diff --git a/README.md b/README.md deleted file mode 100644 index 788ba14e..00000000 --- a/README.md +++ /dev/null @@ -1,221 +0,0 @@ -
- LOGO -
- -# KIOS --- Knowledge-based Intelligent Operation System - -This is a project for robot task planning and execution based on langchain agent (langgraph), LLM, behavior tree and robot expert skills. - -> BB: I'm now finishing my master thesis and the project is still under development. The LLM planning functionality has been fully tested and the behavior tree executor has also been proven to work well. - -> BB: I'm now concentrating on generating more data and using them for local model finetuning. The finetuned gpt-3.5-turbo model is now the llm model for generating the unit behavior trees in the behavior tree offline expanding loop, while the mistral 7B model finetuned with QLora 4bit in the llama-factory platform can work equivalently as the unit behavior tree generator. More data is being generated by gpt-4 and collected for the next step to finetuning state forecasting models and sequential planning models. - -> BB: To save your time, the tutorial part of the project is still unter construction. You may jump to the "something to try" part to see if there are any updates. BB will fill up the tutorial part as soon as possible. - -- About the old version: -The behavior tree executor is implemented based on project BehaviorTree.CPP. See `kios_cpp`. -Some of the components are implemented in python. See `kios_py`. - -> BB: For the old version, please check `old_readme.md`. The old version will be archived and will not be updated anymore. - -- About the new version: -The no-ros version, which aims at simplizing the system structure, is now actively developed. -For this part see `kios_bt_planning`. - -- About robot interface: -For people who don't have access to mios, you may deploy your own methods to generate robot commands in "mios_task_factory.py" and your own methods to execute the commands in "robot_command.py". - -## Intro - -KIOS is a robot task planning system developed by BlackBird for his master thesis. The system is currently under development and is not ready for use. - -The system is mainly written in python. The idea is to integrate the LLM (large language model) into the robot task planning system for automatic behaviortree generation and modification. - -The LLM is used for generating the task plan in the form of behavior trees based on the provided domain knowledge (prompt engineering or RAG). The APIs for generating, modifying and executing the behavior trees are exposed to the LLM agent. With the feedback from the robot(also the nature language feedbacks from the user), the LLM agent can modify the behavior tree and generate new plans dynamically to finish the robotic assembly tasks. - -The usecases are from the siemens robot assembly challenge and the furniture-bench. - -## Contents - -* [What is KIOS?](#what-is-KIOS) -* [Getting started](#getting-started) - * [Requirements](#requirements) - * [Install](#install) - * [Packages](#packages) - * [Usage](#usage) - * [System Structure](#system-structure) - * [Something to try](#something-to-try) - * [Testing](#testing) - * [Development Log](#development-log) -* [Contribute](#contribute) -* [License](#license) -* [Sources](#sources) -* [More](#more) - -## What is KIOS? - - -## Getting Started - -### Requirements - -**For the client side:** -- Ubuntu 20.04 LTS -- conan 1.59.0 (conan 2 is not compatible with the project mios) -- linux Realtime kernal. This is the requirement of robot control interface (1000Hz control loop). For walkthrough please check [here](https://frankaemika.github.io/docs/installation_linux.html#setting-up-the-real-time-kernel). - -**For the server side (if you want to try the local inference models):** -- Ubuntu 20.04 LTS -- CUDA 12.1 or higher -- RAM 32GB or higher -- GPU 24GB or higher - -### Install - -It is highly recommended to use a virtual environment for the project. - -```bash -conda create -n kios python=3.10 -conda activate kios -``` - -1. Install dependency packages. - -```bash -pip3 install -r requirements.txt -sudo apt-get install graphviz -# install the package kios_bt_planning -cd kios_bt_planning -pip3 install -e . -``` - -2. (Skip this if you do not need world state visualization) install neo4j. - -The application can be downloaded from [here](https://github.com/neo4j/neo4j-python-driver). - -After setting up the neo4j server, please change in the neo4j interface in `kios_bt_planning/kios_world` to your pw. - - -3. Set up the mios (branch = kios) and the franka robot. - -4. Install llama.cpp according to the [docs](https://github.com/abetlen/llama-cpp-python?tab=readme-ov-file#supported-backends). Please aware that you need to enable CUDA backend. - -```bash -# in the virtual environment -CMAKE_ARGS="-DLLAMA_CUBLAS=on" pip install llama-cpp-python -``` - -> BB: Go to check the project [mios](https://gitlab.lrz.de/ki_fabrik_integration/MIRMI-public/mios) for more information. The docker image's name is "mirmi/mios", but is not compatible with this project. The skills necessary for the robot manipulation in kios are still being actively developed. A new docker image will be released as soon as possible. - -### Packages -- experiments: the experiment files of different problems. - - chair... - - gearset... - - scene - - domain - - problem -- kios_bt_planning - - kios_agent: the agents for task planning and behavior tree generating - - kios_llm_bt: prompt engineering files for end-to-end behavior tree generating - - kios_bt: modules for basic behavior tree functionality. - - Behavior nodes (actions and conditions) - - The factory class for generating behavior trees - - Behavior tree json interface - - Mios asynchronization module - - ... - - kios_domain: domain knowledge written in pddl with unified-planning - - pddl python interfaces. - - domain knowledge definitions. - - ... - - kios_planner: discarded now - - kios_robot: robot modules for real-world robot manipulation - - kios_vision - - robot_interface: interface methods to execute the actions in behavior trees. - - robot_proprioceptor: class for interacting with the robot (get/set states). - - mios_task_factory: task factory for miosskill, mioscall and kioscall. - - mios_async: asynchronization module for robot_command - - robot_command: the command (list of calls/skills) for the robot. - - kios_scene: modules for the task scene model - - mongodb_interface - - scene_factory - - (scene-world_linker) - - kios_world: modules for the world model - - world_interface: interfaces for query/update the world state. - - graph_interface: interfaces for interacting with inner world graph. - - neo4j_interface: interfaces for neo4j database. - - kios_utils: utility modules - - tests: test files for the modules above. - -(old version packages) -- kios_cpp: the behavior tree executor implemented in c++. discarded now. -- kios_py: the behavior tree executor implemented in python. discarded now. -- kios_cli: the command line interface for the project. discarded now. - -### Usage - -> BB: Now the functionality of the project is still under development. - -> BB: For now please try out the test files in `kios_bt_planning`. - -### System Structure - -
- The Concept -
- -### Something to try - -1. Set up your openai api-key. - -2. Set up langsmith. - -3. Set up huggingface. - -### Testing - -For module testing please check the test folder in `kios_bt_planning`. - -To use ipython for debug, install ipython in your environment (otherwise will use the system-wide interpreter): - -```bash -conda install ipython -``` - -### Development Log - -For the old version see [old_dev_log.md](old_dev_log.md) - -## Contribute - -The project is still under development. You are welcome to contribute to the project by starting an issue or making a pull request. - -## License - -MIT License - -## Sources - -- [langchain](https://python.langchain.com/docs/get_started/introduction) -- [langsmith](https://docs.smith.langchain.com/tracing) -- [llama_cpp_python](https://github.com/abetlen/llama-cpp-python) -- [llama_factory](https://github.com/hiyouga/LLaMA-Factory) -- [Mistral](https://mistral.ai/technology/#models) -- [llama2](https://llama.meta.com/llama2) -- [ChatGPT-Robot-Manipulation-Prompts](https://github.com/microsoft/ChatGPT-Robot-Manipulation-Prompts) -- [furniture-bench](https://github.com/clvrai/furniture-bench) -- [mios_py_interface](https://gitlab.lrz.de/bblab/mios_py_interface) -- [unified-planning](https://github.com/aiplan4eu/unified-planning) -- [mios](https://gitlab.lrz.de/ki_fabrik_integration/MIRMI-public/mios) -- [BehaviorTree.CPP](https://github.com/BehaviorTree/BehaviorTree.CPP) -- [py_trees](https://github.com/splintered-reality/py_trees) -- [neo4j](https://github.com/neo4j/neo4j-python-driver) -- [websocketpp](https://github.com/zaphoyd/websocketpp) -- [ros2](https://docs.ros.org/en/foxy/index.html) - -... - -## More - -The project is still under development. Please feel free to start an issue if you have any question or suggestion. - -The tutorial for the new version is still under construction. It will be released as soon as possible (maybe sometime around 15.04.2024, which is the deadline of my master thesis). diff --git a/TRI.png b/TRI.png deleted file mode 100644 index 12ffedb6..00000000 Binary files a/TRI.png and /dev/null differ diff --git a/data/cook/cook_e2e/DISCARDED.md b/data/cook/cook_e2e/DISCARDED.md deleted file mode 100644 index e69de29b..00000000 diff --git a/data/cook/cook_e2e/behaviortree.txt b/data/cook/cook_e2e/behaviortree.txt deleted file mode 100644 index 4d79b87c..00000000 --- a/data/cook/cook_e2e/behaviortree.txt +++ /dev/null @@ -1,51 +0,0 @@ -[user] -The behavior tree should be in json dictionary format. -Following is a simple example: -""" -{ - "summary": "Selector to change the tool in the left hand from outwardgripper to defaultgripper", - "name": "selector: change_tool(left_hand, outwardgripper, defaultgripper)", - "children": [ - { - "summary": "the target is that the left hand is holding the default gripper", - "name": "target: hold(left_hand, defaultgripper)" - }, - { - "summary": "Sequence to change the tool in the left hand from outwardgripper to defaultgripper", - "name": "sequence: change_tool(left_hand, outwardgripper, defaultgripper)", - "children": [ - { - "summary": "A precondition is that the left hand is holding the outwardgripper", - "name": "precondition: hold(left_hand, outwardgripper)" - }, - { - "summary": "A precondition is that the outwardgripper is empty", - "name": "precondition: is_empty(outwardgripper)" - }, - { - "summary": "The action to change the tool in the left hand from outwardgripper to defaultgripper", - "name": "action: change_tool(left_hand, outwardgripper, defaultgripper)" - } - ] - } - ] -} -""" -1. Every node are described by a summary and a name: - - "summary": a short interpretive description of the node. - - "name": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are "selector", "sequence", "condition" ("target", "precondition"), "action". -2. The control flow nodes, namely "selector" and "sequence", can have "children", which is a list of children nodes. -3. The "condition" nodes that are children of "selectors" are targets. Those that are children of "sequences" are preconditions. -4. All the node should always have their corresponding actions/precidates in the domain knowledge. -5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state. -6. The basic structure template of a unit subtree is: - - A selector as the root node. - - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied. - - A sequence node as the last child of the selector node to achieve the target condition node. - - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions. - - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before. - - The action should have effects that can fulfill the target condition node in the selector. - -7. If any precondition node will be unsatisfied when it is ticked, a unit subtree should be constructed to replace it, which take that precondition node as its target node and aims to achieve it. You should estimate the world state when the precondition is ticked (checked) according to the initial state and the effects of the actions executed before, then do this replacement repeatedly until all the preconditions are satisfied at the time they are ticked. -8. The behavior tree should be constructed based on the action sequence you planned and the action definitions in ROBOT_ACTION_LIST. -9. Control flow nodes (sequence nodes and selector nodes) don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state. \ No newline at end of file diff --git a/data/cook/cook_e2e/chain.txt b/data/cook/cook_e2e/chain.txt deleted file mode 100644 index ffcc4fc7..00000000 --- a/data/cook/cook_e2e/chain.txt +++ /dev/null @@ -1,9 +0,0 @@ -Follow this workflow to solve the task: -1. Read the problem description. Parse the initial state and write into the "initial_state" in the output dictionary. -2. Interpret the goal of the problem as goal_state and write into the "instruction_summary" in the output dictionary. -3. Try to solve the problem, achieve the goal_state from the initial_state based on the domain knowledge provided. Generate a action sequence with the actions defined in and write it into dictionary["task_plan"]["action_sequence"] of the output. -4. If you cannot find a plan to solve the task, ask question with nature language in the "question" of the output dictionary. -5. Explain the actions in the action sequence and write the output dictionary["task_plan"]["action_instructions"]. -6. Collect the objects in the world that are affected by the action sequence, write into dictionary["task_plan"]["manipulated_objects"]. -7. Update the "initial_state" based on the affected objects you collected in the last step so irrelevant objects and relations/constraints are removed. -8. Generate the behavior tree based on the action sequence and write into dictionary["task_plan"]["behavior_tree"]. \ No newline at end of file diff --git a/data/cook/cook_e2e/domain.txt b/data/cook/cook_e2e/domain.txt deleted file mode 100644 index 837729f4..00000000 --- a/data/cook/cook_e2e/domain.txt +++ /dev/null @@ -1,40 +0,0 @@ -Following is the domain knowledge you need, which includes: -- the object types in the world, -- the predicates for describing the states of the world, -- the actions that the robot can apply, with their precondtions and effects defined. - -The object types are explained as follows: -""" -thing: The base class for all objects in the world. Its subclasses are: hand, utensil, food. -hand: The robot hand. A empty can only hold one at a time. -utensil: The utensil to contain, handle and cook . -food: The food to be handled and cooked. -""" - -The precidates are explained as follows: -""" -"PREDICATE_LIST" -- is_empty(): is empty and can hold something. -- can_manipulate(, ): can be used to manipulate (manipulation constraint). This regulates the compatibility between and . -- can_insert_to(, ): can be inserted into (insertion assembly constraint) -- hold(, ): is holding . It can be holding or holding . -- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) -""" - -The actions are explained as follows: -""" -"ROBOT_ACTION_LIST" -- pick_up(, , ): use to pick up . As preconditions, should be holding and should be empty in order to hold the . The will be held by the as a result of this action. -- put_down(, , ): use to put down . As preconditions, should be holding and should be holding . The target is to make the tool empty. The will be empty and will not hold the anymore as a result of this action. -- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding . -- change_tool(, , ): change the equipped tool from to in order to manipulate specific parts. As preconditions, should be holding and should be empty. - -- cut(, , ): use to cut . As preconditions, should be holding and should be empty in order to hold the . The will be cut by the as a result of this action. - -- cook(, , ): use to cook . As preconditions, should be holding and should be empty in order to hold the . The will be cooked by the as a result of this action. -- stir_and_fry(, , ): use to stir and fry . As preconditions, should be holding and should be empty in order to hold the . The will be stirred and fried by the as a result of this action. -- serve(, , ): use to serve . As preconditions, should be holding and should be empty in order to hold the . The will be served by the as a result of this action. - - -""" -the items in <> indicate the type of the instances needed for the predicates and actions. \ No newline at end of file diff --git a/data/cook/cook_e2e/example copy.txt b/data/cook/cook_e2e/example copy.txt deleted file mode 100644 index 615373fc..00000000 --- a/data/cook/cook_e2e/example copy.txt +++ /dev/null @@ -1,185 +0,0 @@ -I will give you some examples of the input and the output you will generate. -Example 1: -""" -- Input: - -- Output: -``` -{"task_plan": { - "action_sequence": [ - "load_tool(left_hand, parallel_box1)", - "pick_up(left_hand, parallel_box1, gear1)", - "insert(left_hand, parallel_box1, gear1, shaft1)" - ], - "action_instructions": [ - "equip the parallel_box1 on the left hand", - "pick up the gear1 with the parallel_box1 in the left hand", - "insert the gear1 into the shaft1 using the parallel_box1 in the left hand" - ], - "behavior_tree": { - "summary": "selector to insert gear1 into shaft1", - "name": "selector: insert gear1 into shaft1", - "children": [ - { - "summary": "check if gear1 is inserted to shaft1", - "name": "target: is_inserted_to(gear1, shaft1)", - }, - { - "summary": "sequence to insert gear1 into shaft1", - "name": "sequence: insert(gear1, shaft1)", - "children": [ - { - "summary": "selector to load parallel_box1", - "name": "selector: load_tool(left_hand, parallel_box1)", - "children": [ - { - "summary": "check if parallel_box1 is hold", - "name": "target: hold(left_hand, parallel_box1)", - }, - { - "summary": "sequence to load parallel_box1", - "name": "sequence: load_tool(left_hand, parallel_box1)", - "children": [ - { - "summary": "check if parallel_box1 is equippable", - "name": "precondition: is_equippable(parallel_box1)", - }, - { - "summary": "check if left hand is free", - "name": "precondition: is_free(left_hand)", - }, - { - "summary": "equip parallel_box1 to left hand", - "name": "action: load_tool(left_hand, parallel_box1)", - }, - ], - }, - ], - }, - { - "summary": "selector to pick_up gear1", - "name": "selector: pick_up(left_hand, parallel_box1, gear1)", - "children": [ - { - "summary": "check if parallel_box1 holds gear1", - "name": "target: hold(parallel_box1, gear1)", - }, - { - "summary": "sequence to pick_up gear1", - "name": "sequence: pick_up(left_hand, parallel_box1, gear1)", - "identifier": 8, - "type_name": "sequence", - "children": [ - { - "summary": "check if parallel_box1 is free", - "name": "precondition: is_free(parallel_box1)", - "identifier": 9, - "type_name": "condition", - "conditions": [ - { - "object_name": "parallel_box1", - "property_name": "is_free", - "property_value": null, - "status": true, - } - ], - }, - { - "summary": "check if left hand holds parallel_box1", - "name": "precondition: hold(left_hand, parallel_box1)", - "identifier": 16, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "parallel_box1", - "status": true, - } - ], - }, - { - "summary": "pick up gear1 using parallel_box1", - "name": "action: pick_up(left_hand, parallel_box1, gear1)", - "identifier": 12, - "type_name": "action", - "effects": [ - { - "object_name": "parallel_box1", - "property_name": "hold", - "property_value": "gear1", - "status": true, - }, - { - "object_name": "parallel_box1", - "property_name": "is_free", - "property_value": null, - "status": false, - }, - ], - }, - ], - }, - ], - }, - { - "summary": "check if gear1 can be inserted to shaft1", - "name": "precondition: can_insert_to(gear1, shaft1)", - "identifier": 18, - "type_name": "condition", - "conditions": [ - { - "object_name": "gear1", - "property_name": "can_insert_to", - "property_value": "shaft1", - "status": true, - } - ], - }, - { - "summary": "insert gear1 to shaft1", - "name": "action: insert(left_hand, parallel_box1, gear1, shaft1)", - "identifier": 19, - "type_name": "action", - "effects": [ - { - "object_name": "parallel_box1", - "property_name": "hold", - "property_value": "gear1", - "status": false, - }, - { - "object_name": "parallel_box1", - "property_name": "is_free", - "property_value": null, - "status": true, - }, - { - "object_name": "gear1", - "property_name": "is_inserted_to", - "property_value": "shaft1", - "status": true, - }, - ], - }, - ], - }, - ], -} -}, -"initial_state": {"objects": [ - {"name": "gear1", "properties": []}, - {"name": "shaft1", "properties": []}, - {"name": "parallel_box1", "properties": ["is_equippable", "is_free"]}, - {"name": "left_hand", "properties": ["is_free"]}, - ], - "constraints": [ - {"source": "gear1", "name": "can_insert_to", "target": "shaft1"}, - {"source": "parallel_box1", "name": "can_manipulate", "target": "gear1"}, - ], - "relations": [], -}, -"instruction_summary": "insert gear1 into shaft1" -} -``` -""" \ No newline at end of file diff --git a/data/cook/cook_e2e/example.txt b/data/cook/cook_e2e/example.txt deleted file mode 100644 index cf713974..00000000 --- a/data/cook/cook_e2e/example.txt +++ /dev/null @@ -1,81 +0,0 @@ -Following is an input/output example. -Example 1: -""" -INPUT: -target: "insert the gear1 into the shaft1" -initial_state: {"objects": [ - {"name": "gear1", "properties": []}, - {"name": "shaft1", "properties": []}, - {"name": "parallel_box1", "properties": ["is_empty"]}, - {"name": "left_hand", "properties": []}, - ], - "constraints": [ - {"source": "gear1", "name": "can_insert_to", "target": "shaft1"}, - {"source": "parallel_box1", "name": "can_manipulate", "target": "gear1"}, - ], - "relations": [ - {"source": "left_hand", "name": "hold", "target": "parallel_box1"}, - ], -} -OUTPUT: -{ - "thought": "At the start, the left hand is holding the parallel_box1, and the parallel_box1 is empty. According to the constraints, the gear1 can be inserted to the shaft1, and the parallel_box1 can manipulate the gear1. So, the gear1 can be picked up with the parallel_box1 in the left hand, and then the precondition hold(parallel_box1, gear1) is satisfied. Finally, the gear1 can be inserted into the shaft1 with the parallel_box1 in the left hand.", - "action_sequence": [ - "pick_up(left_hand, parallel_box1, gear1)", - "insert(left_hand, parallel_box1, gear1, shaft1)" - ], - "behavior_tree": { - "summary": "selector to insert the gear1 into the shaft1 with the parallel_box1 in the left hand", - "name": "selector: insert(gear1, shaft1, parallel_box1, left_hand)", - "children": [ - { - "summary": "check the target that gear1 is inserted to shaft1", - "name": "target: is_inserted_to(gear1, shaft1)" - }, - { - "summary": "sequence to insert the gear1 into the shaft1 with the parallel_box1 in the left hand", - "name": "sequence: insert(gear1, shaft1, parallel_box1, left_hand)", - "children": [ - { - "summary": "check the precondition that the left hand is holding the parallel_box1", - "name": "precondition: hold(left_hand, parallel_box1)" - }, - { - "summary": "selector to pick_up the gear1 with the parallel_box1 in the left hand", - "name": "selector: pick_up(left_hand, parallel_box1, gear1)", - "children": [ - { - "summary": "check the target that the parallel_box1 is holding the gear1", - "name": "target: hold(parallel_box1, gear1)" - }, - { - "summary": "sequence to pick_up the gear1 with the parallel_box1 in the left hand", - "name": "sequence: pick_up(left_hand, parallel_box1, gear1)", - "children": [ - { - "summary": "check the precondition that the parallel_box1 is empty", - "name": "precondition: is_empty(parallel_box1)" - }, - { - "summary": "check the precondition that the left hand is holding the parallel_box1", - "name": "precondition: hold(left_hand, parallel_box1)" - }, - { - "summary": "the action to pick_up the gear1 with the parallel_box1 in the left hand", - "name": "action: pick_up(left_hand, parallel_box1, gear1)" - } - ] - } - ] - }, - { - "summary": "the action to insert the gear1 into the shaft1 with the parallel_box1 in the left hand", - "name": "action: insert(left_hand, parallel_box1, gear1, shaft1)" - } - ] - } - ] - } -} - - diff --git a/data/cook/cook_e2e/output_format.txt b/data/cook/cook_e2e/output_format.txt deleted file mode 100644 index c38fc403..00000000 --- a/data/cook/cook_e2e/output_format.txt +++ /dev/null @@ -1,6 +0,0 @@ -The output json obejct has the following keys: -""" -- "thought": A brief explanation of the thinking process behind the action sequence. -- "action_sequence": A sequential list of robot actions. Only the actions defined in the "ROBOT_ACTION LIST" will be used. -- "behavior_tree": A json dictionary form behavior tree constructed according to the "action_sequence" -""" \ No newline at end of file diff --git a/data/cook/cook_e2e/state.txt b/data/cook/cook_e2e/state.txt deleted file mode 100644 index 665700ba..00000000 --- a/data/cook/cook_e2e/state.txt +++ /dev/null @@ -1,6 +0,0 @@ -Information about world states should be in json dictionary format. -It contains three lists: - - objects: the objects and their properties (as lists of strings) in the world. The properties can be changed by robot actions. - - constraints: the constraint relations that are enforced and should not be changed. They are knowledge about the world or some constraints set by the user. - - relations: the relations in the world. They can be changed by robot actions. -The properties, constraints and relations should always align with the [PREDICATE_LIST] in the domain knowledge. \ No newline at end of file diff --git a/data/cook/cook_e2e/system.txt b/data/cook/cook_e2e/system.txt deleted file mode 100644 index cb531830..00000000 --- a/data/cook/cook_e2e/system.txt +++ /dev/null @@ -1 +0,0 @@ -You are an excellent interpreter of instructions for robotic assembly tasks. \ No newline at end of file diff --git a/data/cook/cook_e2e/task.txt b/data/cook/cook_e2e/task.txt deleted file mode 100644 index e039d429..00000000 --- a/data/cook/cook_e2e/task.txt +++ /dev/null @@ -1,6 +0,0 @@ -The input are: -target: The target that you make the sequential plan and construct the behavior tree to achieve. -initial_state: The initial state that you start from. - -You analyze the target, refer to the domain knowledge to understand it, make an action sequence and construct a behavior tree based on it. -Your output should be a json object which follows the regulated format. \ No newline at end of file diff --git a/data/cook/cook_e2e/template.txt b/data/cook/cook_e2e/template.txt deleted file mode 100644 index 9f3f5b61..00000000 --- a/data/cook/cook_e2e/template.txt +++ /dev/null @@ -1,5 +0,0 @@ -Resume from the input below. -""" -target: {target} -initial_state: {initial_state} -""" \ No newline at end of file diff --git a/data/examples/behavior_tree_skeleton.json b/data/examples/behavior_tree_skeleton.json deleted file mode 100644 index c1dc06a5..00000000 --- a/data/examples/behavior_tree_skeleton.json +++ /dev/null @@ -1,28 +0,0 @@ -{ - "summary": "Selector to change the tool in the left hand from outwardgripper to defaultgripper", - "name": "selector: change_tool(left_hand, outwardgripper, defaultgripper)", - "children": [ - { - "summary": "the target is that the left hand is holding the default gripper", - "name": "target: hold(left_hand, defaultgripper)" - }, - { - "summary": "Sequence to change the tool in the left hand from outwardgripper to defaultgripper", - "name": "sequence: change_tool(left_hand, outwardgripper, defaultgripper)", - "children": [ - { - "summary": "A precondition is that the left hand is holding the outwardgripper", - "name": "precondition: hold(left_hand, outwardgripper)" - }, - { - "summary": "A precondition is that the outwardgripper is empty", - "name": "precondition: is_empty(outwardgripper)" - }, - { - "summary": "The action to change the tool in the left hand from outwardgripper to defaultgripper", - "name": "action: change_tool(left_hand, outwardgripper, defaultgripper)" - } - ] - } - ] -} \ No newline at end of file diff --git a/data/examples/world_state.json b/data/examples/world_state.json deleted file mode 100644 index 19105d78..00000000 --- a/data/examples/world_state.json +++ /dev/null @@ -1,25 +0,0 @@ -{ - "objects": [ - {"name": "clampgripper", "properties": ["is_empty"]}, - {"name": "outwardgripper", "properties": []}, - {"name": "defaultgripper", "properties": ["is_empty"]}, - ... - {"name": "gear1", "properties": []}, - {"name": "shaft1", "properties": []}, - ... - {"name": "left_hand", "properties": []} - ], - "constraints": [ - {"source": "clampgripper", "name": "can_manipulate", "target": "shaft1"}, - {"source": "parallelgripper", "name": "can_manipulate", "target": "gear1"}, - ... - {"source": "shaft1", "name": "can_insert_to", "target": "gearbase_hole1"}, - {"source": "gear1", "name": "can_insert_to", "target": "shaft1"} - ... - ], - "relations": [ - {"source": "left_hand", "name": "hold", "target": "outwardgripper"}, - {"source": "outwardgripper", "name": "hold", "target": "gear3"} - ] -} - diff --git a/data/finetuning/finetuning_messages.jsonl b/data/finetuning/finetuning_messages.jsonl deleted file mode 100644 index a1cef79a..00000000 --- a/data/finetuning/finetuning_messages.jsonl +++ /dev/null @@ -1,10 +0,0 @@ -{"messages": [{"role": "system", "content": "You are an excellent behavior tree builder for robotic assembly tasks"}, {"role": "user", "content": "You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you construct robot task plans represented by behaiovr trees. \n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object that is ready to be parsed.\n\n Following is the domain knowledge you need in pddl domain format, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\"\"\"\n(define (domain robot_assembly_problem-domain)\n (:requirements :strips :typing)\n (:types\n thing - object\n tool part hand - thing\n )\n (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part))\n (:action pick_up\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part))\n :effect (and (hold ?tool ?part) (not (is_free ?tool))))\n (:action put_down\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (hold ?tool ?part) (hold ?hand ?tool))\n :effect (and (not (hold ?hand ?part)) (is_free ?tool)))\n (:action place\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2)))\n (:action detach\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2))))\n (:action insert\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2)))\n (:action pull\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2))))\n (:action screw\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2)))\n (:action unscrew\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2))))\n (:action load_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (is_equippable ?tool) (is_free ?hand))\n :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool)))\n (:action unload_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (hold ?hand ?tool) (is_free ?tool))\n :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool))))\n)\n\"\"\"\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_free( or ): or is free and can hold something. (default: true)\n- is_equippable(): is ready to be eqquipped by a robot hand. (default: true)\n- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false)\n- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false)\n- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false)\n- can_place_to(, ): can be placed into (placement assembly constraint) (default: false)\n- hold(, ): is holding (default: false)\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding .\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into .\n- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free.\n- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: unload_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to unload the outwardgripper in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\",\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding a outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\",\n },\n {\n \"summary\": \"Unload the outwardgripper in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outwardgripper)\",\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: insert(left_hand, defaultgripper, gear1, shaft1)\n---------------\n\n "}, {"role": "assistant", "content": "{\n \"summary\": \"Selector to insert gear1 into shaft1 using defaultgripper in the left hand\",\n \"name\": \"selector: insert(left_hand, defaultgripper, gear1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"Check the target that gear1 is inserted into shaft1\",\n \"name\": \"target: is_inserted_to(gear1, shaft1)\",\n },\n {\n \"summary\": \"Sequence to insert gear1 into shaft1 using defaultgripper in the left hand\",\n \"name\": \"sequence: insert(left_hand, defaultgripper, gear1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"Check the precondition that the left hand is holding defaultgripper\",\n \"name\": \"precondition: hold(left_hand, defaultgripper)\",\n },\n {\n \"summary\": \"Check the precondition that defaultgripper is holding gear1\",\n \"name\": \"precondition: hold(defaultgripper, gear1)\",\n },\n {\n \"summary\": \"Insert gear1 into shaft1 using defaultgripper\",\n \"name\": \"action: insert(left_hand, defaultgripper, gear1, shaft1)\",\n }\n ]\n }\n ]\n}"}]} -{"messages": [{"role": "system", "content": "You are an excellent behavior tree builder for robotic assembly tasks"}, {"role": "user", "content": "You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you construct robot task plans represented by behaiovr trees. \n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need in pddl domain format, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\"\"\"\n(define (domain robot_assembly_problem-domain)\n (:requirements :strips :typing)\n (:types\n thing - object\n tool part hand - thing\n )\n (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part))\n (:action pick_up\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part))\n :effect (and (hold ?tool ?part) (not (is_free ?tool))))\n (:action put_down\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (hold ?tool ?part) (hold ?hand ?tool))\n :effect (and (not (hold ?hand ?part)) (is_free ?tool)))\n (:action place\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2)))\n (:action detach\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2))))\n (:action insert\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2)))\n (:action pull\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2))))\n (:action screw\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2)))\n (:action unscrew\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2))))\n (:action load_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (is_equippable ?tool) (is_free ?hand))\n :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool)))\n (:action unload_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (hold ?hand ?tool) (is_free ?tool))\n :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool))))\n)\n\"\"\"\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_free( or ): or is free and can hold something. (default: true)\n- is_equippable(): is ready to be eqquipped by a robot hand. (default: true)\n- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false)\n- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false)\n- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false)\n- can_place_to(, ): can be placed into (placement assembly constraint) (default: false)\n- hold(, ): is holding (default: false)\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding .\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into .\n- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free.\n- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: unload_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to unload the outwardgripper in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\",\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding a outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\",\n },\n {\n \"summary\": \"Unload the outwardgripper in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outwardgripper)\",\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: insert(left_hand, defaultgripper, gear1, shaft1)\n---------------\n\n "}, {"role": "assistant", "content": "{\n \"summary\": \"Selector to insert gear1 into shaft1 using defaultgripper in the left hand\",\n \"name\": \"selector: insert(left_hand, defaultgripper, gear1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"Check the target that gear1 is inserted into shaft1\",\n \"name\": \"target: is_inserted_to(gear1, shaft1)\",\n },\n {\n \"summary\": \"Sequence to insert gear1 into shaft1 using defaultgripper in the left hand\",\n \"name\": \"sequence: insert(left_hand, defaultgripper, gear1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"Check the precondition that the left hand is holding defaultgripper\",\n \"name\": \"precondition: hold(left_hand, defaultgripper)\",\n },\n {\n \"summary\": \"Check the precondition that defaultgripper is holding gear1\",\n \"name\": \"precondition: hold(defaultgripper, gear1)\",\n },\n {\n \"summary\": \"Insert gear1 into shaft1 using defaultgripper\",\n \"name\": \"action: insert(left_hand, defaultgripper, gear1, shaft1)\",\n }\n ]\n }\n ]\n}"}]} -{"messages": [{"role": "system", "content": "You are an excellent behavior tree builder for robotic assembly tasks"}, {"role": "user", "content": "You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you construct robot task plans represented by behaiovr trees. \n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need in pddl domain format, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\"\"\"\n(define (domain robot_assembly_problem-domain)\n (:requirements :strips :typing)\n (:types\n thing - object\n tool part hand - thing\n )\n (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part))\n (:action pick_up\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part))\n :effect (and (hold ?tool ?part) (not (is_free ?tool))))\n (:action put_down\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (hold ?tool ?part) (hold ?hand ?tool))\n :effect (and (not (hold ?hand ?part)) (is_free ?tool)))\n (:action place\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2)))\n (:action detach\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2))))\n (:action insert\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2)))\n (:action pull\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2))))\n (:action screw\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2)))\n (:action unscrew\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2))))\n (:action load_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (is_equippable ?tool) (is_free ?hand))\n :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool)))\n (:action unload_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (hold ?hand ?tool) (is_free ?tool))\n :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool))))\n)\n\"\"\"\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_free( or ): or is free and can hold something. (default: true)\n- is_equippable(): is ready to be eqquipped by a robot hand. (default: true)\n- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false)\n- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false)\n- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false)\n- can_place_to(, ): can be placed into (placement assembly constraint) (default: false)\n- hold(, ): is holding (default: false)\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding .\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into .\n- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free.\n- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: unload_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to unload the outwardgripper in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\",\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding a outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\",\n },\n {\n \"summary\": \"Unload the outwardgripper in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outwardgripper)\",\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: insert(left_hand, defaultgripper, gear1, shaft1)\n---------------\n\n\n Return a JSON object.\n "}, {"role": "assistant", "content": "{\n \"summary\": \"Selector to insert gear1 into shaft1 using defaultgripper in the left hand\",\n \"name\": \"selector: insert(left_hand, defaultgripper, gear1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"Check the target that gear1 is inserted into shaft1\",\n \"name\": \"target: is_inserted_to(gear1, shaft1)\",\n },\n {\n \"summary\": \"Sequence to insert gear1 into shaft1 using defaultgripper in the left hand\",\n \"name\": \"sequence: insert(left_hand, defaultgripper, gear1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"Check the precondition that the left hand is holding defaultgripper\",\n \"name\": \"precondition: hold(left_hand, defaultgripper)\",\n },\n {\n \"summary\": \"Check the precondition that defaultgripper is holding gear1\",\n \"name\": \"precondition: hold(defaultgripper, gear1)\",\n },\n {\n \"summary\": \"Insert gear1 into shaft1 using defaultgripper\",\n \"name\": \"action: insert(left_hand, defaultgripper, gear1, shaft1)\",\n }\n ]\n }\n ]\n}"}]} -{"messages": [{"role": "system", "content": "You are an excellent behavior tree builder for robotic assembly tasks"}, {"role": "user", "content": "You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you construct robot task plans represented by behaiovr trees. \n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need in pddl domain format, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\"\"\"\n(define (domain robot_assembly_problem-domain)\n (:requirements :strips :typing)\n (:types\n thing - object\n tool part hand - thing\n )\n (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part))\n (:action pick_up\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part))\n :effect (and (hold ?tool ?part) (not (is_free ?tool))))\n (:action put_down\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (hold ?tool ?part) (hold ?hand ?tool))\n :effect (and (not (hold ?hand ?part)) (is_free ?tool)))\n (:action place\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2)))\n (:action detach\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2))))\n (:action insert\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2)))\n (:action pull\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2))))\n (:action screw\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2)))\n (:action unscrew\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2))))\n (:action load_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (is_equippable ?tool) (is_free ?hand))\n :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool)))\n (:action unload_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (hold ?hand ?tool) (is_free ?tool))\n :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool))))\n)\n\"\"\"\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_free( or ): or is free and can hold something. (default: true)\n- is_equippable(): is ready to be eqquipped by a robot hand. (default: true)\n- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false)\n- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false)\n- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false)\n- can_place_to(, ): can be placed into (placement assembly constraint) (default: false)\n- hold(, ): is holding (default: false)\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding .\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into .\n- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free.\n- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: unload_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to unload the outwardgripper in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outwardgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\"\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding a outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"Unload the outwardgripper in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outwardgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: insert(left_hand, defaultgripper, gear1, shaft1)\n---------------\n\n\n Return a JSON object.\n "}, {"role": "assistant", "content": "{\n \"summary\": \"Selector to insert gear1 into shaft1 using defaultgripper in the left hand\",\n \"name\": \"selector: insert(left_hand, defaultgripper, gear1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"Check the target that gear1 is inserted into shaft1\",\n \"name\": \"target: is_inserted_to(gear1, shaft1)\"\n },\n {\n \"summary\": \"Sequence to insert gear1 into shaft1 using defaultgripper in the left hand\",\n \"name\": \"sequence: insert(left_hand, defaultgripper, gear1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"Check the precondition that the left hand is holding defaultgripper\",\n \"name\": \"precondition: hold(left_hand, defaultgripper)\"\n },\n {\n \"summary\": \"Check the precondition that defaultgripper is holding gear1\",\n \"name\": \"precondition: hold(defaultgripper, gear1)\"\n },\n {\n \"summary\": \"Insert gear1 into shaft1 using defaultgripper\",\n \"name\": \"action: insert(left_hand, defaultgripper, gear1, shaft1)\"\n }\n ]\n }\n ]\n}"}]} -{"messages": [{"role": "system", "content": "You are an excellent behavior tree builder for robotic assembly tasks"}, {"role": "user", "content": "You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you construct robot task plans represented by behaiovr trees. \n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need in pddl domain format, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\"\"\"\n(define (domain robot_assembly_problem-domain)\n (:requirements :strips :typing)\n (:types\n thing - object\n tool part hand - thing\n )\n (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part))\n (:action pick_up\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part))\n :effect (and (hold ?tool ?part) (not (is_free ?tool))))\n (:action put_down\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (hold ?tool ?part) (hold ?hand ?tool))\n :effect (and (not (hold ?hand ?part)) (is_free ?tool)))\n (:action place\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2)))\n (:action detach\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2))))\n (:action insert\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2)))\n (:action pull\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2))))\n (:action screw\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2)))\n (:action unscrew\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2))))\n (:action load_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (is_equippable ?tool) (is_free ?hand))\n :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool)))\n (:action unload_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (hold ?hand ?tool) (is_free ?tool))\n :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool))))\n)\n\"\"\"\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_free( or ): or is free and can hold something. (default: true)\n- is_equippable(): is ready to be eqquipped by a robot hand. (default: true)\n- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false)\n- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false)\n- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false)\n- can_place_to(, ): can be placed into (placement assembly constraint) (default: false)\n- hold(, ): is holding (default: false)\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding .\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into .\n- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free.\n- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: unload_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to unload the outwardgripper in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outwardgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\"\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding a outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"Unload the outwardgripper in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outwardgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)\n---------------\n\n\n Return a JSON object.\n "}, {"role": "assistant", "content": "{\n \"summary\": \"Selector to insert shaft1 into gearbase_hole1 using parallel_box1 in the left hand\",\n \"name\": \"selector: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)\",\n \"children\": [\n {\n \"summary\": \"Check the target that shaft1 is inserted into gearbase_hole1\",\n \"name\": \"target: is_inserted_to(shaft1, gearbase_hole1)\"\n },\n {\n \"summary\": \"Sequence to insert shaft1 into gearbase_hole1 using parallel_box1 in the left hand\",\n \"name\": \"sequence: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)\",\n \"children\": [\n {\n \"summary\": \"Check the precondition that the left hand is holding parallel_box1\",\n \"name\": \"precondition: hold(left_hand, parallel_box1)\"\n },\n {\n \"summary\": \"Check the precondition that parallel_box1 is holding shaft1\",\n \"name\": \"precondition: hold(parallel_box1, shaft1)\"\n },\n {\n \"summary\": \"Insert shaft1 into gearbase_hole1 using parallel_box1\",\n \"name\": \"action: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)\"\n }\n ]\n }\n ]\n}"}]} -{"messages": [{"role": "system", "content": "You are an excellent behavior tree builder for robotic assembly tasks"}, {"role": "user", "content": "You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you construct robot task plans represented by behaiovr trees. \n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need in pddl domain format, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\"\"\"\n(define (domain robot_assembly_problem-domain)\n (:requirements :strips :typing)\n (:types\n thing - object\n tool part hand - thing\n )\n (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part))\n (:action pick_up\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part))\n :effect (and (hold ?tool ?part) (not (is_free ?tool))))\n (:action put_down\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (hold ?tool ?part) (hold ?hand ?tool))\n :effect (and (not (hold ?hand ?part)) (is_free ?tool)))\n (:action place\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2)))\n (:action detach\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2))))\n (:action insert\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2)))\n (:action pull\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2))))\n (:action screw\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2)))\n (:action unscrew\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2))))\n (:action load_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (is_equippable ?tool) (is_free ?hand))\n :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool)))\n (:action unload_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (hold ?hand ?tool) (is_free ?tool))\n :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool))))\n)\n\"\"\"\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_free( or ): or is free and can hold something. (default: true)\n- is_equippable(): is ready to be eqquipped by a robot hand. (default: true)\n- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false)\n- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false)\n- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false)\n- can_place_to(, ): can be placed into (placement assembly constraint) (default: false)\n- hold(, ): is holding (default: false)\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding .\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into .\n- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free.\n- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: unload_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to unload the outwardgripper in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outwardgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\"\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding a outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"Unload the outwardgripper in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outwardgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)\n---------------\n\n\n Return a JSON object.\n "}, {"role": "assistant", "content": "{\n \"summary\": \"Selector to insert shaft1 into gearbase_hole1 using parallel_box1 in the left hand\",\n \"name\": \"selector: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)\",\n \"children\": [\n {\n \"summary\": \"Check the target that shaft1 is inserted into gearbase_hole1\",\n \"name\": \"target: is_inserted_to(shaft1, gearbase_hole1)\"\n },\n {\n \"summary\": \"Sequence to insert shaft1 into gearbase_hole1 using parallel_box1 in the left hand\",\n \"name\": \"sequence: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)\",\n \"children\": [\n {\n \"summary\": \"Check the precondition that the left hand is holding parallel_box1\",\n \"name\": \"precondition: hold(left_hand, parallel_box1)\"\n },\n {\n \"summary\": \"Check the precondition that parallel_box1 is holding shaft1\",\n \"name\": \"precondition: hold(parallel_box1, shaft1)\"\n },\n {\n \"summary\": \"Insert shaft1 into gearbase_hole1 using parallel_box1\",\n \"name\": \"action: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)\"\n }\n ]\n }\n ]\n}"}]} -{"messages": [{"role": "system", "content": "You are an excellent behavior tree builder for robotic assembly tasks"}, {"role": "user", "content": "You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you construct robot task plans represented by behaiovr trees. \n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need in pddl domain format, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\"\"\"\n(define (domain robot_assembly_problem-domain)\n (:requirements :strips :typing)\n (:types\n thing - object\n tool part hand - thing\n )\n (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part))\n (:action pick_up\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part))\n :effect (and (hold ?tool ?part) (not (is_free ?tool))))\n (:action put_down\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (hold ?tool ?part) (hold ?hand ?tool))\n :effect (and (not (hold ?hand ?part)) (is_free ?tool)))\n (:action place\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2)))\n (:action detach\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2))))\n (:action insert\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2)))\n (:action pull\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2))))\n (:action screw\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2)))\n (:action unscrew\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2))))\n (:action load_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (is_equippable ?tool) (is_free ?hand))\n :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool)))\n (:action unload_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (hold ?hand ?tool) (is_free ?tool))\n :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool))))\n)\n\"\"\"\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_free( or ): or is free and can hold something. (default: true)\n- is_equippable(): is ready to be eqquipped by a robot hand. (default: true)\n- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false)\n- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false)\n- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false)\n- can_place_to(, ): can be placed into (placement assembly constraint) (default: false)\n- hold(, ): is holding (default: false)\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding .\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into .\n- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free.\n- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: unload_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to unload the outwardgripper in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outwardgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\"\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding a outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"Unload the outwardgripper in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outwardgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)\n---------------\n\n\n Return a JSON object.\n "}, {"role": "assistant", "content": "{\n \"summary\": \"Selector to insert shaft1 into gearbase_hole1 using parallel_box1 in the left hand\",\n \"name\": \"selector: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)\",\n \"children\": [\n {\n \"summary\": \"Check the target that shaft1 is inserted into gearbase_hole1\",\n \"name\": \"target: is_inserted_to(shaft1, gearbase_hole1)\"\n },\n {\n \"summary\": \"Sequence to insert shaft1 into gearbase_hole1 using parallel_box1 in the left hand\",\n \"name\": \"sequence: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)\",\n \"children\": [\n {\n \"summary\": \"Check the precondition that the left hand is holding parallel_box1\",\n \"name\": \"precondition: hold(left_hand, parallel_box1)\"\n },\n {\n \"summary\": \"Check the precondition that parallel_box1 is holding shaft1\",\n \"name\": \"precondition: hold(parallel_box1, shaft1)\"\n },\n {\n \"summary\": \"Insert shaft1 into gearbase_hole1 using parallel_box1\",\n \"name\": \"action: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)\"\n }\n ]\n }\n ]\n}"}]} -{"messages": [{"role": "system", "content": "You are an excellent behavior tree builder for robotic assembly tasks"}, {"role": "user", "content": "You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you construct robot task plans represented by behaiovr trees. \n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need in pddl domain format, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\"\"\"\n(define (domain robot_assembly_problem-domain)\n (:requirements :strips :typing)\n (:types\n thing - object\n tool part hand - thing\n )\n (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part))\n (:action pick_up\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part))\n :effect (and (hold ?tool ?part) (not (is_free ?tool))))\n (:action put_down\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (hold ?tool ?part) (hold ?hand ?tool))\n :effect (and (not (hold ?hand ?part)) (is_free ?tool)))\n (:action place\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2)))\n (:action detach\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2))))\n (:action insert\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2)))\n (:action pull\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2))))\n (:action screw\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2)))\n (:action unscrew\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2))))\n (:action load_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (is_equippable ?tool) (is_free ?hand))\n :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool)))\n (:action unload_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (hold ?hand ?tool) (is_free ?tool))\n :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool))))\n)\n\"\"\"\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_free( or ): or is free and can hold something. (default: true)\n- is_equippable(): is ready to be eqquipped by a robot hand. (default: true)\n- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false)\n- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false)\n- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false)\n- can_place_to(, ): can be placed into (placement assembly constraint) (default: false)\n- hold(, ): is holding (default: false)\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding .\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into .\n- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free.\n- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: unload_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to unload the outwardgripper in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outwardgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\"\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding a outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"Unload the outwardgripper in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outwardgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: load_tool(left_hand, parallel_box1)\n---------------\n\n\n Return a JSON object.\n "}, {"role": "assistant", "content": "{\n \"summary\": \"Selector to load the parallel_box1 in the left hand\",\n \"name\": \"selector: load_tool(left_hand, parallel_box1)\",\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\"\n },\n {\n \"summary\": \"Sequence to load the tool in the left hand\",\n \"name\": \"sequence: load_tool(left_hand, parallel_box1)\",\n \"children\": [\n {\n \"summary\": \"Check the precondition that the parallel_box1 is equippable\",\n \"name\": \"precondition: is_equippable(parallel_box1)\"\n },\n {\n \"summary\": \"Check the precondition that the left hand is free\",\n \"name\": \"precondition: is_free(left_hand)\"\n },\n {\n \"summary\": \"Load the parallel_box1 in the left hand\",\n \"name\": \"action: load_tool(left_hand, parallel_box1)\"\n }\n ]\n }\n ]\n}"}]} -{"messages": [{"role": "system", "content": "You are an excellent behavior tree builder for robotic assembly tasks"}, {"role": "user", "content": "You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you construct robot task plans represented by behaiovr trees. \n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need in pddl domain format, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\"\"\"\n(define (domain robot_assembly_problem-domain)\n (:requirements :strips :typing)\n (:types\n thing - object\n tool part hand - thing\n )\n (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part))\n (:action pick_up\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part))\n :effect (and (hold ?tool ?part) (not (is_free ?tool))))\n (:action put_down\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (hold ?tool ?part) (hold ?hand ?tool))\n :effect (and (not (hold ?hand ?part)) (is_free ?tool)))\n (:action place\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2)))\n (:action detach\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2))))\n (:action insert\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2)))\n (:action pull\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2))))\n (:action screw\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2)))\n (:action unscrew\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2))))\n (:action load_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (is_equippable ?tool) (is_free ?hand))\n :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool)))\n (:action unload_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (hold ?hand ?tool) (is_free ?tool))\n :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool))))\n)\n\"\"\"\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_free( or ): or is free and can hold something. (default: true)\n- is_equippable(): is ready to be eqquipped by a robot hand. (default: true)\n- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false)\n- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false)\n- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false)\n- can_place_to(, ): can be placed into (placement assembly constraint) (default: false)\n- hold(, ): is holding (default: false)\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding .\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into .\n- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free.\n- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: unload_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to unload the outwardgripper in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outwardgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\"\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding a outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"Unload the outwardgripper in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outwardgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: unload_tool(left_hand, outward_claw)\n---------------\n\n\n Return a JSON object.\n "}, {"role": "assistant", "content": "{\n \"summary\": \"Selector to unload the outward_claw in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outward_claw)\",\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\"\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outward_claw)\",\n \"children\": [\n {\n \"summary\": \"Check the precondition that the left hand is holding a outward_claw\",\n \"name\": \"precondition: hold(left_hand, outward_claw)\"\n },\n {\n \"summary\": \"Unload the outward_claw in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outward_claw)\"\n }\n ]\n }\n ]\n}"}]} -{"messages": [{"role": "system", "content": "You are an excellent behavior tree builder for robotic assembly tasks"}, {"role": "user", "content": "You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you construct robot task plans represented by behaiovr trees. \n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need in pddl domain format, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\"\"\"\n(define (domain robot_assembly_problem-domain)\n (:requirements :strips :typing)\n (:types\n thing - object\n tool part hand - thing\n )\n (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part))\n (:action pick_up\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part))\n :effect (and (hold ?tool ?part) (not (is_free ?tool))))\n (:action put_down\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (hold ?tool ?part) (hold ?hand ?tool))\n :effect (and (not (hold ?hand ?part)) (is_free ?tool)))\n (:action place\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2)))\n (:action detach\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2))))\n (:action insert\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2)))\n (:action pull\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2))))\n (:action screw\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2)))\n (:action unscrew\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2))))\n (:action load_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (is_equippable ?tool) (is_free ?hand))\n :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool)))\n (:action unload_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (hold ?hand ?tool) (is_free ?tool))\n :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool))))\n)\n\"\"\"\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_free( or ): or is free and can hold something. (default: true)\n- is_equippable(): is ready to be eqquipped by a robot hand. (default: true)\n- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false)\n- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false)\n- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false)\n- can_place_to(, ): can be placed into (placement assembly constraint) (default: false)\n- hold(, ): is holding (default: false)\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding .\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into .\n- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free.\n- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: unload_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to unload the outwardgripper in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outwardgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\"\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding a outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"Unload the outwardgripper in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outwardgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: pick_up(left_hand, parallel_box1, shaft1)\n---------------\n\n\n Return a JSON object.\n "}, {"role": "assistant", "content": "{\n \"summary\": \"Selector to pick up the shaft1 with the parallel_box1 in the left hand\",\n \"name\": \"selector: pick_up(left_hand, parallel_box1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"Check the target that the parallel_box1 is holding the shaft1\",\n \"name\": \"target: hold(parallel_box1, shaft1)\"\n },\n {\n \"summary\": \"Sequence to pick up the shaft1 with the parallel_box1 in the left hand\",\n \"name\": \"sequence: pick_up(left_hand, parallel_box1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"Check the precondition that the left hand is holding the parallel_box1\",\n \"name\": \"precondition: hold(left_hand, parallel_box1)\"\n },\n {\n \"summary\": \"Pick up the shaft1 with the parallel_box1 in the left hand\",\n \"name\": \"action: pick_up(left_hand, parallel_box1, shaft1)\"\n }\n ]\n }\n ]\n}"}]} diff --git a/data/finetuning/finetuning_messages_20240303.jsonl b/data/finetuning/finetuning_messages_20240303.jsonl deleted file mode 100644 index c39d3802..00000000 --- a/data/finetuning/finetuning_messages_20240303.jsonl +++ /dev/null @@ -1,60 +0,0 @@ -{"messages": [{"role": "system", "content": "You are an excellent behavior tree builder for robotic assembly tasks"}, {"role": "user", "content": "You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you construct robot task plans represented by behaiovr trees. \n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object that is ready to be parsed.\n\n Following is the domain knowledge you need in pddl domain format, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\"\"\"\n(define (domain robot_assembly_problem-domain)\n (:requirements :strips :typing)\n (:types\n thing - object\n tool part hand - thing\n )\n (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part))\n (:action pick_up\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part))\n :effect (and (hold ?tool ?part) (not (is_free ?tool))))\n (:action put_down\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (hold ?tool ?part) (hold ?hand ?tool))\n :effect (and (not (hold ?hand ?part)) (is_free ?tool)))\n (:action place\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2)))\n (:action detach\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2))))\n (:action insert\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2)))\n (:action pull\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2))))\n (:action screw\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2)))\n (:action unscrew\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2))))\n (:action load_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (is_equippable ?tool) (is_free ?hand))\n :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool)))\n (:action unload_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (hold ?hand ?tool) (is_free ?tool))\n :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool))))\n)\n\"\"\"\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_free( or ): or is free and can hold something. (default: true)\n- is_equippable(): is ready to be eqquipped by a robot hand. (default: true)\n- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false)\n- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false)\n- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false)\n- can_place_to(, ): can be placed into (placement assembly constraint) (default: false)\n- hold(, ): is holding (default: false)\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding .\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into .\n- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free.\n- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: unload_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to unload the outwardgripper in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\",\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding a outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\",\n },\n {\n \"summary\": \"Unload the outwardgripper in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outwardgripper)\",\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: insert(left_hand, defaultgripper, gear1, shaft1)\n---------------\n\n "}, {"role": "assistant", "content": "{\n \"summary\": \"Selector to insert gear1 into shaft1 using defaultgripper in the left hand\",\n \"name\": \"selector: insert(left_hand, defaultgripper, gear1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"Check the target that gear1 is inserted into shaft1\",\n \"name\": \"target: is_inserted_to(gear1, shaft1)\",\n },\n {\n \"summary\": \"Sequence to insert gear1 into shaft1 using defaultgripper in the left hand\",\n \"name\": \"sequence: insert(left_hand, defaultgripper, gear1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"Check the precondition that the left hand is holding defaultgripper\",\n \"name\": \"precondition: hold(left_hand, defaultgripper)\",\n },\n {\n \"summary\": \"Check the precondition that defaultgripper is holding gear1\",\n \"name\": \"precondition: hold(defaultgripper, gear1)\",\n },\n {\n \"summary\": \"Insert gear1 into shaft1 using defaultgripper\",\n \"name\": \"action: insert(left_hand, defaultgripper, gear1, shaft1)\",\n }\n ]\n }\n ]\n}"}]} -{"messages": [{"role": "system", "content": "You are an excellent behavior tree builder for robotic assembly tasks"}, {"role": "user", "content": "You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you construct robot task plans represented by behaiovr trees. \n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need in pddl domain format, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\"\"\"\n(define (domain robot_assembly_problem-domain)\n (:requirements :strips :typing)\n (:types\n thing - object\n tool part hand - thing\n )\n (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part))\n (:action pick_up\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part))\n :effect (and (hold ?tool ?part) (not (is_free ?tool))))\n (:action put_down\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (hold ?tool ?part) (hold ?hand ?tool))\n :effect (and (not (hold ?hand ?part)) (is_free ?tool)))\n (:action place\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2)))\n (:action detach\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2))))\n (:action insert\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2)))\n (:action pull\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2))))\n (:action screw\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2)))\n (:action unscrew\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2))))\n (:action load_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (is_equippable ?tool) (is_free ?hand))\n :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool)))\n (:action unload_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (hold ?hand ?tool) (is_free ?tool))\n :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool))))\n)\n\"\"\"\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_free( or ): or is free and can hold something. (default: true)\n- is_equippable(): is ready to be eqquipped by a robot hand. (default: true)\n- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false)\n- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false)\n- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false)\n- can_place_to(, ): can be placed into (placement assembly constraint) (default: false)\n- hold(, ): is holding (default: false)\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding .\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into .\n- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free.\n- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: unload_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to unload the outwardgripper in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\",\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding a outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\",\n },\n {\n \"summary\": \"Unload the outwardgripper in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outwardgripper)\",\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: insert(left_hand, defaultgripper, gear1, shaft1)\n---------------\n\n "}, {"role": "assistant", "content": "{\n \"summary\": \"Selector to insert gear1 into shaft1 using defaultgripper in the left hand\",\n \"name\": \"selector: insert(left_hand, defaultgripper, gear1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"Check the target that gear1 is inserted into shaft1\",\n \"name\": \"target: is_inserted_to(gear1, shaft1)\",\n },\n {\n \"summary\": \"Sequence to insert gear1 into shaft1 using defaultgripper in the left hand\",\n \"name\": \"sequence: insert(left_hand, defaultgripper, gear1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"Check the precondition that the left hand is holding defaultgripper\",\n \"name\": \"precondition: hold(left_hand, defaultgripper)\",\n },\n {\n \"summary\": \"Check the precondition that defaultgripper is holding gear1\",\n \"name\": \"precondition: hold(defaultgripper, gear1)\",\n },\n {\n \"summary\": \"Insert gear1 into shaft1 using defaultgripper\",\n \"name\": \"action: insert(left_hand, defaultgripper, gear1, shaft1)\",\n }\n ]\n }\n ]\n}"}]} -{"messages": [{"role": "system", "content": "You are an excellent behavior tree builder for robotic assembly tasks"}, {"role": "user", "content": "You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you construct robot task plans represented by behaiovr trees. \n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need in pddl domain format, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\"\"\"\n(define (domain robot_assembly_problem-domain)\n (:requirements :strips :typing)\n (:types\n thing - object\n tool part hand - thing\n )\n (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part))\n (:action pick_up\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part))\n :effect (and (hold ?tool ?part) (not (is_free ?tool))))\n (:action put_down\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (hold ?tool ?part) (hold ?hand ?tool))\n :effect (and (not (hold ?hand ?part)) (is_free ?tool)))\n (:action place\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2)))\n (:action detach\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2))))\n (:action insert\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2)))\n (:action pull\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2))))\n (:action screw\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2)))\n (:action unscrew\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2))))\n (:action load_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (is_equippable ?tool) (is_free ?hand))\n :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool)))\n (:action unload_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (hold ?hand ?tool) (is_free ?tool))\n :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool))))\n)\n\"\"\"\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_free( or ): or is free and can hold something. (default: true)\n- is_equippable(): is ready to be eqquipped by a robot hand. (default: true)\n- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false)\n- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false)\n- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false)\n- can_place_to(, ): can be placed into (placement assembly constraint) (default: false)\n- hold(, ): is holding (default: false)\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding .\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into .\n- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free.\n- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: unload_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to unload the outwardgripper in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\",\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding a outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\",\n },\n {\n \"summary\": \"Unload the outwardgripper in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outwardgripper)\",\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: insert(left_hand, defaultgripper, gear1, shaft1)\n---------------\n\n\n Return a JSON object.\n "}, {"role": "assistant", "content": "{\n \"summary\": \"Selector to insert gear1 into shaft1 using defaultgripper in the left hand\",\n \"name\": \"selector: insert(left_hand, defaultgripper, gear1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"Check the target that gear1 is inserted into shaft1\",\n \"name\": \"target: is_inserted_to(gear1, shaft1)\",\n },\n {\n \"summary\": \"Sequence to insert gear1 into shaft1 using defaultgripper in the left hand\",\n \"name\": \"sequence: insert(left_hand, defaultgripper, gear1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"Check the precondition that the left hand is holding defaultgripper\",\n \"name\": \"precondition: hold(left_hand, defaultgripper)\",\n },\n {\n \"summary\": \"Check the precondition that defaultgripper is holding gear1\",\n \"name\": \"precondition: hold(defaultgripper, gear1)\",\n },\n {\n \"summary\": \"Insert gear1 into shaft1 using defaultgripper\",\n \"name\": \"action: insert(left_hand, defaultgripper, gear1, shaft1)\",\n }\n ]\n }\n ]\n}"}]} -{"messages": [{"role": "system", "content": "You are an excellent behavior tree builder for robotic assembly tasks"}, {"role": "user", "content": "You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you construct robot task plans represented by behaiovr trees. \n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need in pddl domain format, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\"\"\"\n(define (domain robot_assembly_problem-domain)\n (:requirements :strips :typing)\n (:types\n thing - object\n tool part hand - thing\n )\n (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part))\n (:action pick_up\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part))\n :effect (and (hold ?tool ?part) (not (is_free ?tool))))\n (:action put_down\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (hold ?tool ?part) (hold ?hand ?tool))\n :effect (and (not (hold ?hand ?part)) (is_free ?tool)))\n (:action place\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2)))\n (:action detach\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2))))\n (:action insert\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2)))\n (:action pull\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2))))\n (:action screw\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2)))\n (:action unscrew\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2))))\n (:action load_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (is_equippable ?tool) (is_free ?hand))\n :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool)))\n (:action unload_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (hold ?hand ?tool) (is_free ?tool))\n :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool))))\n)\n\"\"\"\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_free( or ): or is free and can hold something. (default: true)\n- is_equippable(): is ready to be eqquipped by a robot hand. (default: true)\n- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false)\n- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false)\n- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false)\n- can_place_to(, ): can be placed into (placement assembly constraint) (default: false)\n- hold(, ): is holding (default: false)\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding .\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into .\n- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free.\n- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: unload_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to unload the outwardgripper in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outwardgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\"\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding a outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"Unload the outwardgripper in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outwardgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: insert(left_hand, defaultgripper, gear1, shaft1)\n---------------\n\n\n Return a JSON object.\n "}, {"role": "assistant", "content": "{\n \"summary\": \"Selector to insert gear1 into shaft1 using defaultgripper in the left hand\",\n \"name\": \"selector: insert(left_hand, defaultgripper, gear1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"Check the target that gear1 is inserted into shaft1\",\n \"name\": \"target: is_inserted_to(gear1, shaft1)\"\n },\n {\n \"summary\": \"Sequence to insert gear1 into shaft1 using defaultgripper in the left hand\",\n \"name\": \"sequence: insert(left_hand, defaultgripper, gear1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"Check the precondition that the left hand is holding defaultgripper\",\n \"name\": \"precondition: hold(left_hand, defaultgripper)\"\n },\n {\n \"summary\": \"Check the precondition that defaultgripper is holding gear1\",\n \"name\": \"precondition: hold(defaultgripper, gear1)\"\n },\n {\n \"summary\": \"Insert gear1 into shaft1 using defaultgripper\",\n \"name\": \"action: insert(left_hand, defaultgripper, gear1, shaft1)\"\n }\n ]\n }\n ]\n}"}]} -{"messages": [{"role": "system", "content": "You are an excellent behavior tree builder for robotic assembly tasks"}, {"role": "user", "content": "You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you construct robot task plans represented by behaiovr trees. \n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need in pddl domain format, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\"\"\"\n(define (domain robot_assembly_problem-domain)\n (:requirements :strips :typing)\n (:types\n thing - object\n tool part hand - thing\n )\n (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part))\n (:action pick_up\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part))\n :effect (and (hold ?tool ?part) (not (is_free ?tool))))\n (:action put_down\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (hold ?tool ?part) (hold ?hand ?tool))\n :effect (and (not (hold ?hand ?part)) (is_free ?tool)))\n (:action place\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2)))\n (:action detach\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2))))\n (:action insert\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2)))\n (:action pull\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2))))\n (:action screw\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2)))\n (:action unscrew\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2))))\n (:action load_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (is_equippable ?tool) (is_free ?hand))\n :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool)))\n (:action unload_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (hold ?hand ?tool) (is_free ?tool))\n :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool))))\n)\n\"\"\"\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_free( or ): or is free and can hold something. (default: true)\n- is_equippable(): is ready to be eqquipped by a robot hand. (default: true)\n- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false)\n- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false)\n- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false)\n- can_place_to(, ): can be placed into (placement assembly constraint) (default: false)\n- hold(, ): is holding (default: false)\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding .\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into .\n- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free.\n- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: unload_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to unload the outwardgripper in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outwardgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\"\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding a outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"Unload the outwardgripper in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outwardgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)\n---------------\n\n\n Return a JSON object.\n "}, {"role": "assistant", "content": "{\n \"summary\": \"Selector to insert shaft1 into gearbase_hole1 using parallel_box1 in the left hand\",\n \"name\": \"selector: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)\",\n \"children\": [\n {\n \"summary\": \"Check the target that shaft1 is inserted into gearbase_hole1\",\n \"name\": \"target: is_inserted_to(shaft1, gearbase_hole1)\"\n },\n {\n \"summary\": \"Sequence to insert shaft1 into gearbase_hole1 using parallel_box1 in the left hand\",\n \"name\": \"sequence: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)\",\n \"children\": [\n {\n \"summary\": \"Check the precondition that the left hand is holding parallel_box1\",\n \"name\": \"precondition: hold(left_hand, parallel_box1)\"\n },\n {\n \"summary\": \"Check the precondition that parallel_box1 is holding shaft1\",\n \"name\": \"precondition: hold(parallel_box1, shaft1)\"\n },\n {\n \"summary\": \"Insert shaft1 into gearbase_hole1 using parallel_box1\",\n \"name\": \"action: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)\"\n }\n ]\n }\n ]\n}"}]} -{"messages": [{"role": "system", "content": "You are an excellent behavior tree builder for robotic assembly tasks"}, {"role": "user", "content": "You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you construct robot task plans represented by behaiovr trees. \n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need in pddl domain format, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\"\"\"\n(define (domain robot_assembly_problem-domain)\n (:requirements :strips :typing)\n (:types\n thing - object\n tool part hand - thing\n )\n (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part))\n (:action pick_up\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part))\n :effect (and (hold ?tool ?part) (not (is_free ?tool))))\n (:action put_down\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (hold ?tool ?part) (hold ?hand ?tool))\n :effect (and (not (hold ?hand ?part)) (is_free ?tool)))\n (:action place\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2)))\n (:action detach\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2))))\n (:action insert\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2)))\n (:action pull\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2))))\n (:action screw\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2)))\n (:action unscrew\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2))))\n (:action load_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (is_equippable ?tool) (is_free ?hand))\n :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool)))\n (:action unload_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (hold ?hand ?tool) (is_free ?tool))\n :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool))))\n)\n\"\"\"\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_free( or ): or is free and can hold something. (default: true)\n- is_equippable(): is ready to be eqquipped by a robot hand. (default: true)\n- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false)\n- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false)\n- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false)\n- can_place_to(, ): can be placed into (placement assembly constraint) (default: false)\n- hold(, ): is holding (default: false)\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding .\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into .\n- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free.\n- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: unload_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to unload the outwardgripper in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outwardgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\"\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding a outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"Unload the outwardgripper in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outwardgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)\n---------------\n\n\n Return a JSON object.\n "}, {"role": "assistant", "content": "{\n \"summary\": \"Selector to insert shaft1 into gearbase_hole1 using parallel_box1 in the left hand\",\n \"name\": \"selector: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)\",\n \"children\": [\n {\n \"summary\": \"Check the target that shaft1 is inserted into gearbase_hole1\",\n \"name\": \"target: is_inserted_to(shaft1, gearbase_hole1)\"\n },\n {\n \"summary\": \"Sequence to insert shaft1 into gearbase_hole1 using parallel_box1 in the left hand\",\n \"name\": \"sequence: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)\",\n \"children\": [\n {\n \"summary\": \"Check the precondition that the left hand is holding parallel_box1\",\n \"name\": \"precondition: hold(left_hand, parallel_box1)\"\n },\n {\n \"summary\": \"Check the precondition that parallel_box1 is holding shaft1\",\n \"name\": \"precondition: hold(parallel_box1, shaft1)\"\n },\n {\n \"summary\": \"Insert shaft1 into gearbase_hole1 using parallel_box1\",\n \"name\": \"action: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)\"\n }\n ]\n }\n ]\n}"}]} -{"messages": [{"role": "system", "content": "You are an excellent behavior tree builder for robotic assembly tasks"}, {"role": "user", "content": "You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you construct robot task plans represented by behaiovr trees. \n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need in pddl domain format, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\"\"\"\n(define (domain robot_assembly_problem-domain)\n (:requirements :strips :typing)\n (:types\n thing - object\n tool part hand - thing\n )\n (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part))\n (:action pick_up\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part))\n :effect (and (hold ?tool ?part) (not (is_free ?tool))))\n (:action put_down\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (hold ?tool ?part) (hold ?hand ?tool))\n :effect (and (not (hold ?hand ?part)) (is_free ?tool)))\n (:action place\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2)))\n (:action detach\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2))))\n (:action insert\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2)))\n (:action pull\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2))))\n (:action screw\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2)))\n (:action unscrew\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2))))\n (:action load_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (is_equippable ?tool) (is_free ?hand))\n :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool)))\n (:action unload_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (hold ?hand ?tool) (is_free ?tool))\n :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool))))\n)\n\"\"\"\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_free( or ): or is free and can hold something. (default: true)\n- is_equippable(): is ready to be eqquipped by a robot hand. (default: true)\n- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false)\n- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false)\n- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false)\n- can_place_to(, ): can be placed into (placement assembly constraint) (default: false)\n- hold(, ): is holding (default: false)\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding .\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into .\n- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free.\n- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: unload_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to unload the outwardgripper in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outwardgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\"\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding a outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"Unload the outwardgripper in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outwardgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)\n---------------\n\n\n Return a JSON object.\n "}, {"role": "assistant", "content": "{\n \"summary\": \"Selector to insert shaft1 into gearbase_hole1 using parallel_box1 in the left hand\",\n \"name\": \"selector: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)\",\n \"children\": [\n {\n \"summary\": \"Check the target that shaft1 is inserted into gearbase_hole1\",\n \"name\": \"target: is_inserted_to(shaft1, gearbase_hole1)\"\n },\n {\n \"summary\": \"Sequence to insert shaft1 into gearbase_hole1 using parallel_box1 in the left hand\",\n \"name\": \"sequence: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)\",\n \"children\": [\n {\n \"summary\": \"Check the precondition that the left hand is holding parallel_box1\",\n \"name\": \"precondition: hold(left_hand, parallel_box1)\"\n },\n {\n \"summary\": \"Check the precondition that parallel_box1 is holding shaft1\",\n \"name\": \"precondition: hold(parallel_box1, shaft1)\"\n },\n {\n \"summary\": \"Insert shaft1 into gearbase_hole1 using parallel_box1\",\n \"name\": \"action: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)\"\n }\n ]\n }\n ]\n}"}]} -{"messages": [{"role": "system", "content": "You are an excellent behavior tree builder for robotic assembly tasks"}, {"role": "user", "content": "You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you construct robot task plans represented by behaiovr trees. \n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need in pddl domain format, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\"\"\"\n(define (domain robot_assembly_problem-domain)\n (:requirements :strips :typing)\n (:types\n thing - object\n tool part hand - thing\n )\n (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part))\n (:action pick_up\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part))\n :effect (and (hold ?tool ?part) (not (is_free ?tool))))\n (:action put_down\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (hold ?tool ?part) (hold ?hand ?tool))\n :effect (and (not (hold ?hand ?part)) (is_free ?tool)))\n (:action place\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2)))\n (:action detach\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2))))\n (:action insert\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2)))\n (:action pull\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2))))\n (:action screw\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2)))\n (:action unscrew\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2))))\n (:action load_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (is_equippable ?tool) (is_free ?hand))\n :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool)))\n (:action unload_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (hold ?hand ?tool) (is_free ?tool))\n :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool))))\n)\n\"\"\"\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_free( or ): or is free and can hold something. (default: true)\n- is_equippable(): is ready to be eqquipped by a robot hand. (default: true)\n- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false)\n- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false)\n- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false)\n- can_place_to(, ): can be placed into (placement assembly constraint) (default: false)\n- hold(, ): is holding (default: false)\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding .\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into .\n- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free.\n- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: unload_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to unload the outwardgripper in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outwardgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\"\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding a outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"Unload the outwardgripper in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outwardgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: load_tool(left_hand, parallel_box1)\n---------------\n\n\n Return a JSON object.\n "}, {"role": "assistant", "content": "{\n \"summary\": \"Selector to load the parallel_box1 in the left hand\",\n \"name\": \"selector: load_tool(left_hand, parallel_box1)\",\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\"\n },\n {\n \"summary\": \"Sequence to load the tool in the left hand\",\n \"name\": \"sequence: load_tool(left_hand, parallel_box1)\",\n \"children\": [\n {\n \"summary\": \"Check the precondition that the parallel_box1 is equippable\",\n \"name\": \"precondition: is_equippable(parallel_box1)\"\n },\n {\n \"summary\": \"Check the precondition that the left hand is free\",\n \"name\": \"precondition: is_free(left_hand)\"\n },\n {\n \"summary\": \"Load the parallel_box1 in the left hand\",\n \"name\": \"action: load_tool(left_hand, parallel_box1)\"\n }\n ]\n }\n ]\n}"}]} -{"messages": [{"role": "system", "content": "You are an excellent behavior tree builder for robotic assembly tasks"}, {"role": "user", "content": "You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you construct robot task plans represented by behaiovr trees. \n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need in pddl domain format, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\"\"\"\n(define (domain robot_assembly_problem-domain)\n (:requirements :strips :typing)\n (:types\n thing - object\n tool part hand - thing\n )\n (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part))\n (:action pick_up\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part))\n :effect (and (hold ?tool ?part) (not (is_free ?tool))))\n (:action put_down\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (hold ?tool ?part) (hold ?hand ?tool))\n :effect (and (not (hold ?hand ?part)) (is_free ?tool)))\n (:action place\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2)))\n (:action detach\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2))))\n (:action insert\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2)))\n (:action pull\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2))))\n (:action screw\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2)))\n (:action unscrew\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2))))\n (:action load_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (is_equippable ?tool) (is_free ?hand))\n :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool)))\n (:action unload_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (hold ?hand ?tool) (is_free ?tool))\n :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool))))\n)\n\"\"\"\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_free( or ): or is free and can hold something. (default: true)\n- is_equippable(): is ready to be eqquipped by a robot hand. (default: true)\n- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false)\n- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false)\n- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false)\n- can_place_to(, ): can be placed into (placement assembly constraint) (default: false)\n- hold(, ): is holding (default: false)\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding .\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into .\n- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free.\n- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: unload_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to unload the outwardgripper in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outwardgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\"\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding a outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"Unload the outwardgripper in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outwardgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: unload_tool(left_hand, outward_claw)\n---------------\n\n\n Return a JSON object.\n "}, {"role": "assistant", "content": "{\n \"summary\": \"Selector to unload the outward_claw in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outward_claw)\",\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\"\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outward_claw)\",\n \"children\": [\n {\n \"summary\": \"Check the precondition that the left hand is holding a outward_claw\",\n \"name\": \"precondition: hold(left_hand, outward_claw)\"\n },\n {\n \"summary\": \"Unload the outward_claw in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outward_claw)\"\n }\n ]\n }\n ]\n}"}]} -{"messages": [{"role": "system", "content": "You are an excellent behavior tree builder for robotic assembly tasks"}, {"role": "user", "content": "You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you construct robot task plans represented by behaiovr trees. \n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need in pddl domain format, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\"\"\"\n(define (domain robot_assembly_problem-domain)\n (:requirements :strips :typing)\n (:types\n thing - object\n tool part hand - thing\n )\n (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part))\n (:action pick_up\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part))\n :effect (and (hold ?tool ?part) (not (is_free ?tool))))\n (:action put_down\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (hold ?tool ?part) (hold ?hand ?tool))\n :effect (and (not (hold ?hand ?part)) (is_free ?tool)))\n (:action place\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2)))\n (:action detach\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2))))\n (:action insert\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2)))\n (:action pull\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2))))\n (:action screw\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2)))\n (:action unscrew\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2))))\n (:action load_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (is_equippable ?tool) (is_free ?hand))\n :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool)))\n (:action unload_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (hold ?hand ?tool) (is_free ?tool))\n :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool))))\n)\n\"\"\"\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_free( or ): or is free and can hold something. (default: true)\n- is_equippable(): is ready to be eqquipped by a robot hand. (default: true)\n- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false)\n- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false)\n- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false)\n- can_place_to(, ): can be placed into (placement assembly constraint) (default: false)\n- hold(, ): is holding (default: false)\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding .\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into .\n- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free.\n- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: unload_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to unload the outwardgripper in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outwardgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\"\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding a outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"Unload the outwardgripper in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outwardgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: pick_up(left_hand, parallel_box1, shaft1)\n---------------\n\n\n Return a JSON object.\n "}, {"role": "assistant", "content": "{\n \"summary\": \"Selector to pick up the shaft1 with the parallel_box1 in the left hand\",\n \"name\": \"selector: pick_up(left_hand, parallel_box1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"Check the target that the parallel_box1 is holding the shaft1\",\n \"name\": \"target: hold(parallel_box1, shaft1)\"\n },\n {\n \"summary\": \"Sequence to pick up the shaft1 with the parallel_box1 in the left hand\",\n \"name\": \"sequence: pick_up(left_hand, parallel_box1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"Check the precondition that the left hand is holding the parallel_box1\",\n \"name\": \"precondition: hold(left_hand, parallel_box1)\"\n },\n {\n \"summary\": \"Pick up the shaft1 with the parallel_box1 in the left hand\",\n \"name\": \"action: pick_up(left_hand, parallel_box1, shaft1)\"\n }\n ]\n }\n ]\n}"}]} -{"messages": [{"role": "system", "content": "You are an excellent behavior tree builder for robotic assembly tasks"}, {"role": "user", "content": "You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you construct robot task plans represented by behaiovr trees. \n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need in pddl domain format, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\"\"\"\n(define (domain robot_assembly_problem-domain)\n (:requirements :strips :typing)\n (:types\n thing - object\n tool part hand - thing\n )\n (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part))\n (:action pick_up\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part))\n :effect (and (hold ?tool ?part) (not (is_free ?tool))))\n (:action put_down\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (hold ?tool ?part) (hold ?hand ?tool))\n :effect (and (not (hold ?hand ?part)) (is_free ?tool)))\n (:action place\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2)))\n (:action detach\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2))))\n (:action insert\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2)))\n (:action pull\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2))))\n (:action screw\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2)))\n (:action unscrew\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2))))\n (:action load_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (is_equippable ?tool) (is_free ?hand))\n :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool)))\n (:action unload_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (hold ?hand ?tool) (is_free ?tool))\n :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool))))\n)\n\"\"\"\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_free( or ): or is free and can hold something. (default: true)\n- is_equippable(): is ready to be eqquipped by a robot hand. (default: true)\n- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false)\n- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false)\n- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false)\n- can_place_to(, ): can be placed into (placement assembly constraint) (default: false)\n- hold(, ): is holding (default: false)\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding .\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into .\n- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free.\n- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: unload_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to unload the outwardgripper in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outwardgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\"\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding a outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"Unload the outwardgripper in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outwardgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)\n---------------\n\n\n Return a JSON object.\n "}, {"role": "assistant", "content": "{\n \"summary\": \"Selector to insert shaft1 into gearbase_hole1 using parallel_box1 in the left hand\",\n \"name\": \"selector: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)\",\n \"children\": [\n {\n \"summary\": \"Check the target that shaft1 is inserted into gearbase_hole1\",\n \"name\": \"target: is_inserted_to(shaft1, gearbase_hole1)\"\n },\n {\n \"summary\": \"Sequence to insert shaft1 into gearbase_hole1 using parallel_box1 in the left hand\",\n \"name\": \"sequence: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding parallel_box1\",\n \"name\": \"precondition: hold(left_hand, parallel_box1)\"\n },\n {\n \"summary\": \"check the precondition that parallel_box1 is holding shaft1\",\n \"name\": \"precondition: hold(parallel_box1, shaft1)\"\n },\n {\n \"summary\": \"check the precondition that shaft1 can be inserted into gearbase_hole1\",\n \"name\": \"precondition: can_insert_to(shaft1, gearbase_hole1)\"\n },\n {\n \"summary\": \"Insert shaft1 into gearbase_hole1 using parallel_box1 in the left hand\",\n \"name\": \"action: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)\"\n }\n ]\n }\n ]\n}"}]} -{"messages": [{"role": "system", "content": "You are an excellent behavior tree builder for robotic assembly tasks"}, {"role": "user", "content": "You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you construct robot task plans represented by behaiovr trees. \n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need in pddl domain format, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\"\"\"\n(define (domain robot_assembly_problem-domain)\n (:requirements :strips :typing)\n (:types\n thing - object\n tool part hand - thing\n )\n (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part))\n (:action pick_up\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part))\n :effect (and (hold ?tool ?part) (not (is_free ?tool))))\n (:action put_down\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (hold ?tool ?part) (hold ?hand ?tool))\n :effect (and (not (hold ?hand ?part)) (is_free ?tool)))\n (:action place\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2)))\n (:action detach\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2))))\n (:action insert\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2)))\n (:action pull\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2))))\n (:action screw\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2)))\n (:action unscrew\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2))))\n (:action load_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (is_equippable ?tool) (is_free ?hand))\n :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool)))\n (:action unload_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (hold ?hand ?tool) (is_free ?tool))\n :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool))))\n)\n\"\"\"\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_free( or ): or is free and can hold something. (default: true)\n- is_equippable(): is ready to be eqquipped by a robot hand. (default: true)\n- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false)\n- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false)\n- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false)\n- can_place_to(, ): can be placed into (placement assembly constraint) (default: false)\n- hold(, ): is holding (default: false)\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding .\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into .\n- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free.\n- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: unload_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to unload the outwardgripper in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outwardgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\"\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding a outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"Unload the outwardgripper in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outwardgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: load_tool(left_hand, parallel_box1)\n---------------\n\n\n Return a JSON object.\n "}, {"role": "assistant", "content": "{\n \"summary\": \"Selector to load the parallel_box1 in the left hand\",\n \"name\": \"selector: load_tool(left_hand, parallel_box1)\",\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is holding parallel_box1\",\n \"name\": \"target: hold(left_hand, parallel_box1)\"\n },\n {\n \"summary\": \"Sequence to load the tool in the left hand\",\n \"name\": \"sequence: load_tool(left_hand, parallel_box1)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is free\",\n \"name\": \"precondition: is_free(left_hand)\"\n },\n {\n \"summary\": \"check the precondition that the parallel_box1 is equippable\",\n \"name\": \"precondition: is_equippable(parallel_box1)\"\n },\n {\n \"summary\": \"Load the parallel_box1 in the left hand\",\n \"name\": \"action: load_tool(left_hand, parallel_box1)\"\n }\n ]\n }\n ]\n}"}]} -{"messages": [{"role": "system", "content": "You are an excellent behavior tree builder for robotic assembly tasks"}, {"role": "user", "content": "You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you construct robot task plans represented by behaiovr trees. \n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need in pddl domain format, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\"\"\"\n(define (domain robot_assembly_problem-domain)\n (:requirements :strips :typing)\n (:types\n thing - object\n tool part hand - thing\n )\n (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part))\n (:action pick_up\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part))\n :effect (and (hold ?tool ?part) (not (is_free ?tool))))\n (:action put_down\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (hold ?tool ?part) (hold ?hand ?tool))\n :effect (and (not (hold ?hand ?part)) (is_free ?tool)))\n (:action place\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2)))\n (:action detach\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2))))\n (:action insert\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2)))\n (:action pull\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2))))\n (:action screw\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2)))\n (:action unscrew\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2))))\n (:action load_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (is_equippable ?tool) (is_free ?hand))\n :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool)))\n (:action unload_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (hold ?hand ?tool) (is_free ?tool))\n :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool))))\n)\n\"\"\"\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_free( or ): or is free and can hold something. (default: true)\n- is_equippable(): is ready to be eqquipped by a robot hand. (default: true)\n- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false)\n- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false)\n- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false)\n- can_place_to(, ): can be placed into (placement assembly constraint) (default: false)\n- hold(, ): is holding (default: false)\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding .\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into .\n- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free.\n- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: unload_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to unload the outwardgripper in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outwardgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\"\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding a outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"Unload the outwardgripper in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outwardgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: unload_tool(left_hand, outward_claw)\n---------------\n\n\n Return a JSON object.\n "}, {"role": "assistant", "content": "{\n \"summary\": \"Selector to unload the outward_claw in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outward_claw)\",\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\"\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outward_claw)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding an outward_claw\",\n \"name\": \"precondition: hold(left_hand, outward_claw)\"\n },\n {\n \"summary\": \"Unload the outward_claw in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outward_claw)\"\n }\n ]\n }\n ]\n}"}]} -{"messages": [{"role": "system", "content": "You are an excellent behavior tree builder for robotic assembly tasks"}, {"role": "user", "content": "You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you construct robot task plans represented by behaiovr trees. \n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need in pddl domain format, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\"\"\"\n(define (domain robot_assembly_problem-domain)\n (:requirements :strips :typing)\n (:types\n thing - object\n tool part hand - thing\n )\n (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part))\n (:action pick_up\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part))\n :effect (and (hold ?tool ?part) (not (is_free ?tool))))\n (:action put_down\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (hold ?tool ?part) (hold ?hand ?tool))\n :effect (and (not (hold ?hand ?part)) (is_free ?tool)))\n (:action place\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2)))\n (:action detach\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2))))\n (:action insert\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2)))\n (:action pull\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2))))\n (:action screw\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2)))\n (:action unscrew\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2))))\n (:action load_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (is_equippable ?tool) (is_free ?hand))\n :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool)))\n (:action unload_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (hold ?hand ?tool) (is_free ?tool))\n :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool))))\n)\n\"\"\"\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_free( or ): or is free and can hold something. (default: true)\n- is_equippable(): is ready to be eqquipped by a robot hand. (default: true)\n- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false)\n- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false)\n- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false)\n- can_place_to(, ): can be placed into (placement assembly constraint) (default: false)\n- hold(, ): is holding (default: false)\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding .\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into .\n- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free.\n- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: unload_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to unload the outwardgripper in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outwardgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\"\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding a outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"Unload the outwardgripper in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outwardgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: load_tool(left_hand, parallel_box1)\n---------------\n\n\n Return a JSON object.\n "}, {"role": "assistant", "content": "{\n \"summary\": \"Selector to load the parallel_box1 in the left hand\",\n \"name\": \"selector: load_tool(left_hand, parallel_box1)\",\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is holding parallel_box1\",\n \"name\": \"target: hold(left_hand, parallel_box1)\"\n },\n {\n \"summary\": \"Sequence to load the tool in the left hand\",\n \"name\": \"sequence: load_tool(left_hand, parallel_box1)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is free\",\n \"name\": \"precondition: is_free(left_hand)\"\n },\n {\n \"summary\": \"check the precondition that the parallel_box1 is equippable\",\n \"name\": \"precondition: is_equippable(parallel_box1)\"\n },\n {\n \"summary\": \"Load the parallel_box1 in the left hand\",\n \"name\": \"action: load_tool(left_hand, parallel_box1)\"\n }\n ]\n }\n ]\n}"}]} -{"messages": [{"role": "system", "content": "You are an excellent behavior tree builder for robotic assembly tasks"}, {"role": "user", "content": "You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you construct robot task plans represented by behaiovr trees. \n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need in pddl domain format, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\"\"\"\n(define (domain robot_assembly_problem-domain)\n (:requirements :strips :typing)\n (:types\n thing - object\n tool part hand - thing\n )\n (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part))\n (:action pick_up\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part))\n :effect (and (hold ?tool ?part) (not (is_free ?tool))))\n (:action put_down\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (hold ?tool ?part) (hold ?hand ?tool))\n :effect (and (not (hold ?hand ?part)) (is_free ?tool)))\n (:action place\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2)))\n (:action detach\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2))))\n (:action insert\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2)))\n (:action pull\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2))))\n (:action screw\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2)))\n (:action unscrew\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2))))\n (:action load_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (is_equippable ?tool) (is_free ?hand))\n :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool)))\n (:action unload_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (hold ?hand ?tool) (is_free ?tool))\n :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool))))\n)\n\"\"\"\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_free( or ): or is free and can hold something. (default: true)\n- is_equippable(): is ready to be eqquipped by a robot hand. (default: true)\n- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false)\n- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false)\n- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false)\n- can_place_to(, ): can be placed into (placement assembly constraint) (default: false)\n- hold(, ): is holding (default: false)\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding .\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into .\n- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free.\n- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: unload_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to unload the outwardgripper in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outwardgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\"\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding a outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"Unload the outwardgripper in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outwardgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: load_tool(left_hand, parallel_box1)\n---------------\n\n\n Return a JSON object.\n "}, {"role": "assistant", "content": "{\n \"summary\": \"Selector to load the parallel_box1 in the left hand\",\n \"name\": \"selector: load_tool(left_hand, parallel_box1)\",\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is holding parallel_box1\",\n \"name\": \"target: hold(left_hand, parallel_box1)\"\n },\n {\n \"summary\": \"Sequence to load the tool in the left hand\",\n \"name\": \"sequence: load_tool(left_hand, parallel_box1)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is free\",\n \"name\": \"precondition: is_free(left_hand)\"\n },\n {\n \"summary\": \"check the precondition that the parallel_box1 is equippable\",\n \"name\": \"precondition: is_equippable(parallel_box1)\"\n },\n {\n \"summary\": \"Load the parallel_box1 in the left hand\",\n \"name\": \"action: load_tool(left_hand, parallel_box1)\"\n }\n ]\n }\n ]\n}"}]} -{"messages": [{"role": "system", "content": "You are an excellent behavior tree builder for robotic assembly tasks"}, {"role": "user", "content": "You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you construct robot task plans represented by behaiovr trees. \n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need in pddl domain format, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\"\"\"\n(define (domain robot_assembly_problem-domain)\n (:requirements :strips :typing)\n (:types\n thing - object\n tool part hand - thing\n )\n (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part))\n (:action pick_up\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part))\n :effect (and (hold ?tool ?part) (not (is_free ?tool))))\n (:action put_down\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (hold ?tool ?part) (hold ?hand ?tool))\n :effect (and (not (hold ?hand ?part)) (is_free ?tool)))\n (:action place\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2)))\n (:action detach\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2))))\n (:action insert\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2)))\n (:action pull\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2))))\n (:action screw\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2)))\n (:action unscrew\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2))))\n (:action load_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (is_equippable ?tool) (is_free ?hand))\n :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool)))\n (:action unload_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (hold ?hand ?tool) (is_free ?tool))\n :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool))))\n)\n\"\"\"\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_free( or ): or is free and can hold something. (default: true)\n- is_equippable(): is ready to be eqquipped by a robot hand. (default: true)\n- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false)\n- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false)\n- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false)\n- can_place_to(, ): can be placed into (placement assembly constraint) (default: false)\n- hold(, ): is holding (default: false)\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding .\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into .\n- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free.\n- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: unload_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to unload the outwardgripper in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outwardgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\"\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding a outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"Unload the outwardgripper in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outwardgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: load_tool(left_hand, parallel_box1)\n---------------\n\n\n Return a JSON object.\n "}, {"role": "assistant", "content": "{\n \"summary\": \"Selector to load the parallel_box1 in the left hand\",\n \"name\": \"selector: load_tool(left_hand, parallel_box1)\",\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is holding parallel_box1\",\n \"name\": \"target: hold(left_hand, parallel_box1)\"\n },\n {\n \"summary\": \"Sequence to load the tool in the left hand\",\n \"name\": \"sequence: load_tool(left_hand, parallel_box1)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the parallel_box1 is equippable\",\n \"name\": \"precondition: is_equippable(parallel_box1)\"\n },\n {\n \"summary\": \"check the precondition that the left hand is free\",\n \"name\": \"precondition: is_free(left_hand)\"\n },\n {\n \"summary\": \"Load the parallel_box1 in the left hand\",\n \"name\": \"action: load_tool(left_hand, parallel_box1)\"\n }\n ]\n }\n ]\n}"}]} -{"messages": [{"role": "system", "content": "You are an excellent behavior tree builder for robotic assembly tasks"}, {"role": "user", "content": "You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you construct robot task plans represented by behaiovr trees. \n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need in pddl domain format, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\"\"\"\n(define (domain robot_assembly_problem-domain)\n (:requirements :strips :typing)\n (:types\n thing - object\n tool part hand - thing\n )\n (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part))\n (:action pick_up\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part))\n :effect (and (hold ?tool ?part) (not (is_free ?tool))))\n (:action put_down\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (hold ?tool ?part) (hold ?hand ?tool))\n :effect (and (not (hold ?hand ?part)) (is_free ?tool)))\n (:action place\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2)))\n (:action detach\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2))))\n (:action insert\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2)))\n (:action pull\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2))))\n (:action screw\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2)))\n (:action unscrew\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2))))\n (:action load_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (is_equippable ?tool) (is_free ?hand))\n :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool)))\n (:action unload_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (hold ?hand ?tool) (is_free ?tool))\n :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool))))\n)\n\"\"\"\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_free( or ): or is free and can hold something. (default: true)\n- is_equippable(): is ready to be eqquipped by a robot hand. (default: true)\n- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false)\n- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false)\n- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false)\n- can_place_to(, ): can be placed into (placement assembly constraint) (default: false)\n- hold(, ): is holding (default: false)\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding .\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into .\n- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free.\n- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: unload_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to unload the outwardgripper in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outwardgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\"\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding a outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"Unload the outwardgripper in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outwardgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)\n---------------\n\n\n Return a JSON object.\n "}, {"role": "assistant", "content": "{\n \"summary\": \"Selector to insert shaft1 into gearbase_hole1 using parallel_box1 in the left hand\",\n \"name\": \"selector: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)\",\n \"children\": [\n {\n \"summary\": \"Check the target that shaft1 is inserted into gearbase_hole1\",\n \"name\": \"target: is_inserted_to(shaft1, gearbase_hole1)\"\n },\n {\n \"summary\": \"Sequence to insert shaft1 into gearbase_hole1 using parallel_box1 in the left hand\",\n \"name\": \"sequence: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding parallel_box1\",\n \"name\": \"precondition: hold(left_hand, parallel_box1)\"\n },\n {\n \"summary\": \"check the precondition that parallel_box1 is holding shaft1\",\n \"name\": \"precondition: hold(parallel_box1, shaft1)\"\n },\n {\n \"summary\": \"check the precondition that shaft1 can be inserted into gearbase_hole1\",\n \"name\": \"precondition: can_insert_to(shaft1, gearbase_hole1)\"\n },\n {\n \"summary\": \"Insert shaft1 into gearbase_hole1 using parallel_box1 in the left hand\",\n \"name\": \"action: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)\"\n }\n ]\n }\n ]\n}"}]} -{"messages": [{"role": "system", "content": "You are an excellent behavior tree builder for robotic assembly tasks"}, {"role": "user", "content": "You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you construct robot task plans represented by behaiovr trees. \n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need in pddl domain format, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\"\"\"\n(define (domain robot_assembly_problem-domain)\n (:requirements :strips :typing)\n (:types\n thing - object\n tool part hand - thing\n )\n (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part))\n (:action pick_up\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part))\n :effect (and (hold ?tool ?part) (not (is_free ?tool))))\n (:action put_down\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (hold ?tool ?part) (hold ?hand ?tool))\n :effect (and (not (hold ?hand ?part)) (is_free ?tool)))\n (:action place\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2)))\n (:action detach\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2))))\n (:action insert\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2)))\n (:action pull\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2))))\n (:action screw\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2)))\n (:action unscrew\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2))))\n (:action load_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (is_equippable ?tool) (is_free ?hand))\n :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool)))\n (:action unload_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (hold ?hand ?tool) (is_free ?tool))\n :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool))))\n)\n\"\"\"\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_free( or ): or is free and can hold something. (default: true)\n- is_equippable(): is ready to be eqquipped by a robot hand. (default: true)\n- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false)\n- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false)\n- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false)\n- can_place_to(, ): can be placed into (placement assembly constraint) (default: false)\n- hold(, ): is holding (default: false)\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding .\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into .\n- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free.\n- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: unload_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to unload the outwardgripper in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outwardgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\"\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding a outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"Unload the outwardgripper in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outwardgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: load_tool(left_hand, parallel_box1)\n---------------\n\n\n Return a JSON object.\n "}, {"role": "assistant", "content": "{\n \"summary\": \"Selector to load the parallel_box1 in the left hand\",\n \"name\": \"selector: load_tool(left_hand, parallel_box1)\",\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is holding parallel_box1\",\n \"name\": \"target: hold(left_hand, parallel_box1)\"\n },\n {\n \"summary\": \"Sequence to load the tool in the left hand\",\n \"name\": \"sequence: load_tool(left_hand, parallel_box1)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is free\",\n \"name\": \"precondition: is_free(left_hand)\"\n },\n {\n \"summary\": \"check the precondition that the parallel_box1 is equippable\",\n \"name\": \"precondition: is_equippable(parallel_box1)\"\n },\n {\n \"summary\": \"Load the parallel_box1 in the left hand\",\n \"name\": \"action: load_tool(left_hand, parallel_box1)\"\n }\n ]\n }\n ]\n}"}]} -{"messages": [{"role": "system", "content": "You are an excellent behavior tree builder for robotic assembly tasks"}, {"role": "user", "content": "You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you construct robot task plans represented by behaiovr trees. \n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need in pddl domain format, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\"\"\"\n(define (domain robot_assembly_problem-domain)\n (:requirements :strips :typing)\n (:types\n thing - object\n tool part hand - thing\n )\n (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part))\n (:action pick_up\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part))\n :effect (and (hold ?tool ?part) (not (is_free ?tool))))\n (:action put_down\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (hold ?tool ?part) (hold ?hand ?tool))\n :effect (and (not (hold ?hand ?part)) (is_free ?tool)))\n (:action place\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2)))\n (:action detach\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2))))\n (:action insert\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2)))\n (:action pull\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2))))\n (:action screw\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2)))\n (:action unscrew\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2))))\n (:action load_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (is_equippable ?tool) (is_free ?hand))\n :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool)))\n (:action unload_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (hold ?hand ?tool) (is_free ?tool))\n :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool))))\n)\n\"\"\"\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_free( or ): or is free and can hold something. (default: true)\n- is_equippable(): is ready to be eqquipped by a robot hand. (default: true)\n- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false)\n- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false)\n- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false)\n- can_place_to(, ): can be placed into (placement assembly constraint) (default: false)\n- hold(, ): is holding (default: false)\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding .\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into .\n- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free.\n- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: unload_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to unload the outwardgripper in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outwardgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\"\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding a outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"Unload the outwardgripper in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outwardgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: unload_tool(left_hand, outward_claw)\n---------------\n\n\n Return a JSON object.\n "}, {"role": "assistant", "content": "{\n \"summary\": \"Selector to unload the outward_claw in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outward_claw)\",\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\"\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outward_claw)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding an outward_claw\",\n \"name\": \"precondition: hold(left_hand, outward_claw)\"\n },\n {\n \"summary\": \"Unload the outward_claw in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outward_claw)\"\n }\n ]\n }\n ]\n}"}]} -{"messages": [{"role": "system", "content": "You are an excellent behavior tree builder for robotic assembly tasks"}, {"role": "user", "content": "You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you construct robot task plans represented by behaiovr trees. \n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need in pddl domain format, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\"\"\"\n(define (domain robot_assembly_problem-domain)\n (:requirements :strips :typing)\n (:types\n thing - object\n tool part hand - thing\n )\n (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part))\n (:action pick_up\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part))\n :effect (and (hold ?tool ?part) (not (is_free ?tool))))\n (:action put_down\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (hold ?tool ?part) (hold ?hand ?tool))\n :effect (and (not (hold ?hand ?part)) (is_free ?tool)))\n (:action place\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2)))\n (:action detach\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2))))\n (:action insert\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2)))\n (:action pull\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2))))\n (:action screw\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2)))\n (:action unscrew\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2))))\n (:action load_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (is_equippable ?tool) (is_free ?hand))\n :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool)))\n (:action unload_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (hold ?hand ?tool) (is_free ?tool))\n :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool))))\n)\n\"\"\"\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_free( or ): or is free and can hold something. (default: true)\n- is_equippable(): is ready to be eqquipped by a robot hand. (default: true)\n- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false)\n- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false)\n- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false)\n- can_place_to(, ): can be placed into (placement assembly constraint) (default: false)\n- hold(, ): is holding (default: false)\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding .\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into .\n- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free.\n- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: unload_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to unload the outwardgripper in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outwardgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\"\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding a outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"Unload the outwardgripper in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outwardgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: load_tool(left_hand, parallel_box1)\n---------------\n\n\n Return a JSON object.\n "}, {"role": "assistant", "content": "{\n \"summary\": \"Selector to load the parallel_box1 in the left hand\",\n \"name\": \"selector: load_tool(left_hand, parallel_box1)\",\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is holding parallel_box1\",\n \"name\": \"target: hold(left_hand, parallel_box1)\"\n },\n {\n \"summary\": \"Sequence to load the tool in the left hand\",\n \"name\": \"sequence: load_tool(left_hand, parallel_box1)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is free\",\n \"name\": \"precondition: is_free(left_hand)\"\n },\n {\n \"summary\": \"check the precondition that the parallel_box1 is equippable\",\n \"name\": \"precondition: is_equippable(parallel_box1)\"\n },\n {\n \"summary\": \"Load the parallel_box1 in the left hand\",\n \"name\": \"action: load_tool(left_hand, parallel_box1)\"\n }\n ]\n }\n ]\n}"}]} -{"messages": [{"role": "system", "content": "You are an excellent behavior tree builder for robotic assembly tasks"}, {"role": "user", "content": "You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you construct robot task plans represented by behaiovr trees. \n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need in pddl domain format, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\"\"\"\n(define (domain robot_assembly_problem-domain)\n (:requirements :strips :typing)\n (:types\n thing - object\n tool part hand - thing\n )\n (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part))\n (:action pick_up\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part))\n :effect (and (hold ?tool ?part) (not (is_free ?tool))))\n (:action put_down\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (hold ?tool ?part) (hold ?hand ?tool))\n :effect (and (not (hold ?hand ?part)) (is_free ?tool)))\n (:action place\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2)))\n (:action detach\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2))))\n (:action insert\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2)))\n (:action pull\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2))))\n (:action screw\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2)))\n (:action unscrew\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2))))\n (:action load_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (is_equippable ?tool) (is_free ?hand))\n :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool)))\n (:action unload_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (hold ?hand ?tool) (is_free ?tool))\n :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool))))\n)\n\"\"\"\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_free( or ): or is free and can hold something. (default: true)\n- is_equippable(): is ready to be eqquipped by a robot hand. (default: true)\n- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false)\n- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false)\n- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false)\n- can_place_to(, ): can be placed into (placement assembly constraint) (default: false)\n- hold(, ): is holding (default: false)\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding .\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into .\n- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free.\n- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: unload_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to unload the outwardgripper in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outwardgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\"\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding a outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"Unload the outwardgripper in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outwardgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)\n---------------\n\n\n Return a JSON object.\n "}, {"role": "assistant", "content": "{\n \"summary\": \"Selector to insert shaft1 into gearbase_hole1 using parallel_box1 in the left hand\",\n \"name\": \"selector: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)\",\n \"children\": [\n {\n \"summary\": \"Check the target that shaft1 is inserted into gearbase_hole1\",\n \"name\": \"target: is_inserted_to(shaft1, gearbase_hole1)\"\n },\n {\n \"summary\": \"Sequence to insert shaft1 into gearbase_hole1 using parallel_box1 in the left hand\",\n \"name\": \"sequence: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding parallel_box1\",\n \"name\": \"precondition: hold(left_hand, parallel_box1)\"\n },\n {\n \"summary\": \"check the precondition that parallel_box1 is holding shaft1\",\n \"name\": \"precondition: hold(parallel_box1, shaft1)\"\n },\n {\n \"summary\": \"check the precondition that shaft1 can be inserted into gearbase_hole1\",\n \"name\": \"precondition: can_insert_to(shaft1, gearbase_hole1)\"\n },\n {\n \"summary\": \"Insert shaft1 into gearbase_hole1 using parallel_box1 in the left hand\",\n \"name\": \"action: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)\"\n }\n ]\n }\n ]\n}"}]} -{"messages": [{"role": "system", "content": "You are an excellent behavior tree builder for robotic assembly tasks"}, {"role": "user", "content": "You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you construct robot task plans represented by behaiovr trees. \n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need in pddl domain format, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\"\"\"\n(define (domain robot_assembly_problem-domain)\n (:requirements :strips :typing)\n (:types\n thing - object\n tool part hand - thing\n )\n (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part))\n (:action pick_up\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part))\n :effect (and (hold ?tool ?part) (not (is_free ?tool))))\n (:action put_down\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (hold ?tool ?part) (hold ?hand ?tool))\n :effect (and (not (hold ?hand ?part)) (is_free ?tool)))\n (:action place\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2)))\n (:action detach\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2))))\n (:action insert\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2)))\n (:action pull\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2))))\n (:action screw\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2)))\n (:action unscrew\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2))))\n (:action load_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (is_equippable ?tool) (is_free ?hand))\n :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool)))\n (:action unload_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (hold ?hand ?tool) (is_free ?tool))\n :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool))))\n)\n\"\"\"\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_free( or ): or is free and can hold something. (default: true)\n- is_equippable(): is ready to be eqquipped by a robot hand. (default: true)\n- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false)\n- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false)\n- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false)\n- can_place_to(, ): can be placed into (placement assembly constraint) (default: false)\n- hold(, ): is holding (default: false)\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding .\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into .\n- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free.\n- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: unload_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to unload the outwardgripper in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outwardgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\"\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding a outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"Unload the outwardgripper in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outwardgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: load_tool(left_hand, parallel_box1)\n---------------\n\n\n Return a JSON object.\n "}, {"role": "assistant", "content": "{\n \"summary\": \"Selector to load the parallel_box1 in the left hand\",\n \"name\": \"selector: load_tool(left_hand, parallel_box1)\",\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is holding parallel_box1\",\n \"name\": \"target: hold(left_hand, parallel_box1)\"\n },\n {\n \"summary\": \"Sequence to load the tool in the left hand\",\n \"name\": \"sequence: load_tool(left_hand, parallel_box1)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is free\",\n \"name\": \"precondition: is_free(left_hand)\"\n },\n {\n \"summary\": \"check the precondition that the parallel_box1 is equippable\",\n \"name\": \"precondition: is_equippable(parallel_box1)\"\n },\n {\n \"summary\": \"Load the parallel_box1 in the left hand\",\n \"name\": \"action: load_tool(left_hand, parallel_box1)\"\n }\n ]\n }\n ]\n}"}]} -{"messages": [{"role": "system", "content": "You are an excellent behavior tree builder for robotic assembly tasks"}, {"role": "user", "content": "You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you construct robot task plans represented by behaiovr trees. \n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need in pddl domain format, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\"\"\"\n(define (domain robot_assembly_problem-domain)\n (:requirements :strips :typing)\n (:types\n thing - object\n tool part hand - thing\n )\n (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part))\n (:action pick_up\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part))\n :effect (and (hold ?tool ?part) (not (is_free ?tool))))\n (:action put_down\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (hold ?tool ?part) (hold ?hand ?tool))\n :effect (and (not (hold ?hand ?part)) (is_free ?tool)))\n (:action place\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2)))\n (:action detach\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2))))\n (:action insert\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2)))\n (:action pull\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2))))\n (:action screw\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2)))\n (:action unscrew\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2))))\n (:action load_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (is_equippable ?tool) (is_free ?hand))\n :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool)))\n (:action unload_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (hold ?hand ?tool) (is_free ?tool))\n :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool))))\n)\n\"\"\"\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_free( or ): or is free and can hold something. (default: true)\n- is_equippable(): is ready to be eqquipped by a robot hand. (default: true)\n- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false)\n- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false)\n- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false)\n- can_place_to(, ): can be placed into (placement assembly constraint) (default: false)\n- hold(, ): is holding (default: false)\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding .\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into .\n- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free.\n- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: unload_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to unload the outwardgripper in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outwardgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\"\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding a outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"Unload the outwardgripper in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outwardgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: unload_tool(left_hand, outward_claw)\n---------------\n\n\n Return a JSON object.\n "}, {"role": "assistant", "content": "{\n \"summary\": \"Selector to unload the outward_claw in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outward_claw)\",\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\"\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outward_claw)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding an outward_claw\",\n \"name\": \"precondition: hold(left_hand, outward_claw)\"\n },\n {\n \"summary\": \"Unload the outward_claw in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outward_claw)\"\n }\n ]\n }\n ]\n}"}]} -{"messages": [{"role": "system", "content": "You are an excellent behavior tree builder for robotic assembly tasks"}, {"role": "user", "content": "You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you construct robot task plans represented by behaiovr trees. \n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need in pddl domain format, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\"\"\"\n(define (domain robot_assembly_problem-domain)\n (:requirements :strips :typing)\n (:types\n thing - object\n tool part hand - thing\n )\n (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part))\n (:action pick_up\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part))\n :effect (and (hold ?tool ?part) (not (is_free ?tool))))\n (:action put_down\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (hold ?tool ?part) (hold ?hand ?tool))\n :effect (and (not (hold ?hand ?part)) (is_free ?tool)))\n (:action place\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2)))\n (:action detach\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2))))\n (:action insert\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2)))\n (:action pull\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2))))\n (:action screw\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2)))\n (:action unscrew\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2))))\n (:action load_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (is_equippable ?tool) (is_free ?hand))\n :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool)))\n (:action unload_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (hold ?hand ?tool) (is_free ?tool))\n :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool))))\n)\n\"\"\"\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_free( or ): or is free and can hold something. (default: true)\n- is_equippable(): is ready to be eqquipped by a robot hand. (default: true)\n- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false)\n- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false)\n- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false)\n- can_place_to(, ): can be placed into (placement assembly constraint) (default: false)\n- hold(, ): is holding (default: false)\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding .\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into .\n- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free.\n- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: unload_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to unload the outwardgripper in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outwardgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\"\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding a outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"Unload the outwardgripper in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outwardgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: pick_up(left_hand, parallel_box1, shaft1)\n---------------\n\n\n Return a JSON object.\n "}, {"role": "assistant", "content": "{\n \"summary\": \"Selector to pick up shaft1 using parallel_box1 in the left hand\",\n \"name\": \"selector: pick_up(left_hand, parallel_box1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"Check the target that parallel_box1 is holding shaft1\",\n \"name\": \"target: hold(parallel_box1, shaft1)\"\n },\n {\n \"summary\": \"Sequence to pick up shaft1 using parallel_box1 in the left hand\",\n \"name\": \"sequence: pick_up(left_hand, parallel_box1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that parallel_box1 is free\",\n \"name\": \"precondition: is_free(parallel_box1)\"\n },\n {\n \"summary\": \"check the precondition that left hand is holding parallel_box1\",\n \"name\": \"precondition: hold(left_hand, parallel_box1)\"\n },\n {\n \"summary\": \"check the precondition that parallel_box1 can manipulate shaft1\",\n \"name\": \"precondition: can_manipulate(parallel_box1, shaft1)\"\n },\n {\n \"summary\": \"Pick up shaft1 using parallel_box1 in the left hand\",\n \"name\": \"action: pick_up(left_hand, parallel_box1, shaft1)\"\n }\n ]\n }\n ]\n}"}]} -{"messages": [{"role": "system", "content": "You are an excellent behavior tree builder for robotic assembly tasks"}, {"role": "user", "content": "You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you construct robot task plans represented by behaiovr trees. \n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need in pddl domain format, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\"\"\"\n(define (domain robot_assembly_problem-domain)\n (:requirements :strips :typing)\n (:types\n thing - object\n tool part hand - thing\n )\n (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part))\n (:action pick_up\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part))\n :effect (and (hold ?tool ?part) (not (is_free ?tool))))\n (:action put_down\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (hold ?tool ?part) (hold ?hand ?tool))\n :effect (and (not (hold ?hand ?part)) (is_free ?tool)))\n (:action place\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2)))\n (:action detach\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2))))\n (:action insert\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2)))\n (:action pull\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2))))\n (:action screw\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2)))\n (:action unscrew\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2))))\n (:action load_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (is_equippable ?tool) (is_free ?hand))\n :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool)))\n (:action unload_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (hold ?hand ?tool) (is_free ?tool))\n :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool))))\n)\n\"\"\"\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_free( or ): or is free and can hold something. (default: true)\n- is_equippable(): is ready to be eqquipped by a robot hand. (default: true)\n- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false)\n- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false)\n- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false)\n- can_place_to(, ): can be placed into (placement assembly constraint) (default: false)\n- hold(, ): is holding (default: false)\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding .\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into .\n- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free.\n- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: unload_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to unload the outwardgripper in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outwardgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\"\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding a outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"Unload the outwardgripper in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outwardgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: put_down(left_hand, parallel_box1, shaft1)\n---------------\n\n\n Return a JSON object.\n "}, {"role": "assistant", "content": "{\n \"summary\": \"Selector to put down the shaft1 in the left hand using parallel_box1\",\n \"name\": \"selector: put_down(left_hand, parallel_box1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"Check the target that the parallel_box1 is free\",\n \"name\": \"target: is_free(parallel_box1)\"\n },\n {\n \"summary\": \"Sequence to put down the shaft1 in the left hand using parallel_box1\",\n \"name\": \"sequence: put_down(left_hand, parallel_box1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding a parallel_box1\",\n \"name\": \"precondition: hold(left_hand, parallel_box1)\"\n },\n {\n \"summary\": \"check the precondition that the parallel_box1 is holding a shaft1\",\n \"name\": \"precondition: hold(parallel_box1, shaft1)\"\n },\n {\n \"summary\": \"Put down the shaft1 in the left hand using parallel_box1\",\n \"name\": \"action: put_down(left_hand, parallel_box1, shaft1)\"\n }\n ]\n }\n ]\n}"}]} -{"messages": [{"role": "system", "content": "You are an excellent behavior tree builder for robotic assembly tasks"}, {"role": "user", "content": "You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you construct robot task plans represented by behaiovr trees. \n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need in pddl domain format, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\"\"\"\n(define (domain robot_assembly_problem-domain)\n (:requirements :strips :typing)\n (:types\n thing - object\n tool part hand - thing\n )\n (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part))\n (:action pick_up\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part))\n :effect (and (hold ?tool ?part) (not (is_free ?tool))))\n (:action put_down\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (hold ?tool ?part) (hold ?hand ?tool))\n :effect (and (not (hold ?hand ?part)) (is_free ?tool)))\n (:action place\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2)))\n (:action detach\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2))))\n (:action insert\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2)))\n (:action pull\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2))))\n (:action screw\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2)))\n (:action unscrew\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2))))\n (:action load_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (is_equippable ?tool) (is_free ?hand))\n :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool)))\n (:action unload_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (hold ?hand ?tool) (is_free ?tool))\n :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool))))\n)\n\"\"\"\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_free( or ): or is free and can hold something. (default: true)\n- is_equippable(): is ready to be eqquipped by a robot hand. (default: true)\n- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false)\n- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false)\n- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false)\n- can_place_to(, ): can be placed into (placement assembly constraint) (default: false)\n- hold(, ): is holding (default: false)\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding .\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into .\n- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free.\n- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: unload_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to unload the outwardgripper in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outwardgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\"\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding a outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"Unload the outwardgripper in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outwardgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)\n---------------\n\n\n Return a JSON object.\n "}, {"role": "assistant", "content": "{\n \"summary\": \"Selector to insert shaft1 into gearbase_hole1 using parallel_box1 in the left hand\",\n \"name\": \"selector: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)\",\n \"children\": [\n {\n \"summary\": \"Check the target that shaft1 is inserted into gearbase_hole1\",\n \"name\": \"target: is_inserted_to(shaft1, gearbase_hole1)\"\n },\n {\n \"summary\": \"Sequence to insert shaft1 into gearbase_hole1 using parallel_box1 in the left hand\",\n \"name\": \"sequence: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding a parallel_box1\",\n \"name\": \"precondition: hold(left_hand, parallel_box1)\"\n },\n {\n \"summary\": \"check the precondition that the parallel_box1 is holding a shaft1\",\n \"name\": \"precondition: hold(parallel_box1, shaft1)\"\n },\n {\n \"summary\": \"check the precondition that shaft1 can be inserted into gearbase_hole1\",\n \"name\": \"precondition: can_insert_to(shaft1, gearbase_hole1)\"\n },\n {\n \"summary\": \"Insert shaft1 into gearbase_hole1 using parallel_box1 in the left hand\",\n \"name\": \"action: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)\"\n }\n ]\n }\n ]\n}"}]} -{"messages": [{"role": "system", "content": "You are an excellent behavior tree builder for robotic assembly tasks"}, {"role": "user", "content": "You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you construct robot task plans represented by behaiovr trees. \n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need in pddl domain format, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\"\"\"\n(define (domain robot_assembly_problem-domain)\n (:requirements :strips :typing)\n (:types\n thing - object\n tool part hand - thing\n )\n (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part))\n (:action pick_up\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part))\n :effect (and (hold ?tool ?part) (not (is_free ?tool))))\n (:action put_down\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (hold ?tool ?part) (hold ?hand ?tool))\n :effect (and (not (hold ?hand ?part)) (is_free ?tool)))\n (:action place\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2)))\n (:action detach\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2))))\n (:action insert\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2)))\n (:action pull\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2))))\n (:action screw\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2)))\n (:action unscrew\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2))))\n (:action load_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (is_equippable ?tool) (is_free ?hand))\n :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool)))\n (:action unload_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (hold ?hand ?tool) (is_free ?tool))\n :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool))))\n)\n\"\"\"\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_free( or ): or is free and can hold something. (default: true)\n- is_equippable(): is ready to be eqquipped by a robot hand. (default: true)\n- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false)\n- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false)\n- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false)\n- can_place_to(, ): can be placed into (placement assembly constraint) (default: false)\n- hold(, ): is holding (default: false)\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding .\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into .\n- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free.\n- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: unload_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to unload the outwardgripper in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outwardgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\"\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding a outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"Unload the outwardgripper in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outwardgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: load_tool(left_hand, parallel_box1)\n---------------\n\n\n Return a JSON object.\n "}, {"role": "assistant", "content": "{\n \"summary\": \"Selector to load the parallel_box1 in the left hand\",\n \"name\": \"selector: load_tool(left_hand, parallel_box1)\",\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is holding a parallel_box1\",\n \"name\": \"target: hold(left_hand, parallel_box1)\"\n },\n {\n \"summary\": \"Sequence to load the tool in the left hand\",\n \"name\": \"sequence: load_tool(left_hand, parallel_box1)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is free\",\n \"name\": \"precondition: is_free(left_hand)\"\n },\n {\n \"summary\": \"check the precondition that the parallel_box1 is equippable\",\n \"name\": \"precondition: is_equippable(parallel_box1)\"\n },\n {\n \"summary\": \"Load the parallel_box1 in the left hand\",\n \"name\": \"action: load_tool(left_hand, parallel_box1)\"\n }\n ]\n }\n ]\n}"}]} -{"messages": [{"role": "system", "content": "You are an excellent behavior tree builder for robotic assembly tasks"}, {"role": "user", "content": "You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you construct robot task plans represented by behaiovr trees. \n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need in pddl domain format, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\"\"\"\n(define (domain robot_assembly_problem-domain)\n (:requirements :strips :typing)\n (:types\n thing - object\n tool part hand - thing\n )\n (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part))\n (:action pick_up\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part))\n :effect (and (hold ?tool ?part) (not (is_free ?tool))))\n (:action put_down\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (hold ?tool ?part) (hold ?hand ?tool))\n :effect (and (not (hold ?hand ?part)) (is_free ?tool)))\n (:action place\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2)))\n (:action detach\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2))))\n (:action insert\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2)))\n (:action pull\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2))))\n (:action screw\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2)))\n (:action unscrew\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2))))\n (:action load_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (is_equippable ?tool) (is_free ?hand))\n :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool)))\n (:action unload_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (hold ?hand ?tool) (is_free ?tool))\n :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool))))\n)\n\"\"\"\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_free( or ): or is free and can hold something. (default: true)\n- is_equippable(): is ready to be eqquipped by a robot hand. (default: true)\n- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false)\n- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false)\n- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false)\n- can_place_to(, ): can be placed into (placement assembly constraint) (default: false)\n- hold(, ): is holding (default: false)\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding .\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into .\n- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free.\n- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: unload_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to unload the outwardgripper in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outwardgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\"\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding a outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"Unload the outwardgripper in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outwardgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: unload_tool(left_hand, outward_claw)\n---------------\n\n\n Return a JSON object.\n "}, {"role": "assistant", "content": "{\n \"summary\": \"Selector to unload the outward_claw in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outward_claw)\",\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\"\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outward_claw)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding an outward_claw\",\n \"name\": \"precondition: hold(left_hand, outward_claw)\"\n },\n {\n \"summary\": \"Unload the outward_claw in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outward_claw)\"\n }\n ]\n }\n ]\n}"}]} -{"messages": [{"role": "system", "content": "You are an excellent behavior tree builder for robotic assembly tasks"}, {"role": "user", "content": "You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you construct robot task plans represented by behaiovr trees. \n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need in pddl domain format, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\"\"\"\n(define (domain robot_assembly_problem-domain)\n (:requirements :strips :typing)\n (:types\n thing - object\n tool part hand - thing\n )\n (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part))\n (:action pick_up\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part))\n :effect (and (hold ?tool ?part) (not (is_free ?tool))))\n (:action put_down\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (hold ?tool ?part) (hold ?hand ?tool))\n :effect (and (not (hold ?hand ?part)) (is_free ?tool)))\n (:action place\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2)))\n (:action detach\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2))))\n (:action insert\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2)))\n (:action pull\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2))))\n (:action screw\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2)))\n (:action unscrew\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2))))\n (:action load_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (is_equippable ?tool) (is_free ?hand))\n :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool)))\n (:action unload_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (hold ?hand ?tool) (is_free ?tool))\n :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool))))\n)\n\"\"\"\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_free( or ): or is free and can hold something. (default: true)\n- is_equippable(): is ready to be eqquipped by a robot hand. (default: true)\n- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false)\n- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false)\n- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false)\n- can_place_to(, ): can be placed into (placement assembly constraint) (default: false)\n- hold(, ): is holding (default: false)\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding .\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into .\n- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free.\n- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: unload_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to unload the outwardgripper in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outwardgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\"\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding a outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"Unload the outwardgripper in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outwardgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: load_tool(left_hand, parallel_box1)\n---------------\n\n\n Return a JSON object.\n "}, {"role": "assistant", "content": "{\n \"summary\": \"Selector to load the parallel_box1 in the left hand\",\n \"name\": \"selector: load_tool(left_hand, parallel_box1)\",\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is holding parallel_box1\",\n \"name\": \"target: hold(left_hand, parallel_box1)\"\n },\n {\n \"summary\": \"Sequence to load the tool in the left hand\",\n \"name\": \"sequence: load_tool(left_hand, parallel_box1)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is free\",\n \"name\": \"precondition: is_free(left_hand)\"\n },\n {\n \"summary\": \"check the precondition that the parallel_box1 is equippable\",\n \"name\": \"precondition: is_equippable(parallel_box1)\"\n },\n {\n \"summary\": \"Load the parallel_box1 in the left hand\",\n \"name\": \"action: load_tool(left_hand, parallel_box1)\"\n }\n ]\n }\n ]\n}"}]} -{"messages": [{"role": "system", "content": "You are an excellent behavior tree builder for robotic assembly tasks"}, {"role": "user", "content": "You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you construct robot task plans represented by behaiovr trees. \n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need in pddl domain format, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\"\"\"\n(define (domain robot_assembly_problem-domain)\n (:requirements :strips :typing)\n (:types\n thing - object\n tool part hand - thing\n )\n (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part))\n (:action pick_up\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part))\n :effect (and (hold ?tool ?part) (not (is_free ?tool))))\n (:action put_down\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (hold ?tool ?part) (hold ?hand ?tool))\n :effect (and (not (hold ?hand ?part)) (is_free ?tool)))\n (:action place\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2)))\n (:action detach\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2))))\n (:action insert\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2)))\n (:action pull\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2))))\n (:action screw\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2)))\n (:action unscrew\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2))))\n (:action load_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (is_equippable ?tool) (is_free ?hand))\n :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool)))\n (:action unload_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (hold ?hand ?tool) (is_free ?tool))\n :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool))))\n)\n\"\"\"\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_free( or ): or is free and can hold something. (default: true)\n- is_equippable(): is ready to be eqquipped by a robot hand. (default: true)\n- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false)\n- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false)\n- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false)\n- can_place_to(, ): can be placed into (placement assembly constraint) (default: false)\n- hold(, ): is holding (default: false)\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding .\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into .\n- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free.\n- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: unload_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to unload the outwardgripper in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outwardgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\"\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding a outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"Unload the outwardgripper in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outwardgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)\n---------------\n\n\n Return a JSON object.\n "}, {"role": "assistant", "content": "{\n \"summary\": \"Selector to insert shaft1 into gearbase_hole1 using parallel_box1 in the left hand\",\n \"name\": \"selector: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)\",\n \"children\": [\n {\n \"summary\": \"Check the target that shaft1 is inserted into gearbase_hole1\",\n \"name\": \"target: is_inserted_to(shaft1, gearbase_hole1)\"\n },\n {\n \"summary\": \"Sequence to insert shaft1 into gearbase_hole1 using parallel_box1 in the left hand\",\n \"name\": \"sequence: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding a parallel_box1\",\n \"name\": \"precondition: hold(left_hand, parallel_box1)\"\n },\n {\n \"summary\": \"check the precondition that the parallel_box1 is holding a shaft1\",\n \"name\": \"precondition: hold(parallel_box1, shaft1)\"\n },\n {\n \"summary\": \"check the precondition that the shaft1 can be inserted into gearbase_hole1\",\n \"name\": \"precondition: can_insert_to(shaft1, gearbase_hole1)\"\n },\n {\n \"summary\": \"Insert shaft1 into gearbase_hole1 using parallel_box1 in the left hand\",\n \"name\": \"action: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)\"\n }\n ]\n }\n ]\n}"}]} -{"messages": [{"role": "system", "content": "You are an excellent behavior tree builder for robotic assembly tasks"}, {"role": "user", "content": "You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you construct robot task plans represented by behaiovr trees. \n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need in pddl domain format, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\"\"\"\n(define (domain robot_assembly_problem-domain)\n (:requirements :strips :typing)\n (:types\n thing - object\n tool part hand - thing\n )\n (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part))\n (:action pick_up\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part))\n :effect (and (hold ?tool ?part) (not (is_free ?tool))))\n (:action put_down\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (hold ?tool ?part) (hold ?hand ?tool))\n :effect (and (not (hold ?hand ?part)) (is_free ?tool)))\n (:action place\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2)))\n (:action detach\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2))))\n (:action insert\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2)))\n (:action pull\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2))))\n (:action screw\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2)))\n (:action unscrew\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2))))\n (:action load_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (is_equippable ?tool) (is_free ?hand))\n :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool)))\n (:action unload_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (hold ?hand ?tool) (is_free ?tool))\n :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool))))\n)\n\"\"\"\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_free( or ): or is free and can hold something. (default: true)\n- is_equippable(): is ready to be eqquipped by a robot hand. (default: true)\n- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false)\n- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false)\n- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false)\n- can_place_to(, ): can be placed into (placement assembly constraint) (default: false)\n- hold(, ): is holding (default: false)\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding .\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into .\n- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free.\n- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: unload_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to unload the outwardgripper in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outwardgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\"\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding a outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"Unload the outwardgripper in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outwardgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: load_tool(left_hand, parallel_box1)\n---------------\n\n\n Return a JSON object.\n "}, {"role": "assistant", "content": "{\n \"summary\": \"Selector to load the parallel_box1 in the left hand\",\n \"name\": \"selector: load_tool(left_hand, parallel_box1)\",\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is holding parallel_box1\",\n \"name\": \"target: hold(left_hand, parallel_box1)\"\n },\n {\n \"summary\": \"Sequence to load the tool in the left hand\",\n \"name\": \"sequence: load_tool(left_hand, parallel_box1)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is free\",\n \"name\": \"precondition: is_free(left_hand)\"\n },\n {\n \"summary\": \"check the precondition that the parallel_box1 is equippable\",\n \"name\": \"precondition: is_equippable(parallel_box1)\"\n },\n {\n \"summary\": \"Load the parallel_box1 in the left hand\",\n \"name\": \"action: load_tool(left_hand, parallel_box1)\"\n }\n ]\n }\n ]\n}"}]} -{"messages": [{"role": "system", "content": "You are an excellent behavior tree builder for robotic assembly tasks"}, {"role": "user", "content": "You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you construct robot task plans represented by behaiovr trees. \n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need in pddl domain format, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\"\"\"\n(define (domain robot_assembly_problem-domain)\n (:requirements :strips :typing)\n (:types\n thing - object\n tool part hand - thing\n )\n (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part))\n (:action pick_up\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part))\n :effect (and (hold ?tool ?part) (not (is_free ?tool))))\n (:action put_down\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (hold ?tool ?part) (hold ?hand ?tool))\n :effect (and (not (hold ?hand ?part)) (is_free ?tool)))\n (:action place\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2)))\n (:action detach\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2))))\n (:action insert\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2)))\n (:action pull\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2))))\n (:action screw\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2)))\n (:action unscrew\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2))))\n (:action load_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (is_equippable ?tool) (is_free ?hand))\n :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool)))\n (:action unload_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (hold ?hand ?tool) (is_free ?tool))\n :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool))))\n)\n\"\"\"\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_free( or ): or is free and can hold something. (default: true)\n- is_equippable(): is ready to be eqquipped by a robot hand. (default: true)\n- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false)\n- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false)\n- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false)\n- can_place_to(, ): can be placed into (placement assembly constraint) (default: false)\n- hold(, ): is holding (default: false)\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding .\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into .\n- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free.\n- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: unload_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to unload the outwardgripper in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outwardgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\"\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding a outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"Unload the outwardgripper in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outwardgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: unload_tool(left_hand, outward_claw)\n---------------\n\n\n Return a JSON object.\n "}, {"role": "assistant", "content": "{\n \"summary\": \"Selector to unload the outward_claw in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outward_claw)\",\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\"\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outward_claw)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding an outward_claw\",\n \"name\": \"precondition: hold(left_hand, outward_claw)\"\n },\n {\n \"summary\": \"Unload the outward_claw in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outward_claw)\"\n }\n ]\n }\n ]\n}"}]} -{"messages": [{"role": "system", "content": "You are an excellent behavior tree builder for robotic assembly tasks"}, {"role": "user", "content": "You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you construct robot task plans represented by behaiovr trees. \n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need in pddl domain format, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\"\"\"\n(define (domain robot_assembly_problem-domain)\n (:requirements :strips :typing)\n (:types\n thing - object\n tool part hand - thing\n )\n (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part))\n (:action pick_up\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part))\n :effect (and (hold ?tool ?part) (not (is_free ?tool))))\n (:action put_down\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (hold ?tool ?part) (hold ?hand ?tool))\n :effect (and (not (hold ?hand ?part)) (is_free ?tool)))\n (:action place\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2)))\n (:action detach\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2))))\n (:action insert\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2)))\n (:action pull\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2))))\n (:action screw\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2)))\n (:action unscrew\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2))))\n (:action load_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (is_equippable ?tool) (is_free ?hand))\n :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool)))\n (:action unload_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (hold ?hand ?tool) (is_free ?tool))\n :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool))))\n)\n\"\"\"\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_free( or ): or is free and can hold something. (default: true)\n- is_equippable(): is ready to be eqquipped by a robot hand. (default: true)\n- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false)\n- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false)\n- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false)\n- can_place_to(, ): can be placed into (placement assembly constraint) (default: false)\n- hold(, ): is holding (default: false)\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding .\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into .\n- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free.\n- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: unload_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to unload the outwardgripper in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outwardgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\"\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding a outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"Unload the outwardgripper in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outwardgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: pick_up(left_hand, parallel_box1, shaft1)\n---------------\n\n\n Return a JSON object.\n "}, {"role": "assistant", "content": "{\n \"summary\": \"Selector to pick up shaft1 using parallel_box1 in the left hand\",\n \"name\": \"selector: pick_up(left_hand, parallel_box1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"Check the target that parallel_box1 is holding shaft1\",\n \"name\": \"target: hold(parallel_box1, shaft1)\"\n },\n {\n \"summary\": \"Sequence to pick up shaft1 using parallel_box1 in the left hand\",\n \"name\": \"sequence: pick_up(left_hand, parallel_box1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that parallel_box1 is free\",\n \"name\": \"precondition: is_free(parallel_box1)\"\n },\n {\n \"summary\": \"check the precondition that the left hand is holding parallel_box1\",\n \"name\": \"precondition: hold(left_hand, parallel_box1)\"\n },\n {\n \"summary\": \"check the precondition that parallel_box1 can manipulate shaft1\",\n \"name\": \"precondition: can_manipulate(parallel_box1, shaft1)\"\n },\n {\n \"summary\": \"Pick up shaft1 using parallel_box1 in the left hand\",\n \"name\": \"action: pick_up(left_hand, parallel_box1, shaft1)\"\n }\n ]\n }\n ]\n}"}]} -{"messages": [{"role": "system", "content": "You are an excellent behavior tree builder for robotic assembly tasks"}, {"role": "user", "content": "You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you construct robot task plans represented by behaiovr trees. \n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need in pddl domain format, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\"\"\"\n(define (domain robot_assembly_problem-domain)\n (:requirements :strips :typing)\n (:types\n thing - object\n tool part hand - thing\n )\n (:predicates (is_empty ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part))\n (:action pick_up\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (is_empty ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part))\n :effect (and (hold ?tool ?part) (not (is_empty ?tool))))\n (:action put_down\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (hold ?tool ?part) (hold ?hand ?tool))\n :effect (and (not (hold ?hand ?part)) (is_empty ?tool)))\n (:action place\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_empty ?tool) (is_placed_to ?part1 ?part2)))\n (:action detach\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_empty ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2))\n :effect (and (hold ?tool ?part1) (not (is_empty ?tool)) (not (is_placed_to ?part1 ?part2))))\n (:action insert\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_empty ?tool) (is_inserted_to ?part1 ?part2)))\n (:action pull\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_empty ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_empty ?tool)) (not (is_inserted_to ?part1 ?part2))))\n (:action screw\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_empty ?tool) (is_screwed_to ?part1 ?part2)))\n (:action unscrew\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_empty ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_empty ?tool)) (not (is_screwed_to ?part1 ?part2))))\n (:action load_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (is_equippable ?tool) (is_empty ?hand))\n :effect (and (not (is_empty ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool)))\n (:action unload_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (hold ?hand ?tool) (is_empty ?tool))\n :effect (and (is_empty ?hand) (is_equippable ?tool) (not (hold ?hand ?tool))))\n)\n\"\"\"\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_empty( or ): or is empty and can hold something. (default: true)\n- is_equippable(): is ready to be eqquipped by a robot hand. (default: true)\n- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false)\n- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false)\n- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false)\n- can_place_to(, ): can be placed into (placement assembly constraint) (default: false)\n- hold(, ): is holding (default: false)\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be empty to hold something.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding .\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be screwed into .\n- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be empty.\n- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be empty.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: unload_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to unload the outwardgripper in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outwardgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is empty\",\n \"name\": \"target: is_empty(left_hand)\"\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding a outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"Unload the outwardgripper in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outwardgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: pick_up(left_hand, parallel_box1, shaft1)\n---------------\n\n\n Return a JSON object.\n "}, {"role": "assistant", "content": "{\n \"summary\": \"Selector to pick up the shaft1 with parallel_box1 in the left hand\",\n \"name\": \"selector: pick_up(left_hand, parallel_box1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"Check the target that the parallel_box1 is holding shaft1\",\n \"name\": \"target: hold(parallel_box1, shaft1)\"\n },\n {\n \"summary\": \"Sequence to pick up the shaft1 with parallel_box1 in the left hand\",\n \"name\": \"sequence: pick_up(left_hand, parallel_box1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the parallel_box1 is empty\",\n \"name\": \"precondition: is_empty(parallel_box1)\"\n },\n {\n \"summary\": \"check the precondition that the left hand is holding a parallel_box1\",\n \"name\": \"precondition: hold(left_hand, parallel_box1)\"\n },\n {\n \"summary\": \"check the precondition that the parallel_box1 can manipulate shaft1\",\n \"name\": \"precondition: can_manipulate(parallel_box1, shaft1)\"\n },\n {\n \"summary\": \"Pick up the shaft1 with parallel_box1 in the left hand\",\n \"name\": \"action: pick_up(left_hand, parallel_box1, shaft1)\"\n }\n ]\n }\n ]\n}"}]} -{"messages": [{"role": "system", "content": "You are an excellent behavior tree builder for robotic assembly tasks"}, {"role": "user", "content": "You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you construct robot task plans represented by behaiovr trees. \n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need in pddl domain format, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\"\"\"\n(define (domain robot_assembly_problem-domain)\n (:requirements :strips :typing)\n (:types\n thing - object\n tool part hand - thing\n )\n (:predicates (is_empty ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part))\n (:action pick_up\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (is_empty ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part))\n :effect (and (hold ?tool ?part) (not (is_empty ?tool))))\n (:action put_down\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (hold ?tool ?part) (hold ?hand ?tool))\n :effect (and (not (hold ?hand ?part)) (is_empty ?tool)))\n (:action place\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_empty ?tool) (is_placed_to ?part1 ?part2)))\n (:action detach\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_empty ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2))\n :effect (and (hold ?tool ?part1) (not (is_empty ?tool)) (not (is_placed_to ?part1 ?part2))))\n (:action insert\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_empty ?tool) (is_inserted_to ?part1 ?part2)))\n (:action pull\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_empty ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_empty ?tool)) (not (is_inserted_to ?part1 ?part2))))\n (:action screw\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_empty ?tool) (is_screwed_to ?part1 ?part2)))\n (:action unscrew\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_empty ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_empty ?tool)) (not (is_screwed_to ?part1 ?part2))))\n (:action load_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (is_equippable ?tool) (is_empty ?hand))\n :effect (and (not (is_empty ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool)))\n (:action unload_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (hold ?hand ?tool) (is_empty ?tool))\n :effect (and (is_empty ?hand) (is_equippable ?tool) (not (hold ?hand ?tool))))\n)\n\"\"\"\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_empty( or ): or is empty and can hold something. (default: true)\n- is_equippable(): is ready to be eqquipped by a robot hand. (default: true)\n- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false)\n- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false)\n- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false)\n- can_place_to(, ): can be placed into (placement assembly constraint) (default: false)\n- hold(, ): is holding (default: false)\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be empty to hold something.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding .\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be screwed into .\n- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be empty.\n- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be empty.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: unload_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to unload the outwardgripper in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outwardgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is empty\",\n \"name\": \"target: is_empty(left_hand)\"\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding a outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"Unload the outwardgripper in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outwardgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: pick_up(left_hand, parallel_box1, shaft1)\n---------------\n\n\n Return a JSON object.\n "}, {"role": "assistant", "content": "{\n \"summary\": \"Selector to pick up shaft1 with parallel_box1 using left hand\",\n \"name\": \"selector: pick_up(left_hand, parallel_box1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"Check the target that shaft1 is held by parallel_box1\",\n \"name\": \"target: hold(parallel_box1, shaft1)\"\n },\n {\n \"summary\": \"Sequence to pick up shaft1 with parallel_box1 using left hand\",\n \"name\": \"sequence: pick_up(left_hand, parallel_box1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"Check the precondition that left hand is holding parallel_box1 and parallel_box1 is empty\",\n \"name\": \"precondition: hold(left_hand, parallel_box1) and is_empty(parallel_box1)\"\n },\n {\n \"summary\": \"Pick up shaft1 with parallel_box1 using left hand\",\n \"name\": \"action: pick_up(left_hand, parallel_box1, shaft1)\"\n }\n ]\n }\n ]\n}"}]} -{"messages": [{"role": "system", "content": "You are an excellent behavior tree builder for robotic assembly tasks"}, {"role": "user", "content": "You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you construct robot task plans represented by behaiovr trees. \n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need in pddl domain format, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\"\"\"\n(define (domain robot_assembly_problem-domain)\n (:requirements :strips :typing)\n (:types\n thing - object\n tool part hand - thing\n )\n (:predicates (is_empty ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part))\n (:action pick_up\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (is_empty ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part))\n :effect (and (hold ?tool ?part) (not (is_empty ?tool))))\n (:action put_down\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (hold ?tool ?part) (hold ?hand ?tool))\n :effect (and (not (hold ?hand ?part)) (is_empty ?tool)))\n (:action place\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_empty ?tool) (is_placed_to ?part1 ?part2)))\n (:action detach\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_empty ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2))\n :effect (and (hold ?tool ?part1) (not (is_empty ?tool)) (not (is_placed_to ?part1 ?part2))))\n (:action insert\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_empty ?tool) (is_inserted_to ?part1 ?part2)))\n (:action pull\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_empty ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_empty ?tool)) (not (is_inserted_to ?part1 ?part2))))\n (:action screw\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_empty ?tool) (is_screwed_to ?part1 ?part2)))\n (:action unscrew\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_empty ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_empty ?tool)) (not (is_screwed_to ?part1 ?part2))))\n (:action load_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (is_equippable ?tool) (is_empty ?hand))\n :effect (and (not (is_empty ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool)))\n (:action unload_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (hold ?hand ?tool) (is_empty ?tool))\n :effect (and (is_empty ?hand) (is_equippable ?tool) (not (hold ?hand ?tool))))\n)\n\"\"\"\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_empty( or ): or is empty and can hold something. (default: true)\n- is_equippable(): is ready to be eqquipped by a robot hand. (default: true)\n- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false)\n- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false)\n- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false)\n- can_place_to(, ): can be placed into (placement assembly constraint) (default: false)\n- hold(, ): is holding (default: false)\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be empty to hold something.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding .\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be screwed into .\n- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be empty.\n- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be empty.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: unload_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to unload the outwardgripper in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outwardgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is empty\",\n \"name\": \"target: is_empty(left_hand)\"\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding a outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"Unload the outwardgripper in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outwardgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: pick_up(left_hand, parallel_box1, shaft1)\n---------------\n\n\n Return a JSON object.\n "}, {"role": "assistant", "content": "{\n \"summary\": \"Selector to pick up shaft1 using parallel_box1 in the left hand\",\n \"name\": \"selector: pick_up(left_hand, parallel_box1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"Check the target that parallel_box1 is holding shaft1\",\n \"name\": \"target: hold(parallel_box1, shaft1)\"\n },\n {\n \"summary\": \"Sequence to pick up shaft1 using parallel_box1 in the left hand\",\n \"name\": \"sequence: pick_up(left_hand, parallel_box1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"Check the precondition that parallel_box1 is empty\",\n \"name\": \"precondition: is_empty(parallel_box1)\"\n },\n {\n \"summary\": \"Check the precondition that left hand is holding parallel_box1\",\n \"name\": \"precondition: hold(left_hand, parallel_box1)\"\n },\n {\n \"summary\": \"Check the precondition that parallel_box1 can manipulate shaft1\",\n \"name\": \"precondition: can_manipulate(parallel_box1, shaft1)\"\n },\n {\n \"summary\": \"Pick up shaft1 using parallel_box1 in the left hand\",\n \"name\": \"action: pick_up(left_hand, parallel_box1, shaft1)\"\n }\n ]\n }\n ]\n}"}]} -{"messages": [{"role": "system", "content": "You are an excellent behavior tree builder for robotic assembly tasks"}, {"role": "user", "content": "You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you construct robot task plans represented by behaiovr trees. \n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need in pddl domain format, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\"\"\"\n(define (domain robot_assembly_problem-domain)\n (:requirements :strips :typing)\n (:types\n thing - object\n tool part hand - thing\n )\n (:predicates (is_empty ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part))\n (:action pick_up\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (is_empty ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part))\n :effect (and (hold ?tool ?part) (not (is_empty ?tool))))\n (:action put_down\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (hold ?tool ?part) (hold ?hand ?tool))\n :effect (and (not (hold ?hand ?part)) (is_empty ?tool)))\n (:action place\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_empty ?tool) (is_placed_to ?part1 ?part2)))\n (:action detach\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_empty ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2))\n :effect (and (hold ?tool ?part1) (not (is_empty ?tool)) (not (is_placed_to ?part1 ?part2))))\n (:action insert\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_empty ?tool) (is_inserted_to ?part1 ?part2)))\n (:action pull\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_empty ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_empty ?tool)) (not (is_inserted_to ?part1 ?part2))))\n (:action screw\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_empty ?tool) (is_screwed_to ?part1 ?part2)))\n (:action unscrew\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_empty ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_empty ?tool)) (not (is_screwed_to ?part1 ?part2))))\n (:action load_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (is_equippable ?tool) (is_empty ?hand))\n :effect (and (not (is_empty ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool)))\n (:action unload_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (hold ?hand ?tool) (is_empty ?tool))\n :effect (and (is_empty ?hand) (is_equippable ?tool) (not (hold ?hand ?tool))))\n)\n\"\"\"\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_empty( or ): or is empty and can hold something. (default: true)\n- is_equippable(): is ready to be eqquipped by a robot hand. (default: true)\n- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false)\n- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false)\n- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false)\n- can_place_to(, ): can be placed into (placement assembly constraint) (default: false)\n- hold(, ): is holding (default: false)\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be empty to hold something.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding .\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be screwed into .\n- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be empty.\n- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be empty.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: unload_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to unload the outwardgripper in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outwardgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is empty\",\n \"name\": \"target: is_empty(left_hand)\"\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding a outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"Unload the outwardgripper in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outwardgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: pick_up(left_hand, parallel_box1, shaft1)\n---------------\n\n\n Return a JSON object.\n "}, {"role": "assistant", "content": "{\n \"summary\": \"Selector to pick up shaft1 using parallel_box1 in the left hand\",\n \"name\": \"selector: pick_up(left_hand, parallel_box1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"Check the target that parallel_box1 is holding shaft1\",\n \"name\": \"target: hold(parallel_box1, shaft1)\"\n },\n {\n \"summary\": \"Sequence to pick up shaft1 using parallel_box1 in the left hand\",\n \"name\": \"sequence: pick_up(left_hand, parallel_box1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"Check the precondition that parallel_box1 is empty\",\n \"name\": \"precondition: is_empty(parallel_box1)\"\n },\n {\n \"summary\": \"Check the precondition that left hand is holding parallel_box1\",\n \"name\": \"precondition: hold(left_hand, parallel_box1)\"\n },\n {\n \"summary\": \"Check the precondition that parallel_box1 can manipulate shaft1\",\n \"name\": \"precondition: can_manipulate(parallel_box1, shaft1)\"\n },\n {\n \"summary\": \"Pick up shaft1 using parallel_box1 in the left hand\",\n \"name\": \"action: pick_up(left_hand, parallel_box1, shaft1)\"\n }\n ]\n }\n ]\n}"}]} -{"messages": [{"role": "system", "content": "You are an excellent behavior tree builder for robotic assembly tasks"}, {"role": "user", "content": "You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you construct robot task plans represented by behaiovr trees. \n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need in pddl domain format, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\"\"\"\n(define (domain robot_assembly_problem-domain)\n (:requirements :strips :typing)\n (:types\n thing - object\n tool part hand - thing\n )\n (:predicates (is_empty ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part))\n (:action pick_up\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (is_empty ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part))\n :effect (and (hold ?tool ?part) (not (is_empty ?tool))))\n (:action put_down\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (hold ?tool ?part) (hold ?hand ?tool))\n :effect (and (not (hold ?hand ?part)) (is_empty ?tool)))\n (:action place\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_empty ?tool) (is_placed_to ?part1 ?part2)))\n (:action detach\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_empty ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2))\n :effect (and (hold ?tool ?part1) (not (is_empty ?tool)) (not (is_placed_to ?part1 ?part2))))\n (:action insert\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_empty ?tool) (is_inserted_to ?part1 ?part2)))\n (:action pull\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_empty ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_empty ?tool)) (not (is_inserted_to ?part1 ?part2))))\n (:action screw\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_empty ?tool) (is_screwed_to ?part1 ?part2)))\n (:action unscrew\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_empty ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_empty ?tool)) (not (is_screwed_to ?part1 ?part2))))\n (:action load_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (is_equippable ?tool) (is_empty ?hand))\n :effect (and (not (is_empty ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool)))\n (:action unload_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (hold ?hand ?tool) (is_empty ?tool))\n :effect (and (is_empty ?hand) (is_equippable ?tool) (not (hold ?hand ?tool))))\n)\n\"\"\"\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_empty( or ): or is empty and can hold something. (default: true)\n- is_equippable(): is ready to be eqquipped by a robot hand. (default: true)\n- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false)\n- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false)\n- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false)\n- can_place_to(, ): can be placed into (placement assembly constraint) (default: false)\n- hold(, ): is holding (default: false)\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be empty to hold something.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding .\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be screwed into .\n- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be empty.\n- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be empty.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: unload_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to unload the outwardgripper in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outwardgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is empty\",\n \"name\": \"target: is_empty(left_hand)\"\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding a outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"Unload the outwardgripper in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outwardgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: pick_up(left_hand, parallel_box1, shaft1)\n---------------\n\n\n Return a JSON object.\n "}, {"role": "assistant", "content": "{\n \"summary\": \"Selector to pick up shaft1 using parallel_box1 in the left hand\",\n \"name\": \"selector: pick_up(left_hand, parallel_box1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"Check the target that parallel_box1 is holding shaft1\",\n \"name\": \"target: hold(parallel_box1, shaft1)\"\n },\n {\n \"summary\": \"Sequence to pick up shaft1 using parallel_box1 in the left hand\",\n \"name\": \"sequence: pick_up(left_hand, parallel_box1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that parallel_box1 is empty\",\n \"name\": \"precondition: is_empty(parallel_box1)\"\n },\n {\n \"summary\": \"check the precondition that left hand is holding parallel_box1\",\n \"name\": \"precondition: hold(left_hand, parallel_box1)\"\n },\n {\n \"summary\": \"check the precondition that parallel_box1 can manipulate shaft1\",\n \"name\": \"precondition: can_manipulate(parallel_box1, shaft1)\"\n },\n {\n \"summary\": \"Pick up shaft1 using parallel_box1 in the left hand\",\n \"name\": \"action: pick_up(left_hand, parallel_box1, shaft1)\"\n }\n ]\n }\n ]\n}"}]} -{"messages": [{"role": "system", "content": "You are an excellent behavior tree builder for robotic assembly tasks"}, {"role": "user", "content": "You are an excellent behavior tree builder for robotic assembly tasks. Given an action, you construct a unit subtree for the action based on its definition in the domain knowledge.\n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\nThe object types are explained as follows:\n\"\"\"\nthing: The base class for all objects in the world. Its subclasses are: hand, tool, part.\nhand: The robot hand. A empty can only hold one at a time.\ntool: The tool to manipulate . A empty can hold one at a time.\npart: The part to be manipulated.\n\"\"\"\n\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_empty(): is empty and can hold something.\n- can_manipulate(, ): can be used to manipulate (manipulation constraint). This regulates the compatibility between and .\n- can_insert_to(, ): can be inserted into (insertion assembly constraint)\n- can_screw_to(, ): can be screwed into (screw assembly constraint)\n- can_place_to(, ): can be placed into (placement assembly constraint)\n- hold(, ): is holding . It can be holding or holding .\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be empty in order to hold the . The will be held by the as a result of this action.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding . The will be empty and will not hold the anymore as a result of this action.\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be screwed into .\n- change_tool(, , ): change the equipped tool from to in order to manipulate specific parts. As preconditions, should be holding and should be empty.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: change_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to change the tool in the left hand from outwardgripper to defaultgripper\",\n \"name\": \"selector: change_tool(left_hand, outwardgripper, defaultgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is holding the default gripper\",\n \"name\": \"target: hold(left_hand, defaultgripper)\"\n },\n {\n \"summary\": \"Sequence to change the tool in the left hand from outwardgripper to defaultgripper\", \n \"name\": \"sequence: change_tool(left_hand, outwardgripper, defaultgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding the outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"check the precondition that the outwardgripper is empty\",\n \"name\": \"precondition: is_empty(outwardgripper)\"\n },\n {\n \"summary\": \"Action to change the tool in the left hand from outwardgripper to defaultgripper\",\n \"name\": \"action: change_tool(left_hand, outwardgripper, defaultgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: pick_up(left_hand, parallel_box1, shaft1)\n---------------\n\n\n Return a JSON object.\n "}, {"role": "assistant", "content": "{\n \"summary\": \"Selector to pick up shaft1 using parallel_box1 in the left hand\",\n \"name\": \"selector: pick_up(left_hand, parallel_box1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"Check the target that parallel_box1 is holding shaft1\",\n \"name\": \"target: hold(parallel_box1, shaft1)\"\n },\n {\n \"summary\": \"Sequence to pick up shaft1 using parallel_box1 in the left hand\", \n \"name\": \"sequence: pick_up(left_hand, parallel_box1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding parallel_box1\",\n \"name\": \"precondition: hold(left_hand, parallel_box1)\"\n },\n {\n \"summary\": \"check the precondition that parallel_box1 is empty\",\n \"name\": \"precondition: is_empty(parallel_box1)\"\n },\n {\n \"summary\": \"Action to pick up shaft1 using parallel_box1 in the left hand\",\n \"name\": \"action: pick_up(left_hand, parallel_box1, shaft1)\"\n }\n ]\n }\n ]\n}"}]} -{"messages": [{"role": "system", "content": "You are an excellent behavior tree builder for robotic assembly tasks"}, {"role": "user", "content": "You are an excellent behavior tree builder for robotic assembly tasks. Given an action, you construct a unit subtree for the action based on its definition in the domain knowledge.\n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\nThe object types are explained as follows:\n\"\"\"\nthing: The base class for all objects in the world. Its subclasses are: hand, tool, part.\nhand: The robot hand. A empty can only hold one at a time.\ntool: The tool to manipulate . A empty can hold one at a time.\npart: The part to be manipulated.\n\"\"\"\n\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_empty(): is empty and can hold something.\n- can_manipulate(, ): can be used to manipulate (manipulation constraint). This regulates the compatibility between and .\n- can_insert_to(, ): can be inserted into (insertion assembly constraint)\n- can_screw_to(, ): can be screwed into (screw assembly constraint)\n- can_place_to(, ): can be placed into (placement assembly constraint)\n- hold(, ): is holding . It can be holding or holding .\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be empty in order to hold the . The will be held by the as a result of this action.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding . The will be empty and will not hold the anymore as a result of this action.\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be screwed into .\n- change_tool(, , ): change the equipped tool from to in order to manipulate specific parts. As preconditions, should be holding and should be empty.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: change_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to change the tool in the left hand from outwardgripper to defaultgripper\",\n \"name\": \"selector: change_tool(left_hand, outwardgripper, defaultgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is holding the default gripper\",\n \"name\": \"target: hold(left_hand, defaultgripper)\"\n },\n {\n \"summary\": \"Sequence to change the tool in the left hand from outwardgripper to defaultgripper\", \n \"name\": \"sequence: change_tool(left_hand, outwardgripper, defaultgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding the outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"check the precondition that the outwardgripper is empty\",\n \"name\": \"precondition: is_empty(outwardgripper)\"\n },\n {\n \"summary\": \"Action to change the tool in the left hand from outwardgripper to defaultgripper\",\n \"name\": \"action: change_tool(left_hand, outwardgripper, defaultgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: insert(left_hand, outward_claw, gear2, shaft2)\n---------------\n\n\n Return a JSON object.\n "}, {"role": "assistant", "content": "{\n \"summary\": \"Selector to insert gear2 into shaft2 using the outward claw in the left hand\",\n \"name\": \"selector: insert(left_hand, outward_claw, gear2, shaft2)\",\n \"children\": [\n {\n \"summary\": \"Check the target that gear2 is inserted into shaft2\",\n \"name\": \"target: is_inserted_to(gear2, shaft2)\"\n },\n {\n \"summary\": \"Sequence to insert gear2 into shaft2 using the outward claw in the left hand\", \n \"name\": \"sequence: insert(left_hand, outward_claw, gear2, shaft2)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding the outward claw\",\n \"name\": \"precondition: hold(left_hand, outward_claw)\"\n },\n {\n \"summary\": \"check the precondition that the outward claw is holding gear2\",\n \"name\": \"precondition: hold(outward_claw, gear2)\"\n },\n {\n \"summary\": \"Action to insert gear2 into shaft2 using the outward claw in the left hand\",\n \"name\": \"action: insert(left_hand, outward_claw, gear2, shaft2)\"\n }\n ]\n }\n ]\n}"}]} -{"messages": [{"role": "system", "content": "You are an excellent behavior tree builder for robotic assembly tasks"}, {"role": "user", "content": "You are an excellent behavior tree builder for robotic assembly tasks. Given an action, you construct a unit subtree for the action based on its definition in the domain knowledge.\n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\nThe object types are explained as follows:\n\"\"\"\nthing: The base class for all objects in the world. Its subclasses are: hand, tool, part.\nhand: The robot hand. A empty can only hold one at a time.\ntool: The tool to manipulate . A empty can hold one at a time.\npart: The part to be manipulated.\n\"\"\"\n\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_empty(): is empty and can hold something.\n- can_manipulate(, ): can be used to manipulate (manipulation constraint). This regulates the compatibility between and .\n- can_insert_to(, ): can be inserted into (insertion assembly constraint)\n- can_screw_to(, ): can be screwed into (screw assembly constraint)\n- can_place_to(, ): can be placed into (placement assembly constraint)\n- hold(, ): is holding . It can be holding or holding .\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be empty in order to hold the . The will be held by the as a result of this action.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding . The will be empty and will not hold the anymore as a result of this action.\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be screwed into .\n- change_tool(, , ): change the equipped tool from to in order to manipulate specific parts. As preconditions, should be holding and should be empty.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: change_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to change the tool in the left hand from outwardgripper to defaultgripper\",\n \"name\": \"selector: change_tool(left_hand, outwardgripper, defaultgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is holding the default gripper\",\n \"name\": \"target: hold(left_hand, defaultgripper)\"\n },\n {\n \"summary\": \"Sequence to change the tool in the left hand from outwardgripper to defaultgripper\", \n \"name\": \"sequence: change_tool(left_hand, outwardgripper, defaultgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding the outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"check the precondition that the outwardgripper is empty\",\n \"name\": \"precondition: is_empty(outwardgripper)\"\n },\n {\n \"summary\": \"Action to change the tool in the left hand from outwardgripper to defaultgripper\",\n \"name\": \"action: change_tool(left_hand, outwardgripper, defaultgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: change_tool(left_hand, parallel_box1, outward_claw)\n---------------\n\n\n Return a JSON object.\n "}, {"role": "assistant", "content": "{\n \"summary\": \"Selector to change the tool in the left hand from parallel_box1 to outward_claw\",\n \"name\": \"selector: change_tool(left_hand, parallel_box1, outward_claw)\",\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is holding the outward_claw\",\n \"name\": \"target: hold(left_hand, outward_claw)\"\n },\n {\n \"summary\": \"Sequence to change the tool in the left hand from parallel_box1 to outward_claw\", \n \"name\": \"sequence: change_tool(left_hand, parallel_box1, outward_claw)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding the parallel_box1\",\n \"name\": \"precondition: hold(left_hand, parallel_box1)\"\n },\n {\n \"summary\": \"check the precondition that the parallel_box1 is empty\",\n \"name\": \"precondition: is_empty(parallel_box1)\"\n },\n {\n \"summary\": \"Action to change the tool in the left hand from parallel_box1 to outward_claw\",\n \"name\": \"action: change_tool(left_hand, parallel_box1, outward_claw)\"\n }\n ]\n }\n ]\n}"}]} -{"messages": [{"role": "system", "content": "You are an excellent behavior tree builder for robotic assembly tasks"}, {"role": "user", "content": "You are an excellent behavior tree builder for robotic assembly tasks. Given an action, you construct a unit subtree for the action based on its definition in the domain knowledge.\n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\nThe object types are explained as follows:\n\"\"\"\nthing: The base class for all objects in the world. Its subclasses are: hand, tool, part.\nhand: The robot hand. A empty can only hold one at a time.\ntool: The tool to manipulate . A empty can hold one at a time.\npart: The part to be manipulated.\n\"\"\"\n\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_empty(): is empty and can hold something.\n- can_manipulate(, ): can be used to manipulate (manipulation constraint). This regulates the compatibility between and .\n- can_insert_to(, ): can be inserted into (insertion assembly constraint)\n- can_screw_to(, ): can be screwed into (screw assembly constraint)\n- can_place_to(, ): can be placed into (placement assembly constraint)\n- hold(, ): is holding . It can be holding or holding .\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be empty in order to hold the . The will be held by the as a result of this action.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding . The will be empty and will not hold the anymore as a result of this action.\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be screwed into .\n- change_tool(, , ): change the equipped tool from to in order to manipulate specific parts. As preconditions, should be holding and should be empty.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: change_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to change the tool in the left hand from outwardgripper to defaultgripper\",\n \"name\": \"selector: change_tool(left_hand, outwardgripper, defaultgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is holding the default gripper\",\n \"name\": \"target: hold(left_hand, defaultgripper)\"\n },\n {\n \"summary\": \"Sequence to change the tool in the left hand from outwardgripper to defaultgripper\", \n \"name\": \"sequence: change_tool(left_hand, outwardgripper, defaultgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding the outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"check the precondition that the outwardgripper is empty\",\n \"name\": \"precondition: is_empty(outwardgripper)\"\n },\n {\n \"summary\": \"Action to change the tool in the left hand from outwardgripper to defaultgripper\",\n \"name\": \"action: change_tool(left_hand, outwardgripper, defaultgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: pick_up(left_hand, outward_claw, gear2)\n---------------\n\n\n Return a JSON object.\n "}, {"role": "assistant", "content": "{\n \"summary\": \"Selector to pick up gear2 with outward_claw using left_hand\",\n \"name\": \"selector: pick_up(left_hand, outward_claw, gear2)\",\n \"children\": [\n {\n \"summary\": \"Check the target that the outward_claw is holding gear2\",\n \"name\": \"target: hold(outward_claw, gear2)\"\n },\n {\n \"summary\": \"Sequence to pick up gear2 with outward_claw using left_hand\", \n \"name\": \"sequence: pick_up(left_hand, outward_claw, gear2)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left_hand is holding the outward_claw\",\n \"name\": \"precondition: hold(left_hand, outward_claw)\"\n },\n {\n \"summary\": \"check the precondition that the outward_claw is empty\",\n \"name\": \"precondition: is_empty(outward_claw)\"\n },\n {\n \"summary\": \"Action to pick up gear2 with outward_claw using left_hand\",\n \"name\": \"action: pick_up(left_hand, outward_claw, gear2)\"\n }\n ]\n }\n ]\n}"}]} -{"messages": [{"role": "system", "content": "You are an excellent behavior tree builder for robotic assembly tasks"}, {"role": "user", "content": "You are an excellent behavior tree builder for robotic assembly tasks. Given an action, you construct a unit subtree for the action based on its definition in the domain knowledge.\n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\nThe object types are explained as follows:\n\"\"\"\nthing: The base class for all objects in the world. Its subclasses are: hand, tool, part.\nhand: The robot hand. A empty can only hold one at a time.\ntool: The tool to manipulate . A empty can hold one at a time.\npart: The part to be manipulated.\n\"\"\"\n\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_empty(): is empty and can hold something.\n- can_manipulate(, ): can be used to manipulate (manipulation constraint). This regulates the compatibility between and .\n- can_insert_to(, ): can be inserted into (insertion assembly constraint)\n- can_screw_to(, ): can be screwed into (screw assembly constraint)\n- can_place_to(, ): can be placed into (placement assembly constraint)\n- hold(, ): is holding . It can be holding or holding .\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be empty in order to hold the . The will be held by the as a result of this action.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding . The will be empty and will not hold the anymore as a result of this action.\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be screwed into .\n- change_tool(, , ): change the equipped tool from to in order to manipulate specific parts. As preconditions, should be holding and should be empty.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: change_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to change the tool in the left hand from outwardgripper to defaultgripper\",\n \"name\": \"selector: change_tool(left_hand, outwardgripper, defaultgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is holding the default gripper\",\n \"name\": \"target: hold(left_hand, defaultgripper)\"\n },\n {\n \"summary\": \"Sequence to change the tool in the left hand from outwardgripper to defaultgripper\", \n \"name\": \"sequence: change_tool(left_hand, outwardgripper, defaultgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding the outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"check the precondition that the outwardgripper is empty\",\n \"name\": \"precondition: is_empty(outwardgripper)\"\n },\n {\n \"summary\": \"Action to change the tool in the left hand from outwardgripper to defaultgripper\",\n \"name\": \"action: change_tool(left_hand, outwardgripper, defaultgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: insert(left_hand, parallel_box1, gear2, shaft2)\n---------------\n\n\n Return a JSON object.\n "}, {"role": "assistant", "content": "{\n \"summary\": \"Selector to insert gear2 into shaft2 using parallel_box1 with left hand\",\n \"name\": \"selector: insert(left_hand, parallel_box1, gear2, shaft2)\",\n \"children\": [\n {\n \"summary\": \"Check the target that gear2 is inserted into shaft2\",\n \"name\": \"target: is_inserted_to(gear2, shaft2)\"\n },\n {\n \"summary\": \"Sequence to insert gear2 into shaft2 using parallel_box1 with left hand\",\n \"name\": \"sequence: insert(left_hand, parallel_box1, gear2, shaft2)\",\n \"children\": [\n {\n \"summary\": \"Check the precondition that left hand is holding parallel_box1\",\n \"name\": \"precondition: hold(left_hand, parallel_box1)\"\n },\n {\n \"summary\": \"Check the precondition that parallel_box1 is holding gear2\",\n \"name\": \"precondition: hold(parallel_box1, gear2)\"\n },\n {\n \"summary\": \"Action to insert gear2 into shaft2 using parallel_box1 with left hand\",\n \"name\": \"action: insert(left_hand, parallel_box1, gear2, shaft2)\"\n }\n ]\n }\n ]\n}"}]} -{"messages": [{"role": "system", "content": "You are an excellent behavior tree builder for robotic assembly tasks"}, {"role": "user", "content": "You are an excellent behavior tree builder for robotic assembly tasks. Given an action, you construct a unit subtree for the action based on its definition in the domain knowledge.\n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\nThe object types are explained as follows:\n\"\"\"\nthing: The base class for all objects in the world. Its subclasses are: hand, tool, part.\nhand: The robot hand. A empty can only hold one at a time.\ntool: The tool to manipulate . A empty can hold one at a time.\npart: The part to be manipulated.\n\"\"\"\n\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_empty(): is empty and can hold something.\n- can_manipulate(, ): can be used to manipulate (manipulation constraint). This regulates the compatibility between and .\n- can_insert_to(, ): can be inserted into (insertion assembly constraint)\n- can_screw_to(, ): can be screwed into (screw assembly constraint)\n- can_place_to(, ): can be placed into (placement assembly constraint)\n- hold(, ): is holding . It can be holding or holding .\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be empty in order to hold the . The will be held by the as a result of this action.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding . The will be empty and will not hold the anymore as a result of this action.\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be screwed into .\n- change_tool(, , ): change the equipped tool from to in order to manipulate specific parts. As preconditions, should be holding and should be empty.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: change_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to change the tool in the left hand from outwardgripper to defaultgripper\",\n \"name\": \"selector: change_tool(left_hand, outwardgripper, defaultgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is holding the default gripper\",\n \"name\": \"target: hold(left_hand, defaultgripper)\"\n },\n {\n \"summary\": \"Sequence to change the tool in the left hand from outwardgripper to defaultgripper\", \n \"name\": \"sequence: change_tool(left_hand, outwardgripper, defaultgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding the outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"check the precondition that the outwardgripper is empty\",\n \"name\": \"precondition: is_empty(outwardgripper)\"\n },\n {\n \"summary\": \"Action to change the tool in the left hand from outwardgripper to defaultgripper\",\n \"name\": \"action: change_tool(left_hand, outwardgripper, defaultgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: insert(left_hand, outward_claw, gear2, shaft2)\n---------------\n\n\n Return a JSON object.\n "}, {"role": "assistant", "content": "{\n \"summary\": \"Selector to insert gear2 into shaft2 using outward_claw in the left hand\",\n \"name\": \"selector: insert(left_hand, outward_claw, gear2, shaft2)\",\n \"children\": [\n {\n \"summary\": \"Check the target that gear2 is inserted into shaft2\",\n \"name\": \"target: is_inserted_to(gear2, shaft2)\"\n },\n {\n \"summary\": \"Sequence to insert gear2 into shaft2 using outward_claw in the left hand\",\n \"name\": \"sequence: insert(left_hand, outward_claw, gear2, shaft2)\",\n \"children\": [\n {\n \"summary\": \"Check the precondition that the left hand is holding outward_claw\",\n \"name\": \"precondition: hold(left_hand, outward_claw)\"\n },\n {\n \"summary\": \"Check the precondition that outward_claw is holding gear2\",\n \"name\": \"precondition: hold(outward_claw, gear2)\"\n },\n {\n \"summary\": \"Action to insert gear2 into shaft2 using outward_claw in the left hand\",\n \"name\": \"action: insert(left_hand, outward_claw, gear2, shaft2)\"\n }\n ]\n }\n ]\n}"}]} -{"messages": [{"role": "system", "content": "You are an excellent behavior tree builder for robotic assembly tasks"}, {"role": "user", "content": "You are an excellent behavior tree builder for robotic assembly tasks. Given an action, you construct a unit subtree for the action based on its definition in the domain knowledge.\n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\nThe object types are explained as follows:\n\"\"\"\nthing: The base class for all objects in the world. Its subclasses are: hand, tool, part.\nhand: The robot hand. A empty can only hold one at a time.\ntool: The tool to manipulate . A empty can hold one at a time.\npart: The part to be manipulated.\n\"\"\"\n\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_empty(): is empty and can hold something.\n- can_manipulate(, ): can be used to manipulate (manipulation constraint). This regulates the compatibility between and .\n- can_insert_to(, ): can be inserted into (insertion assembly constraint)\n- can_screw_to(, ): can be screwed into (screw assembly constraint)\n- can_place_to(, ): can be placed into (placement assembly constraint)\n- hold(, ): is holding . It can be holding or holding .\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be empty in order to hold the . The will be held by the as a result of this action.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding . The will be empty and will not hold the anymore as a result of this action.\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be screwed into .\n- change_tool(, , ): change the equipped tool from to in order to manipulate specific parts. As preconditions, should be holding and should be empty.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: change_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to change the tool in the left hand from outwardgripper to defaultgripper\",\n \"name\": \"selector: change_tool(left_hand, outwardgripper, defaultgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is holding the default gripper\",\n \"name\": \"target: hold(left_hand, defaultgripper)\"\n },\n {\n \"summary\": \"Sequence to change the tool in the left hand from outwardgripper to defaultgripper\", \n \"name\": \"sequence: change_tool(left_hand, outwardgripper, defaultgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding the outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"check the precondition that the outwardgripper is empty\",\n \"name\": \"precondition: is_empty(outwardgripper)\"\n },\n {\n \"summary\": \"Action to change the tool in the left hand from outwardgripper to defaultgripper\",\n \"name\": \"action: change_tool(left_hand, outwardgripper, defaultgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: change_tool(left_hand, parallel_box1, outward_claw)\n---------------\n\n\n Return a JSON object.\n "}, {"role": "assistant", "content": "{\n \"summary\": \"Selector to change the tool in the left hand from outward_claw to parallel_box1\",\n \"name\": \"selector: change_tool(left_hand, outward_claw, parallel_box1)\",\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is holding the parallel_box1\",\n \"name\": \"target: hold(left_hand, parallel_box1)\"\n },\n {\n \"summary\": \"Sequence to change the tool in the left hand from outward_claw to parallel_box1\",\n \"name\": \"sequence: change_tool(left_hand, outward_claw, parallel_box1)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding the outward_claw\",\n \"name\": \"precondition: hold(left_hand, outward_claw)\"\n },\n {\n \"summary\": \"check the precondition that the outward_claw is empty\",\n \"name\": \"precondition: is_empty(outward_claw)\"\n },\n {\n \"summary\": \"Action to change the tool in the left hand from outward_claw to parallel_box1\",\n \"name\": \"action: change_tool(left_hand, outward_claw, parallel_box1)\"\n }\n ]\n }\n ]\n}"}]} -{"messages": [{"role": "system", "content": "You are an excellent behavior tree builder for robotic assembly tasks"}, {"role": "user", "content": "You are an excellent behavior tree builder for robotic assembly tasks. Given an action, you construct a unit subtree for the action based on its definition in the domain knowledge.\n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\nThe object types are explained as follows:\n\"\"\"\nthing: The base class for all objects in the world. Its subclasses are: hand, tool, part.\nhand: The robot hand. A empty can only hold one at a time.\ntool: The tool to manipulate . A empty can hold one at a time.\npart: The part to be manipulated.\n\"\"\"\n\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_empty(): is empty and can hold something.\n- can_manipulate(, ): can be used to manipulate (manipulation constraint). This regulates the compatibility between and .\n- can_insert_to(, ): can be inserted into (insertion assembly constraint)\n- can_screw_to(, ): can be screwed into (screw assembly constraint)\n- can_place_to(, ): can be placed into (placement assembly constraint)\n- hold(, ): is holding . It can be holding or holding .\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be empty in order to hold the . The will be held by the as a result of this action.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding . The will be empty and will not hold the anymore as a result of this action.\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be screwed into .\n- change_tool(, , ): change the equipped tool from to in order to manipulate specific parts. As preconditions, should be holding and should be empty.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: change_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to change the tool in the left hand from outwardgripper to defaultgripper\",\n \"name\": \"selector: change_tool(left_hand, outwardgripper, defaultgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is holding the default gripper\",\n \"name\": \"target: hold(left_hand, defaultgripper)\"\n },\n {\n \"summary\": \"Sequence to change the tool in the left hand from outwardgripper to defaultgripper\", \n \"name\": \"sequence: change_tool(left_hand, outwardgripper, defaultgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding the outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"check the precondition that the outwardgripper is empty\",\n \"name\": \"precondition: is_empty(outwardgripper)\"\n },\n {\n \"summary\": \"Action to change the tool in the left hand from outwardgripper to defaultgripper\",\n \"name\": \"action: change_tool(left_hand, outwardgripper, defaultgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: insert(left_hand, outward_claw, gear2, shaft2)\n---------------\n\n\n Return a JSON object.\n "}, {"role": "assistant", "content": "{\n \"summary\": \"Selector to insert gear2 into shaft2 using the outward claw in the left hand\",\n \"name\": \"selector: insert(left_hand, outward_claw, gear2, shaft2)\",\n \"children\": [\n {\n \"summary\": \"Check the target that gear2 is inserted into shaft2\",\n \"name\": \"target: is_inserted_to(gear2, shaft2)\"\n },\n {\n \"summary\": \"Sequence to insert gear2 into shaft2 using the outward claw in the left hand\", \n \"name\": \"sequence: insert(left_hand, outward_claw, gear2, shaft2)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding the outward claw\",\n \"name\": \"precondition: hold(left_hand, outward_claw)\"\n },\n {\n \"summary\": \"check the precondition that the outward claw is holding gear2\",\n \"name\": \"precondition: hold(outward_claw, gear2)\"\n },\n {\n \"summary\": \"Action to insert gear2 into shaft2 using the outward claw in the left hand\",\n \"name\": \"action: insert(left_hand, outward_claw, gear2, shaft2)\"\n }\n ]\n }\n ]\n}"}]} -{"messages": [{"role": "system", "content": "You are an excellent behavior tree builder for robotic assembly tasks"}, {"role": "user", "content": "You are an excellent behavior tree builder for robotic assembly tasks. Given an action, you construct a unit subtree for the action based on its definition in the domain knowledge.\n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\nThe object types are explained as follows:\n\"\"\"\nthing: The base class for all objects in the world. Its subclasses are: hand, tool, part.\nhand: The robot hand. A empty can only hold one at a time.\ntool: The tool to manipulate . A empty can hold one at a time.\npart: The part to be manipulated.\n\"\"\"\n\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_empty(): is empty and can hold something.\n- can_manipulate(, ): can be used to manipulate (manipulation constraint). This regulates the compatibility between and .\n- can_insert_to(, ): can be inserted into (insertion assembly constraint)\n- can_screw_to(, ): can be screwed into (screw assembly constraint)\n- can_place_to(, ): can be placed into (placement assembly constraint)\n- hold(, ): is holding . It can be holding or holding .\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be empty in order to hold the . The will be held by the as a result of this action.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding . The will be empty and will not hold the anymore as a result of this action.\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be screwed into .\n- change_tool(, , ): change the equipped tool from to in order to manipulate specific parts. As preconditions, should be holding and should be empty.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: change_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to change the tool in the left hand from outwardgripper to defaultgripper\",\n \"name\": \"selector: change_tool(left_hand, outwardgripper, defaultgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is holding the default gripper\",\n \"name\": \"target: hold(left_hand, defaultgripper)\"\n },\n {\n \"summary\": \"Sequence to change the tool in the left hand from outwardgripper to defaultgripper\", \n \"name\": \"sequence: change_tool(left_hand, outwardgripper, defaultgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding the outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"check the precondition that the outwardgripper is empty\",\n \"name\": \"precondition: is_empty(outwardgripper)\"\n },\n {\n \"summary\": \"Action to change the tool in the left hand from outwardgripper to defaultgripper\",\n \"name\": \"action: change_tool(left_hand, outwardgripper, defaultgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: change_tool(left_hand, parallel_box1, outward_claw)\n---------------\n\n\n Return a JSON object.\n "}, {"role": "assistant", "content": "{\n \"summary\": \"Selector to change the tool in the left hand from parallel_box1 to outward_claw\",\n \"name\": \"selector: change_tool(left_hand, parallel_box1, outward_claw)\",\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is holding the outward_claw\",\n \"name\": \"target: hold(left_hand, outward_claw)\"\n },\n {\n \"summary\": \"Sequence to change the tool in the left hand from parallel_box1 to outward_claw\", \n \"name\": \"sequence: change_tool(left_hand, parallel_box1, outward_claw)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding the parallel_box1\",\n \"name\": \"precondition: hold(left_hand, parallel_box1)\"\n },\n {\n \"summary\": \"check the precondition that the parallel_box1 is empty\",\n \"name\": \"precondition: is_empty(parallel_box1)\"\n },\n {\n \"summary\": \"Action to change the tool in the left hand from parallel_box1 to outward_claw\",\n \"name\": \"action: change_tool(left_hand, parallel_box1, outward_claw)\"\n }\n ]\n }\n ]\n}"}]} -{"messages": [{"role": "system", "content": "You are an excellent behavior tree builder for robotic assembly tasks"}, {"role": "user", "content": "You are an excellent behavior tree builder for robotic assembly tasks. Given an action, you construct a unit subtree for the action based on its definition in the domain knowledge.\n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\nThe object types are explained as follows:\n\"\"\"\nthing: The base class for all objects in the world. Its subclasses are: hand, tool, part.\nhand: The robot hand. A empty can only hold one at a time.\ntool: The tool to manipulate . A empty can hold one at a time.\npart: The part to be manipulated.\n\"\"\"\n\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_empty(): is empty and can hold something.\n- can_manipulate(, ): can be used to manipulate (manipulation constraint). This regulates the compatibility between and .\n- can_insert_to(, ): can be inserted into (insertion assembly constraint)\n- can_screw_to(, ): can be screwed into (screw assembly constraint)\n- can_place_to(, ): can be placed into (placement assembly constraint)\n- hold(, ): is holding . It can be holding or holding .\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be empty in order to hold the . The will be held by the as a result of this action.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding . The will be empty and will not hold the anymore as a result of this action.\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be screwed into .\n- change_tool(, , ): change the equipped tool from to in order to manipulate specific parts. As preconditions, should be holding and should be empty.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: change_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to change the tool in the left hand from outwardgripper to defaultgripper\",\n \"name\": \"selector: change_tool(left_hand, outwardgripper, defaultgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is holding the default gripper\",\n \"name\": \"target: hold(left_hand, defaultgripper)\"\n },\n {\n \"summary\": \"Sequence to change the tool in the left hand from outwardgripper to defaultgripper\", \n \"name\": \"sequence: change_tool(left_hand, outwardgripper, defaultgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding the outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"check the precondition that the outwardgripper is empty\",\n \"name\": \"precondition: is_empty(outwardgripper)\"\n },\n {\n \"summary\": \"Action to change the tool in the left hand from outwardgripper to defaultgripper\",\n \"name\": \"action: change_tool(left_hand, outwardgripper, defaultgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: pick_up(left_hand, outward_claw, gear2)\n---------------\n\n\n Return a JSON object.\n "}, {"role": "assistant", "content": "{\n \"summary\": \"Selector to pick up gear2 with outward_claw using left_hand\",\n \"name\": \"selector: pick_up(left_hand, outward_claw, gear2)\",\n \"children\": [\n {\n \"summary\": \"Check the target that the outward_claw is holding gear2\",\n \"name\": \"target: hold(outward_claw, gear2)\"\n },\n {\n \"summary\": \"Sequence to pick up gear2 with outward_claw using left_hand\", \n \"name\": \"sequence: pick_up(left_hand, outward_claw, gear2)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left_hand is holding the outward_claw\",\n \"name\": \"precondition: hold(left_hand, outward_claw)\"\n },\n {\n \"summary\": \"check the precondition that the outward_claw is empty\",\n \"name\": \"precondition: is_empty(outward_claw)\"\n },\n {\n \"summary\": \"Action to pick up gear2 with outward_claw using left_hand\",\n \"name\": \"action: pick_up(left_hand, outward_claw, gear2)\"\n }\n ]\n }\n ]\n}"}]} -{"messages": [{"role": "system", "content": "You are an excellent behavior tree builder for robotic assembly tasks"}, {"role": "user", "content": "You are an excellent behavior tree builder for robotic assembly tasks. Given an action, you construct a unit subtree for the action based on its definition in the domain knowledge.\n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\nThe object types are explained as follows:\n\"\"\"\nthing: The base class for all objects in the world. Its subclasses are: hand, tool, part.\nhand: The robot hand. A empty can only hold one at a time.\ntool: The tool to manipulate . A empty can hold one at a time.\npart: The part to be manipulated.\n\"\"\"\n\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_empty(): is empty and can hold something.\n- can_manipulate(, ): can be used to manipulate (manipulation constraint). This regulates the compatibility between and .\n- can_insert_to(, ): can be inserted into (insertion assembly constraint)\n- can_screw_to(, ): can be screwed into (screw assembly constraint)\n- can_place_to(, ): can be placed into (placement assembly constraint)\n- hold(, ): is holding . It can be holding or holding .\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be empty in order to hold the . The will be held by the as a result of this action.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding . The will be empty and will not hold the anymore as a result of this action.\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be screwed into .\n- change_tool(, , ): change the equipped tool from to in order to manipulate specific parts. As preconditions, should be holding and should be empty.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: change_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to change the tool in the left hand from outwardgripper to defaultgripper\",\n \"name\": \"selector: change_tool(left_hand, outwardgripper, defaultgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is holding the default gripper\",\n \"name\": \"target: hold(left_hand, defaultgripper)\"\n },\n {\n \"summary\": \"Sequence to change the tool in the left hand from outwardgripper to defaultgripper\", \n \"name\": \"sequence: change_tool(left_hand, outwardgripper, defaultgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding the outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"check the precondition that the outwardgripper is empty\",\n \"name\": \"precondition: is_empty(outwardgripper)\"\n },\n {\n \"summary\": \"Action to change the tool in the left hand from outwardgripper to defaultgripper\",\n \"name\": \"action: change_tool(left_hand, outwardgripper, defaultgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: put_down(left_hand, outward_claw, gear2)\n---------------\n\n\n Return a JSON object.\n "}, {"role": "assistant", "content": "{\n \"summary\": \"Selector to put down gear2 using the outward claw in the left hand\",\n \"name\": \"selector: put_down(left_hand, outward_claw, gear2)\",\n \"children\": [\n {\n \"summary\": \"Check the target that the outward claw is not holding gear2\",\n \"name\": \"target: not hold(outward_claw, gear2)\"\n },\n {\n \"summary\": \"Sequence to put down gear2 using the outward claw in the left hand\", \n \"name\": \"sequence: put_down(left_hand, outward_claw, gear2)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding the outward claw\",\n \"name\": \"precondition: hold(left_hand, outward_claw)\"\n },\n {\n \"summary\": \"check the precondition that the outward claw is holding gear2\",\n \"name\": \"precondition: hold(outward_claw, gear2)\"\n },\n {\n \"summary\": \"Action to put down gear2 using the outward claw in the left hand\",\n \"name\": \"action: put_down(left_hand, outward_claw, gear2)\"\n }\n ]\n }\n ]\n}"}]} -{"messages": [{"role": "system", "content": "You are an excellent behavior tree builder for robotic assembly tasks"}, {"role": "user", "content": "You are an excellent behavior tree builder for robotic assembly tasks. Given an action, you construct a unit subtree for the action based on its definition in the domain knowledge.\n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\nThe object types are explained as follows:\n\"\"\"\nthing: The base class for all objects in the world. Its subclasses are: hand, tool, part.\nhand: The robot hand. A empty can only hold one at a time.\ntool: The tool to manipulate . A empty can hold one at a time.\npart: The part to be manipulated.\n\"\"\"\n\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_empty(): is empty and can hold something.\n- can_manipulate(, ): can be used to manipulate (manipulation constraint). This regulates the compatibility between and .\n- can_insert_to(, ): can be inserted into (insertion assembly constraint)\n- can_screw_to(, ): can be screwed into (screw assembly constraint)\n- can_place_to(, ): can be placed into (placement assembly constraint)\n- hold(, ): is holding . It can be holding or holding .\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be empty in order to hold the . The will be held by the as a result of this action.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding . The will be empty and will not hold the anymore as a result of this action.\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be screwed into .\n- change_tool(, , ): change the equipped tool from to in order to manipulate specific parts. As preconditions, should be holding and should be empty.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: change_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to change the tool in the left hand from outwardgripper to defaultgripper\",\n \"name\": \"selector: change_tool(left_hand, outwardgripper, defaultgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is holding the default gripper\",\n \"name\": \"target: hold(left_hand, defaultgripper)\"\n },\n {\n \"summary\": \"Sequence to change the tool in the left hand from outwardgripper to defaultgripper\", \n \"name\": \"sequence: change_tool(left_hand, outwardgripper, defaultgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding the outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"check the precondition that the outwardgripper is empty\",\n \"name\": \"precondition: is_empty(outwardgripper)\"\n },\n {\n \"summary\": \"Action to change the tool in the left hand from outwardgripper to defaultgripper\",\n \"name\": \"action: change_tool(left_hand, outwardgripper, defaultgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: insert(left_hand, outward_claw, gear2, shaft2)\n---------------\n\n\n Return a JSON object.\n "}, {"role": "assistant", "content": "{\n \"summary\": \"Selector to insert gear2 into shaft2 using the outward claw in the left hand\",\n \"name\": \"selector: insert(left_hand, outward_claw, gear2, shaft2)\",\n \"children\": [\n {\n \"summary\": \"Check the target that gear2 is inserted into shaft2\",\n \"name\": \"target: is_inserted_to(gear2, shaft2)\"\n },\n {\n \"summary\": \"Sequence to insert gear2 into shaft2 using the outward claw in the left hand\", \n \"name\": \"sequence: insert(left_hand, outward_claw, gear2, shaft2)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding the outward claw\",\n \"name\": \"precondition: hold(left_hand, outward_claw)\"\n },\n {\n \"summary\": \"check the precondition that the outward claw is holding gear2\",\n \"name\": \"precondition: hold(outward_claw, gear2)\"\n },\n {\n \"summary\": \"Action to insert gear2 into shaft2 using the outward claw in the left hand\",\n \"name\": \"action: insert(left_hand, outward_claw, gear2, shaft2)\"\n }\n ]\n }\n ]\n}"}]} -{"messages": [{"role": "system", "content": "You are an excellent behavior tree builder for robotic assembly tasks"}, {"role": "user", "content": "You are an excellent behavior tree builder for robotic assembly tasks. Given an action, you construct a unit subtree for the action based on its definition in the domain knowledge.\n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\nThe object types are explained as follows:\n\"\"\"\nthing: The base class for all objects in the world. Its subclasses are: hand, tool, part.\nhand: The robot hand. A empty can only hold one at a time.\ntool: The tool to manipulate . A empty can hold one at a time.\npart: The part to be manipulated.\n\"\"\"\n\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_empty(): is empty and can hold something.\n- can_manipulate(, ): can be used to manipulate (manipulation constraint). This regulates the compatibility between and .\n- can_insert_to(, ): can be inserted into (insertion assembly constraint)\n- can_screw_to(, ): can be screwed into (screw assembly constraint)\n- can_place_to(, ): can be placed into (placement assembly constraint)\n- hold(, ): is holding . It can be holding or holding .\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be empty in order to hold the . The will be held by the as a result of this action.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding . The will be empty and will not hold the anymore as a result of this action.\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be screwed into .\n- change_tool(, , ): change the equipped tool from to in order to manipulate specific parts. As preconditions, should be holding and should be empty.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: change_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to change the tool in the left hand from outwardgripper to defaultgripper\",\n \"name\": \"selector: change_tool(left_hand, outwardgripper, defaultgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is holding the default gripper\",\n \"name\": \"target: hold(left_hand, defaultgripper)\"\n },\n {\n \"summary\": \"Sequence to change the tool in the left hand from outwardgripper to defaultgripper\", \n \"name\": \"sequence: change_tool(left_hand, outwardgripper, defaultgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding the outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"check the precondition that the outwardgripper is empty\",\n \"name\": \"precondition: is_empty(outwardgripper)\"\n },\n {\n \"summary\": \"Action to change the tool in the left hand from outwardgripper to defaultgripper\",\n \"name\": \"action: change_tool(left_hand, outwardgripper, defaultgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: change_tool(left_hand, parallel_box1, outward_claw)\n---------------\n\n\n Return a JSON object.\n "}, {"role": "assistant", "content": "{\n \"summary\": \"Selector to change the tool in the left hand from parallel_box1 to outward_claw\",\n \"name\": \"selector: change_tool(left_hand, parallel_box1, outward_claw)\",\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is holding the outward_claw\",\n \"name\": \"target: hold(left_hand, outward_claw)\"\n },\n {\n \"summary\": \"Sequence to change the tool in the left hand from parallel_box1 to outward_claw\", \n \"name\": \"sequence: change_tool(left_hand, parallel_box1, outward_claw)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding the parallel_box1\",\n \"name\": \"precondition: hold(left_hand, parallel_box1)\"\n },\n {\n \"summary\": \"check the precondition that the parallel_box1 is empty\",\n \"name\": \"precondition: is_empty(parallel_box1)\"\n },\n {\n \"summary\": \"Action to change the tool in the left hand from parallel_box1 to outward_claw\",\n \"name\": \"action: change_tool(left_hand, parallel_box1, outward_claw)\"\n }\n ]\n }\n ]\n}"}]} -{"messages": [{"role": "system", "content": "You are an excellent behavior tree builder for robotic assembly tasks"}, {"role": "user", "content": "You are an excellent behavior tree builder for robotic assembly tasks. Given an action, you construct a unit subtree for the action based on its definition in the domain knowledge.\n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\nThe object types are explained as follows:\n\"\"\"\nthing: The base class for all objects in the world. Its subclasses are: hand, tool, part.\nhand: The robot hand. A empty can only hold one at a time.\ntool: The tool to manipulate . A empty can hold one at a time.\npart: The part to be manipulated.\n\"\"\"\n\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_empty(): is empty and can hold something.\n- can_manipulate(, ): can be used to manipulate (manipulation constraint). This regulates the compatibility between and .\n- can_insert_to(, ): can be inserted into (insertion assembly constraint)\n- can_screw_to(, ): can be screwed into (screw assembly constraint)\n- can_place_to(, ): can be placed into (placement assembly constraint)\n- hold(, ): is holding . It can be holding or holding .\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be empty in order to hold the . The will be held by the as a result of this action.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding . The will be empty and will not hold the anymore as a result of this action.\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be screwed into .\n- change_tool(, , ): change the equipped tool from to in order to manipulate specific parts. As preconditions, should be holding and should be empty.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: change_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to change the tool in the left hand from outwardgripper to defaultgripper\",\n \"name\": \"selector: change_tool(left_hand, outwardgripper, defaultgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is holding the default gripper\",\n \"name\": \"target: hold(left_hand, defaultgripper)\"\n },\n {\n \"summary\": \"Sequence to change the tool in the left hand from outwardgripper to defaultgripper\", \n \"name\": \"sequence: change_tool(left_hand, outwardgripper, defaultgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding the outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"check the precondition that the outwardgripper is empty\",\n \"name\": \"precondition: is_empty(outwardgripper)\"\n },\n {\n \"summary\": \"Action to change the tool in the left hand from outwardgripper to defaultgripper\",\n \"name\": \"action: change_tool(left_hand, outwardgripper, defaultgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: pick_up(left_hand, outward_claw, gear2)\n---------------\n\n\n Return a JSON object.\n "}, {"role": "assistant", "content": "{\n \"summary\": \"Selector to pick up gear2 with outward_claw using left_hand\",\n \"name\": \"selector: pick_up(left_hand, outward_claw, gear2)\",\n \"children\": [\n {\n \"summary\": \"Check the target that the outward_claw is holding gear2\",\n \"name\": \"target: hold(outward_claw, gear2)\"\n },\n {\n \"summary\": \"Sequence to pick up gear2 with outward_claw using left_hand\", \n \"name\": \"sequence: pick_up(left_hand, outward_claw, gear2)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left_hand is holding the outward_claw\",\n \"name\": \"precondition: hold(left_hand, outward_claw)\"\n },\n {\n \"summary\": \"check the precondition that the outward_claw is empty\",\n \"name\": \"precondition: is_empty(outward_claw)\"\n },\n {\n \"summary\": \"Action to pick up gear2 with outward_claw using left_hand\",\n \"name\": \"action: pick_up(left_hand, outward_claw, gear2)\"\n }\n ]\n }\n ]\n}"}]} -{"messages": [{"role": "system", "content": "You are an excellent behavior tree builder for robotic assembly tasks"}, {"role": "user", "content": "You are an excellent behavior tree builder for robotic assembly tasks. Given an action, you construct a unit subtree for the action based on its definition in the domain knowledge.\n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\nThe object types are explained as follows:\n\"\"\"\nthing: The base class for all objects in the world. Its subclasses are: hand, tool, part.\nhand: The robot hand. A empty can only hold one at a time.\ntool: The tool to manipulate . A empty can hold one at a time.\npart: The part to be manipulated.\n\"\"\"\n\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_empty(): is empty and can hold something.\n- can_manipulate(, ): can be used to manipulate (manipulation constraint). This regulates the compatibility between and .\n- can_insert_to(, ): can be inserted into (insertion assembly constraint)\n- can_screw_to(, ): can be screwed into (screw assembly constraint)\n- can_place_to(, ): can be placed into (placement assembly constraint)\n- hold(, ): is holding . It can be holding or holding .\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be empty in order to hold the . The will be held by the as a result of this action.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding . The will be empty and will not hold the anymore as a result of this action.\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be screwed into .\n- change_tool(, , ): change the equipped tool from to in order to manipulate specific parts. As preconditions, should be holding and should be empty.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: change_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to change the tool in the left hand from outwardgripper to defaultgripper\",\n \"name\": \"selector: change_tool(left_hand, outwardgripper, defaultgripper)\"\n \"children\": [\n {\n \"summary\": \"The target is to make the left hand hold the default gripper\",\n \"name\": \"target: hold(left_hand, defaultgripper)\"\n },\n {\n \"summary\": \"Sequence to change the tool in the left hand from outwardgripper to defaultgripper\", \n \"name\": \"sequence: change_tool(left_hand, outwardgripper, defaultgripper)\",\n \"children\": [\n {\n \"summary\": \"A precondition is that the left hand is holding the outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"A precondition is that the outwardgripper is empty\",\n \"name\": \"precondition: is_empty(outwardgripper)\"\n },\n {\n \"summary\": \"Take the action to change the tool in the left hand from outwardgripper to defaultgripper\",\n \"name\": \"action: change_tool(left_hand, outwardgripper, defaultgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: insert(left_hand, defaultgripper, gear1, shaft1)\n---------------\n\n\n Return a JSON object.\n "}, {"role": "assistant", "content": "{\n \"summary\": \"Selector to insert gear1 into shaft1 using left hand with defaultgripper\",\n \"name\": \"selector: insert(left_hand, defaultgripper, gear1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"The target is to make gear1 inserted into shaft1\",\n \"name\": \"target: is_inserted_to(gear1, shaft1)\"\n },\n {\n \"summary\": \"Sequence to insert gear1 into shaft1 using left hand with defaultgripper\", \n \"name\": \"sequence: insert(left_hand, defaultgripper, gear1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"A precondition is that the left hand is holding the defaultgripper\",\n \"name\": \"precondition: hold(left_hand, defaultgripper)\"\n },\n {\n \"summary\": \"A precondition is that the defaultgripper is holding gear1\",\n \"name\": \"precondition: hold(defaultgripper, gear1)\"\n },\n {\n \"summary\": \"Take the action to insert gear1 into shaft1 using left hand with defaultgripper\",\n \"name\": \"action: insert(left_hand, defaultgripper, gear1, shaft1)\"\n }\n ]\n }\n ]\n}"}]} -{"messages": [{"role": "system", "content": "You are an excellent behavior tree builder for robotic assembly tasks"}, {"role": "user", "content": "You are an excellent behavior tree builder for robotic assembly tasks. Given an action, you construct a unit subtree for the action based on its definition in the domain knowledge.\n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\nThe object types are explained as follows:\n\"\"\"\nthing: The base class for all objects in the world. Its subclasses are: hand, tool, part.\nhand: The robot hand. A empty can only hold one at a time.\ntool: The tool to manipulate . A empty can hold one at a time.\npart: The part to be manipulated.\n\"\"\"\n\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_empty(): is empty and can hold something.\n- can_manipulate(, ): can be used to manipulate (manipulation constraint). This regulates the compatibility between and .\n- can_insert_to(, ): can be inserted into (insertion assembly constraint)\n- can_screw_to(, ): can be screwed into (screw assembly constraint)\n- can_place_to(, ): can be placed into (placement assembly constraint)\n- hold(, ): is holding . It can be holding or holding .\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be empty in order to hold the . The will be held by the as a result of this action.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding . The will be empty and will not hold the anymore as a result of this action.\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be screwed into .\n- change_tool(, , ): change the equipped tool from to in order to manipulate specific parts. As preconditions, should be holding and should be empty.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: change_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to change the tool in the left hand from outwardgripper to defaultgripper\",\n \"name\": \"selector: change_tool(left_hand, outwardgripper, defaultgripper)\"\n \"children\": [\n {\n \"summary\": \"The target is to make the left hand hold the default gripper\",\n \"name\": \"target: hold(left_hand, defaultgripper)\"\n },\n {\n \"summary\": \"Sequence to change the tool in the left hand from outwardgripper to defaultgripper\", \n \"name\": \"sequence: change_tool(left_hand, outwardgripper, defaultgripper)\",\n \"children\": [\n {\n \"summary\": \"A precondition is that the left hand is holding the outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"A precondition is that the outwardgripper is empty\",\n \"name\": \"precondition: is_empty(outwardgripper)\"\n },\n {\n \"summary\": \"Take the action to change the tool in the left hand from outwardgripper to defaultgripper\",\n \"name\": \"action: change_tool(left_hand, outwardgripper, defaultgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: insert(left_hand, defaultgripper, gear1, shaft1)\n---------------\n\n\n Return a JSON object.\n "}, {"role": "assistant", "content": "{\n \"summary\": \"Selector to insert gear1 into shaft1 using left hand with defaultgripper\",\n \"name\": \"selector: insert(left_hand, defaultgripper, gear1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"The target is to make gear1 inserted into shaft1\",\n \"name\": \"target: is_inserted_to(gear1, shaft1)\"\n },\n {\n \"summary\": \"Sequence to insert gear1 into shaft1 using left hand with defaultgripper\", \n \"name\": \"sequence: insert(left_hand, defaultgripper, gear1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"A precondition is that the left hand is holding the defaultgripper\",\n \"name\": \"precondition: hold(left_hand, defaultgripper)\"\n },\n {\n \"summary\": \"A precondition is that the defaultgripper is holding gear1\",\n \"name\": \"precondition: hold(defaultgripper, gear1)\"\n },\n {\n \"summary\": \"Take the action to insert gear1 into shaft1 using left hand with defaultgripper\",\n \"name\": \"action: insert(left_hand, defaultgripper, gear1, shaft1)\"\n }\n ]\n }\n ]\n}"}]} -{"messages": [{"role": "system", "content": "You are an excellent behavior tree builder for robotic assembly tasks"}, {"role": "user", "content": "You are an excellent behavior tree builder for robotic assembly tasks. Given an action, you construct a unit subtree for the action based on its definition in the domain knowledge.\n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\nThe object types are explained as follows:\n\"\"\"\nthing: The base class for all objects in the world. Its subclasses are: hand, tool, part.\nhand: The robot hand. A empty can only hold one at a time.\ntool: The tool to manipulate . A empty can hold one at a time.\npart: The part to be manipulated.\n\"\"\"\n\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_empty(): is empty and can hold something.\n- can_manipulate(, ): can be used to manipulate (manipulation constraint). This regulates the compatibility between and .\n- can_insert_to(, ): can be inserted into (insertion assembly constraint)\n- can_screw_to(, ): can be screwed into (screw assembly constraint)\n- can_place_to(, ): can be placed into (placement assembly constraint)\n- hold(, ): is holding . It can be holding or holding .\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be empty in order to hold the . The will be held by the as a result of this action.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding . The will be empty and will not hold the anymore as a result of this action.\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be screwed into .\n- change_tool(, , ): change the equipped tool from to in order to manipulate specific parts. As preconditions, should be holding and should be empty.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: change_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to change the tool in the left hand from outwardgripper to defaultgripper\",\n \"name\": \"selector: change_tool(left_hand, outwardgripper, defaultgripper)\"\n \"children\": [\n {\n \"summary\": \"The target is to make the left hand hold the default gripper\",\n \"name\": \"target: hold(left_hand, defaultgripper)\"\n },\n {\n \"summary\": \"Sequence to change the tool in the left hand from outwardgripper to defaultgripper\", \n \"name\": \"sequence: change_tool(left_hand, outwardgripper, defaultgripper)\",\n \"children\": [\n {\n \"summary\": \"A precondition is that the left hand is holding the outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"A precondition is that the outwardgripper is empty\",\n \"name\": \"precondition: is_empty(outwardgripper)\"\n },\n {\n \"summary\": \"Take the action to change the tool in the left hand from outwardgripper to defaultgripper\",\n \"name\": \"action: change_tool(left_hand, outwardgripper, defaultgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: insert(left_hand, defaultgripper, gear1, shaft1)\n---------------\n\n\n Return a JSON object.\n "}, {"role": "assistant", "content": "{\n \"summary\": \"Selector to insert gear1 into shaft1 using left hand with defaultgripper\",\n \"name\": \"selector: insert(left_hand, defaultgripper, gear1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"The target is to make gear1 inserted into shaft1\",\n \"name\": \"target: is_inserted_to(gear1, shaft1)\"\n },\n {\n \"summary\": \"Sequence to insert gear1 into shaft1 using left hand with defaultgripper\", \n \"name\": \"sequence: insert(left_hand, defaultgripper, gear1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"A precondition is that the left hand is holding the defaultgripper\",\n \"name\": \"precondition: hold(left_hand, defaultgripper)\"\n },\n {\n \"summary\": \"A precondition is that the defaultgripper is holding gear1\",\n \"name\": \"precondition: hold(defaultgripper, gear1)\"\n },\n {\n \"summary\": \"A precondition is that the defaultgripper can manipulate gear1\",\n \"name\": \"precondition: can_manipulate(defaultgripper, gear1)\"\n },\n {\n \"summary\": \"A precondition is that gear1 can be inserted into shaft1\",\n \"name\": \"precondition: can_insert_to(gear1, shaft1)\"\n },\n {\n \"summary\": \"Take the action to insert gear1 into shaft1 using left hand with defaultgripper\",\n \"name\": \"action: insert(left_hand, defaultgripper, gear1, shaft1)\"\n }\n ]\n }\n ]\n}"}]} -{"messages": [{"role": "system", "content": "You are an excellent behavior tree builder for robotic assembly tasks"}, {"role": "user", "content": "You are an excellent behavior tree builder for robotic assembly tasks. Given an action, you construct a unit subtree for the action based on its definition in the domain knowledge.\n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\nThe object types are explained as follows:\n\"\"\"\nthing: The base class for all objects in the world. Its subclasses are: hand, tool, part.\nhand: The robot hand. A empty can only hold one at a time.\ntool: The tool to manipulate . A empty can hold one at a time.\npart: The part to be manipulated.\n\"\"\"\n\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_empty(): is empty and can hold something.\n- can_manipulate(, ): can be used to manipulate (manipulation constraint). This regulates the compatibility between and .\n- can_insert_to(, ): can be inserted into (insertion assembly constraint)\n- can_screw_to(, ): can be screwed into (screw assembly constraint)\n- can_place_to(, ): can be placed into (placement assembly constraint)\n- hold(, ): is holding . It can be holding or holding .\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be empty in order to hold the . The will be held by the as a result of this action.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding . The will be empty and will not hold the anymore as a result of this action.\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be screwed into .\n- change_tool(, , ): change the equipped tool from to in order to manipulate specific parts. As preconditions, should be holding and should be empty.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: change_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to change the tool in the left hand from outwardgripper to defaultgripper\",\n \"name\": \"selector: change_tool(left_hand, outwardgripper, defaultgripper)\"\n \"children\": [\n {\n \"summary\": \"The target is to make the left hand hold the default gripper\",\n \"name\": \"target: hold(left_hand, defaultgripper)\"\n },\n {\n \"summary\": \"Sequence to change the tool in the left hand from outwardgripper to defaultgripper\", \n \"name\": \"sequence: change_tool(left_hand, outwardgripper, defaultgripper)\",\n \"children\": [\n {\n \"summary\": \"A precondition is that the left hand is holding the outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"A precondition is that the outwardgripper is empty\",\n \"name\": \"precondition: is_empty(outwardgripper)\"\n },\n {\n \"summary\": \"Take the action to change the tool in the left hand from outwardgripper to defaultgripper\",\n \"name\": \"action: change_tool(left_hand, outwardgripper, defaultgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: insert(left_hand, defaultgripper, gear1, shaft1)\n---------------\n\n\n Return a JSON object.\n "}, {"role": "assistant", "content": "{\n \"summary\": \"Selector to insert gear1 into shaft1 using left hand with defaultgripper\",\n \"name\": \"selector: insert(left_hand, defaultgripper, gear1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"The target is to make gear1 inserted into shaft1\",\n \"name\": \"target: is_inserted_to(gear1, shaft1)\"\n },\n {\n \"summary\": \"Sequence to insert gear1 into shaft1 using left hand with defaultgripper\", \n \"name\": \"sequence: insert(left_hand, defaultgripper, gear1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"A precondition is that the left hand is holding the defaultgripper\",\n \"name\": \"precondition: hold(left_hand, defaultgripper)\"\n },\n {\n \"summary\": \"A precondition is that the defaultgripper is holding gear1\",\n \"name\": \"precondition: hold(defaultgripper, gear1)\"\n },\n {\n \"summary\": \"A precondition is that gear1 can be inserted into shaft1\",\n \"name\": \"precondition: can_insert_to(gear1, shaft1)\"\n },\n {\n \"summary\": \"Take the action to insert gear1 into shaft1 using left hand with defaultgripper\",\n \"name\": \"action: insert(left_hand, defaultgripper, gear1, shaft1)\"\n }\n ]\n }\n ]\n}"}]} -{"messages": [{"role": "system", "content": "You are an excellent behavior tree builder for robotic assembly tasks"}, {"role": "user", "content": "You are an excellent behavior tree builder for robotic assembly tasks. Given an action, you construct a unit subtree for the action based on its definition in the domain knowledge.\n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\nThe object types are explained as follows:\n\"\"\"\nthing: The base class for all objects in the world. Its subclasses are: hand, tool, part.\nhand: The robot hand. A empty can only hold one at a time.\ntool: The tool to manipulate . A empty can hold one at a time.\npart: The part to be manipulated.\n\"\"\"\n\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_empty(): is empty and can hold something.\n- can_manipulate(, ): can be used to manipulate (manipulation constraint). This regulates the compatibility between and .\n- can_insert_to(, ): can be inserted into (insertion assembly constraint)\n- can_screw_to(, ): can be screwed into (screw assembly constraint)\n- can_place_to(, ): can be placed into (placement assembly constraint)\n- hold(, ): is holding . It can be holding or holding .\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be empty in order to hold the . The will be held by the as a result of this action.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding . The will be empty and will not hold the anymore as a result of this action.\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be screwed into .\n- change_tool(, , ): change the equipped tool from to in order to manipulate specific parts. As preconditions, should be holding and should be empty.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: change_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to change the tool in the left hand from outwardgripper to defaultgripper\",\n \"name\": \"selector: change_tool(left_hand, outwardgripper, defaultgripper)\"\n \"children\": [\n {\n \"summary\": \"The target is to make the left hand hold the default gripper\",\n \"name\": \"target: hold(left_hand, defaultgripper)\"\n },\n {\n \"summary\": \"Sequence to change the tool in the left hand from outwardgripper to defaultgripper\", \n \"name\": \"sequence: change_tool(left_hand, outwardgripper, defaultgripper)\",\n \"children\": [\n {\n \"summary\": \"A precondition is that the left hand is holding the outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"A precondition is that the outwardgripper is empty\",\n \"name\": \"precondition: is_empty(outwardgripper)\"\n },\n {\n \"summary\": \"Take the action to change the tool in the left hand from outwardgripper to defaultgripper\",\n \"name\": \"action: change_tool(left_hand, outwardgripper, defaultgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: insert(left_hand, defaultgripper, gear1, shaft1)\n---------------\n\n\n Return a JSON object.\n "}, {"role": "assistant", "content": "{\n \"summary\": \"Selector to insert gear1 into shaft1 using left hand with defaultgripper\",\n \"name\": \"selector: insert(left_hand, defaultgripper, gear1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"The target is to make gear1 inserted into shaft1\",\n \"name\": \"target: is_inserted_to(gear1, shaft1)\"\n },\n {\n \"summary\": \"Sequence to insert gear1 into shaft1 using left hand with defaultgripper\", \n \"name\": \"sequence: insert(left_hand, defaultgripper, gear1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"A precondition is that the left hand is holding the defaultgripper\",\n \"name\": \"precondition: hold(left_hand, defaultgripper)\"\n },\n {\n \"summary\": \"A precondition is that the defaultgripper is holding gear1\",\n \"name\": \"precondition: hold(defaultgripper, gear1)\"\n },\n {\n \"summary\": \"Take the action to insert gear1 into shaft1 using left hand with defaultgripper\",\n \"name\": \"action: insert(left_hand, defaultgripper, gear1, shaft1)\"\n }\n ]\n }\n ]\n}"}]} -{"messages": [{"role": "system", "content": "You are an excellent behavior tree builder for robotic assembly tasks"}, {"role": "user", "content": "You are an excellent behavior tree builder for robotic assembly tasks. Given an action, you construct a unit subtree for the action based on its definition in the domain knowledge.\n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\nThe object types are explained as follows:\n\"\"\"\nthing: The base class for all objects in the world. Its subclasses are: hand, tool, part.\nhand: The robot hand. A empty can only hold one at a time.\ntool: The tool to manipulate . A empty can hold one at a time.\npart: The part to be manipulated.\n\"\"\"\n\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_empty(): is empty and can hold something.\n- can_manipulate(, ): can be used to manipulate (manipulation constraint). This regulates the compatibility between and .\n- can_insert_to(, ): can be inserted into (insertion assembly constraint)\n- can_screw_to(, ): can be screwed into (screw assembly constraint)\n- can_place_to(, ): can be placed into (placement assembly constraint)\n- hold(, ): is holding . It can be holding or holding .\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be empty in order to hold the . The will be held by the as a result of this action.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding . The will be empty and will not hold the anymore as a result of this action.\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be screwed into .\n- change_tool(, , ): change the equipped tool from to in order to manipulate specific parts. As preconditions, should be holding and should be empty.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: change_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to change the tool in the left hand from outwardgripper to defaultgripper\",\n \"name\": \"selector: change_tool(left_hand, outwardgripper, defaultgripper)\"\n \"children\": [\n {\n \"summary\": \"The target is to make the left hand hold the default gripper\",\n \"name\": \"target: hold(left_hand, defaultgripper)\"\n },\n {\n \"summary\": \"Sequence to change the tool in the left hand from outwardgripper to defaultgripper\", \n \"name\": \"sequence: change_tool(left_hand, outwardgripper, defaultgripper)\",\n \"children\": [\n {\n \"summary\": \"A precondition is that the left hand is holding the outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"A precondition is that the outwardgripper is empty\",\n \"name\": \"precondition: is_empty(outwardgripper)\"\n },\n {\n \"summary\": \"Take the action to change the tool in the left hand from outwardgripper to defaultgripper\",\n \"name\": \"action: change_tool(left_hand, outwardgripper, defaultgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: insert(left_hand, defaultgripper, gear1, shaft1)\n---------------\n\n\n Return a JSON object.\n "}, {"role": "assistant", "content": "{\n \"summary\": \"Selector to insert gear1 into shaft1 using the default gripper in the left hand\",\n \"name\": \"selector: insert(left_hand, defaultgripper, gear1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"The target is to have gear1 inserted into shaft1\",\n \"name\": \"target: is_inserted_to(gear1, shaft1)\"\n },\n {\n \"summary\": \"Sequence to insert gear1 into shaft1 using the default gripper in the left hand\",\n \"name\": \"sequence: insert(left_hand, defaultgripper, gear1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"A precondition is that the left hand is holding the default gripper\",\n \"name\": \"precondition: hold(left_hand, defaultgripper)\"\n },\n {\n \"summary\": \"A precondition is that the default gripper is holding gear1\",\n \"name\": \"precondition: hold(defaultgripper, gear1)\"\n },\n {\n \"summary\": \"Take the action to insert gear1 into shaft1 using the default gripper in the left hand\",\n \"name\": \"action: insert(left_hand, defaultgripper, gear1, shaft1)\"\n }\n ]\n }\n ]\n}"}]} -{"messages": [{"role": "system", "content": "You are an excellent behavior tree builder for robotic assembly tasks"}, {"role": "user", "content": "You are an excellent behavior tree builder for robotic assembly tasks. Given an action, you construct a unit subtree for the action based on its definition in the domain knowledge.\n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\nThe object types are explained as follows:\n\"\"\"\nthing: The base class for all objects in the world. Its subclasses are: hand, tool, part.\nhand: The robot hand. A empty can only hold one at a time.\ntool: The tool to manipulate . A empty can hold one at a time.\npart: The part to be manipulated.\n\"\"\"\n\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_empty(): is empty and can hold something.\n- can_manipulate(, ): can be used to manipulate (manipulation constraint). This regulates the compatibility between and .\n- can_insert_to(, ): can be inserted into (insertion assembly constraint)\n- can_screw_to(, ): can be screwed into (screw assembly constraint)\n- can_place_to(, ): can be placed into (placement assembly constraint)\n- hold(, ): is holding . It can be holding or holding .\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be empty in order to hold the . The will be held by the as a result of this action.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding . The will be empty and will not hold the anymore as a result of this action.\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be screwed into .\n- change_tool(, , ): change the equipped tool from to in order to manipulate specific parts. As preconditions, should be holding and should be empty.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: change_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to change the tool in the left hand from outwardgripper to defaultgripper\",\n \"name\": \"selector: change_tool(left_hand, outwardgripper, defaultgripper)\"\n \"children\": [\n {\n \"summary\": \"The target is to make the left hand hold the default gripper\",\n \"name\": \"target: hold(left_hand, defaultgripper)\"\n },\n {\n \"summary\": \"Sequence to change the tool in the left hand from outwardgripper to defaultgripper\", \n \"name\": \"sequence: change_tool(left_hand, outwardgripper, defaultgripper)\",\n \"children\": [\n {\n \"summary\": \"A precondition is that the left hand is holding the outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"A precondition is that the outwardgripper is empty\",\n \"name\": \"precondition: is_empty(outwardgripper)\"\n },\n {\n \"summary\": \"Take the action to change the tool in the left hand from outwardgripper to defaultgripper\",\n \"name\": \"action: change_tool(left_hand, outwardgripper, defaultgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: insert(left_hand, defaultgripper, gear1, shaft1)\n---------------\n\n\n Return a JSON object.\n "}, {"role": "assistant", "content": "{\n \"summary\": \"Selector to insert gear1 into shaft1 using the default gripper in the left hand\",\n \"name\": \"selector: insert(left_hand, defaultgripper, gear1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"The target is to have gear1 inserted into shaft1\",\n \"name\": \"target: is_inserted_to(gear1, shaft1)\"\n },\n {\n \"summary\": \"Sequence to insert gear1 into shaft1 using the default gripper in the left hand\",\n \"name\": \"sequence: insert(left_hand, defaultgripper, gear1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"A precondition is that the left hand is holding the default gripper\",\n \"name\": \"precondition: hold(left_hand, defaultgripper)\"\n },\n {\n \"summary\": \"A precondition is that the default gripper is holding gear1\",\n \"name\": \"precondition: hold(defaultgripper, gear1)\"\n },\n {\n \"summary\": \"Take the action to insert gear1 into shaft1 using the default gripper in the left hand\",\n \"name\": \"action: insert(left_hand, defaultgripper, gear1, shaft1)\"\n }\n ]\n }\n ]\n}"}]} -{"messages": [{"role": "system", "content": "You are an excellent behavior tree builder for robotic assembly tasks"}, {"role": "user", "content": "You are an excellent behavior tree builder for robotic assembly tasks. Given an action, you construct a unit subtree for the action based on its definition in the domain knowledge.\n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\nThe object types are explained as follows:\n\"\"\"\nthing: The base class for all objects in the world. Its subclasses are: hand, tool, part.\nhand: The robot hand. A empty can only hold one at a time.\ntool: The tool to manipulate . A empty can hold one at a time.\npart: The part to be manipulated.\n\"\"\"\n\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_empty(): is empty and can hold something.\n- can_manipulate(, ): can be used to manipulate (manipulation constraint). This regulates the compatibility between and .\n- can_insert_to(, ): can be inserted into (insertion assembly constraint)\n- can_screw_to(, ): can be screwed into (screw assembly constraint)\n- can_place_to(, ): can be placed into (placement assembly constraint)\n- hold(, ): is holding . It can be holding or holding .\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be empty in order to hold the . The will be held by the as a result of this action.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding . The will be empty and will not hold the anymore as a result of this action.\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be screwed into .\n- change_tool(, , ): change the equipped tool from to in order to manipulate specific parts. As preconditions, should be holding and should be empty.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: change_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to change the tool in the left hand from outwardgripper to defaultgripper\",\n \"name\": \"selector: change_tool(left_hand, outwardgripper, defaultgripper)\"\n \"children\": [\n {\n \"summary\": \"The target is to make the left hand hold the default gripper\",\n \"name\": \"target: hold(left_hand, defaultgripper)\"\n },\n {\n \"summary\": \"Sequence to change the tool in the left hand from outwardgripper to defaultgripper\", \n \"name\": \"sequence: change_tool(left_hand, outwardgripper, defaultgripper)\",\n \"children\": [\n {\n \"summary\": \"A precondition is that the left hand is holding the outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"A precondition is that the outwardgripper is empty\",\n \"name\": \"precondition: is_empty(outwardgripper)\"\n },\n {\n \"summary\": \"Take the action to change the tool in the left hand from outwardgripper to defaultgripper\",\n \"name\": \"action: change_tool(left_hand, outwardgripper, defaultgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: change_tool(left_hand, parallel_box1, defaultgripper)\n---------------\n\n\n Return a JSON object.\n "}, {"role": "assistant", "content": "{\n \"summary\": \"Selector to change the tool in the left hand from parallel_box1 to defaultgripper\",\n \"name\": \"selector: change_tool(left_hand, parallel_box1, defaultgripper)\",\n \"children\": [\n {\n \"summary\": \"The target is to make the left hand hold the default gripper\",\n \"name\": \"target: hold(left_hand, defaultgripper)\"\n },\n {\n \"summary\": \"Sequence to change the tool in the left hand from parallel_box1 to defaultgripper\",\n \"name\": \"sequence: change_tool(left_hand, parallel_box1, defaultgripper)\",\n \"children\": [\n {\n \"summary\": \"A precondition is that the left hand is holding the parallel_box1\",\n \"name\": \"precondition: hold(left_hand, parallel_box1)\"\n },\n {\n \"summary\": \"A precondition is that the parallel_box1 is empty\",\n \"name\": \"precondition: is_empty(parallel_box1)\"\n },\n {\n \"summary\": \"Take the action to change the tool in the left hand from parallel_box1 to defaultgripper\",\n \"name\": \"action: change_tool(left_hand, parallel_box1, defaultgripper)\"\n }\n ]\n }\n ]\n}"}]} diff --git a/data/finetuning/ft_job.py b/data/finetuning/ft_job.py deleted file mode 100644 index 3f4bcd3e..00000000 --- a/data/finetuning/ft_job.py +++ /dev/null @@ -1,10 +0,0 @@ -from openai import OpenAI - -client = OpenAI() - -client.fine_tuning.jobs.create( - training_file="file-D8aBgTaXbp3g42RcV2bTiKDK", - model="ft:gpt-3.5-turbo-0125:kifabrik-mirmi:kios-ut-gen-v2:8z2KbPsr", - suffix="kios_ut_gen_v3", - hyperparameters={"batch_size": 3, "learning_rate_multiplier": 0.5, "n_epochs": 1}, -) diff --git a/data/finetuning/ft_monitor.py b/data/finetuning/ft_monitor.py deleted file mode 100644 index 9da77ca4..00000000 --- a/data/finetuning/ft_monitor.py +++ /dev/null @@ -1,6 +0,0 @@ -from openai import OpenAI - -client = OpenAI() - -# List 10 fine-tuning jobs -print(client.fine_tuning.jobs.list(limit=10)) diff --git a/data/finetuning/langsmith_database.py b/data/finetuning/langsmith_database.py deleted file mode 100644 index 2aba2398..00000000 --- a/data/finetuning/langsmith_database.py +++ /dev/null @@ -1,71 +0,0 @@ -from langsmith import Client -from pprint import pprint -import json - -client = Client() - -# import datetime - -# project_name = "kios_agent" -# run_type = "llm" -# end_time = datetime.datetime.now() - -# runs = client.list_runs( -# project_name=project_name, -# run_type=run_type, -# error=False, -# ) - -from langsmith import schemas -from langchain import load - - -def convert_messages(example: schemas.Example) -> dict: - pprint(example.inputs) - pprint(example.outputs) - user_input = load.load(example.inputs)["input"][0]["data"]["content"] - ai_output = load.load(example.outputs)["output"]["data"]["content"] - return { - "user": user_input, - "ai": ai_output, - } - - -def convert_openai_ft_msg(msg: dict) -> dict[str, list[dict[str, str]]]: - return { - "messages": [ - { - "role": "system", - "content": "You are an excellent behavior tree builder for robotic assembly tasks", - }, - { - "role": "user", - "content": msg["user"], - }, - { - "role": "assistant", - "content": msg["ai"], - }, - ] - } - - -messages = [ - convert_messages(example) - for example in client.list_examples(dataset_name="UT-gen-data") -] - -# from langchain.adapters import openai as openai_adapter - -finetuning_messages = [convert_openai_ft_msg(msg) for msg in messages] - -# pprint(finetuning_messages[0]) - -# ! just try -# finetuning_messages = finetuning_messages[:10] - -# Write finetuning_messages to a jsonl file -with open("/home/blackbird/kios/data/finetuning/finetuning_messages_20240303.jsonl", "w") as f: - for msg in finetuning_messages: - json.dump(msg, f) - f.write("\n") diff --git a/data/finetuning/langsmith_ft_tt.py b/data/finetuning/langsmith_ft_tt.py deleted file mode 100644 index 3ab8467b..00000000 --- a/data/finetuning/langsmith_ft_tt.py +++ /dev/null @@ -1,31 +0,0 @@ -from openai import OpenAI -import os - -openai_client = OpenAI() - -file_name = "finetuning_messages_20240303.jsonl" - -training_file = openai_client.files.create( - file=open( - os.path.join(os.path.dirname(__file__), file_name), - "rb", - ), - purpose="fine-tune", -) - -# # Wait while the file is processed -# status = openai_client.files.retrieve("file-EhbEPGHSCMj1bD6TGi6WQMxT").status - - -# start_time = time.time() -# while status != "processed": -# print(f"Status=[{status}]... {time.time() - start_time:.2f}s", end="\r", flush=True) -# time.sleep(5) -# status = openai.File.retrieve(training_file.id).status -# print(f"File {training_file.id} ready after {time.time() - start_time:.2f} seconds.") - -# from openai import OpenAI - -# client = OpenAI() - -# client.fine_tuning.jobs.create(training_file="UT_gen_test", model="gpt-3.5-turbo") diff --git a/data/finetuning/material_check.py b/data/finetuning/material_check.py deleted file mode 100644 index 082e4e8b..00000000 --- a/data/finetuning/material_check.py +++ /dev/null @@ -1,149 +0,0 @@ -import json -import tiktoken # for token counting -import numpy as np -from collections import defaultdict -import os - -file_name = "finetuning_messages_20240303.jsonl" - -data_path = os.path.join(os.path.dirname(__file__), file_name) - -# Load the dataset -with open(data_path, "r", encoding="utf-8") as f: - dataset = [json.loads(line) for line in f] - -# # Initial dataset stats -print("Num examples:", len(dataset)) -# print("First example:") -# for message in dataset[0]["messages"]: -# print(message) - -# * Format error checks -format_errors = defaultdict(int) - - -for ex in dataset: - if not isinstance(ex, dict): - format_errors["data_type"] += 1 - continue - - messages = ex.get("messages", None) - if not messages: - format_errors["missing_messages_list"] += 1 - continue - - for message in messages: - if "role" not in message or "content" not in message: - format_errors["message_missing_key"] += 1 - - if any( - k not in ("role", "content", "name", "function_call", "weight") - for k in message - ): - format_errors["message_unrecognized_key"] += 1 - - if message.get("role", None) not in ("system", "user", "assistant", "function"): - format_errors["unrecognized_role"] += 1 - - content = message.get("content", None) - function_call = message.get("function_call", None) - - if (not content and not function_call) or not isinstance(content, str): - format_errors["missing_content"] += 1 - - if not any(message.get("role", None) == "assistant" for message in messages): - format_errors["example_missing_assistant_message"] += 1 - -if format_errors: - print("Found errors:") - for k, v in format_errors.items(): - print(f"{k}: {v}") -else: - print("No errors found") - - -encoding = tiktoken.get_encoding("cl100k_base") - - -# not exact! -# simplified from https://github.com/openai/openai-cookbook/blob/main/examples/How_to_count_tokens_with_tiktoken.ipynb -def num_tokens_from_messages(messages, tokens_per_message=3, tokens_per_name=1): - num_tokens = 0 - for message in messages: - num_tokens += tokens_per_message - for key, value in message.items(): - num_tokens += len(encoding.encode(value)) - if key == "name": - num_tokens += tokens_per_name - num_tokens += 3 - return num_tokens - - -def num_assistant_tokens_from_messages(messages): - num_tokens = 0 - for message in messages: - if message["role"] == "assistant": - num_tokens += len(encoding.encode(message["content"])) - return num_tokens - - -def print_distribution(values, name): - print(f"\n#### Distribution of {name}:") - print(f"min / max: {min(values)}, {max(values)}") - print(f"mean / median: {np.mean(values)}, {np.median(values)}") - print(f"p5 / p95: {np.quantile(values, 0.1)}, {np.quantile(values, 0.9)}") - - -# * Warnings and tokens counts -n_missing_system = 0 -n_missing_user = 0 -n_messages = [] -convo_lens = [] -assistant_message_lens = [] - -for ex in dataset: - messages = ex["messages"] - if not any(message["role"] == "system" for message in messages): - n_missing_system += 1 - if not any(message["role"] == "user" for message in messages): - n_missing_user += 1 - n_messages.append(len(messages)) - convo_lens.append(num_tokens_from_messages(messages)) - assistant_message_lens.append(num_assistant_tokens_from_messages(messages)) - -print("Num examples missing system message:", n_missing_system) -print("Num examples missing user message:", n_missing_user) -print_distribution(n_messages, "num_messages_per_example") -print_distribution(convo_lens, "num_total_tokens_per_example") -print_distribution(assistant_message_lens, "num_assistant_tokens_per_example") -n_too_long = sum(l > 4096 for l in convo_lens) -print( - f"\n{n_too_long} examples may be over the 4096 token limit, they will be truncated during fine-tuning" -) - -# * Pricing and default n_epochs estimate -MAX_TOKENS_PER_EXAMPLE = 4096 - -TARGET_EPOCHS = 10 -MIN_TARGET_EXAMPLES = 100 -MAX_TARGET_EXAMPLES = 25000 -MIN_DEFAULT_EPOCHS = 1 -MAX_DEFAULT_EPOCHS = 25 - -n_epochs = TARGET_EPOCHS -n_train_examples = len(dataset) -if n_train_examples * TARGET_EPOCHS < MIN_TARGET_EXAMPLES: - n_epochs = min(MAX_DEFAULT_EPOCHS, MIN_TARGET_EXAMPLES // n_train_examples) -elif n_train_examples * TARGET_EPOCHS > MAX_TARGET_EXAMPLES: - n_epochs = max(MIN_DEFAULT_EPOCHS, MAX_TARGET_EXAMPLES // n_train_examples) - -n_billing_tokens_in_dataset = sum( - min(MAX_TOKENS_PER_EXAMPLE, length) for length in convo_lens -) -print( - f"Dataset has ~{n_billing_tokens_in_dataset} tokens that will be charged for during training" -) -print(f"By default, you'll train for {n_epochs} epochs on this dataset") -print( - f"By default, you'll be charged for ~{n_epochs * n_billing_tokens_in_dataset} tokens" -) diff --git a/data/finetuning/openai_file_pre.py b/data/finetuning/openai_file_pre.py deleted file mode 100644 index 467c30a4..00000000 --- a/data/finetuning/openai_file_pre.py +++ /dev/null @@ -1,7 +0,0 @@ -from openai import OpenAI - -openai_client = OpenAI() - -status = openai_client.files.retrieve("file-EhbEPGHSCMj1bD6TGi6WQMxT").status - -print(status) diff --git a/data/finetuning/test.jsonl b/data/finetuning/test.jsonl deleted file mode 100644 index e69de29b..00000000 diff --git a/data/finetuning/visualize.py b/data/finetuning/visualize.py deleted file mode 100644 index 641eaffa..00000000 --- a/data/finetuning/visualize.py +++ /dev/null @@ -1,5 +0,0 @@ -from wandb.integration.openai.fine_tuning import WandbLogger - -# Finetuning logic - -WandbLogger.sync(fine_tune_job_id="ftjob-L36vT4dDnkWzL9GckJ0SssVm") diff --git a/data/prompt_skeletons/cot_skeleton.json b/data/prompt_skeletons/cot_skeleton.json deleted file mode 100644 index b49f2bad..00000000 --- a/data/prompt_skeletons/cot_skeleton.json +++ /dev/null @@ -1,16 +0,0 @@ -{ - "skeleton_name": "cot_skeleton", - "model_name": "gpt-4-1106-preview", - "version": "V0", - "prompt_dir": "prompts/cot_skeleton_chain", - "prompt_load_order": [ - "cot_sk_role", - "cot_sk_output_format", - "cot_sk_domain", - "cot_sk_problem", - "cot_sk_state", - "cot_sk_bt", - "cot_sk_chain", - "cot_sk_example" - ] -} \ No newline at end of file diff --git a/data/prompt_skeletons/refine_skeleton.json b/data/prompt_skeletons/refine_skeleton.json deleted file mode 100644 index 07c9e259..00000000 --- a/data/prompt_skeletons/refine_skeleton.json +++ /dev/null @@ -1,16 +0,0 @@ -{ - "skeleton_name": "skeleton_refiner", - "model_name": "gpt-4-1106-preview", - "version": "V0", - "prompt_dir": "prompts/skeleton_refiner", - "prompt_load_order": [ - "refine_role", - "refine_input_format", - "refine_state", - "refine_help", - "refine_controlflow", - "refine_condition", - "refine_action", - "refine_output_format" - ] -} diff --git a/data/prompt_skeletons/skeleton_generator copy.json b/data/prompt_skeletons/skeleton_generator copy.json deleted file mode 100644 index df83c891..00000000 --- a/data/prompt_skeletons/skeleton_generator copy.json +++ /dev/null @@ -1,17 +0,0 @@ -{ - "skeleton_name": "skeleton_generator", - "model_name": "gpt-4-1106-preview", - "version": "V0", - "prompt_dir": "prompts/skeleton_generator", - "prompt_load_order": [ - "cot_sk_role", - "cot_sk_output_format", - "cot_sk_domain", - "cot_sk_problem", - "cot_sk_instruction", - "cot_sk_state", - "cot_sk_bt", - "cot_sk_chain", - "cot_sk_example" - ] -} \ No newline at end of file diff --git a/data/prompt_skeletons/skeleton_generator.json b/data/prompt_skeletons/skeleton_generator.json deleted file mode 100644 index cfe39b87..00000000 --- a/data/prompt_skeletons/skeleton_generator.json +++ /dev/null @@ -1,17 +0,0 @@ -{ - "skeleton_name": "skeleton_generator", - "model_name": "gpt-4-1106-preview", - "version": "V0", - "prompt_dir": "prompts/skeleton_generator", - "prompt_load_order": [ - "cot_sk_role", - "cot_sk_output_format", - "cot_sk_domain", - "cot_sk_objects", - "cot_sk_instruction", - "cot_sk_state", - "cot_sk_bt", - "cot_sk_chain", - "cot_sk_example" - ] -} \ No newline at end of file diff --git a/data/prompt_skeletons/skeleton_refiner.json b/data/prompt_skeletons/skeleton_refiner.json deleted file mode 100644 index 07c9e259..00000000 --- a/data/prompt_skeletons/skeleton_refiner.json +++ /dev/null @@ -1,16 +0,0 @@ -{ - "skeleton_name": "skeleton_refiner", - "model_name": "gpt-4-1106-preview", - "version": "V0", - "prompt_dir": "prompts/skeleton_refiner", - "prompt_load_order": [ - "refine_role", - "refine_input_format", - "refine_state", - "refine_help", - "refine_controlflow", - "refine_condition", - "refine_action", - "refine_output_format" - ] -} diff --git a/data/prompts/cot_skeleton/backup_do_not_use b/data/prompts/cot_skeleton/backup_do_not_use deleted file mode 100644 index e69de29b..00000000 diff --git a/data/prompts/cot_skeleton/prompt/cot_sk_bt.txt b/data/prompts/cot_skeleton/prompt/cot_sk_bt.txt deleted file mode 100644 index 54cefdd0..00000000 --- a/data/prompts/cot_skeleton/prompt/cot_sk_bt.txt +++ /dev/null @@ -1,83 +0,0 @@ -[user] -The behavior tree should be in json dictionary format and have a structure like the following example: -""" -{ - "summary": "selector to use left hand to load inward_claw", - "name": "selector: load_tool(left_hand, inward_claw)", - "children": [ - { - "summary": "condition node to check if left_hand holds inward_claw", - "name": "target: hold(left_hand, inward_claw)", - }, - { - "summary": "sequence to use left hand to load inward_claw", - "name": "sequence: load_tool(left_hand, inward_claw)", - "children": [ - { - "summary": "condition node to check if inward_claw is equippable", - "name": "precondition: is_equippable(inward_claw)", - }, - { - "summary": "condition node to check if left hand is free", - "name": "precondition: is_free(left_hand)", - }, - { - "summary": "action node to use left hand to load inward_claw", - "name": "action: load_tool(left_hand, inward_claw)", - "effects": [ - {"summary": "left hand will be not free anymore",}, - {"summary": "inward_claw will be not equippable anymore",}, - {"summary": "left_hand will hold inward_claw",}, - ], - }, - ], - }, - ], -} -""" -Follow these rules: -1. Every node are described by a summary and a name: - - The "summary" key includes a short interpretive description of the node. - - The "name" key includes the type of the node and the formal form of the node according to the domain knowledge. -2. The control flow nodes, "selector" and "sequence", can have "children", which is a list of children nodes. -3. The "condition" nodes that are children of "selectors" are targets. Those that are children of "sequences" are preconditions. -4. The "action" nodes should have a key "effects" with a list of dictionaries, which describes the to-be-exerted effects of the action. The effects are shortly described under the key "summary". -5. All the subtrees always have their corresponding action in ROBOT_ACTION_LIST. The recommended structure is: - - A selector as the root node. - - (Usually one) target condition node as the first child of the selector to check if the target of the action has been fulfilled. - - A sequence node as the last child of the selector node to achieve the target. - - (Usually a number of) precondition nodes as the front children of the sequence node to check the preconditions for executing the action. - - One action node as the last child of the sequence node, whose preconditions are checked before and will immediate start the job when being ticked. - - The action will exert its defined effects after finishing the job successfully. Those effects should fulfilled all the target conditions in the selector node. - A brief example is as follows: -{ - selector: load_tool selector, - children: [ - { - target: check if inward_claw is equipped by left hand, - }, - { - sequence: load_tool, - children: [ - {precondition: check if inward_claw is equippable}, - {precondition: check if left hand is free}, - { - action: equip inward_claw to left hand, - effects: [ - left hand not free, - inward_claw not equippable, - left_hand holds inward_claw, - ] - }, - ], - }, - ], -} - -6. When performing a new action to fulfill a unsatisfied precondition, you can construct a subtree to replace the precondition and take that unfulfilled precondition as the target of the subtree. Then you construct the sequence of the subtree according to the new action you want to perform. -7. All of the control flow nodes, namely selectors and sequences, are without memory. A new behavior tree tick will always start from the root and traverse all the nodes even if they are ticked in the last tick. - -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/data/prompts/cot_skeleton/prompt/cot_sk_chain copy.txt b/data/prompts/cot_skeleton/prompt/cot_sk_chain copy.txt deleted file mode 100644 index 8ab56a97..00000000 --- a/data/prompts/cot_skeleton/prompt/cot_sk_chain copy.txt +++ /dev/null @@ -1,15 +0,0 @@ -[user] -Follow this workflow to solve the task: - -1. Read the problem description. Parse and generate the "initial_state" in the output dictionary. -2. Interpret the goal of the problem as goal_state and write into the "instruction_summary" in the output dictionary. -3. Try to solve the problem, achieve the goal_state from the initial_state based on the domain knowledge provided. Generate a action sequence with the actions defined in and write it into dictionary["task_plan"]["action_sequence"] of the output. -4. If you cannot find a plan to solve the task, ask question with nature language in the "question" of the output dictionary. -5. Explain the actions in the action sequence and write the output dictionary["task_plan"]["action_instructions"]. -6. Collect the objects in the world that are effected by the action sequence, write into dictionary["task_plan"]["manipulated_objects"]. -7. Generate the behavior tree based on the action sequence and write into dictionary["task_plan"]["behavior_tree"]. - -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/data/prompts/cot_skeleton/prompt/cot_sk_chain.txt b/data/prompts/cot_skeleton/prompt/cot_sk_chain.txt deleted file mode 100644 index 3b0777a5..00000000 --- a/data/prompts/cot_skeleton/prompt/cot_sk_chain.txt +++ /dev/null @@ -1,16 +0,0 @@ -[user] -Follow this workflow to solve the task: - -1. Read the problem description. Parse it and fill the "constraints" in "initial_state". -2. Fill the "objects" and "relations" in "initial_state" based on the input world state. -3. Interpret the input instructions as goal_state and write into the "instruction_summary" in the output dictionary. -3. Try to solve the problem, achieve the goal_state from the initial_state based on the domain knowledge provided. Generate a action sequence with the actions defined in and write it into dictionary["task_plan"]["action_sequence"] of the output. -4. If you cannot find a plan to solve the task, ask question with nature language in the "question" of the output dictionary. -5. Explain the actions in the action sequence and write the output dictionary["task_plan"]["action_instructions"]. -6. Collect the objects in the world that are effected by the action sequence, write into dictionary["task_plan"]["manipulated_objects"]. -7. Generate the behavior tree based on the action sequence and write into dictionary["task_plan"]["behavior_tree"]. - -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/data/prompts/cot_skeleton/prompt/cot_sk_domain.txt b/data/prompts/cot_skeleton/prompt/cot_sk_domain.txt deleted file mode 100644 index 7f48353e..00000000 --- a/data/prompts/cot_skeleton/prompt/cot_sk_domain.txt +++ /dev/null @@ -1,89 +0,0 @@ -[user] -Following is the domain knowledge you need in pddl domain format, which includes: -- the object types in the world, -- the predicates for describing the states of the world, -- the actions that the robot can apply, with their precondtions and effects defined. -""" -(define (domain robot_assembly_problem-domain) - (:requirements :strips :typing) - (:types - thing - object - tool part hand - thing - ) - (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part)) - (:action pick_up - :parameters ( ?hand - hand ?tool - tool ?part - part) - :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part)) - :effect (and (hold ?tool ?part) (not (is_free ?tool)))) - (:action put_down - :parameters ( ?hand - hand ?tool - tool ?part - part) - :precondition (and (hold ?tool ?part) (hold ?hand ?tool)) - :effect (and (not (hold ?hand ?part)) (is_free ?tool))) - (:action place - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2))) - (:action detach - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2)))) - (:action insert - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2))) - (:action pull - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2)))) - (:action screw - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2))) - (:action unscrew - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2)))) - (:action load_tool - :parameters ( ?hand - hand ?tool - tool) - :precondition (and (is_equippable ?tool) (is_free ?hand)) - :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool))) - (:action unload_tool - :parameters ( ?hand - hand ?tool - tool) - :precondition (and (hold ?hand ?tool) (is_free ?tool)) - :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool)))) -) -""" -The precidates are explained as follows: -""" -"PREDICATE_LIST" -- is_free( or ): or is free and can hold something. (default: true) -- is_equippable(): is ready to be eqquipped by a robot hand. (default: true) -- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false) -- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false) -- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false) -- can_place_to(, ): can be placed into (placement assembly constraint) (default: false) -- hold(, ): is holding (default: false) -- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false) -- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false) -- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false) -""" - -The actions are explained as follows: -""" -"ROBOT_ACTION_LIST" -- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something. -- put_down(, , ): use to put down . As preconditions, should be holding and should be holding . -- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding . -- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into . -- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding . -- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into . -- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding . -- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into . -- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free. -- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free. -""" -the items in <> indicate the type of the instances needed for the predicates and actions. -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. Waiting for next input. diff --git a/data/prompts/cot_skeleton/prompt/cot_sk_example copy.txt b/data/prompts/cot_skeleton/prompt/cot_sk_example copy.txt deleted file mode 100644 index e1fb3256..00000000 --- a/data/prompts/cot_skeleton/prompt/cot_sk_example copy.txt +++ /dev/null @@ -1,144 +0,0 @@ -[user] -I will give you some examples of the input and the output you will generate. -Example 1: -""" -- Input: -(define (problem robot_assembly_problem-problem) - (:domain robot_assembly_problem-domain) - (:objects - parallel_box1 parallel_box2 inward_claw outward_claw no_tool - tool - gear1 gear2 gear3 shaft1 shaft2 base - part - left_hand - hand - ) - (:init (can_manipulate parallel_box1 gear1) (can_manipulate outward_claw gear2) (can_manipulate inward_claw gear3) (can_manipulate parallel_box2 shaft1) (can_manipulate no_tool shaft2) (can_screw_to leg1 seat) (can_screw_to leg2 seat) (can_insert_to back seat) (can_screw_to nut1 seat) (can_screw_to nut2 seat) (can_screw_to blub base) (can_place_to lamp blub) (can_insert_to shaft1 base) (can_insert_to shaft2 base) (can_insert_to gear3 shaft2) (can_insert_to gear2 base) (can_insert_to gear1 shaft1) (is_inserted_to shaft1 base) (is_free left_hand) (is_free parallel_box1) (is_free parallel_box2) (is_free inward_claw) (is_free outward_claw) (is_free no_tool) (is_equippable parallel_box1) (is_equippable parallel_box2) (is_equippable inward_claw) (is_equippable outward_claw) (is_equippable no_tool)) - (:goal (and (is_inserted_to gear1 shaft1))) -) -- Output: -``` -{"task_plan": { - "action_sequence": [ - "load_tool(left_hand, parallel_box1)", - "pick_up(left_hand, parallel_box1, gear1)", - "insert(left_hand, parallel_box1, gear1, shaft1)" - ], - "action_instructions": [ - "equip the parallel_box1 on the left hand", - "pick up the gear1 with the parallel_box1 in the left hand", - "insert the gear1 into the shaft1 using the parallel_box1 in the left hand" - ], - "object_name": ["left_hand", "gear1", "parallel_box1", "shaft1"], - "behavior_tree": { - "summary": "selector to insert gear1 into shaft1", - "name": "selector: insert(left_hand, parallel_box1, gear1, shaft1)", - "children": [ - { - "summary": "condition node to check if gear1 is inserted to shaft1", - "name": "target: is_inserted_to(gear1, shaft1)", - }, - { - "summary": "sequence to insert gear1 into shaft1", - "name": "sequence: insert(left_hand, parallel_box1, gear1, shaft1)", - "children": [ - { - "summary": "selector to load parallel_box1", - "name": "selector: load_tool(left_hand, parallel_box1)", - "children": [ - { - "summary": "condition node to check if left_hand holds parallel_box1", - "name": "target: hold(left_hand, parallel_box1)" - }, - { - "summary": "sequence to load parallel_box1", - "name": "sequence: load_tool(left_hand, parallel_box1)", - "children": [ - { - "summary": "condition node to check if parallel_box1 is equippable", - "name": "precondition: is_equippable(parallel_box1)" - }, - { - "summary": "condition node to check if left hand is free", - "name": "precondition: is_free(left_hand)" - }, - { - "summary": "action node to equip parallel_box1 to left hand", - "name": "action: load_tool(left_hand, parallel_box1)", - "effects": [ - {"summary": "left_hand will be not free",}, - {"summary": "parallel_box1 will be not equippable",}, - {"summary": "left_hand will hold parallel_box1",}, - ], - }, - ], - }, - ], - }, - { - "summary": "selector to use left_hand with parallel_box1 to pick up gear1", - "name": "selector: pick_up(left_hand, parallel_box1, gear1)", - "children": [ - { - "summary": "condition node to check if parallel_box1 holds gear1", - "name": "target: hold(parallel_box1, gear1)" - }, - { - "summary": "sequence to use left hand with parallel_box1 to pick up gear1", - "name": "sequence: pick_up(left_hand, parallel_box1, gear1)", - "children": [ - { - "summary": "condition node to check if parallel_box1 is free", - "name": "precondition: is_free(parallel_box1)" - }, - { - "summary": "condition node to check if left hand holds parallel_box1", - "name": "precondition: hold(left_hand, parallel_box1)" - }, - { - "summary": "action node to use left hand with parallel_box1 to pick up gear1", - "name": "action: pick_up(left_hand, parallel_box1, gear1)", - "effects": [ - {"summary": "parallel_box1 will hold gear1",}, - {"summary": "parallel_box1 will be not free",}, - ], - }, - ], - }, - ], - }, - { - "summary": "condition node to check if gear1 can be inserted to shaft1", - "name": "precondition: can_insert_to(gear1, shaft1)", - }, - { - "summary": "action node to use left_hand with parallel_box1 to insert gear1 to shaft1", - "name": "action: insert(left_hand, parallel_box1, gear1, shaft1)", - "effects": [ - {"summary": "parallel_box1 will be not holding gear1",}, - {"summary": "parallel_box1 will be free",}, - {"summary": "gear1 will be inserted to shaft1",}, - ], - }, - ], - }, - ], -} -}, -"initial_state": {"objects": [ - {"name": "gear1", "properties": []}, - {"name": "shaft1", "properties": []}, - {"name": "parallel_box1", "properties": ["is_equippable", "is_free"]}, - {"name": "left_hand", "properties": ["is_free"]}, - ], - "constraints": [ - {"source": "gear1", "name": "can_insert_to", "target": "shaft1"}, - {"source": "parallel_box1", "name": "can_manipulate", "target": "gear1"}, - ], - "relations": [], -}, -"instruction_summary": "insert gear1 into shaft1", -"question":""} -``` -""" -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/data/prompts/cot_skeleton/prompt/cot_sk_example.txt b/data/prompts/cot_skeleton/prompt/cot_sk_example.txt deleted file mode 100644 index 8e311506..00000000 --- a/data/prompts/cot_skeleton/prompt/cot_sk_example.txt +++ /dev/null @@ -1,156 +0,0 @@ -[user] -I will give you some examples of the input and the output you will generate. -Example 1: -""" -- Input: -problem: -(define (problem robot_assembly_problem-problem) - (:domain robot_assembly_problem-domain) - (:objects - parallel_box1 parallel_box2 inward_claw outward_claw no_tool - tool - gear1 gear2 gear3 shaft1 shaft2 base - part - left_hand - hand - ) - (:init (can_manipulate parallel_box1 gear1) (can_manipulate outward_claw gear2) (can_manipulate inward_claw gear3) (can_manipulate parallel_box2 shaft1) (can_manipulate no_tool shaft2) (can_screw_to leg1 seat) (can_screw_to leg2 seat) (can_insert_to back seat) (can_screw_to nut1 seat) (can_screw_to nut2 seat) (can_screw_to blub base) (can_place_to lamp blub) (can_insert_to shaft1 base) (can_insert_to shaft2 base) (can_insert_to gear3 shaft2) (can_insert_to gear2 base) (can_insert_to gear1 shaft1) (is_inserted_to shaft1 base) (is_free left_hand) (is_free parallel_box1) (is_free parallel_box2) (is_free inward_claw) (is_free outward_claw) (is_free no_tool) (is_equippable parallel_box1) (is_equippable parallel_box2) (is_equippable inward_claw) (is_equippable outward_claw) (is_equippable no_tool)) -) -world_state: -{"objects": [ - {"name": "gear1", "properties": []}, - {"name": "shaft1", "properties": []}, - {"name": "parallel_box1", "properties": ["is_equippable", "is_free"]}, - {"name": "left_hand", "properties": ["is_free"]}, - ], - "constraints": [], - "relations": [], -} -instructions: insert gear1 into shaft1 - -- Output: -``` -{"task_plan": { - "action_sequence": [ - "load_tool(left_hand, parallel_box1)", - "pick_up(left_hand, parallel_box1, gear1)", - "insert(left_hand, parallel_box1, gear1, shaft1)" - ], - "action_instructions": [ - "equip the parallel_box1 on the left hand", - "pick up the gear1 with the parallel_box1 in the left hand", - "insert the gear1 into the shaft1 using the parallel_box1 in the left hand" - ], - "object_name": ["left_hand", "gear1", "parallel_box1", "shaft1"], - "behavior_tree": { - "summary": "selector to insert gear1 into shaft1", - "name": "selector: insert(left_hand, parallel_box1, gear1, shaft1)", - "children": [ - { - "summary": "condition node to check if gear1 is inserted to shaft1", - "name": "target: is_inserted_to(gear1, shaft1)", - }, - { - "summary": "sequence to insert gear1 into shaft1", - "name": "sequence: insert(left_hand, parallel_box1, gear1, shaft1)", - "children": [ - { - "summary": "selector to load parallel_box1", - "name": "selector: load_tool(left_hand, parallel_box1)", - "children": [ - { - "summary": "condition node to check if left_hand holds parallel_box1", - "name": "target: hold(left_hand, parallel_box1)" - }, - { - "summary": "sequence to load parallel_box1", - "name": "sequence: load_tool(left_hand, parallel_box1)", - "children": [ - { - "summary": "condition node to check if parallel_box1 is equippable", - "name": "precondition: is_equippable(parallel_box1)" - }, - { - "summary": "condition node to check if left hand is free", - "name": "precondition: is_free(left_hand)" - }, - { - "summary": "action node to equip parallel_box1 to left hand", - "name": "action: load_tool(left_hand, parallel_box1)", - "effects": [ - {"summary": "left_hand will be not free",}, - {"summary": "parallel_box1 will be not equippable",}, - {"summary": "left_hand will hold parallel_box1",}, - ], - }, - ], - }, - ], - }, - { - "summary": "selector to use left_hand with parallel_box1 to pick up gear1", - "name": "selector: pick_up(left_hand, parallel_box1, gear1)", - "children": [ - { - "summary": "condition node to check if parallel_box1 holds gear1", - "name": "target: hold(parallel_box1, gear1)" - }, - { - "summary": "sequence to use left hand with parallel_box1 to pick up gear1", - "name": "sequence: pick_up(left_hand, parallel_box1, gear1)", - "children": [ - { - "summary": "condition node to check if parallel_box1 is free", - "name": "precondition: is_free(parallel_box1)" - }, - { - "summary": "condition node to check if left hand holds parallel_box1", - "name": "precondition: hold(left_hand, parallel_box1)" - }, - { - "summary": "action node to use left hand with parallel_box1 to pick up gear1", - "name": "action: pick_up(left_hand, parallel_box1, gear1)", - "effects": [ - {"summary": "parallel_box1 will hold gear1",}, - {"summary": "parallel_box1 will be not free",}, - ], - }, - ], - }, - ], - }, - { - "summary": "condition node to check if gear1 can be inserted to shaft1", - "name": "precondition: can_insert_to(gear1, shaft1)", - }, - { - "summary": "action node to use left_hand with parallel_box1 to insert gear1 to shaft1", - "name": "action: insert(left_hand, parallel_box1, gear1, shaft1)", - "effects": [ - {"summary": "parallel_box1 will be not holding gear1",}, - {"summary": "parallel_box1 will be free",}, - {"summary": "gear1 will be inserted to shaft1",}, - ], - }, - ], - }, - ], -} -}, -"initial_state": {"objects": [ - {"name": "gear1", "properties": []}, - {"name": "shaft1", "properties": []}, - {"name": "parallel_box1", "properties": ["is_equippable", "is_free"]}, - {"name": "left_hand", "properties": ["is_free"]}, - ], - "constraints": [ - {"source": "gear1", "name": "can_insert_to", "target": "shaft1"}, - {"source": "parallel_box1", "name": "can_manipulate", "target": "gear1"}, - ], - "relations": [], -}, -"instruction_summary": "insert gear1 into shaft1", -"question":""} -``` -""" -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/data/prompts/cot_skeleton/prompt/cot_sk_instruction.txt b/data/prompts/cot_skeleton/prompt/cot_sk_instruction.txt deleted file mode 100644 index a3d98c9b..00000000 --- a/data/prompts/cot_skeleton/prompt/cot_sk_instruction.txt +++ /dev/null @@ -1,10 +0,0 @@ -[user] -User's instruction may have different forms. -it can be one instruction string or a list of instruction strings. -You interpret the user instruction into goal state based on the domain knowledge. -Then you make the plan to achieve it. - -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/data/prompts/cot_skeleton/prompt/cot_sk_output_format.txt b/data/prompts/cot_skeleton/prompt/cot_sk_output_format.txt deleted file mode 100644 index d47c6af5..00000000 --- a/data/prompts/cot_skeleton/prompt/cot_sk_output_format.txt +++ /dev/null @@ -1,21 +0,0 @@ -[user] -You will be provided with a problem and the necessary domain knowledge. -You generate an action sequence to solve the task and contruct a behavior tree based on it in json format. -The output dictionary has five keys. -""" -- dictionary["task_plan"]: A dictionary containing the task plan. -- dictionary["initial_state"]: The initial world state before executing the plan. -- dictionary["instruction_summary"]: A brief summary of the task plan. -- dictionary["question"]: If you don't have enough knowledge for solving the problem, you can ask questions about the knowledge you need. Leave this key empty if you can solve the task. -""" -Four keys exist in dictionary["task_plan"]. -""" -- dictionary["task_plan"]["action_sequence"]: Contains a list of robot actions. Only the actions defined in the "ROBOT_ACTION LIST" will be used. -- dictionary["task_plan"]["action_instructions"]: contains a list of instructions corresponding to dictionary["task_plan"]["action_sequence"] to explain the actions. -- dictionary["task_plan"]["manipulated_objects"]: The list of the manipulated objects. It only includes the objects that are related to the current task. -- dictionary["task_plan"]["behavior_tree"]: contains a dictionary of the behavior tree constructed according to the dictionary ["task_plan"]["action_sequence"] -""" -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/data/prompts/cot_skeleton/prompt/cot_sk_problem copy.txt b/data/prompts/cot_skeleton/prompt/cot_sk_problem copy.txt deleted file mode 100644 index 405cbaaf..00000000 --- a/data/prompts/cot_skeleton/prompt/cot_sk_problem copy.txt +++ /dev/null @@ -1,24 +0,0 @@ -[user] -Information about the problem is given as a pddl problem. -It includes the following information: -- objects: all the objects in the world. They should be of a type defined in the domain knowledge. For example, "parallel_box1" is an instance of the type "tool", while "gear1" is an instance of the type "part". -- init: the initial world state to start with, described with [PREDICATE_LIST] from domain knowledge. For example, "is_free hand" means the object "hand" is free and holds nothing. -- goal: the world state to achieve, described with [PREDICATE_LIST] from domain knowledge. -An example: -""" -(define (problem robot_assembly_problem-problem) - (:domain robot_assembly_problem-domain) - (:objects - parallel_box1 parallel_box2 inward_claw outward_claw no_tool - tool - gear1 gear2 gear3 shaft1 shaft2 base - part - left_hand - hand - ) - (:init (can_manipulate parallel_box1 gear1) (can_manipulate outward_claw gear2) (can_manipulate inward_claw gear3) (can_manipulate parallel_box2 shaft1) (can_manipulate no_tool shaft2) (can_screw_to leg1 seat) (can_screw_to leg2 seat) (can_insert_to back seat) (can_screw_to nut1 seat) (can_screw_to nut2 seat) (can_screw_to blub base) (can_place_to lamp blub) (can_insert_to shaft1 base) (can_insert_to shaft2 base) (can_insert_to gear3 shaft2) (can_insert_to gear2 base) (can_insert_to gear1 shaft1) (is_free left_hand) (is_free parallel_box1) (is_free parallel_box2) (is_free inward_claw) (is_free outward_claw) (is_free no_tool) (is_equippable parallel_box1) (is_equippable parallel_box2) (is_equippable inward_claw) (is_equippable outward_claw) (is_equippable no_tool)) - (:goal (and (is_inserted_to shaft1 base))) -) -""" - -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/data/prompts/cot_skeleton/prompt/cot_sk_problem.txt b/data/prompts/cot_skeleton/prompt/cot_sk_problem.txt deleted file mode 100644 index 916311f0..00000000 --- a/data/prompts/cot_skeleton/prompt/cot_sk_problem.txt +++ /dev/null @@ -1,24 +0,0 @@ -[user] -Information about the problem is given as a pddl problem. -It includes the following information: -- objects: all the objects in the world. They should be of a type defined in the domain knowledge. For example, "parallel_box1" is an instance of the type "tool", while "gear1" is an instance of the type "part". -- init: the constraint relations in the world, described with [PREDICATE_LIST] from domain knowledge. They cannot be changed by robot actions. -- goal: IGNORE this and follow user's instructions instead. -An example: -""" -(define (problem robot_assembly_problem-problem) - (:domain robot_assembly_problem-domain) - (:objects - parallel_box1 parallel_box2 inward_claw outward_claw no_tool - tool - gear1 gear2 gear3 shaft1 shaft2 base - part - left_hand - hand - ) - (:init (can_manipulate parallel_box1 gear1) (can_manipulate outward_claw gear2) (can_manipulate inward_claw gear3) (can_manipulate parallel_box2 shaft1) (can_manipulate no_tool shaft2) (can_screw_to leg1 seat) (can_screw_to leg2 seat) (can_insert_to back seat) (can_screw_to nut1 seat) (can_screw_to nut2 seat) (can_screw_to blub base) (can_place_to lamp blub) (can_insert_to shaft1 base) (can_insert_to shaft2 base) (can_insert_to gear3 shaft2) (can_insert_to gear2 base) (can_insert_to gear1 shaft1)S) - (:goal (and )) -) -""" - -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/data/prompts/cot_skeleton/prompt/cot_sk_role.txt b/data/prompts/cot_skeleton/prompt/cot_sk_role.txt deleted file mode 100644 index 6ce736a0..00000000 --- a/data/prompts/cot_skeleton/prompt/cot_sk_role.txt +++ /dev/null @@ -1,5 +0,0 @@ -[user] -You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you break it down and construct robot task plans represented by behaiovr trees. -Please do not begin working until I say "Start working." Instead, simply output the message "Waiting for next input." Understood? -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/data/prompts/cot_skeleton/prompt/cot_sk_state copy.txt b/data/prompts/cot_skeleton/prompt/cot_sk_state copy.txt deleted file mode 100644 index eb2743b8..00000000 --- a/data/prompts/cot_skeleton/prompt/cot_sk_state copy.txt +++ /dev/null @@ -1,28 +0,0 @@ -[user] -Information about world states should be in json dictionary format. Example: -""" -{ - "objects": [ - {"name": "leg1", "properties": []}, # leg1 is an object with no properties - {"name": "seat", "properties": []}, # seat is an object with no properties - {"name": "hand", "properties": ["is_free"]}, # hand is an object with a property "is_free" - ], - "constraints": [ - {"source": "leg1", "name": "can_screw_to", "target": "seat"}, # means leg1 can be screwed to seat - ], - "relations": [ - {"source": "leg1", "name": "is_screwed_to", "target": "seat"}, # means leg1 is screwed to seat - ], -} -""" -It contains three lists: - - "objects": the objects and their properties (as lists of strings) in the world. The properties can be changed by robot actions. Refer to the input world state for them. - - "constraints": the constraint relations that are enforced and should not be changed. Refer to the pddl problem for the constraints. - - "relations": the relations in the world. They can be changed by robot actions. Refer to input world state for them. -The properties, constraints and relations should always align with the [PREDICATE_LIST] in domain knowledge. -Only concentrate on the objects, constraints and relations that are related to the current task. - -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/data/prompts/cot_skeleton/prompt/cot_sk_state.txt b/data/prompts/cot_skeleton/prompt/cot_sk_state.txt deleted file mode 100644 index 8c7d2692..00000000 --- a/data/prompts/cot_skeleton/prompt/cot_sk_state.txt +++ /dev/null @@ -1,27 +0,0 @@ -[user] -Information about world states should be in json dictionary format. Example: -""" -{ - "objects": [ - {"name": "leg1", "properties": []}, # leg1 is an object with no properties - {"name": "seat", "properties": []}, # seat is an object with no properties - {"name": "hand", "properties": ["is_free"]}, # hand is an object with a property "is_free" - ], - "constraints": [ - {"source": "leg1", "name": "can_screw_to", "target": "seat"}, # means leg1 can be screwed to seat - ], - "relations": [ - {"source": "leg1", "name": "is_screwed_to", "target": "seat"}, # means leg1 is screwed to seat - ], -} -""" -It contains three lists: - - "objects": the objects and their properties (as lists of strings) in the world. The properties can be changed by robot actions. Refer to the input world state for them. - - "constraints": the constraint relations that are enforced and should not be changed. Refer to the pddl problem for the constraints. - - "relations": the relations in the world. They can be changed by robot actions. Refer to input world state for them. -The properties, constraints and relations should always align with the [PREDICATE_LIST] in domain knowledge. - -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/data/prompts/cot_skeleton/query/query copy.txt b/data/prompts/cot_skeleton/query/query copy.txt deleted file mode 100644 index cfeb4e8e..00000000 --- a/data/prompts/cot_skeleton/query/query copy.txt +++ /dev/null @@ -1,16 +0,0 @@ -Start working. Resume from the problem below. -""" -{"problem": [PROBLEM]} -""" -The dictionary that you return should be formatted as json file. Follow these rules: -1. Always equip the proper tools before performing specific object manipulaion. -2. Make sure that each element of the ["action_instructions"] explains corresponding element of the ["action_sequence"]. Refer to the "ROBOT ACTION LIST" to understand the elements of ["action_sequence"]. -3, The length of the ["action_instructions"] list and the ["action_sequence"] list should be the same. -4. Never left ',' at the end of the list. Use json boolean values, "true" or "false", instead of "True" or "False". -5. Keep track of all items listed in the "objects" section of the "initial_state" field. -6. Use the "PREDICATE_LIST" for the initial states (:init) in the problem to generate ["initial_state"]. -7. All keys of the dictionary should be double-quoted. -8. Insert ``` at the beginning and the end of the dictionary to separate it from the rest of your response. Don't write any additional things. -9. Make sure that you output a consistent manipultation as a single arm robot. For example, grasping an object should not occur in successive steps. -10. When generating the behavior tree, make sure the tree has the asked structure. -Adhere to the output format I defined above. Follow the rules above. Think step by step. \ No newline at end of file diff --git a/data/prompts/cot_skeleton/query/query.txt b/data/prompts/cot_skeleton/query/query.txt deleted file mode 100644 index 4afcfc2a..00000000 --- a/data/prompts/cot_skeleton/query/query.txt +++ /dev/null @@ -1,17 +0,0 @@ -Start working. Resume from the problem below. -""" -{"problem": [PROBLEM]} - -""" -The dictionary that you return should be formatted as json file. Follow these rules: -1. Always equip the proper tools before performing specific object manipulaion. -2. Make sure that each element of the ["action_instructions"] explains corresponding element of the ["action_sequence"]. Refer to the "ROBOT ACTION LIST" to understand the elements of ["action_sequence"]. -3, The length of the ["action_instructions"] list and the ["action_sequence"] list should be the same. -4. Never left ',' at the end of the list. Use json boolean values, "true" or "false", instead of "True" or "False". -5. Keep track of all items listed in the "objects" section of the "initial_state" field. -6. Use the "PREDICATE_LIST" for the initial states (:init) in the problem to generate ["initial_state"]. -7. All keys of the dictionary should be double-quoted. -8. Insert ``` at the beginning and the end of the dictionary to separate it from the rest of your response. Don't write any additional things. -9. Make sure that you output a consistent manipultation as a single arm robot. For example, grasping an object should not occur in successive steps. -10. When generating the behavior tree, make sure the tree has the asked structure. -Adhere to the output format I defined above. Follow the rules above. Think step by step. \ No newline at end of file diff --git a/data/prompts/cot_skeleton/system/system.txt b/data/prompts/cot_skeleton/system/system.txt deleted file mode 100644 index 26f2afe2..00000000 --- a/data/prompts/cot_skeleton/system/system.txt +++ /dev/null @@ -1 +0,0 @@ -You are an excellent interpreter of instructions for robotic assembly tasks. Given the knowledge you need, the world states and some instructions, you break it down and construct robot task plans represented by behaiovr trees based on the action sequences. \ No newline at end of file diff --git a/data/prompts/cot_skeleton_chain/backup_do_not_use b/data/prompts/cot_skeleton_chain/backup_do_not_use deleted file mode 100644 index e69de29b..00000000 diff --git a/data/prompts/cot_skeleton_chain/prompt/cot_sk_bt.txt b/data/prompts/cot_skeleton_chain/prompt/cot_sk_bt.txt deleted file mode 100644 index 7219d47f..00000000 --- a/data/prompts/cot_skeleton_chain/prompt/cot_sk_bt.txt +++ /dev/null @@ -1,83 +0,0 @@ -[user] -The behavior tree should be in json dictionary format and have a structure like the following example: -""" -{ - "summary": "selector to use left hand to load inward_claw", - "name": "selector: load_tool(left_hand, inward_claw)", - "children": [ - { - "summary": "condition node to check if left_hand holds inward_claw", - "name": "target: hold(left_hand, inward_claw)", - }, - { - "summary": "sequence to use left hand to load inward_claw", - "name": "sequence: load_tool(left_hand, inward_claw)", - "children": [ - { - "summary": "condition node to check if inward_claw is equippable", - "name": "precondition: is_equippable(inward_claw)", - }, - { - "summary": "condition node to check if left hand is free", - "name": "precondition: is_free(left_hand)", - }, - { - "summary": "action node to use left hand to load inward_claw", - "name": "action: load_tool(left_hand, inward_claw)", - "effects": [ - {"summary": "left hand will be not free anymore",}, - {"summary": "inward_claw will be not equippable anymore",}, - {"summary": "left_hand will hold inward_claw",}, - ], - }, - ], - }, - ], -} -""" -Follow these rules: -1. Every node are described by a summary and a name: - - The "summary" key includes a short interpretive description of the node. - - The "name" key includes the type of the node and the formal form of the node according to the domain knowledge. -2. The control flow nodes, "selector" and "sequence", can have "children", which is a list of children nodes. -3. The "condition" nodes that are children of "selectors" are targets. Those that are children of "sequences" are preconditions. -4. The "action" nodes should have a key "effects" with a list of dictionaries, which describes the to-be-exerted effects of the action. The effects are shortly described under the key "summary". -5. All the subtrees always have their corresponding action in ROBOT_ACTION_LIST. The recommended structure is: - - A selector as the root node. - - (Usually one) target condition node as the first child of the selector to check if the target of the action has been fulfilled. - - A sequence node as the last child of the selector node to achieve the target. - - (Usually a number of) precondition nodes as the front children of the sequence node to check the preconditions for executing the action. - - One action node as the last child of the sequence node, whose preconditions are checked before and will immediate start the job when being ticked. - - The action will exert its defined effects after finishing the job successfully. Those effects should fulfilled all the target conditions in the selector node. - A brief example is as follows: -{ - selector: load_tool selector, - children: [ - { - target: check if inward_claw is equipped by left hand, - }, - { - sequence: load_tool, - children: [ - {precondition: check if inward_claw is equippable}, - {precondition: check if left hand is free}, - { - action: equip inward_claw to left hand, - effects: [ - left hand not free, - inward_claw not equippable, - left_hand holds inward_claw, - ] - }, - ], - }, - ], -} - -6. When performing a new action to fulfill a unsatisfied precondition, you can construct a subtree to replace the precondition and take that unfulfilled precondition as the target of the subtree. Then you construct the sequence of the subtree according to the new action you want to perform. -7. All of the control flow nodes, namely selectors and sequences, are without memory. A new behavior tree tick will always start from the root and traverse all the nodes even if they are ticked in the last tick. - -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/data/prompts/cot_skeleton_chain/prompt/cot_sk_chain copy.txt b/data/prompts/cot_skeleton_chain/prompt/cot_sk_chain copy.txt deleted file mode 100644 index 8ab56a97..00000000 --- a/data/prompts/cot_skeleton_chain/prompt/cot_sk_chain copy.txt +++ /dev/null @@ -1,15 +0,0 @@ -[user] -Follow this workflow to solve the task: - -1. Read the problem description. Parse and generate the "initial_state" in the output dictionary. -2. Interpret the goal of the problem as goal_state and write into the "instruction_summary" in the output dictionary. -3. Try to solve the problem, achieve the goal_state from the initial_state based on the domain knowledge provided. Generate a action sequence with the actions defined in and write it into dictionary["task_plan"]["action_sequence"] of the output. -4. If you cannot find a plan to solve the task, ask question with nature language in the "question" of the output dictionary. -5. Explain the actions in the action sequence and write the output dictionary["task_plan"]["action_instructions"]. -6. Collect the objects in the world that are effected by the action sequence, write into dictionary["task_plan"]["manipulated_objects"]. -7. Generate the behavior tree based on the action sequence and write into dictionary["task_plan"]["behavior_tree"]. - -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/data/prompts/cot_skeleton_chain/prompt/cot_sk_chain.txt b/data/prompts/cot_skeleton_chain/prompt/cot_sk_chain.txt deleted file mode 100644 index 231c0b8f..00000000 --- a/data/prompts/cot_skeleton_chain/prompt/cot_sk_chain.txt +++ /dev/null @@ -1,16 +0,0 @@ -[user] -Follow this workflow to solve the task: - -1. Read the problem description. Parse it and fill the "constraints" in "initial_state". -2. Fill the "objects" and "relations" in "initial_state" based on the input world state. -3. Interpret the input instructions as goal_state and write into the "instruction_summary" in the output dictionary. -4. Try to make a plan, achieve the goal_state from the initial_state based on the domain knowledge provided. Generate a action sequence with the actions defined in and write it into dictionary["task_plan"]["action_sequence"] of the output. -5. If you cannot find a plan to solve the task, ask question with nature language in the "question" of the output dictionary. -6. Explain the actions in the action sequence and write the output dictionary["task_plan"]["action_instructions"]. -7. Collect the objects that are effected by the action sequence, write into dictionary["task_plan"]["manipulated_objects"]. -8. Generate the behavior tree based on the action sequence and write into dictionary["task_plan"]["behavior_tree"]. - -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/data/prompts/cot_skeleton_chain/prompt/cot_sk_domain.txt b/data/prompts/cot_skeleton_chain/prompt/cot_sk_domain.txt deleted file mode 100644 index bf3c7859..00000000 --- a/data/prompts/cot_skeleton_chain/prompt/cot_sk_domain.txt +++ /dev/null @@ -1,89 +0,0 @@ -[user] -Following is the domain knowledge you need in pddl domain format, which includes: -- the object types in the world, -- the predicates for describing the states of the world, -- the actions that the robot can apply, with their precondtions and effects defined. -""" -(define (domain robot_assembly_problem-domain) - (:requirements :strips :typing) - (:types - thing - object - tool part hand - thing - ) - (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part)) - (:action pick_up - :parameters ( ?hand - hand ?tool - tool ?part - part) - :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part)) - :effect (and (hold ?tool ?part) (not (is_free ?tool)))) - (:action put_down - :parameters ( ?hand - hand ?tool - tool ?part - part) - :precondition (and (hold ?tool ?part) (hold ?hand ?tool)) - :effect (and (not (hold ?hand ?part)) (is_free ?tool))) - (:action place - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2))) - (:action detach - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2)))) - (:action insert - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2))) - (:action pull - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2)))) - (:action screw - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2))) - (:action unscrew - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2)))) - (:action load_tool - :parameters ( ?hand - hand ?tool - tool) - :precondition (and (is_equippable ?tool) (is_free ?hand)) - :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool))) - (:action unload_tool - :parameters ( ?hand - hand ?tool - tool) - :precondition (and (hold ?hand ?tool) (is_free ?tool)) - :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool)))) -) -""" -The precidates are explained as follows: -""" -"PREDICATE_LIST" -- is_free( or ): or is free and can hold something. (default: true) -- is_equippable(): is ready to be eqquipped by a robot hand. (default: true) -- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false) -- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false) -- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false) -- can_place_to(, ): can be placed into (placement assembly constraint) (default: false) -- hold(, ): is holding (default: false) -- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false) -- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false) -- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false) -""" - -The actions are explained as follows: -""" -"ROBOT_ACTION_LIST" -- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something. -- put_down(, , ): use to put down . As preconditions, should be holding and should be holding . -- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding . -- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into . -- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding . -- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into . -- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding . -- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into . -- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free. -- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free. -""" -the items in <> indicate the type of the instances needed for the predicates and actions. -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. diff --git a/data/prompts/cot_skeleton_chain/prompt/cot_sk_example copy.txt b/data/prompts/cot_skeleton_chain/prompt/cot_sk_example copy.txt deleted file mode 100644 index 02853177..00000000 --- a/data/prompts/cot_skeleton_chain/prompt/cot_sk_example copy.txt +++ /dev/null @@ -1,144 +0,0 @@ -[user] -I will give you some examples of the input and the output you will generate. -Example 1: -""" -- Input: -(define (problem robot_assembly_problem-problem) - (:domain robot_assembly_problem-domain) - (:objects - parallel_box1 parallel_box2 inward_claw outward_claw no_tool - tool - gear1 gear2 gear3 shaft1 shaft2 base - part - left_hand - hand - ) - (:init (can_manipulate parallel_box1 gear1) (can_manipulate outward_claw gear2) (can_manipulate inward_claw gear3) (can_manipulate parallel_box2 shaft1) (can_manipulate no_tool shaft2) (can_screw_to leg1 seat) (can_screw_to leg2 seat) (can_insert_to back seat) (can_screw_to nut1 seat) (can_screw_to nut2 seat) (can_screw_to blub base) (can_place_to lamp blub) (can_insert_to shaft1 base) (can_insert_to shaft2 base) (can_insert_to gear3 shaft2) (can_insert_to gear2 base) (can_insert_to gear1 shaft1) (is_inserted_to shaft1 base) (is_free left_hand) (is_free parallel_box1) (is_free parallel_box2) (is_free inward_claw) (is_free outward_claw) (is_free no_tool) (is_equippable parallel_box1) (is_equippable parallel_box2) (is_equippable inward_claw) (is_equippable outward_claw) (is_equippable no_tool)) - (:goal (and (is_inserted_to gear1 shaft1))) -) -- Output: -``` -{"task_plan": { - "action_sequence": [ - "load_tool(left_hand, parallel_box1)", - "pick_up(left_hand, parallel_box1, gear1)", - "insert(left_hand, parallel_box1, gear1, shaft1)" - ], - "action_instructions": [ - "equip the parallel_box1 on the left hand", - "pick up the gear1 with the parallel_box1 in the left hand", - "insert the gear1 into the shaft1 using the parallel_box1 in the left hand" - ], - "object_name": ["left_hand", "gear1", "parallel_box1", "shaft1"], - "behavior_tree": { - "summary": "selector to insert gear1 into shaft1", - "name": "selector: insert(left_hand, parallel_box1, gear1, shaft1)", - "children": [ - { - "summary": "condition node to check if gear1 is inserted to shaft1", - "name": "target: is_inserted_to(gear1, shaft1)", - }, - { - "summary": "sequence to insert gear1 into shaft1", - "name": "sequence: insert(left_hand, parallel_box1, gear1, shaft1)", - "children": [ - { - "summary": "selector to load parallel_box1", - "name": "selector: load_tool(left_hand, parallel_box1)", - "children": [ - { - "summary": "condition node to check if left_hand holds parallel_box1", - "name": "target: hold(left_hand, parallel_box1)" - }, - { - "summary": "sequence to load parallel_box1", - "name": "sequence: load_tool(left_hand, parallel_box1)", - "children": [ - { - "summary": "condition node to check if parallel_box1 is equippable", - "name": "precondition: is_equippable(parallel_box1)" - }, - { - "summary": "condition node to check if left hand is free", - "name": "precondition: is_free(left_hand)" - }, - { - "summary": "action node to equip parallel_box1 to left hand", - "name": "action: load_tool(left_hand, parallel_box1)", - "effects": [ - {"summary": "left_hand will be not free",}, - {"summary": "parallel_box1 will be not equippable",}, - {"summary": "left_hand will hold parallel_box1",}, - ], - }, - ], - }, - ], - }, - { - "summary": "selector to use left_hand with parallel_box1 to pick up gear1", - "name": "selector: pick_up(left_hand, parallel_box1, gear1)", - "children": [ - { - "summary": "condition node to check if parallel_box1 holds gear1", - "name": "target: hold(parallel_box1, gear1)" - }, - { - "summary": "sequence to use left hand with parallel_box1 to pick up gear1", - "name": "sequence: pick_up(left_hand, parallel_box1, gear1)", - "children": [ - { - "summary": "condition node to check if parallel_box1 is free", - "name": "precondition: is_free(parallel_box1)" - }, - { - "summary": "condition node to check if left hand holds parallel_box1", - "name": "precondition: hold(left_hand, parallel_box1)" - }, - { - "summary": "action node to use left hand with parallel_box1 to pick up gear1", - "name": "action: pick_up(left_hand, parallel_box1, gear1)", - "effects": [ - {"summary": "parallel_box1 will hold gear1",}, - {"summary": "parallel_box1 will be not free",}, - ], - }, - ], - }, - ], - }, - { - "summary": "condition node to check if gear1 can be inserted to shaft1", - "name": "precondition: can_insert_to(gear1, shaft1)", - }, - { - "summary": "action node to use left_hand with parallel_box1 to insert gear1 to shaft1", - "name": "action: insert(left_hand, parallel_box1, gear1, shaft1)", - "effects": [ - {"summary": "parallel_box1 will be not holding gear1",}, - {"summary": "parallel_box1 will be free",}, - {"summary": "gear1 will be inserted to shaft1",}, - ], - }, - ], - }, - ], -} -}, -"initial_state": {"objects": [ - {"name": "gear1", "properties": []}, - {"name": "shaft1", "properties": []}, - {"name": "parallel_box1", "properties": ["is_equippable", "is_free"]}, - {"name": "left_hand", "properties": ["is_free"]}, - ], - "constraints": [ - {"source": "gear1", "name": "can_insert_to", "target": "shaft1"}, - {"source": "parallel_box1", "name": "can_manipulate", "target": "gear1"}, - ], - "relations": [], -}, -"instruction_summary": "insert gear1 into shaft1", -"question":""} -``` -""" -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/data/prompts/cot_skeleton_chain/prompt/cot_sk_example.txt b/data/prompts/cot_skeleton_chain/prompt/cot_sk_example.txt deleted file mode 100644 index 8e311506..00000000 --- a/data/prompts/cot_skeleton_chain/prompt/cot_sk_example.txt +++ /dev/null @@ -1,156 +0,0 @@ -[user] -I will give you some examples of the input and the output you will generate. -Example 1: -""" -- Input: -problem: -(define (problem robot_assembly_problem-problem) - (:domain robot_assembly_problem-domain) - (:objects - parallel_box1 parallel_box2 inward_claw outward_claw no_tool - tool - gear1 gear2 gear3 shaft1 shaft2 base - part - left_hand - hand - ) - (:init (can_manipulate parallel_box1 gear1) (can_manipulate outward_claw gear2) (can_manipulate inward_claw gear3) (can_manipulate parallel_box2 shaft1) (can_manipulate no_tool shaft2) (can_screw_to leg1 seat) (can_screw_to leg2 seat) (can_insert_to back seat) (can_screw_to nut1 seat) (can_screw_to nut2 seat) (can_screw_to blub base) (can_place_to lamp blub) (can_insert_to shaft1 base) (can_insert_to shaft2 base) (can_insert_to gear3 shaft2) (can_insert_to gear2 base) (can_insert_to gear1 shaft1) (is_inserted_to shaft1 base) (is_free left_hand) (is_free parallel_box1) (is_free parallel_box2) (is_free inward_claw) (is_free outward_claw) (is_free no_tool) (is_equippable parallel_box1) (is_equippable parallel_box2) (is_equippable inward_claw) (is_equippable outward_claw) (is_equippable no_tool)) -) -world_state: -{"objects": [ - {"name": "gear1", "properties": []}, - {"name": "shaft1", "properties": []}, - {"name": "parallel_box1", "properties": ["is_equippable", "is_free"]}, - {"name": "left_hand", "properties": ["is_free"]}, - ], - "constraints": [], - "relations": [], -} -instructions: insert gear1 into shaft1 - -- Output: -``` -{"task_plan": { - "action_sequence": [ - "load_tool(left_hand, parallel_box1)", - "pick_up(left_hand, parallel_box1, gear1)", - "insert(left_hand, parallel_box1, gear1, shaft1)" - ], - "action_instructions": [ - "equip the parallel_box1 on the left hand", - "pick up the gear1 with the parallel_box1 in the left hand", - "insert the gear1 into the shaft1 using the parallel_box1 in the left hand" - ], - "object_name": ["left_hand", "gear1", "parallel_box1", "shaft1"], - "behavior_tree": { - "summary": "selector to insert gear1 into shaft1", - "name": "selector: insert(left_hand, parallel_box1, gear1, shaft1)", - "children": [ - { - "summary": "condition node to check if gear1 is inserted to shaft1", - "name": "target: is_inserted_to(gear1, shaft1)", - }, - { - "summary": "sequence to insert gear1 into shaft1", - "name": "sequence: insert(left_hand, parallel_box1, gear1, shaft1)", - "children": [ - { - "summary": "selector to load parallel_box1", - "name": "selector: load_tool(left_hand, parallel_box1)", - "children": [ - { - "summary": "condition node to check if left_hand holds parallel_box1", - "name": "target: hold(left_hand, parallel_box1)" - }, - { - "summary": "sequence to load parallel_box1", - "name": "sequence: load_tool(left_hand, parallel_box1)", - "children": [ - { - "summary": "condition node to check if parallel_box1 is equippable", - "name": "precondition: is_equippable(parallel_box1)" - }, - { - "summary": "condition node to check if left hand is free", - "name": "precondition: is_free(left_hand)" - }, - { - "summary": "action node to equip parallel_box1 to left hand", - "name": "action: load_tool(left_hand, parallel_box1)", - "effects": [ - {"summary": "left_hand will be not free",}, - {"summary": "parallel_box1 will be not equippable",}, - {"summary": "left_hand will hold parallel_box1",}, - ], - }, - ], - }, - ], - }, - { - "summary": "selector to use left_hand with parallel_box1 to pick up gear1", - "name": "selector: pick_up(left_hand, parallel_box1, gear1)", - "children": [ - { - "summary": "condition node to check if parallel_box1 holds gear1", - "name": "target: hold(parallel_box1, gear1)" - }, - { - "summary": "sequence to use left hand with parallel_box1 to pick up gear1", - "name": "sequence: pick_up(left_hand, parallel_box1, gear1)", - "children": [ - { - "summary": "condition node to check if parallel_box1 is free", - "name": "precondition: is_free(parallel_box1)" - }, - { - "summary": "condition node to check if left hand holds parallel_box1", - "name": "precondition: hold(left_hand, parallel_box1)" - }, - { - "summary": "action node to use left hand with parallel_box1 to pick up gear1", - "name": "action: pick_up(left_hand, parallel_box1, gear1)", - "effects": [ - {"summary": "parallel_box1 will hold gear1",}, - {"summary": "parallel_box1 will be not free",}, - ], - }, - ], - }, - ], - }, - { - "summary": "condition node to check if gear1 can be inserted to shaft1", - "name": "precondition: can_insert_to(gear1, shaft1)", - }, - { - "summary": "action node to use left_hand with parallel_box1 to insert gear1 to shaft1", - "name": "action: insert(left_hand, parallel_box1, gear1, shaft1)", - "effects": [ - {"summary": "parallel_box1 will be not holding gear1",}, - {"summary": "parallel_box1 will be free",}, - {"summary": "gear1 will be inserted to shaft1",}, - ], - }, - ], - }, - ], -} -}, -"initial_state": {"objects": [ - {"name": "gear1", "properties": []}, - {"name": "shaft1", "properties": []}, - {"name": "parallel_box1", "properties": ["is_equippable", "is_free"]}, - {"name": "left_hand", "properties": ["is_free"]}, - ], - "constraints": [ - {"source": "gear1", "name": "can_insert_to", "target": "shaft1"}, - {"source": "parallel_box1", "name": "can_manipulate", "target": "gear1"}, - ], - "relations": [], -}, -"instruction_summary": "insert gear1 into shaft1", -"question":""} -``` -""" -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/data/prompts/cot_skeleton_chain/prompt/cot_sk_output_format.txt b/data/prompts/cot_skeleton_chain/prompt/cot_sk_output_format.txt deleted file mode 100644 index d47c6af5..00000000 --- a/data/prompts/cot_skeleton_chain/prompt/cot_sk_output_format.txt +++ /dev/null @@ -1,21 +0,0 @@ -[user] -You will be provided with a problem and the necessary domain knowledge. -You generate an action sequence to solve the task and contruct a behavior tree based on it in json format. -The output dictionary has five keys. -""" -- dictionary["task_plan"]: A dictionary containing the task plan. -- dictionary["initial_state"]: The initial world state before executing the plan. -- dictionary["instruction_summary"]: A brief summary of the task plan. -- dictionary["question"]: If you don't have enough knowledge for solving the problem, you can ask questions about the knowledge you need. Leave this key empty if you can solve the task. -""" -Four keys exist in dictionary["task_plan"]. -""" -- dictionary["task_plan"]["action_sequence"]: Contains a list of robot actions. Only the actions defined in the "ROBOT_ACTION LIST" will be used. -- dictionary["task_plan"]["action_instructions"]: contains a list of instructions corresponding to dictionary["task_plan"]["action_sequence"] to explain the actions. -- dictionary["task_plan"]["manipulated_objects"]: The list of the manipulated objects. It only includes the objects that are related to the current task. -- dictionary["task_plan"]["behavior_tree"]: contains a dictionary of the behavior tree constructed according to the dictionary ["task_plan"]["action_sequence"] -""" -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/data/prompts/cot_skeleton_chain/prompt/cot_sk_problem copy.txt b/data/prompts/cot_skeleton_chain/prompt/cot_sk_problem copy.txt deleted file mode 100644 index 6b1ddfb7..00000000 --- a/data/prompts/cot_skeleton_chain/prompt/cot_sk_problem copy.txt +++ /dev/null @@ -1,24 +0,0 @@ -[user] -Information about the problem is given as a pddl problem. -It includes the following information: -- objects: all the objects in the world. They should be of a type defined in the domain knowledge. For example, "parallel_box1" is an instance of the type "tool", while "gear1" is an instance of the type "part". -- init: the initial world state to start with, described with [PREDICATE_LIST] from domain knowledge. For example, "is_free hand" means the object "hand" is free and holds nothing. -- goal: the world state to achieve, described with [PREDICATE_LIST] from domain knowledge. -An example: -""" -(define (problem robot_assembly_problem-problem) - (:domain robot_assembly_problem-domain) - (:objects - parallel_box1 parallel_box2 inward_claw outward_claw no_tool - tool - gear1 gear2 gear3 shaft1 shaft2 base - part - left_hand - hand - ) - (:init (can_manipulate parallel_box1 gear1) (can_manipulate outward_claw gear2) (can_manipulate inward_claw gear3) (can_manipulate parallel_box2 shaft1) (can_manipulate no_tool shaft2) (can_screw_to leg1 seat) (can_screw_to leg2 seat) (can_insert_to back seat) (can_screw_to nut1 seat) (can_screw_to nut2 seat) (can_screw_to blub base) (can_place_to lamp blub) (can_insert_to shaft1 base) (can_insert_to shaft2 base) (can_insert_to gear3 shaft2) (can_insert_to gear2 base) (can_insert_to gear1 shaft1) (is_free left_hand) (is_free parallel_box1) (is_free parallel_box2) (is_free inward_claw) (is_free outward_claw) (is_free no_tool) (is_equippable parallel_box1) (is_equippable parallel_box2) (is_equippable inward_claw) (is_equippable outward_claw) (is_equippable no_tool)) - (:goal (and (is_inserted_to shaft1 base))) -) -""" - -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/data/prompts/cot_skeleton_chain/prompt/cot_sk_problem.txt b/data/prompts/cot_skeleton_chain/prompt/cot_sk_problem.txt deleted file mode 100644 index 916311f0..00000000 --- a/data/prompts/cot_skeleton_chain/prompt/cot_sk_problem.txt +++ /dev/null @@ -1,24 +0,0 @@ -[user] -Information about the problem is given as a pddl problem. -It includes the following information: -- objects: all the objects in the world. They should be of a type defined in the domain knowledge. For example, "parallel_box1" is an instance of the type "tool", while "gear1" is an instance of the type "part". -- init: the constraint relations in the world, described with [PREDICATE_LIST] from domain knowledge. They cannot be changed by robot actions. -- goal: IGNORE this and follow user's instructions instead. -An example: -""" -(define (problem robot_assembly_problem-problem) - (:domain robot_assembly_problem-domain) - (:objects - parallel_box1 parallel_box2 inward_claw outward_claw no_tool - tool - gear1 gear2 gear3 shaft1 shaft2 base - part - left_hand - hand - ) - (:init (can_manipulate parallel_box1 gear1) (can_manipulate outward_claw gear2) (can_manipulate inward_claw gear3) (can_manipulate parallel_box2 shaft1) (can_manipulate no_tool shaft2) (can_screw_to leg1 seat) (can_screw_to leg2 seat) (can_insert_to back seat) (can_screw_to nut1 seat) (can_screw_to nut2 seat) (can_screw_to blub base) (can_place_to lamp blub) (can_insert_to shaft1 base) (can_insert_to shaft2 base) (can_insert_to gear3 shaft2) (can_insert_to gear2 base) (can_insert_to gear1 shaft1)S) - (:goal (and )) -) -""" - -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/data/prompts/cot_skeleton_chain/prompt/cot_sk_role.txt b/data/prompts/cot_skeleton_chain/prompt/cot_sk_role.txt deleted file mode 100644 index 6ce736a0..00000000 --- a/data/prompts/cot_skeleton_chain/prompt/cot_sk_role.txt +++ /dev/null @@ -1,5 +0,0 @@ -[user] -You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you break it down and construct robot task plans represented by behaiovr trees. -Please do not begin working until I say "Start working." Instead, simply output the message "Waiting for next input." Understood? -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/data/prompts/cot_skeleton_chain/prompt/cot_sk_state copy.txt b/data/prompts/cot_skeleton_chain/prompt/cot_sk_state copy.txt deleted file mode 100644 index 00d748ea..00000000 --- a/data/prompts/cot_skeleton_chain/prompt/cot_sk_state copy.txt +++ /dev/null @@ -1,28 +0,0 @@ -[user] -Information about world states should be in json dictionary format. Example: -""" -{ - "objects": [ - {"name": "leg1", "properties": []}, # leg1 is an object with no properties - {"name": "seat", "properties": []}, # seat is an object with no properties - {"name": "hand", "properties": ["is_free"]}, # hand is an object with a property "is_free" - ], - "constraints": [ - {"source": "leg1", "name": "can_screw_to", "target": "seat"}, # means leg1 can be screwed to seat - ], - "relations": [ - {"source": "leg1", "name": "is_screwed_to", "target": "seat"}, # means leg1 is screwed to seat - ], -} -""" -It contains three lists: - - "objects": the objects and their properties (as lists of strings) in the world. The properties can be changed by robot actions. - - "constraints": the constraint relations that are enforced and should not be changed. They are knowledge about the world or some constraints set by the user. - - "relations": the relations in the world. They can be changed by robot actions. -The properties, constraints and relations should always align with the [PREDICATE_LIST] in domain knowledge. -Only concentrate on the objects, constraints and relations that are related to the current task. - -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/data/prompts/cot_skeleton_chain/prompt/cot_sk_state.txt b/data/prompts/cot_skeleton_chain/prompt/cot_sk_state.txt deleted file mode 100644 index 8c7d2692..00000000 --- a/data/prompts/cot_skeleton_chain/prompt/cot_sk_state.txt +++ /dev/null @@ -1,27 +0,0 @@ -[user] -Information about world states should be in json dictionary format. Example: -""" -{ - "objects": [ - {"name": "leg1", "properties": []}, # leg1 is an object with no properties - {"name": "seat", "properties": []}, # seat is an object with no properties - {"name": "hand", "properties": ["is_free"]}, # hand is an object with a property "is_free" - ], - "constraints": [ - {"source": "leg1", "name": "can_screw_to", "target": "seat"}, # means leg1 can be screwed to seat - ], - "relations": [ - {"source": "leg1", "name": "is_screwed_to", "target": "seat"}, # means leg1 is screwed to seat - ], -} -""" -It contains three lists: - - "objects": the objects and their properties (as lists of strings) in the world. The properties can be changed by robot actions. Refer to the input world state for them. - - "constraints": the constraint relations that are enforced and should not be changed. Refer to the pddl problem for the constraints. - - "relations": the relations in the world. They can be changed by robot actions. Refer to input world state for them. -The properties, constraints and relations should always align with the [PREDICATE_LIST] in domain knowledge. - -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/data/prompts/cot_skeleton_chain/query/query copy.txt b/data/prompts/cot_skeleton_chain/query/query copy.txt deleted file mode 100644 index cbe4d915..00000000 --- a/data/prompts/cot_skeleton_chain/query/query copy.txt +++ /dev/null @@ -1,16 +0,0 @@ -Start working. Resume from the problem below. ---------------- -{problem} ---------------- -The dictionary that you return should be formatted as json file. Follow these rules: -1. Always equip the proper tools before performing specific object manipulaion. -2. Make sure that each element of the ["action_instructions"] explains corresponding element of the ["action_sequence"]. Refer to the "ROBOT ACTION LIST" to understand the elements of ["action_sequence"]. -3, The length of the ["action_instructions"] list and the ["action_sequence"] list should be the same. -4. Never left ',' at the end of the list. Use json boolean values, "true" or "false", instead of "True" or "False". -5. Keep track of all items listed in the "objects" section of the "initial_state" field. -6. Use the "PREDICATE_LIST" for the initial states (:init) in the problem to generate ["initial_state"]. -7. All keys of the dictionary should be double-quoted. -8. Return the json dictionary without adding anything at the beginning and the end of it. You return should be ready to be parsed as a json object directly. -9. Make sure that you output a consistent manipultation as a single arm robot. For example, grasping an object should not occur in successive steps. -10. When generating the behavior tree, make sure the tree has the asked structure. -Adhere to the output format I defined above. Follow the rules above. Think step by step. \ No newline at end of file diff --git a/data/prompts/cot_skeleton_chain/query/query.txt b/data/prompts/cot_skeleton_chain/query/query.txt deleted file mode 100644 index 35684e9e..00000000 --- a/data/prompts/cot_skeleton_chain/query/query.txt +++ /dev/null @@ -1,20 +0,0 @@ -Start working. Resume from the input below. ---------------- -problem: {problem} - -world_state: {world_state} - -instructions: {instructions} ---------------- -The dictionary that you return should be formatted as json file. Follow these rules: -1. Always equip the proper tools before performing specific object manipulaion. -2. Make sure that each element of the ["action_instructions"] explains corresponding element of the ["action_sequence"]. Refer to the "ROBOT ACTION LIST" to understand the elements of ["action_sequence"]. -3, The length of the ["action_instructions"] list and the ["action_sequence"] list should be the same. -4. Never left ',' at the end of the list. Use json boolean values, "true" or "false", instead of "True" or "False". -5. Keep track of all items listed in the "objects" section of the "initial_state" field. -6. The ["initial_state"] should be generated based on "PREDICATE_LIST". -7. All keys of the dictionary should be double-quoted. -8. Return the json dictionary without adding anything at the beginning and the end of it. You return should be ready to be parsed as a json object directly. -9. Make sure that you output a consistent manipultation as a single arm robot. For example, grasping an object should not occur in successive steps. -10. When generating the behavior tree, make sure the tree has the asked structure. -Adhere to the output format I defined above. Follow the rules above. Think step by step. \ No newline at end of file diff --git a/data/prompts/cot_skeleton_chain/system/system.txt b/data/prompts/cot_skeleton_chain/system/system.txt deleted file mode 100644 index 26f2afe2..00000000 --- a/data/prompts/cot_skeleton_chain/system/system.txt +++ /dev/null @@ -1 +0,0 @@ -You are an excellent interpreter of instructions for robotic assembly tasks. Given the knowledge you need, the world states and some instructions, you break it down and construct robot task plans represented by behaiovr trees based on the action sequences. \ No newline at end of file diff --git a/data/prompts/cot_sp_skeleton/backup_do_not_use b/data/prompts/cot_sp_skeleton/backup_do_not_use deleted file mode 100644 index e69de29b..00000000 diff --git a/data/prompts/cot_sp_skeleton/prompt/cot_bt.txt b/data/prompts/cot_sp_skeleton/prompt/cot_bt.txt deleted file mode 100644 index e2fc475c..00000000 --- a/data/prompts/cot_sp_skeleton/prompt/cot_bt.txt +++ /dev/null @@ -1,68 +0,0 @@ -[user] -The behavior tree should be in json dictionary format and have a structure like the following example: -""" -{ - "summary": "selector to load inward_claw to left hand", - "children": [ - { - "summary": "condition node to check if inward_claw is held by left hand", - }, - { - "summary": "sequence to load inward_claw to left hand", - "children": [ - { - "summary": "condition node to check if inward_claw is equippable", - }, - { - "summary": "condition node to check if left hand is free", - }, - { - "summary": "action node to equip inward_claw to left hand", - "effects": [ - {"summary": "left hand will be not free anymore",}, - {"summary": "inward_claw will be not equippable anymore",}, - {"summary": "left_hand will hold inward_claw",}, - ], - }, - ], - }, - ], -} -""" -Follow these rules: -1. Every node are described by a summary, which is a short interpretive description of the node. It should include the type of the node and the purpose of the node. -2. The control flow nodes, "selector" and "sequence", can have "children", which is a list of children nodes. -3. The "condition" nodes that are children of "selectors" are "targets". Those that are children of "sequences" are preconditions. -4. The "action" nodes should have a key "effects" with a list of dictionaries, which describes the to-be-exerted effects of the action. The effects are shortly described under the key "summary". -5. All the subtrees always have their corresponding action in ROBOT_ACTION_LIST. They are used to represent the actions in the action sequence. The recommended structure is: - - A selector as the root node. - - (Usually one) target condition node as the first child of the selector to check if the target of the action has been fulfilled. - - A sequence node as the last child of the selector node to achieve the target. - - (Usually a number of) precondition nodes as the front children of the sequence node to check the preconditions for executing the action. - - One action node as the last child of the sequence node, whose preconditions are checked before and will immediate start the job when being ticked. - - The action will exert its defined effects after finishing the job successfully. Those effects should fulfilled all the target conditions in the selector node. - A brief example is as follows: -{ - selector: load_tool selector, - children: [ - { - target: check if inward_claw is equipped by left hand, - }, - { - sequence: load_tool, - children: [ - precondition: check if inward_claw is equippable, - precondition: check if left hand is free, - {action: equip inward_claw to left hand, effects: [left hand not free, inward_claw not equippable, left_hand holds inward_claw]}, - ], - }, - ], -} - -6. When performing a new action to fulfill a unsatisfied precondition, you can construct a subtree to replace the precondition and take that unfulfilled precondition as the target of the subtree. Then you construct the sequence of the subtree according to the new action you want to perform. -7. All of the control flow nodes, namely selectors and sequences, are without memory. A new behavior tree tick will always start from the root and traverse all the nodes even if they are ticked in the last tick. - -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/data/prompts/cot_sp_skeleton/prompt/cot_chain.txt b/data/prompts/cot_sp_skeleton/prompt/cot_chain.txt deleted file mode 100644 index 8ab56a97..00000000 --- a/data/prompts/cot_sp_skeleton/prompt/cot_chain.txt +++ /dev/null @@ -1,15 +0,0 @@ -[user] -Follow this workflow to solve the task: - -1. Read the problem description. Parse and generate the "initial_state" in the output dictionary. -2. Interpret the goal of the problem as goal_state and write into the "instruction_summary" in the output dictionary. -3. Try to solve the problem, achieve the goal_state from the initial_state based on the domain knowledge provided. Generate a action sequence with the actions defined in and write it into dictionary["task_plan"]["action_sequence"] of the output. -4. If you cannot find a plan to solve the task, ask question with nature language in the "question" of the output dictionary. -5. Explain the actions in the action sequence and write the output dictionary["task_plan"]["action_instructions"]. -6. Collect the objects in the world that are effected by the action sequence, write into dictionary["task_plan"]["manipulated_objects"]. -7. Generate the behavior tree based on the action sequence and write into dictionary["task_plan"]["behavior_tree"]. - -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/data/prompts/cot_sp_skeleton/prompt/cot_domain.txt b/data/prompts/cot_sp_skeleton/prompt/cot_domain.txt deleted file mode 100644 index bf3c7859..00000000 --- a/data/prompts/cot_sp_skeleton/prompt/cot_domain.txt +++ /dev/null @@ -1,89 +0,0 @@ -[user] -Following is the domain knowledge you need in pddl domain format, which includes: -- the object types in the world, -- the predicates for describing the states of the world, -- the actions that the robot can apply, with their precondtions and effects defined. -""" -(define (domain robot_assembly_problem-domain) - (:requirements :strips :typing) - (:types - thing - object - tool part hand - thing - ) - (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part)) - (:action pick_up - :parameters ( ?hand - hand ?tool - tool ?part - part) - :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part)) - :effect (and (hold ?tool ?part) (not (is_free ?tool)))) - (:action put_down - :parameters ( ?hand - hand ?tool - tool ?part - part) - :precondition (and (hold ?tool ?part) (hold ?hand ?tool)) - :effect (and (not (hold ?hand ?part)) (is_free ?tool))) - (:action place - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2))) - (:action detach - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2)))) - (:action insert - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2))) - (:action pull - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2)))) - (:action screw - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2))) - (:action unscrew - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2)))) - (:action load_tool - :parameters ( ?hand - hand ?tool - tool) - :precondition (and (is_equippable ?tool) (is_free ?hand)) - :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool))) - (:action unload_tool - :parameters ( ?hand - hand ?tool - tool) - :precondition (and (hold ?hand ?tool) (is_free ?tool)) - :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool)))) -) -""" -The precidates are explained as follows: -""" -"PREDICATE_LIST" -- is_free( or ): or is free and can hold something. (default: true) -- is_equippable(): is ready to be eqquipped by a robot hand. (default: true) -- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false) -- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false) -- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false) -- can_place_to(, ): can be placed into (placement assembly constraint) (default: false) -- hold(, ): is holding (default: false) -- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false) -- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false) -- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false) -""" - -The actions are explained as follows: -""" -"ROBOT_ACTION_LIST" -- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something. -- put_down(, , ): use to put down . As preconditions, should be holding and should be holding . -- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding . -- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into . -- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding . -- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into . -- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding . -- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into . -- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free. -- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free. -""" -the items in <> indicate the type of the instances needed for the predicates and actions. -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. diff --git a/data/prompts/cot_sp_skeleton/prompt/cot_example.txt b/data/prompts/cot_sp_skeleton/prompt/cot_example.txt deleted file mode 100644 index 3d8d6d5b..00000000 --- a/data/prompts/cot_sp_skeleton/prompt/cot_example.txt +++ /dev/null @@ -1,127 +0,0 @@ -[user] -I will give you some examples of the input and the output you will generate. -Example 1: -""" -- Input: -(define (problem robot_assembly_problem-problem) - (:domain robot_assembly_problem-domain) - (:objects - parallel_box1 parallel_box2 inward_claw outward_claw no_tool - tool - gear1 gear2 gear3 shaft1 shaft2 base - part - left_hand - hand - ) - (:init (can_manipulate parallel_box1 gear1) (can_manipulate outward_claw gear2) (can_manipulate inward_claw gear3) (can_manipulate parallel_box2 shaft1) (can_manipulate no_tool shaft2) (can_screw_to leg1 seat) (can_screw_to leg2 seat) (can_insert_to back seat) (can_screw_to nut1 seat) (can_screw_to nut2 seat) (can_screw_to blub base) (can_place_to lamp blub) (can_insert_to shaft1 base) (can_insert_to shaft2 base) (can_insert_to gear3 shaft2) (can_insert_to gear2 base) (can_insert_to gear1 shaft1) (is_inserted_to shaft1 base) (is_free left_hand) (is_free parallel_box1) (is_free parallel_box2) (is_free inward_claw) (is_free outward_claw) (is_free no_tool) (is_equippable parallel_box1) (is_equippable parallel_box2) (is_equippable inward_claw) (is_equippable outward_claw) (is_equippable no_tool)) - (:goal (and (is_inserted_to gear1 shaft1))) -) -- Output: -``` -{"task_plan": { - "action_sequence": [ - "load_tool(left_hand, parallel_box1)", - "pick_up(left_hand, parallel_box1, gear1)", - "insert(left_hand, parallel_box1, gear1, shaft1)" - ], - "action_instructions": [ - "equip the parallel_box1 on the left hand", - "pick up the gear1 with the parallel_box1 in the left hand", - "insert the gear1 into the shaft1 using the parallel_box1 in the left hand" - ], - "object_name": ["left_hand", "gear1", "parallel_box1", "shaft1"], - "behavior_tree": { - "summary": "selector to insert gear1 into shaft1", - "children": [ - { - "summary": "condition node to check if gear1 is inserted to shaft1", - }, - { - "summary": "sequence to insert gear1 into shaft1", - "children": [ - { - "summary": "selector to load_tool - "children": [ - { - "summary": "condition node to check if parallel_box1 is hold", - }, - { - "summary": "sequence to load_tool", - "children": [ - { - "summary": "condition node to check if parallel_box1 is equippable", - }, - { - "summary": "condition node to check if left hand is free", - }, - { - "summary": "action node to equip parallel_box1 to left hand", - "effects": [ - {"summary": "left_hand will be not free",}, - {"summary": "parallel_box1 will be not equippable",}, - {"summary": "left_hand will hold parallel_box1",}, - ], - }, - ], - }, - ], - }, - { - "summary": "selector to pick_up", - "children": [ - { - "summary": "condition node to check if parallel_box1 holds gear1", - }, - { - "summary": "sequence to pick_up", - "children": [ - { - "summary": "condition node to check if parallel_box1 is free", - }, - { - "summary": "condition node to check if left hand holds parallel_box1", - }, - { - "summary": "action node to pick up gear1 using parallel_box1", - "effects": [ - {"summary": "parallel_box1 will hold gear1",}, - {"summary": "parallel_box1 will be not free",}, - ], - }, - ], - }, - ], - }, - { - "summary": "condition node to check if gear1 can be inserted to shaft1", - }, - { - "summary": "action node to insert gear1 to shaft1", - "effects": [ - {"summary": "parallel_box1 will be not holding gear1",}, - {"summary": "parallel_box1 will be free",}, - {"summary": "gear1 will be inserted to shaft1",}, - ], - }, - ], - }, - ], -} -}, -"initial_state": {"objects": [ - {"name": "gear1", "properties": []}, - {"name": "shaft1", "properties": []}, - {"name": "parallel_box1", "properties": ["is_equippable", "is_free"]}, - {"name": "left_hand", "properties": ["is_free"]}, - ], - "constraints": [ - {"source": "gear1", "name": "can_insert_to", "target": "shaft1"}, - {"source": "parallel_box1", "name": "can_manipulate", "target": "gear1"}, - ], - "relations": [], -}, -"instruction_summary": "insert gear1 into shaft1", -"question":""} -``` -""" -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/data/prompts/cot_sp_skeleton/prompt/cot_output_format.txt b/data/prompts/cot_sp_skeleton/prompt/cot_output_format.txt deleted file mode 100644 index d47c6af5..00000000 --- a/data/prompts/cot_sp_skeleton/prompt/cot_output_format.txt +++ /dev/null @@ -1,21 +0,0 @@ -[user] -You will be provided with a problem and the necessary domain knowledge. -You generate an action sequence to solve the task and contruct a behavior tree based on it in json format. -The output dictionary has five keys. -""" -- dictionary["task_plan"]: A dictionary containing the task plan. -- dictionary["initial_state"]: The initial world state before executing the plan. -- dictionary["instruction_summary"]: A brief summary of the task plan. -- dictionary["question"]: If you don't have enough knowledge for solving the problem, you can ask questions about the knowledge you need. Leave this key empty if you can solve the task. -""" -Four keys exist in dictionary["task_plan"]. -""" -- dictionary["task_plan"]["action_sequence"]: Contains a list of robot actions. Only the actions defined in the "ROBOT_ACTION LIST" will be used. -- dictionary["task_plan"]["action_instructions"]: contains a list of instructions corresponding to dictionary["task_plan"]["action_sequence"] to explain the actions. -- dictionary["task_plan"]["manipulated_objects"]: The list of the manipulated objects. It only includes the objects that are related to the current task. -- dictionary["task_plan"]["behavior_tree"]: contains a dictionary of the behavior tree constructed according to the dictionary ["task_plan"]["action_sequence"] -""" -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/data/prompts/cot_sp_skeleton/prompt/cot_problem.txt b/data/prompts/cot_sp_skeleton/prompt/cot_problem.txt deleted file mode 100644 index 658940e7..00000000 --- a/data/prompts/cot_sp_skeleton/prompt/cot_problem.txt +++ /dev/null @@ -1,27 +0,0 @@ -[user] -Information about the problem is given as a pddl problem. -It includes the following information: -- objects: all the objects in the world. They should be of a type defined in the domain knowledge. For example, "parallel_box1" is an instance of the type "tool", while "gear1" is an instance of the type "part". -- init: the initial world state to start with, described with [PREDICATE_LIST] from domain knowledge. For example, "is_free hand" means the object "hand" is free and holds nothing. -- goal: the world state to achieve, described with [PREDICATE_LIST] from domain knowledge. -An example: -""" -(define (problem robot_assembly_problem-problem) - (:domain robot_assembly_problem-domain) - (:objects - parallel_box1 parallel_box2 inward_claw outward_claw no_tool - tool - gear1 gear2 gear3 shaft1 shaft2 base - part - left_hand - hand - ) - (:init (can_manipulate parallel_box1 gear1) (can_manipulate outward_claw gear2) (can_manipulate inward_claw gear3) (can_manipulate parallel_box2 shaft1) (can_manipulate no_tool shaft2) (can_screw_to leg1 seat) (can_screw_to leg2 seat) (can_insert_to back seat) (can_screw_to nut1 seat) (can_screw_to nut2 seat) (can_screw_to blub base) (can_place_to lamp blub) (can_insert_to shaft1 base) (can_insert_to shaft2 base) (can_insert_to gear3 shaft2) (can_insert_to gear2 base) (can_insert_to gear1 shaft1) (is_free left_hand) (is_free parallel_box1) (is_free parallel_box2) (is_free inward_claw) (is_free outward_claw) (is_free no_tool) (is_equippable parallel_box1) (is_equippable parallel_box2) (is_equippable inward_claw) (is_equippable outward_claw) (is_equippable no_tool)) - (:goal (and (is_inserted_to shaft1 base))) -) -""" - - - - -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/data/prompts/cot_sp_skeleton/prompt/cot_role.txt b/data/prompts/cot_sp_skeleton/prompt/cot_role.txt deleted file mode 100644 index 6ce736a0..00000000 --- a/data/prompts/cot_sp_skeleton/prompt/cot_role.txt +++ /dev/null @@ -1,5 +0,0 @@ -[user] -You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you break it down and construct robot task plans represented by behaiovr trees. -Please do not begin working until I say "Start working." Instead, simply output the message "Waiting for next input." Understood? -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/data/prompts/cot_sp_skeleton/prompt/cot_state.txt b/data/prompts/cot_sp_skeleton/prompt/cot_state.txt deleted file mode 100644 index a8ea9cd6..00000000 --- a/data/prompts/cot_sp_skeleton/prompt/cot_state.txt +++ /dev/null @@ -1,30 +0,0 @@ -[user] -Information about world states should be in json dictionary format. Example: -""" -{ - "objects": [ - {"name": "leg1", "properties": []}, # leg1 is an object with no properties - {"name": "seat", "properties": []}, # seat is an object with no properties - {"name": "cabinet", "properties": ["open"]}, # cabinet is an object with a property "open" - {"name": "table", "properties": []}, # table is an object with no properties - ], - "constraints": [ - {"source": "leg1", "name": "can_screw_to", "target": "seat"}, # means leg1 can be screwed to seat - {"source": "cabinet", "name": "on_something", "target": "table"}, # means cabinet is on table - ], - "relations": [ - {"source": "leg1", "name": "is_screwed_to", "target": "seat"}, # means leg1 is screwed to seat - ], -} -""" -It contains three lists: - - "objects": the objects and their properties (as lists of strings) in the world. The properties can be changed by robot actions. - - "constraints": the constraint relations that are enforced and should not be changed. They are knowledge about the world or some constraints set by the user. - - "relations": the relations in the world. They can be changed by robot actions. -The properties, constraints and relations should always align with the [PREDICATE_LIST] in domain knowledge. -Only concentrate on the objects, constraints and relations that are related to the current task. - -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/data/prompts/cot_sp_skeleton/query/old_query.txt b/data/prompts/cot_sp_skeleton/query/old_query.txt deleted file mode 100644 index d9ff871a..00000000 --- a/data/prompts/cot_sp_skeleton/query/old_query.txt +++ /dev/null @@ -1,16 +0,0 @@ -Start working. Resume from the problem below. -""" -{"problem": [PROBLEM]} -""" -The dictionary that you return should be formatted as json file. Follow these rules: -1. Always equip the proper tools before performing specific object manipulaion. -2. Make sure that each element of the ["action_instructions"] explains corresponding element of the ["action_sequence"]. Refer to the "ROBOT ACTION LIST" to understand the elements of ["action_sequence"]. -3, The length of the ["action_instructions"] list and the ["action_sequence"] list should be the same. -4. Never left ',' at the end of the list. Use json boolean values, "true" or "false", instead of "True" or "False". -5. Keep track of all items listed in the "objects" section of the "initial_state" field. Please make sure you fill out the "objects" section for all listed items. -6. Use the "PREDICATE_LIST" for the initial states (:init) in the problem to generate ["initial_state"]. -7. All keys of the dictionary should be double-quoted. -8. Insert ``` at the beginning and the end of the dictionary to separate it from the rest of your response. Don't write any additional things. -9. Make sure that you output a consistent manipultation as a single arm robot. For example, grasping an object should not occur in successive steps. -10. When generating the behavior tree, make sure the tree has the asked structure. -Adhere to the output format I defined above. Follow the rules above. Think step by step. \ No newline at end of file diff --git a/data/prompts/cot_sp_skeleton/query/query.txt b/data/prompts/cot_sp_skeleton/query/query.txt deleted file mode 100644 index cfeb4e8e..00000000 --- a/data/prompts/cot_sp_skeleton/query/query.txt +++ /dev/null @@ -1,16 +0,0 @@ -Start working. Resume from the problem below. -""" -{"problem": [PROBLEM]} -""" -The dictionary that you return should be formatted as json file. Follow these rules: -1. Always equip the proper tools before performing specific object manipulaion. -2. Make sure that each element of the ["action_instructions"] explains corresponding element of the ["action_sequence"]. Refer to the "ROBOT ACTION LIST" to understand the elements of ["action_sequence"]. -3, The length of the ["action_instructions"] list and the ["action_sequence"] list should be the same. -4. Never left ',' at the end of the list. Use json boolean values, "true" or "false", instead of "True" or "False". -5. Keep track of all items listed in the "objects" section of the "initial_state" field. -6. Use the "PREDICATE_LIST" for the initial states (:init) in the problem to generate ["initial_state"]. -7. All keys of the dictionary should be double-quoted. -8. Insert ``` at the beginning and the end of the dictionary to separate it from the rest of your response. Don't write any additional things. -9. Make sure that you output a consistent manipultation as a single arm robot. For example, grasping an object should not occur in successive steps. -10. When generating the behavior tree, make sure the tree has the asked structure. -Adhere to the output format I defined above. Follow the rules above. Think step by step. \ No newline at end of file diff --git a/data/prompts/cot_sp_skeleton/system/system.txt b/data/prompts/cot_sp_skeleton/system/system.txt deleted file mode 100644 index 26f2afe2..00000000 --- a/data/prompts/cot_sp_skeleton/system/system.txt +++ /dev/null @@ -1 +0,0 @@ -You are an excellent interpreter of instructions for robotic assembly tasks. Given the knowledge you need, the world states and some instructions, you break it down and construct robot task plans represented by behaiovr trees based on the action sequences. \ No newline at end of file diff --git a/data/prompts/end_to_end_v1/prompt/backup b/data/prompts/end_to_end_v1/prompt/backup deleted file mode 100644 index c4a0f6f6..00000000 --- a/data/prompts/end_to_end_v1/prompt/backup +++ /dev/null @@ -1,43 +0,0 @@ - -There are also some counterexamples for you to refer to: -1. One sequence should never have multiple action nodes as its children like the behavior tree below. -{ - "summary": "selector to get the apple", - "name": "Get the apple", - "type_name": "selector", - "children": [ - { - "summary": "check if the apple is in hand", - "name": "target: in_hand(apple)", - "type_name": "condition", - }, - { - "summary": "sequence to get the apple", - "name": "sequence: get apple", - "type_name": "sequence", - "children": [ - { - "summary": "check if the apple is on the table", - "name": "precondition: on(apple, table)", - "type_name": "condition", - }, - { - "summary": "go to the table", - "name": "action: go_to(table)", - "type_name": "action", - }, - { - "summary": "check if the hand is free", - "name": "precondition: is_free(hand)", - "type_name": "condition", - }, - { - "summary": "pick up the apple", - "name": "action: pick_up(apple)", - "type_name": "action", - (effect omitted) - }, - ], - }, - ], -} \ No newline at end of file diff --git a/data/prompts/end_to_end_v1/prompt/old_prompt_example.txt b/data/prompts/end_to_end_v1/prompt/old_prompt_example.txt deleted file mode 100644 index 8d71b9b6..00000000 --- a/data/prompts/end_to_end_v1/prompt/old_prompt_example.txt +++ /dev/null @@ -1,232 +0,0 @@ -[user] -I will give you some examples of the input and the output you will generate. -Example 1: -""" -- Input: -{"assets":["", "", "", "", ""], -"asset_states": {"": "on_something(
)", - "": "on_something()"}, -"objects": ["", ""], -"object_states": {"": "on_something(
)", - "": "on_something()"}, -"instruction": "Put the juice on top of the shelf"} -- Output: -``` -{"task_cohesion": { - "task_sequence": [ - "move_hand()", - "grasp_object()", - "detach_from_plane()", - "move_object()", - "move_object()", - "move_object()", - "attach_to_plane()", - "release_object()" - ], - "step_instructions": [ - "move the hand near the juice", - "grasp the juice", - "pick up the juice", - "take the juice from the bottom of the shelf", - "move the juice upward", - "move the juice above the upper shelf", - "place the juice", - "release the juice" - ], - "object_name": ""}, -"environment_before": {"assets":["
", "", "", "", ""], - "asset_states": {"": "on_something(
)", - "": "on_something()"}, - "objects": ["", ""], - "object_states": {"": "on_something(
)", - "": "on_something()"}}, -"environment_after": {"assets":["
", "", "", "", ""], - "asset_states": {"": "on_something(
)", - "": "on_something()"}, - "objects": ["", ""], - "object_states": {"": "on_something(
)", - "": "on_something()"}}, -"instruction_summary": "put the juice on top of the shelf", -"question":""} -``` -""" -Example 2: -""" -- Input: -{"assets":["
", "", "", "", ""], -"asset_states": {"": "on_something(
)", - "": "on_something()"}, -"objects": ["", ""], -"object_states": {"": "on_something(
)", - "": "on_something()"}, -"instruction": "Throw away the spam into the trash bin."} -- Output: -``` -{"task_cohesion": { - "task_sequence": [ - "move_hand()", - "grasp_object()", - "detach_from_plane()", - "move_object()", - "release_object()" - ], - "step_instructions": [ - "move your hand near the spam", - "grasp the spam", - "pick up the spam", - "bring the spam above the trash bin", - "release the spam to drop it in the trash bin" - ], - "object_name": ""}, -"environment_before": {"assets":["
", "", "", "", ""], - "asset_states": {"": "on_something(
)", - "": "on_something()"}, - "objects": ["", ""], - "object_states": {"": "on_something(
)", - "": "on_something()"}}, -"environment_after": {"assets":["
", "", "", "", ""], - "asset_states": {"": "on_something(
)", - "": "on_something()"}, - "objects": ["", ""], - "object_states": {"": "inside_something()", - "": "on_something()"}}, -"instruction_summary": "throw away the spam", -"question":""} -``` -""" -Example 3: -""" --Input: -{"assets":["", ""], -"asset_states": {"": "on_something()"}, -"objects": [""], -"object_states": {"": "closed()"}, -"instruction": "Open the fridge."} -- Output: -``` -{"task_cohesion": { - "task_sequence": [ - "move_hand()", - "grasp_object()", - "open_by_rotate()", - "release_object()" - ], - "step_instructions": [ - "move the hand near the fridge handle", - "grasp the handle", - "open the fridge by pulling the handle", - "release the handle" - ], - "object_name": ""}, -"environment_before": {"assets":["", ""], - "asset_states": {"": "on_something()"}, - "objects": [""], - "object_states": {"": "closed()"}}, -"environment_after": {"assets":["", ""], - "asset_states": {"": "on_something()"}, - "objects": [""], - "object_states": {"": "open()"}}, -"instruction_summary": "open a fridge", -"question":""} -``` -""" -Example 4: -""" --Input: -{"assets":["
", ""], -"asset_states": {"
": "next_to()"}, -"objects": [""], -"object_states": {"": "on_something(
)"}, -"instruction": "Wipe the window with the sponge. Then, put the sponge on the table."} -- Output: -``` -{"task_cohesion": { - "task_sequence": [ - "move_hand()", - "grasp_object()", - "detach_from_plane()", - "move_object()", - "attach_to_plane()", - "wipe_on_plane()", - "detach_from_plane()", - "move_object()", - "attach_to_plane()", - "release_object()" - ], - "step_instructions": [ - "move the hand near the sponge", - "grasp the sponge", - "pick up the sponge", - "bring the sponge above the window", - "attach the sponge to the window", - "wipe the window with the sponge", - "detach the sponge from the window", - "bring the sponge above the table", - "put down the sponge", - "release the sponge" - ], - "object_name": ""}, -"environment_before": {"assets":["
", ""], - "asset_states": {"
": "next_to()"}, - "objects": [""], - "object_states": {"": "on_something(
)"},} -"environment_after": {"assets":["
", ""], - "asset_states": {"
": "next_to()"}, - "objects": [""], - "object_states": {"": "on_something(
)"}}, -"instruction_summary": "wipe the window with the sponge, then put the sponge on the table", -"question":""} -``` -""" -Example 5: -""" --Input: -{"assets":["
"], -"asset_states": {"": "can_manipulate()"}, -"objects": ["", "", ""], -"object_states": {"": "on_something(
)", "": "on_something(
), "": "can_manipulate()"}, -"instruction": "screw the nut into the bolt. Tool should be used and be put back at the end."} -- Output: -``` -{"task_cohesion": { - "task_sequence": [ - "load_tool()", - "approach()", - "grasp_object()", - "pick_up()", - "approach()", - "insert()", - "screw()", - "release_object()", - "unload_tool()" - ], - "step_instructions": [ - "load the parallel_toolbox", - "move the hand close to the nut", - "grasp the nut", - "pick up the nut", - "move the hand close to the bolt", - "insert the nut onto the bolt", - "screw the nut into the bold", - "release the nut", - "unload the parallel_toolbox" - ], - "object_name": ""}, -"environment_before": {"assets":["
"], - "asset_states": {"": "can_manipulate()"}, - "objects": ["", "", ""], - "object_states": {"": "on_something(
)", "": "on_something(
)"},} -"environment_after": {"assets":["
"], - "asset_states": {"": "can_manipulate()"}, - "objects": ["", "", ""], - "object_states": {"": "screwed_into()", "": "on_something(
)"}, -"instruction_summary": "screw the nut into the bolt. Tool should be used and be put back at the end.", -"question":""} -} -``` -""" -From these examples, learn that some robotic actions have dependencies with the actions before and after them. -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/data/prompts/end_to_end_v1/prompt/old_prompt_output_format.txt b/data/prompts/end_to_end_v1/prompt/old_prompt_output_format.txt deleted file mode 100644 index 7be3d38b..00000000 --- a/data/prompts/end_to_end_v1/prompt/old_prompt_output_format.txt +++ /dev/null @@ -1,20 +0,0 @@ -[user] -You divide the actions given in the text into detailed robot actions and put them together as a python dictionary. -The dictionary has five keys. -""" -- dictionary["task_cohesion"]: A dictionary containing information about the robot's actions that have been split up. -- dictionary["environment_before"]: The state of the environment before the manipulation. -- dictionary["environment_after"]: The state of the environment after the manipulation. -- dictionary["instruction_summary"]: contains a brief summary of the given sentence. -- dictionary["question"]: If you cannot understand the given sentence, you can ask the user to rephrase the sentence. Leave this key empty if you can understand the given sentence. -""" -Three keys exist in dictionary["task_cohesion"]. -""" -- dictionary["task_cohesion"]["task_sequence"]: Contains a list of robot actions. Only the behaviors defined in the "ROBOT ACTION LIST" will be used. -- dictionary["task_cohesion"]["step_instructions"]: contains a list of instructions corresponding to dictionary["task_cohesion"]["task_sequence"]. -- dictionary["task_cohesion"]["object_name"]: The name of the manipulated object. Only objects defined in the input dictionary will be used for the object name. -""" -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/data/prompts/end_to_end_v1/prompt/oldprompt_behaviortree.txt b/data/prompts/end_to_end_v1/prompt/oldprompt_behaviortree.txt deleted file mode 100644 index 585ecc2f..00000000 --- a/data/prompts/end_to_end_v1/prompt/oldprompt_behaviortree.txt +++ /dev/null @@ -1,170 +0,0 @@ -[user] -The behavior tree should be in python dictionary format and have a structure like the following example: -""" -{ - "name": "Pick Up Apple", - "identifier": 0, - "type_name": "selector", - "children": [ - { - "summary": "check if the apple is in hand", - "name": "check apple in hand", - "identifier": 1, - "type_name": "condition", - "conditions": [ - { - "object_name": "apple", - "property_name": "in", - "property_value": "hand", - "status": True, - } - ], - }, - { - "name": "Pick Up Sequence", - "identifier": 2, - "type_name": "sequence", - "children": [ - { - "summary": "check if the apple is on the ground", - "name": "check apple on the ground", - "identifier": 3, - "type_name": "condition", - "conditions": [ - { - "object_name": "apple", - "property_name": "on_the_ground", - "property_value": None, - "status": True, - } - ], - }, - { - "summary": "check if the hand is free", - "name": "check hand free", - "identifier": 4, - "type_name": "condition", - "conditions": [ - { - "object_name": "hand", - "property_name": "free", - "property_value": None, - "status": True, - } - ], - }, - { - "summary": "pick up the apple", - "name": "pick_up(apple)", - "identifier": 5, - "type_name": "action", - "effects": [ - { - "object_name": "apple", - "property_name": "on_the_ground", - "property_value": None, - "status": False, - }, - { - "object_name": "apple", - "property_name": "in", - "property_value": "hand", - "status": True, - }, - { - "object_name": "hand", - "property_name": "free", - "property_value": None, - "status": False, - }, - ], - }, - ], - }, - ], -} -""" -The dictionary that you return should be formatted as python dictionary. Follow these rules: -1. Every node should have three basic keys: - - "name": the name of the node (for action node, the name should be the same as defined in the domain knowledge) - - "identifier": the identifier of the node, which should be unique - - "type_name": the type of the node, its value can be: "selector", "sequence", "condition", "action" -2. The "selector" and "sequence" nodes can (and should) have a key "children" with a list of children nodes. -3. The "action" and "condition" nodes should have a key "summary" with a short description of the action/condition. -4. The "condition" nodes should have a key "conditions" with a list of conditions to be checked. Each condition should have the following keys: - - "object_name": the object to check - - "property_name": the property/relation of the object to be checked - - "property_value": this should be the other object if this is a relation. For property it should be None - - "status": the boolean status of the property/relation the condition expects, True if the property/relation should be True, False if the property/relation should be False -5. The "action" nodes should have a key "effects" with a list of effects to be exerted after the execution of the action. Each effect should have the following keys: - - "object_name": the object to be affected - - "property_name": the property/relation of the object to be affected - - "property_value": this should be the other object if this is a relation. For property it should be None - - "status": True if the effect is to add this property/relation. False if is to remove. -6. The behavior tree is used to represent an action in the action sequence. Its basic structure is: - - A selector as the root node. - - (Usually one) (Goal-)Condition node as the first child of the selector. It should check the goal conditions the action intends to achieve. So if the goal condition is met, the selector will directly return success. - - A sequence node as the last child of the selector node. The aim of the sequence is to achieve the goal conditions. - - (Pre-)Condition nodes as the front children of the sequence node. They should check the preconditions of the action, which is the last child node of the sequence node. So if any of the preconditions is not met, the sequence node will return failure. - - Only one action node as the last child of the sequence node. Its preconditions are checked by the condition nodes in the front of it. Its effects are exerted after the execution of the action, and should be able to satisfy the goal condition(s), which is the first child of the selector node. -7. When there is any (Pre-)condition node that is not satisfied, a behavior tree can be constructed according to the rules above, which replaces the unsatisfied (Pre-)condition, takes that (Pre-)condition as the (Goal-)condition to achieve and includes the action to achieve it. -8. The selector can have multiple (goal-)condition nodes as children, but must have only and no more than one sequence node as the last child. -9. The sequence node can have multiple (pre-)condition nodes (or selectors) as children, but must have only and no more than one action node as the last child. -10. The behavior tree should be constructed based on the action sequence you planned and the action definitions from the domain knowledge. The depth of the behavior tree is not limited, and when being expanded, it should expanded in a vertical way instead of inserting new actions into the old sequence. - -There are also some counterexamples for you to refer to: -1. One sequence should never have multiple action nodes as its children like the behavior tree below. -{ - "name": "Get the apple", - "identifier": 0, - "type_name": "selector", - "children": [ - { - "summary": "check if the apple is in hand", - "name": "check apple in hand", - "identifier": 1, - "type_name": "condition", - (condition omitted) - }, - { - "name": "Pick Up Sequence", - "identifier": 2, - "type_name": "sequence", - "children": [ - { - "summary": "check if the apple is on the table", - "name": "check apple on the table", - "identifier": 3, - "type_name": "condition", - (condition omitted) - }, - { - "summary": "go to the table", - "name": "go_to(table)", - "identifier": 4, - "type_name": "action", - (effect omitted) - }, - { - "summary": "check if the hand is free", - "name": "check hand free", - "identifier": 5, - "type_name": "condition", - (condition omitted) - }, - { - "summary": "pick up the apple", - "name": "pick_up(apple)", - "identifier": 6, - "type_name": "action", - (effect omitted) - }, - ], - }, - ], -} - -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/data/prompts/end_to_end_v1/prompt/oldprompt_environment.txt b/data/prompts/end_to_end_v1/prompt/oldprompt_environment.txt deleted file mode 100644 index ac8293ab..00000000 --- a/data/prompts/end_to_end_v1/prompt/oldprompt_environment.txt +++ /dev/null @@ -1,28 +0,0 @@ -[user] -Information about world states are given as python dictionary. Example: -""" -{"environment":{"assets":["
", "", "", "", ""], - "asset_states": {"": "on_something(
)", - "": "on_something()"}, - "objects": ["", ""], - "object_states": {"": "on_something(
)", - "": "on_something()"}}} -""" -Asset states and object states are represented using those state sets: -""" -"STATE LIST" -- on_something(): Object is located on -- inside_something(): Object is located inside -- inside_hand(): Object is being grasped by a robot hand -- closed(): Object can be opened -- open(): Object can be closed or kept opened -- can_manipulate(): Object can be used to grasp and manipulate -- equipped(): Object is being equipped by the robot hand. -- screwed_into(): Object is screwed into by robot hand -- inserted_into(): Object is inserted into by robot hand -""" - should be one of the assets or objects in the environment. -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/data/prompts/end_to_end_v1/prompt/prompt_behaviortree.txt b/data/prompts/end_to_end_v1/prompt/prompt_behaviortree.txt deleted file mode 100644 index 6a88afae..00000000 --- a/data/prompts/end_to_end_v1/prompt/prompt_behaviortree.txt +++ /dev/null @@ -1,189 +0,0 @@ -[user] -The behavior tree should be in json dictionary format and have a structure like the following example: -""" -{ - "name": "load_tool selector", - "identifier": 3, - "type_name": "selector", - "children": [ - { - "summary": "check if inward_claw is held by left hand", - "name": "check left_hand hold inward_claw", - "identifier": 4, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "inward_claw", - "status": True, - } - ], - }, - { - "name": "load_tool sequence", - "identifier": 5, - "type_name": "sequence", - "children": [ - { - "summary": "check if inward_claw is equippable", - "name": "check inward_claw is_equippable", - "identifier": 6, - "type_name": "condition", - "conditions": [ - { - "object_name": "inward_claw", - "property_name": "is_equippable", - "property_value": None, - "status": True, - } - ], - }, - { - "summary": "check if left hand is free", - "name": "check left_hand is_free", - "identifier": 7, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "is_free", - "property_value": None, - "status": True, - } - ], - }, - { - "summary": "equip inward_claw to left hand", - "name": "load_tool(left_hand, inward_claw)", - "identifier": 8, - "type_name": "action", - "effects": [ - { - "object_name": "left_hand", - "property_name": "is_free", - "property_value": None, - "status": False, - }, - { - "object_name": "inward_claw", - "property_name": "is_equippable", - "property_value": None, - "status": False, - }, - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "inward_claw", - "status": True, - }, - ], - }, - ], - }, - ], -} -""" -The dictionary that you return should be formatted as json dictionary. Follow these rules: -1. Every node should have three basic keys: - - "name": the name of the node (for action node, the name should be the same as defined in the domain knowledge) - - "identifier": the unique id of the node - - "type_name": node_type, can be: "selector", "sequence", "condition", "action" -2. The control flow node, "selector" and "sequence" nodes, can have "children", which is a list of children nodes. -3. The "action" and "condition" nodes should have "summary", which is a short description of the action/condition. -4. The "condition" nodes should have "conditions", which is a list of conditions to be checked. Each condition should have the following keys: - - "object_name": the object to check - - "property_name": the property/relation of the object to be checked - - "property_value": this should be the other object if this is a relation. For property it should be None - - "status": the boolean status of the property/relation the condition expects, True if the property/relation should be True, False otherwise -5. The "action" nodes should have a key "effects" with a list of effects to be exerted after the execution of the action. Each effect should have the following keys: - - "object_name": the affected object - - "property_name": the affected property/relation of the object - - "property_value": this should be the other object if this is a relation. For property it should be None - - "status": True if the effect is to add this property/relation. False if is to remove. -6. All the subtrees always have their corresponding action in ROBOT_ACTION_LIST. They are used to represent the actions in the action sequence. Its basic structure is: - - A selector as the root node. - - (Usually one) (Goal-)Condition node as the first child of the selector. It should check the goal conditions the action intends to achieve. So if the goal condition is met, the selector will directly return success. - - A sequence node as the last child of the selector node. The aim of the sequence is to achieve the goal conditions. - - (Pre-)Condition nodes as the front children of the sequence node. They should check the preconditions of the action node after them. - - Only one action node as the last child of the sequence node. Its preconditions are checked by the condition nodes in the front of it. Its effects are exerted after the execution of the action, and should be able to satisfy the goal condition(s). - A brief example: -{ - selector: load_tool selector, - "children": [ - { - (goal)condition: check if inward_claw is equipped by left hand, - }, - { - sequence: load_tool sequence, - children: [ - (pre)condition: check if inward_claw is equippable, (if not satisfied, this condition node can be expanded. A new subtree can be constructed to satisfy this) - (pre)condition: check if left hand is free, (if not satisfied, this condition node can be expanded. A new subtree can be constructed to satisfy this) - (more preconditions can be added here), - action: equip inward_claw to left hand, - ], - }, - ], -} - -7. When there is any (Pre-)condition node that is not satisfied, a behavior tree can be constructed according to the rules above, which replaces the unsatisfied (Pre-)condition, takes that (Pre-)condition as the (Goal-)condition to achieve and includes the action to achieve it. -8. The selector can have multiple (goal-)condition nodes as children, but must have only and no more than one sequence node as the last child. -9. The sequence node can have multiple (pre-)condition nodes (or selectors) as children, but must have only and no more than one action node as the last child. -10. The behavior tree should be constructed based on the action sequence you planned and the action definitions in ROBOT_ACTION_LIST. - -There are also some counterexamples for you to refer to: -1. One sequence should never have multiple action nodes as its children like the behavior tree below. -{ - "name": "Get the apple", - "identifier": 0, - "type_name": "selector", - "children": [ - { - "summary": "check if the apple is in hand", - "name": "check apple in hand", - "identifier": 1, - "type_name": "condition", - (condition omitted) - }, - { - "name": "Pick Up Sequence", - "identifier": 2, - "type_name": "sequence", - "children": [ - { - "summary": "check if the apple is on the table", - "name": "check apple on the table", - "identifier": 3, - "type_name": "condition", - (condition omitted) - }, - { - "summary": "go to the table", - "name": "go_to(table)", - "identifier": 4, - "type_name": "action", - (effect omitted) - }, - { - "summary": "check if the hand is free", - "name": "check hand free", - "identifier": 5, - "type_name": "condition", - (condition omitted) - }, - { - "summary": "pick up the apple", - "name": "pick_up(apple)", - "identifier": 6, - "type_name": "action", - (effect omitted) - }, - ], - }, - ], -} - -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/data/prompts/end_to_end_v1/prompt/prompt_bt_skeleton.txt b/data/prompts/end_to_end_v1/prompt/prompt_bt_skeleton.txt deleted file mode 100644 index b1225fcc..00000000 --- a/data/prompts/end_to_end_v1/prompt/prompt_bt_skeleton.txt +++ /dev/null @@ -1,83 +0,0 @@ -[user] -The behavior tree should be in json dictionary format and have a structure like the following example: -""" -{ - "summary": "selector to load inward_claw to left hand", - "name": "selector: load_tool", - "type_name": "selector", - "children": [ - { - "summary": "check if inward_claw is held by left hand", - "name": "target: hold(left_hand, inward_claw)", - "type_name": "condition", - }, - { - "summary": "sequence to load inward_claw to left hand", - "name": "sequence: load_tool", - "type_name": "sequence", - "children": [ - { - "summary": "check if inward_claw is equippable", - "name": "precondition: is_equippable(inward_claw)", - "type_name": "condition", - }, - { - "summary": "check if left hand is free", - "name": "precondition: is_free(left_hand)", - "type_name": "condition", - }, - { - "summary": "equip inward_claw to left hand", - "name": "action: load_tool(left_hand, inward_claw)", - "type_name": "action", - "effects": [ - {"summary": "left hand will be not free anymore",}, - {"summary": "inward_claw will be not equippable anymore",}, - {"summary": "left_hand will hold inward_claw",}, - ], - }, - ], - }, - ], -} -""" -The dictionary that you return should be formatted as json dictionary. Follow these rules: -1. Every node should have three basic keys: - - "summary": a short interpretive description of the node - - "name": the name of the node. The name should follow the definitions in the domain knowledge. - - "type_name": can be: "selector", "sequence", "condition", "action" -2. The control flow nodes, "selector" and "sequence", can have "children", which is a list of children nodes. -3. The "condition" nodes that are children of "selectors" are "targets". Those that are children of "sequences" are preconditions. -4. The "action" nodes should have a key "effects" with a list of dictionaries, which describes the to-be-exerted effects of the action. The effects are shortly described under the key "summary". -5. All the subtrees always have their corresponding action in ROBOT_ACTION_LIST. They are used to represent the actions in the action sequence. The recommended structure is: - - A selector as the root node. - - (Usually one) target condition node as the first child of the selector to check if the target of the action has been fulfilled. - - A sequence node as the last child of the selector node to achieve the target. - - (Usually a number of) precondition nodes as the front children of the sequence node to check the preconditions for executing the action. - - One action node as the last child of the sequence node, whose preconditions are checked before and will immediate start the job when being ticked. - - The action will exert its defined effects after finishing the job successfully. Those effects should fulfilled all the target conditions in the selector node. - A brief example is as follows: -{ - selector: load_tool selector, - children: [ - { - target: check if inward_claw is equipped by left hand, - }, - { - sequence: load_tool, - children: [ - precondition: check if inward_claw is equippable, - precondition: check if left hand is free, - {action: equip inward_claw to left hand, effects: [left hand not free, inward_claw not equippable, left_hand holds inward_claw]}, - ], - }, - ], -} - -6. When performing a new action to fulfill a unsatisfied precondition, you can construct a subtree to replace the precondition and take that precondition as the target of the subtree. Then you construct the sequence of the subtree according to the new action you want to perform. -7. All of the control flow nodes, namely selectors and sequences, are without memory. A new behavior tree tick will always start from the root and traverse all the nodes even if they are ticked in the last tick. - -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/data/prompts/end_to_end_v1/prompt/prompt_bt_skeleton_example.txt b/data/prompts/end_to_end_v1/prompt/prompt_bt_skeleton_example.txt deleted file mode 100644 index e84cdfb9..00000000 --- a/data/prompts/end_to_end_v1/prompt/prompt_bt_skeleton_example.txt +++ /dev/null @@ -1,195 +0,0 @@ -[user] -I will give you some examples of the input and the output you will generate. -Example 1: -""" -- Input: -(define (problem robot_assembly_problem-problem) - (:domain robot_assembly_problem-domain) - (:objects - parallel_box1 parallel_box2 inward_claw outward_claw no_tool - tool - gear1 gear2 gear3 shaft1 shaft2 base - part - left_hand - hand - ) - (:init (can_manipulate parallel_box1 gear1) (can_manipulate outward_claw gear2) (can_manipulate inward_claw gear3) (can_manipulate parallel_box2 shaft1) (can_manipulate no_tool shaft2) (can_screw_to leg1 seat) (can_screw_to leg2 seat) (can_insert_to back seat) (can_screw_to nut1 seat) (can_screw_to nut2 seat) (can_screw_to blub base) (can_place_to lamp blub) (can_insert_to shaft1 base) (can_insert_to shaft2 base) (can_insert_to gear3 shaft2) (can_insert_to gear2 base) (can_insert_to gear1 shaft1) (is_inserted_to shaft1 base) (is_free left_hand) (is_free parallel_box1) (is_free parallel_box2) (is_free inward_claw) (is_free outward_claw) (is_free no_tool) (is_free gear1) (is_free gear2) (is_free gear3) (is_free shaft1) (is_free shaft2) (is_free base) (is_equippable parallel_box1) (is_equippable parallel_box2) (is_equippable inward_claw) (is_equippable outward_claw) (is_equippable no_tool)) - (:goal (and (is_inserted_to gear1 shaft1))) -) -- Output: -``` -{"task_plan": { - "action_sequence": [ - "load_tool(left_hand, parallel_box1)", - "pick_up(left_hand, parallel_box1, gear1)", - "insert(left_hand, parallel_box1, gear1, shaft1)" - ], - "action_instructions": [ - "equip the parallel_box1 on the left hand", - "pick up the gear1 with the left hand", - "insert the gear1 into the shaft1 using the left hand with the parallel_box1" - ], - "object_name": ["left_hand", "gear1", "parallel_box1", "shaft1"], - "behavior_tree": { - "summary": "selector to insert gear1 into shaft1", - "name": "selector: insert gear1 into shaft1", - "type_name": "selector", - "children": [ - { - "summary": "check if gear1 is inserted to shaft1", - "name": "target: is_inserted_to(gear1, shaft1)", - "type_name": "condition", - }, - { - "summary": "sequence to insert gear1 into shaft1", - "name": "sequence: insert(gear1, shaft1)", - "type_name": "sequence", - "children": [ - { - "summary": "selector to load_tool - "name": "load_tool selector", - "type_name": "selector", - "children": [ - { - "summary": "check if parallel_box1 is hold", - "name": "target: hold(left_hand, parallel_box1)", - "type_name": "condition", - }, - { - "summary": "sequence to load_tool", - "name": "sequence: load_tool", - "type_name": "sequence", - "children": [ - { - "summary": "check if parallel_box1 is equippable", - "name": "precondition: is_equippable(parallel_box1)", - "type_name": "condition", - }, - { - "summary": "check if left hand is free", - "name": "precondition: is_free(left_hand)", - "type_name": "condition", - }, - { - "summary": "equip parallel_box1 to left hand", - "name": "action: load_tool(left_hand, parallel_box1)", - "type_name": "action", - "effects": [ - { - "summary": "left_hand will be not free", - }, - { - "summary": "parallel_box1 will be not equippable", - }, - { - "summary": "left_hand will hold parallel_box1", - }, - ], - }, - ], - }, - ], - }, - { - "summary": "selector to pick_up", - "name": "selector: pick_up", - "type_name": "selector", - "children": [ - { - "summary": "check if parallel_box1 holds gear1", - "name": "precondition: hold(parallel_box1, gear1)", - "type_name": "condition", - }, - { - "summary": "sequence to pick_up", - "name": "sequence: pick_up", - "type_name": "sequence", - "children": [ - { - "summary": "check if parallel_box1 is free", - "name": "precondition: is_free(parallel_box1)", - "type_name": "condition", - }, - { - "summary": "check if left hand holds parallel_box1", - "name": "precondition: hold(left_hand, parallel_box1)", - "type_name": "condition", - }, - { - "summary": "pick up gear1 using parallel_box1", - "name": "action: pick_up(left_hand, parallel_box1, gear1)", - "type_name": "action", - "effects": [ - { - "summary": "parallel_box1 will hold gear1", - }, - { - "summary": "parallel_box1 will be not free", - }, - ], - }, - ], - }, - ], - }, - { - "summary": "check if gear1 can be inserted to shaft1", - "name": "precondition: can_insert_to(gear1, shaft1)", - "type_name": "condition", - }, - { - "summary": "insert gear1 to shaft1", - "name": "action: insert(left_hand, parallel_box1, gear1, shaft1)", - "type_name": "action", - "effects": [ - { - "summary": "parallel_box1 will be not holding gear1", - }, - { - "summary": "parallel_box1 will be free", - }, - { - "summary": "gear1 will be inserted to shaft1", - }, - ], - }, - ], - }, - ], -} -}, -"environment_before": {"objects": [ - {"name": "gear1", "properties": []}, - {"name": "shaft1", "properties": []}, - {"name": "parallel_box1", "properties": ["is_equippable", "is_free"]}, - {"name": "left_hand", "properties": ["is_free"]}, - ], - "constraints": [ - {"source": "gear1", "name": "can_insert_to", "target": "shaft1"}, - {"source": "parallel_box1", "name": "can_manipulate", "target": "gear1"}, - ], - "relations": [ - - ], -}, -"environment_after": {"objects": [ - {"name": "gear1", "properties": []}, - {"name": "shaft1", "properties": []}, - {"name": "parallel_box1", "properties": ["is_equippable", "is_free"]}, - {"name": "left_hand", "properties": []}, - ], - "constraints": [ - {"source": "gear1", "name": "can_insert_to", "target": "shaft1"}, - {"source": "parallel_box1", "name": "can_manipulate", "target": "gear1"}, - ], - "relations": [ - {"source": "gear1", "name": "is_inserted_to", "target": "shaft1"}, - {"source": "left_hand", "name": "hold", "target": "parallel_box1"}, - ], -}, -"instruction_summary": "insert gear1 into shaft1", -"question":""} -``` -""" -From these examples, learn that some robotic actions have dependencies with the actions before and after them. -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/data/prompts/end_to_end_v1/prompt/prompt_bt_validation.txt b/data/prompts/end_to_end_v1/prompt/prompt_bt_validation.txt deleted file mode 100644 index e69de29b..00000000 diff --git a/data/prompts/end_to_end_v1/prompt/prompt_domain.txt b/data/prompts/end_to_end_v1/prompt/prompt_domain.txt deleted file mode 100644 index 670f1581..00000000 --- a/data/prompts/end_to_end_v1/prompt/prompt_domain.txt +++ /dev/null @@ -1,89 +0,0 @@ -[user] -Following is the domain knowledge you need in pddl domain format, which includes: -- the object types in the world, -- the predicates for describing the states of the world, -- the actions that the robot can apply, with their precondtions and effects defined. -""" -(define (domain robot_assembly_problem-domain) - (:requirements :strips :typing) - (:types - thing - object - tool part hand - thing - ) - (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part)) - (:action pick_up - :parameters ( ?hand - hand ?tool - tool ?part - part) - :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part)) - :effect (and (hold ?tool ?part) (not (is_free ?tool)))) - (:action put_down - :parameters ( ?hand - hand ?tool - tool ?part - part) - :precondition (and (hold ?tool ?part) (hold ?hand ?tool)) - :effect (and (not (hold ?hand ?part)) (is_free ?tool))) - (:action place - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2))) - (:action detach - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2)))) - (:action insert - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2))) - (:action pull - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2)))) - (:action screw - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2))) - (:action unscrew - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2)))) - (:action load_tool - :parameters ( ?hand - hand ?tool - tool) - :precondition (and (is_equippable ?tool) (is_free ?hand)) - :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool))) - (:action unload_tool - :parameters ( ?hand - hand ?tool - tool) - :precondition (and (hold ?hand ?tool) (is_free ?tool)) - :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool)))) -) -""" -The precidates are explained as follows: -""" -"PREDICATE_LIST" -- is_free(): is free and can hold something. (default: true) -- is_equippable(): is ready to be eqquipped by a robot hand. (default: true) -- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false) -- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false) -- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false) -- can_place_to(, ): can be placed into (placement assembly constraint) (default: false) -- hold(, ): is holding (default: false) -- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false) -- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false) -- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false) -""" - -The actions are explained as follows: -""" -"ROBOT_ACTION_LIST" -- pick_up(, , ): use to pick up -- put_down(, , ): use to put down -- place(, , , ): use to place to (to satisfy the placement assembly constraint) -- detach(, , , ): use to detach from (to lift the existing placement assembly constraint) -- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint) -- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint) -- screw(, , , ): use to screw into (to satisfy the screw assembly constraint) -- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint) -- load_tool(, ): equips to manipulate other objects -- unload_tool(, ): unloads to manipulate other objects -""" -the items in <> indicate the type of the instances needed for the predicates and actions. -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. diff --git a/data/prompts/end_to_end_v1/prompt/prompt_environment.txt b/data/prompts/end_to_end_v1/prompt/prompt_environment.txt deleted file mode 100644 index 4cd08f12..00000000 --- a/data/prompts/end_to_end_v1/prompt/prompt_environment.txt +++ /dev/null @@ -1,31 +0,0 @@ -[user] -Information about world states are given as json dictionary. Example: -""" -{ - "objects": [ - {"name": "leg1", "properties": []}, - {"name": "seat", "properties": []}, - {"name": "cabinet", "properties": ["open"]}, - {"name": "parallel_box1", "properties": []}, - {"name": "table", "properties": []}, - ], - "constraints": [ - {"source": "leg1", "name": "can_screw_to", "target": "seat"}, - {"source": "parallel_box1", "name": "can_manipulate", "target": "leg1"}, - {"source": "cabinet", "name": "on_something", "target": "table"}, - ], - "relations": [ - {"source": "leg1", "name": "is_screwed_to", "target": "seat"}, - ], -} -""" -It contains three lists: - - "objects": the objects and their properties (as lists of strings) in the environment. The properties can be changed by robot actions. - - "constraints": the constraint relations that are enforced and should not be changed. They are knowledge about the world or some constraints set by the user. - - "relations": the relations in the environment. They can be changed by robot actions. -The properties, constraints and relations should always align with the [PREDICATE_LIST] in domain knowledge. -You should only concentrate on the objects, constraints and relations that are related to (or can be effected by) the current task. -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/data/prompts/end_to_end_v1/prompt/prompt_example.txt b/data/prompts/end_to_end_v1/prompt/prompt_example.txt deleted file mode 100644 index a079e986..00000000 --- a/data/prompts/end_to_end_v1/prompt/prompt_example.txt +++ /dev/null @@ -1,294 +0,0 @@ -[user] -I will give you some examples of the input and the output you will generate. -Example 1: -""" -- Input: -(define (problem robot_assembly_problem-problem) - (:domain robot_assembly_problem-domain) - (:objects - parallel_box1 parallel_box2 inward_claw outward_claw no_tool - tool - gear1 gear2 gear3 shaft1 shaft2 base - part - left_hand - hand - ) - (:init (can_manipulate parallel_box1 gear1) (can_manipulate outward_claw gear2) (can_manipulate inward_claw gear3) (can_manipulate parallel_box2 shaft1) (can_manipulate no_tool shaft2) (can_screw_to leg1 seat) (can_screw_to leg2 seat) (can_insert_to back seat) (can_screw_to nut1 seat) (can_screw_to nut2 seat) (can_screw_to blub base) (can_place_to lamp blub) (can_insert_to shaft1 base) (can_insert_to shaft2 base) (can_insert_to gear3 shaft2) (can_insert_to gear2 base) (can_insert_to gear1 shaft1) (is_inserted_to shaft1 base) (is_free left_hand) (is_free parallel_box1) (is_free parallel_box2) (is_free inward_claw) (is_free outward_claw) (is_free no_tool) (is_free gear1) (is_free gear2) (is_free gear3) (is_free shaft1) (is_free shaft2) (is_free base) (is_equippable parallel_box1) (is_equippable parallel_box2) (is_equippable inward_claw) (is_equippable outward_claw) (is_equippable no_tool)) - (:goal (and (is_inserted_to gear1 shaft1))) -) -- Output: -``` -{"task_cohesion": { - "action_sequence": [ - "load_tool(left_hand, parallel_box1)", - "pick_up(left_hand, parallel_box1, gear1)", - "insert(left_hand, parallel_box1, gear1, shaft1)" - ], - "step_instructions": [ - "equip the parallel_box1 on the left hand", - "pick up the gear1 with the left hand", - "insert the gear1 into the shaft1 using the left hand with the parallel_box1" - ], - "object_name": ["left_hand", "gear1", "parallel_box1", "shaft1"], - "behavior_tree": { - "name": "insert selector", - "identifier": 13, - "type_name": "selector", - "children": [ - { - "summary": "check if gear1 is inserted to shaft1", - "name": "check gear1 is_inserted_to shaft1", - "identifier": 14, - "type_name": "condition", - "conditions": [ - { - "object_name": "gear1", - "property_name": "is_inserted_to", - "property_value": "shaft1", - "status": True, - } - ], - }, - { - "name": "insert sequence", - "identifier": 15, - "type_name": "sequence", - "children": [ - { - "name": "load_tool selector", - "identifier": 0, - "type_name": "selector", - "children": [ - { - "summary": "check if parallel_box1 is hold", - "name": "check left_hand hold parallel_box1", - "identifier": 1, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "parallel_box1", - "status": True, - } - ], - }, - { - "name": "load_tool Sequence", - "identifier": 2, - "type_name": "sequence", - "children": [ - { - "summary": "check if parallel_box1 is equippable", - "name": "check parallel_box1 is_equippable", - "identifier": 3, - "type_name": "condition", - "conditions": [ - { - "object_name": "parallel_box1", - "property_name": "is_equippable", - "property_value": None, - "status": True, - } - ], - }, - { - "summary": "check if left hand is free", - "name": "check left_hand is_free", - "identifier": 4, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "is_free", - "property_value": None, - "status": True, - } - ], - }, - { - "summary": "equip parallel_box1 to left hand", - "name": "load_tool(left_hand, parallel_box1)", - "identifier": 5, - "type_name": "action", - "effects": [ - { - "object_name": "hand", - "property_name": "is_free", - "property_value": None, - "status": False, - }, - { - "object_name": "parallel_box1", - "property_name": "is_equippable", - "property_value": None, - "status": False, - }, - { - "object_name": "hand", - "property_name": "hold", - "property_value": "parallel_box1", - "status": True, - }, - ], - }, - ], - }, - ], - }, - { - "name": "pick_up selector", - "identifier": 6, - "type_name": "selector", - "children": [ - { - "summary": "check if parallel_box1 holds gear1", - "name": "check parallel_box1 hold gear1", - "identifier": 7, - "type_name": "condition", - "conditions": [ - { - "object_name": "parallel_box1", - "property_name": "hold", - "property_value": "gear1", - "status": True, - } - ], - }, - { - "name": "pick_up Sequence", - "identifier": 8, - "type_name": "sequence", - "children": [ - { - "summary": "check if parallel_box1 is free", - "name": "check parallel_box1 is_free", - "identifier": 9, - "type_name": "condition", - "conditions": [ - { - "object_name": "parallel_box1", - "property_name": "is_free", - "property_value": None, - "status": True, - } - ], - }, - { - "summary": "check if left hand holds parallel_box1", - "name": "check left_hand hold parallel_box1", - "identifier": 16, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "parallel_box1", - "status": True, - } - ], - }, - { - "summary": "pick up gear1 using parallel_box1", - "name": "pick_up(left_hand, parallel_box1, gear1)", - "identifier": 12, - "type_name": "action", - "effects": [ - { - "object_name": "parallel_box1", - "property_name": "hold", - "property_value": "gear1", - "status": True, - }, - { - "object_name": "parallel_box1", - "property_name": "is_free", - "property_value": None, - "status": False, - }, - ], - }, - ], - }, - ], - }, - { - "summary": "check if gear1 can be inserted to shaft1", - "name": "check gear1 can_insert_to shaft1", - "identifier": 18, - "type_name": "condition", - "conditions": [ - { - "object_name": "gear1", - "property_name": "can_insert_to", - "property_value": "shaft1", - "status": True, - } - ], - }, - { - "summary": "insert gear1 to shaft1", - "name": "insert(left_hand, parallel_box1, gear1, shaft1)", - "identifier": 19, - "type_name": "action", - "effects": [ - { - "object_name": "parallel_box1", - "property_name": "hold", - "property_value": "gear1", - "status": False, - }, - { - "object_name": "parallel_box1", - "property_name": "is_free", - "property_value": None, - "status": True, - }, - { - "object_name": "gear1", - "property_name": "is_inserted_to", - "property_value": "shaft1", - "status": True, - }, - ], - }, - ], - }, - ], -} -}, -"environment_before": {"objects": [ - {"name": "gear1", "properties": []}, - {"name": "shaft1", "properties": []}, - {"name": "parallel_box1", "properties": ["is_equippable", "is_free"]}, - {"name": "left_hand", "properties": ["is_free"]}, - ], - "constraints": [ - {"source": "gear1", "name": "can_insert_to", "target": "shaft1"}, - {"source": "parallel_box1", "name": "can_manipulate", "target": "gear1"}, - ], - "relations": [ - - ], -}, -"environment_after": {"objects": [ - {"name": "gear1", "properties": []}, - {"name": "shaft1", "properties": []}, - {"name": "parallel_box1", "properties": ["is_equippable", "is_free"]}, - {"name": "left_hand", "properties": []}, - ], - "constraints": [ - {"source": "gear1", "name": "can_insert_to", "target": "shaft1"}, - {"source": "parallel_box1", "name": "can_manipulate", "target": "gear1"}, - ], - "relations": [ - {"source": "gear1", "name": "is_inserted_to", "target": "shaft1"}, - {"source": "left_hand", "name": "hold", "target": "parallel_box1"}, - ], -}, -"instruction_summary": "insert gear1 into shaft1", -"question":""} -``` -""" -From these examples, learn that some robotic actions have dependencies with the actions before and after them. -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/data/prompts/end_to_end_v1/prompt/prompt_output_format.txt b/data/prompts/end_to_end_v1/prompt/prompt_output_format.txt deleted file mode 100644 index 6eaad1e0..00000000 --- a/data/prompts/end_to_end_v1/prompt/prompt_output_format.txt +++ /dev/null @@ -1,21 +0,0 @@ -[user] -You generate an action sequence to solve the task and contruct a behavior tree based on it in json format. -The output dictionary has five keys. -""" -- dictionary["task_plan"]: A dictionary containing information about the task plan. -- dictionary["environment_before"]: The environment state before executing the plan. -- dictionary["environment_after"]: The environment state after executing the plan. -- dictionary["instruction_summary"]: A brief summary of the task plan. -- dictionary["question"]: If you don't have enough knowledge for solving the problem, you can ask questions about the knowledge you need. Leave this key empty if you can solve the task. -""" -Three keys exist in dictionary["task_plan"]. -""" -- dictionary["task_plan"]["action_sequence"]: Contains a list of robot actions. Only the actions defined in the "ROBOT_ACTION LIST" will be used. -- dictionary["task_plan"]["action_instructions"]: contains a list of instructions corresponding to dictionary["task_plan"]["action_sequence"] to explain the actions. -- dictionary["task_plan"]["manipulated_objects"]: The list of the names of the manipulated object. It only includes the objects that are related to the current task. -- dictionary["task_plan"]["behavior_tree"]: contains a dictionary of the behavior tree constructed according to the dictionary ["task_plan"]["action_sequence"] -""" -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/data/prompts/end_to_end_v1/prompt/prompt_problem.txt b/data/prompts/end_to_end_v1/prompt/prompt_problem.txt deleted file mode 100644 index 22734b26..00000000 --- a/data/prompts/end_to_end_v1/prompt/prompt_problem.txt +++ /dev/null @@ -1,23 +0,0 @@ -[user] -Information about the problem is given as a pddl problem. -It includes the following information: -- Objects in the world: all instances of the types defined in the domain knowledge. For example, "parallel_box1" is an instance of the type "tool", while "gear1" is an instance of the type "part". -- Initial states: the world states to start with, described with [PREDICATE_LIST] from domain knowledge. For example, "is_free hand" means the object "hand" is free and holds nothing. -- Goal states: the world states to achieve, described with [PREDICATE_LIST] from domain knowledge. -Example: -""" -(define (problem robot_assembly_problem-problem) - (:domain robot_assembly_problem-domain) - (:objects - parallel_box1 parallel_box2 inward_claw outward_claw no_tool - tool - gear1 gear2 gear3 shaft1 shaft2 base - part - left_hand - hand - ) - (:init (can_manipulate parallel_box1 gear1) (can_manipulate outward_claw gear2) (can_manipulate inward_claw gear3) (can_manipulate parallel_box2 shaft1) (can_manipulate no_tool shaft2) (can_screw_to leg1 seat) (can_screw_to leg2 seat) (can_insert_to back seat) (can_screw_to nut1 seat) (can_screw_to nut2 seat) (can_screw_to blub base) (can_place_to lamp blub) (can_insert_to shaft1 base) (can_insert_to shaft2 base) (can_insert_to gear3 shaft2) (can_insert_to gear2 base) (can_insert_to gear1 shaft1) (is_free left_hand) (is_free parallel_box1) (is_free parallel_box2) (is_free inward_claw) (is_free outward_claw) (is_free no_tool) (is_equippable parallel_box1) (is_equippable parallel_box2) (is_equippable inward_claw) (is_equippable outward_claw) (is_equippable no_tool)) - (:goal (and (is_inserted_to shaft1 base))) -) -""" -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/data/prompts/end_to_end_v1/prompt/prompt_role.txt b/data/prompts/end_to_end_v1/prompt/prompt_role.txt deleted file mode 100644 index 6ce736a0..00000000 --- a/data/prompts/end_to_end_v1/prompt/prompt_role.txt +++ /dev/null @@ -1,5 +0,0 @@ -[user] -You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you break it down and construct robot task plans represented by behaiovr trees. -Please do not begin working until I say "Start working." Instead, simply output the message "Waiting for next input." Understood? -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/data/prompts/end_to_end_v1/query/old_query.txt b/data/prompts/end_to_end_v1/query/old_query.txt deleted file mode 100644 index f652fe76..00000000 --- a/data/prompts/end_to_end_v1/query/old_query.txt +++ /dev/null @@ -1,23 +0,0 @@ -Start working. Resume from the environment below. -""" -{"environment":[ENVIRONMENT]} -""" -The problem is as follows: -""" -{"problem": [PROBLEM]} -""" -The instruction is as follows: -""" -{"instruction": [INSTRUCTION]} -""" -The dictionary that you return should be formatted as python dictionary. Follow these rules: -1. Always equip the proper tools before performing specific object manipulaion. The robot should not hold any tool or object at the end. -2. Make sure that each element of the ["action_instructions"] explains corresponding element of the ["action_sequence"]. Refer to the "ROBOT ACTION LIST" to understand the elements of ["action_sequence"]. -3, The length of the ["action_instructions"] list must be the same as the length of the ["action_sequence"] list. -4. Never left ',' at the end of the list. -5. Keep track of all items listed in the "objects" section of the "environment_before" field. Please ensure that you fill out both the "objects" and "object_states" sections for all listed items. -6. Use the "PREDICATE_LIST" for the "object states" field. -7. All keys of the dictionary should be double-quoted. -8. Insert ``` at the beginning and the end of the dictionary to separate it from the rest of your response. -9. Make sure that you output a consistent manipultation as a single arm robot. For example, grasping an object should not occur in successive steps. -Adhere to the output format I defined above. Follow the nine rules. Think step by step. \ No newline at end of file diff --git a/data/prompts/end_to_end_v1/query/query.txt b/data/prompts/end_to_end_v1/query/query.txt deleted file mode 100644 index 4eaf4407..00000000 --- a/data/prompts/end_to_end_v1/query/query.txt +++ /dev/null @@ -1,16 +0,0 @@ -Start working. Resume from problem below. -""" -{"problem": [PROBLEM]} -""" -The dictionary that you return should be formatted as json file. Follow these rules: -1. Always equip the proper tools before performing specific object manipulaion. -2. Make sure that each element of the ["action_instructions"] explains corresponding element of the ["action_sequence"]. Refer to the "ROBOT ACTION LIST" to understand the elements of ["action_sequence"]. -3, The length of the ["action_instructions"] list and the ["action_sequence"] list should be the same. -4. Never left ',' at the end of the list. Use json boolean values, "true" or "false", instead of "True" or "False". -5. Keep track of all items listed in the "objects" section of the "environment_before" field. Please make sure you fill out the "objects" section for all listed items. -6. Use the "PREDICATE_LIST" for the states in the problem. -7. All keys of the dictionary should be double-quoted. -8. Insert ``` at the beginning and the end of the dictionary to separate it from the rest of your response. -9. Make sure that you output a consistent manipultation as a single arm robot. For example, grasping an object should not occur in successive steps. -10. When generating the behavior tree, make sure the tree has the asked structure. -Adhere to the output format I defined above. Follow the rules above. Think step by step. \ No newline at end of file diff --git a/data/prompts/end_to_end_v1/query/second_old_query.txt b/data/prompts/end_to_end_v1/query/second_old_query.txt deleted file mode 100644 index 530173f3..00000000 --- a/data/prompts/end_to_end_v1/query/second_old_query.txt +++ /dev/null @@ -1,16 +0,0 @@ -Start working. Resume from problem below. -""" -{"problem": [PROBLEM]} -""" -The dictionary that you return should be formatted as python dictionary. Follow these rules: -1. Always equip the proper tools before performing specific object manipulaion. -2. Make sure that each element of the ["action_instructions"] explains corresponding element of the ["action_sequence"]. Refer to the "ROBOT ACTION LIST" to understand the elements of ["action_sequence"]. -3, The length of the ["action_instructions"] list must be the same as the length of the ["action_sequence"] list. -4. Never left ',' at the end of the list. -5. Keep track of all items listed in the "objects" section of the "environment_before" field. Please make sure you fill out the "objects" section for all listed items. -6. Use the "PREDICATE_LIST" for the "object states" field. -7. All keys of the dictionary should be double-quoted. -8. Insert ``` at the beginning and the end of the dictionary to separate it from the rest of your response. -9. Make sure that you output a consistent manipultation as a single arm robot. For example, grasping an object should not occur in successive steps. -10. For boolean values don't use capital letters. Use "true" or "false" instead of "True" or "False". -Adhere to the output format I defined above. Follow the rules above. Think step by step. \ No newline at end of file diff --git a/data/prompts/end_to_end_v1/system/system.txt b/data/prompts/end_to_end_v1/system/system.txt deleted file mode 100644 index 26f2afe2..00000000 --- a/data/prompts/end_to_end_v1/system/system.txt +++ /dev/null @@ -1 +0,0 @@ -You are an excellent interpreter of instructions for robotic assembly tasks. Given the knowledge you need, the world states and some instructions, you break it down and construct robot task plans represented by behaiovr trees based on the action sequences. \ No newline at end of file diff --git a/data/prompts/end_to_end_v2/prompt/backup b/data/prompts/end_to_end_v2/prompt/backup deleted file mode 100644 index e45ce6c2..00000000 --- a/data/prompts/end_to_end_v2/prompt/backup +++ /dev/null @@ -1,52 +0,0 @@ - -There are also some counterexamples for you to refer to: -1. One sequence should never have multiple action nodes as its children like the behavior tree below. -{ - "name": "Get the apple", - "identifier": 0, - "type_name": "selector", - "children": [ - { - "summary": "check if the apple is in hand", - "name": "check apple in hand", - "identifier": 1, - "type_name": "condition", - (condition omitted) - }, - { - "name": "Pick Up Sequence", - "identifier": 2, - "type_name": "sequence", - "children": [ - { - "summary": "check if the apple is on the table", - "name": "check apple on the table", - "identifier": 3, - "type_name": "condition", - (condition omitted) - }, - { - "summary": "go to the table", - "name": "go_to(table)", - "identifier": 4, - "type_name": "action", - (effect omitted) - }, - { - "summary": "check if the hand is free", - "name": "check hand free", - "identifier": 5, - "type_name": "condition", - (condition omitted) - }, - { - "summary": "pick up the apple", - "name": "pick_up(apple)", - "identifier": 6, - "type_name": "action", - (effect omitted) - }, - ], - }, - ], -} \ No newline at end of file diff --git a/data/prompts/end_to_end_v2/prompt/e2e_bt.txt b/data/prompts/end_to_end_v2/prompt/e2e_bt.txt deleted file mode 100644 index 21e56f03..00000000 --- a/data/prompts/end_to_end_v2/prompt/e2e_bt.txt +++ /dev/null @@ -1,140 +0,0 @@ -[user] -The behavior tree should be in json dictionary format and have a structure like the following example: -""" -{ - "summary": "selector to load inward_claw to left hand", - "name": "selector: load_tool(left_hand, inward_claw)", - "identifier": 3, - "type_name": "selector", - "children": [ - { - "summary": "check if left_hand is holding inward_claw", - "name": "target: hold(left_hand, inward_claw)", - "identifier": 4, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "inward_claw", - "status": true, - } - ], - }, - { - - "summary": "sequence to load inward_claw to left hand", - "name": "sequence: load_tool(left_hand, inward_claw)", - "identifier": 5, - "type_name": "sequence", - "children": [ - { - "summary": "check if inward_claw is equippable", - "name": "precondition: is_equippable(inward_claw)", - "identifier": 6, - "type_name": "condition", - "conditions": [ - { - "object_name": "inward_claw", - "property_name": "is_equippable", - "property_value": null, - "status": true, - } - ], - }, - { - "summary": "check if left hand is free", - "name": "precondition: is_free(left_hand)", - "identifier": 7, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "is_free", - "property_value": null, - "status": true, - } - ], - }, - { - "summary": "equip inward_claw to left hand", - "name": "action: load_tool(left_hand, inward_claw)", - "identifier": 8, - "type_name": "action", - "effects": [ - { - "object_name": "left_hand", - "property_name": "is_free", - "property_value": null, - "status": false, - }, - { - "object_name": "inward_claw", - "property_name": "is_equippable", - "property_value": null, - "status": false, - }, - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "inward_claw", - "status": true, - }, - ], - }, - ], - }, - ], -} -""" -The dictionary that you return should be formatted as json dictionary. Follow these rules: -1. Every node should have four basic keys: - - "summary": a short description of the node. - - "name": the name of the node. The name should follow the definitions in the domain knowledge. - - "identifier": the unique id of the node. - - "type_name": can be: "selector", "sequence", "condition", "action". -2. The control flow node, "selector" and "sequence" nodes, can have "children", which is a list of children nodes. -3. The "condition" nodes that are children of "selectors" are "targets". Those that are children of "sequences" are preconditions. -4. The "condition" nodes should have the key "conditions", which is a list of conditions to be checked. Each condition should have the following keys: - - "object_name": the object to check. - - "property_name": the property/relation of the object to be checked. - - "property_value": this should be the other object in the relation. For property it should be null. - - "status": true if the property/relation is expected to be true, false otherwise. -5. The "action" nodes should have a key "effects", which is a list of the to-be-exerted effects of the action. Each effect should have the following keys: - - "object_name": the affected object. - - "property_name": the affected property/relation of the object. - - "property_value": this should be the other object if this is a relation. For property it should be null. - - "status": true if the effect is to add this property/relation. false if is to remove it. -6. All the subtrees always have their corresponding action in ROBOT_ACTION_LIST. They are used to represent the actions in the action sequence. Its basic structure is: - - A selector as the root node. - - (Usually one) target condition node as the first child of the selector to check if the target of the action has been fulfilled. - - A sequence node as the last child of the selector node to achieve the target. - - (Usually a number of) precondition nodes as the front children of the sequence node to check the preconditions for executing the action. - - One action node as the last child of the sequence node, whose preconditions are checked before and will immediate start the job when being ticked. - - The action will exert its defined effects after finishing the job successfully. Those effects should fulfilled all the target conditions in the selector node. - A brief example: -{ - selector: load_tool selector, - children: [ - { - target: check if inward_claw is equipped by left hand, - }, - { - sequence: load_tool, - children: [ - precondition: check if inward_claw is equippable, - precondition: check if left hand is free, - {action: equip inward_claw to left hand, effects: [left hand not free, inward_claw not equippable, left_hand holds inward_claw]}, - ], - }, - ], -} - -7. When performing a new action to fulfill a unsatisfied precondition, you can construct a subtree to replace the precondition and take that unfulfilled precondition as the target of the subtree. Then you construct the sequence of the subtree according to the new action you want to perform. -8. The behavior tree should be constructed based on the action sequence you planned and the action definitions in ROBOT_ACTION_LIST. -9. All of the control flow nodes, namely selectors and sequences, are without memory. A new behavior tree tick will always start from the root and traverse all the nodes even if they are ticked in the last tick. - -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/data/prompts/end_to_end_v2/prompt/e2e_chain.txt b/data/prompts/end_to_end_v2/prompt/e2e_chain.txt deleted file mode 100644 index abd348a9..00000000 --- a/data/prompts/end_to_end_v2/prompt/e2e_chain.txt +++ /dev/null @@ -1,16 +0,0 @@ -[user] -Follow this workflow to solve the task: - -1. Read the problem description. Parse the initial state and write into the "initial_state" in the output dictionary. -2. Interpret the goal of the problem as goal_state and write into the "instruction_summary" in the output dictionary. -3. Try to solve the problem, achieve the goal_state from the initial_state based on the domain knowledge provided. Generate a action sequence with the actions defined in and write it into dictionary["task_plan"]["action_sequence"] of the output. -4. If you cannot find a plan to solve the task, ask question with nature language in the "question" of the output dictionary. -5. Explain the actions in the action sequence and write the output dictionary["task_plan"]["action_instructions"]. -6. Collect the objects in the world that are affected by the action sequence, write into dictionary["task_plan"]["manipulated_objects"]. -7. Update the "initial_state" based on the affected objects you collected in the last step so irrelevant objects and relations/constraints are removed. -8. Generate the behavior tree based on the action sequence and write into dictionary["task_plan"]["behavior_tree"]. - -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/data/prompts/end_to_end_v2/prompt/e2e_domain.txt b/data/prompts/end_to_end_v2/prompt/e2e_domain.txt deleted file mode 100644 index bf3c7859..00000000 --- a/data/prompts/end_to_end_v2/prompt/e2e_domain.txt +++ /dev/null @@ -1,89 +0,0 @@ -[user] -Following is the domain knowledge you need in pddl domain format, which includes: -- the object types in the world, -- the predicates for describing the states of the world, -- the actions that the robot can apply, with their precondtions and effects defined. -""" -(define (domain robot_assembly_problem-domain) - (:requirements :strips :typing) - (:types - thing - object - tool part hand - thing - ) - (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part)) - (:action pick_up - :parameters ( ?hand - hand ?tool - tool ?part - part) - :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part)) - :effect (and (hold ?tool ?part) (not (is_free ?tool)))) - (:action put_down - :parameters ( ?hand - hand ?tool - tool ?part - part) - :precondition (and (hold ?tool ?part) (hold ?hand ?tool)) - :effect (and (not (hold ?hand ?part)) (is_free ?tool))) - (:action place - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2))) - (:action detach - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2)))) - (:action insert - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2))) - (:action pull - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2)))) - (:action screw - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2))) - (:action unscrew - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2)))) - (:action load_tool - :parameters ( ?hand - hand ?tool - tool) - :precondition (and (is_equippable ?tool) (is_free ?hand)) - :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool))) - (:action unload_tool - :parameters ( ?hand - hand ?tool - tool) - :precondition (and (hold ?hand ?tool) (is_free ?tool)) - :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool)))) -) -""" -The precidates are explained as follows: -""" -"PREDICATE_LIST" -- is_free( or ): or is free and can hold something. (default: true) -- is_equippable(): is ready to be eqquipped by a robot hand. (default: true) -- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false) -- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false) -- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false) -- can_place_to(, ): can be placed into (placement assembly constraint) (default: false) -- hold(, ): is holding (default: false) -- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false) -- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false) -- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false) -""" - -The actions are explained as follows: -""" -"ROBOT_ACTION_LIST" -- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something. -- put_down(, , ): use to put down . As preconditions, should be holding and should be holding . -- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding . -- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into . -- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding . -- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into . -- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding . -- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into . -- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free. -- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free. -""" -the items in <> indicate the type of the instances needed for the predicates and actions. -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. diff --git a/data/prompts/end_to_end_v2/prompt/e2e_example.txt b/data/prompts/end_to_end_v2/prompt/e2e_example.txt deleted file mode 100644 index f0dccece..00000000 --- a/data/prompts/end_to_end_v2/prompt/e2e_example.txt +++ /dev/null @@ -1,281 +0,0 @@ -[user] -I will give you some examples of the input and the output you will generate. -Example 1: -""" -- Input: -(define (problem robot_assembly_problem-problem) - (:domain robot_assembly_problem-domain) - (:objects - parallel_box1 parallel_box2 inward_claw outward_claw no_tool - tool - gear1 gear2 gear3 shaft1 shaft2 base - part - left_hand - hand - ) - (:init (can_manipulate parallel_box1 gear1) (can_manipulate outward_claw gear2) (can_manipulate inward_claw gear3) (can_manipulate parallel_box2 shaft1) (can_manipulate no_tool shaft2) (can_screw_to leg1 seat) (can_screw_to leg2 seat) (can_insert_to back seat) (can_screw_to nut1 seat) (can_screw_to nut2 seat) (can_screw_to blub base) (can_place_to lamp blub) (can_insert_to shaft1 base) (can_insert_to shaft2 base) (can_insert_to gear3 shaft2) (can_insert_to gear2 base) (can_insert_to gear1 shaft1) (is_inserted_to shaft1 base) (is_free left_hand) (is_free parallel_box1) (is_free parallel_box2) (is_free inward_claw) (is_free outward_claw) (is_free no_tool) (is_equippable parallel_box1) (is_equippable parallel_box2) (is_equippable inward_claw) (is_equippable outward_claw) (is_equippable no_tool)) - (:goal (and (is_inserted_to gear1 shaft1))) -) -- Output: -``` -{"task_plan": { - "action_sequence": [ - "load_tool(left_hand, parallel_box1)", - "pick_up(left_hand, parallel_box1, gear1)", - "insert(left_hand, parallel_box1, gear1, shaft1)" - ], - "action_instructions": [ - "equip the parallel_box1 on the left hand", - "pick up the gear1 with the parallel_box1 in the left hand", - "insert the gear1 into the shaft1 using the parallel_box1 in the left hand" - ], - "object_name": ["left_hand", "gear1", "parallel_box1", "shaft1"], - "behavior_tree": { - "summary": "selector to insert gear1 into shaft1", - "name": "selector: insert gear1 into shaft1", - "identifier": 13, - "type_name": "selector", - "children": [ - { - "summary": "check if gear1 is inserted to shaft1", - "name": "target: is_inserted_to(gear1, shaft1)", - "type_name": "condition", - "conditions": [ - { - "object_name": "gear1", - "property_name": "is_inserted_to", - "property_value": "shaft1", - "status": true, - } - ], - }, - { - "summary": "sequence to insert gear1 into shaft1", - "name": "sequence: insert(gear1, shaft1)", - "identifier": 15, - "type_name": "sequence", - "children": [ - { - "summary": "selector to load parallel_box1", - "name": "selector: load_tool(left_hand, parallel_box1)", - "identifier": 0, - "type_name": "selector", - "children": [ - { - "summary": "check if parallel_box1 is hold", - "name": "target: hold(left_hand, parallel_box1)", - "identifier": 1, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "parallel_box1", - "status": true, - } - ], - }, - { - "summary": "sequence to load parallel_box1", - "name": "sequence: load_tool(left_hand, parallel_box1)", - "identifier": 2, - "type_name": "sequence", - "children": [ - { - "summary": "check if parallel_box1 is equippable", - "name": "precondition: is_equippable(parallel_box1)", - "identifier": 3, - "type_name": "condition", - "conditions": [ - { - "object_name": "parallel_box1", - "property_name": "is_equippable", - "property_value": null, - "status": true, - } - ], - }, - { - "summary": "check if left hand is free", - "name": "precondition: is_free(left_hand)", - "identifier": 4, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "is_free", - "property_value": null, - "status": true, - } - ], - }, - { - "summary": "equip parallel_box1 to left hand", - "name": "action: load_tool(left_hand, parallel_box1)", - "identifier": 5, - "type_name": "action", - "effects": [ - { - "object_name": "hand", - "property_name": "is_free", - "property_value": null, - "status": false, - }, - { - "object_name": "parallel_box1", - "property_name": "is_equippable", - "property_value": null, - "status": false, - }, - { - "object_name": "hand", - "property_name": "hold", - "property_value": "parallel_box1", - "status": true, - }, - ], - }, - ], - }, - ], - }, - { - "summary": "selector to pick_up gear1", - "name": "selector: pick_up(left_hand, parallel_box1, gear1)", - "identifier": 6, - "type_name": "selector", - "children": [ - { - "summary": "check if parallel_box1 holds gear1", - "name": "target: hold(parallel_box1, gear1)", - "identifier": 7, - "type_name": "condition", - "conditions": [ - { - "object_name": "parallel_box1", - "property_name": "hold", - "property_value": "gear1", - "status": true, - } - ], - }, - { - "summary": "sequence to pick_up gear1", - "name": "sequence: pick_up(left_hand, parallel_box1, gear1)", - "identifier": 8, - "type_name": "sequence", - "children": [ - { - "summary": "check if parallel_box1 is free", - "name": "precondition: is_free(parallel_box1)", - "identifier": 9, - "type_name": "condition", - "conditions": [ - { - "object_name": "parallel_box1", - "property_name": "is_free", - "property_value": null, - "status": true, - } - ], - }, - { - "summary": "check if left hand holds parallel_box1", - "name": "precondition: hold(left_hand, parallel_box1)", - "identifier": 16, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "parallel_box1", - "status": true, - } - ], - }, - { - "summary": "pick up gear1 using parallel_box1", - "name": "action: pick_up(left_hand, parallel_box1, gear1)", - "identifier": 12, - "type_name": "action", - "effects": [ - { - "object_name": "parallel_box1", - "property_name": "hold", - "property_value": "gear1", - "status": true, - }, - { - "object_name": "parallel_box1", - "property_name": "is_free", - "property_value": null, - "status": false, - }, - ], - }, - ], - }, - ], - }, - { - "summary": "check if gear1 can be inserted to shaft1", - "name": "precondition: can_insert_to(gear1, shaft1)", - "identifier": 18, - "type_name": "condition", - "conditions": [ - { - "object_name": "gear1", - "property_name": "can_insert_to", - "property_value": "shaft1", - "status": true, - } - ], - }, - { - "summary": "insert gear1 to shaft1", - "name": "action: insert(left_hand, parallel_box1, gear1, shaft1)", - "identifier": 19, - "type_name": "action", - "effects": [ - { - "object_name": "parallel_box1", - "property_name": "hold", - "property_value": "gear1", - "status": false, - }, - { - "object_name": "parallel_box1", - "property_name": "is_free", - "property_value": null, - "status": true, - }, - { - "object_name": "gear1", - "property_name": "is_inserted_to", - "property_value": "shaft1", - "status": true, - }, - ], - }, - ], - }, - ], -} -}, -"initial_state": {"objects": [ - {"name": "gear1", "properties": []}, - {"name": "shaft1", "properties": []}, - {"name": "parallel_box1", "properties": ["is_equippable", "is_free"]}, - {"name": "left_hand", "properties": ["is_free"]}, - ], - "constraints": [ - {"source": "gear1", "name": "can_insert_to", "target": "shaft1"}, - {"source": "parallel_box1", "name": "can_manipulate", "target": "gear1"}, - ], - "relations": [], -}, -"instruction_summary": "insert gear1 into shaft1", -"question":""} -``` -""" -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/data/prompts/end_to_end_v2/prompt/e2e_output_format.txt b/data/prompts/end_to_end_v2/prompt/e2e_output_format.txt deleted file mode 100644 index a194e955..00000000 --- a/data/prompts/end_to_end_v2/prompt/e2e_output_format.txt +++ /dev/null @@ -1,21 +0,0 @@ -[user] -You will be provided with a problem and the necessary domain knowledge. -You generate an action sequence to solve the task and contruct a behavior tree based on it in json format. -The output dictionary has five keys. -""" -- dictionary["task_plan"]: A dictionary containing the task plan. -- dictionary["initial_state"]: The initial world state before executing the plan. -- dictionary["instruction_summary"]: A brief summary of the task plan. -- dictionary["question"]: If you don't have enough knowledge for solving the problem, you can ask questions about the knowledge you need. Leave this key empty if you can solve the task. -""" -Three keys exist in dictionary["task_plan"]. -""" -- dictionary["task_plan"]["action_sequence"]: Contains a list of robot actions. Only the actions defined in the "ROBOT_ACTION LIST" will be used. -- dictionary["task_plan"]["action_instructions"]: contains a list of instructions corresponding to dictionary["task_plan"]["action_sequence"] to explain the actions. -- dictionary["task_plan"]["manipulated_objects"]: The list of the manipulated objects. It only includes the objects that are related to the current task. -- dictionary["task_plan"]["behavior_tree"]: contains a dictionary of the behavior tree constructed according to the dictionary ["task_plan"]["action_sequence"] -""" -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/data/prompts/end_to_end_v2/prompt/e2e_problem.txt b/data/prompts/end_to_end_v2/prompt/e2e_problem.txt deleted file mode 100644 index 6b1ddfb7..00000000 --- a/data/prompts/end_to_end_v2/prompt/e2e_problem.txt +++ /dev/null @@ -1,24 +0,0 @@ -[user] -Information about the problem is given as a pddl problem. -It includes the following information: -- objects: all the objects in the world. They should be of a type defined in the domain knowledge. For example, "parallel_box1" is an instance of the type "tool", while "gear1" is an instance of the type "part". -- init: the initial world state to start with, described with [PREDICATE_LIST] from domain knowledge. For example, "is_free hand" means the object "hand" is free and holds nothing. -- goal: the world state to achieve, described with [PREDICATE_LIST] from domain knowledge. -An example: -""" -(define (problem robot_assembly_problem-problem) - (:domain robot_assembly_problem-domain) - (:objects - parallel_box1 parallel_box2 inward_claw outward_claw no_tool - tool - gear1 gear2 gear3 shaft1 shaft2 base - part - left_hand - hand - ) - (:init (can_manipulate parallel_box1 gear1) (can_manipulate outward_claw gear2) (can_manipulate inward_claw gear3) (can_manipulate parallel_box2 shaft1) (can_manipulate no_tool shaft2) (can_screw_to leg1 seat) (can_screw_to leg2 seat) (can_insert_to back seat) (can_screw_to nut1 seat) (can_screw_to nut2 seat) (can_screw_to blub base) (can_place_to lamp blub) (can_insert_to shaft1 base) (can_insert_to shaft2 base) (can_insert_to gear3 shaft2) (can_insert_to gear2 base) (can_insert_to gear1 shaft1) (is_free left_hand) (is_free parallel_box1) (is_free parallel_box2) (is_free inward_claw) (is_free outward_claw) (is_free no_tool) (is_equippable parallel_box1) (is_equippable parallel_box2) (is_equippable inward_claw) (is_equippable outward_claw) (is_equippable no_tool)) - (:goal (and (is_inserted_to shaft1 base))) -) -""" - -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/data/prompts/end_to_end_v2/prompt/e2e_role.txt b/data/prompts/end_to_end_v2/prompt/e2e_role.txt deleted file mode 100644 index 6ce736a0..00000000 --- a/data/prompts/end_to_end_v2/prompt/e2e_role.txt +++ /dev/null @@ -1,5 +0,0 @@ -[user] -You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you break it down and construct robot task plans represented by behaiovr trees. -Please do not begin working until I say "Start working." Instead, simply output the message "Waiting for next input." Understood? -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/data/prompts/end_to_end_v2/prompt/e2e_state.txt b/data/prompts/end_to_end_v2/prompt/e2e_state.txt deleted file mode 100644 index a8ea9cd6..00000000 --- a/data/prompts/end_to_end_v2/prompt/e2e_state.txt +++ /dev/null @@ -1,30 +0,0 @@ -[user] -Information about world states should be in json dictionary format. Example: -""" -{ - "objects": [ - {"name": "leg1", "properties": []}, # leg1 is an object with no properties - {"name": "seat", "properties": []}, # seat is an object with no properties - {"name": "cabinet", "properties": ["open"]}, # cabinet is an object with a property "open" - {"name": "table", "properties": []}, # table is an object with no properties - ], - "constraints": [ - {"source": "leg1", "name": "can_screw_to", "target": "seat"}, # means leg1 can be screwed to seat - {"source": "cabinet", "name": "on_something", "target": "table"}, # means cabinet is on table - ], - "relations": [ - {"source": "leg1", "name": "is_screwed_to", "target": "seat"}, # means leg1 is screwed to seat - ], -} -""" -It contains three lists: - - "objects": the objects and their properties (as lists of strings) in the world. The properties can be changed by robot actions. - - "constraints": the constraint relations that are enforced and should not be changed. They are knowledge about the world or some constraints set by the user. - - "relations": the relations in the world. They can be changed by robot actions. -The properties, constraints and relations should always align with the [PREDICATE_LIST] in domain knowledge. -Only concentrate on the objects, constraints and relations that are related to the current task. - -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/data/prompts/end_to_end_v2/query/query.txt b/data/prompts/end_to_end_v2/query/query.txt deleted file mode 100644 index cfeb4e8e..00000000 --- a/data/prompts/end_to_end_v2/query/query.txt +++ /dev/null @@ -1,16 +0,0 @@ -Start working. Resume from the problem below. -""" -{"problem": [PROBLEM]} -""" -The dictionary that you return should be formatted as json file. Follow these rules: -1. Always equip the proper tools before performing specific object manipulaion. -2. Make sure that each element of the ["action_instructions"] explains corresponding element of the ["action_sequence"]. Refer to the "ROBOT ACTION LIST" to understand the elements of ["action_sequence"]. -3, The length of the ["action_instructions"] list and the ["action_sequence"] list should be the same. -4. Never left ',' at the end of the list. Use json boolean values, "true" or "false", instead of "True" or "False". -5. Keep track of all items listed in the "objects" section of the "initial_state" field. -6. Use the "PREDICATE_LIST" for the initial states (:init) in the problem to generate ["initial_state"]. -7. All keys of the dictionary should be double-quoted. -8. Insert ``` at the beginning and the end of the dictionary to separate it from the rest of your response. Don't write any additional things. -9. Make sure that you output a consistent manipultation as a single arm robot. For example, grasping an object should not occur in successive steps. -10. When generating the behavior tree, make sure the tree has the asked structure. -Adhere to the output format I defined above. Follow the rules above. Think step by step. \ No newline at end of file diff --git a/data/prompts/end_to_end_v2/system/system.txt b/data/prompts/end_to_end_v2/system/system.txt deleted file mode 100644 index 26f2afe2..00000000 --- a/data/prompts/end_to_end_v2/system/system.txt +++ /dev/null @@ -1 +0,0 @@ -You are an excellent interpreter of instructions for robotic assembly tasks. Given the knowledge you need, the world states and some instructions, you break it down and construct robot task plans represented by behaiovr trees based on the action sequences. \ No newline at end of file diff --git a/data/prompts/end_to_end_v3/behaviortree.txt b/data/prompts/end_to_end_v3/behaviortree.txt deleted file mode 100644 index 4d79b87c..00000000 --- a/data/prompts/end_to_end_v3/behaviortree.txt +++ /dev/null @@ -1,51 +0,0 @@ -[user] -The behavior tree should be in json dictionary format. -Following is a simple example: -""" -{ - "summary": "Selector to change the tool in the left hand from outwardgripper to defaultgripper", - "name": "selector: change_tool(left_hand, outwardgripper, defaultgripper)", - "children": [ - { - "summary": "the target is that the left hand is holding the default gripper", - "name": "target: hold(left_hand, defaultgripper)" - }, - { - "summary": "Sequence to change the tool in the left hand from outwardgripper to defaultgripper", - "name": "sequence: change_tool(left_hand, outwardgripper, defaultgripper)", - "children": [ - { - "summary": "A precondition is that the left hand is holding the outwardgripper", - "name": "precondition: hold(left_hand, outwardgripper)" - }, - { - "summary": "A precondition is that the outwardgripper is empty", - "name": "precondition: is_empty(outwardgripper)" - }, - { - "summary": "The action to change the tool in the left hand from outwardgripper to defaultgripper", - "name": "action: change_tool(left_hand, outwardgripper, defaultgripper)" - } - ] - } - ] -} -""" -1. Every node are described by a summary and a name: - - "summary": a short interpretive description of the node. - - "name": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are "selector", "sequence", "condition" ("target", "precondition"), "action". -2. The control flow nodes, namely "selector" and "sequence", can have "children", which is a list of children nodes. -3. The "condition" nodes that are children of "selectors" are targets. Those that are children of "sequences" are preconditions. -4. All the node should always have their corresponding actions/precidates in the domain knowledge. -5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state. -6. The basic structure template of a unit subtree is: - - A selector as the root node. - - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied. - - A sequence node as the last child of the selector node to achieve the target condition node. - - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions. - - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before. - - The action should have effects that can fulfill the target condition node in the selector. - -7. If any precondition node will be unsatisfied when it is ticked, a unit subtree should be constructed to replace it, which take that precondition node as its target node and aims to achieve it. You should estimate the world state when the precondition is ticked (checked) according to the initial state and the effects of the actions executed before, then do this replacement repeatedly until all the preconditions are satisfied at the time they are ticked. -8. The behavior tree should be constructed based on the action sequence you planned and the action definitions in ROBOT_ACTION_LIST. -9. Control flow nodes (sequence nodes and selector nodes) don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state. \ No newline at end of file diff --git a/data/prompts/end_to_end_v3/chain.txt b/data/prompts/end_to_end_v3/chain.txt deleted file mode 100644 index ffcc4fc7..00000000 --- a/data/prompts/end_to_end_v3/chain.txt +++ /dev/null @@ -1,9 +0,0 @@ -Follow this workflow to solve the task: -1. Read the problem description. Parse the initial state and write into the "initial_state" in the output dictionary. -2. Interpret the goal of the problem as goal_state and write into the "instruction_summary" in the output dictionary. -3. Try to solve the problem, achieve the goal_state from the initial_state based on the domain knowledge provided. Generate a action sequence with the actions defined in and write it into dictionary["task_plan"]["action_sequence"] of the output. -4. If you cannot find a plan to solve the task, ask question with nature language in the "question" of the output dictionary. -5. Explain the actions in the action sequence and write the output dictionary["task_plan"]["action_instructions"]. -6. Collect the objects in the world that are affected by the action sequence, write into dictionary["task_plan"]["manipulated_objects"]. -7. Update the "initial_state" based on the affected objects you collected in the last step so irrelevant objects and relations/constraints are removed. -8. Generate the behavior tree based on the action sequence and write into dictionary["task_plan"]["behavior_tree"]. \ No newline at end of file diff --git a/data/prompts/end_to_end_v3/domain.txt b/data/prompts/end_to_end_v3/domain.txt deleted file mode 100644 index 18fdc90b..00000000 --- a/data/prompts/end_to_end_v3/domain.txt +++ /dev/null @@ -1,41 +0,0 @@ -Following is the domain knowledge you need, which includes: -- the object types in the world, -- the predicates for describing the states of the world, -- the actions that the robot can apply, with their precondtions and effects defined. - -The object types are explained as follows: -""" -thing: The base class for all objects in the world. Its subclasses are: hand, tool, part. -hand: The robot hand. A empty can only hold one at a time. -tool: The tool to manipulate . A empty can hold one at a time. -part: The part to be manipulated. -""" - -The precidates are explained as follows: -""" -"PREDICATE_LIST" -- is_empty(): is empty and can hold something. -- can_manipulate(, ): can be used to manipulate (manipulation constraint). This regulates the compatibility between and . -- can_insert_to(, ): can be inserted into (insertion assembly constraint) -- can_screw_to(, ): can be screwed into (screw assembly constraint) -- can_place_to(, ): can be placed into (placement assembly constraint) -- hold(, ): is holding . It can be holding or holding . -- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) -- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) -- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) -""" - -The actions are explained as follows: -""" -"ROBOT_ACTION_LIST" -- pick_up(, , ): use to pick up . As preconditions, should be holding and should be empty in order to hold the . The will be held by the as a result of this action. -- put_down(, , ): use to put down . As preconditions, should be holding and should be holding . The target is to make the tool empty. The will be empty and will not hold the anymore as a result of this action. -- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding . -- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be placed into . -- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding . -- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be inserted into . -- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding . -- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be screwed into . -- change_tool(, , ): change the equipped tool from to in order to manipulate specific parts. As preconditions, should be holding and should be empty. -""" -the items in <> indicate the type of the instances needed for the predicates and actions. \ No newline at end of file diff --git a/data/prompts/end_to_end_v3/example copy.txt b/data/prompts/end_to_end_v3/example copy.txt deleted file mode 100644 index 615373fc..00000000 --- a/data/prompts/end_to_end_v3/example copy.txt +++ /dev/null @@ -1,185 +0,0 @@ -I will give you some examples of the input and the output you will generate. -Example 1: -""" -- Input: - -- Output: -``` -{"task_plan": { - "action_sequence": [ - "load_tool(left_hand, parallel_box1)", - "pick_up(left_hand, parallel_box1, gear1)", - "insert(left_hand, parallel_box1, gear1, shaft1)" - ], - "action_instructions": [ - "equip the parallel_box1 on the left hand", - "pick up the gear1 with the parallel_box1 in the left hand", - "insert the gear1 into the shaft1 using the parallel_box1 in the left hand" - ], - "behavior_tree": { - "summary": "selector to insert gear1 into shaft1", - "name": "selector: insert gear1 into shaft1", - "children": [ - { - "summary": "check if gear1 is inserted to shaft1", - "name": "target: is_inserted_to(gear1, shaft1)", - }, - { - "summary": "sequence to insert gear1 into shaft1", - "name": "sequence: insert(gear1, shaft1)", - "children": [ - { - "summary": "selector to load parallel_box1", - "name": "selector: load_tool(left_hand, parallel_box1)", - "children": [ - { - "summary": "check if parallel_box1 is hold", - "name": "target: hold(left_hand, parallel_box1)", - }, - { - "summary": "sequence to load parallel_box1", - "name": "sequence: load_tool(left_hand, parallel_box1)", - "children": [ - { - "summary": "check if parallel_box1 is equippable", - "name": "precondition: is_equippable(parallel_box1)", - }, - { - "summary": "check if left hand is free", - "name": "precondition: is_free(left_hand)", - }, - { - "summary": "equip parallel_box1 to left hand", - "name": "action: load_tool(left_hand, parallel_box1)", - }, - ], - }, - ], - }, - { - "summary": "selector to pick_up gear1", - "name": "selector: pick_up(left_hand, parallel_box1, gear1)", - "children": [ - { - "summary": "check if parallel_box1 holds gear1", - "name": "target: hold(parallel_box1, gear1)", - }, - { - "summary": "sequence to pick_up gear1", - "name": "sequence: pick_up(left_hand, parallel_box1, gear1)", - "identifier": 8, - "type_name": "sequence", - "children": [ - { - "summary": "check if parallel_box1 is free", - "name": "precondition: is_free(parallel_box1)", - "identifier": 9, - "type_name": "condition", - "conditions": [ - { - "object_name": "parallel_box1", - "property_name": "is_free", - "property_value": null, - "status": true, - } - ], - }, - { - "summary": "check if left hand holds parallel_box1", - "name": "precondition: hold(left_hand, parallel_box1)", - "identifier": 16, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "parallel_box1", - "status": true, - } - ], - }, - { - "summary": "pick up gear1 using parallel_box1", - "name": "action: pick_up(left_hand, parallel_box1, gear1)", - "identifier": 12, - "type_name": "action", - "effects": [ - { - "object_name": "parallel_box1", - "property_name": "hold", - "property_value": "gear1", - "status": true, - }, - { - "object_name": "parallel_box1", - "property_name": "is_free", - "property_value": null, - "status": false, - }, - ], - }, - ], - }, - ], - }, - { - "summary": "check if gear1 can be inserted to shaft1", - "name": "precondition: can_insert_to(gear1, shaft1)", - "identifier": 18, - "type_name": "condition", - "conditions": [ - { - "object_name": "gear1", - "property_name": "can_insert_to", - "property_value": "shaft1", - "status": true, - } - ], - }, - { - "summary": "insert gear1 to shaft1", - "name": "action: insert(left_hand, parallel_box1, gear1, shaft1)", - "identifier": 19, - "type_name": "action", - "effects": [ - { - "object_name": "parallel_box1", - "property_name": "hold", - "property_value": "gear1", - "status": false, - }, - { - "object_name": "parallel_box1", - "property_name": "is_free", - "property_value": null, - "status": true, - }, - { - "object_name": "gear1", - "property_name": "is_inserted_to", - "property_value": "shaft1", - "status": true, - }, - ], - }, - ], - }, - ], -} -}, -"initial_state": {"objects": [ - {"name": "gear1", "properties": []}, - {"name": "shaft1", "properties": []}, - {"name": "parallel_box1", "properties": ["is_equippable", "is_free"]}, - {"name": "left_hand", "properties": ["is_free"]}, - ], - "constraints": [ - {"source": "gear1", "name": "can_insert_to", "target": "shaft1"}, - {"source": "parallel_box1", "name": "can_manipulate", "target": "gear1"}, - ], - "relations": [], -}, -"instruction_summary": "insert gear1 into shaft1" -} -``` -""" \ No newline at end of file diff --git a/data/prompts/end_to_end_v3/example.txt b/data/prompts/end_to_end_v3/example.txt deleted file mode 100644 index cf713974..00000000 --- a/data/prompts/end_to_end_v3/example.txt +++ /dev/null @@ -1,81 +0,0 @@ -Following is an input/output example. -Example 1: -""" -INPUT: -target: "insert the gear1 into the shaft1" -initial_state: {"objects": [ - {"name": "gear1", "properties": []}, - {"name": "shaft1", "properties": []}, - {"name": "parallel_box1", "properties": ["is_empty"]}, - {"name": "left_hand", "properties": []}, - ], - "constraints": [ - {"source": "gear1", "name": "can_insert_to", "target": "shaft1"}, - {"source": "parallel_box1", "name": "can_manipulate", "target": "gear1"}, - ], - "relations": [ - {"source": "left_hand", "name": "hold", "target": "parallel_box1"}, - ], -} -OUTPUT: -{ - "thought": "At the start, the left hand is holding the parallel_box1, and the parallel_box1 is empty. According to the constraints, the gear1 can be inserted to the shaft1, and the parallel_box1 can manipulate the gear1. So, the gear1 can be picked up with the parallel_box1 in the left hand, and then the precondition hold(parallel_box1, gear1) is satisfied. Finally, the gear1 can be inserted into the shaft1 with the parallel_box1 in the left hand.", - "action_sequence": [ - "pick_up(left_hand, parallel_box1, gear1)", - "insert(left_hand, parallel_box1, gear1, shaft1)" - ], - "behavior_tree": { - "summary": "selector to insert the gear1 into the shaft1 with the parallel_box1 in the left hand", - "name": "selector: insert(gear1, shaft1, parallel_box1, left_hand)", - "children": [ - { - "summary": "check the target that gear1 is inserted to shaft1", - "name": "target: is_inserted_to(gear1, shaft1)" - }, - { - "summary": "sequence to insert the gear1 into the shaft1 with the parallel_box1 in the left hand", - "name": "sequence: insert(gear1, shaft1, parallel_box1, left_hand)", - "children": [ - { - "summary": "check the precondition that the left hand is holding the parallel_box1", - "name": "precondition: hold(left_hand, parallel_box1)" - }, - { - "summary": "selector to pick_up the gear1 with the parallel_box1 in the left hand", - "name": "selector: pick_up(left_hand, parallel_box1, gear1)", - "children": [ - { - "summary": "check the target that the parallel_box1 is holding the gear1", - "name": "target: hold(parallel_box1, gear1)" - }, - { - "summary": "sequence to pick_up the gear1 with the parallel_box1 in the left hand", - "name": "sequence: pick_up(left_hand, parallel_box1, gear1)", - "children": [ - { - "summary": "check the precondition that the parallel_box1 is empty", - "name": "precondition: is_empty(parallel_box1)" - }, - { - "summary": "check the precondition that the left hand is holding the parallel_box1", - "name": "precondition: hold(left_hand, parallel_box1)" - }, - { - "summary": "the action to pick_up the gear1 with the parallel_box1 in the left hand", - "name": "action: pick_up(left_hand, parallel_box1, gear1)" - } - ] - } - ] - }, - { - "summary": "the action to insert the gear1 into the shaft1 with the parallel_box1 in the left hand", - "name": "action: insert(left_hand, parallel_box1, gear1, shaft1)" - } - ] - } - ] - } -} - - diff --git a/data/prompts/end_to_end_v3/output_format.txt b/data/prompts/end_to_end_v3/output_format.txt deleted file mode 100644 index c38fc403..00000000 --- a/data/prompts/end_to_end_v3/output_format.txt +++ /dev/null @@ -1,6 +0,0 @@ -The output json obejct has the following keys: -""" -- "thought": A brief explanation of the thinking process behind the action sequence. -- "action_sequence": A sequential list of robot actions. Only the actions defined in the "ROBOT_ACTION LIST" will be used. -- "behavior_tree": A json dictionary form behavior tree constructed according to the "action_sequence" -""" \ No newline at end of file diff --git a/data/prompts/end_to_end_v3/state.txt b/data/prompts/end_to_end_v3/state.txt deleted file mode 100644 index 665700ba..00000000 --- a/data/prompts/end_to_end_v3/state.txt +++ /dev/null @@ -1,6 +0,0 @@ -Information about world states should be in json dictionary format. -It contains three lists: - - objects: the objects and their properties (as lists of strings) in the world. The properties can be changed by robot actions. - - constraints: the constraint relations that are enforced and should not be changed. They are knowledge about the world or some constraints set by the user. - - relations: the relations in the world. They can be changed by robot actions. -The properties, constraints and relations should always align with the [PREDICATE_LIST] in the domain knowledge. \ No newline at end of file diff --git a/data/prompts/end_to_end_v3/system.txt b/data/prompts/end_to_end_v3/system.txt deleted file mode 100644 index cb531830..00000000 --- a/data/prompts/end_to_end_v3/system.txt +++ /dev/null @@ -1 +0,0 @@ -You are an excellent interpreter of instructions for robotic assembly tasks. \ No newline at end of file diff --git a/data/prompts/end_to_end_v3/task.txt b/data/prompts/end_to_end_v3/task.txt deleted file mode 100644 index e039d429..00000000 --- a/data/prompts/end_to_end_v3/task.txt +++ /dev/null @@ -1,6 +0,0 @@ -The input are: -target: The target that you make the sequential plan and construct the behavior tree to achieve. -initial_state: The initial state that you start from. - -You analyze the target, refer to the domain knowledge to understand it, make an action sequence and construct a behavior tree based on it. -Your output should be a json object which follows the regulated format. \ No newline at end of file diff --git a/data/prompts/end_to_end_v3/template.txt b/data/prompts/end_to_end_v3/template.txt deleted file mode 100644 index 9f3f5b61..00000000 --- a/data/prompts/end_to_end_v3/template.txt +++ /dev/null @@ -1,5 +0,0 @@ -Resume from the input below. -""" -target: {target} -initial_state: {initial_state} -""" \ No newline at end of file diff --git a/data/prompts/human_instruction/Zchain.txt b/data/prompts/human_instruction/Zchain.txt deleted file mode 100644 index f60d7911..00000000 --- a/data/prompts/human_instruction/Zchain.txt +++ /dev/null @@ -1,2 +0,0 @@ -This is how you should work: -1. If last_behavior_tree doesn't exist, you interpret user_instruction into a target condition. Then you find one action from the domain knowledge that can fulfill the target condition. You finally return the \ No newline at end of file diff --git a/data/prompts/human_instruction/Zexample.txt b/data/prompts/human_instruction/Zexample.txt deleted file mode 100644 index fa5c6cdd..00000000 --- a/data/prompts/human_instruction/Zexample.txt +++ /dev/null @@ -1,177 +0,0 @@ -Following are some examples of the input and output. -Example 1: -""" -- Input: -runtime_world_state: -{ - "objects": [ - {"name": "parallel_box1", "properties": ["is_free", "is_equippable"]}, - {"name": "inward_claw", "properties": ["is_free", "is_equippable"]}, - {"name": "outward_claw", "properties": ["is_free"]}, - {"name": "gear1", "properties": []}, - {"name": "shaft1", "properties": []}, - {"name": "gearbase_hole1", "properties": []}, - {"name": "left_hand", "properties": []} - ], - "constraints": [ - {"source": "parallel_box1", "name": "can_manipulate", "target": "gear1"}, - {"source": "gear1", "name": "can_insert_to", "target": "shaft1"}, - ], - "relations": [ - {"source": "left_hand","name": "hold","target": "outward_claw"} - ] -} -last_behavior_tree: -{ - "summary": "selector to load parallel_box1", - "name": "selector: load_tool(left_hand, parallel_box1)", - "identifier": 0, - "children": [ - { - "summary": "condition node to check if left_hand holds parallel_box1", - "name": "target: hold(left_hand, parallel_box1)" - "identifier": 1, - }, - { - "summary": "sequence to load parallel_box1", - "name": "sequence: load_tool(left_hand, parallel_box1)", - "identifier": 2, - "children": [ - { - "summary": "condition node to check if parallel_box1 is equippable", - "name": "precondition: is_equippable(parallel_box1)" - "identifier": 3, - }, - { - "summary": "condition node to check if left_hand is free, - "name": "target: is_free(left_hand)" - "identifier": 4, - }, - { - "summary": "action node to equip parallel_box1 to left hand", - "name": "action: load_tool(left_hand, parallel_box1)", - "identifier": 5, - }, - ], - }, - ], -} -user_instruction: "insert gear1 into shaft1" - -- Output: -{ - "summary": "selector to load parallel_box1", - "name": "selector: load_tool(left_hand, parallel_box1)", - "identifier": 0, - "children": [ - { - "summary": "condition node to check if left_hand holds parallel_box1", - "name": "target: hold(left_hand, parallel_box1)" - "identifier": 1, - }, - { - "summary": "sequence to load parallel_box1", - "name": "sequence: load_tool(left_hand, parallel_box1)", - "identifier": 2, - "children": [ - { - "summary": "condition node to check if parallel_box1 is equippable", - "name": "precondition: is_equippable(parallel_box1)" - "identifier": 3, - }, - { - "summary": "selector to make left_hand unload outward_claw", - "name": "selector: unload_tool(left_hand, outward_claw)", - "children": [ - { - "summary": "condition node to check if left_hand is free, - "name": "target: is_free(left_hand)" - }, - { - "summary": "sequence to make left_hand unload outward_claw", - "name": "sequence: unload_tool(left_hand, outward_claw)", - "children": [ - { - "summary": "condition node to check if outward_claw is free", - "name": "precondition: is_free(outward_claw)" - }, - { - "summary": "condition node to check if left hand holds outward_claw", - "name": "precondition: hold(left_hand, outward_claw)" - }, - { - "summary": "action node to make left hand unload outward_claw", - "name": "action: unload_tool(left_hand, outward_claw)", - }, - ], - }, - ], - }, - { - "summary": "action node to equip parallel_box1 to left hand", - "name": "action: load_tool(left_hand, parallel_box1)", - "identifier": 5, - }, - ], - }, - ], -} -""" - -Example 2: -""" -- Input: -objects: -{ - "tool": ["inward_claw"], - "part": ["nut1", "bolt1"] - "hand": ["left_hand"] -} -runtime_world_state: -{ - "objects": [ - {"name": "inward_claw", "properties": ["is_free", "is_equippable"]}, - {"name": "nut1", "properties": []}, - {"name": "bolt1", "properties": []}, - {"name": "left_hand", "properties": ["is_free"]} - ], - "constraints": [ - {"source": "inward_claw", "name": "can_manipulate", "target": "nut1"}, - {"source": "nut1", "name": "can_screw_to", "target": "bolt1"}, - ], - "relations": [] -} -last_behavior_tree: None -last_failed_node: None -instructions: "screw nut1 into bolt1" - -- Output: -{ - "summary": "selector to use left_hand with inward_claw to screw nut1 into bolt1", - "name": "selector: screw(left_hand, inward_claw, nut1, bolt1)", - "children": [ - { - "summary": "condition node to check if nut1 is screwed into bolt1", - "name": "target: is_screwed_to(nut1, bolt1)" - }, - { - "summary": "sequence to use left_hand with inward_claw to screw nut1 into bolt1", - "name": "sequence: screw(left_hand, inward_claw, nut1, bolt1)", - "children": [ - { - "summary": "condition node to check if inward_claw holds nut1", - "name": "precondition: hold(inward_claw, nut1)" - }, - { - "summary": "condition node to check if nut1 can be screwed into bolt1", - "name": "precondition: can_screw_to(nut1, bolt1)" - }, - { - "summary": "action node to use left_hand with inward_claw to screw nut1 into bolt1", - "name": "action: screw(left_hand, inward_claw, nut1, bolt1)" - }, - ], - }, - ], -} -""" \ No newline at end of file diff --git a/data/prompts/human_instruction/Zobject.txt b/data/prompts/human_instruction/Zobject.txt deleted file mode 100644 index 2a4947ec..00000000 --- a/data/prompts/human_instruction/Zobject.txt +++ /dev/null @@ -1,7 +0,0 @@ -objects are given in a dictionary. -Its keys are the types and the values are lists of instances of that type. -An example: -""" -"part": ["gear", "nut", "bolt"] -""" -Refer to the domain knowledge to understand which predicates and actions are applicable to which types. \ No newline at end of file diff --git a/data/prompts/human_instruction/Zstate.txt b/data/prompts/human_instruction/Zstate.txt deleted file mode 100644 index 792f3ac5..00000000 --- a/data/prompts/human_instruction/Zstate.txt +++ /dev/null @@ -1,21 +0,0 @@ -Information about world states is in json dictionary format. Example: -""" -{ - "objects": [ - {"name": "leg1", "properties": []}, # leg1 is an object with no properties - {"name": "seat", "properties": []}, # seat is an object with no properties - {"name": "hand", "properties": ["is_free"]}, # hand is an object with a property "is_free" - ], - "constraints": [ - {"source": "leg1", "name": "can_screw_to", "target": "seat"}, # means leg1 can be screwed to seat - ], - "relations": [ - {"source": "leg1", "name": "is_screwed_to", "target": "seat"}, # means leg1 is screwed to seat - ], -} -""" -It contains three lists: - - "objects": the objects in the world and their properties. The properties can be changed by robot actions. - - "constraints": the constraint relations that are enforced and should not be changed. - - "relations": the relations in the world. They can be changed by robot actions. -The properties, constraints and relations should always align with the [PREDICATE_LIST] in domain knowledge. \ No newline at end of file diff --git a/data/prompts/human_instruction/behaviortree.txt b/data/prompts/human_instruction/behaviortree.txt deleted file mode 100644 index 4d79b87c..00000000 --- a/data/prompts/human_instruction/behaviortree.txt +++ /dev/null @@ -1,51 +0,0 @@ -[user] -The behavior tree should be in json dictionary format. -Following is a simple example: -""" -{ - "summary": "Selector to change the tool in the left hand from outwardgripper to defaultgripper", - "name": "selector: change_tool(left_hand, outwardgripper, defaultgripper)", - "children": [ - { - "summary": "the target is that the left hand is holding the default gripper", - "name": "target: hold(left_hand, defaultgripper)" - }, - { - "summary": "Sequence to change the tool in the left hand from outwardgripper to defaultgripper", - "name": "sequence: change_tool(left_hand, outwardgripper, defaultgripper)", - "children": [ - { - "summary": "A precondition is that the left hand is holding the outwardgripper", - "name": "precondition: hold(left_hand, outwardgripper)" - }, - { - "summary": "A precondition is that the outwardgripper is empty", - "name": "precondition: is_empty(outwardgripper)" - }, - { - "summary": "The action to change the tool in the left hand from outwardgripper to defaultgripper", - "name": "action: change_tool(left_hand, outwardgripper, defaultgripper)" - } - ] - } - ] -} -""" -1. Every node are described by a summary and a name: - - "summary": a short interpretive description of the node. - - "name": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are "selector", "sequence", "condition" ("target", "precondition"), "action". -2. The control flow nodes, namely "selector" and "sequence", can have "children", which is a list of children nodes. -3. The "condition" nodes that are children of "selectors" are targets. Those that are children of "sequences" are preconditions. -4. All the node should always have their corresponding actions/precidates in the domain knowledge. -5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state. -6. The basic structure template of a unit subtree is: - - A selector as the root node. - - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied. - - A sequence node as the last child of the selector node to achieve the target condition node. - - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions. - - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before. - - The action should have effects that can fulfill the target condition node in the selector. - -7. If any precondition node will be unsatisfied when it is ticked, a unit subtree should be constructed to replace it, which take that precondition node as its target node and aims to achieve it. You should estimate the world state when the precondition is ticked (checked) according to the initial state and the effects of the actions executed before, then do this replacement repeatedly until all the preconditions are satisfied at the time they are ticked. -8. The behavior tree should be constructed based on the action sequence you planned and the action definitions in ROBOT_ACTION_LIST. -9. Control flow nodes (sequence nodes and selector nodes) don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state. \ No newline at end of file diff --git a/data/prompts/human_instruction/domain.txt b/data/prompts/human_instruction/domain.txt deleted file mode 100644 index b67bcb08..00000000 --- a/data/prompts/human_instruction/domain.txt +++ /dev/null @@ -1,41 +0,0 @@ -Following is the domain knowledge you need, which includes: -- the object types in the world, -- the predicates for describing the states of the world, -- the actions that the robot can apply, with their precondtions and effects defined. - -The object types are explained as follows: -""" -thing: The base class for all objects in the world. Its subclasses are: hand, tool, part. -hand: The robot hand. A empty can only hold one at a time. -tool: The tool to manipulate . A empty can hold one at a time. -part: The part to be manipulated. -""" - -The precidates are explained as follows: -""" -"PREDICATE_LIST" -- is_empty(): is empty and can hold something. -- can_manipulate(, ): can be used to manipulate (manipulation constraint). This regulates the compatibility between and . -- can_insert_to(, ): can be inserted into (insertion assembly constraint) -- can_screw_to(, ): can be screwed into (screw assembly constraint) -- can_place_to(, ): can be placed into (placement assembly constraint) -- hold(, ): is holding . It can be holding or holding . -- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) -- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) -- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) -""" - -The actions are explained as follows: -""" -"ROBOT_ACTION_LIST" -- pick_up(, , ): use to pick up . As preconditions, should be holding and should be empty in order to hold the . The will be held by the as a result of this action. -- put_down(, , ): use to put down . As preconditions, should be holding and should be holding . The will be empty and will not hold the anymore as a result of this action. -- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding . -- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be placed into . -- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding . -- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be inserted into . -- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding . -- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be screwed into . -- change_tool(, , ): change the equipped tool from to in order to manipulate specific parts. As preconditions, should be holding and should be empty. -""" -the items in <> indicate the type of the instances needed for the predicates and actions. \ No newline at end of file diff --git a/data/prompts/human_instruction/readme.md b/data/prompts/human_instruction/readme.md deleted file mode 100644 index cbf3e891..00000000 --- a/data/prompts/human_instruction/readme.md +++ /dev/null @@ -1,2 +0,0 @@ -This is for human in the loop. -Instead of using simulation feedback, use human feedback to modify the tree. \ No newline at end of file diff --git a/data/prompts/human_instruction/system.txt b/data/prompts/human_instruction/system.txt deleted file mode 100644 index 7ebac297..00000000 --- a/data/prompts/human_instruction/system.txt +++ /dev/null @@ -1 +0,0 @@ -You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you construct robot task plans represented by behaiovr trees. \ No newline at end of file diff --git a/data/prompts/human_instruction/task.txt b/data/prompts/human_instruction/task.txt deleted file mode 100644 index 20ac660e..00000000 --- a/data/prompts/human_instruction/task.txt +++ /dev/null @@ -1,9 +0,0 @@ -The input are: -runtime_world_state: the world state when the last behaior tree stopped. -last_behavior_tree: the last executed behavior tree. -user_instruction: the instruction from the user to help you understand the failure reason. -action_sequence: the action_sequence that your behavior tree should align with. - -You analyze the failure reason with the help of user_instruction, then fix the behavior tree. -If no last_behavior_tree is given, you generate a behavior tree based on the action sequence. The behavior tree should follow the regulated format. -You only return the behavior tree as a json object that is ready to be parsed. \ No newline at end of file diff --git a/data/prompts/human_instruction/template.txt b/data/prompts/human_instruction/template.txt deleted file mode 100644 index 788be399..00000000 --- a/data/prompts/human_instruction/template.txt +++ /dev/null @@ -1,8 +0,0 @@ -Start working. Resume from the input below. ---------------- -last_behavior_tree: {last_behavior_tree} - -user_instruction: {user_instruction} - -action_sequence: {action_sequence} ---------------- diff --git a/data/prompts/new/readme.md b/data/prompts/new/readme.md deleted file mode 100644 index 19bb8ada..00000000 --- a/data/prompts/new/readme.md +++ /dev/null @@ -1,7 +0,0 @@ -This folder serves for generating the entire behavior tree in a recursively manner. The subfolders are: - -1. `action_sequence`: generate an action sequence based on world state and target. -2. `state_est`: estimate the state of the world after executing an action at the given initial state. -3. `unit_tree`: generate a unit tree for an action. - -For explanation see my thesis or the main readme file [TBD]. \ No newline at end of file diff --git a/data/prompts/new/seq_plan/chain.txt b/data/prompts/new/seq_plan/chain.txt deleted file mode 100644 index 9116e19d..00000000 --- a/data/prompts/new/seq_plan/chain.txt +++ /dev/null @@ -1,5 +0,0 @@ -You follow the work flow below: -1. analyze the world state. describe the current state of the world, including the objects, the objects' properties, the constraints and the relationships. -2. you find the goal state. -3. you find the action steps that can be performed to reach the goal state. -4. you format the actions and put them in a sequence to form the output plan. \ No newline at end of file diff --git a/data/prompts/new/seq_plan/domain_nl.txt b/data/prompts/new/seq_plan/domain_nl.txt deleted file mode 100644 index 6d0052bd..00000000 --- a/data/prompts/new/seq_plan/domain_nl.txt +++ /dev/null @@ -1,43 +0,0 @@ -Following is the domain knowledge you need, which includes: -- the object types in the world, -- the predicates for describing the states of the world, -- the actions that the robot can apply, with their precondtions and effects defined. - -The object types are explained as follows: -""" -Thing: The base class for all objects in the world. Its subclasses are: hand, tool, part. -hand: The robot hand. A empty can only hold one at a time. -tool: The tool to manipulate . A empty can hold one at a time. -part: The part to be manipulated. -""" - -The precidates are explained as follows: -""" -"PREDICATE_LIST" -- is_empty(): is empty and can hold something. The thing here can be or . -- is_equippable(): is ready and can be eqquipped by a robot hand. -- can_manipulate(, ): can be used to manipulate (manipulation constraint). This regulates the compatibility between and . -- can_insert_to(, ): can be inserted into (insertion assembly constraint) -- can_screw_to(, ): can be screwed into (screw assembly constraint) -- can_place_to(, ): can be placed into (placement assembly constraint) -- hold(, ): is holding . It can be holding or holding . -- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) -- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) -- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) -""" - -The actions are explained as follows: -""" -"ROBOT_ACTION_LIST" -- pick_up(, , ): use to pick up . As preconditions, should be holding and should be empty in order to hold the . The will be held by the as a result of this action. -- put_down(, , ): use to put down . As preconditions, should be holding and should be holding . The will be empty and will not hold the anymore as a result of this action. -- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding . -- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be placed into . -- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding . -- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be inserted into . -- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding . -- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be screwed into . -- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be empty. -- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be empty. -""" -the items in <> indicate the type of the instances needed for the predicates and actions. \ No newline at end of file diff --git a/data/prompts/new/seq_plan/domain_pddl copy.txt b/data/prompts/new/seq_plan/domain_pddl copy.txt deleted file mode 100644 index 33fbcd43..00000000 --- a/data/prompts/new/seq_plan/domain_pddl copy.txt +++ /dev/null @@ -1,84 +0,0 @@ -Following is the domain knowledge you need in pddl domain format, which includes: -- the object types in the world, -- the predicates for describing the states of the world, -- the actions that the robot can apply, with their precondtions and effects defined. -""" -(define (domain robot_assembly_problem-domain) - (:requirements :strips :typing) - (:types - thing - object - tool part hand - thing - ) - (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part)) - (:action pick_up - :parameters ( ?hand - hand ?tool - tool ?part - part) - :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part)) - :effect (and (hold ?tool ?part) (not (is_free ?tool)))) - (:action put_down - :parameters ( ?hand - hand ?tool - tool ?part - part) - :precondition (and (hold ?tool ?part) (hold ?hand ?tool)) - :effect (and (not (hold ?hand ?part)) (is_free ?tool))) - (:action place - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2))) - (:action detach - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2)))) - (:action insert - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2))) - (:action pull - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2)))) - (:action screw - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2))) - (:action unscrew - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2)))) - (:action load_tool - :parameters ( ?hand - hand ?tool - tool) - :precondition (and (is_equippable ?tool) (is_free ?hand)) - :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool))) - (:action unload_tool - :parameters ( ?hand - hand ?tool - tool) - :precondition (and (hold ?hand ?tool) (is_free ?tool)) - :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool)))) -) -""" -The precidates are explained as follows: -""" -"PREDICATE_LIST" -- is_free( or ): or is free and can hold something. -- is_equippable(): is ready to be eqquipped by a robot hand. -- can_manipulate(, ): can be used to manipulate (manipulation constraint) -- can_insert_to(, ): can be inserted into (insertion assembly constraint) -- can_screw_to(, ): can be screwed into (screw assembly constraint) -- can_place_to(, ): can be placed into (placement assembly constraint) -- hold(, ): is holding . It can be holding or holding . -- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) -- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) -- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) -""" - -The actions are explained as follows: -""" -"ROBOT_ACTION_LIST" -- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something. -- put_down(, , ): use to put down . As preconditions, should be holding and should be holding . -- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding . -- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into . -- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding . -- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into . -- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding . -- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into . -- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free. -- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free. -""" -the items in <> indicate the type of the instances needed for the predicates and actions. \ No newline at end of file diff --git a/data/prompts/new/seq_plan/domain_pddl.txt b/data/prompts/new/seq_plan/domain_pddl.txt deleted file mode 100644 index b91a7a64..00000000 --- a/data/prompts/new/seq_plan/domain_pddl.txt +++ /dev/null @@ -1,84 +0,0 @@ -Following is the domain knowledge you need in pddl domain format, which includes: -- the object types in the world, -- the predicates for describing the states of the world, -- the actions that the robot can apply, with their precondtions and effects defined. -""" -(define (domain robot_assembly_problem-domain) - (:requirements :strips :typing) - (:types - thing - object - tool part hand - thing - ) - (:predicates (is_empty ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part)) - (:action pick_up - :parameters ( ?hand - hand ?tool - tool ?part - part) - :precondition (and (is_empty ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part)) - :effect (and (hold ?tool ?part) (not (is_empty ?tool)))) - (:action put_down - :parameters ( ?hand - hand ?tool - tool ?part - part) - :precondition (and (hold ?tool ?part) (hold ?hand ?tool)) - :effect (and (not (hold ?hand ?part)) (is_empty ?tool))) - (:action place - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_empty ?tool) (is_placed_to ?part1 ?part2))) - (:action detach - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_empty ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2)) - :effect (and (hold ?tool ?part1) (not (is_empty ?tool)) (not (is_placed_to ?part1 ?part2)))) - (:action insert - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_empty ?tool) (is_inserted_to ?part1 ?part2))) - (:action pull - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_empty ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1)) - :effect (and (hold ?tool ?part1) (not (is_empty ?tool)) (not (is_inserted_to ?part1 ?part2)))) - (:action screw - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_empty ?tool) (is_screwed_to ?part1 ?part2))) - (:action unscrew - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_empty ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1)) - :effect (and (hold ?tool ?part1) (not (is_empty ?tool)) (not (is_screwed_to ?part1 ?part2)))) - (:action load_tool - :parameters ( ?hand - hand ?tool - tool) - :precondition (and (is_equippable ?tool) (is_empty ?hand)) - :effect (and (not (is_empty ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool))) - (:action unload_tool - :parameters ( ?hand - hand ?tool - tool) - :precondition (and (hold ?hand ?tool) (is_empty ?tool)) - :effect (and (is_empty ?hand) (is_equippable ?tool) (not (hold ?hand ?tool)))) -) -""" -The precidates are explained as follows: -""" -"PREDICATE_LIST" -- is_empty( or ): or is empty and can hold something. -- is_equippable(): is ready to be eqquipped by a robot hand. -- can_manipulate(, ): can be used to manipulate (manipulation constraint) -- can_insert_to(, ): can be inserted into (insertion assembly constraint) -- can_screw_to(, ): can be screwed into (screw assembly constraint) -- can_place_to(, ): can be placed into (placement assembly constraint) -- hold(, ): is holding . It can be holding or holding . -- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) -- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) -- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) -""" - -The actions are explained as follows: -""" -"ROBOT_ACTION_LIST" -- pick_up(, , ): use to pick up . As preconditions, should be holding and should be empty to hold something. -- put_down(, , ): use to put down . As preconditions, should be holding and should be holding . -- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding . -- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be placed into . -- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding . -- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be inserted into . -- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding . -- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be screwed into . -- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be empty. -- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be empty. -""" -the items in <> indicate the type of the instances needed for the predicates and actions. \ No newline at end of file diff --git a/data/prompts/new/seq_plan/example.txt b/data/prompts/new/seq_plan/example.txt deleted file mode 100644 index 20d43a11..00000000 --- a/data/prompts/new/seq_plan/example.txt +++ /dev/null @@ -1,26 +0,0 @@ -Following is an input/output example: -input: -''' -start_world_state: -{ -"objects": [ - {"name": "outward_claw", "properties": ["is_empty", "is_equippable"]}, - {"name": "shaft1", "properties": []}, - {"name": "left_hand", "properties": ["is_empty"]} -], -"constraints": [ - {"source": "outward_claw", "name": "can_manipulate", "target": "gear1"} -], -"relations": [] -} -target: "hold(left_hand, outward_claw)" -''' -output: -''' -{ - "explanation": "we need to load the outward_claw into the left_hand to make the left hand hold it. The outward_claw is empty and equippable, and the left_hand is empty. So we can load the outward_claw into the left_hand.", - "task_plan": [ - "load_tool(left_hand, outward_claw)" - ] -} -''' \ No newline at end of file diff --git a/data/prompts/new/seq_plan/new_domain_nl.txt b/data/prompts/new/seq_plan/new_domain_nl.txt deleted file mode 100644 index b67bcb08..00000000 --- a/data/prompts/new/seq_plan/new_domain_nl.txt +++ /dev/null @@ -1,41 +0,0 @@ -Following is the domain knowledge you need, which includes: -- the object types in the world, -- the predicates for describing the states of the world, -- the actions that the robot can apply, with their precondtions and effects defined. - -The object types are explained as follows: -""" -thing: The base class for all objects in the world. Its subclasses are: hand, tool, part. -hand: The robot hand. A empty can only hold one at a time. -tool: The tool to manipulate . A empty can hold one at a time. -part: The part to be manipulated. -""" - -The precidates are explained as follows: -""" -"PREDICATE_LIST" -- is_empty(): is empty and can hold something. -- can_manipulate(, ): can be used to manipulate (manipulation constraint). This regulates the compatibility between and . -- can_insert_to(, ): can be inserted into (insertion assembly constraint) -- can_screw_to(, ): can be screwed into (screw assembly constraint) -- can_place_to(, ): can be placed into (placement assembly constraint) -- hold(, ): is holding . It can be holding or holding . -- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) -- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) -- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) -""" - -The actions are explained as follows: -""" -"ROBOT_ACTION_LIST" -- pick_up(, , ): use to pick up . As preconditions, should be holding and should be empty in order to hold the . The will be held by the as a result of this action. -- put_down(, , ): use to put down . As preconditions, should be holding and should be holding . The will be empty and will not hold the anymore as a result of this action. -- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding . -- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be placed into . -- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding . -- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be inserted into . -- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding . -- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be screwed into . -- change_tool(, , ): change the equipped tool from to in order to manipulate specific parts. As preconditions, should be holding and should be empty. -""" -the items in <> indicate the type of the instances needed for the predicates and actions. \ No newline at end of file diff --git a/data/prompts/new/seq_plan/new_example.txt b/data/prompts/new/seq_plan/new_example.txt deleted file mode 100644 index 2b58c241..00000000 --- a/data/prompts/new/seq_plan/new_example.txt +++ /dev/null @@ -1,26 +0,0 @@ -Following is an input/output example: -INPUT: -start_world_state: -{ -"objects": [ - {"name": "outward_claw", "properties": ["is_empty"]}, - {"name": "shaft1", "properties": []}, - {"name": "left_hand", "properties": []}, - {"name": "parallel_box1", "properties": [is_empty]} -], -"constraints": [ - {"source": "outward_claw", "name": "can_manipulate", "target": "gear1"} -], -"relations": [ - {"source": "left_hand", "name": "hold", "target": "parallel_box1"} -] -} -target: "change the tool from the parallel_box1 to the outward_claw" - -OUTPUT: -{ - "explanation": "The left_hand is holding the parallel_box1 now and the parallel_box1 is empty, so the change_tool action can be conducted to change the tool in the hand from the parallel_box1 to outward_claw.", - "task_plan": [ - "change_tool(left_hand, parallel_box1, outward_claw)" - ] -} \ No newline at end of file diff --git a/data/prompts/new/seq_plan/output_format.txt b/data/prompts/new/seq_plan/output_format.txt deleted file mode 100644 index 05d8c8ee..00000000 --- a/data/prompts/new/seq_plan/output_format.txt +++ /dev/null @@ -1,3 +0,0 @@ -The output json object has the following keys. -- "task_plan": Contains a list of robot actions in the defined form. Only the actions defined in the "ROBOT_ACTION LIST" will be used. The actions should be in pddl action format. -- "explanation": A string that explains your reasoning process briefly. \ No newline at end of file diff --git a/data/prompts/new/seq_plan/state.txt b/data/prompts/new/seq_plan/state.txt deleted file mode 100644 index 4b315a03..00000000 --- a/data/prompts/new/seq_plan/state.txt +++ /dev/null @@ -1,21 +0,0 @@ -World states should be in json dictionary format. Example: -""" -{ - "objects": [ - {"name": "leg1", "properties": []}, # leg1 is an object with no properties - {"name": "seat", "properties": []}, # seat is an object with no properties - {"name": "hand", "properties": ["is_empty"]} # hand is an object with a property "is_empty" - ], - "constraints": [ - {"source": "leg1", "name": "can_screw_to", "target": "seat"} # means leg1 can be screwed to seat - ], - "relations": [ - {"source": "leg1", "name": "is_screwed_to", "target": "seat"} # means leg1 is screwed to seat - ] -} -""" -It contains three lists: - - "objects": the objects in the world and their properties. The properties can be changed by robot actions. - - "constraints": the constraint relations that are regulated and can not be changed. - - "relations": the relations in the world. They can be changed by robot actions. -The properties, constraints and relations should always align with the [PREDICATE_LIST] in the domain knowledge. \ No newline at end of file diff --git a/data/prompts/new/seq_plan/system.txt b/data/prompts/new/seq_plan/system.txt deleted file mode 100644 index c0b37f9f..00000000 --- a/data/prompts/new/seq_plan/system.txt +++ /dev/null @@ -1 +0,0 @@ -You are an expert in making plans for robotic assembly tasks. \ No newline at end of file diff --git a/data/prompts/new/seq_plan/task.txt b/data/prompts/new/seq_plan/task.txt deleted file mode 100644 index bd444795..00000000 --- a/data/prompts/new/seq_plan/task.txt +++ /dev/null @@ -1,8 +0,0 @@ -The input are: -start_world_state: the world state when the behavior tree started. -target: the goal that the plan should achieve. - -You summarize the target into a goal world state. -You check if the target has been fulfilled. If so, return your explanation and an empty task plan in one json object. -If the target has not been fulfilled, you make a bullet list task plan to achieve it from the start_world_state. Use the definitions in the domain knowledge to construct actions. Do not add any superfluous steps. Then you return the task plan and your explanation in one json object. -You only output an json object, which is ready to be parsed directly. Any other superfluous output is forbidden. \ No newline at end of file diff --git a/data/prompts/new/seq_plan/template.txt b/data/prompts/new/seq_plan/template.txt deleted file mode 100644 index ccd1c78b..00000000 --- a/data/prompts/new/seq_plan/template.txt +++ /dev/null @@ -1,3 +0,0 @@ -Now start to work with the following inputs: -start_world_state: {start_world_state} -target: {target} \ No newline at end of file diff --git a/data/prompts/new/state_est/chain.txt b/data/prompts/new/state_est/chain.txt deleted file mode 100644 index 6c94f571..00000000 --- a/data/prompts/new/state_est/chain.txt +++ /dev/null @@ -1,5 +0,0 @@ -This is the workflow you should follow: -1. Take the actions from the input. -2. For each action, think about the possible effects it could have on the world state according to the domain knowledge. -3. Update the world state according to the effects of the actions in the provided order. -4. Return the updated world state. \ No newline at end of file diff --git a/data/prompts/new/state_est/domain.txt b/data/prompts/new/state_est/domain.txt deleted file mode 100644 index 65b6a2df..00000000 --- a/data/prompts/new/state_est/domain.txt +++ /dev/null @@ -1,84 +0,0 @@ -Following is the domain knowledge you need in pddl domain format, which includes: -- the object types in the world, -- the predicates for describing the states of the world, -- the actions that the robot can apply, with their precondtions and effects defined. -""" -(define (domain robot_assembly_problem-domain) - (:requirements :strips :typing) - (:types - thing - object - tool part hand - thing - ) - (:predicates (is_empty ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part)) - (:action pick_up - :parameters ( ?hand - hand ?tool - tool ?part - part) - :precondition (and (is_empty ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part)) - :effect (and (hold ?tool ?part) (not (is_empty ?tool)))) - (:action put_down - :parameters ( ?hand - hand ?tool - tool ?part - part) - :precondition (and (hold ?tool ?part) (hold ?hand ?tool)) - :effect (and (not (hold ?hand ?part)) (is_empty ?tool))) - (:action place - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_empty ?tool) (is_placed_to ?part1 ?part2))) - (:action detach - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_empty ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2)) - :effect (and (hold ?tool ?part1) (not (is_empty ?tool)) (not (is_placed_to ?part1 ?part2)))) - (:action insert - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_empty ?tool) (is_inserted_to ?part1 ?part2))) - (:action pull - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_empty ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1)) - :effect (and (hold ?tool ?part1) (not (is_empty ?tool)) (not (is_inserted_to ?part1 ?part2)))) - (:action screw - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_empty ?tool) (is_screwed_to ?part1 ?part2))) - (:action unscrew - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_empty ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1)) - :effect (and (hold ?tool ?part1) (not (is_empty ?tool)) (not (is_screwed_to ?part1 ?part2)))) - (:action load_tool - :parameters ( ?hand - hand ?tool - tool) - :precondition (and (is_equippable ?tool) (is_empty ?hand)) - :effect (and (not (is_empty ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool))) - (:action unload_tool - :parameters ( ?hand - hand ?tool - tool) - :precondition (and (hold ?hand ?tool) (is_empty ?tool)) - :effect (and (is_empty ?hand) (is_equippable ?tool) (not (hold ?hand ?tool)))) -) -""" -The precidates are explained as follows: -""" -"PREDICATE_LIST" -- is_empty( or ): or is free and can hold something. (default: true) -- is_equippable(): is ready to be eqquipped by a robot hand. (default: true) -- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false) -- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false) -- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false) -- can_place_to(, ): can be placed into (placement assembly constraint) (default: false) -- hold(, ): is holding (default: false) -- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false) -- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false) -- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false) -""" - -The actions are explained as follows: -""" -"ROBOT_ACTION_LIST" -- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something. -- put_down(, , ): use to put down . As preconditions, should be holding and should be holding . -- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding . -- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into . -- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding . -- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into . -- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding . -- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into . -- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free. -- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free. -""" -the items in <> indicate the type of the instances needed for the predicates and actions. \ No newline at end of file diff --git a/data/prompts/new/state_est/new_domain_nl.txt b/data/prompts/new/state_est/new_domain_nl.txt deleted file mode 100644 index b67bcb08..00000000 --- a/data/prompts/new/state_est/new_domain_nl.txt +++ /dev/null @@ -1,41 +0,0 @@ -Following is the domain knowledge you need, which includes: -- the object types in the world, -- the predicates for describing the states of the world, -- the actions that the robot can apply, with their precondtions and effects defined. - -The object types are explained as follows: -""" -thing: The base class for all objects in the world. Its subclasses are: hand, tool, part. -hand: The robot hand. A empty can only hold one at a time. -tool: The tool to manipulate . A empty can hold one at a time. -part: The part to be manipulated. -""" - -The precidates are explained as follows: -""" -"PREDICATE_LIST" -- is_empty(): is empty and can hold something. -- can_manipulate(, ): can be used to manipulate (manipulation constraint). This regulates the compatibility between and . -- can_insert_to(, ): can be inserted into (insertion assembly constraint) -- can_screw_to(, ): can be screwed into (screw assembly constraint) -- can_place_to(, ): can be placed into (placement assembly constraint) -- hold(, ): is holding . It can be holding or holding . -- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) -- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) -- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) -""" - -The actions are explained as follows: -""" -"ROBOT_ACTION_LIST" -- pick_up(, , ): use to pick up . As preconditions, should be holding and should be empty in order to hold the . The will be held by the as a result of this action. -- put_down(, , ): use to put down . As preconditions, should be holding and should be holding . The will be empty and will not hold the anymore as a result of this action. -- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding . -- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be placed into . -- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding . -- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be inserted into . -- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding . -- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be screwed into . -- change_tool(, , ): change the equipped tool from to in order to manipulate specific parts. As preconditions, should be holding and should be empty. -""" -the items in <> indicate the type of the instances needed for the predicates and actions. \ No newline at end of file diff --git a/data/prompts/new/state_est/new_example.txt b/data/prompts/new/state_est/new_example.txt deleted file mode 100644 index f54d4065..00000000 --- a/data/prompts/new/state_est/new_example.txt +++ /dev/null @@ -1,45 +0,0 @@ -Following is an input/output example: -INPUT: -start_world_state: -{ -"objects": [ - {"name": "outwardgripper", "properties": ["is_empty"]}, - {"name": "shaft1", "properties": []}, - {"name": "hole1", "properties": []}, - {"name": "defaultgripper", "properties": ["is_empty"]}, - {"name": "left_hand", "properties": []} -], -"constraints": [ - {"source": "outwardgripper", "name": "can_manipulate", "target": "shaft1"}, - {"source": "shaft1", "name": "can_insert_to", "target": "hole1"} -], -"relations": [ - {"source": "left_hand", "name": "hold", "target": "defaultgripper"} -] -} -action_plan: -[ - "change_tool(left_hand, defaultgripper, outwardgripper)", - "pick_up(left_hand, outwardgripper, shaft1)", - "insert(left_hand, outwardgripper, gear1, hole1)", -] -OUTPUT: -{ - "estimated_world_state": { - "objects": [ - {"name": "outwardgripper", "properties": ["is_empty"]}, - {"name": "shaft1", "properties": []}, - {"name": "hole1", "properties": []}, - {"name": "defaultgripper", "properties": ["is_empty"]}, - {"name": "left_hand", "properties": []} - ], - "constraints": [ - {"source": "outwardgripper", "name": "can_manipulate", "target": "shaft1"}, - {"source": "shaft1", "name": "can_insert_to", "target": "hole1"} - ], - "relations": [ - {"source": "left_hand", "name": "hold", "target": "outwardgripper"} - {"source": "shaft1", "name": "is_inserted_to", "target": "hole1"} - ] - } -} \ No newline at end of file diff --git a/data/prompts/new/state_est/output_format.txt b/data/prompts/new/state_est/output_format.txt deleted file mode 100644 index ccf11c35..00000000 --- a/data/prompts/new/state_est/output_format.txt +++ /dev/null @@ -1,2 +0,0 @@ -The output json object has the following keys. -- "estimated_world_state": The world state after executing the plan. diff --git a/data/prompts/new/state_est/state.txt b/data/prompts/new/state_est/state.txt deleted file mode 100644 index 15dd521a..00000000 --- a/data/prompts/new/state_est/state.txt +++ /dev/null @@ -1,21 +0,0 @@ -World states should be in json dictionary format. Example: -""" -{ - "objects": [ - {"name": "leg1", "properties": []}, # leg1 is an object with no properties - {"name": "seat", "properties": []}, # seat is an object with no properties - {"name": "tool1", "properties": ["is_empty"]} # hand is an object with a property "is_free" - ], - "constraints": [ - {"source": "leg1", "name": "can_screw_to", "target": "seat"} # means leg1 can be screwed to seat - ], - "relations": [ - {"source": "leg1", "name": "is_screwed_to", "target": "seat"} # means leg1 is screwed to seat - ] -} -""" -It contains three lists: - - "objects": the objects in the world and their properties. The properties can be changed by robot actions. - - "constraints": the constraint relations that are regulated and can not be changed. - - "relations": the relations in the world. They can be changed by robot actions. -The properties, constraints and relations should always align with the [PREDICATE_LIST] in the domain knowledge. \ No newline at end of file diff --git a/data/prompts/new/state_est/system.txt b/data/prompts/new/state_est/system.txt deleted file mode 100644 index 61593243..00000000 --- a/data/prompts/new/state_est/system.txt +++ /dev/null @@ -1 +0,0 @@ -You are an expert in estimating world state. \ No newline at end of file diff --git a/data/prompts/new/state_est/task.txt b/data/prompts/new/state_est/task.txt deleted file mode 100644 index 95ad05ad..00000000 --- a/data/prompts/new/state_est/task.txt +++ /dev/null @@ -1,6 +0,0 @@ -The input are: -start_world_state: the world state when the behavior tree started. -action_plan: a sequence of action to be performed. - -You estimate the world state after executing the task plan. -You return the estimated world state in a json object. diff --git a/data/prompts/new/state_est/template.txt b/data/prompts/new/state_est/template.txt deleted file mode 100644 index cf62bdac..00000000 --- a/data/prompts/new/state_est/template.txt +++ /dev/null @@ -1,3 +0,0 @@ -Now make a plan with the following information: -start_world_state: {start_world_state} -action_plan: {action_plan} \ No newline at end of file diff --git a/data/prompts/new/ut_gen/behaviortree.txt b/data/prompts/new/ut_gen/behaviortree.txt deleted file mode 100644 index a1a67ca9..00000000 --- a/data/prompts/new/ut_gen/behaviortree.txt +++ /dev/null @@ -1,44 +0,0 @@ -The behavior tree should be in json dictionary format. -When building a unit subtree, follow these rules: -1. Every node are described by a summary and a name: - - "summary": a short interpretive description of the node. - - "name": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are "selector", "sequence", "condition" ("target", "precondition"), "action". -2. The control flow nodes, namely "selector" and "sequence", can have "children", which is a list of children nodes. -3. The "condition" nodes that are children of "selectors" are targets. Those that are children of "sequences" are preconditions. -4. All the node should always have their corresponding actions/precidates in the domain knowledge. -5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state. -6. The basic structure template of a unit subtree is: - - A selector as the root node. - - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied. - - A sequence node as the last child of the selector node to achieve the target condition node. - - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions. - - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before. - - The action should have effects that can fulfill the target condition node in the selector. -Following is an example: -action: unload_tool(left_hand, outwardgripper) -unit subtree: -{ - "summary": "Selector to unload the outwardgripper in the left hand", - "name": "selector: unload_tool(left_hand, outwardgripper)" - "children": [ - { - "summary": "Check the target that the left hand is empty", - "name": "target: is_empty(left_hand)" - }, - { - "summary": "Sequence to unload the tool in the left hand", - "name": "sequence: unload_tool(left_hand, outwardgripper)", - "children": [ - { - "summary": "check the precondition that the left hand is holding a outwardgripper", - "name": "precondition: hold(left_hand, outwardgripper)" - }, - { - "summary": "Unload the outwardgripper in the left hand", - "name": "action: unload_tool(left_hand, outwardgripper)" - } - ] - } - ] -} - diff --git a/data/prompts/new/ut_gen/domain.txt b/data/prompts/new/ut_gen/domain.txt deleted file mode 100644 index a2c0847f..00000000 --- a/data/prompts/new/ut_gen/domain.txt +++ /dev/null @@ -1,84 +0,0 @@ -Following is the domain knowledge you need in pddl domain format, which includes: -- the object types in the world, -- the predicates for describing the states of the world, -- the actions that the robot can apply, with their precondtions and effects defined. -""" -(define (domain robot_assembly_problem-domain) - (:requirements :strips :typing) - (:types - thing - object - tool part hand - thing - ) - (:predicates (is_empty ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part)) - (:action pick_up - :parameters ( ?hand - hand ?tool - tool ?part - part) - :precondition (and (is_empty ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part)) - :effect (and (hold ?tool ?part) (not (is_empty ?tool)))) - (:action put_down - :parameters ( ?hand - hand ?tool - tool ?part - part) - :precondition (and (hold ?tool ?part) (hold ?hand ?tool)) - :effect (and (not (hold ?hand ?part)) (is_empty ?tool))) - (:action place - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_empty ?tool) (is_placed_to ?part1 ?part2))) - (:action detach - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_empty ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2)) - :effect (and (hold ?tool ?part1) (not (is_empty ?tool)) (not (is_placed_to ?part1 ?part2)))) - (:action insert - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_empty ?tool) (is_inserted_to ?part1 ?part2))) - (:action pull - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_empty ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1)) - :effect (and (hold ?tool ?part1) (not (is_empty ?tool)) (not (is_inserted_to ?part1 ?part2)))) - (:action screw - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_empty ?tool) (is_screwed_to ?part1 ?part2))) - (:action unscrew - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_empty ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1)) - :effect (and (hold ?tool ?part1) (not (is_empty ?tool)) (not (is_screwed_to ?part1 ?part2)))) - (:action load_tool - :parameters ( ?hand - hand ?tool - tool) - :precondition (and (is_equippable ?tool) (is_empty ?hand)) - :effect (and (not (is_empty ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool))) - (:action unload_tool - :parameters ( ?hand - hand ?tool - tool) - :precondition (and (hold ?hand ?tool) (is_empty ?tool)) - :effect (and (is_empty ?hand) (is_equippable ?tool) (not (hold ?hand ?tool)))) -) -""" -The precidates are explained as follows: -""" -"PREDICATE_LIST" -- is_empty( or ): or is empty and can hold something. (default: true) -- is_equippable(): is ready to be eqquipped by a robot hand. (default: true) -- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false) -- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false) -- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false) -- can_place_to(, ): can be placed into (placement assembly constraint) (default: false) -- hold(, ): is holding (default: false) -- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false) -- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false) -- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false) -""" - -The actions are explained as follows: -""" -"ROBOT_ACTION_LIST" -- pick_up(, , ): use to pick up . As preconditions, should be holding and should be empty to hold something. -- put_down(, , ): use to put down . As preconditions, should be holding and should be holding . -- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding . -- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be placed into . -- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding . -- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be inserted into . -- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding . -- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be screwed into . -- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be empty. -- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be empty. -""" -the items in <> indicate the type of the instances needed for the predicates and actions. \ No newline at end of file diff --git a/data/prompts/new/ut_gen/new_behaviortree.txt b/data/prompts/new/ut_gen/new_behaviortree.txt deleted file mode 100644 index 1c44921e..00000000 --- a/data/prompts/new/ut_gen/new_behaviortree.txt +++ /dev/null @@ -1,48 +0,0 @@ -The behavior tree should be in json dictionary format. -When building a unit subtree, follow these rules: -1. Every node are described by a summary and a name: - - "summary": a short interpretive description of the node. - - "name": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are "selector", "sequence", "condition" ("target", "precondition"), "action". -2. The control flow nodes, namely "selector" and "sequence", can have "children", which is a list of children nodes. -3. The "condition" nodes that are children of "selectors" are targets. Those that are children of "sequences" are preconditions. -4. All the node should always have their corresponding actions/precidates in the domain knowledge. -5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state. -6. The basic structure template of a unit subtree is: - - A selector as the root node. - - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied. - - A sequence node as the last child of the selector node to achieve the target condition node. - - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions. - - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before. - - The action should have effects that can fulfill the target condition node in the selector. -Following is an example: -action: change_tool(left_hand, outwardgripper, defaultgripper) -unit subtree: -{ - "summary": "Selector to change the tool in the left hand from outwardgripper to defaultgripper", - "name": "selector: change_tool(left_hand, outwardgripper, defaultgripper)" - "children": [ - { - "summary": "The target is to make the left hand hold the default gripper", - "name": "target: hold(left_hand, defaultgripper)" - }, - { - "summary": "Sequence to change the tool in the left hand from outwardgripper to defaultgripper", - "name": "sequence: change_tool(left_hand, outwardgripper, defaultgripper)", - "children": [ - { - "summary": "A precondition is that the left hand is holding the outwardgripper", - "name": "precondition: hold(left_hand, outwardgripper)" - }, - { - "summary": "A precondition is that the outwardgripper is empty", - "name": "precondition: is_empty(outwardgripper)" - }, - { - "summary": "Take the action to change the tool in the left hand from outwardgripper to defaultgripper", - "name": "action: change_tool(left_hand, outwardgripper, defaultgripper)" - } - ] - } - ] -} - diff --git a/data/prompts/new/ut_gen/new_domain_nl.txt b/data/prompts/new/ut_gen/new_domain_nl.txt deleted file mode 100644 index b67bcb08..00000000 --- a/data/prompts/new/ut_gen/new_domain_nl.txt +++ /dev/null @@ -1,41 +0,0 @@ -Following is the domain knowledge you need, which includes: -- the object types in the world, -- the predicates for describing the states of the world, -- the actions that the robot can apply, with their precondtions and effects defined. - -The object types are explained as follows: -""" -thing: The base class for all objects in the world. Its subclasses are: hand, tool, part. -hand: The robot hand. A empty can only hold one at a time. -tool: The tool to manipulate . A empty can hold one at a time. -part: The part to be manipulated. -""" - -The precidates are explained as follows: -""" -"PREDICATE_LIST" -- is_empty(): is empty and can hold something. -- can_manipulate(, ): can be used to manipulate (manipulation constraint). This regulates the compatibility between and . -- can_insert_to(, ): can be inserted into (insertion assembly constraint) -- can_screw_to(, ): can be screwed into (screw assembly constraint) -- can_place_to(, ): can be placed into (placement assembly constraint) -- hold(, ): is holding . It can be holding or holding . -- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) -- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) -- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) -""" - -The actions are explained as follows: -""" -"ROBOT_ACTION_LIST" -- pick_up(, , ): use to pick up . As preconditions, should be holding and should be empty in order to hold the . The will be held by the as a result of this action. -- put_down(, , ): use to put down . As preconditions, should be holding and should be holding . The will be empty and will not hold the anymore as a result of this action. -- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding . -- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be placed into . -- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding . -- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be inserted into . -- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding . -- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be screwed into . -- change_tool(, , ): change the equipped tool from to in order to manipulate specific parts. As preconditions, should be holding and should be empty. -""" -the items in <> indicate the type of the instances needed for the predicates and actions. \ No newline at end of file diff --git a/data/prompts/new/ut_gen/system.txt b/data/prompts/new/ut_gen/system.txt deleted file mode 100644 index d76b391a..00000000 --- a/data/prompts/new/ut_gen/system.txt +++ /dev/null @@ -1 +0,0 @@ -You are an excellent behavior tree builder for robotic assembly tasks. Given an action, you construct a unit subtree for the action based on its definition in the domain knowledge. \ No newline at end of file diff --git a/data/prompts/new/ut_gen/task.txt b/data/prompts/new/ut_gen/task.txt deleted file mode 100644 index 2a3a4062..00000000 --- a/data/prompts/new/ut_gen/task.txt +++ /dev/null @@ -1,6 +0,0 @@ -The input are: -action: The action that you construct the unit subtree from. - -You analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it. -The unit subtree should follow the regulated format. -You only return the unit subtree as a json object. \ No newline at end of file diff --git a/data/prompts/new/ut_gen/template.txt b/data/prompts/new/ut_gen/template.txt deleted file mode 100644 index 2182ebce..00000000 --- a/data/prompts/new/ut_gen/template.txt +++ /dev/null @@ -1,4 +0,0 @@ -Resume from the input below. ---------------- -action: {action} ---------------- diff --git a/data/prompts/plan_updater/domain.txt b/data/prompts/plan_updater/domain.txt deleted file mode 100644 index 2ced4acf..00000000 --- a/data/prompts/plan_updater/domain.txt +++ /dev/null @@ -1,17 +0,0 @@ -You should only use formal assembly actions as the action in the steps. -The available assembly actions are used to achieve the assembly constraints between two parts. -The assembly actions are: -- insert something into something -- place something onto something -- screw something into something - -You should also refer to the constraints in the world state, which provides the assembly constraints between the parts. - -Example: -user_input: "you should assembly the red block with the blue hole." -in world state's constraint: can_insert_into(red_block, blue_hole) -plan: "insert the red block into the blue hole" - -Example: -user_input: "intert the blub into the base, then put the cover onto it." -plan: ["insert the blub into the base", "put the cover onto the blub"] \ No newline at end of file diff --git a/data/prompts/plan_updater/readme.md b/data/prompts/plan_updater/readme.md deleted file mode 100644 index ec1bd528..00000000 --- a/data/prompts/plan_updater/readme.md +++ /dev/null @@ -1,3 +0,0 @@ -This is a plan updater prompt for updating the assembly plan after the execution of one step. - -It should remove the executed action. Currently it just returns the updated plan. \ No newline at end of file diff --git a/data/prompts/plan_updater/system.txt b/data/prompts/plan_updater/system.txt deleted file mode 100644 index b2b897ae..00000000 --- a/data/prompts/plan_updater/system.txt +++ /dev/null @@ -1 +0,0 @@ -You are an assistant to make plans for robotic assembly tasks. \ No newline at end of file diff --git a/data/prompts/plan_updater/task copy 2.txt b/data/prompts/plan_updater/task copy 2.txt deleted file mode 100644 index 32df086f..00000000 --- a/data/prompts/plan_updater/task copy 2.txt +++ /dev/null @@ -1,16 +0,0 @@ -You are provided with: -- a user instruction -- an original robotic assembly plan made based on the user instruction. -- the current world state -- the steps in the original plan that have been executed and their result. - -You update the original plan, remove the unnecessary steps if their targets have been achieved, -and add the steps that still need to be done to achieve the objective defined by the user instruction. -Then you return the updated plan. - -Only add steps to the plan that still NEED to be done. Do not return previously done steps as part of the plan. -Do not add any superfluous steps. -Make sure that each step has all the information needed - do not skip steps. - -If the user instruction has been accomplished, respond to the user for more input with explanation in the response. -If the you some past steps failed, and you cannot find a way to achieve the user instruction, respond to the user for more information with a question in the response. \ No newline at end of file diff --git a/data/prompts/plan_updater/task copy.txt b/data/prompts/plan_updater/task copy.txt deleted file mode 100644 index 7d673116..00000000 --- a/data/prompts/plan_updater/task copy.txt +++ /dev/null @@ -1,16 +0,0 @@ -You are provided with: -- a user instruction -- an original robotic assembly plan made based on the user instruction. -- the current world state -- the steps in the original plan that have been executed and their result. - -You update the original plan, remove the unnecessary steps if their targets have been achieved, -and add the steps that still need to be done to achieve the objective defined by the user instruction. -Then you return the updated plan. - -Do not add any superfluous steps. -Make sure that each step has all the information needed - do not skip steps. - -Read the world state. Check the original plan from the first step and update your plan with the steps need to be conducted next according to the world state. -If no more steps are needed to achieve the objective defined by user input, respond to the user for more input. Otherwise, fill out the plan. -Only add steps to the plan that still NEED to be done. Do not return previously done steps as part of the plan. diff --git a/data/prompts/plan_updater/task.txt b/data/prompts/plan_updater/task.txt deleted file mode 100644 index a0f60f38..00000000 --- a/data/prompts/plan_updater/task.txt +++ /dev/null @@ -1,15 +0,0 @@ -You are provided with: -- a user instruction -- an original robotic assembly plan made based on the user instruction. -- the current world state -- the steps in the original plan that have been executed and their result. - -You update the original plan, remove the unnecessary steps if their targets have been achieved, -and add the steps that still need to be done to achieve the objective defined by the user instruction. -Then you return the updated plan. - -Only add steps to the plan that still NEED to be done. Do not return previously done steps as part of the plan. -Do not add any superfluous steps. -Make sure that each step has all the information needed - do not skip steps. - -If the user instruction has been accomplished, return an empty plan list. \ No newline at end of file diff --git a/data/prompts/plan_updater/template.txt b/data/prompts/plan_updater/template.txt deleted file mode 100644 index ebd101e0..00000000 --- a/data/prompts/plan_updater/template.txt +++ /dev/null @@ -1,12 +0,0 @@ -Now start to work with the following information: -The user input is: -{user_input} - -The origianal plan is: -{plan} - -The steps that have been executed and their result: -{past_steps} - -The world state is: -{world_state} \ No newline at end of file diff --git a/data/prompts/planner/domain.txt b/data/prompts/planner/domain.txt deleted file mode 100644 index 2ced4acf..00000000 --- a/data/prompts/planner/domain.txt +++ /dev/null @@ -1,17 +0,0 @@ -You should only use formal assembly actions as the action in the steps. -The available assembly actions are used to achieve the assembly constraints between two parts. -The assembly actions are: -- insert something into something -- place something onto something -- screw something into something - -You should also refer to the constraints in the world state, which provides the assembly constraints between the parts. - -Example: -user_input: "you should assembly the red block with the blue hole." -in world state's constraint: can_insert_into(red_block, blue_hole) -plan: "insert the red block into the blue hole" - -Example: -user_input: "intert the blub into the base, then put the cover onto it." -plan: ["insert the blub into the base", "put the cover onto the blub"] \ No newline at end of file diff --git a/data/prompts/planner/system.txt b/data/prompts/planner/system.txt deleted file mode 100644 index c0b37f9f..00000000 --- a/data/prompts/planner/system.txt +++ /dev/null @@ -1 +0,0 @@ -You are an expert in making plans for robotic assembly tasks. \ No newline at end of file diff --git a/data/prompts/planner/task.txt b/data/prompts/planner/task.txt deleted file mode 100644 index 16327668..00000000 --- a/data/prompts/planner/task.txt +++ /dev/null @@ -1,5 +0,0 @@ -The given user input is an instruction of a robotic assembly task. -You summarize it and come up with a bullet list task plan from it. -This plan should involve individual assembly step(s) of robot assembly task. -Do not add any superfluous steps. -Make sure that each step has all the information needed - do not skip steps. \ No newline at end of file diff --git a/data/prompts/planner/template copy.txt b/data/prompts/planner/template copy.txt deleted file mode 100644 index 53b29a0c..00000000 --- a/data/prompts/planner/template copy.txt +++ /dev/null @@ -1,22 +0,0 @@ -The given user input is an instruction of a robotic assembly task. -You summarize it and come up with a bullet list task plan from it. -This plan should involve individual assembly step(s) of robot assembly task. -Do not add any superfluous steps. -Make sure that each step has all the information needed - do not skip steps. - -You should only use formal assembly actions as the action in the steps. -The available assembly actions are used to achieve the assembly constraints between two parts. -The assembly actions are: -- insert something into something -- place something onto something -- screw something into something - -You should also refer to the constraints in the world state, which provides the assembly constraints between the parts. - -Example: -user_input: "you should assembly the red block with the blue hole." -in world state's constraint: can_insert_into(red_block, blue_hole) -plan: "insert the red block into the blue hole" - -user_input: "intert the blub into the base, then put the cover onto it." -plan: ["insert the blub into the base", "put the cover onto the blub"] diff --git a/data/prompts/planner/template.txt b/data/prompts/planner/template.txt deleted file mode 100644 index f1d28224..00000000 --- a/data/prompts/planner/template.txt +++ /dev/null @@ -1,3 +0,0 @@ -Now make a plan with the following information: -world_state: {world_state} -user_input: {user_input} \ No newline at end of file diff --git a/data/prompts/rec_sk_gen/behaviortree copy 2.txt b/data/prompts/rec_sk_gen/behaviortree copy 2.txt deleted file mode 100644 index 8bf28578..00000000 --- a/data/prompts/rec_sk_gen/behaviortree copy 2.txt +++ /dev/null @@ -1,19 +0,0 @@ -Follow these rules: -1. Every node are described by a summary and a name: - - "summary": a short interpretive description of the node. - - "name": includes the node type and the formal form of the node according to the domain knowledge. -2. The control flow nodes, namely "selector" and "sequence", can have "children", which is a list of children nodes. -3. The "condition" nodes that are children of "selectors" are targets. Those that are children of "sequences" are preconditions. -4. All the node always have their corresponding actions/precidates in the domain knowledge. -5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state. -6. The basic structure template of a unit subtree is: - - A selector as the root node. - - One target condition node as the first child of the selector to check if the purpose of executing an action has been fulfilled. - - A sequence node as the last child of the selector node to achieve the target. - - A number of precondition nodes as the first children of the sequence node to check the action's preconditions. - - One action node as the last child of the sequence node, whose preconditions are checked before and will immediate start its action when being ticked. -7. This is how you use a unit subtree to modify the original tree: - - First, find the failed condition node. Refer to runtime world state to understand its failure reason. - - Find an action from domain knowledge that can fulfill the failed condition node. - - Create a subtree to fulfill that condition node, with the failed condition node as target node. - - Replace the failed condition node with the subtree in the original tree. \ No newline at end of file diff --git a/data/prompts/rec_sk_gen/behaviortree copy.txt b/data/prompts/rec_sk_gen/behaviortree copy.txt deleted file mode 100644 index a04cc3a9..00000000 --- a/data/prompts/rec_sk_gen/behaviortree copy.txt +++ /dev/null @@ -1,56 +0,0 @@ -The behavior tree should be in json dictionary format. -Follow these rules: -1. Every node are described by a summary and a name: - - "summary": a short interpretive description of the node. - - "name": includes the node type and the formal form of the node according to the domain knowledge. -2. The control flow nodes, namely "selector" and "sequence", can have "children", which is a list of children nodes. -3. The "condition" nodes that are children of "selectors" are targets. Those that are children of "sequences" are preconditions. -4. All the node always have their corresponding actions/precidates in the domain knowledge. -5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state. -6. The basic structure template of a subtree is: - - A selector as the root node. - - One target condition node as the first child of the selector to check if the target of the action has been fulfilled. - - A sequence node as the last child of the selector node to achieve the target. - - A number of precondition nodes as the first children of the sequence node to check the action's preconditions. - - One action node as the last child of the sequence node, whose preconditions are checked before and will immediate start its action when being ticked. - - The action should have effects that can fulfill the target condition node in the selector. -7. This is how you generate and use a new subtree: - - First, find the failed condition node. Refer to runtime world state to understand its failure reason. - - Find the corresponding action from domain knowledge that can fulfill that condition node. - - Create a subtree to fulfill that condition node, with that condition node as target node. - - Create the sequence in this subtree with the action's preconditions and the action. - - Replace the failed condition node with the subtree in the original tree. -Following is an example of how this works: -old tree: -{ -- selector - - target - - sequence - - precondition1 - - precondition2 (failed according to the runtime world state) - - action -} -new subtree: -{ -- new selector - - new target (precondition2) - - new sequence - - new action's preconditions - - new action -} -the new behavior tree: -{ -- selector - - target - - sequence - - precondition1 - { - - new selector - - new target (precondition2) - - new sequence - - new action's precondition1 - - new action's precondition2 - - new action - } - - action -} diff --git a/data/prompts/rec_sk_gen/behaviortree.txt b/data/prompts/rec_sk_gen/behaviortree.txt deleted file mode 100644 index 7976e67e..00000000 --- a/data/prompts/rec_sk_gen/behaviortree.txt +++ /dev/null @@ -1,13 +0,0 @@ -Follow these rules: -1. Every node has: - - "summary": a short interpretive description of the node. - - "name": includes the node type and the formal name of the node according to the domain knowledge. -2. Selector nodes and sequence nodes can have child nodes. -3. Refer to PREDICATE_LIST to understand condition nodes -4. Refer to ROBOT_ACTION_LIST to understand action nodes. -4. Make sure for each action node, its preconditions are checked by condition nodes before it is ticked. -5. One condition node can only check one condition. -6. An action node will return RUNNING immediately even if it is ticked. -7. The action node will execute the action when it is ticked again, even if it has been executed and returned success before. Use selector nodes with a condition node to avoid this. -8. Don't put two action nodes in one sequence node because in this case the first action node will be executed repeatedly and the second one will never get executed. -9. Due to the same reason, don't put two sequence nodes in one sequence node. \ No newline at end of file diff --git a/data/prompts/rec_sk_gen/chain.txt b/data/prompts/rec_sk_gen/chain.txt deleted file mode 100644 index f60d7911..00000000 --- a/data/prompts/rec_sk_gen/chain.txt +++ /dev/null @@ -1,2 +0,0 @@ -This is how you should work: -1. If last_behavior_tree doesn't exist, you interpret user_instruction into a target condition. Then you find one action from the domain knowledge that can fulfill the target condition. You finally return the \ No newline at end of file diff --git a/data/prompts/rec_sk_gen/domain copy.txt b/data/prompts/rec_sk_gen/domain copy.txt deleted file mode 100644 index 7eec5ea4..00000000 --- a/data/prompts/rec_sk_gen/domain copy.txt +++ /dev/null @@ -1,84 +0,0 @@ -Following is the domain knowledge you need in pddl domain format, which includes: -- the object types in the world, -- the predicates for describing the states of the world, -- the actions that the robot can apply, with their precondtions and effects defined. -""" -(define (domain robot_assembly_problem-domain) - (:requirements :strips :typing) - (:types - thing - object - tool part hand - thing - ) - (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part)) - (:action pick_up - :parameters ( ?hand - hand ?tool - tool ?part - part) - :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part)) - :effect (and (hold ?tool ?part) (not (is_free ?tool)))) - (:action put_down - :parameters ( ?hand - hand ?tool - tool ?part - part) - :precondition (and (hold ?tool ?part) (hold ?hand ?tool)) - :effect (and (not (hold ?hand ?part)) (is_free ?tool))) - (:action place - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2))) - (:action detach - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2)))) - (:action insert - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2))) - (:action pull - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2)))) - (:action screw - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2))) - (:action unscrew - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2)))) - (:action load_tool - :parameters ( ?hand - hand ?tool - tool) - :precondition (and (is_equippable ?tool) (is_free ?hand)) - :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool))) - (:action unload_tool - :parameters ( ?hand - hand ?tool - tool) - :precondition (and (hold ?hand ?tool) (is_free ?tool)) - :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool)))) -) -""" -The precidates are explained as follows: -""" -"PREDICATE_LIST" -- is_free( or ): or is free and can hold something. (default: true) -- is_equippable(): is ready to be eqquipped by a robot hand. (default: true) -- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false) -- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false) -- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false) -- can_place_to(, ): can be placed into (placement assembly constraint) (default: false) -- hold(, ): is holding (default: false) -- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false) -- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false) -- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false) -""" - -The actions are explained as follows: -""" -"ROBOT_ACTION_LIST" -- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something. -- put_down(, , ): use to put down . As preconditions, should be holding and should be holding . -- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding . -- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into . -- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding . -- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into . -- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding . -- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into . -- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free. -- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free. -""" -the items in <> indicate the type of the instances needed for the predicates and actions. \ No newline at end of file diff --git a/data/prompts/rec_sk_gen/domain.txt b/data/prompts/rec_sk_gen/domain.txt deleted file mode 100644 index 8ee984d3..00000000 --- a/data/prompts/rec_sk_gen/domain.txt +++ /dev/null @@ -1,35 +0,0 @@ -Following is the domain knowledge you need, which includes: -- the object types in the world, -- the predicates for describing the states of the world, -- the actions that the robot can apply, with their precondtions and effects defined. - -The precidates are explained as follows: -""" -"PREDICATE_LIST" -- is_free( or ): or is free and can hold something. -- is_equippable(): is ready to be eqquipped by a robot hand. -- can_manipulate(, ): can be used to manipulate (manipulation constraint) -- can_insert_to(, ): can be inserted into (insertion assembly constraint) -- can_screw_to(, ): can be screwed into (screw assembly constraint) -- can_place_to(, ): can be placed into (placement assembly constraint) -- hold(, ): is holding -- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) -- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) -- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) -""" - -The actions are explained as follows: -""" -"ROBOT_ACTION_LIST" -- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something. -- put_down(, , ): use to put down . As preconditions, should be holding and should be holding . -- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding . -- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into . -- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding . -- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into . -- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding . -- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into . -- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free. -- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free. -""" -the items in <> indicate the type of the instances needed for the predicates and actions. \ No newline at end of file diff --git a/data/prompts/rec_sk_gen/example.txt b/data/prompts/rec_sk_gen/example.txt deleted file mode 100644 index 5fdda217..00000000 --- a/data/prompts/rec_sk_gen/example.txt +++ /dev/null @@ -1,189 +0,0 @@ -Following are some examples of the input and output. -Example 1: -""" -- Input: -objects: -{ - "tool": ["parallel_box1", "inward_claw", "outward_claw"], - "part": ["gear1", "shaft1", "gearbase_hole1", "gearbase_hole3"] - "hand": ["left_hand"] -} -runtime_world_state: -{ - "objects": [ - {"name": "parallel_box1", "properties": ["is_free", "is_equippable"]}, - {"name": "inward_claw", "properties": ["is_free", "is_equippable"]}, - {"name": "outward_claw", "properties": ["is_free"]}, - {"name": "gear1", "properties": []}, - {"name": "shaft1", "properties": []}, - {"name": "gearbase_hole1", "properties": []}, - {"name": "left_hand", "properties": []} - ], - "constraints": [ - {"source": "parallel_box1", "name": "can_manipulate", "target": "gear1"}, - {"source": "gear1", "name": "can_insert_to", "target": "shaft1"}, - ], - "relations": [ - {"source": "left_hand","name": "hold","target": "outward_claw"} - ] -} -last_behavior_tree: -{ - "summary": "selector to load parallel_box1", - "name": "selector: load_tool(left_hand, parallel_box1)", - "identifier": 0, - "children": [ - { - "summary": "condition node to check if left_hand holds parallel_box1", - "name": "target: hold(left_hand, parallel_box1)" - "identifier": 1, - }, - { - "summary": "sequence to load parallel_box1", - "name": "sequence: load_tool(left_hand, parallel_box1)", - "identifier": 2, - "children": [ - { - "summary": "condition node to check if parallel_box1 is equippable", - "name": "precondition: is_equippable(parallel_box1)" - "identifier": 3, - }, - { - "summary": "condition node to check if left_hand is free, - "name": "target: is_free(left_hand)" - "identifier": 4, - }, - { - "summary": "action node to equip parallel_box1 to left hand", - "name": "action: load_tool(left_hand, parallel_box1)", - "identifier": 5, - }, - ], - }, - ], -} -last_failed_node: -{ - "summary": "condition node to check if left_hand is free, - "name": "target: is_free(left_hand)" - "identifier": 4, -} -instructions: "insert gear1 into shaft1" - -- Output: -{ - "summary": "selector to load parallel_box1", - "name": "selector: load_tool(left_hand, parallel_box1)", - "identifier": 0, - "children": [ - { - "summary": "condition node to check if left_hand holds parallel_box1", - "name": "target: hold(left_hand, parallel_box1)" - "identifier": 1, - }, - { - "summary": "sequence to load parallel_box1", - "name": "sequence: load_tool(left_hand, parallel_box1)", - "identifier": 2, - "children": [ - { - "summary": "condition node to check if parallel_box1 is equippable", - "name": "precondition: is_equippable(parallel_box1)" - "identifier": 3, - }, - { - "summary": "selector to make left_hand unload outward_claw", - "name": "selector: unload_tool(left_hand, outward_claw)", - "children": [ - { - "summary": "condition node to check if left_hand is free, - "name": "target: is_free(left_hand)" - }, - { - "summary": "sequence to make left_hand unload outward_claw", - "name": "sequence: unload_tool(left_hand, outward_claw)", - "children": [ - { - "summary": "condition node to check if outward_claw is free", - "name": "precondition: is_free(outward_claw)" - }, - { - "summary": "condition node to check if left hand holds outward_claw", - "name": "precondition: hold(left_hand, outward_claw)" - }, - { - "summary": "action node to make left hand unload outward_claw", - "name": "action: unload_tool(left_hand, outward_claw)", - }, - ], - }, - ], - }, - { - "summary": "action node to equip parallel_box1 to left hand", - "name": "action: load_tool(left_hand, parallel_box1)", - "identifier": 5, - }, - ], - }, - ], -} -""" - -Example 2: -""" -- Input: -objects: -{ - "tool": ["inward_claw"], - "part": ["nut1", "bolt1"] - "hand": ["left_hand"] -} -runtime_world_state: -{ - "objects": [ - {"name": "inward_claw", "properties": ["is_free", "is_equippable"]}, - {"name": "nut1", "properties": []}, - {"name": "bolt1", "properties": []}, - {"name": "left_hand", "properties": ["is_free"]} - ], - "constraints": [ - {"source": "inward_claw", "name": "can_manipulate", "target": "nut1"}, - {"source": "nut1", "name": "can_screw_to", "target": "bolt1"}, - ], - "relations": [] -} -last_behavior_tree: None -last_failed_node: None -instructions: "screw nut1 into bolt1" - -- Output: -{ - "summary": "selector to use left_hand with inward_claw to screw nut1 into bolt1", - "name": "selector: screw(left_hand, inward_claw, nut1, bolt1)", - "children": [ - { - "summary": "condition node to check if nut1 is screwed into bolt1", - "name": "target: is_screwed_to(nut1, bolt1)" - }, - { - "summary": "sequence to use left_hand with inward_claw to screw nut1 into bolt1", - "name": "sequence: screw(left_hand, inward_claw, nut1, bolt1)", - "children": [ - { - "summary": "condition node to check if inward_claw holds nut1", - "name": "precondition: hold(inward_claw, nut1)" - }, - { - "summary": "condition node to check if nut1 can be screwed into bolt1", - "name": "precondition: can_screw_to(nut1, bolt1)" - }, - { - "summary": "action node to use left_hand with inward_claw to screw nut1 into bolt1", - "name": "action: screw(left_hand, inward_claw, nut1, bolt1)" - }, - ], - }, - ], -} -""" \ No newline at end of file diff --git a/data/prompts/rec_sk_gen/object.txt b/data/prompts/rec_sk_gen/object.txt deleted file mode 100644 index 2a4947ec..00000000 --- a/data/prompts/rec_sk_gen/object.txt +++ /dev/null @@ -1,7 +0,0 @@ -objects are given in a dictionary. -Its keys are the types and the values are lists of instances of that type. -An example: -""" -"part": ["gear", "nut", "bolt"] -""" -Refer to the domain knowledge to understand which predicates and actions are applicable to which types. \ No newline at end of file diff --git a/data/prompts/rec_sk_gen/state.txt b/data/prompts/rec_sk_gen/state.txt deleted file mode 100644 index 792f3ac5..00000000 --- a/data/prompts/rec_sk_gen/state.txt +++ /dev/null @@ -1,21 +0,0 @@ -Information about world states is in json dictionary format. Example: -""" -{ - "objects": [ - {"name": "leg1", "properties": []}, # leg1 is an object with no properties - {"name": "seat", "properties": []}, # seat is an object with no properties - {"name": "hand", "properties": ["is_free"]}, # hand is an object with a property "is_free" - ], - "constraints": [ - {"source": "leg1", "name": "can_screw_to", "target": "seat"}, # means leg1 can be screwed to seat - ], - "relations": [ - {"source": "leg1", "name": "is_screwed_to", "target": "seat"}, # means leg1 is screwed to seat - ], -} -""" -It contains three lists: - - "objects": the objects in the world and their properties. The properties can be changed by robot actions. - - "constraints": the constraint relations that are enforced and should not be changed. - - "relations": the relations in the world. They can be changed by robot actions. -The properties, constraints and relations should always align with the [PREDICATE_LIST] in domain knowledge. \ No newline at end of file diff --git a/data/prompts/rec_sk_gen/system.txt b/data/prompts/rec_sk_gen/system.txt deleted file mode 100644 index 7ebac297..00000000 --- a/data/prompts/rec_sk_gen/system.txt +++ /dev/null @@ -1 +0,0 @@ -You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you construct robot task plans represented by behaiovr trees. \ No newline at end of file diff --git a/data/prompts/rec_sk_gen/task.txt b/data/prompts/rec_sk_gen/task.txt deleted file mode 100644 index 05e02a11..00000000 --- a/data/prompts/rec_sk_gen/task.txt +++ /dev/null @@ -1,12 +0,0 @@ -The input are: -objects: a list of objects in the world. -runtime_world_state: the world state when the last behaior tree stopped. -last_behavior_tree: the last executed behavior tree. -last_failed_node: the last node that failed in the last behavior tree. -action_sequence: the action_sequence that your behavior tree should align with. - -You analyze the failure reason, then fix the behavior tree. -If no last_behavior_tree is given, you generate a behavior tree based on the action sequence. The behavior tree should follow the regulated format. -Learn from the examples to understand the regulated format of the behavior tree. -The identifiers of the nodes are not necessary when generating the behavior tree. -You only return the behavior tree as a json object that is ready to be parsed. \ No newline at end of file diff --git a/data/prompts/rec_sk_gen/template copy 2.txt b/data/prompts/rec_sk_gen/template copy 2.txt deleted file mode 100644 index 93aff710..00000000 --- a/data/prompts/rec_sk_gen/template copy 2.txt +++ /dev/null @@ -1,14 +0,0 @@ -Start working. Resume from the input below. ---------------- -objects: {objects} - -runtime_world_state: {runtime_world_state} - -last_behavior_tree: {last_behavior_tree} - -last_failed_node: {last_failed_node} - -summary: {summary} - -instructions: {instructions} ---------------- diff --git a/data/prompts/rec_sk_gen/template copy.txt b/data/prompts/rec_sk_gen/template copy.txt deleted file mode 100644 index 88ae902b..00000000 --- a/data/prompts/rec_sk_gen/template copy.txt +++ /dev/null @@ -1,14 +0,0 @@ -Start working. Resume from the input below. ---------------- -objects: {objects} - -runtime_world_state: {runtime_world_state} - -last_behavior_tree: {last_behavior_tree} - -last_failed_node: {last_failed_node} - -summary: {summary} - -action_sequence: {action_sequence} ---------------- diff --git a/data/prompts/rec_sk_gen/template.txt b/data/prompts/rec_sk_gen/template.txt deleted file mode 100644 index 88ae902b..00000000 --- a/data/prompts/rec_sk_gen/template.txt +++ /dev/null @@ -1,14 +0,0 @@ -Start working. Resume from the input below. ---------------- -objects: {objects} - -runtime_world_state: {runtime_world_state} - -last_behavior_tree: {last_behavior_tree} - -last_failed_node: {last_failed_node} - -summary: {summary} - -action_sequence: {action_sequence} ---------------- diff --git a/data/prompts/rec_sk_gen_bare/behaviortree copy.txt b/data/prompts/rec_sk_gen_bare/behaviortree copy.txt deleted file mode 100644 index a04cc3a9..00000000 --- a/data/prompts/rec_sk_gen_bare/behaviortree copy.txt +++ /dev/null @@ -1,56 +0,0 @@ -The behavior tree should be in json dictionary format. -Follow these rules: -1. Every node are described by a summary and a name: - - "summary": a short interpretive description of the node. - - "name": includes the node type and the formal form of the node according to the domain knowledge. -2. The control flow nodes, namely "selector" and "sequence", can have "children", which is a list of children nodes. -3. The "condition" nodes that are children of "selectors" are targets. Those that are children of "sequences" are preconditions. -4. All the node always have their corresponding actions/precidates in the domain knowledge. -5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state. -6. The basic structure template of a subtree is: - - A selector as the root node. - - One target condition node as the first child of the selector to check if the target of the action has been fulfilled. - - A sequence node as the last child of the selector node to achieve the target. - - A number of precondition nodes as the first children of the sequence node to check the action's preconditions. - - One action node as the last child of the sequence node, whose preconditions are checked before and will immediate start its action when being ticked. - - The action should have effects that can fulfill the target condition node in the selector. -7. This is how you generate and use a new subtree: - - First, find the failed condition node. Refer to runtime world state to understand its failure reason. - - Find the corresponding action from domain knowledge that can fulfill that condition node. - - Create a subtree to fulfill that condition node, with that condition node as target node. - - Create the sequence in this subtree with the action's preconditions and the action. - - Replace the failed condition node with the subtree in the original tree. -Following is an example of how this works: -old tree: -{ -- selector - - target - - sequence - - precondition1 - - precondition2 (failed according to the runtime world state) - - action -} -new subtree: -{ -- new selector - - new target (precondition2) - - new sequence - - new action's preconditions - - new action -} -the new behavior tree: -{ -- selector - - target - - sequence - - precondition1 - { - - new selector - - new target (precondition2) - - new sequence - - new action's precondition1 - - new action's precondition2 - - new action - } - - action -} diff --git a/data/prompts/rec_sk_gen_bare/behaviortree.txt b/data/prompts/rec_sk_gen_bare/behaviortree.txt deleted file mode 100644 index 205a0b95..00000000 --- a/data/prompts/rec_sk_gen_bare/behaviortree.txt +++ /dev/null @@ -1,22 +0,0 @@ -The behavior tree should be in json dictionary format. -Follow these rules: -1. Every node are described by a summary and a name: - - "summary": a short interpretive description of the node. - - "name": includes the node type and the formal form of the node according to the domain knowledge. -2. The control flow nodes, namely "selector" and "sequence", can have "children", which is a list of children nodes. -3. The "condition" nodes that are children of "selectors" are targets. Those that are children of "sequences" are preconditions. -4. All the node always have their corresponding actions/precidates in the domain knowledge. -5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state. -6. The basic structure template of a unit subtree is: - - A selector as the root node. - - One target condition node as the first child of the selector to check if the target of the action has been fulfilled. - - A sequence node as the last child of the selector node to achieve the target. - - A number of precondition nodes as the first children of the sequence node to check the action's preconditions. - - One action node as the last child of the sequence node, whose preconditions are checked before and will immediate start its action when being ticked. - - The action should have effects that can fulfill the target condition node in the selector. -7. This is how you generate and use a new unit subtree: - - First, find the failed condition node. Refer to runtime world state to understand its failure reason. - - Find the corresponding action from domain knowledge that can fulfill that condition node. - - Create a subtree to fulfill that condition node, with that condition node as target node. - - Create the sequence in this subtree with the action's preconditions and the action. - - Replace the failed condition node with the subtree in the original tree. \ No newline at end of file diff --git a/data/prompts/rec_sk_gen_bare/domain.txt b/data/prompts/rec_sk_gen_bare/domain.txt deleted file mode 100644 index 7eec5ea4..00000000 --- a/data/prompts/rec_sk_gen_bare/domain.txt +++ /dev/null @@ -1,84 +0,0 @@ -Following is the domain knowledge you need in pddl domain format, which includes: -- the object types in the world, -- the predicates for describing the states of the world, -- the actions that the robot can apply, with their precondtions and effects defined. -""" -(define (domain robot_assembly_problem-domain) - (:requirements :strips :typing) - (:types - thing - object - tool part hand - thing - ) - (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part)) - (:action pick_up - :parameters ( ?hand - hand ?tool - tool ?part - part) - :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part)) - :effect (and (hold ?tool ?part) (not (is_free ?tool)))) - (:action put_down - :parameters ( ?hand - hand ?tool - tool ?part - part) - :precondition (and (hold ?tool ?part) (hold ?hand ?tool)) - :effect (and (not (hold ?hand ?part)) (is_free ?tool))) - (:action place - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2))) - (:action detach - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2)))) - (:action insert - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2))) - (:action pull - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2)))) - (:action screw - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2))) - (:action unscrew - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2)))) - (:action load_tool - :parameters ( ?hand - hand ?tool - tool) - :precondition (and (is_equippable ?tool) (is_free ?hand)) - :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool))) - (:action unload_tool - :parameters ( ?hand - hand ?tool - tool) - :precondition (and (hold ?hand ?tool) (is_free ?tool)) - :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool)))) -) -""" -The precidates are explained as follows: -""" -"PREDICATE_LIST" -- is_free( or ): or is free and can hold something. (default: true) -- is_equippable(): is ready to be eqquipped by a robot hand. (default: true) -- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false) -- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false) -- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false) -- can_place_to(, ): can be placed into (placement assembly constraint) (default: false) -- hold(, ): is holding (default: false) -- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false) -- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false) -- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false) -""" - -The actions are explained as follows: -""" -"ROBOT_ACTION_LIST" -- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something. -- put_down(, , ): use to put down . As preconditions, should be holding and should be holding . -- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding . -- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into . -- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding . -- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into . -- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding . -- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into . -- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free. -- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free. -""" -the items in <> indicate the type of the instances needed for the predicates and actions. \ No newline at end of file diff --git a/data/prompts/rec_sk_gen_bare/example.txt b/data/prompts/rec_sk_gen_bare/example.txt deleted file mode 100644 index 5fdda217..00000000 --- a/data/prompts/rec_sk_gen_bare/example.txt +++ /dev/null @@ -1,189 +0,0 @@ -Following are some examples of the input and output. -Example 1: -""" -- Input: -objects: -{ - "tool": ["parallel_box1", "inward_claw", "outward_claw"], - "part": ["gear1", "shaft1", "gearbase_hole1", "gearbase_hole3"] - "hand": ["left_hand"] -} -runtime_world_state: -{ - "objects": [ - {"name": "parallel_box1", "properties": ["is_free", "is_equippable"]}, - {"name": "inward_claw", "properties": ["is_free", "is_equippable"]}, - {"name": "outward_claw", "properties": ["is_free"]}, - {"name": "gear1", "properties": []}, - {"name": "shaft1", "properties": []}, - {"name": "gearbase_hole1", "properties": []}, - {"name": "left_hand", "properties": []} - ], - "constraints": [ - {"source": "parallel_box1", "name": "can_manipulate", "target": "gear1"}, - {"source": "gear1", "name": "can_insert_to", "target": "shaft1"}, - ], - "relations": [ - {"source": "left_hand","name": "hold","target": "outward_claw"} - ] -} -last_behavior_tree: -{ - "summary": "selector to load parallel_box1", - "name": "selector: load_tool(left_hand, parallel_box1)", - "identifier": 0, - "children": [ - { - "summary": "condition node to check if left_hand holds parallel_box1", - "name": "target: hold(left_hand, parallel_box1)" - "identifier": 1, - }, - { - "summary": "sequence to load parallel_box1", - "name": "sequence: load_tool(left_hand, parallel_box1)", - "identifier": 2, - "children": [ - { - "summary": "condition node to check if parallel_box1 is equippable", - "name": "precondition: is_equippable(parallel_box1)" - "identifier": 3, - }, - { - "summary": "condition node to check if left_hand is free, - "name": "target: is_free(left_hand)" - "identifier": 4, - }, - { - "summary": "action node to equip parallel_box1 to left hand", - "name": "action: load_tool(left_hand, parallel_box1)", - "identifier": 5, - }, - ], - }, - ], -} -last_failed_node: -{ - "summary": "condition node to check if left_hand is free, - "name": "target: is_free(left_hand)" - "identifier": 4, -} -instructions: "insert gear1 into shaft1" - -- Output: -{ - "summary": "selector to load parallel_box1", - "name": "selector: load_tool(left_hand, parallel_box1)", - "identifier": 0, - "children": [ - { - "summary": "condition node to check if left_hand holds parallel_box1", - "name": "target: hold(left_hand, parallel_box1)" - "identifier": 1, - }, - { - "summary": "sequence to load parallel_box1", - "name": "sequence: load_tool(left_hand, parallel_box1)", - "identifier": 2, - "children": [ - { - "summary": "condition node to check if parallel_box1 is equippable", - "name": "precondition: is_equippable(parallel_box1)" - "identifier": 3, - }, - { - "summary": "selector to make left_hand unload outward_claw", - "name": "selector: unload_tool(left_hand, outward_claw)", - "children": [ - { - "summary": "condition node to check if left_hand is free, - "name": "target: is_free(left_hand)" - }, - { - "summary": "sequence to make left_hand unload outward_claw", - "name": "sequence: unload_tool(left_hand, outward_claw)", - "children": [ - { - "summary": "condition node to check if outward_claw is free", - "name": "precondition: is_free(outward_claw)" - }, - { - "summary": "condition node to check if left hand holds outward_claw", - "name": "precondition: hold(left_hand, outward_claw)" - }, - { - "summary": "action node to make left hand unload outward_claw", - "name": "action: unload_tool(left_hand, outward_claw)", - }, - ], - }, - ], - }, - { - "summary": "action node to equip parallel_box1 to left hand", - "name": "action: load_tool(left_hand, parallel_box1)", - "identifier": 5, - }, - ], - }, - ], -} -""" - -Example 2: -""" -- Input: -objects: -{ - "tool": ["inward_claw"], - "part": ["nut1", "bolt1"] - "hand": ["left_hand"] -} -runtime_world_state: -{ - "objects": [ - {"name": "inward_claw", "properties": ["is_free", "is_equippable"]}, - {"name": "nut1", "properties": []}, - {"name": "bolt1", "properties": []}, - {"name": "left_hand", "properties": ["is_free"]} - ], - "constraints": [ - {"source": "inward_claw", "name": "can_manipulate", "target": "nut1"}, - {"source": "nut1", "name": "can_screw_to", "target": "bolt1"}, - ], - "relations": [] -} -last_behavior_tree: None -last_failed_node: None -instructions: "screw nut1 into bolt1" - -- Output: -{ - "summary": "selector to use left_hand with inward_claw to screw nut1 into bolt1", - "name": "selector: screw(left_hand, inward_claw, nut1, bolt1)", - "children": [ - { - "summary": "condition node to check if nut1 is screwed into bolt1", - "name": "target: is_screwed_to(nut1, bolt1)" - }, - { - "summary": "sequence to use left_hand with inward_claw to screw nut1 into bolt1", - "name": "sequence: screw(left_hand, inward_claw, nut1, bolt1)", - "children": [ - { - "summary": "condition node to check if inward_claw holds nut1", - "name": "precondition: hold(inward_claw, nut1)" - }, - { - "summary": "condition node to check if nut1 can be screwed into bolt1", - "name": "precondition: can_screw_to(nut1, bolt1)" - }, - { - "summary": "action node to use left_hand with inward_claw to screw nut1 into bolt1", - "name": "action: screw(left_hand, inward_claw, nut1, bolt1)" - }, - ], - }, - ], -} -""" \ No newline at end of file diff --git a/data/prompts/rec_sk_gen_bare/object.txt b/data/prompts/rec_sk_gen_bare/object.txt deleted file mode 100644 index 2a4947ec..00000000 --- a/data/prompts/rec_sk_gen_bare/object.txt +++ /dev/null @@ -1,7 +0,0 @@ -objects are given in a dictionary. -Its keys are the types and the values are lists of instances of that type. -An example: -""" -"part": ["gear", "nut", "bolt"] -""" -Refer to the domain knowledge to understand which predicates and actions are applicable to which types. \ No newline at end of file diff --git a/data/prompts/rec_sk_gen_bare/state.txt b/data/prompts/rec_sk_gen_bare/state.txt deleted file mode 100644 index c686cb84..00000000 --- a/data/prompts/rec_sk_gen_bare/state.txt +++ /dev/null @@ -1,21 +0,0 @@ -Information about world states is in json dictionary format. Example: -""" -{ - "objects": [ - {"name": "leg1", "properties": []}, # leg1 is an object with no properties - {"name": "seat", "properties": []}, # seat is an object with no properties - {"name": "hand", "properties": ["is_free"]}, # hand is an object with a property "is_free" - ], - "constraints": [ - {"source": "leg1", "name": "can_screw_to", "target": "seat"}, # means leg1 can be screwed to seat - ], - "relations": [ - {"source": "leg1", "name": "is_screwed_to", "target": "seat"}, # means leg1 is screwed to seat - ], -} -""" -It contains three lists: - - "objects": the objects and their properties (as lists of strings) in the world. The properties can be changed by robot actions. Refer to the input world state for them. - - "constraints": the constraint relations that are enforced and should not be changed. Refer to the pddl problem for the constraints. - - "relations": the relations in the world. They can be changed by robot actions. Refer to input world state for them. -The properties, constraints and relations should always align with the [PREDICATE_LIST] in domain knowledge. \ No newline at end of file diff --git a/data/prompts/rec_sk_gen_bare/system.txt b/data/prompts/rec_sk_gen_bare/system.txt deleted file mode 100644 index 7ebac297..00000000 --- a/data/prompts/rec_sk_gen_bare/system.txt +++ /dev/null @@ -1 +0,0 @@ -You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you construct robot task plans represented by behaiovr trees. \ No newline at end of file diff --git a/data/prompts/rec_sk_gen_bare/task.txt b/data/prompts/rec_sk_gen_bare/task.txt deleted file mode 100644 index 93979c67..00000000 --- a/data/prompts/rec_sk_gen_bare/task.txt +++ /dev/null @@ -1,12 +0,0 @@ -The input are: -objects: a list of objects in the world. -runtime_world_state: the world state when the last behaior tree stopped. -last_behavior_tree: the last executed behavior tree. -last_failed_node: the last node that failed in the last behavior tree. -user_instruction: the instruction that the tree is trying to fulfill. - -You analyze the failure reason of the failed node, construct a subtree that can fulfill the failed condition node, replace the failed node with the subtree, and return the new behavior tree. -If no last_behavior_tree is given, you interprete the user instruction into a target condition, pick one action from the domain knowledge that can fulfill the target condition, and return a unit subtree constructed based on it as a start. - -The identifiers of the nodes are not necessary when generating the behavior tree. -You only return the behavior tree as a json object that is ready to be parsed. \ No newline at end of file diff --git a/data/prompts/rec_sk_gen_bare/template.txt b/data/prompts/rec_sk_gen_bare/template.txt deleted file mode 100644 index 85fe167d..00000000 --- a/data/prompts/rec_sk_gen_bare/template.txt +++ /dev/null @@ -1,12 +0,0 @@ -Start working. Resume from the input below. ---------------- -objects: {objects} - -runtime_world_state: {runtime_world_state} - -last_behavior_tree: {last_behavior_tree} - -last_failed_node: {last_failed_node} - -instructions: {instructions} ---------------- diff --git a/data/prompts/seq_planner/Zdomain.txt b/data/prompts/seq_planner/Zdomain.txt deleted file mode 100644 index 7eec5ea4..00000000 --- a/data/prompts/seq_planner/Zdomain.txt +++ /dev/null @@ -1,84 +0,0 @@ -Following is the domain knowledge you need in pddl domain format, which includes: -- the object types in the world, -- the predicates for describing the states of the world, -- the actions that the robot can apply, with their precondtions and effects defined. -""" -(define (domain robot_assembly_problem-domain) - (:requirements :strips :typing) - (:types - thing - object - tool part hand - thing - ) - (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part)) - (:action pick_up - :parameters ( ?hand - hand ?tool - tool ?part - part) - :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part)) - :effect (and (hold ?tool ?part) (not (is_free ?tool)))) - (:action put_down - :parameters ( ?hand - hand ?tool - tool ?part - part) - :precondition (and (hold ?tool ?part) (hold ?hand ?tool)) - :effect (and (not (hold ?hand ?part)) (is_free ?tool))) - (:action place - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2))) - (:action detach - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2)))) - (:action insert - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2))) - (:action pull - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2)))) - (:action screw - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2))) - (:action unscrew - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2)))) - (:action load_tool - :parameters ( ?hand - hand ?tool - tool) - :precondition (and (is_equippable ?tool) (is_free ?hand)) - :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool))) - (:action unload_tool - :parameters ( ?hand - hand ?tool - tool) - :precondition (and (hold ?hand ?tool) (is_free ?tool)) - :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool)))) -) -""" -The precidates are explained as follows: -""" -"PREDICATE_LIST" -- is_free( or ): or is free and can hold something. (default: true) -- is_equippable(): is ready to be eqquipped by a robot hand. (default: true) -- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false) -- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false) -- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false) -- can_place_to(, ): can be placed into (placement assembly constraint) (default: false) -- hold(, ): is holding (default: false) -- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false) -- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false) -- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false) -""" - -The actions are explained as follows: -""" -"ROBOT_ACTION_LIST" -- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something. -- put_down(, , ): use to put down . As preconditions, should be holding and should be holding . -- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding . -- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into . -- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding . -- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into . -- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding . -- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into . -- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free. -- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free. -""" -the items in <> indicate the type of the instances needed for the predicates and actions. \ No newline at end of file diff --git a/data/prompts/seq_planner/domain.txt b/data/prompts/seq_planner/domain.txt deleted file mode 100644 index b67bcb08..00000000 --- a/data/prompts/seq_planner/domain.txt +++ /dev/null @@ -1,41 +0,0 @@ -Following is the domain knowledge you need, which includes: -- the object types in the world, -- the predicates for describing the states of the world, -- the actions that the robot can apply, with their precondtions and effects defined. - -The object types are explained as follows: -""" -thing: The base class for all objects in the world. Its subclasses are: hand, tool, part. -hand: The robot hand. A empty can only hold one at a time. -tool: The tool to manipulate . A empty can hold one at a time. -part: The part to be manipulated. -""" - -The precidates are explained as follows: -""" -"PREDICATE_LIST" -- is_empty(): is empty and can hold something. -- can_manipulate(, ): can be used to manipulate (manipulation constraint). This regulates the compatibility between and . -- can_insert_to(, ): can be inserted into (insertion assembly constraint) -- can_screw_to(, ): can be screwed into (screw assembly constraint) -- can_place_to(, ): can be placed into (placement assembly constraint) -- hold(, ): is holding . It can be holding or holding . -- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) -- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) -- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) -""" - -The actions are explained as follows: -""" -"ROBOT_ACTION_LIST" -- pick_up(, , ): use to pick up . As preconditions, should be holding and should be empty in order to hold the . The will be held by the as a result of this action. -- put_down(, , ): use to put down . As preconditions, should be holding and should be holding . The will be empty and will not hold the anymore as a result of this action. -- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding . -- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be placed into . -- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding . -- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be inserted into . -- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding . -- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be screwed into . -- change_tool(, , ): change the equipped tool from to in order to manipulate specific parts. As preconditions, should be holding and should be empty. -""" -the items in <> indicate the type of the instances needed for the predicates and actions. \ No newline at end of file diff --git a/data/prompts/seq_planner/log.md b/data/prompts/seq_planner/log.md deleted file mode 100644 index fc869ff9..00000000 --- a/data/prompts/seq_planner/log.md +++ /dev/null @@ -1,3 +0,0 @@ -29.03.2024 -- removed pddl from domain. use the new domain. -- removed objects from the prompts. diff --git a/data/prompts/seq_planner/readme.md b/data/prompts/seq_planner/readme.md deleted file mode 100644 index bb4cf2b0..00000000 --- a/data/prompts/seq_planner/readme.md +++ /dev/null @@ -1,3 +0,0 @@ -This is a sequential planner prompt for one step in the assembly plan. - -The action level should be robot-motion level. \ No newline at end of file diff --git a/data/prompts/seq_planner/system.txt b/data/prompts/seq_planner/system.txt deleted file mode 100644 index c0b37f9f..00000000 --- a/data/prompts/seq_planner/system.txt +++ /dev/null @@ -1 +0,0 @@ -You are an expert in making plans for robotic assembly tasks. \ No newline at end of file diff --git a/data/prompts/seq_planner/task.txt b/data/prompts/seq_planner/task.txt deleted file mode 100644 index 5bca1690..00000000 --- a/data/prompts/seq_planner/task.txt +++ /dev/null @@ -1,11 +0,0 @@ -The input are: -start_world_state: the world state when the behavior tree started. -user_instruction: an instruction of a robotic assembly task. - -You summarize the user_instruction into a goal world state, then make a bullet list task plan to fulfill the user_instruction. - -The output is a list of actions that the robot should take to fulfill the user_instruction. - -Use the definitions in the domain knowledge to construct actions. - -Do not add any superfluous steps. Make sure that each step has all the information needed. Do not skip steps. \ No newline at end of file diff --git a/data/prompts/seq_planner/template.txt b/data/prompts/seq_planner/template.txt deleted file mode 100644 index c1ed98a5..00000000 --- a/data/prompts/seq_planner/template.txt +++ /dev/null @@ -1,3 +0,0 @@ -Now make a plan with the following information: -start_world_state: {start_world_state} -user_instruction: {user_instruction} \ No newline at end of file diff --git a/data/prompts/seq_planner_est/domain.txt b/data/prompts/seq_planner_est/domain.txt deleted file mode 100644 index a2c0847f..00000000 --- a/data/prompts/seq_planner_est/domain.txt +++ /dev/null @@ -1,84 +0,0 @@ -Following is the domain knowledge you need in pddl domain format, which includes: -- the object types in the world, -- the predicates for describing the states of the world, -- the actions that the robot can apply, with their precondtions and effects defined. -""" -(define (domain robot_assembly_problem-domain) - (:requirements :strips :typing) - (:types - thing - object - tool part hand - thing - ) - (:predicates (is_empty ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part)) - (:action pick_up - :parameters ( ?hand - hand ?tool - tool ?part - part) - :precondition (and (is_empty ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part)) - :effect (and (hold ?tool ?part) (not (is_empty ?tool)))) - (:action put_down - :parameters ( ?hand - hand ?tool - tool ?part - part) - :precondition (and (hold ?tool ?part) (hold ?hand ?tool)) - :effect (and (not (hold ?hand ?part)) (is_empty ?tool))) - (:action place - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_empty ?tool) (is_placed_to ?part1 ?part2))) - (:action detach - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_empty ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2)) - :effect (and (hold ?tool ?part1) (not (is_empty ?tool)) (not (is_placed_to ?part1 ?part2)))) - (:action insert - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_empty ?tool) (is_inserted_to ?part1 ?part2))) - (:action pull - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_empty ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1)) - :effect (and (hold ?tool ?part1) (not (is_empty ?tool)) (not (is_inserted_to ?part1 ?part2)))) - (:action screw - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_empty ?tool) (is_screwed_to ?part1 ?part2))) - (:action unscrew - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_empty ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1)) - :effect (and (hold ?tool ?part1) (not (is_empty ?tool)) (not (is_screwed_to ?part1 ?part2)))) - (:action load_tool - :parameters ( ?hand - hand ?tool - tool) - :precondition (and (is_equippable ?tool) (is_empty ?hand)) - :effect (and (not (is_empty ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool))) - (:action unload_tool - :parameters ( ?hand - hand ?tool - tool) - :precondition (and (hold ?hand ?tool) (is_empty ?tool)) - :effect (and (is_empty ?hand) (is_equippable ?tool) (not (hold ?hand ?tool)))) -) -""" -The precidates are explained as follows: -""" -"PREDICATE_LIST" -- is_empty( or ): or is empty and can hold something. (default: true) -- is_equippable(): is ready to be eqquipped by a robot hand. (default: true) -- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false) -- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false) -- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false) -- can_place_to(, ): can be placed into (placement assembly constraint) (default: false) -- hold(, ): is holding (default: false) -- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false) -- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false) -- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false) -""" - -The actions are explained as follows: -""" -"ROBOT_ACTION_LIST" -- pick_up(, , ): use to pick up . As preconditions, should be holding and should be empty to hold something. -- put_down(, , ): use to put down . As preconditions, should be holding and should be holding . -- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding . -- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be placed into . -- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding . -- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be inserted into . -- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding . -- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be screwed into . -- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be empty. -- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be empty. -""" -the items in <> indicate the type of the instances needed for the predicates and actions. \ No newline at end of file diff --git a/data/prompts/seq_planner_est/example.txt b/data/prompts/seq_planner_est/example.txt deleted file mode 100644 index 1a4f63ca..00000000 --- a/data/prompts/seq_planner_est/example.txt +++ /dev/null @@ -1,38 +0,0 @@ -Following is an input/output example: -input: -''' -start_world_state: -{ -"objects": [ - {"name": "outward_claw", "properties": ["is_empty"]}, - {"name": "shaft1", "properties": []}, - {"name": "left_hand", "properties": ["is_empty"]} -], -"constraints": [ - {"source": "outward_claw", "name": "can_manipulate", "target": "gear1"} -], -"relations": [] -} -target: "hold(left_hand, outward_claw)" -''' -output: -''' -{ - "task_plan": [ - "load_tool(left_hand, outward_claw)" - ], - "estimated_world_state": { - "objects": [ - {"name": "outward_claw", "properties": ["is_empty"]}, - {"name": "shaft1", "properties": []}, - {"name": "left_hand", "properties": [""]} - ], - "constraints": [ - {"source": "outward_claw", "name": "can_manipulate", "target": "gear1"} - ], - "relations": [ - {"source": "left_hand", "name": "hold", "target": "outward_claw"} - ] - } -} -''' \ No newline at end of file diff --git a/data/prompts/seq_planner_est/output_format.txt b/data/prompts/seq_planner_est/output_format.txt deleted file mode 100644 index 84eaf6ad..00000000 --- a/data/prompts/seq_planner_est/output_format.txt +++ /dev/null @@ -1,4 +0,0 @@ -You generate an action sequence to achieve the goal and estimate the world state after executing the plan. -The output json object has the following keys. -- "task_plan": Contains a list of robot actions in the defined form. Only the actions defined in the "ROBOT_ACTION LIST" will be used. The actions should be in pddl action format. -- "estimated_world_state": The world state after executing the plan. diff --git a/data/prompts/seq_planner_est/state.txt b/data/prompts/seq_planner_est/state.txt deleted file mode 100644 index 4b315a03..00000000 --- a/data/prompts/seq_planner_est/state.txt +++ /dev/null @@ -1,21 +0,0 @@ -World states should be in json dictionary format. Example: -""" -{ - "objects": [ - {"name": "leg1", "properties": []}, # leg1 is an object with no properties - {"name": "seat", "properties": []}, # seat is an object with no properties - {"name": "hand", "properties": ["is_empty"]} # hand is an object with a property "is_empty" - ], - "constraints": [ - {"source": "leg1", "name": "can_screw_to", "target": "seat"} # means leg1 can be screwed to seat - ], - "relations": [ - {"source": "leg1", "name": "is_screwed_to", "target": "seat"} # means leg1 is screwed to seat - ] -} -""" -It contains three lists: - - "objects": the objects in the world and their properties. The properties can be changed by robot actions. - - "constraints": the constraint relations that are regulated and can not be changed. - - "relations": the relations in the world. They can be changed by robot actions. -The properties, constraints and relations should always align with the [PREDICATE_LIST] in the domain knowledge. \ No newline at end of file diff --git a/data/prompts/seq_planner_est/system.txt b/data/prompts/seq_planner_est/system.txt deleted file mode 100644 index c0b37f9f..00000000 --- a/data/prompts/seq_planner_est/system.txt +++ /dev/null @@ -1 +0,0 @@ -You are an expert in making plans for robotic assembly tasks. \ No newline at end of file diff --git a/data/prompts/seq_planner_est/task.txt b/data/prompts/seq_planner_est/task.txt deleted file mode 100644 index 8e9dcee6..00000000 --- a/data/prompts/seq_planner_est/task.txt +++ /dev/null @@ -1,8 +0,0 @@ -The input are: -start_world_state: the world state when the behavior tree started. -target: the goal that the plan should achieve. - -You summarize the target into a goal world state, then make a bullet list task plan to achieve it from the start_world_state. Use the definitions in the domain knowledge to construct actions. -If no action needs to be perform, return an empty task plan. -You estimate the world state after the task plan is executed based on the task plan and the start_world_state. -You return the task plan and the estimated world state in a json object. diff --git a/data/prompts/seq_planner_est/template.txt b/data/prompts/seq_planner_est/template.txt deleted file mode 100644 index 3464c4e0..00000000 --- a/data/prompts/seq_planner_est/template.txt +++ /dev/null @@ -1,3 +0,0 @@ -Now make a plan with the following information: -start_world_state: {start_world_state} -target: {target} \ No newline at end of file diff --git a/data/prompts/sk_refine/backup_do_not_use b/data/prompts/sk_refine/backup_do_not_use deleted file mode 100644 index e69de29b..00000000 diff --git a/data/prompts/sk_refine/prompt/dicard_refine_bt.txt b/data/prompts/sk_refine/prompt/dicard_refine_bt.txt deleted file mode 100644 index d7233761..00000000 --- a/data/prompts/sk_refine/prompt/dicard_refine_bt.txt +++ /dev/null @@ -1,140 +0,0 @@ -[user] -You should refine the behavior tree skeleton into a full behavior tree, which is in json dictionary format and have a structure like the following example: -""" -{ - "summary": "selector to load inward_claw to left hand", - "name": "selector: load_tool(left_hand, inward_claw)", - "identifier": 3, - "type_name": "selector", - "children": [ - { - "summary": "check if left_hand is holding inward_claw", - "name": "target: hold(left_hand, inward_claw)", - "identifier": 4, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "inward_claw", - "status": true, - } - ], - }, - { - - "summary": "sequence to load inward_claw to left hand", - "name": "sequence: load_tool(left_hand, inward_claw)", - "identifier": 5, - "type_name": "sequence", - "children": [ - { - "summary": "check if inward_claw is equippable", - "name": "precondition: is_equippable(inward_claw)", - "identifier": 6, - "type_name": "condition", - "conditions": [ - { - "object_name": "inward_claw", - "property_name": "is_equippable", - "property_value": null, - "status": true, - } - ], - }, - { - "summary": "check if left hand is free", - "name": "precondition: is_free(left_hand)", - "identifier": 7, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "is_free", - "property_value": null, - "status": true, - } - ], - }, - { - "summary": "equip inward_claw to left hand", - "name": "action: load_tool(left_hand, inward_claw)", - "identifier": 8, - "type_name": "action", - "effects": [ - { - "object_name": "left_hand", - "property_name": "is_free", - "property_value": null, - "status": false, - }, - { - "object_name": "inward_claw", - "property_name": "is_equippable", - "property_value": null, - "status": false, - }, - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "inward_claw", - "status": true, - }, - ], - }, - ], - }, - ], -} -""" -The dictionary that you return should be formatted as json dictionary. Follow these rules: -1. Every node should have four basic keys: - - "summary": a short description of the node. - - "name": the name of the node. The name should follow the definitions in the domain knowledge. - - "identifier": the unique id of the node. - - "type_name": can be: "selector", "sequence", "condition", "action". -2. The control flow node, "selector" and "sequence" nodes, can have "children", which is a list of children nodes. -3. The "condition" nodes that are children of "selectors" are "targets". Those that are children of "sequences" are preconditions. -4. The "condition" nodes should have the key "conditions", which is a list of conditions to be checked. Each condition should have the following keys: - - "object_name": the object to check. - - "property_name": the property/relation of the object to be checked. - - "property_value": this should be the other object in the relation. For property it should be null. - - "status": true if the property/relation is expected to be true, false otherwise. -5. The "action" nodes should have a key "effects", which is a list of the to-be-exerted effects of the action. Each effect should have the following keys: - - "object_name": the affected object. - - "property_name": the affected property/relation of the object. - - "property_value": this should be the other object if this is a relation. For property it should be null. - - "status": true if the effect is to add this property/relation. false if is to remove it. -6. All the subtrees always have their corresponding action in ROBOT_ACTION_LIST. They are used to represent the actions in the action sequence. Its basic structure is: - - A selector as the root node. - - (Usually one) target condition node as the first child of the selector to check if the target of the action has been fulfilled. - - A sequence node as the last child of the selector node to achieve the target. - - (Usually a number of) precondition nodes as the front children of the sequence node to check the preconditions for executing the action. - - One action node as the last child of the sequence node, whose preconditions are checked before and will immediate start the job when being ticked. - - The action will exert its defined effects after finishing the job successfully. Those effects should fulfilled all the target conditions in the selector node. - A brief example: -{ - selector: load_tool selector, - children: [ - { - target: check if inward_claw is equipped by left hand, - }, - { - sequence: load_tool, - children: [ - precondition: check if inward_claw is equippable, - precondition: check if left hand is free, - {action: equip inward_claw to left hand, effects: [left hand not free, inward_claw not equippable, left_hand holds inward_claw]}, - ], - }, - ], -} - -7. When performing a new action to fulfill a unsatisfied precondition, you can construct a subtree to replace the precondition and take that unfulfilled precondition as the target of the subtree. Then you construct the sequence of the subtree according to the new action you want to perform. -8. The behavior tree should be constructed based on the action sequence you planned and the action definitions in ROBOT_ACTION_LIST. -9. All of the control flow nodes, namely selectors and sequences, are without memory. A new behavior tree tick will always start from the root and traverse all the nodes even if they are ticked in the last tick. - -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/data/prompts/sk_refine/prompt/discard_cot_bt.txt b/data/prompts/sk_refine/prompt/discard_cot_bt.txt deleted file mode 100644 index e2fc475c..00000000 --- a/data/prompts/sk_refine/prompt/discard_cot_bt.txt +++ /dev/null @@ -1,68 +0,0 @@ -[user] -The behavior tree should be in json dictionary format and have a structure like the following example: -""" -{ - "summary": "selector to load inward_claw to left hand", - "children": [ - { - "summary": "condition node to check if inward_claw is held by left hand", - }, - { - "summary": "sequence to load inward_claw to left hand", - "children": [ - { - "summary": "condition node to check if inward_claw is equippable", - }, - { - "summary": "condition node to check if left hand is free", - }, - { - "summary": "action node to equip inward_claw to left hand", - "effects": [ - {"summary": "left hand will be not free anymore",}, - {"summary": "inward_claw will be not equippable anymore",}, - {"summary": "left_hand will hold inward_claw",}, - ], - }, - ], - }, - ], -} -""" -Follow these rules: -1. Every node are described by a summary, which is a short interpretive description of the node. It should include the type of the node and the purpose of the node. -2. The control flow nodes, "selector" and "sequence", can have "children", which is a list of children nodes. -3. The "condition" nodes that are children of "selectors" are "targets". Those that are children of "sequences" are preconditions. -4. The "action" nodes should have a key "effects" with a list of dictionaries, which describes the to-be-exerted effects of the action. The effects are shortly described under the key "summary". -5. All the subtrees always have their corresponding action in ROBOT_ACTION_LIST. They are used to represent the actions in the action sequence. The recommended structure is: - - A selector as the root node. - - (Usually one) target condition node as the first child of the selector to check if the target of the action has been fulfilled. - - A sequence node as the last child of the selector node to achieve the target. - - (Usually a number of) precondition nodes as the front children of the sequence node to check the preconditions for executing the action. - - One action node as the last child of the sequence node, whose preconditions are checked before and will immediate start the job when being ticked. - - The action will exert its defined effects after finishing the job successfully. Those effects should fulfilled all the target conditions in the selector node. - A brief example is as follows: -{ - selector: load_tool selector, - children: [ - { - target: check if inward_claw is equipped by left hand, - }, - { - sequence: load_tool, - children: [ - precondition: check if inward_claw is equippable, - precondition: check if left hand is free, - {action: equip inward_claw to left hand, effects: [left hand not free, inward_claw not equippable, left_hand holds inward_claw]}, - ], - }, - ], -} - -6. When performing a new action to fulfill a unsatisfied precondition, you can construct a subtree to replace the precondition and take that unfulfilled precondition as the target of the subtree. Then you construct the sequence of the subtree according to the new action you want to perform. -7. All of the control flow nodes, namely selectors and sequences, are without memory. A new behavior tree tick will always start from the root and traverse all the nodes even if they are ticked in the last tick. - -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/data/prompts/sk_refine/prompt/discard_cot_example.txt b/data/prompts/sk_refine/prompt/discard_cot_example.txt deleted file mode 100644 index 3d8d6d5b..00000000 --- a/data/prompts/sk_refine/prompt/discard_cot_example.txt +++ /dev/null @@ -1,127 +0,0 @@ -[user] -I will give you some examples of the input and the output you will generate. -Example 1: -""" -- Input: -(define (problem robot_assembly_problem-problem) - (:domain robot_assembly_problem-domain) - (:objects - parallel_box1 parallel_box2 inward_claw outward_claw no_tool - tool - gear1 gear2 gear3 shaft1 shaft2 base - part - left_hand - hand - ) - (:init (can_manipulate parallel_box1 gear1) (can_manipulate outward_claw gear2) (can_manipulate inward_claw gear3) (can_manipulate parallel_box2 shaft1) (can_manipulate no_tool shaft2) (can_screw_to leg1 seat) (can_screw_to leg2 seat) (can_insert_to back seat) (can_screw_to nut1 seat) (can_screw_to nut2 seat) (can_screw_to blub base) (can_place_to lamp blub) (can_insert_to shaft1 base) (can_insert_to shaft2 base) (can_insert_to gear3 shaft2) (can_insert_to gear2 base) (can_insert_to gear1 shaft1) (is_inserted_to shaft1 base) (is_free left_hand) (is_free parallel_box1) (is_free parallel_box2) (is_free inward_claw) (is_free outward_claw) (is_free no_tool) (is_equippable parallel_box1) (is_equippable parallel_box2) (is_equippable inward_claw) (is_equippable outward_claw) (is_equippable no_tool)) - (:goal (and (is_inserted_to gear1 shaft1))) -) -- Output: -``` -{"task_plan": { - "action_sequence": [ - "load_tool(left_hand, parallel_box1)", - "pick_up(left_hand, parallel_box1, gear1)", - "insert(left_hand, parallel_box1, gear1, shaft1)" - ], - "action_instructions": [ - "equip the parallel_box1 on the left hand", - "pick up the gear1 with the parallel_box1 in the left hand", - "insert the gear1 into the shaft1 using the parallel_box1 in the left hand" - ], - "object_name": ["left_hand", "gear1", "parallel_box1", "shaft1"], - "behavior_tree": { - "summary": "selector to insert gear1 into shaft1", - "children": [ - { - "summary": "condition node to check if gear1 is inserted to shaft1", - }, - { - "summary": "sequence to insert gear1 into shaft1", - "children": [ - { - "summary": "selector to load_tool - "children": [ - { - "summary": "condition node to check if parallel_box1 is hold", - }, - { - "summary": "sequence to load_tool", - "children": [ - { - "summary": "condition node to check if parallel_box1 is equippable", - }, - { - "summary": "condition node to check if left hand is free", - }, - { - "summary": "action node to equip parallel_box1 to left hand", - "effects": [ - {"summary": "left_hand will be not free",}, - {"summary": "parallel_box1 will be not equippable",}, - {"summary": "left_hand will hold parallel_box1",}, - ], - }, - ], - }, - ], - }, - { - "summary": "selector to pick_up", - "children": [ - { - "summary": "condition node to check if parallel_box1 holds gear1", - }, - { - "summary": "sequence to pick_up", - "children": [ - { - "summary": "condition node to check if parallel_box1 is free", - }, - { - "summary": "condition node to check if left hand holds parallel_box1", - }, - { - "summary": "action node to pick up gear1 using parallel_box1", - "effects": [ - {"summary": "parallel_box1 will hold gear1",}, - {"summary": "parallel_box1 will be not free",}, - ], - }, - ], - }, - ], - }, - { - "summary": "condition node to check if gear1 can be inserted to shaft1", - }, - { - "summary": "action node to insert gear1 to shaft1", - "effects": [ - {"summary": "parallel_box1 will be not holding gear1",}, - {"summary": "parallel_box1 will be free",}, - {"summary": "gear1 will be inserted to shaft1",}, - ], - }, - ], - }, - ], -} -}, -"initial_state": {"objects": [ - {"name": "gear1", "properties": []}, - {"name": "shaft1", "properties": []}, - {"name": "parallel_box1", "properties": ["is_equippable", "is_free"]}, - {"name": "left_hand", "properties": ["is_free"]}, - ], - "constraints": [ - {"source": "gear1", "name": "can_insert_to", "target": "shaft1"}, - {"source": "parallel_box1", "name": "can_manipulate", "target": "gear1"}, - ], - "relations": [], -}, -"instruction_summary": "insert gear1 into shaft1", -"question":""} -``` -""" -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/data/prompts/sk_refine/prompt/discard_refine_refine.txt b/data/prompts/sk_refine/prompt/discard_refine_refine.txt deleted file mode 100644 index 777e8f26..00000000 --- a/data/prompts/sk_refine/prompt/discard_refine_refine.txt +++ /dev/null @@ -1,247 +0,0 @@ -[user] -You should refine the behavior tree skeleton into a full behavior tree in json dictionary format. - -For each node, you should: -1. add an unique integer "identifier". -2. add its "type_name" accordingly. (precondition and target are both "condition") - -For control flow nodes, namely selectors and sequences, an example is shown as follows: -the skeleton: -""" -{ - "summary": "selector to load no_tool to hand", - "name": "selector: load_tool", - "children": [...], -} - -""" -after refinement: -""" -{ - "summary": "selector to load no_tool to hand", - "name": "selector: load_tool(left_hand, no_tool)", - "identifier": 3, - "type_name": "selector", - "children": [...], -} -""" - -For condition nodes, you should: -1. refine and add the key "conditions", which is a list of conditions to be checked. Each condition should have the following keys: - - "object_name": the object to check. - - "property_name": the property/relation of the object to be checked. - - "property_value": this should be the other object in the relation. For property it should be null. - - "status": true if the property/relation is expected to be true, false otherwise. -An example is shown below: -the skeleton: -""" -{ - "summary": "check if left_hand is holding parallel_box1", - "name": "target: hold(left_hand, parallel_box1)", -} -""" -after refinement: -""" -{ - "summary": "check if left_hand is holding parallel_box1", - "name": "target: hold(left_hand, parallel_box1)", - "identifier": 4, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "parallel_box1", - "status": true, - } - ], -}, -""" - -For action nodes, you should: -1. refine and replace the summaries in the key "effects", which is a list of the to-be-exerted effects of the action. Each effect should have the following keys: - - "object_name": the affected object. - - "property_name": the affected property/relation of the object. - - "property_value": this should be the other object if this is a relation. For property it should be null. - - "status": true if the effect is to add this property/relation. false if is to remove it. -An example is shown below: -the skeleton: -""" -{ - "summary": "left hand equips tool1", - "name": "action: load_tool(left_hand, tool1)", - "effects": [ - {"summary": "left hand will be not free anymore",}, - {"summary": "tool1 will be not equippable anymore",}, - {"summary": "left_hand will hold tool1",}, - ], -}, -""" -after refinement: -""" -{ - "summary": "left hand equips tool1", - "name": "action: load_tool(left_hand, tool1)", - "identifier": 8, - "type_name": "action", - "effects": [ - { - "object_name": "left_hand", - "property_name": "is_free", - "property_value": null, - "status": false, - }, - { - "object_name": "tool1", - "property_name": "is_equippable", - "property_value": null, - "status": false, - }, - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "tool1", - "status": true, - }, - ], -}, -""" - -""" -{ - "summary": "selector to load inward_claw to left hand", - "name": "selector: load_tool(left_hand, inward_claw)", - "identifier": 3, - "type_name": "selector", - "children": [ - { - "summary": "check if left_hand is holding inward_claw", - "name": "target: hold(left_hand, inward_claw)", - "identifier": 4, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "inward_claw", - "status": true, - } - ], - }, - { - - "summary": "sequence to load inward_claw to left hand", - "name": "sequence: load_tool(left_hand, inward_claw)", - "identifier": 5, - "type_name": "sequence", - "children": [ - { - "summary": "check if inward_claw is equippable", - "name": "precondition: is_equippable(inward_claw)", - "identifier": 6, - "type_name": "condition", - "conditions": [ - { - "object_name": "inward_claw", - "property_name": "is_equippable", - "property_value": null, - "status": true, - } - ], - }, - { - "summary": "check if left hand is free", - "name": "precondition: is_free(left_hand)", - "identifier": 7, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "is_free", - "property_value": null, - "status": true, - } - ], - }, - { - "summary": "equip inward_claw to left hand", - "name": "action: load_tool(left_hand, inward_claw)", - "identifier": 8, - "type_name": "action", - "effects": [ - { - "object_name": "left_hand", - "property_name": "is_free", - "property_value": null, - "status": false, - }, - { - "object_name": "inward_claw", - "property_name": "is_equippable", - "property_value": null, - "status": false, - }, - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "inward_claw", - "status": true, - }, - ], - }, - ], - }, - ], -} -""" -The dictionary that you return should be formatted as json dictionary. Follow these rules: -1. Every node should have four basic keys: - - "summary": a short description of the node. - - "name": the name of the node. The name should follow the definitions in the domain knowledge. - - "identifier": the unique id of the node. - - "type_name": can be: "selector", "sequence", "condition", "action". -2. The control flow node, "selector" and "sequence" nodes, can have "children", which is a list of children nodes. -3. The "condition" nodes that are children of "selectors" are "targets". Those that are children of "sequences" are preconditions. -4. The "condition" nodes should have the key "conditions", which is a list of conditions to be checked. Each condition should have the following keys: - - "object_name": the object to check. - - "property_name": the property/relation of the object to be checked. - - "property_value": this should be the other object in the relation. For property it should be null. - - "status": true if the property/relation is expected to be true, false otherwise. -5. The "action" nodes should have a key "effects", which is a list of the to-be-exerted effects of the action. Each effect should have the following keys: - - "object_name": the affected object. - - "property_name": the affected property/relation of the object. - - "property_value": this should be the other object if this is a relation. For property it should be null. - - "status": true if the effect is to add this property/relation. false if is to remove it. -6. All the subtrees always have their corresponding action in ROBOT_ACTION_LIST. They are used to represent the actions in the action sequence. Its basic structure is: - - A selector as the root node. - - (Usually one) target condition node as the first child of the selector to check if the target of the action has been fulfilled. - - A sequence node as the last child of the selector node to achieve the target. - - (Usually a number of) precondition nodes as the front children of the sequence node to check the preconditions for executing the action. - - One action node as the last child of the sequence node, whose preconditions are checked before and will immediate start the job when being ticked. - - The action will exert its defined effects after finishing the job successfully. Those effects should fulfilled all the target conditions in the selector node. - A brief example: -{ - selector: load_tool selector, - children: [ - { - target: check if inward_claw is equipped by left hand, - }, - { - sequence: load_tool, - children: [ - precondition: check if inward_claw is equippable, - precondition: check if left hand is free, - {action: equip inward_claw to left hand, effects: [left hand not free, inward_claw not equippable, left_hand holds inward_claw]}, - ], - }, - ], -} - -7. When performing a new action to fulfill a unsatisfied precondition, you can construct a subtree to replace the precondition and take that unfulfilled precondition as the target of the subtree. Then you construct the sequence of the subtree according to the new action you want to perform. -8. The behavior tree should be constructed based on the action sequence you planned and the action definitions in ROBOT_ACTION_LIST. -9. All of the control flow nodes, namely selectors and sequences, are without memory. A new behavior tree tick will always start from the root and traverse all the nodes even if they are ticked in the last tick. - -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/data/prompts/sk_refine/prompt/refine_action.txt b/data/prompts/sk_refine/prompt/refine_action.txt deleted file mode 100644 index ebba9c19..00000000 --- a/data/prompts/sk_refine/prompt/refine_action.txt +++ /dev/null @@ -1,54 +0,0 @@ -[user] -For action nodes, you should: -1. refine and replace the summaries in the key "effects", which is a list of the to-be-exerted effects of the action. Each effect should have the following keys: - - "object_name": the affected object. - - "property_name": the affected property/relation of the object. - - "property_value": this should be the other object if this is a relation. For property it should be null. - - "status": true if the effect is to add this property/relation. false if is to remove it. -An example is shown below: -the skeleton: -""" -{ - "summary": "left hand equips tool1", - "name": "action: load_tool(left_hand, tool1)", - "effects": [ - {"summary": "left hand will be not free anymore",}, - {"summary": "tool1 will be not equippable anymore",}, - {"summary": "left_hand will hold tool1",}, - ], -}, -""" -after refinement: -""" -{ - "summary": "left hand equips tool1", - "name": "action: load_tool(left_hand, tool1)", - "identifier": 8, - "type_name": "action", - "effects": [ - { - "object_name": "left_hand", - "property_name": "is_free", - "property_value": null, - "status": false, - }, - { - "object_name": "tool1", - "property_name": "is_equippable", - "property_value": null, - "status": false, - }, - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "tool1", - "status": true, - }, - ], -}, -""" - -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/data/prompts/sk_refine/prompt/refine_chain.txt b/data/prompts/sk_refine/prompt/refine_chain.txt deleted file mode 100644 index 8ab56a97..00000000 --- a/data/prompts/sk_refine/prompt/refine_chain.txt +++ /dev/null @@ -1,15 +0,0 @@ -[user] -Follow this workflow to solve the task: - -1. Read the problem description. Parse and generate the "initial_state" in the output dictionary. -2. Interpret the goal of the problem as goal_state and write into the "instruction_summary" in the output dictionary. -3. Try to solve the problem, achieve the goal_state from the initial_state based on the domain knowledge provided. Generate a action sequence with the actions defined in and write it into dictionary["task_plan"]["action_sequence"] of the output. -4. If you cannot find a plan to solve the task, ask question with nature language in the "question" of the output dictionary. -5. Explain the actions in the action sequence and write the output dictionary["task_plan"]["action_instructions"]. -6. Collect the objects in the world that are effected by the action sequence, write into dictionary["task_plan"]["manipulated_objects"]. -7. Generate the behavior tree based on the action sequence and write into dictionary["task_plan"]["behavior_tree"]. - -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/data/prompts/sk_refine/prompt/refine_condition.txt b/data/prompts/sk_refine/prompt/refine_condition.txt deleted file mode 100644 index bb08d441..00000000 --- a/data/prompts/sk_refine/prompt/refine_condition.txt +++ /dev/null @@ -1,37 +0,0 @@ -[user] -For condition nodes, you should: -1. refine and add the key "conditions", which is a list of conditions to be checked. Each condition should have the following keys: - - "object_name": the object to check. - - "property_name": the property/relation of the object to be checked. - - "property_value": this should be the other object in the relation. For property it should be null. - - "status": true if the property/relation is expected to be true, false otherwise. -An example is shown below: -the skeleton: -""" -{ - "summary": "check if left_hand is holding parallel_box1", - "name": "target: hold(left_hand, parallel_box1)", -} -""" -after refinement: -""" -{ - "summary": "check if left_hand is holding parallel_box1", - "name": "target: hold(left_hand, parallel_box1)", - "identifier": 4, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "parallel_box1", - "status": true, - } - ], -}, -""" - -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/data/prompts/sk_refine/prompt/refine_controlflow.txt b/data/prompts/sk_refine/prompt/refine_controlflow.txt deleted file mode 100644 index d0f4c953..00000000 --- a/data/prompts/sk_refine/prompt/refine_controlflow.txt +++ /dev/null @@ -1,26 +0,0 @@ -[user] -For control flow nodes, namely selectors and sequences, an example is shown as follows: -the skeleton: -""" -{ - "summary": "selector to load no_tool to hand", - "name": "selector: load_tool", - "children": [...], -} - -""" -after refinement: -""" -{ - "summary": "selector to load no_tool to hand", - "name": "selector: load_tool(left_hand, no_tool)", - "identifier": 3, - "type_name": "selector", - "children": [...], -} -""" - -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/data/prompts/sk_refine/prompt/refine_domain.txt b/data/prompts/sk_refine/prompt/refine_domain.txt deleted file mode 100644 index 7f48353e..00000000 --- a/data/prompts/sk_refine/prompt/refine_domain.txt +++ /dev/null @@ -1,89 +0,0 @@ -[user] -Following is the domain knowledge you need in pddl domain format, which includes: -- the object types in the world, -- the predicates for describing the states of the world, -- the actions that the robot can apply, with their precondtions and effects defined. -""" -(define (domain robot_assembly_problem-domain) - (:requirements :strips :typing) - (:types - thing - object - tool part hand - thing - ) - (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part)) - (:action pick_up - :parameters ( ?hand - hand ?tool - tool ?part - part) - :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part)) - :effect (and (hold ?tool ?part) (not (is_free ?tool)))) - (:action put_down - :parameters ( ?hand - hand ?tool - tool ?part - part) - :precondition (and (hold ?tool ?part) (hold ?hand ?tool)) - :effect (and (not (hold ?hand ?part)) (is_free ?tool))) - (:action place - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2))) - (:action detach - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2)))) - (:action insert - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2))) - (:action pull - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2)))) - (:action screw - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2))) - (:action unscrew - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2)))) - (:action load_tool - :parameters ( ?hand - hand ?tool - tool) - :precondition (and (is_equippable ?tool) (is_free ?hand)) - :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool))) - (:action unload_tool - :parameters ( ?hand - hand ?tool - tool) - :precondition (and (hold ?hand ?tool) (is_free ?tool)) - :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool)))) -) -""" -The precidates are explained as follows: -""" -"PREDICATE_LIST" -- is_free( or ): or is free and can hold something. (default: true) -- is_equippable(): is ready to be eqquipped by a robot hand. (default: true) -- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false) -- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false) -- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false) -- can_place_to(, ): can be placed into (placement assembly constraint) (default: false) -- hold(, ): is holding (default: false) -- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false) -- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false) -- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false) -""" - -The actions are explained as follows: -""" -"ROBOT_ACTION_LIST" -- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something. -- put_down(, , ): use to put down . As preconditions, should be holding and should be holding . -- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding . -- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into . -- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding . -- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into . -- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding . -- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into . -- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free. -- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free. -""" -the items in <> indicate the type of the instances needed for the predicates and actions. -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. Waiting for next input. diff --git a/data/prompts/sk_refine/prompt/refine_help.txt b/data/prompts/sk_refine/prompt/refine_help.txt deleted file mode 100644 index 1ca165db..00000000 --- a/data/prompts/sk_refine/prompt/refine_help.txt +++ /dev/null @@ -1,11 +0,0 @@ -[user] -You should refine the behavior tree skeleton into a full behavior tree in json dictionary format. - -1. Refer to "initial_state" of input to find the necessary objects, properties and relationships. -2. Refer to the domain knowledge to understand the actions (corresponds to "ROBOT_ACTION_LIST") and conditions (corresponds to "PREDICATE_LIST"). -3. The name of the node (if it has) also provides information for its refinement. - -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/data/prompts/sk_refine/prompt/refine_input_format.txt b/data/prompts/sk_refine/prompt/refine_input_format.txt deleted file mode 100644 index 27a2fdef..00000000 --- a/data/prompts/sk_refine/prompt/refine_input_format.txt +++ /dev/null @@ -1,127 +0,0 @@ -[user] -The input you get and the output you return should be in json dictionary format. -The dictionary contains the following keys: -""" -- dictionary["task_plan"]: A dictionary containing the task plan. -- dictionary["initial_state"]: The initial world state before executing the plan. -- dictionary["instruction_summary"]: A brief summary of the task plan. -- dictionary["question"]: Question for more knowledge that are unknown yet but necessary for making the plan. Normally empty. -""" -Four keys exist in dictionary["task_plan"]. -""" -- dictionary["task_plan"]["action_sequence"]: Contains a list of robot actions. -- dictionary["task_plan"]["action_instructions"]: contains a list of instructions to explain the actions in dictionary["task_plan"]["action_sequence"]. -- dictionary["task_plan"]["manipulated_objects"]: The list of the objects that are related to the current task. -- dictionary["task_plan"]["behavior_tree"]: contains a dictionary of the behavior tree skeleton constructed according to the dictionary ["task_plan"]["action_sequence"] -""" -The following is an input example: -``` -{"task_plan": { - "action_sequence": [ - "load_tool(left_hand, parallel_box1)", - "pick_up(left_hand, parallel_box1, gear1)", - "insert(left_hand, parallel_box1, gear1, shaft1)" - ], - "action_instructions": [ - "equip the parallel_box1 on the left hand", - "pick up the gear1 with the parallel_box1 in the left hand", - "insert the gear1 into the shaft1 using the parallel_box1 in the left hand" - ], - "object_name": ["left_hand", "gear1", "parallel_box1", "shaft1"], - "behavior_tree": { - "summary": "selector to insert gear1 into shaft1", - "children": [ - { - "summary": "condition node to check if gear1 is inserted to shaft1", - }, - { - "summary": "sequence to insert gear1 into shaft1", - "children": [ - { - "summary": "selector to load_tool - "children": [ - { - "summary": "condition node to check if parallel_box1 is hold", - }, - { - "summary": "sequence to load_tool", - "children": [ - { - "summary": "condition node to check if parallel_box1 is equippable", - }, - { - "summary": "condition node to check if left hand is free", - }, - { - "summary": "action node to equip parallel_box1 to left hand", - "effects": [ - {"summary": "left_hand will be not free",}, - {"summary": "parallel_box1 will be not equippable",}, - {"summary": "left_hand will hold parallel_box1",}, - ], - }, - ], - }, - ], - }, - { - "summary": "selector to pick_up", - "children": [ - { - "summary": "condition node to check if parallel_box1 holds gear1", - }, - { - "summary": "sequence to pick_up", - "children": [ - { - "summary": "condition node to check if parallel_box1 is free", - }, - { - "summary": "condition node to check if left hand holds parallel_box1", - }, - { - "summary": "action node to pick up gear1 using parallel_box1", - "effects": [ - {"summary": "parallel_box1 will hold gear1",}, - {"summary": "parallel_box1 will be not free",}, - ], - }, - ], - }, - ], - }, - { - "summary": "condition node to check if gear1 can be inserted to shaft1", - }, - { - "summary": "action node to insert gear1 to shaft1", - "effects": [ - {"summary": "parallel_box1 will be not holding gear1",}, - {"summary": "parallel_box1 will be free",}, - {"summary": "gear1 will be inserted to shaft1",}, - ], - }, - ], - }, - ], -} -}, -"initial_state": {"objects": [ - {"name": "gear1", "properties": []}, - {"name": "shaft1", "properties": []}, - {"name": "parallel_box1", "properties": ["is_equippable", "is_free"]}, - {"name": "left_hand", "properties": ["is_free"]}, - ], - "constraints": [ - {"source": "gear1", "name": "can_insert_to", "target": "shaft1"}, - {"source": "parallel_box1", "name": "can_manipulate", "target": "gear1"}, - ], - "relations": [], -}, -"instruction_summary": "insert gear1 into shaft1", -"question":""} -``` -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/data/prompts/sk_refine/prompt/refine_output_format.txt b/data/prompts/sk_refine/prompt/refine_output_format.txt deleted file mode 100644 index d40e17ea..00000000 --- a/data/prompts/sk_refine/prompt/refine_output_format.txt +++ /dev/null @@ -1,19 +0,0 @@ -[user] -The output dictionary has four keys. -""" -- dictionary["task_plan"] -- dictionary["initial_state"] -- dictionary["instruction_summary"] -- dictionary["question"] -""" -Four keys exist in dictionary["task_plan"]. -""" -- dictionary["task_plan"]["action_sequence"] -- dictionary["task_plan"]["action_instructions"] -- dictionary["task_plan"]["manipulated_objects"] -- dictionary["task_plan"]["behavior_tree"] -""" -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/data/prompts/sk_refine/prompt/refine_role.txt b/data/prompts/sk_refine/prompt/refine_role.txt deleted file mode 100644 index de12cb04..00000000 --- a/data/prompts/sk_refine/prompt/refine_role.txt +++ /dev/null @@ -1,5 +0,0 @@ -[user] -You are an excellent task planner for robotic assembly tasks. Given the world state and a rough plan, you break it down and refine the plan into a well formatted and executable behavior tree. -Please do not start to work until I say "Start working." Instead, simply output the message "Waiting for next input." Understood? -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/data/prompts/sk_refine/prompt/refine_state.txt b/data/prompts/sk_refine/prompt/refine_state.txt deleted file mode 100644 index b84f1778..00000000 --- a/data/prompts/sk_refine/prompt/refine_state.txt +++ /dev/null @@ -1,27 +0,0 @@ -[user] -The initial state in the input is explained as follows: - - "objects": the objects and their properties (as lists of strings) in the world. The properties can be changed by robot actions. - - "constraints": the constraint relations that are enforced and should not be changed. They are knowledge about the world or some constraints set by the user. - - "relations": the relations in the world. They can be changed by robot actions. -following is another example: -""" -{ - "objects": [ - {"name": "leg1", "properties": []}, # leg1 is an object with no properties - {"name": "seat", "properties": []}, # seat is an object with no properties - {"name": "hand", "properties": ["is_free"]}, # hand is an object with a property "is_free" - ], - "constraints": [ - {"source": "leg1", "name": "can_screw_to", "target": "seat"}, # means leg1 can be screwed to seat - ], - "relations": [ - {"source": "leg1", "name": "is_screwed_to", "target": "seat"}, # means leg1 is screwed to seat - ], -} -""" -Only concentrate on the objects, constraints and relations that are related to the current task. - -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/data/prompts/sk_refine/query/dup_query.txt b/data/prompts/sk_refine/query/dup_query.txt deleted file mode 100644 index c1a7955e..00000000 --- a/data/prompts/sk_refine/query/dup_query.txt +++ /dev/null @@ -1,16 +0,0 @@ -Start working. Resume from the input below. -""" -{"input": [PROBLEM]} -""" -The dictionary that you return should be formatted as json file. Follow these rules: -1. Always equip the proper tools before performing specific object manipulaion. -2. Make sure that each element of the ["action_instructions"] explains corresponding element of the ["action_sequence"]. Refer to the "ROBOT ACTION LIST" to understand the elements of ["action_sequence"]. -3, The length of the ["action_instructions"] list and the ["action_sequence"] list should be the same. -4. Never left ',' at the end of the list. Use json boolean values, "true" or "false", instead of "True" or "False". -5. Keep track of all items listed in the "objects" section of the "initial_state" field. -6. Use the "PREDICATE_LIST" for the initial states (:init) in the problem to generate ["initial_state"]. -7. All keys of the dictionary should be double-quoted. -8. Insert ``` at the beginning and the end of the dictionary to separate it from the rest of your response. Don't write any additional things. -9. Make sure that you output a consistent manipultation as a single arm robot. For example, grasping an object should not occur in successive steps. -10. When generating the behavior tree, make sure the tree has the asked structure. -Adhere to the output format I defined above. Follow the rules above. Think step by step. \ No newline at end of file diff --git a/data/prompts/sk_refine/query/query.txt b/data/prompts/sk_refine/query/query.txt deleted file mode 100644 index 7abaa278..00000000 --- a/data/prompts/sk_refine/query/query.txt +++ /dev/null @@ -1,14 +0,0 @@ -Start working. Resume from the input below. -""" -{"input": [PROBLEM]} -""" -The dictionary that you return should be formatted as json file. Follow these rules: -1. Only refine the skeleton behavior tree in the "behavior_tree" field. Refine the tree according to rules provided before. Replace the old behavior tree with the refined behavior tree. -2. Refer to the domain knowledge, namely "ROBOT_ACTION_LIST" and "PREDICATE_LIST", to understand and refine the skeleton behavior tree. -3. Do not change other fields of the dictionary and return the new dictionary as json file. -4. Never left ',' at the end of the list. -5. Use json boolean values, "true" or "false", instead of "True" or "False". -6. All keys of the dictionary should be double-quoted. -7. Insert ``` at the beginning and the end of the dictionary to separate it from the rest of your response. Don't write any additional things. -8. When generating the behavior tree, make sure the tree has the asked structure. -Adhere to the output format I defined above. Follow the rules above. Think step by step. \ No newline at end of file diff --git a/data/prompts/sk_refine/system/system.txt b/data/prompts/sk_refine/system/system.txt deleted file mode 100644 index 5bc3c5a6..00000000 --- a/data/prompts/sk_refine/system/system.txt +++ /dev/null @@ -1 +0,0 @@ -You are an excellent robot behavior tree expert in robotic assembly tasks. Given the knowledge you need, the world state and a rough plan, you break it down and refine the task plan to make it well formatted and ready to be executed. \ No newline at end of file diff --git a/data/prompts/skeleton/prompt/backup b/data/prompts/skeleton/prompt/backup deleted file mode 100644 index c4a0f6f6..00000000 --- a/data/prompts/skeleton/prompt/backup +++ /dev/null @@ -1,43 +0,0 @@ - -There are also some counterexamples for you to refer to: -1. One sequence should never have multiple action nodes as its children like the behavior tree below. -{ - "summary": "selector to get the apple", - "name": "Get the apple", - "type_name": "selector", - "children": [ - { - "summary": "check if the apple is in hand", - "name": "target: in_hand(apple)", - "type_name": "condition", - }, - { - "summary": "sequence to get the apple", - "name": "sequence: get apple", - "type_name": "sequence", - "children": [ - { - "summary": "check if the apple is on the table", - "name": "precondition: on(apple, table)", - "type_name": "condition", - }, - { - "summary": "go to the table", - "name": "action: go_to(table)", - "type_name": "action", - }, - { - "summary": "check if the hand is free", - "name": "precondition: is_free(hand)", - "type_name": "condition", - }, - { - "summary": "pick up the apple", - "name": "action: pick_up(apple)", - "type_name": "action", - (effect omitted) - }, - ], - }, - ], -} \ No newline at end of file diff --git a/data/prompts/skeleton/prompt/sk_bt_skeleton.txt b/data/prompts/skeleton/prompt/sk_bt_skeleton.txt deleted file mode 100644 index b1225fcc..00000000 --- a/data/prompts/skeleton/prompt/sk_bt_skeleton.txt +++ /dev/null @@ -1,83 +0,0 @@ -[user] -The behavior tree should be in json dictionary format and have a structure like the following example: -""" -{ - "summary": "selector to load inward_claw to left hand", - "name": "selector: load_tool", - "type_name": "selector", - "children": [ - { - "summary": "check if inward_claw is held by left hand", - "name": "target: hold(left_hand, inward_claw)", - "type_name": "condition", - }, - { - "summary": "sequence to load inward_claw to left hand", - "name": "sequence: load_tool", - "type_name": "sequence", - "children": [ - { - "summary": "check if inward_claw is equippable", - "name": "precondition: is_equippable(inward_claw)", - "type_name": "condition", - }, - { - "summary": "check if left hand is free", - "name": "precondition: is_free(left_hand)", - "type_name": "condition", - }, - { - "summary": "equip inward_claw to left hand", - "name": "action: load_tool(left_hand, inward_claw)", - "type_name": "action", - "effects": [ - {"summary": "left hand will be not free anymore",}, - {"summary": "inward_claw will be not equippable anymore",}, - {"summary": "left_hand will hold inward_claw",}, - ], - }, - ], - }, - ], -} -""" -The dictionary that you return should be formatted as json dictionary. Follow these rules: -1. Every node should have three basic keys: - - "summary": a short interpretive description of the node - - "name": the name of the node. The name should follow the definitions in the domain knowledge. - - "type_name": can be: "selector", "sequence", "condition", "action" -2. The control flow nodes, "selector" and "sequence", can have "children", which is a list of children nodes. -3. The "condition" nodes that are children of "selectors" are "targets". Those that are children of "sequences" are preconditions. -4. The "action" nodes should have a key "effects" with a list of dictionaries, which describes the to-be-exerted effects of the action. The effects are shortly described under the key "summary". -5. All the subtrees always have their corresponding action in ROBOT_ACTION_LIST. They are used to represent the actions in the action sequence. The recommended structure is: - - A selector as the root node. - - (Usually one) target condition node as the first child of the selector to check if the target of the action has been fulfilled. - - A sequence node as the last child of the selector node to achieve the target. - - (Usually a number of) precondition nodes as the front children of the sequence node to check the preconditions for executing the action. - - One action node as the last child of the sequence node, whose preconditions are checked before and will immediate start the job when being ticked. - - The action will exert its defined effects after finishing the job successfully. Those effects should fulfilled all the target conditions in the selector node. - A brief example is as follows: -{ - selector: load_tool selector, - children: [ - { - target: check if inward_claw is equipped by left hand, - }, - { - sequence: load_tool, - children: [ - precondition: check if inward_claw is equippable, - precondition: check if left hand is free, - {action: equip inward_claw to left hand, effects: [left hand not free, inward_claw not equippable, left_hand holds inward_claw]}, - ], - }, - ], -} - -6. When performing a new action to fulfill a unsatisfied precondition, you can construct a subtree to replace the precondition and take that precondition as the target of the subtree. Then you construct the sequence of the subtree according to the new action you want to perform. -7. All of the control flow nodes, namely selectors and sequences, are without memory. A new behavior tree tick will always start from the root and traverse all the nodes even if they are ticked in the last tick. - -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/data/prompts/skeleton/prompt/sk_bt_skeleton_example.txt b/data/prompts/skeleton/prompt/sk_bt_skeleton_example.txt deleted file mode 100644 index af7cbe72..00000000 --- a/data/prompts/skeleton/prompt/sk_bt_skeleton_example.txt +++ /dev/null @@ -1,177 +0,0 @@ -[user] -I will give you some examples of the input and the output you will generate. -Example 1: -""" -- Input: -(define (problem robot_assembly_problem-problem) - (:domain robot_assembly_problem-domain) - (:objects - parallel_box1 parallel_box2 inward_claw outward_claw no_tool - tool - gear1 gear2 gear3 shaft1 shaft2 base - part - left_hand - hand - ) - (:init (can_manipulate parallel_box1 gear1) (can_manipulate outward_claw gear2) (can_manipulate inward_claw gear3) (can_manipulate parallel_box2 shaft1) (can_manipulate no_tool shaft2) (can_screw_to leg1 seat) (can_screw_to leg2 seat) (can_insert_to back seat) (can_screw_to nut1 seat) (can_screw_to nut2 seat) (can_screw_to blub base) (can_place_to lamp blub) (can_insert_to shaft1 base) (can_insert_to shaft2 base) (can_insert_to gear3 shaft2) (can_insert_to gear2 base) (can_insert_to gear1 shaft1) (is_inserted_to shaft1 base) (is_free left_hand) (is_free parallel_box1) (is_free parallel_box2) (is_free inward_claw) (is_free outward_claw) (is_free no_tool) (is_equippable parallel_box1) (is_equippable parallel_box2) (is_equippable inward_claw) (is_equippable outward_claw) (is_equippable no_tool)) - (:goal (and (is_inserted_to gear1 shaft1))) -) -- Output: -``` -{"task_plan": { - "action_sequence": [ - "load_tool(left_hand, parallel_box1)", - "pick_up(left_hand, parallel_box1, gear1)", - "insert(left_hand, parallel_box1, gear1, shaft1)" - ], - "action_instructions": [ - "equip the parallel_box1 on the left hand", - "pick up the gear1 with the parallel_box1 in the left hand", - "insert the gear1 into the shaft1 using the parallel_box1 in the left hand" - ], - "object_name": ["left_hand", "gear1", "parallel_box1", "shaft1"], - "behavior_tree": { - "summary": "selector to insert gear1 into shaft1", - "name": "selector: insert gear1 into shaft1", - "type_name": "selector", - "children": [ - { - "summary": "check if gear1 is inserted to shaft1", - "name": "target: is_inserted_to(gear1, shaft1)", - "type_name": "condition", - }, - { - "summary": "sequence to insert gear1 into shaft1", - "name": "sequence: insert(gear1, shaft1)", - "type_name": "sequence", - "children": [ - { - "summary": "selector to load parallel_box1", - "name": "selector: load_tool(left_hand, parallel_box1)", - "type_name": "selector", - "children": [ - { - "summary": "check if parallel_box1 is hold", - "name": "target: hold(left_hand, parallel_box1)", - "type_name": "condition", - }, - { - "summary": "sequence to load parallel_box1", - "name": "sequence: load_tool(left_hand, parallel_box1)", - "type_name": "sequence", - "children": [ - { - "summary": "check if parallel_box1 is equippable", - "name": "precondition: is_equippable(parallel_box1)", - "type_name": "condition", - }, - { - "summary": "check if left hand is free", - "name": "precondition: is_free(left_hand)", - "type_name": "condition", - }, - { - "summary": "equip parallel_box1 to left hand", - "name": "action: load_tool(left_hand, parallel_box1)", - "type_name": "action", - "effects": [ - { - "summary": "left_hand will be not free", - }, - { - "summary": "parallel_box1 will be not equippable", - }, - { - "summary": "left_hand will hold parallel_box1", - }, - ], - }, - ], - }, - ], - }, - { - "summary": "selector to pick_up gear1", - "name": "selector: pick_up(left_hand, parallel_box1, gear1)", - "type_name": "selector", - "children": [ - { - "summary": "check if parallel_box1 holds gear1", - "name": "target: hold(parallel_box1, gear1)", - "type_name": "condition", - }, - { - "summary": "sequence to pick_up gear1", - "name": "sequence: pick_up(left_hand, parallel_box1, gear1)", - "type_name": "sequence", - "children": [ - { - "summary": "check if parallel_box1 is free", - "name": "precondition: is_free(parallel_box1)", - "type_name": "condition", - }, - { - "summary": "check if left hand holds parallel_box1", - "name": "precondition: hold(left_hand, parallel_box1)", - "type_name": "condition", - }, - { - "summary": "pick up gear1 using parallel_box1", - "name": "action: pick_up(left_hand, parallel_box1, gear1)", - "type_name": "action", - "effects": [ - { - "summary": "parallel_box1 will hold gear1", - }, - { - "summary": "parallel_box1 will be not free", - }, - ], - }, - ], - }, - ], - }, - { - "summary": "check if gear1 can be inserted to shaft1", - "name": "precondition: can_insert_to(gear1, shaft1)", - "type_name": "condition", - }, - { - "summary": "insert gear1 to shaft1", - "name": "action: insert(left_hand, parallel_box1, gear1, shaft1)", - "type_name": "action", - "effects": [ - { - "summary": "parallel_box1 will be not holding gear1", - }, - { - "summary": "parallel_box1 will be free", - }, - { - "summary": "gear1 will be inserted to shaft1", - }, - ], - }, - ], - }, - ], -} -}, -"environment_before": {"objects": [ - {"name": "gear1", "properties": []}, - {"name": "shaft1", "properties": []}, - {"name": "parallel_box1", "properties": ["is_equippable", "is_free"]}, - {"name": "left_hand", "properties": ["is_free"]}, - ], - "constraints": [ - {"source": "gear1", "name": "can_insert_to", "target": "shaft1"}, - {"source": "parallel_box1", "name": "can_manipulate", "target": "gear1"}, - ], - "relations": [], -}, -"instruction_summary": "insert gear1 into shaft1", -"question":""} -``` -""" -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/data/prompts/skeleton/prompt/sk_bt_validation.txt b/data/prompts/skeleton/prompt/sk_bt_validation.txt deleted file mode 100644 index e69de29b..00000000 diff --git a/data/prompts/skeleton/prompt/sk_domain.txt b/data/prompts/skeleton/prompt/sk_domain.txt deleted file mode 100644 index bf3c7859..00000000 --- a/data/prompts/skeleton/prompt/sk_domain.txt +++ /dev/null @@ -1,89 +0,0 @@ -[user] -Following is the domain knowledge you need in pddl domain format, which includes: -- the object types in the world, -- the predicates for describing the states of the world, -- the actions that the robot can apply, with their precondtions and effects defined. -""" -(define (domain robot_assembly_problem-domain) - (:requirements :strips :typing) - (:types - thing - object - tool part hand - thing - ) - (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part)) - (:action pick_up - :parameters ( ?hand - hand ?tool - tool ?part - part) - :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part)) - :effect (and (hold ?tool ?part) (not (is_free ?tool)))) - (:action put_down - :parameters ( ?hand - hand ?tool - tool ?part - part) - :precondition (and (hold ?tool ?part) (hold ?hand ?tool)) - :effect (and (not (hold ?hand ?part)) (is_free ?tool))) - (:action place - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2))) - (:action detach - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2)))) - (:action insert - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2))) - (:action pull - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2)))) - (:action screw - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2))) - (:action unscrew - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2)))) - (:action load_tool - :parameters ( ?hand - hand ?tool - tool) - :precondition (and (is_equippable ?tool) (is_free ?hand)) - :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool))) - (:action unload_tool - :parameters ( ?hand - hand ?tool - tool) - :precondition (and (hold ?hand ?tool) (is_free ?tool)) - :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool)))) -) -""" -The precidates are explained as follows: -""" -"PREDICATE_LIST" -- is_free( or ): or is free and can hold something. (default: true) -- is_equippable(): is ready to be eqquipped by a robot hand. (default: true) -- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false) -- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false) -- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false) -- can_place_to(, ): can be placed into (placement assembly constraint) (default: false) -- hold(, ): is holding (default: false) -- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false) -- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false) -- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false) -""" - -The actions are explained as follows: -""" -"ROBOT_ACTION_LIST" -- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something. -- put_down(, , ): use to put down . As preconditions, should be holding and should be holding . -- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding . -- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into . -- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding . -- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into . -- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding . -- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into . -- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free. -- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free. -""" -the items in <> indicate the type of the instances needed for the predicates and actions. -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. diff --git a/data/prompts/skeleton/prompt/sk_environment.txt b/data/prompts/skeleton/prompt/sk_environment.txt deleted file mode 100644 index 4cd08f12..00000000 --- a/data/prompts/skeleton/prompt/sk_environment.txt +++ /dev/null @@ -1,31 +0,0 @@ -[user] -Information about world states are given as json dictionary. Example: -""" -{ - "objects": [ - {"name": "leg1", "properties": []}, - {"name": "seat", "properties": []}, - {"name": "cabinet", "properties": ["open"]}, - {"name": "parallel_box1", "properties": []}, - {"name": "table", "properties": []}, - ], - "constraints": [ - {"source": "leg1", "name": "can_screw_to", "target": "seat"}, - {"source": "parallel_box1", "name": "can_manipulate", "target": "leg1"}, - {"source": "cabinet", "name": "on_something", "target": "table"}, - ], - "relations": [ - {"source": "leg1", "name": "is_screwed_to", "target": "seat"}, - ], -} -""" -It contains three lists: - - "objects": the objects and their properties (as lists of strings) in the environment. The properties can be changed by robot actions. - - "constraints": the constraint relations that are enforced and should not be changed. They are knowledge about the world or some constraints set by the user. - - "relations": the relations in the environment. They can be changed by robot actions. -The properties, constraints and relations should always align with the [PREDICATE_LIST] in domain knowledge. -You should only concentrate on the objects, constraints and relations that are related to (or can be effected by) the current task. -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/data/prompts/skeleton/prompt/sk_output_format.txt b/data/prompts/skeleton/prompt/sk_output_format.txt deleted file mode 100644 index c7e7c60c..00000000 --- a/data/prompts/skeleton/prompt/sk_output_format.txt +++ /dev/null @@ -1,20 +0,0 @@ -[user] -You generate an action sequence to solve the task and contruct a behavior tree based on it in json format. -The output dictionary has five keys. -""" -- dictionary["task_plan"]: A dictionary containing information about the task plan. -- dictionary["environment_before"]: The environment state before executing the plan, described by the problem provided. -- dictionary["instruction_summary"]: A brief summary of the task plan. -- dictionary["question"]: If you don't have enough knowledge for solving the problem, you can ask questions about the knowledge you need. Leave this key empty if you can solve the task. -""" -Three keys exist in dictionary["task_plan"]. -""" -- dictionary["task_plan"]["action_sequence"]: Contains a list of robot actions. Only the actions defined in the "ROBOT_ACTION LIST" will be used. -- dictionary["task_plan"]["action_instructions"]: contains a list of instructions corresponding to dictionary["task_plan"]["action_sequence"] to explain the actions. -- dictionary["task_plan"]["manipulated_objects"]: The list of the names of the manipulated object. It only includes the objects that are related to the current task. -- dictionary["task_plan"]["behavior_tree"]: contains a dictionary of the behavior tree constructed according to the dictionary ["task_plan"]["action_sequence"] -""" -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/data/prompts/skeleton/prompt/sk_problem.txt b/data/prompts/skeleton/prompt/sk_problem.txt deleted file mode 100644 index ec47c472..00000000 --- a/data/prompts/skeleton/prompt/sk_problem.txt +++ /dev/null @@ -1,23 +0,0 @@ -[user] -Information about the problem is given as a pddl problem. -It includes the following information: -- Objects in the world: all instances of the types defined in the domain knowledge. For example, "parallel_box1" is an instance of the type "tool", while "gear1" is an instance of the type "part". -- Initial states: the world states to start with, described with [PREDICATE_LIST] from domain knowledge. For example, "is_free hand" means the object "hand" is free and holds nothing. -- Goal states: the world states to achieve, described with [PREDICATE_LIST] from domain knowledge. -Example: -""" -(define (problem robot_assembly_problem-problem) - (:domain robot_assembly_problem-domain) - (:objects - parallel_box1 parallel_box2 inward_claw outward_claw no_tool - tool - gear1 gear2 gear3 shaft1 shaft2 base - part - left_hand - hand - ) - (:init (can_manipulate parallel_box1 gear1) (can_manipulate outward_claw gear2) (can_manipulate inward_claw gear3) (can_manipulate parallel_box2 shaft1) (can_manipulate no_tool shaft2) (can_screw_to leg1 seat) (can_screw_to leg2 seat) (can_insert_to back seat) (can_screw_to nut1 seat) (can_screw_to nut2 seat) (can_screw_to blub base) (can_place_to lamp blub) (can_insert_to shaft1 base) (can_insert_to shaft2 base) (can_insert_to gear3 shaft2) (can_insert_to gear2 base) (can_insert_to gear1 shaft1) (is_free left_hand) (is_free parallel_box1) (is_free parallel_box2) (is_free inward_claw) (is_free outward_claw) (is_free no_tool) 1) (is_equippable parallel_box1) (is_equippable parallel_box2) (is_equippable inward_claw) (is_equippable outward_claw) (is_equippable no_tool)) - (:goal (and (is_inserted_to shaft1 base))) -) -""" -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/data/prompts/skeleton/prompt/sk_role.txt b/data/prompts/skeleton/prompt/sk_role.txt deleted file mode 100644 index 6ce736a0..00000000 --- a/data/prompts/skeleton/prompt/sk_role.txt +++ /dev/null @@ -1,5 +0,0 @@ -[user] -You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you break it down and construct robot task plans represented by behaiovr trees. -Please do not begin working until I say "Start working." Instead, simply output the message "Waiting for next input." Understood? -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/data/prompts/skeleton/query/query.txt b/data/prompts/skeleton/query/query.txt deleted file mode 100644 index 8ec7310d..00000000 --- a/data/prompts/skeleton/query/query.txt +++ /dev/null @@ -1,16 +0,0 @@ -Start working. Resume from the problem below. -""" -{"problem": [PROBLEM]} -""" -The dictionary that you return should be formatted as json file. Follow these rules: -1. Always equip the proper tools before performing specific object manipulaion. -2. Make sure that each element of the ["action_instructions"] explains corresponding element of the ["action_sequence"]. Refer to the "ROBOT ACTION LIST" to understand the elements of ["action_sequence"]. -3, The length of the ["action_instructions"] list and the ["action_sequence"] list should be the same. -4. Never left ',' at the end of the list. Use json boolean values, "true" or "false", instead of "True" or "False". -5. Keep track of all items listed in the "objects" section of the "environment_before" field. Please make sure you fill out the "objects" section for all listed items. -6. Use the "PREDICATE_LIST" for the states in the problem. -7. All keys of the dictionary should be double-quoted. -8. Insert ``` at the beginning and the end of the dictionary to separate it from the rest of your response. -9. Make sure that you output a consistent manipultation as a single arm robot. For example, grasping an object should not occur in successive steps. -10. When generating the behavior tree, make sure the tree has the asked structure. -Adhere to the output format I defined above. Follow the rules above. Think step by step. \ No newline at end of file diff --git a/data/prompts/skeleton/system/system.txt b/data/prompts/skeleton/system/system.txt deleted file mode 100644 index 26f2afe2..00000000 --- a/data/prompts/skeleton/system/system.txt +++ /dev/null @@ -1 +0,0 @@ -You are an excellent interpreter of instructions for robotic assembly tasks. Given the knowledge you need, the world states and some instructions, you break it down and construct robot task plans represented by behaiovr trees based on the action sequences. \ No newline at end of file diff --git a/data/prompts/skeleton_generator/prompt v2/cot_sk_bt copy.txt b/data/prompts/skeleton_generator/prompt v2/cot_sk_bt copy.txt deleted file mode 100644 index 10faf8a0..00000000 --- a/data/prompts/skeleton_generator/prompt v2/cot_sk_bt copy.txt +++ /dev/null @@ -1,95 +0,0 @@ -[user] -The behavior tree should be in json dictionary format and have a structure like the following example: -""" -{ - "summary": "selector to use left hand to load inward_claw", - "name": "selector: load_tool(left_hand, inward_claw)", - "children": [ - { - "summary": "condition node to check if left_hand holds inward_claw", - "name": "target: hold(left_hand, inward_claw)", - }, - { - "summary": "sequence to use left hand to load inward_claw", - "name": "sequence: load_tool(left_hand, inward_claw)", - "children": [ - { - "summary": "condition node to check if inward_claw is equippable", - "name": "precondition: is_equippable(inward_claw)", - }, - { - "summary": "condition node to check if left hand is free", - "name": "precondition: is_free(left_hand)", - }, - { - "summary": "action node to use left hand to load inward_claw", - "name": "action: load_tool(left_hand, inward_claw)", - "effects": [ - {"summary": "left hand will be not free anymore",}, - {"summary": "inward_claw will be not equippable anymore",}, - {"summary": "left_hand will hold inward_claw",}, - ], - }, - ], - }, - ], -} -""" -Follow these rules: -1. Every node are described by a summary and a name: - - "summary": a short interpretive description of the node. - - "name": includes the node type and the formal form of the node according to the domain knowledge. -2. The control flow nodes, "selector" and "sequence", can have "children", which is a list of children nodes. -3. The "condition" nodes that are children of "selectors" are targets. Those that are children of "sequences" are preconditions. -4. The "action" nodes should have a key "effects" with a list of dictionaries, which describes the to-be-exerted effects of the action. The effects are shortly described under the key "summary". -5. All the subtrees always have their corresponding action in ROBOT_ACTION_LIST. -This is how their structure is: - - A selector as the root node. - - (Usually one) target condition node as the first child of the selector to check if the target of the action has been fulfilled. - - A sequence node as the last child of the selector node to achieve the target. - - (Usually a number of) precondition nodes as the front children of the sequence node to check the preconditions for executing the action. - - One action node as the last child of the sequence node, whose preconditions are checked before and will immediate start the job when being ticked. - - The action will exert its defined effects after finishing the job successfully. Those effects should fulfilled all the target conditions in the selector node. -6. To perform a action to fulfill a unsatisfied precondition, you can construct a subtree to replace the precondition at where it is and take that unfulfilled precondition as the target of the subtree. -7. All of the control flow nodes, namely selectors and sequences, are without memory. A new behavior tree tick will always start from the root and traverse all the nodes even if they are ticked in the last tick. - -An example: -The action constructed based on the "load_tool" from ROBOT_ACTION_LIST: -load_tool(left_hand, parallel_box1) -The bahavior tree constructed according to its definition in ROBOT_ACTION_LIST: -{ - "summary": "selector to load left_hand with parallel_box1", - "name": "selector: load_tool(left_hand, parallel_box1)", - "children": [ - { - "summary": "condition node to check if left_hand holds parallel_box1", - "name": "target: hold(left_hand, parallel_box1)" - }, - { - "summary": "sequence to load left_hand with parallel_box1", - "name": "sequence: load_tool(left_hand, parallel_box1)", - "children": [ - { - "summary": "condition node to check if left_hand is free, - "name": "precondition: is_free(left_hand)" - }, - { - "summary": "condition node to check if parallel_box1 is equippable", - "name": "precondition: is_equippable(parallel_box1)" - }, - { - "summary": "action node to equip left hand with parallel_box1", - "name": "action: load_tool(left_hand, parallel_box1)", - "effects": [ - {"summary": "left_hand will be not free",}, - {"summary": "parallel_box1 will be not equippable",}, - {"summary": "left_hand will hold parallel_box1",}, - ], - } - ], - }, - ], -} - -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/data/prompts/skeleton_generator/prompt v2/cot_sk_bt.txt b/data/prompts/skeleton_generator/prompt v2/cot_sk_bt.txt deleted file mode 100644 index 10faf8a0..00000000 --- a/data/prompts/skeleton_generator/prompt v2/cot_sk_bt.txt +++ /dev/null @@ -1,95 +0,0 @@ -[user] -The behavior tree should be in json dictionary format and have a structure like the following example: -""" -{ - "summary": "selector to use left hand to load inward_claw", - "name": "selector: load_tool(left_hand, inward_claw)", - "children": [ - { - "summary": "condition node to check if left_hand holds inward_claw", - "name": "target: hold(left_hand, inward_claw)", - }, - { - "summary": "sequence to use left hand to load inward_claw", - "name": "sequence: load_tool(left_hand, inward_claw)", - "children": [ - { - "summary": "condition node to check if inward_claw is equippable", - "name": "precondition: is_equippable(inward_claw)", - }, - { - "summary": "condition node to check if left hand is free", - "name": "precondition: is_free(left_hand)", - }, - { - "summary": "action node to use left hand to load inward_claw", - "name": "action: load_tool(left_hand, inward_claw)", - "effects": [ - {"summary": "left hand will be not free anymore",}, - {"summary": "inward_claw will be not equippable anymore",}, - {"summary": "left_hand will hold inward_claw",}, - ], - }, - ], - }, - ], -} -""" -Follow these rules: -1. Every node are described by a summary and a name: - - "summary": a short interpretive description of the node. - - "name": includes the node type and the formal form of the node according to the domain knowledge. -2. The control flow nodes, "selector" and "sequence", can have "children", which is a list of children nodes. -3. The "condition" nodes that are children of "selectors" are targets. Those that are children of "sequences" are preconditions. -4. The "action" nodes should have a key "effects" with a list of dictionaries, which describes the to-be-exerted effects of the action. The effects are shortly described under the key "summary". -5. All the subtrees always have their corresponding action in ROBOT_ACTION_LIST. -This is how their structure is: - - A selector as the root node. - - (Usually one) target condition node as the first child of the selector to check if the target of the action has been fulfilled. - - A sequence node as the last child of the selector node to achieve the target. - - (Usually a number of) precondition nodes as the front children of the sequence node to check the preconditions for executing the action. - - One action node as the last child of the sequence node, whose preconditions are checked before and will immediate start the job when being ticked. - - The action will exert its defined effects after finishing the job successfully. Those effects should fulfilled all the target conditions in the selector node. -6. To perform a action to fulfill a unsatisfied precondition, you can construct a subtree to replace the precondition at where it is and take that unfulfilled precondition as the target of the subtree. -7. All of the control flow nodes, namely selectors and sequences, are without memory. A new behavior tree tick will always start from the root and traverse all the nodes even if they are ticked in the last tick. - -An example: -The action constructed based on the "load_tool" from ROBOT_ACTION_LIST: -load_tool(left_hand, parallel_box1) -The bahavior tree constructed according to its definition in ROBOT_ACTION_LIST: -{ - "summary": "selector to load left_hand with parallel_box1", - "name": "selector: load_tool(left_hand, parallel_box1)", - "children": [ - { - "summary": "condition node to check if left_hand holds parallel_box1", - "name": "target: hold(left_hand, parallel_box1)" - }, - { - "summary": "sequence to load left_hand with parallel_box1", - "name": "sequence: load_tool(left_hand, parallel_box1)", - "children": [ - { - "summary": "condition node to check if left_hand is free, - "name": "precondition: is_free(left_hand)" - }, - { - "summary": "condition node to check if parallel_box1 is equippable", - "name": "precondition: is_equippable(parallel_box1)" - }, - { - "summary": "action node to equip left hand with parallel_box1", - "name": "action: load_tool(left_hand, parallel_box1)", - "effects": [ - {"summary": "left_hand will be not free",}, - {"summary": "parallel_box1 will be not equippable",}, - {"summary": "left_hand will hold parallel_box1",}, - ], - } - ], - }, - ], -} - -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/data/prompts/skeleton_generator/prompt v2/cot_sk_chain.txt b/data/prompts/skeleton_generator/prompt v2/cot_sk_chain.txt deleted file mode 100644 index 50016de5..00000000 --- a/data/prompts/skeleton_generator/prompt v2/cot_sk_chain.txt +++ /dev/null @@ -1,14 +0,0 @@ -[user] -Follow this workflow to solve the task: - -1. Read the problem description. Parse it and use it to complete the "constraints" in "initial_state". -2. Fill the "objects" and "relations" in "initial_state" based on the input world state. -3. Interpret the input instructions as goal_state. -4. Try to make a plan, achieve the goal_state from the initial_state based on the domain knowledge provided. Generate a action sequence with the actions defined in and write it into dictionary["task_plan"]["action_sequence"] of the output. -5. If you cannot find a plan to solve the task, ask question with nature language in the "question" of the output dictionary. -6. Collect the objects that are effected by the action sequence, write into dictionary["task_plan"]["manipulated_objects"]. -7. Generate the behavior tree based on the action sequence. -8. check the behavior tree, run it through with the world state and expand the behavior tree if necessary. Write the expanded behavior tree into dictionary["task_plan"]["behavior_tree"]. - -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/data/prompts/skeleton_generator/prompt v2/cot_sk_domain.txt b/data/prompts/skeleton_generator/prompt v2/cot_sk_domain.txt deleted file mode 100644 index ba5c088b..00000000 --- a/data/prompts/skeleton_generator/prompt v2/cot_sk_domain.txt +++ /dev/null @@ -1,88 +0,0 @@ -[user] -Following is the domain knowledge you need in pddl domain format, which includes: -- the object types in the world, -- the predicates for describing the states of the world, -- the actions that the robot can apply, with their precondtions and effects defined. -""" -(define (domain robot_assembly_problem-domain) - (:requirements :strips :typing) - (:types - thing - object - tool part hand - thing - ) - (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part)) - (:action pick_up - :parameters ( ?hand - hand ?tool - tool ?part - part) - :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part)) - :effect (and (hold ?tool ?part) (not (is_free ?tool)))) - (:action put_down - :parameters ( ?hand - hand ?tool - tool ?part - part) - :precondition (and (hold ?tool ?part) (hold ?hand ?tool)) - :effect (and (not (hold ?hand ?part)) (is_free ?tool))) - (:action place - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2))) - (:action detach - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2)))) - (:action insert - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2))) - (:action pull - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2)))) - (:action screw - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2))) - (:action unscrew - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2)))) - (:action load_tool - :parameters ( ?hand - hand ?tool - tool) - :precondition (and (is_equippable ?tool) (is_free ?hand)) - :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool))) - (:action unload_tool - :parameters ( ?hand - hand ?tool - tool) - :precondition (and (hold ?hand ?tool) (is_free ?tool)) - :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool)))) -) -""" -The precidates are explained as follows: -""" -"PREDICATE_LIST" -- is_free( or ): or is free and can hold something. (default: true) -- is_equippable(): is ready to be eqquipped by a robot hand. (default: true) -- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false) -- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false) -- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false) -- can_place_to(, ): can be placed into (placement assembly constraint) (default: false) -- hold(, ): is holding (default: false) -- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false) -- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false) -- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false) -""" - -The actions are explained as follows: -""" -"ROBOT_ACTION_LIST" -- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something. -- put_down(, , ): use to put down . As preconditions, should be holding and should be holding . -- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding . -- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into . -- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding . -- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into . -- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding . -- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into . -- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free. -- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free. -""" -the items in <> indicate the type of the instances needed for the predicates and actions. - -[assistant] -Understood. I will wait for further instructions before starting to work. diff --git a/data/prompts/skeleton_generator/prompt v2/cot_sk_example copy 3.txt b/data/prompts/skeleton_generator/prompt v2/cot_sk_example copy 3.txt deleted file mode 100644 index f68b479a..00000000 --- a/data/prompts/skeleton_generator/prompt v2/cot_sk_example copy 3.txt +++ /dev/null @@ -1,283 +0,0 @@ -[user] -I will give you some examples of the input and the output you will generate. -Example 1: -""" -- Input: -problem: -(define (problem robot_assembly_problem-problem) - (:domain robot_assembly_problem-domain) - (:objects - parallel_box1 parallel_box2 inward_claw outward_claw no_tool - tool - gear1 gear2 gear3 shaft1 shaft2 base - part - left_hand - hand - ) - (:init (can_manipulate parallel_box1 gear1) (can_manipulate outward_claw gear2) (can_manipulate inward_claw gear3) (can_manipulate parallel_box2 shaft1) (can_manipulate no_tool shaft2) (can_screw_to leg1 seat) (can_screw_to leg2 seat) (can_insert_to back seat) (can_screw_to nut1 seat) (can_screw_to nut2 seat) (can_screw_to blub base) (can_place_to lamp blub) (can_insert_to shaft1 base) (can_insert_to shaft2 base) (can_insert_to gear3 shaft2) (can_insert_to gear2 base) (can_insert_to gear1 shaft1) (is_inserted_to shaft1 base) (is_free left_hand) (is_free parallel_box1) (is_free parallel_box2) (is_free inward_claw) (is_free outward_claw) (is_free no_tool) (is_equippable parallel_box1) (is_equippable parallel_box2) (is_equippable inward_claw) (is_equippable outward_claw) (is_equippable no_tool)) -) -world_state: -{"objects": [ - {"name": "gear1", "properties": []}, - {"name": "shaft1", "properties": []}, - {"name": "parallel_box1", "properties": ["is_equippable", "is_free"]}, - {"name": "left_hand", "properties": ["is_free"]}, - ], - "constraints": [], - "relations": [], -} -instructions: insert gear1 into shaft1 - -- Output: -``` -{"task_plan": { - "action_sequence": [ - "load_tool(left_hand, parallel_box1)", - "pick_up(left_hand, parallel_box1, gear1)", - "insert(left_hand, parallel_box1, gear1, shaft1)" - ], - "action_instructions": [ - "equip the parallel_box1 on the left hand", - "pick up the gear1 with the parallel_box1 in the left hand", - "insert the gear1 into the shaft1 using the parallel_box1 in the left hand" - ], - "object_name": ["left_hand", "gear1", "parallel_box1", "shaft1"], - "behavior_tree": { - "summary": "selector to insert gear1 into shaft1", - "name": "selector: insert(left_hand, parallel_box1, gear1, shaft1)", - "children": [ - { - "summary": "condition node to check if gear1 is inserted to shaft1", - "name": "target: is_inserted_to(gear1, shaft1)", - }, - { - "summary": "sequence to insert gear1 into shaft1", - "name": "sequence: insert(left_hand, parallel_box1, gear1, shaft1)", - "children": [ - { - "summary": "selector to load parallel_box1", - "name": "selector: load_tool(left_hand, parallel_box1)", - "children": [ - { - "summary": "condition node to check if left_hand holds parallel_box1", - "name": "target: hold(left_hand, parallel_box1)" - }, - { - "summary": "sequence to load parallel_box1", - "name": "sequence: load_tool(left_hand, parallel_box1)", - "children": [ - { - "summary": "condition node to check if parallel_box1 is equippable", - "name": "precondition: is_equippable(parallel_box1)" - }, - { - "summary": "condition node to check if left hand is free", - "name": "precondition: is_free(left_hand)" - }, - { - "summary": "action node to equip parallel_box1 to left hand", - "name": "action: load_tool(left_hand, parallel_box1)", - "effects": [ - {"summary": "left_hand will be not free",}, - {"summary": "parallel_box1 will be not equippable",}, - {"summary": "left_hand will hold parallel_box1",}, - ], - }, - ], - }, - ], - }, - { - "summary": "selector to use left_hand with parallel_box1 to pick up gear1", - "name": "selector: pick_up(left_hand, parallel_box1, gear1)", - "children": [ - { - "summary": "condition node to check if parallel_box1 holds gear1", - "name": "target: hold(parallel_box1, gear1)" - }, - { - "summary": "sequence to use left hand with parallel_box1 to pick up gear1", - "name": "sequence: pick_up(left_hand, parallel_box1, gear1)", - "children": [ - { - "summary": "condition node to check if parallel_box1 is free", - "name": "precondition: is_free(parallel_box1)" - }, - { - "summary": "condition node to check if left hand holds parallel_box1", - "name": "precondition: hold(left_hand, parallel_box1)" - }, - { - "summary": "action node to use left hand with parallel_box1 to pick up gear1", - "name": "action: pick_up(left_hand, parallel_box1, gear1)", - "effects": [ - {"summary": "parallel_box1 will hold gear1",}, - {"summary": "parallel_box1 will be not free",}, - ], - }, - ], - }, - ], - }, - { - "summary": "condition node to check if gear1 can be inserted to shaft1", - "name": "precondition: can_insert_to(gear1, shaft1)", - }, - { - "summary": "action node to use left_hand with parallel_box1 to insert gear1 to shaft1", - "name": "action: insert(left_hand, parallel_box1, gear1, shaft1)", - "effects": [ - {"summary": "parallel_box1 will be not holding gear1",}, - {"summary": "parallel_box1 will be free",}, - {"summary": "gear1 will be inserted to shaft1",}, - ], - }, - ], - }, - ], -} -}, -"initial_state": {"objects": [ - {"name": "gear1", "properties": []}, - {"name": "shaft1", "properties": []}, - {"name": "parallel_box1", "properties": ["is_equippable", "is_free"]}, - {"name": "left_hand", "properties": ["is_free"]}, - ], - "constraints": [ - {"source": "gear1", "name": "can_insert_to", "target": "shaft1"}, - {"source": "parallel_box1", "name": "can_manipulate", "target": "gear1"}, - ], - "relations": [], -}, -"question":""} -``` -""" - -Example 2: -""" -- Input: -problem: -(define (problem robot_assembly_problem-problem) - (:domain robot_assembly_problem-domain) - (:objects - parallel_box1 parallel_box2 inward_claw outward_claw no_tool - tool - gear1 gear2 gear3 shaft1 shaft2 base - part - left_hand - hand - ) - (:init (can_manipulate parallel_box1 gear1) (can_manipulate outward_claw gear2) (can_manipulate inward_claw gear3) (can_manipulate parallel_box2 shaft1) (can_manipulate no_tool shaft2) (can_screw_to leg1 seat) (can_screw_to leg2 seat) (can_insert_to back seat) (can_screw_to nut1 seat) (can_screw_to nut2 seat) (can_screw_to blub base) (can_place_to lamp blub) (can_insert_to shaft1 base) (can_insert_to shaft2 base) (can_insert_to gear3 shaft2) (can_insert_to gear2 base) (can_insert_to gear1 shaft1) (is_inserted_to shaft1 base) (is_free left_hand) (is_free parallel_box1) (is_free parallel_box2) (is_free inward_claw) (is_free outward_claw) (is_free no_tool) (is_equippable parallel_box1) (is_equippable parallel_box2) (is_equippable inward_claw) (is_equippable outward_claw) (is_equippable no_tool)) -) -world_state: -{"objects": [ - {"name": "gear1", "properties": []}, - {"name": "shaft1", "properties": []}, - {"name": "outward_claw", "properties": ["is_free"]}, - {"name": "parallel_box1", "properties": ["is_equippable", "is_free"]}, - {"name": "left_hand", "properties": []}, - ], - "constraints": [], - "relations": [ - { - "source": "left_hand", - "name": "hold", - "target": "outward_claw" - } - ], -} -instructions: load left_hand with parallel_box1 - -- Output: -``` -{"task_plan": { - "action_sequence": [ - "unload_tool(left_hand, outward_claw)", - "load_tool(left_hand, parallel_box1)" - ], - "action_instructions": [ - "unload the outward_claw from the left hand", - "equip the left_hand with the parallel_box1" - ], - "object_name": ["left_hand", "outward_claw", "parallel_box1"], - "behavior_tree": { - "summary": "selector to load left_hand with parallel_box1", - "name": "selector: load_tool(left_hand, parallel_box1)", - "children": [ - { - "summary": "condition node to check if left_hand holds parallel_box1", - "name": "target: hold(left_hand, parallel_box1)" - }, - { - "summary": "sequence to load left_hand with parallel_box1", - "name": "sequence: load_tool(left_hand, parallel_box1)", - "children": [ - { - "summary": "selector to make left_hand unload outward_claw", - "name": "selector: unload_tool(left_hand, outward_claw)", - "children": [ - { - "summary": "condition node to check if left_hand is free", - "name": "target: is_free(left_hand)" - }, - { - "summary": "sequence to make left_hand unload outward_claw", - "name": "sequence: unload_tool(left_hand, outward_claw)", - "children": [ - { - "summary": "condition node to check if outward_claw is free", - "name": "precondition: is_free(outward_claw)" - }, - { - "summary": "condition node to check if left hand holds outward_claw", - "name": "precondition: hold(left_hand, outward_claw)" - }, - { - "summary": "action node to make left hand unload outward_claw", - "name": "action: unload_tool(left_hand, outward_claw)", - "effects": [ - {"summary": "left_hand will be free",}, - {"summary": "outward_claw will be equippable",}, - {"summary": "left_hand will not hold outward_claw",}, - ], - }, - ], - }, - ], - }, - { - "summary": "condition node to check if parallel_box1 is equippable", - "name": "precondition: is_equippable(parallel_box1)" - }, - { - "summary": "action node to equip left hand with parallel_box1", - "name": "action: load_tool(left_hand, parallel_box1)", - "effects": [ - {"summary": "left_hand will be not free",}, - {"summary": "parallel_box1 will be not equippable",}, - {"summary": "left_hand will hold parallel_box1",}, - ], - } - ], - }, - ], -} -}, -"initial_state": {"objects": [ - {"name": "gear1", "properties": []}, - {"name": "shaft1", "properties": []}, - {"name": "outward_claw", "properties": ["is_free"]}, - {"name": "parallel_box1", "properties": ["is_equippable", "is_free"]}, - {"name": "left_hand", "properties": []}, - ], - "constraints": [ - {"source": "gear1", "name": "can_insert_to", "target": "shaft1"}, - {"source": "parallel_box1", "name": "can_manipulate", "target": "gear1"}, - ], - "relations": [ - { - "source": "left_hand", - "name": "hold", - "target": "outward_claw" - } - ], -}, -"question":""} -``` -""" -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/data/prompts/skeleton_generator/prompt v2/cot_sk_example.txt b/data/prompts/skeleton_generator/prompt v2/cot_sk_example.txt deleted file mode 100644 index e4de8094..00000000 --- a/data/prompts/skeleton_generator/prompt v2/cot_sk_example.txt +++ /dev/null @@ -1,196 +0,0 @@ -[user] -I will give you some examples of the input and the output you will generate. -Example 1: -""" -- Input: -problem: -(define (problem robot_assembly_problem-problem) - (:domain robot_assembly_problem-domain) - (:objects - parallel_box1 parallel_box2 inward_claw outward_claw no_tool - tool - gear1 gear2 gear3 shaft1 shaft2 base - part - left_hand - hand - ) - (:init (can_manipulate parallel_box1 gear1) (can_manipulate outward_claw gear2) (can_manipulate inward_claw gear3) (can_manipulate parallel_box2 shaft1) (can_manipulate no_tool shaft2) (can_screw_to leg1 seat) (can_screw_to leg2 seat) (can_insert_to back seat) (can_screw_to nut1 seat) (can_screw_to nut2 seat) (can_screw_to blub base) (can_place_to lamp blub) (can_insert_to shaft1 base) (can_insert_to shaft2 base) (can_insert_to gear3 shaft2) (can_insert_to gear2 base) (can_insert_to gear1 shaft1)) -) -world_state: -{ - "objects": [ - {"name": "parallel_box1", "properties": ["is_free", "is_equippable"]}, - {"name": "parallel_box2", "properties": ["is_free", "is_equippable"]}, - {"name": "inward_claw", "properties": ["is_free", "is_equippable"]}, - {"name": "outward_claw", "properties": ["is_free"]}, - {"name": "no_tool", "properties": ["is_free", "is_equippable"]}, - {"name": "gear1", "properties": []}, - {"name": "gear2", "properties": []}, - {"name": "gear3", "properties": []}, - {"name": "shaft1", "properties": []}, - {"name": "shaft2", "properties": []}, - {"name": "shaft3", "properties": []}, - {"name": "gearbase", "properties": []}, - {"name": "gearbase_hole1", "properties": []}, - {"name": "gearbase_hole3", "properties": []}, - {"name": "left_hand", "properties": []} - ], - "constraints": [], - "relations": [ - {"source": "left_hand","name": "hold","target": "outward_claw"} - ] -} -instructions: insert gear1 into shaft1 - -- Output: -``` -{"task_plan": { - "action_sequence": [ - "unload_tool(left_hand, outward_claw)", - "load_tool(left_hand, parallel_box1)", - "pick_up(left_hand, parallel_box1, gear1)", - "insert(left_hand, parallel_box1, gear1, shaft1)" - ], - "object_name": ["left_hand", "gear1","outward_claw", "parallel_box1", "shaft1"], - "behavior_tree": { - "summary": "selector to insert gear1 into shaft1", - "name": "selector: insert(left_hand, parallel_box1, gear1, shaft1)", - "children": [ - { - "summary": "condition node to check if gear1 is inserted to shaft1", - "name": "target: is_inserted_to(gear1, shaft1)", - }, - { - "summary": "sequence to insert gear1 into shaft1", - "name": "sequence: insert(left_hand, parallel_box1, gear1, shaft1)", - "children": [ - { - "summary": "selector to load parallel_box1", - "name": "selector: load_tool(left_hand, parallel_box1)", - "children": [ - { - "summary": "condition node to check if left_hand holds parallel_box1", - "name": "target: hold(left_hand, parallel_box1)" - }, - { - "summary": "sequence to load parallel_box1", - "name": "sequence: load_tool(left_hand, parallel_box1)", - "children": [ - { - "summary": "condition node to check if parallel_box1 is equippable", - "name": "precondition: is_equippable(parallel_box1)" - }, - { - "summary": "selector to make left_hand unload outward_claw", - "name": "selector: unload_tool(left_hand, outward_claw)", - "children": [ - { - "summary": "condition node to check if left_hand is free, - "name": "target: is_free(left_hand)" - }, - { - "summary": "sequence to make left_hand unload outward_claw", - "name": "sequence: unload_tool(left_hand, outward_claw)", - "children": [ - { - "summary": "condition node to check if outward_claw is free", - "name": "precondition: is_free(outward_claw)" - }, - { - "summary": "condition node to check if left hand holds outward_claw", - "name": "precondition: hold(left_hand, outward_claw)" - }, - { - "summary": "action node to make left hand unload outward_claw", - "name": "action: unload_tool(left_hand, outward_claw)", - "effects": [ - {"summary": "left_hand will be free",}, - {"summary": "outward_claw will be equippable",}, - {"summary": "left_hand will not hold outward_claw",}, - ], - }, - ], - }, - ], - }, - { - "summary": "action node to equip parallel_box1 to left hand", - "name": "action: load_tool(left_hand, parallel_box1)", - "effects": [ - {"summary": "left_hand will be not free",}, - {"summary": "parallel_box1 will be not equippable",}, - {"summary": "left_hand will hold parallel_box1",}, - ], - }, - ], - }, - ], - }, - { - "summary": "selector to use left_hand with parallel_box1 to pick up gear1", - "name": "selector: pick_up(left_hand, parallel_box1, gear1)", - "children": [ - { - "summary": "condition node to check if parallel_box1 holds gear1", - "name": "target: hold(parallel_box1, gear1)" - }, - { - "summary": "sequence to use left hand with parallel_box1 to pick up gear1", - "name": "sequence: pick_up(left_hand, parallel_box1, gear1)", - "children": [ - { - "summary": "condition node to check if parallel_box1 is free", - "name": "precondition: is_free(parallel_box1)" - }, - { - "summary": "condition node to check if left hand holds parallel_box1", - "name": "precondition: hold(left_hand, parallel_box1)" - }, - { - "summary": "action node to use left hand with parallel_box1 to pick up gear1", - "name": "action: pick_up(left_hand, parallel_box1, gear1)", - "effects": [ - {"summary": "parallel_box1 will hold gear1",}, - {"summary": "parallel_box1 will be not free",}, - ], - }, - ], - }, - ], - }, - { - "summary": "condition node to check if gear1 can be inserted to shaft1", - "name": "precondition: can_insert_to(gear1, shaft1)", - }, - { - "summary": "action node to use left_hand with parallel_box1 to insert gear1 to shaft1", - "name": "action: insert(left_hand, parallel_box1, gear1, shaft1)", - "effects": [ - {"summary": "parallel_box1 will be not holding gear1",}, - {"summary": "parallel_box1 will be free",}, - {"summary": "gear1 will be inserted to shaft1",}, - ], - }, - ], - }, - ], -} -}, -"initial_state": {"objects": [ - {"name": "gear1", "properties": []}, - {"name": "shaft1", "properties": []}, - {"name": "outward_claw", "properties": ["is_free"]}, - {"name": "parallel_box1", "properties": ["is_equippable", "is_free"]}, - {"name": "left_hand", "properties": []}, - ], - "constraints": [ - {"source": "gear1", "name": "can_insert_to", "target": "shaft1"}, - {"source": "parallel_box1", "name": "can_manipulate", "target": "gear1"}, - ], - "relations": [ - {"source": "left_hand", "name": "hold", "target": "outward_claw"}, - ], -}, -"question":""} -``` -""" - -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/data/prompts/skeleton_generator/prompt v2/cot_sk_instruction.txt b/data/prompts/skeleton_generator/prompt v2/cot_sk_instruction.txt deleted file mode 100644 index 29546e4b..00000000 --- a/data/prompts/skeleton_generator/prompt v2/cot_sk_instruction.txt +++ /dev/null @@ -1,7 +0,0 @@ -[user] -User's instruction is an instruction string. -You interpret the user instruction into goal state based on the domain knowledge. -Then you make the plan to achieve it. - -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/data/prompts/skeleton_generator/prompt v2/cot_sk_output_format.txt b/data/prompts/skeleton_generator/prompt v2/cot_sk_output_format.txt deleted file mode 100644 index 9eafd742..00000000 --- a/data/prompts/skeleton_generator/prompt v2/cot_sk_output_format.txt +++ /dev/null @@ -1,17 +0,0 @@ -[user] -You will be provided with a problem and the necessary domain knowledge. -You generate an action sequence to solve the task and contruct a behavior tree based on it in json format. -The output dictionary has five keys. -""" -- dictionary["task_plan"]: A dictionary containing the task plan. -- dictionary["initial_state"]: The initial world state before executing the plan. -- dictionary["question"]: If you don't have enough knowledge for solving the problem, you can ask questions about the knowledge you need. Leave this key empty if you can solve the task. -""" -Three keys exist in dictionary["task_plan"]. -""" -- dictionary["task_plan"]["action_sequence"]: Contains a list of robot actions. Only the actions defined in the "ROBOT_ACTION LIST" will be used. -- dictionary["task_plan"]["manipulated_objects"]: The list of the manipulated objects. It only includes the objects that are related to the current task. -- dictionary["task_plan"]["behavior_tree"]: contains a dictionary of the behavior tree constructed according to the dictionary ["task_plan"]["action_sequence"] -""" -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/data/prompts/skeleton_generator/prompt v2/cot_sk_problem.txt b/data/prompts/skeleton_generator/prompt v2/cot_sk_problem.txt deleted file mode 100644 index d6c7fa26..00000000 --- a/data/prompts/skeleton_generator/prompt v2/cot_sk_problem.txt +++ /dev/null @@ -1,22 +0,0 @@ -[user] -Information about the problem is given as a pddl problem. -It includes the following information: -- objects: all the objects in the world. They should be of a type defined in the domain knowledge. For example, "parallel_box1" is an instance of the type "tool", while "gear1" is an instance of the type "part". -- init: the constraint relations in the world, described with [PREDICATE_LIST] from domain knowledge. They cannot be changed by robot actions. -- goal: IGNORE this and follow user's instructions instead. -An example: -""" -(define (problem robot_assembly_problem-problem) - (:domain robot_assembly_problem-domain) - (:objects - parallel_box1 parallel_box2 inward_claw outward_claw no_tool - tool - gear1 gear2 gear3 shaft1 shaft2 base - part - left_hand - hand - ) - (:init (can_manipulate parallel_box1 gear1) (can_manipulate outward_claw gear2) (can_manipulate inward_claw gear3) (can_manipulate parallel_box2 shaft1) (can_manipulate no_tool shaft2) (can_screw_to leg1 seat) (can_screw_to leg2 seat) (can_insert_to back seat) (can_screw_to nut1 seat) (can_screw_to nut2 seat) (can_screw_to blub base) (can_place_to lamp blub) (can_insert_to shaft1 base) (can_insert_to shaft2 base) (can_insert_to gear3 shaft2) (can_insert_to gear2 base) (can_insert_to gear1 shaft1)S) - (:goal (and )) -) -""" - -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/data/prompts/skeleton_generator/prompt v2/cot_sk_role.txt b/data/prompts/skeleton_generator/prompt v2/cot_sk_role.txt deleted file mode 100644 index 19f9b3f0..00000000 --- a/data/prompts/skeleton_generator/prompt v2/cot_sk_role.txt +++ /dev/null @@ -1,4 +0,0 @@ -[user] -You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you break it down and construct robot task plans represented by behaiovr trees. -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/data/prompts/skeleton_generator/prompt v2/cot_sk_state.txt b/data/prompts/skeleton_generator/prompt v2/cot_sk_state.txt deleted file mode 100644 index 66541898..00000000 --- a/data/prompts/skeleton_generator/prompt v2/cot_sk_state.txt +++ /dev/null @@ -1,25 +0,0 @@ -[user] -Information about world states should be in json dictionary format. Example: -""" -{ - "objects": [ - {"name": "leg1", "properties": []}, # leg1 is an object with no properties - {"name": "seat", "properties": []}, # seat is an object with no properties - {"name": "hand", "properties": ["is_free"]}, # hand is an object with a property "is_free" - ], - "constraints": [ - {"source": "leg1", "name": "can_screw_to", "target": "seat"}, # means leg1 can be screwed to seat - ], - "relations": [ - {"source": "leg1", "name": "is_screwed_to", "target": "seat"}, # means leg1 is screwed to seat - ], -} -""" -It contains three lists: - - "objects": the objects and their properties (as lists of strings) in the world. The properties can be changed by robot actions. Refer to the input world state for them. - - "constraints": the constraint relations that are enforced and should not be changed. Refer to the pddl problem for the constraints. - - "relations": the relations in the world. They can be changed by robot actions. Refer to input world state for them. -The properties, constraints and relations should always align with the [PREDICATE_LIST] in domain knowledge. - -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/data/prompts/skeleton_generator/prompt/cot_sk_bt copy.txt b/data/prompts/skeleton_generator/prompt/cot_sk_bt copy.txt deleted file mode 100644 index 39fabdbc..00000000 --- a/data/prompts/skeleton_generator/prompt/cot_sk_bt copy.txt +++ /dev/null @@ -1,55 +0,0 @@ -[user] -The behavior tree should be in json dictionary format. -Follow these rules: -1. Every node are described by a summary and a name: - - "summary": a short interpretive description of the node. - - "name": includes the node type and the formal form of the node according to the domain knowledge. -2. The control flow nodes, "selector" and "sequence", can have "children", which is a list of children nodes. -3. The "condition" nodes that are children of "selectors" are targets. Those that are children of "sequences" are preconditions. -4. All the subtrees always have their corresponding action in ROBOT_ACTION_LIST. -Their structure is: - - A selector as the root node. - - (Usually one) target condition node as the first child of the selector to check if the target of the action has been fulfilled. - - A sequence node as the last child of the selector node to achieve the target. - - (Usually a number of) precondition nodes as the front children of the sequence node to check the preconditions for executing the action. - - One action node as the last child of the sequence node, whose preconditions are checked before and will immediate start the job when being ticked. - - The action will exert its defined effects after finishing the job successfully. Those effects should fulfilled all the target conditions in the selector node. -6. To fulfill an unsatisfied precondition, you can replace it with a subtree that treats the precondition as the target. -7. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state. -8. The selector node can only be used to replace an existing unsatisfied condition node. When using a selector node, you should check if success. If not, it should return failure. - -An example: -The action based on the "load_tool" from : -load_tool(left_hand, parallel_box1) -The bahavior tree constructed according to its definition in : -{ - "summary": "selector to load left_hand with parallel_box1", - "name": "selector: load_tool(left_hand, parallel_box1)", - "children": [ - { - "summary": "condition node to check if left_hand holds parallel_box1", - "name": "target: hold(left_hand, parallel_box1)" - }, - { - "summary": "sequence to load left_hand with parallel_box1", - "name": "sequence: load_tool(left_hand, parallel_box1)", - "children": [ - { - "summary": "condition node to check if left_hand is free, - "name": "precondition: is_free(left_hand)" - }, - { - "summary": "condition node to check if parallel_box1 is equippable", - "name": "precondition: is_equippable(parallel_box1)" - }, - { - "summary": "action node to equip left hand with parallel_box1", - "name": "action: load_tool(left_hand, parallel_box1)", - } - ], - }, - ], -} - -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/data/prompts/skeleton_generator/prompt/cot_sk_bt effect.txt b/data/prompts/skeleton_generator/prompt/cot_sk_bt effect.txt deleted file mode 100644 index 9e5b15e2..00000000 --- a/data/prompts/skeleton_generator/prompt/cot_sk_bt effect.txt +++ /dev/null @@ -1,95 +0,0 @@ -[user] -The behavior tree should be in json dictionary format and have a structure like the following example: -""" -{ - "summary": "selector to use left hand to load inward_claw", - "name": "selector: load_tool(left_hand, inward_claw)", - "children": [ - { - "summary": "condition node to check if left_hand holds inward_claw", - "name": "target: hold(left_hand, inward_claw)", - }, - { - "summary": "sequence to use left hand to load inward_claw", - "name": "sequence: load_tool(left_hand, inward_claw)", - "children": [ - { - "summary": "condition node to check if inward_claw is equippable", - "name": "precondition: is_equippable(inward_claw)", - }, - { - "summary": "condition node to check if left hand is free", - "name": "precondition: is_free(left_hand)", - }, - { - "summary": "action node to use left hand to load inward_claw", - "name": "action: load_tool(left_hand, inward_claw)", - "effects": [ - {"summary": "left hand will be not free anymore",}, - {"summary": "inward_claw will be not equippable anymore",}, - {"summary": "left_hand will hold inward_claw",}, - ], - }, - ], - }, - ], -} -""" -Follow these rules: -1. Every node are described by a summary and a name: - - "summary": a short interpretive description of the node. - - "name": includes the node type and the formal form of the node according to the domain knowledge. -2. The control flow nodes, "selector" and "sequence", can have "children", which is a list of children nodes. -3. The "condition" nodes that are children of "selectors" are targets. Those that are children of "sequences" are preconditions. -4. The "action" nodes should have a key "effects" with a list of dictionaries, which describes the to-be-exerted effects of the action. The effects are shortly described under the key "summary". -5. All the subtrees always have their corresponding action in ROBOT_ACTION_LIST. -This is how their structure is: - - A selector as the root node. - - (Usually one) target condition node as the first child of the selector to check if the target of the action has been fulfilled. - - A sequence node as the last child of the selector node to achieve the target. - - (Usually a number of) precondition nodes as the front children of the sequence node to check the preconditions for executing the action. - - One action node as the last child of the sequence node, whose preconditions are checked before and will immediate start the job when being ticked. - - The action will exert its defined effects after finishing the job successfully. Those effects should fulfilled all the target conditions in the selector node. -6. To perform a action to fulfill a unsatisfied precondition, you can construct a subtree to replace the precondition at where it is and take that unfulfilled precondition as the target of the subtree. -7. All of the control flow nodes, namely selectors and sequences, are without memory. A new behavior tree tick will always start from the root and traverse all the nodes even if they are ticked in the last tick. - -An example: -The action based on the "load_tool" from : -load_tool(left_hand, parallel_box1) -The bahavior tree constructed according to its definition in : -{ - "summary": "selector to load left_hand with parallel_box1", - "name": "selector: load_tool(left_hand, parallel_box1)", - "children": [ - { - "summary": "condition node to check if left_hand holds parallel_box1", - "name": "target: hold(left_hand, parallel_box1)" - }, - { - "summary": "sequence to load left_hand with parallel_box1", - "name": "sequence: load_tool(left_hand, parallel_box1)", - "children": [ - { - "summary": "condition node to check if left_hand is free, - "name": "precondition: is_free(left_hand)" - }, - { - "summary": "condition node to check if parallel_box1 is equippable", - "name": "precondition: is_equippable(parallel_box1)" - }, - { - "summary": "action node to equip left hand with parallel_box1", - "name": "action: load_tool(left_hand, parallel_box1)", - "effects": [ - {"summary": "left_hand will be not free",}, - {"summary": "parallel_box1 will be not equippable",}, - {"summary": "left_hand will hold parallel_box1",}, - ], - } - ], - }, - ], -} - -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/data/prompts/skeleton_generator/prompt/cot_sk_bt.txt b/data/prompts/skeleton_generator/prompt/cot_sk_bt.txt deleted file mode 100644 index 6ecf1e65..00000000 --- a/data/prompts/skeleton_generator/prompt/cot_sk_bt.txt +++ /dev/null @@ -1,50 +0,0 @@ -[user] -The behavior tree should be in json dictionary format. -Follow these rules: -1. Every node are described by a summary and a name: - - "summary": a short interpretive description of the node. - - "name": includes the node type and the formal form of the node according to the domain knowledge. -2. The control flow nodes, namely "selector" and "sequence", can have "children", which is a list of children nodes. -3. The "condition" nodes that are children of "selectors" are targets. Those that are children of "sequences" are preconditions. -4. All the subtrees always have their corresponding action in ROBOT_ACTION_LIST. -5. The basic structure template of a behavior tree is: - - A selector as the root node. - - One target condition node as the first child of the selector to check if the target of the action has been fulfilled. - - A sequence node as the last child of the selector node to achieve the target. - - A number of precondition nodes as the front children of the sequence node to check the preconditions for executing the action. - - One action node as the last child of the sequence node, whose preconditions are checked before and will immediate start its action when being ticked. - - The action should be able to fulfill the target condition node in the selector. -6. This is how you expand a behavior tree: - - A selector is only allowed to be used to replace an existing unfulfilled precondition node in the tree. - - When this happens, that precondition node will become the target condition node of the selector. - - After that, find an action from domain knowledge that can fulfill the target condition node. - - Then create a sequence node based on the preconditions of the action and the action itself. - - Finally, add the sequence node as the last child of the selector node. -7. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state. - -Following is an example of how to expand a behavior tree: -old tree: -{ - selector - - target - - sequence - - precondition1 - - precondition2 (failed according to the world state here) - - action -} -new tree: -{ - selector - - target - - sequence - - precondition1 - - new selector - - new target (precondition2) - - new sequence - - new action's preconditions - - new action - - action -} - -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/data/prompts/skeleton_generator/prompt/cot_sk_chain.txt b/data/prompts/skeleton_generator/prompt/cot_sk_chain.txt deleted file mode 100644 index b2656c48..00000000 --- a/data/prompts/skeleton_generator/prompt/cot_sk_chain.txt +++ /dev/null @@ -1,13 +0,0 @@ -[user] -Follow this workflow to solve the task: - -1. Read the objects and the world state. Understand the type of the objects and the current state of the world. -3. Interpret the input instructions as goal_state. -4. Try to make a plan to achieve the goal_state from the initial_state based on the domain knowledge provided. Generate a action sequence with the actions defined in and write it into dictionary["task_plan"]["action_sequence"] of the output. -5. If you cannot find a plan to solve the task, ask question with nature language in the "question" of the output dictionary. -6. Collect the objects that are effected by the action sequence, write into dictionary["task_plan"]["manipulated_objects"]. -7. Generate the behavior tree based on the action sequence. -8. check the behavior tree, run it through with the world state and expand the behavior tree if necessary. Write the expanded behavior tree into dictionary["task_plan"]["behavior_tree"]. - -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/data/prompts/skeleton_generator/prompt/cot_sk_domain.txt b/data/prompts/skeleton_generator/prompt/cot_sk_domain.txt deleted file mode 100644 index ba5c088b..00000000 --- a/data/prompts/skeleton_generator/prompt/cot_sk_domain.txt +++ /dev/null @@ -1,88 +0,0 @@ -[user] -Following is the domain knowledge you need in pddl domain format, which includes: -- the object types in the world, -- the predicates for describing the states of the world, -- the actions that the robot can apply, with their precondtions and effects defined. -""" -(define (domain robot_assembly_problem-domain) - (:requirements :strips :typing) - (:types - thing - object - tool part hand - thing - ) - (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part)) - (:action pick_up - :parameters ( ?hand - hand ?tool - tool ?part - part) - :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part)) - :effect (and (hold ?tool ?part) (not (is_free ?tool)))) - (:action put_down - :parameters ( ?hand - hand ?tool - tool ?part - part) - :precondition (and (hold ?tool ?part) (hold ?hand ?tool)) - :effect (and (not (hold ?hand ?part)) (is_free ?tool))) - (:action place - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2))) - (:action detach - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2)))) - (:action insert - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2))) - (:action pull - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2)))) - (:action screw - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2))) - (:action unscrew - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2)))) - (:action load_tool - :parameters ( ?hand - hand ?tool - tool) - :precondition (and (is_equippable ?tool) (is_free ?hand)) - :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool))) - (:action unload_tool - :parameters ( ?hand - hand ?tool - tool) - :precondition (and (hold ?hand ?tool) (is_free ?tool)) - :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool)))) -) -""" -The precidates are explained as follows: -""" -"PREDICATE_LIST" -- is_free( or ): or is free and can hold something. (default: true) -- is_equippable(): is ready to be eqquipped by a robot hand. (default: true) -- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false) -- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false) -- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false) -- can_place_to(, ): can be placed into (placement assembly constraint) (default: false) -- hold(, ): is holding (default: false) -- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false) -- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false) -- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false) -""" - -The actions are explained as follows: -""" -"ROBOT_ACTION_LIST" -- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something. -- put_down(, , ): use to put down . As preconditions, should be holding and should be holding . -- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding . -- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into . -- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding . -- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into . -- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding . -- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into . -- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free. -- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free. -""" -the items in <> indicate the type of the instances needed for the predicates and actions. - -[assistant] -Understood. I will wait for further instructions before starting to work. diff --git a/data/prompts/skeleton_generator/prompt/cot_sk_example effect.txt b/data/prompts/skeleton_generator/prompt/cot_sk_example effect.txt deleted file mode 100644 index cfea96bb..00000000 --- a/data/prompts/skeleton_generator/prompt/cot_sk_example effect.txt +++ /dev/null @@ -1,192 +0,0 @@ -[user] -I will give you some examples of the input and the output you will generate. -Example 1: -""" -- Input: -objects: -{ - "tool": ["parallel_box1", "parallel_box2", "inward_claw", "outward_claw", "no_tool"], - "part": ["gear1", "gear2", "gear3", "shaft1", "shaft2", "shaft3", "gearbase", "gearbase_hole1", "gearbase_hole3"] - "hand": ["left_hand"] -} -world_state: -{ - "objects": [ - {"name": "parallel_box1", "properties": ["is_free", "is_equippable"]}, - {"name": "parallel_box2", "properties": ["is_free", "is_equippable"]}, - {"name": "inward_claw", "properties": ["is_free", "is_equippable"]}, - {"name": "outward_claw", "properties": ["is_free"]}, - {"name": "no_tool", "properties": ["is_free", "is_equippable"]}, - {"name": "gear1", "properties": []}, - {"name": "gear2", "properties": []}, - {"name": "gear3", "properties": []}, - {"name": "shaft1", "properties": []}, - {"name": "shaft2", "properties": []}, - {"name": "shaft3", "properties": []}, - {"name": "gearbase", "properties": []}, - {"name": "gearbase_hole1", "properties": []}, - {"name": "gearbase_hole3", "properties": []}, - {"name": "left_hand", "properties": []} - ], - "constraints": [ - {"name": "can_manipulate", "args": ["parallel_box1", "gear1"]}, - {"name": "can_manipulate", "args": ["outward_claw", "gear2"]}, - {"name": "can_manipulate", "args": ["inward_claw", "gear3"]}, - {"name": "can_manipulate", "args": ["parallel_box2", "shaft1"]}, - {"name": "can_manipulate", "args": ["no_tool", "shaft2"]}, - {"name": "can_insert_to", "args": ["gear1", "shaft1"]}, - {"name": "can_insert_to", "args": ["gear2", "base"]}, - {"name": "can_insert_to", "args": ["gear3", "shaft2"]}, - {"name": "can_insert_to", "args": ["shaft1", "base"]}, - {"name": "can_insert_to", "args": ["shaft2", "base"]}, - {"name": "can_insert_to", "args": ["shaft3", "base"]}, - {"name": "can_insert_to", "args": ["gearbase", "base"]}, - {"name": "can_insert_to", "args": ["gearbase_hole1", "gearbase"]}, - {"name": "can_insert_to", "args": ["gearbase_hole3", "gearbase"]}, - ], - "relations": [ - {"source": "left_hand","name": "hold","target": "outward_claw"} - ] -} -instructions: "insert gear1 into shaft1" - -- Output: -``` -{"task_plan": { - "action_sequence": [ - "unload_tool(left_hand, outward_claw)", - "load_tool(left_hand, parallel_box1)", - "pick_up(left_hand, parallel_box1, gear1)", - "insert(left_hand, parallel_box1, gear1, shaft1)" - ], - "object_name": ["left_hand", "gear1","outward_claw", "parallel_box1", "shaft1"], - "behavior_tree": { - "summary": "selector to insert gear1 into shaft1", - "name": "selector: insert(left_hand, parallel_box1, gear1, shaft1)", - "children": [ - { - "summary": "condition node to check if gear1 is inserted to shaft1", - "name": "target: is_inserted_to(gear1, shaft1)", - }, - { - "summary": "sequence to insert gear1 into shaft1", - "name": "sequence: insert(left_hand, parallel_box1, gear1, shaft1)", - "children": [ - { - "summary": "selector to load parallel_box1", - "name": "selector: load_tool(left_hand, parallel_box1)", - "children": [ - { - "summary": "condition node to check if left_hand holds parallel_box1", - "name": "target: hold(left_hand, parallel_box1)" - }, - { - "summary": "sequence to load parallel_box1", - "name": "sequence: load_tool(left_hand, parallel_box1)", - "children": [ - { - "summary": "condition node to check if parallel_box1 is equippable", - "name": "precondition: is_equippable(parallel_box1)" - }, - { - "summary": "selector to make left_hand unload outward_claw", - "name": "selector: unload_tool(left_hand, outward_claw)", - "children": [ - { - "summary": "condition node to check if left_hand is free, - "name": "target: is_free(left_hand)" - }, - { - "summary": "sequence to make left_hand unload outward_claw", - "name": "sequence: unload_tool(left_hand, outward_claw)", - "children": [ - { - "summary": "condition node to check if outward_claw is free", - "name": "precondition: is_free(outward_claw)" - }, - { - "summary": "condition node to check if left hand holds outward_claw", - "name": "precondition: hold(left_hand, outward_claw)" - }, - { - "summary": "action node to make left hand unload outward_claw", - "name": "action: unload_tool(left_hand, outward_claw)", - "effects": [ - {"summary": "left_hand will be free",}, - {"summary": "outward_claw will be equippable",}, - {"summary": "left_hand will not hold outward_claw",}, - ], - }, - ], - }, - ], - }, - { - "summary": "action node to equip parallel_box1 to left hand", - "name": "action: load_tool(left_hand, parallel_box1)", - "effects": [ - {"summary": "left_hand will be not free",}, - {"summary": "parallel_box1 will be not equippable",}, - {"summary": "left_hand will hold parallel_box1",}, - ], - }, - ], - }, - ], - }, - { - "summary": "selector to use left_hand with parallel_box1 to pick up gear1", - "name": "selector: pick_up(left_hand, parallel_box1, gear1)", - "children": [ - { - "summary": "condition node to check if parallel_box1 holds gear1", - "name": "target: hold(parallel_box1, gear1)" - }, - { - "summary": "sequence to use left hand with parallel_box1 to pick up gear1", - "name": "sequence: pick_up(left_hand, parallel_box1, gear1)", - "children": [ - { - "summary": "condition node to check if parallel_box1 is free", - "name": "precondition: is_free(parallel_box1)" - }, - { - "summary": "condition node to check if left hand holds parallel_box1", - "name": "precondition: hold(left_hand, parallel_box1)" - }, - { - "summary": "action node to use left hand with parallel_box1 to pick up gear1", - "name": "action: pick_up(left_hand, parallel_box1, gear1)", - "effects": [ - {"summary": "parallel_box1 will hold gear1",}, - {"summary": "parallel_box1 will be not free",}, - ], - }, - ], - }, - ], - }, - { - "summary": "condition node to check if gear1 can be inserted to shaft1", - "name": "precondition: can_insert_to(gear1, shaft1)", - }, - { - "summary": "action node to use left_hand with parallel_box1 to insert gear1 to shaft1", - "name": "action: insert(left_hand, parallel_box1, gear1, shaft1)", - "effects": [ - {"summary": "parallel_box1 will be not holding gear1",}, - {"summary": "parallel_box1 will be free",}, - {"summary": "gear1 will be inserted to shaft1",}, - ], - }, - ], - }, - ], -} -}, -"question":""} -``` -""" - -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/data/prompts/skeleton_generator/prompt/cot_sk_example.txt b/data/prompts/skeleton_generator/prompt/cot_sk_example.txt deleted file mode 100644 index a0d3c147..00000000 --- a/data/prompts/skeleton_generator/prompt/cot_sk_example.txt +++ /dev/null @@ -1,154 +0,0 @@ -[user] -Following are some examples of the input and output. -Learn how to expand behavior tree from the examples. -Example 1: -""" -- Input: -objects: -{ - "tool": ["parallel_box1", "inward_claw", "outward_claw"], - "part": ["gear1", "shaft1", "gearbase_hole1", "gearbase_hole3"] - "hand": ["left_hand"] -} -world_state: -{ - "objects": [ - {"name": "parallel_box1", "properties": ["is_free", "is_equippable"]}, - {"name": "inward_claw", "properties": ["is_free", "is_equippable"]}, - {"name": "outward_claw", "properties": ["is_free"]}, - {"name": "gear1", "properties": []}, - {"name": "shaft1", "properties": []}, - {"name": "gearbase_hole1", "properties": []}, - {"name": "left_hand", "properties": []} - ], - "constraints": [ - {"name": "can_manipulate", "args": ["parallel_box1", "gear1"]}, - {"name": "can_insert_to", "args": ["gear1", "shaft1"]}, - ], - "relations": [ - {"source": "left_hand","name": "hold","target": "outward_claw"} - ] -} -instructions: "insert gear1 into shaft1" - -- Output: -``` -{"task_plan": { - "action_sequence": [ - "unload_tool(left_hand, outward_claw)", - "load_tool(left_hand, parallel_box1)", - "pick_up(left_hand, parallel_box1, gear1)", - "insert(left_hand, parallel_box1, gear1, shaft1)" - ], - "object_name": ["left_hand", "gear1","outward_claw", "parallel_box1", "shaft1"], - "behavior_tree": { - "summary": "selector to insert gear1 into shaft1", - "name": "selector: insert(left_hand, parallel_box1, gear1, shaft1)", - "children": [ - { - "summary": "condition node to check if gear1 is inserted to shaft1", - "name": "target: is_inserted_to(gear1, shaft1)", - }, - { - "summary": "sequence to insert gear1 into shaft1", - "name": "sequence: insert(left_hand, parallel_box1, gear1, shaft1)", - "children": [ - { - "summary": "selector to load parallel_box1", - "name": "selector: load_tool(left_hand, parallel_box1)", - "children": [ - { - "summary": "condition node to check if left_hand holds parallel_box1", - "name": "target: hold(left_hand, parallel_box1)" - }, - { - "summary": "sequence to load parallel_box1", - "name": "sequence: load_tool(left_hand, parallel_box1)", - "children": [ - { - "summary": "condition node to check if parallel_box1 is equippable", - "name": "precondition: is_equippable(parallel_box1)" - }, - { - "summary": "selector to make left_hand unload outward_claw", - "name": "selector: unload_tool(left_hand, outward_claw)", - "children": [ - { - "summary": "condition node to check if left_hand is free, - "name": "target: is_free(left_hand)" - }, - { - "summary": "sequence to make left_hand unload outward_claw", - "name": "sequence: unload_tool(left_hand, outward_claw)", - "children": [ - { - "summary": "condition node to check if outward_claw is free", - "name": "precondition: is_free(outward_claw)" - }, - { - "summary": "condition node to check if left hand holds outward_claw", - "name": "precondition: hold(left_hand, outward_claw)" - }, - { - "summary": "action node to make left hand unload outward_claw", - "name": "action: unload_tool(left_hand, outward_claw)", - }, - ], - }, - ], - }, - { - "summary": "action node to equip parallel_box1 to left hand", - "name": "action: load_tool(left_hand, parallel_box1)", - }, - ], - }, - ], - }, - { - "summary": "selector to use left_hand with parallel_box1 to pick up gear1", - "name": "selector: pick_up(left_hand, parallel_box1, gear1)", - "children": [ - { - "summary": "condition node to check if parallel_box1 holds gear1", - "name": "target: hold(parallel_box1, gear1)" - }, - { - "summary": "sequence to use left hand with parallel_box1 to pick up gear1", - "name": "sequence: pick_up(left_hand, parallel_box1, gear1)", - "children": [ - { - "summary": "condition node to check if parallel_box1 is free", - "name": "precondition: is_free(parallel_box1)" - }, - { - "summary": "condition node to check if left hand holds parallel_box1", - "name": "precondition: hold(left_hand, parallel_box1)" - }, - { - "summary": "action node to use left hand with parallel_box1 to pick up gear1", - "name": "action: pick_up(left_hand, parallel_box1, gear1)", - }, - ], - }, - ], - }, - { - "summary": "condition node to check if gear1 can be inserted to shaft1", - "name": "precondition: can_insert_to(gear1, shaft1)", - }, - { - "summary": "action node to use left_hand with parallel_box1 to insert gear1 to shaft1", - "name": "action: insert(left_hand, parallel_box1, gear1, shaft1)", - }, - ], - }, - ], -} -}, -"question":""} -``` -""" - -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/data/prompts/skeleton_generator/prompt/cot_sk_instruction.txt b/data/prompts/skeleton_generator/prompt/cot_sk_instruction.txt deleted file mode 100644 index 29546e4b..00000000 --- a/data/prompts/skeleton_generator/prompt/cot_sk_instruction.txt +++ /dev/null @@ -1,7 +0,0 @@ -[user] -User's instruction is an instruction string. -You interpret the user instruction into goal state based on the domain knowledge. -Then you make the plan to achieve it. - -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/data/prompts/skeleton_generator/prompt/cot_sk_objects.txt b/data/prompts/skeleton_generator/prompt/cot_sk_objects.txt deleted file mode 100644 index f0249582..00000000 --- a/data/prompts/skeleton_generator/prompt/cot_sk_objects.txt +++ /dev/null @@ -1,15 +0,0 @@ -[user] -Information about the objects is given as a dictionary. Its keys are the types and the values are lists of instances of that type. -An example: -""" -{ - "fruit": ["apple", "banana", "cherry"], - "part": ["gear", "nut", "bolt"], - "tool": ["tool_box", "gripper", "screwdriver"] -} -""" -The types are defined and used in domain knowledge. -Refer to the domain knowledge to understand which predicates and actions are applicable to which types. - -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/data/prompts/skeleton_generator/prompt/cot_sk_output_format.txt b/data/prompts/skeleton_generator/prompt/cot_sk_output_format.txt deleted file mode 100644 index 3a3838c7..00000000 --- a/data/prompts/skeleton_generator/prompt/cot_sk_output_format.txt +++ /dev/null @@ -1,16 +0,0 @@ -[user] -You will be provided with a problem and the necessary domain knowledge. -You generate an action sequence to solve the task and contruct a behavior tree based on it in json format. -The output dictionary has two keys. -""" -- dictionary["task_plan"]: A dictionary containing the task plan. -- dictionary["question"]: If you don't have enough knowledge for solving the problem, you can ask questions about the knowledge you need. Leave this key empty if you can solve the task. -""" -Three keys exist in dictionary["task_plan"]. -""" -- dictionary["task_plan"]["action_sequence"]: Contains a list of robot actions. Only the actions defined in the "ROBOT_ACTION LIST" will be used. -- dictionary["task_plan"]["manipulated_objects"]: The list of the manipulated objects. It only includes the objects that are related to the current task. -- dictionary["task_plan"]["behavior_tree"]: contains a dictionary of the behavior tree constructed according to the dictionary ["task_plan"]["action_sequence"] -""" -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/data/prompts/skeleton_generator/prompt/cot_sk_problem.txt b/data/prompts/skeleton_generator/prompt/cot_sk_problem.txt deleted file mode 100644 index c66ea775..00000000 --- a/data/prompts/skeleton_generator/prompt/cot_sk_problem.txt +++ /dev/null @@ -1,24 +0,0 @@ -[user] - -/// discarded now -Information about the problem is given as a pddl problem. -It includes the following information: -- objects: all the objects in the world. They should be of a type defined in the domain knowledge. For example, "parallel_box1" is an instance of the type "tool", while "gear1" is an instance of the type "part". -- init: the constraint relations in the world, described with [PREDICATE_LIST] from domain knowledge. They cannot be changed by robot actions. -- goal: IGNORE this and follow user's instructions instead. -An example: -""" -(define (problem robot_assembly_problem-problem) - (:domain robot_assembly_problem-domain) - (:objects - parallel_box1 parallel_box2 inward_claw outward_claw no_tool - tool - gear1 gear2 gear3 shaft1 shaft2 base - part - left_hand - hand - ) - (:init (can_manipulate parallel_box1 gear1) (can_manipulate outward_claw gear2) (can_manipulate inward_claw gear3) (can_manipulate parallel_box2 shaft1) (can_manipulate no_tool shaft2) (can_screw_to leg1 seat) (can_screw_to leg2 seat) (can_insert_to back seat) (can_screw_to nut1 seat) (can_screw_to nut2 seat) (can_screw_to blub base) (can_place_to lamp blub) (can_insert_to shaft1 base) (can_insert_to shaft2 base) (can_insert_to gear3 shaft2) (can_insert_to gear2 base) (can_insert_to gear1 shaft1)S) - (:goal (and )) -) -""" - -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/data/prompts/skeleton_generator/prompt/cot_sk_role.txt b/data/prompts/skeleton_generator/prompt/cot_sk_role.txt deleted file mode 100644 index 19f9b3f0..00000000 --- a/data/prompts/skeleton_generator/prompt/cot_sk_role.txt +++ /dev/null @@ -1,4 +0,0 @@ -[user] -You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you break it down and construct robot task plans represented by behaiovr trees. -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/data/prompts/skeleton_generator/prompt/cot_sk_state.txt b/data/prompts/skeleton_generator/prompt/cot_sk_state.txt deleted file mode 100644 index 66541898..00000000 --- a/data/prompts/skeleton_generator/prompt/cot_sk_state.txt +++ /dev/null @@ -1,25 +0,0 @@ -[user] -Information about world states should be in json dictionary format. Example: -""" -{ - "objects": [ - {"name": "leg1", "properties": []}, # leg1 is an object with no properties - {"name": "seat", "properties": []}, # seat is an object with no properties - {"name": "hand", "properties": ["is_free"]}, # hand is an object with a property "is_free" - ], - "constraints": [ - {"source": "leg1", "name": "can_screw_to", "target": "seat"}, # means leg1 can be screwed to seat - ], - "relations": [ - {"source": "leg1", "name": "is_screwed_to", "target": "seat"}, # means leg1 is screwed to seat - ], -} -""" -It contains three lists: - - "objects": the objects and their properties (as lists of strings) in the world. The properties can be changed by robot actions. Refer to the input world state for them. - - "constraints": the constraint relations that are enforced and should not be changed. Refer to the pddl problem for the constraints. - - "relations": the relations in the world. They can be changed by robot actions. Refer to input world state for them. -The properties, constraints and relations should always align with the [PREDICATE_LIST] in domain knowledge. - -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/data/prompts/skeleton_generator/prompt_v1/cot_sk_bt copy.txt b/data/prompts/skeleton_generator/prompt_v1/cot_sk_bt copy.txt deleted file mode 100644 index 7219d47f..00000000 --- a/data/prompts/skeleton_generator/prompt_v1/cot_sk_bt copy.txt +++ /dev/null @@ -1,83 +0,0 @@ -[user] -The behavior tree should be in json dictionary format and have a structure like the following example: -""" -{ - "summary": "selector to use left hand to load inward_claw", - "name": "selector: load_tool(left_hand, inward_claw)", - "children": [ - { - "summary": "condition node to check if left_hand holds inward_claw", - "name": "target: hold(left_hand, inward_claw)", - }, - { - "summary": "sequence to use left hand to load inward_claw", - "name": "sequence: load_tool(left_hand, inward_claw)", - "children": [ - { - "summary": "condition node to check if inward_claw is equippable", - "name": "precondition: is_equippable(inward_claw)", - }, - { - "summary": "condition node to check if left hand is free", - "name": "precondition: is_free(left_hand)", - }, - { - "summary": "action node to use left hand to load inward_claw", - "name": "action: load_tool(left_hand, inward_claw)", - "effects": [ - {"summary": "left hand will be not free anymore",}, - {"summary": "inward_claw will be not equippable anymore",}, - {"summary": "left_hand will hold inward_claw",}, - ], - }, - ], - }, - ], -} -""" -Follow these rules: -1. Every node are described by a summary and a name: - - The "summary" key includes a short interpretive description of the node. - - The "name" key includes the type of the node and the formal form of the node according to the domain knowledge. -2. The control flow nodes, "selector" and "sequence", can have "children", which is a list of children nodes. -3. The "condition" nodes that are children of "selectors" are targets. Those that are children of "sequences" are preconditions. -4. The "action" nodes should have a key "effects" with a list of dictionaries, which describes the to-be-exerted effects of the action. The effects are shortly described under the key "summary". -5. All the subtrees always have their corresponding action in ROBOT_ACTION_LIST. The recommended structure is: - - A selector as the root node. - - (Usually one) target condition node as the first child of the selector to check if the target of the action has been fulfilled. - - A sequence node as the last child of the selector node to achieve the target. - - (Usually a number of) precondition nodes as the front children of the sequence node to check the preconditions for executing the action. - - One action node as the last child of the sequence node, whose preconditions are checked before and will immediate start the job when being ticked. - - The action will exert its defined effects after finishing the job successfully. Those effects should fulfilled all the target conditions in the selector node. - A brief example is as follows: -{ - selector: load_tool selector, - children: [ - { - target: check if inward_claw is equipped by left hand, - }, - { - sequence: load_tool, - children: [ - {precondition: check if inward_claw is equippable}, - {precondition: check if left hand is free}, - { - action: equip inward_claw to left hand, - effects: [ - left hand not free, - inward_claw not equippable, - left_hand holds inward_claw, - ] - }, - ], - }, - ], -} - -6. When performing a new action to fulfill a unsatisfied precondition, you can construct a subtree to replace the precondition and take that unfulfilled precondition as the target of the subtree. Then you construct the sequence of the subtree according to the new action you want to perform. -7. All of the control flow nodes, namely selectors and sequences, are without memory. A new behavior tree tick will always start from the root and traverse all the nodes even if they are ticked in the last tick. - -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/data/prompts/skeleton_generator/prompt_v1/cot_sk_bt.txt b/data/prompts/skeleton_generator/prompt_v1/cot_sk_bt.txt deleted file mode 100644 index cafa7482..00000000 --- a/data/prompts/skeleton_generator/prompt_v1/cot_sk_bt.txt +++ /dev/null @@ -1,96 +0,0 @@ -[user] -The behavior tree should be in json dictionary format and have a structure like the following example: -""" -{ - "summary": "selector to use left hand to load inward_claw", - "name": "selector: load_tool(left_hand, inward_claw)", - "children": [ - { - "summary": "condition node to check if left_hand holds inward_claw", - "name": "target: hold(left_hand, inward_claw)", - }, - { - "summary": "sequence to use left hand to load inward_claw", - "name": "sequence: load_tool(left_hand, inward_claw)", - "children": [ - { - "summary": "condition node to check if inward_claw is equippable", - "name": "precondition: is_equippable(inward_claw)", - }, - { - "summary": "condition node to check if left hand is free", - "name": "precondition: is_free(left_hand)", - }, - { - "summary": "action node to use left hand to load inward_claw", - "name": "action: load_tool(left_hand, inward_claw)", - "effects": [ - {"summary": "left hand will be not free anymore",}, - {"summary": "inward_claw will be not equippable anymore",}, - {"summary": "left_hand will hold inward_claw",}, - ], - }, - ], - }, - ], -} -""" -Follow these rules: -1. Every node are described by a summary and a name: - - "summary": a short interpretive description of the node. - - "name": includes the node type and the formal form of the node according to the domain knowledge. -2. The control flow nodes, "selector" and "sequence", can have "children", which is a list of children nodes. -3. The "condition" nodes that are children of "selectors" are targets. Those that are children of "sequences" are preconditions. -4. The "action" nodes should have a key "effects" with a list of dictionaries, which describes the to-be-exerted effects of the action. The effects are shortly described under the key "summary". -5. All the subtrees always have their corresponding action in ROBOT_ACTION_LIST. The recommended structure is: - - A selector as the root node. - - (Usually one) target condition node as the first child of the selector to check if the target of the action has been fulfilled. - - A sequence node as the last child of the selector node to achieve the target. - - (Usually a number of) precondition nodes as the front children of the sequence node to check the preconditions for executing the action. - - One action node as the last child of the sequence node, whose preconditions are checked before and will immediate start the job when being ticked. - - The action will exert its defined effects after finishing the job successfully. Those effects should fulfilled all the target conditions in the selector node. - A brief example is as follows: -The action constructed based on the "load_tool" from ROBOT_ACTION_LIST: -load_tool(left_hand, inward_claw) -The bahavior tree constructed according to its definition in ROBOT_ACTION_LIST: -{ - "summary": "selector to load left_hand with parallel_box1", - "name": "selector: load_tool(left_hand, parallel_box1)", - "children": [ - { - "summary": "condition node to check if left_hand holds parallel_box1", - "name": "target: hold(left_hand, parallel_box1)" - }, - { - "summary": "sequence to load left_hand with parallel_box1", - "name": "sequence: load_tool(left_hand, parallel_box1)", - "children": [ - { - "summary": "condition node to check if left_hand is free, - "name": "precondition: is_free(left_hand)" - }, - { - "summary": "condition node to check if parallel_box1 is equippable", - "name": "precondition: is_equippable(parallel_box1)" - }, - { - "summary": "action node to equip left hand with parallel_box1", - "name": "action: load_tool(left_hand, parallel_box1)", - "effects": [ - {"summary": "left_hand will be not free",}, - {"summary": "parallel_box1 will be not equippable",}, - {"summary": "left_hand will hold parallel_box1",}, - ], - } - ], - }, - ], -} - -6. When performing a new action to fulfill a unsatisfied precondition, you can construct a subtree to replace the precondition and take that unfulfilled precondition as the target of the subtree. Then you construct the sequence of the subtree according to the new action you want to perform. -7. All of the control flow nodes, namely selectors and sequences, are without memory. A new behavior tree tick will always start from the root and traverse all the nodes even if they are ticked in the last tick. - -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/data/prompts/skeleton_generator/prompt_v1/cot_sk_chain copy.txt b/data/prompts/skeleton_generator/prompt_v1/cot_sk_chain copy.txt deleted file mode 100644 index 8ab56a97..00000000 --- a/data/prompts/skeleton_generator/prompt_v1/cot_sk_chain copy.txt +++ /dev/null @@ -1,15 +0,0 @@ -[user] -Follow this workflow to solve the task: - -1. Read the problem description. Parse and generate the "initial_state" in the output dictionary. -2. Interpret the goal of the problem as goal_state and write into the "instruction_summary" in the output dictionary. -3. Try to solve the problem, achieve the goal_state from the initial_state based on the domain knowledge provided. Generate a action sequence with the actions defined in and write it into dictionary["task_plan"]["action_sequence"] of the output. -4. If you cannot find a plan to solve the task, ask question with nature language in the "question" of the output dictionary. -5. Explain the actions in the action sequence and write the output dictionary["task_plan"]["action_instructions"]. -6. Collect the objects in the world that are effected by the action sequence, write into dictionary["task_plan"]["manipulated_objects"]. -7. Generate the behavior tree based on the action sequence and write into dictionary["task_plan"]["behavior_tree"]. - -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/data/prompts/skeleton_generator/prompt_v1/cot_sk_chain.txt b/data/prompts/skeleton_generator/prompt_v1/cot_sk_chain.txt deleted file mode 100644 index 8df3cbea..00000000 --- a/data/prompts/skeleton_generator/prompt_v1/cot_sk_chain.txt +++ /dev/null @@ -1,17 +0,0 @@ -[user] -Follow this workflow to solve the task: - -1. Read the problem description. Parse it and use it to complete the "constraints" in "initial_state". -2. Fill the "objects" and "relations" in "initial_state" based on the input world state. -3. Interpret the input instructions as goal_state and write into the "instruction_summary" in the output dictionary. -4. Try to make a plan, achieve the goal_state from the initial_state based on the domain knowledge provided. Generate a action sequence with the actions defined in and write it into dictionary["task_plan"]["action_sequence"] of the output. -5. If you cannot find a plan to solve the task, ask question with nature language in the "question" of the output dictionary. -6. Explain the actions in the action sequence and write the output dictionary["task_plan"]["action_instructions"]. -7. Collect the objects that are effected by the action sequence, write into dictionary["task_plan"]["manipulated_objects"]. -8. Generate the behavior tree based on the action sequence. -9. check the behavior tree, run it through with the world state and expand the behavior tree if necessary. Write the expanded behavior tree into dictionary["task_plan"]["behavior_tree"]. - -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/data/prompts/skeleton_generator/prompt_v1/cot_sk_domain.txt b/data/prompts/skeleton_generator/prompt_v1/cot_sk_domain.txt deleted file mode 100644 index bf3c7859..00000000 --- a/data/prompts/skeleton_generator/prompt_v1/cot_sk_domain.txt +++ /dev/null @@ -1,89 +0,0 @@ -[user] -Following is the domain knowledge you need in pddl domain format, which includes: -- the object types in the world, -- the predicates for describing the states of the world, -- the actions that the robot can apply, with their precondtions and effects defined. -""" -(define (domain robot_assembly_problem-domain) - (:requirements :strips :typing) - (:types - thing - object - tool part hand - thing - ) - (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part)) - (:action pick_up - :parameters ( ?hand - hand ?tool - tool ?part - part) - :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part)) - :effect (and (hold ?tool ?part) (not (is_free ?tool)))) - (:action put_down - :parameters ( ?hand - hand ?tool - tool ?part - part) - :precondition (and (hold ?tool ?part) (hold ?hand ?tool)) - :effect (and (not (hold ?hand ?part)) (is_free ?tool))) - (:action place - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2))) - (:action detach - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2)))) - (:action insert - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2))) - (:action pull - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2)))) - (:action screw - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2))) - (:action unscrew - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2)))) - (:action load_tool - :parameters ( ?hand - hand ?tool - tool) - :precondition (and (is_equippable ?tool) (is_free ?hand)) - :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool))) - (:action unload_tool - :parameters ( ?hand - hand ?tool - tool) - :precondition (and (hold ?hand ?tool) (is_free ?tool)) - :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool)))) -) -""" -The precidates are explained as follows: -""" -"PREDICATE_LIST" -- is_free( or ): or is free and can hold something. (default: true) -- is_equippable(): is ready to be eqquipped by a robot hand. (default: true) -- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false) -- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false) -- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false) -- can_place_to(, ): can be placed into (placement assembly constraint) (default: false) -- hold(, ): is holding (default: false) -- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false) -- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false) -- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false) -""" - -The actions are explained as follows: -""" -"ROBOT_ACTION_LIST" -- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something. -- put_down(, , ): use to put down . As preconditions, should be holding and should be holding . -- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding . -- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into . -- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding . -- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into . -- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding . -- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into . -- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free. -- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free. -""" -the items in <> indicate the type of the instances needed for the predicates and actions. -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. diff --git a/data/prompts/skeleton_generator/prompt_v1/cot_sk_example copy 2.txt b/data/prompts/skeleton_generator/prompt_v1/cot_sk_example copy 2.txt deleted file mode 100644 index 8e311506..00000000 --- a/data/prompts/skeleton_generator/prompt_v1/cot_sk_example copy 2.txt +++ /dev/null @@ -1,156 +0,0 @@ -[user] -I will give you some examples of the input and the output you will generate. -Example 1: -""" -- Input: -problem: -(define (problem robot_assembly_problem-problem) - (:domain robot_assembly_problem-domain) - (:objects - parallel_box1 parallel_box2 inward_claw outward_claw no_tool - tool - gear1 gear2 gear3 shaft1 shaft2 base - part - left_hand - hand - ) - (:init (can_manipulate parallel_box1 gear1) (can_manipulate outward_claw gear2) (can_manipulate inward_claw gear3) (can_manipulate parallel_box2 shaft1) (can_manipulate no_tool shaft2) (can_screw_to leg1 seat) (can_screw_to leg2 seat) (can_insert_to back seat) (can_screw_to nut1 seat) (can_screw_to nut2 seat) (can_screw_to blub base) (can_place_to lamp blub) (can_insert_to shaft1 base) (can_insert_to shaft2 base) (can_insert_to gear3 shaft2) (can_insert_to gear2 base) (can_insert_to gear1 shaft1) (is_inserted_to shaft1 base) (is_free left_hand) (is_free parallel_box1) (is_free parallel_box2) (is_free inward_claw) (is_free outward_claw) (is_free no_tool) (is_equippable parallel_box1) (is_equippable parallel_box2) (is_equippable inward_claw) (is_equippable outward_claw) (is_equippable no_tool)) -) -world_state: -{"objects": [ - {"name": "gear1", "properties": []}, - {"name": "shaft1", "properties": []}, - {"name": "parallel_box1", "properties": ["is_equippable", "is_free"]}, - {"name": "left_hand", "properties": ["is_free"]}, - ], - "constraints": [], - "relations": [], -} -instructions: insert gear1 into shaft1 - -- Output: -``` -{"task_plan": { - "action_sequence": [ - "load_tool(left_hand, parallel_box1)", - "pick_up(left_hand, parallel_box1, gear1)", - "insert(left_hand, parallel_box1, gear1, shaft1)" - ], - "action_instructions": [ - "equip the parallel_box1 on the left hand", - "pick up the gear1 with the parallel_box1 in the left hand", - "insert the gear1 into the shaft1 using the parallel_box1 in the left hand" - ], - "object_name": ["left_hand", "gear1", "parallel_box1", "shaft1"], - "behavior_tree": { - "summary": "selector to insert gear1 into shaft1", - "name": "selector: insert(left_hand, parallel_box1, gear1, shaft1)", - "children": [ - { - "summary": "condition node to check if gear1 is inserted to shaft1", - "name": "target: is_inserted_to(gear1, shaft1)", - }, - { - "summary": "sequence to insert gear1 into shaft1", - "name": "sequence: insert(left_hand, parallel_box1, gear1, shaft1)", - "children": [ - { - "summary": "selector to load parallel_box1", - "name": "selector: load_tool(left_hand, parallel_box1)", - "children": [ - { - "summary": "condition node to check if left_hand holds parallel_box1", - "name": "target: hold(left_hand, parallel_box1)" - }, - { - "summary": "sequence to load parallel_box1", - "name": "sequence: load_tool(left_hand, parallel_box1)", - "children": [ - { - "summary": "condition node to check if parallel_box1 is equippable", - "name": "precondition: is_equippable(parallel_box1)" - }, - { - "summary": "condition node to check if left hand is free", - "name": "precondition: is_free(left_hand)" - }, - { - "summary": "action node to equip parallel_box1 to left hand", - "name": "action: load_tool(left_hand, parallel_box1)", - "effects": [ - {"summary": "left_hand will be not free",}, - {"summary": "parallel_box1 will be not equippable",}, - {"summary": "left_hand will hold parallel_box1",}, - ], - }, - ], - }, - ], - }, - { - "summary": "selector to use left_hand with parallel_box1 to pick up gear1", - "name": "selector: pick_up(left_hand, parallel_box1, gear1)", - "children": [ - { - "summary": "condition node to check if parallel_box1 holds gear1", - "name": "target: hold(parallel_box1, gear1)" - }, - { - "summary": "sequence to use left hand with parallel_box1 to pick up gear1", - "name": "sequence: pick_up(left_hand, parallel_box1, gear1)", - "children": [ - { - "summary": "condition node to check if parallel_box1 is free", - "name": "precondition: is_free(parallel_box1)" - }, - { - "summary": "condition node to check if left hand holds parallel_box1", - "name": "precondition: hold(left_hand, parallel_box1)" - }, - { - "summary": "action node to use left hand with parallel_box1 to pick up gear1", - "name": "action: pick_up(left_hand, parallel_box1, gear1)", - "effects": [ - {"summary": "parallel_box1 will hold gear1",}, - {"summary": "parallel_box1 will be not free",}, - ], - }, - ], - }, - ], - }, - { - "summary": "condition node to check if gear1 can be inserted to shaft1", - "name": "precondition: can_insert_to(gear1, shaft1)", - }, - { - "summary": "action node to use left_hand with parallel_box1 to insert gear1 to shaft1", - "name": "action: insert(left_hand, parallel_box1, gear1, shaft1)", - "effects": [ - {"summary": "parallel_box1 will be not holding gear1",}, - {"summary": "parallel_box1 will be free",}, - {"summary": "gear1 will be inserted to shaft1",}, - ], - }, - ], - }, - ], -} -}, -"initial_state": {"objects": [ - {"name": "gear1", "properties": []}, - {"name": "shaft1", "properties": []}, - {"name": "parallel_box1", "properties": ["is_equippable", "is_free"]}, - {"name": "left_hand", "properties": ["is_free"]}, - ], - "constraints": [ - {"source": "gear1", "name": "can_insert_to", "target": "shaft1"}, - {"source": "parallel_box1", "name": "can_manipulate", "target": "gear1"}, - ], - "relations": [], -}, -"instruction_summary": "insert gear1 into shaft1", -"question":""} -``` -""" -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/data/prompts/skeleton_generator/prompt_v1/cot_sk_example copy.txt b/data/prompts/skeleton_generator/prompt_v1/cot_sk_example copy.txt deleted file mode 100644 index 02853177..00000000 --- a/data/prompts/skeleton_generator/prompt_v1/cot_sk_example copy.txt +++ /dev/null @@ -1,144 +0,0 @@ -[user] -I will give you some examples of the input and the output you will generate. -Example 1: -""" -- Input: -(define (problem robot_assembly_problem-problem) - (:domain robot_assembly_problem-domain) - (:objects - parallel_box1 parallel_box2 inward_claw outward_claw no_tool - tool - gear1 gear2 gear3 shaft1 shaft2 base - part - left_hand - hand - ) - (:init (can_manipulate parallel_box1 gear1) (can_manipulate outward_claw gear2) (can_manipulate inward_claw gear3) (can_manipulate parallel_box2 shaft1) (can_manipulate no_tool shaft2) (can_screw_to leg1 seat) (can_screw_to leg2 seat) (can_insert_to back seat) (can_screw_to nut1 seat) (can_screw_to nut2 seat) (can_screw_to blub base) (can_place_to lamp blub) (can_insert_to shaft1 base) (can_insert_to shaft2 base) (can_insert_to gear3 shaft2) (can_insert_to gear2 base) (can_insert_to gear1 shaft1) (is_inserted_to shaft1 base) (is_free left_hand) (is_free parallel_box1) (is_free parallel_box2) (is_free inward_claw) (is_free outward_claw) (is_free no_tool) (is_equippable parallel_box1) (is_equippable parallel_box2) (is_equippable inward_claw) (is_equippable outward_claw) (is_equippable no_tool)) - (:goal (and (is_inserted_to gear1 shaft1))) -) -- Output: -``` -{"task_plan": { - "action_sequence": [ - "load_tool(left_hand, parallel_box1)", - "pick_up(left_hand, parallel_box1, gear1)", - "insert(left_hand, parallel_box1, gear1, shaft1)" - ], - "action_instructions": [ - "equip the parallel_box1 on the left hand", - "pick up the gear1 with the parallel_box1 in the left hand", - "insert the gear1 into the shaft1 using the parallel_box1 in the left hand" - ], - "object_name": ["left_hand", "gear1", "parallel_box1", "shaft1"], - "behavior_tree": { - "summary": "selector to insert gear1 into shaft1", - "name": "selector: insert(left_hand, parallel_box1, gear1, shaft1)", - "children": [ - { - "summary": "condition node to check if gear1 is inserted to shaft1", - "name": "target: is_inserted_to(gear1, shaft1)", - }, - { - "summary": "sequence to insert gear1 into shaft1", - "name": "sequence: insert(left_hand, parallel_box1, gear1, shaft1)", - "children": [ - { - "summary": "selector to load parallel_box1", - "name": "selector: load_tool(left_hand, parallel_box1)", - "children": [ - { - "summary": "condition node to check if left_hand holds parallel_box1", - "name": "target: hold(left_hand, parallel_box1)" - }, - { - "summary": "sequence to load parallel_box1", - "name": "sequence: load_tool(left_hand, parallel_box1)", - "children": [ - { - "summary": "condition node to check if parallel_box1 is equippable", - "name": "precondition: is_equippable(parallel_box1)" - }, - { - "summary": "condition node to check if left hand is free", - "name": "precondition: is_free(left_hand)" - }, - { - "summary": "action node to equip parallel_box1 to left hand", - "name": "action: load_tool(left_hand, parallel_box1)", - "effects": [ - {"summary": "left_hand will be not free",}, - {"summary": "parallel_box1 will be not equippable",}, - {"summary": "left_hand will hold parallel_box1",}, - ], - }, - ], - }, - ], - }, - { - "summary": "selector to use left_hand with parallel_box1 to pick up gear1", - "name": "selector: pick_up(left_hand, parallel_box1, gear1)", - "children": [ - { - "summary": "condition node to check if parallel_box1 holds gear1", - "name": "target: hold(parallel_box1, gear1)" - }, - { - "summary": "sequence to use left hand with parallel_box1 to pick up gear1", - "name": "sequence: pick_up(left_hand, parallel_box1, gear1)", - "children": [ - { - "summary": "condition node to check if parallel_box1 is free", - "name": "precondition: is_free(parallel_box1)" - }, - { - "summary": "condition node to check if left hand holds parallel_box1", - "name": "precondition: hold(left_hand, parallel_box1)" - }, - { - "summary": "action node to use left hand with parallel_box1 to pick up gear1", - "name": "action: pick_up(left_hand, parallel_box1, gear1)", - "effects": [ - {"summary": "parallel_box1 will hold gear1",}, - {"summary": "parallel_box1 will be not free",}, - ], - }, - ], - }, - ], - }, - { - "summary": "condition node to check if gear1 can be inserted to shaft1", - "name": "precondition: can_insert_to(gear1, shaft1)", - }, - { - "summary": "action node to use left_hand with parallel_box1 to insert gear1 to shaft1", - "name": "action: insert(left_hand, parallel_box1, gear1, shaft1)", - "effects": [ - {"summary": "parallel_box1 will be not holding gear1",}, - {"summary": "parallel_box1 will be free",}, - {"summary": "gear1 will be inserted to shaft1",}, - ], - }, - ], - }, - ], -} -}, -"initial_state": {"objects": [ - {"name": "gear1", "properties": []}, - {"name": "shaft1", "properties": []}, - {"name": "parallel_box1", "properties": ["is_equippable", "is_free"]}, - {"name": "left_hand", "properties": ["is_free"]}, - ], - "constraints": [ - {"source": "gear1", "name": "can_insert_to", "target": "shaft1"}, - {"source": "parallel_box1", "name": "can_manipulate", "target": "gear1"}, - ], - "relations": [], -}, -"instruction_summary": "insert gear1 into shaft1", -"question":""} -``` -""" -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/data/prompts/skeleton_generator/prompt_v1/cot_sk_example.txt b/data/prompts/skeleton_generator/prompt_v1/cot_sk_example.txt deleted file mode 100644 index 495e2168..00000000 --- a/data/prompts/skeleton_generator/prompt_v1/cot_sk_example.txt +++ /dev/null @@ -1,421 +0,0 @@ -[user] -I will give you some examples of the input and the output you will generate. -Example 1: -""" -- Input: -problem: -(define (problem robot_assembly_problem-problem) - (:domain robot_assembly_problem-domain) - (:objects - parallel_box1 parallel_box2 inward_claw outward_claw no_tool - tool - gear1 gear2 gear3 shaft1 shaft2 base - part - left_hand - hand - ) - (:init (can_manipulate parallel_box1 gear1) (can_manipulate outward_claw gear2) (can_manipulate inward_claw gear3) (can_manipulate parallel_box2 shaft1) (can_manipulate no_tool shaft2) (can_screw_to leg1 seat) (can_screw_to leg2 seat) (can_insert_to back seat) (can_screw_to nut1 seat) (can_screw_to nut2 seat) (can_screw_to blub base) (can_place_to lamp blub) (can_insert_to shaft1 base) (can_insert_to shaft2 base) (can_insert_to gear3 shaft2) (can_insert_to gear2 base) (can_insert_to gear1 shaft1) (is_inserted_to shaft1 base) (is_free left_hand) (is_free parallel_box1) (is_free parallel_box2) (is_free inward_claw) (is_free outward_claw) (is_free no_tool) (is_equippable parallel_box1) (is_equippable parallel_box2) (is_equippable inward_claw) (is_equippable outward_claw) (is_equippable no_tool)) -) -world_state: -{"objects": [ - {"name": "gear1", "properties": []}, - {"name": "shaft1", "properties": []}, - {"name": "parallel_box1", "properties": ["is_equippable", "is_free"]}, - {"name": "left_hand", "properties": ["is_free"]}, - ], - "constraints": [], - "relations": [], -} -instructions: insert gear1 into shaft1 - -- Output: -``` -{"task_plan": { - "action_sequence": [ - "load_tool(left_hand, parallel_box1)", - "pick_up(left_hand, parallel_box1, gear1)", - "insert(left_hand, parallel_box1, gear1, shaft1)" - ], - "action_instructions": [ - "equip the parallel_box1 on the left hand", - "pick up the gear1 with the parallel_box1 in the left hand", - "insert the gear1 into the shaft1 using the parallel_box1 in the left hand" - ], - "object_name": ["left_hand", "gear1", "parallel_box1", "shaft1"], - "behavior_tree": { - "summary": "selector to insert gear1 into shaft1", - "name": "selector: insert(left_hand, parallel_box1, gear1, shaft1)", - "children": [ - { - "summary": "condition node to check if gear1 is inserted to shaft1", - "name": "target: is_inserted_to(gear1, shaft1)", - }, - { - "summary": "sequence to insert gear1 into shaft1", - "name": "sequence: insert(left_hand, parallel_box1, gear1, shaft1)", - "children": [ - { - "summary": "selector to load parallel_box1", - "name": "selector: load_tool(left_hand, parallel_box1)", - "children": [ - { - "summary": "condition node to check if left_hand holds parallel_box1", - "name": "target: hold(left_hand, parallel_box1)" - }, - { - "summary": "sequence to load parallel_box1", - "name": "sequence: load_tool(left_hand, parallel_box1)", - "children": [ - { - "summary": "condition node to check if parallel_box1 is equippable", - "name": "precondition: is_equippable(parallel_box1)" - }, - { - "summary": "condition node to check if left hand is free", - "name": "precondition: is_free(left_hand)" - }, - { - "summary": "action node to equip parallel_box1 to left hand", - "name": "action: load_tool(left_hand, parallel_box1)", - "effects": [ - {"summary": "left_hand will be not free",}, - {"summary": "parallel_box1 will be not equippable",}, - {"summary": "left_hand will hold parallel_box1",}, - ], - }, - ], - }, - ], - }, - { - "summary": "selector to use left_hand with parallel_box1 to pick up gear1", - "name": "selector: pick_up(left_hand, parallel_box1, gear1)", - "children": [ - { - "summary": "condition node to check if parallel_box1 holds gear1", - "name": "target: hold(parallel_box1, gear1)" - }, - { - "summary": "sequence to use left hand with parallel_box1 to pick up gear1", - "name": "sequence: pick_up(left_hand, parallel_box1, gear1)", - "children": [ - { - "summary": "condition node to check if parallel_box1 is free", - "name": "precondition: is_free(parallel_box1)" - }, - { - "summary": "condition node to check if left hand holds parallel_box1", - "name": "precondition: hold(left_hand, parallel_box1)" - }, - { - "summary": "action node to use left hand with parallel_box1 to pick up gear1", - "name": "action: pick_up(left_hand, parallel_box1, gear1)", - "effects": [ - {"summary": "parallel_box1 will hold gear1",}, - {"summary": "parallel_box1 will be not free",}, - ], - }, - ], - }, - ], - }, - { - "summary": "condition node to check if gear1 can be inserted to shaft1", - "name": "precondition: can_insert_to(gear1, shaft1)", - }, - { - "summary": "action node to use left_hand with parallel_box1 to insert gear1 to shaft1", - "name": "action: insert(left_hand, parallel_box1, gear1, shaft1)", - "effects": [ - {"summary": "parallel_box1 will be not holding gear1",}, - {"summary": "parallel_box1 will be free",}, - {"summary": "gear1 will be inserted to shaft1",}, - ], - }, - ], - }, - ], -} -}, -"initial_state": {"objects": [ - {"name": "gear1", "properties": []}, - {"name": "shaft1", "properties": []}, - {"name": "parallel_box1", "properties": ["is_equippable", "is_free"]}, - {"name": "left_hand", "properties": ["is_free"]}, - ], - "constraints": [ - {"source": "gear1", "name": "can_insert_to", "target": "shaft1"}, - {"source": "parallel_box1", "name": "can_manipulate", "target": "gear1"}, - ], - "relations": [], -}, -"instruction_summary": "insert gear1 into shaft1", -"question":""} -``` -""" - -Example 2: -""" -- Input: -problem: -(define (problem robot_assembly_problem-problem) - (:domain robot_assembly_problem-domain) - (:objects - parallel_box1 parallel_box2 inward_claw outward_claw no_tool - tool - gear1 gear2 gear3 shaft1 shaft2 base - part - left_hand - hand - ) - (:init (can_manipulate parallel_box1 gear1) (can_manipulate outward_claw gear2) (can_manipulate inward_claw gear3) (can_manipulate parallel_box2 shaft1) (can_manipulate no_tool shaft2) (can_screw_to leg1 seat) (can_screw_to leg2 seat) (can_insert_to back seat) (can_screw_to nut1 seat) (can_screw_to nut2 seat) (can_screw_to blub base) (can_place_to lamp blub) (can_insert_to shaft1 base) (can_insert_to shaft2 base) (can_insert_to gear3 shaft2) (can_insert_to gear2 base) (can_insert_to gear1 shaft1) (is_inserted_to shaft1 base) (is_free left_hand) (is_free parallel_box1) (is_free parallel_box2) (is_free inward_claw) (is_free outward_claw) (is_free no_tool) (is_equippable parallel_box1) (is_equippable parallel_box2) (is_equippable inward_claw) (is_equippable outward_claw) (is_equippable no_tool)) -) -world_state: -{"objects": [ - {"name": "gear1", "properties": []}, - {"name": "shaft1", "properties": []}, - {"name": "outward_claw", "properties": ["is_free"]}, - {"name": "parallel_box1", "properties": ["is_equippable", "is_free"]}, - {"name": "left_hand", "properties": []}, - ], - "constraints": [], - "relations": [ - { - "source": "left_hand", - "name": "hold", - "target": "outward_claw" - } - ], -} -instructions: load left_hand with parallel_box1 - -- Output: -``` -{"task_plan": { - "action_sequence": [ - "unload_tool(left_hand, outward_claw)", - "load_tool(left_hand, parallel_box1)" - ], - "action_instructions": [ - "unload the outward_claw from the left hand", - "equip the left_hand with the parallel_box1" - ], - "object_name": ["left_hand", "outward_claw", "parallel_box1"], - "behavior_tree": { - "summary": "selector to load left_hand with parallel_box1", - "name": "selector: load_tool(left_hand, parallel_box1)", - "children": [ - { - "summary": "condition node to check if left_hand holds parallel_box1", - "name": "target: hold(left_hand, parallel_box1)" - }, - { - "summary": "sequence to load left_hand with parallel_box1", - "name": "sequence: load_tool(left_hand, parallel_box1)", - "children": [ - { - "summary": "selector to make left_hand unload outward_claw", - "name": "selector: unload_tool(left_hand, outward_claw)", - "children": [ - { - "summary": "condition node to check if left_hand is free, - "name": "target: is_free(left_hand)" - }, - { - "summary": "sequence to make left_hand unload outward_claw", - "name": "sequence: unload_tool(left_hand, outward_claw)", - "children": [ - { - "summary": "condition node to check if outward_claw is free", - "name": "precondition: is_free(outward_claw)" - }, - { - "summary": "condition node to check if left hand holds outward_claw", - "name": "precondition: hold(left_hand, outward_claw)" - }, - { - "summary": "action node to make left hand unload outward_claw", - "name": "action: unload_tool(left_hand, outward_claw)", - "effects": [ - {"summary": "left_hand will be free",}, - {"summary": "outward_claw will be equippable",}, - {"summary": "left_hand will not hold outward_claw",}, - ], - }, - ], - }, - ], - }, - { - "summary": "condition node to check if parallel_box1 is equippable", - "name": "precondition: is_equippable(parallel_box1)" - }, - { - "summary": "action node to equip left hand with parallel_box1", - "name": "action: load_tool(left_hand, parallel_box1)", - "effects": [ - {"summary": "left_hand will be not free",}, - {"summary": "parallel_box1 will be not equippable",}, - {"summary": "left_hand will hold parallel_box1",}, - ], - } - ], - }, - ], -} -}, -"initial_state": {"objects": [ - {"name": "gear1", "properties": []}, - {"name": "shaft1", "properties": []}, - {"name": "outward_claw", "properties": ["is_free"]}, - {"name": "parallel_box1", "properties": ["is_equippable", "is_free"]}, - {"name": "left_hand", "properties": []}, - ], - "constraints": [ - {"source": "gear1", "name": "can_insert_to", "target": "shaft1"}, - {"source": "parallel_box1", "name": "can_manipulate", "target": "gear1"}, - ], - "relations": [ - { - "source": "left_hand", - "name": "hold", - "target": "outward_claw" - } - ], -}, -"instruction_summary": "load left_hand with parallel_box1", -"question":""} -``` -""" - - -Example 2: -""" -- Input: -problem: -(define (problem robot_assembly_problem-problem) - (:domain robot_assembly_problem-domain) - (:objects - parallel_box1 parallel_box2 inward_claw outward_claw no_tool - tool - gear1 gear2 gear3 shaft1 shaft2 base - part - left_hand - hand - ) - (:init (can_manipulate parallel_box1 gear1) (can_manipulate outward_claw gear2) (can_manipulate inward_claw gear3) (can_manipulate parallel_box2 shaft1) (can_manipulate no_tool shaft2) (can_screw_to leg1 seat) (can_screw_to leg2 seat) (can_insert_to back seat) (can_screw_to nut1 seat) (can_screw_to nut2 seat) (can_screw_to blub base) (can_place_to lamp blub) (can_insert_to shaft1 base) (can_insert_to shaft2 base) (can_insert_to gear3 shaft2) (can_insert_to gear2 base) (can_insert_to gear1 shaft1) (is_inserted_to shaft1 base) (is_free left_hand) (is_free parallel_box1) (is_free parallel_box2) (is_free inward_claw) (is_free outward_claw) (is_free no_tool) (is_equippable parallel_box1) (is_equippable parallel_box2) (is_equippable inward_claw) (is_equippable outward_claw) (is_equippable no_tool)) -) -world_state: -{"objects": [ - {"name": "gear1", "properties": []}, - {"name": "shaft1", "properties": []}, - {"name": "outward_claw", "properties": ["is_free"]}, - {"name": "parallel_box1", "properties": ["is_equippable", "is_free"]}, - {"name": "left_hand", "properties": []}, - ], - "constraints": [], - "relations": [ - { - "source": "left_hand", - "name": "hold", - "target": "outward_claw" - } - ], -} -instructions: load left_hand with parallel_box1 - -- Output: -``` -{"task_plan": { - "action_sequence": [ - "unload_tool(left_hand, outward_claw)", - "load_tool(left_hand, parallel_box1)" - ], - "action_instructions": [ - "unload the outward_claw from the left hand", - "equip the left_hand with the parallel_box1" - ], - "object_name": ["left_hand", "outward_claw", "parallel_box1"], - "behavior_tree": { - "summary": "selector to load left_hand with parallel_box1", - "name": "selector: load_tool(left_hand, parallel_box1)", - "children": [ - { - "summary": "condition node to check if left_hand holds parallel_box1", - "name": "target: hold(left_hand, parallel_box1)" - }, - { - "summary": "sequence to load left_hand with parallel_box1", - "name": "sequence: load_tool(left_hand, parallel_box1)", - "children": [ - { - "summary": "selector to make left_hand unload outward_claw", - "name": "selector: unload_tool(left_hand, outward_claw)", - "children": [ - { - "summary": "condition node to check if left_hand is free, - "name": "target: is_free(left_hand)" - }, - { - "summary": "sequence to make left_hand unload outward_claw", - "name": "sequence: unload_tool(left_hand, outward_claw)", - "children": [ - { - "summary": "condition node to check if outward_claw is free", - "name": "precondition: is_free(outward_claw)" - }, - { - "summary": "condition node to check if left hand holds outward_claw", - "name": "precondition: hold(left_hand, outward_claw)" - }, - { - "summary": "action node to make left hand unload outward_claw", - "name": "action: unload_tool(left_hand, outward_claw)", - "effects": [ - {"summary": "left_hand will be free",}, - {"summary": "outward_claw will be equippable",}, - {"summary": "left_hand will not hold outward_claw",}, - ], - }, - ], - }, - ], - }, - { - "summary": "condition node to check if parallel_box1 is equippable", - "name": "precondition: is_equippable(parallel_box1)" - }, - { - "summary": "action node to equip left hand with parallel_box1", - "name": "action: load_tool(left_hand, parallel_box1)", - "effects": [ - {"summary": "left_hand will be not free",}, - {"summary": "parallel_box1 will be not equippable",}, - {"summary": "left_hand will hold parallel_box1",}, - ], - } - ], - }, - ], -} -}, -"initial_state": {"objects": [ - {"name": "gear1", "properties": []}, - {"name": "shaft1", "properties": []}, - {"name": "outward_claw", "properties": ["is_free"]}, - {"name": "parallel_box1", "properties": ["is_equippable", "is_free"]}, - {"name": "left_hand", "properties": []}, - ], - "constraints": [ - {"source": "gear1", "name": "can_insert_to", "target": "shaft1"}, - {"source": "parallel_box1", "name": "can_manipulate", "target": "gear1"}, - ], - "relations": [ - { - "source": "left_hand", - "name": "hold", - "target": "outward_claw" - } - ], -}, -"instruction_summary": "load left_hand with parallel_box1", -"question":""} -``` -""" - - -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/data/prompts/skeleton_generator/prompt_v1/cot_sk_instruction.txt b/data/prompts/skeleton_generator/prompt_v1/cot_sk_instruction.txt deleted file mode 100644 index a3d98c9b..00000000 --- a/data/prompts/skeleton_generator/prompt_v1/cot_sk_instruction.txt +++ /dev/null @@ -1,10 +0,0 @@ -[user] -User's instruction may have different forms. -it can be one instruction string or a list of instruction strings. -You interpret the user instruction into goal state based on the domain knowledge. -Then you make the plan to achieve it. - -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/data/prompts/skeleton_generator/prompt_v1/cot_sk_output_format.txt b/data/prompts/skeleton_generator/prompt_v1/cot_sk_output_format.txt deleted file mode 100644 index d47c6af5..00000000 --- a/data/prompts/skeleton_generator/prompt_v1/cot_sk_output_format.txt +++ /dev/null @@ -1,21 +0,0 @@ -[user] -You will be provided with a problem and the necessary domain knowledge. -You generate an action sequence to solve the task and contruct a behavior tree based on it in json format. -The output dictionary has five keys. -""" -- dictionary["task_plan"]: A dictionary containing the task plan. -- dictionary["initial_state"]: The initial world state before executing the plan. -- dictionary["instruction_summary"]: A brief summary of the task plan. -- dictionary["question"]: If you don't have enough knowledge for solving the problem, you can ask questions about the knowledge you need. Leave this key empty if you can solve the task. -""" -Four keys exist in dictionary["task_plan"]. -""" -- dictionary["task_plan"]["action_sequence"]: Contains a list of robot actions. Only the actions defined in the "ROBOT_ACTION LIST" will be used. -- dictionary["task_plan"]["action_instructions"]: contains a list of instructions corresponding to dictionary["task_plan"]["action_sequence"] to explain the actions. -- dictionary["task_plan"]["manipulated_objects"]: The list of the manipulated objects. It only includes the objects that are related to the current task. -- dictionary["task_plan"]["behavior_tree"]: contains a dictionary of the behavior tree constructed according to the dictionary ["task_plan"]["action_sequence"] -""" -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/data/prompts/skeleton_generator/prompt_v1/cot_sk_problem copy.txt b/data/prompts/skeleton_generator/prompt_v1/cot_sk_problem copy.txt deleted file mode 100644 index 6b1ddfb7..00000000 --- a/data/prompts/skeleton_generator/prompt_v1/cot_sk_problem copy.txt +++ /dev/null @@ -1,24 +0,0 @@ -[user] -Information about the problem is given as a pddl problem. -It includes the following information: -- objects: all the objects in the world. They should be of a type defined in the domain knowledge. For example, "parallel_box1" is an instance of the type "tool", while "gear1" is an instance of the type "part". -- init: the initial world state to start with, described with [PREDICATE_LIST] from domain knowledge. For example, "is_free hand" means the object "hand" is free and holds nothing. -- goal: the world state to achieve, described with [PREDICATE_LIST] from domain knowledge. -An example: -""" -(define (problem robot_assembly_problem-problem) - (:domain robot_assembly_problem-domain) - (:objects - parallel_box1 parallel_box2 inward_claw outward_claw no_tool - tool - gear1 gear2 gear3 shaft1 shaft2 base - part - left_hand - hand - ) - (:init (can_manipulate parallel_box1 gear1) (can_manipulate outward_claw gear2) (can_manipulate inward_claw gear3) (can_manipulate parallel_box2 shaft1) (can_manipulate no_tool shaft2) (can_screw_to leg1 seat) (can_screw_to leg2 seat) (can_insert_to back seat) (can_screw_to nut1 seat) (can_screw_to nut2 seat) (can_screw_to blub base) (can_place_to lamp blub) (can_insert_to shaft1 base) (can_insert_to shaft2 base) (can_insert_to gear3 shaft2) (can_insert_to gear2 base) (can_insert_to gear1 shaft1) (is_free left_hand) (is_free parallel_box1) (is_free parallel_box2) (is_free inward_claw) (is_free outward_claw) (is_free no_tool) (is_equippable parallel_box1) (is_equippable parallel_box2) (is_equippable inward_claw) (is_equippable outward_claw) (is_equippable no_tool)) - (:goal (and (is_inserted_to shaft1 base))) -) -""" - -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/data/prompts/skeleton_generator/prompt_v1/cot_sk_problem.txt b/data/prompts/skeleton_generator/prompt_v1/cot_sk_problem.txt deleted file mode 100644 index 916311f0..00000000 --- a/data/prompts/skeleton_generator/prompt_v1/cot_sk_problem.txt +++ /dev/null @@ -1,24 +0,0 @@ -[user] -Information about the problem is given as a pddl problem. -It includes the following information: -- objects: all the objects in the world. They should be of a type defined in the domain knowledge. For example, "parallel_box1" is an instance of the type "tool", while "gear1" is an instance of the type "part". -- init: the constraint relations in the world, described with [PREDICATE_LIST] from domain knowledge. They cannot be changed by robot actions. -- goal: IGNORE this and follow user's instructions instead. -An example: -""" -(define (problem robot_assembly_problem-problem) - (:domain robot_assembly_problem-domain) - (:objects - parallel_box1 parallel_box2 inward_claw outward_claw no_tool - tool - gear1 gear2 gear3 shaft1 shaft2 base - part - left_hand - hand - ) - (:init (can_manipulate parallel_box1 gear1) (can_manipulate outward_claw gear2) (can_manipulate inward_claw gear3) (can_manipulate parallel_box2 shaft1) (can_manipulate no_tool shaft2) (can_screw_to leg1 seat) (can_screw_to leg2 seat) (can_insert_to back seat) (can_screw_to nut1 seat) (can_screw_to nut2 seat) (can_screw_to blub base) (can_place_to lamp blub) (can_insert_to shaft1 base) (can_insert_to shaft2 base) (can_insert_to gear3 shaft2) (can_insert_to gear2 base) (can_insert_to gear1 shaft1)S) - (:goal (and )) -) -""" - -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/data/prompts/skeleton_generator/prompt_v1/cot_sk_role.txt b/data/prompts/skeleton_generator/prompt_v1/cot_sk_role.txt deleted file mode 100644 index 6ce736a0..00000000 --- a/data/prompts/skeleton_generator/prompt_v1/cot_sk_role.txt +++ /dev/null @@ -1,5 +0,0 @@ -[user] -You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you break it down and construct robot task plans represented by behaiovr trees. -Please do not begin working until I say "Start working." Instead, simply output the message "Waiting for next input." Understood? -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/data/prompts/skeleton_generator/prompt_v1/cot_sk_state copy.txt b/data/prompts/skeleton_generator/prompt_v1/cot_sk_state copy.txt deleted file mode 100644 index 00d748ea..00000000 --- a/data/prompts/skeleton_generator/prompt_v1/cot_sk_state copy.txt +++ /dev/null @@ -1,28 +0,0 @@ -[user] -Information about world states should be in json dictionary format. Example: -""" -{ - "objects": [ - {"name": "leg1", "properties": []}, # leg1 is an object with no properties - {"name": "seat", "properties": []}, # seat is an object with no properties - {"name": "hand", "properties": ["is_free"]}, # hand is an object with a property "is_free" - ], - "constraints": [ - {"source": "leg1", "name": "can_screw_to", "target": "seat"}, # means leg1 can be screwed to seat - ], - "relations": [ - {"source": "leg1", "name": "is_screwed_to", "target": "seat"}, # means leg1 is screwed to seat - ], -} -""" -It contains three lists: - - "objects": the objects and their properties (as lists of strings) in the world. The properties can be changed by robot actions. - - "constraints": the constraint relations that are enforced and should not be changed. They are knowledge about the world or some constraints set by the user. - - "relations": the relations in the world. They can be changed by robot actions. -The properties, constraints and relations should always align with the [PREDICATE_LIST] in domain knowledge. -Only concentrate on the objects, constraints and relations that are related to the current task. - -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/data/prompts/skeleton_generator/prompt_v1/cot_sk_state.txt b/data/prompts/skeleton_generator/prompt_v1/cot_sk_state.txt deleted file mode 100644 index 8c7d2692..00000000 --- a/data/prompts/skeleton_generator/prompt_v1/cot_sk_state.txt +++ /dev/null @@ -1,27 +0,0 @@ -[user] -Information about world states should be in json dictionary format. Example: -""" -{ - "objects": [ - {"name": "leg1", "properties": []}, # leg1 is an object with no properties - {"name": "seat", "properties": []}, # seat is an object with no properties - {"name": "hand", "properties": ["is_free"]}, # hand is an object with a property "is_free" - ], - "constraints": [ - {"source": "leg1", "name": "can_screw_to", "target": "seat"}, # means leg1 can be screwed to seat - ], - "relations": [ - {"source": "leg1", "name": "is_screwed_to", "target": "seat"}, # means leg1 is screwed to seat - ], -} -""" -It contains three lists: - - "objects": the objects and their properties (as lists of strings) in the world. The properties can be changed by robot actions. Refer to the input world state for them. - - "constraints": the constraint relations that are enforced and should not be changed. Refer to the pddl problem for the constraints. - - "relations": the relations in the world. They can be changed by robot actions. Refer to input world state for them. -The properties, constraints and relations should always align with the [PREDICATE_LIST] in domain knowledge. - -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/data/prompts/skeleton_generator/query/query copy 2.txt b/data/prompts/skeleton_generator/query/query copy 2.txt deleted file mode 100644 index 2f976561..00000000 --- a/data/prompts/skeleton_generator/query/query copy 2.txt +++ /dev/null @@ -1,20 +0,0 @@ -Start working. Resume from the input below. ---------------- -problem: {problem} - -world_state: {world_state} - -instructions: {instructions} ---------------- -The dictionary that you return should be formatted as json file. Follow these rules: -1. Always equip the proper tools before performing specific object manipulaion. -2. Make sure that each element of the ["action_instructions"] explains corresponding element of the ["action_sequence"]. Refer to the "ROBOT ACTION LIST" to understand the elements of ["action_sequence"]. -3, The length of the ["action_instructions"] list and the ["action_sequence"] list should be the same. -4. Never left ',' at the end of the list. Use json boolean values, "true" or "false", instead of "True" or "False". -5. Keep track of all the objects, constraints and relations in the "initial_state" field. -6. The ["initial_state"] should be generated based on "PREDICATE_LIST". -7. All keys of the dictionary should be double-quoted. -8. Return the json dictionary without adding anything at the beginning and the end of it. You return should be ready to be parsed as a json object directly. -9. Make sure that you output a consistent manipultation as a single arm robot. For example, you need to unload the tool held by your hand before equipping a new tool. -10. When generating the behavior tree, make sure the tree has the asked structure. -Adhere to the output format I defined above. Follow the rules above. Think step by step. \ No newline at end of file diff --git a/data/prompts/skeleton_generator/query/query copy 3.txt b/data/prompts/skeleton_generator/query/query copy 3.txt deleted file mode 100644 index 42cb6d8c..00000000 --- a/data/prompts/skeleton_generator/query/query copy 3.txt +++ /dev/null @@ -1,18 +0,0 @@ -Start working. Resume from the input below. ---------------- -problem: {problem} - -world_state: {world_state} - -instructions: {instructions} ---------------- -The dictionary that you return should be formatted as json file. Follow these rules: -1. Always equip the proper tools before performing specific object manipulaion. -2. Refer to the "ROBOT ACTION LIST" to understand the elements of ["action_sequence"]. -3. Never left ',' at the end of the list. Use json boolean values, "true" or "false", instead of "True" or "False". -4. Keep track of all the objects, constraints and relations in the "world_state" field. -5. All keys of the dictionary should be double-quoted. -6. Return the json dictionary without adding anything at the beginning and the end of it. You return should be ready to be parsed as a json object directly. -7. Make sure that you output a consistent manipultation as a single arm robot. -8. When generating the behavior tree, make sure the tree has the asked structure. -Adhere to the output format I defined above. Follow the rules above. Think step by step. \ No newline at end of file diff --git a/data/prompts/skeleton_generator/query/query copy.txt b/data/prompts/skeleton_generator/query/query copy.txt deleted file mode 100644 index cbe4d915..00000000 --- a/data/prompts/skeleton_generator/query/query copy.txt +++ /dev/null @@ -1,16 +0,0 @@ -Start working. Resume from the problem below. ---------------- -{problem} ---------------- -The dictionary that you return should be formatted as json file. Follow these rules: -1. Always equip the proper tools before performing specific object manipulaion. -2. Make sure that each element of the ["action_instructions"] explains corresponding element of the ["action_sequence"]. Refer to the "ROBOT ACTION LIST" to understand the elements of ["action_sequence"]. -3, The length of the ["action_instructions"] list and the ["action_sequence"] list should be the same. -4. Never left ',' at the end of the list. Use json boolean values, "true" or "false", instead of "True" or "False". -5. Keep track of all items listed in the "objects" section of the "initial_state" field. -6. Use the "PREDICATE_LIST" for the initial states (:init) in the problem to generate ["initial_state"]. -7. All keys of the dictionary should be double-quoted. -8. Return the json dictionary without adding anything at the beginning and the end of it. You return should be ready to be parsed as a json object directly. -9. Make sure that you output a consistent manipultation as a single arm robot. For example, grasping an object should not occur in successive steps. -10. When generating the behavior tree, make sure the tree has the asked structure. -Adhere to the output format I defined above. Follow the rules above. Think step by step. \ No newline at end of file diff --git a/data/prompts/skeleton_generator/query/query.txt b/data/prompts/skeleton_generator/query/query.txt deleted file mode 100644 index 879e1195..00000000 --- a/data/prompts/skeleton_generator/query/query.txt +++ /dev/null @@ -1,17 +0,0 @@ -Start working. Resume from the input below. ---------------- -objects: {objects} - -world_state: {world_state} - -instructions: {instructions} ---------------- -The dictionary that you return should be formatted as json file. Follow these rules: -1. Always equip the proper tools before performing specific object manipulaion. -2. Refer to the "ROBOT ACTION LIST" to understand the elements of ["action_sequence"]. -3. Never left ',' at the end of the list. Use json boolean values, "true" or "false", instead of "True" or "False". -4. All keys of the dictionary should be double-quoted. -5. Your return should be ready to be parsed as a json object directly. -6. Make sure that you output a consistent manipultation as a single arm robot. -7. Make sure the behavior tree has the asked structure. -Adhere to the output format I defined above. Follow the rules above. Think step by step. \ No newline at end of file diff --git a/data/prompts/skeleton_generator/system/system.txt b/data/prompts/skeleton_generator/system/system.txt deleted file mode 100644 index 26f2afe2..00000000 --- a/data/prompts/skeleton_generator/system/system.txt +++ /dev/null @@ -1 +0,0 @@ -You are an excellent interpreter of instructions for robotic assembly tasks. Given the knowledge you need, the world states and some instructions, you break it down and construct robot task plans represented by behaiovr trees based on the action sequences. \ No newline at end of file diff --git a/data/prompts/skeleton_refiner/prompt/refine_action.txt b/data/prompts/skeleton_refiner/prompt/refine_action.txt deleted file mode 100644 index eb6e4839..00000000 --- a/data/prompts/skeleton_refiner/prompt/refine_action.txt +++ /dev/null @@ -1,52 +0,0 @@ -[user] -For action nodes, you should: -1. refine and replace the summaries in the key "effects", which is a list of the to-be-exerted effects of the action. Each effect should have the following keys: - - "object_name": the affected object. - - "property_name": the affected property/relation of the object. - - "property_value": this should be the other object if this is a relation. For property it should be null. - - "status": true if the effect is to add this property/relation. false if is to remove it. -An example is shown below: -the skeleton: -""" -{ - "summary": "left hand equips tool1", - "name": "action: load_tool(left_hand, tool1)", - "effects": [ - {"summary": "left hand will be not free anymore",}, - {"summary": "tool1 will be not equippable anymore",}, - {"summary": "left_hand will hold tool1",}, - ], -}, -""" -after refinement: -""" -{ - "summary": "left hand equips tool1", - "name": "action: load_tool(left_hand, tool1)", - "identifier": 8, - "type_name": "action", - "effects": [ - { - "object_name": "left_hand", - "property_name": "is_free", - "property_value": null, - "status": false, - }, - { - "object_name": "tool1", - "property_name": "is_equippable", - "property_value": null, - "status": false, - }, - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "tool1", - "status": true, - }, - ], -}, -""" - -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/data/prompts/skeleton_refiner/prompt/refine_chain.txt b/data/prompts/skeleton_refiner/prompt/refine_chain.txt deleted file mode 100644 index f92d937d..00000000 --- a/data/prompts/skeleton_refiner/prompt/refine_chain.txt +++ /dev/null @@ -1,14 +0,0 @@ -[user] -Follow this workflow to solve the task: - -//////////////// NOT USED -1. Read the input. Parse and generate the "initial_state" in the output dictionary. -2. Interpret the goal of the problem as goal_state and write into the "instruction_summary" in the output dictionary. -3. Try to solve the problem, achieve the goal_state from the initial_state based on the domain knowledge provided. Generate a action sequence with the actions defined in and write it into dictionary["task_plan"]["action_sequence"] of the output. -4. If you cannot find a plan to solve the task, ask question with nature language in the "question" of the output dictionary. -5. Explain the actions in the action sequence and write the output dictionary["task_plan"]["action_instructions"]. -6. Collect the objects in the world that are effected by the action sequence, write into dictionary["task_plan"]["manipulated_objects"]. -7. Generate the behavior tree based on the action sequence and write into dictionary["task_plan"]["behavior_tree"]. - -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/data/prompts/skeleton_refiner/prompt/refine_condition.txt b/data/prompts/skeleton_refiner/prompt/refine_condition.txt deleted file mode 100644 index e3e6674e..00000000 --- a/data/prompts/skeleton_refiner/prompt/refine_condition.txt +++ /dev/null @@ -1,35 +0,0 @@ -[user] -For condition nodes, you should: -1. refine and add the key "conditions", which is a list of conditions to be checked. Each condition should have the following keys: - - "object_name": the object to check. - - "property_name": the property/relation of the object to be checked. - - "property_value": this should be the other object in the relation. For property it should be null. - - "status": true if the property/relation is expected to be true, false otherwise. -An example is shown below: -the skeleton: -""" -{ - "summary": "check if left_hand is holding parallel_box1", - "name": "target: hold(left_hand, parallel_box1)", -} -""" -after refinement: -""" -{ - "summary": "check if left_hand is holding parallel_box1", - "name": "target: hold(left_hand, parallel_box1)", - "identifier": 4, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "parallel_box1", - "status": true, - } - ], -}, -""" - -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/data/prompts/skeleton_refiner/prompt/refine_controlflow.txt b/data/prompts/skeleton_refiner/prompt/refine_controlflow.txt deleted file mode 100644 index 83c831f2..00000000 --- a/data/prompts/skeleton_refiner/prompt/refine_controlflow.txt +++ /dev/null @@ -1,28 +0,0 @@ -[user] -For each node, you should: -- add an identifier field to the node, which is a unique integer number. -- add a type_name field to the node, which is a string that represents the type of the node. - -For control flow nodes, namely selectors and sequences, an example is shown as follows: -the skeleton: -""" -{ - "summary": "selector to load no_tool to hand", - "name": "selector: load_tool(left_hand, no_tool)", - "children": [...], -} - -""" -after refinement: -""" -{ - "summary": "selector to load no_tool to hand", - "name": "selector: load_tool(left_hand, no_tool)", - "identifier": 3, - "type_name": "selector", - "children": [...], -} -""" - -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/data/prompts/skeleton_refiner/prompt/refine_domain.txt b/data/prompts/skeleton_refiner/prompt/refine_domain.txt deleted file mode 100644 index 44fa0366..00000000 --- a/data/prompts/skeleton_refiner/prompt/refine_domain.txt +++ /dev/null @@ -1,88 +0,0 @@ -[user] -///////////////////// nNOT USED -Following is the domain knowledge you need in pddl domain format, which includes: -- the object types in the world, -- the predicates for describing the states of the world, -- the actions that the robot can apply, with their precondtions and effects defined. -""" -(define (domain robot_assembly_problem-domain) - (:requirements :strips :typing) - (:types - thing - object - tool part hand - thing - ) - (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part)) - (:action pick_up - :parameters ( ?hand - hand ?tool - tool ?part - part) - :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part)) - :effect (and (hold ?tool ?part) (not (is_free ?tool)))) - (:action put_down - :parameters ( ?hand - hand ?tool - tool ?part - part) - :precondition (and (hold ?tool ?part) (hold ?hand ?tool)) - :effect (and (not (hold ?hand ?part)) (is_free ?tool))) - (:action place - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2))) - (:action detach - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2)))) - (:action insert - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2))) - (:action pull - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2)))) - (:action screw - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2))) - (:action unscrew - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2)))) - (:action load_tool - :parameters ( ?hand - hand ?tool - tool) - :precondition (and (is_equippable ?tool) (is_free ?hand)) - :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool))) - (:action unload_tool - :parameters ( ?hand - hand ?tool - tool) - :precondition (and (hold ?hand ?tool) (is_free ?tool)) - :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool)))) -) -""" -The precidates are explained as follows: -""" -"PREDICATE_LIST" -- is_free( or ): or is free and can hold something. (default: true) -- is_equippable(): is ready to be eqquipped by a robot hand. (default: true) -- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false) -- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false) -- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false) -- can_place_to(, ): can be placed into (placement assembly constraint) (default: false) -- hold(, ): is holding (default: false) -- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false) -- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false) -- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false) -""" - -The actions are explained as follows: -""" -"ROBOT_ACTION_LIST" -- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something. -- put_down(, , ): use to put down . As preconditions, should be holding and should be holding . -- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding . -- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into . -- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding . -- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into . -- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding . -- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into . -- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free. -- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free. -""" -the items in <> indicate the type of the instances needed for the predicates and actions. -[assistant] -Understood. Waiting for next input. diff --git a/data/prompts/skeleton_refiner/prompt/refine_help copy.txt b/data/prompts/skeleton_refiner/prompt/refine_help copy.txt deleted file mode 100644 index 01185254..00000000 --- a/data/prompts/skeleton_refiner/prompt/refine_help copy.txt +++ /dev/null @@ -1,9 +0,0 @@ -[user] -You should refine the behavior tree skeleton into a full behavior tree in json dictionary format. - -1. Refer to "initial_state" of input to find the objects, properties, constraints and relationships. -2. Refer to the domain knowledge to understand the actions (corresponds to "ROBOT_ACTION_LIST") and conditions (corresponds to "PREDICATE_LIST"). -3. The name of the node (if it has) also provides information for its refinement. - -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/data/prompts/skeleton_refiner/prompt/refine_help.txt b/data/prompts/skeleton_refiner/prompt/refine_help.txt deleted file mode 100644 index 1b719d60..00000000 --- a/data/prompts/skeleton_refiner/prompt/refine_help.txt +++ /dev/null @@ -1,8 +0,0 @@ -[user] -You should refine the behavior tree skeleton into a full behavior tree in json dictionary format. - -1. Refer to "initial_state" of input to find the objects, properties, constraints and relationships. -2. The name of the node provides information for its refinement. - -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/data/prompts/skeleton_refiner/prompt/refine_input_format copy.txt b/data/prompts/skeleton_refiner/prompt/refine_input_format copy.txt deleted file mode 100644 index 1f5985a0..00000000 --- a/data/prompts/skeleton_refiner/prompt/refine_input_format copy.txt +++ /dev/null @@ -1,153 +0,0 @@ -[user] -The input you get and the output you return should be in json dictionary format. -The dictionary contains the following keys: -""" -- dictionary["task_plan"]: A dictionary containing the task plan. -- dictionary["initial_state"]: The initial world state before executing the plan. -- dictionary["question"]: Question for more knowledge that are unknown yet but necessary for making the plan. Normally empty. -""" -Three keys exist in dictionary["task_plan"]. -""" -- dictionary["task_plan"]["action_sequence"]: Contains a list of robot actions. -- dictionary["task_plan"]["manipulated_objects"]: The list of the objects that are related to the current task. -- dictionary["task_plan"]["behavior_tree"]: contains a dictionary of the behavior tree skeleton constructed according to the dictionary ["task_plan"]["action_sequence"] -""" -The following is an input example: -``` -{"task_plan": { - "action_sequence": [ - "load_tool(left_hand, parallel_box1)", - "pick_up(left_hand, parallel_box1, gear1)", - "insert(left_hand, parallel_box1, gear1, shaft1)" - ], - "object_name": ["left_hand", "gear1", "parallel_box1", "shaft1"], - "behavior_tree": { - "summary": "selector to use left hand with parallel_box1 to insert gear1 to shaft1", - "name": "selector: insert(left_hand, parallel_box1, gear1, shaft1)", - "children": [ - { - "summary": "condition node to check if gear1 is inserted to shaft1", - "name": "target: is_inserted_to(gear1, shaft1)" - }, - { - "summary": "sequence to insert gear1 into shaft1", - "name": "sequence: insert(left_hand, parallel_box1, gear1, shaft1)", - "children": [ - { - "summary": "selector to load left hand with parallel_box1", - "name": "selector: load_tool(left_hand, parallel_box1)", - "children": [ - { - "summary": "condition node to check if left hand holds parallel_box1", - "name": "target: hold(left_hand, parallel_box1)" - }, - { - "summary": "sequence to load left hand with parallel_box1", - "name": "sequence: load_tool(left_hand, parallel_box1)", - "children": [ - { - "summary": "condition node to check if parallel_box1 is equippable", - "name": "precondition: is_equippable(parallel_box1)" - }, - { - "summary": "condition node to check if left hand is free", - "name": "precondition: is_free(left_hand)" - }, - { - "summary": "action node to equip parallel_box1 to left hand", - "name": "action: load_tool(left_hand, parallel_box1)", - "effects": [ - {"summary": "left_hand will be not free",}, - {"summary": "parallel_box1 will be not equippable",}, - {"summary": "left_hand will hold parallel_box1",}, - ], - }, - ], - }, - ], - }, - { - "summary": "selector to use left_hand with parallel_box1 to pick up gear1", - "name": "selector: pick_up(left_hand, parallel_box1, gear1)", - "children": [ - { - "summary": "condition node to check if parallel_box1 holds gear1", - "name": "target: hold(parallel_box1, gear1)" - }, - { - "summary": "sequence to use left hand with parallel_box1 to pick up gear1", - "name": "sequence: pick_up(left_hand, parallel_box1, gear1)", - "children": [ - { - "summary": "condition node to check if parallel_box1 is free", - "name": "precondition: is_free(parallel_box1)" - }, - { - "summary": "condition node to check if left hand holds parallel_box1", - "name": "precondition: hold(left_hand, parallel_box1)" - }, - { - "summary": "action node to use left hand with parallel_box1 to pick up gear1", - "name": "action: pick_up(left_hand, parallel_box1, gear1)", - "effects": [ - {"summary": "parallel_box1 will hold gear1",}, - {"summary": "parallel_box1 will be not free",}, - ], - }, - ], - }, - ], - }, - { - "summary": "condition node to check if gear1 can be inserted to shaft1", - "name": "precondition: can_insert_to(gear1, shaft1)" - }, - { - "summary": "action node to use left hand with parallel_box1 to insert gear1 to shaft1", - "name": "action: insert(left_hand, parallel_box1, gear1, shaft1)", - "effects": [ - {"summary": "parallel_box1 will be not holding gear1",}, - {"summary": "parallel_box1 will be free",}, - {"summary": "gear1 will be inserted to shaft1",}, - ], - }, - ], - }, - ], -} -}, -"initial_state": { - "objects": [ - {"name": "parallel_box1", "properties": ["is_free", "is_equippable"]}, - {"name": "parallel_box2", "properties": ["is_free", "is_equippable"]}, - {"name": "inward_claw", "properties": ["is_free", "is_equippable"]}, - {"name": "outward_claw", "properties": ["is_free", "is_equippable"]}, - {"name": "no_tool", "properties": ["is_free", "is_equippable"]}, - {"name": "gear1", "properties": []}, - {"name": "gear2", "properties": []}, - {"name": "gear3", "properties": []}, - {"name": "shaft1", "properties": []}, - {"name": "shaft2", "properties": []}, - {"name": "shaft3", "properties": []}, - {"name": "gearbase", "properties": []}, - {"name": "gearbase_hole1", "properties": []}, - {"name": "gearbase_hole3", "properties": []}, - {"name": "left_hand", "properties": ["is_free"]} - ], - "constraints": [ - {"source": "parallel_box1", "name": "can_manipulate", "target": "shaft1"}, - {"source": "parallel_box2", "name": "can_manipulate", "target": "gear1"}, - {"source": "outward_claw", "name": "can_manipulate", "target": "gear2"}, - {"source": "outward_claw", "name": "can_manipulate", "target": "gear3"}, - {"source": "no_tool", "name": "can_manipulate", "target": "shaft3"}, - {"source": "shaft1", "name": "can_insert_to", "target": "gearbase_hole1"}, - {"source": "shaft3", "name": "can_insert_to", "target": "gearbase_hole3"}, - {"source": "gear3", "name": "can_insert_to", "target": "shaft3"} - ], - "relations": [] -}, -"question":""} -``` - -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/data/prompts/skeleton_refiner/prompt/refine_input_format.txt b/data/prompts/skeleton_refiner/prompt/refine_input_format.txt deleted file mode 100644 index ed6dfdd5..00000000 --- a/data/prompts/skeleton_refiner/prompt/refine_input_format.txt +++ /dev/null @@ -1,69 +0,0 @@ -[user] -The input you get and the output you return should be in json dictionary format. -The dictionary contains the following keys: -""" -- dictionary["task_plan"]: A dictionary containing the task plan. -- dictionary["initial_state"]: The initial world state before executing the plan. -- dictionary["question"]: Question for more knowledge that are unknown yet but necessary for making the plan. Normally empty. -""" -Three keys exist in dictionary["task_plan"]. -""" -- dictionary["task_plan"]["action_sequence"]: Contains a list of robot actions. -- dictionary["task_plan"]["manipulated_objects"]: The list of the objects that are related to the current task. -- dictionary["task_plan"]["behavior_tree"]: contains a dictionary of the behavior tree skeleton constructed according to the dictionary ["task_plan"]["action_sequence"] -""" -The following is an input example: -``` -{"task_plan": { - "action_sequence": [ - "load_tool(left_hand, parallel_box1)", - ], - "object_name": ["left_hand", "parallel_box1"], - "behavior_tree": { - "summary": "selector to load left hand with parallel_box1", - "name": "selector: load_tool(left_hand, parallel_box1)", - "children": [ - { - "summary": "condition node to check if left hand holds parallel_box1", - "name": "target: hold(left_hand, parallel_box1)" - }, - { - "summary": "sequence to load left hand with parallel_box1", - "name": "sequence: load_tool(left_hand, parallel_box1)", - "children": [ - { - "summary": "condition node to check if parallel_box1 is equippable", - "name": "precondition: is_equippable(parallel_box1)" - }, - { - "summary": "condition node to check if left hand is free", - "name": "precondition: is_free(left_hand)" - }, - { - "summary": "action node to equip parallel_box1 to left hand", - "name": "action: load_tool(left_hand, parallel_box1)", - "effects": [ - {"summary": "left_hand will be not free",}, - {"summary": "parallel_box1 will be not equippable",}, - {"summary": "left_hand will hold parallel_box1",}, - ], - }, - ], - }, - ], -} -}, -"initial_state": { - "objects": [ - {"name": "parallel_box1", "properties": ["is_free", "is_equippable"]}, - {"name": "left_hand", "properties": ["is_free"]} - ], - "constraints": [ - ], - "relations": [] -}, -"question":""} -``` - -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/data/prompts/skeleton_refiner/prompt/refine_output_format.txt b/data/prompts/skeleton_refiner/prompt/refine_output_format.txt deleted file mode 100644 index 6e346611..00000000 --- a/data/prompts/skeleton_refiner/prompt/refine_output_format.txt +++ /dev/null @@ -1,16 +0,0 @@ -[user] -The output dictionary has four keys. -""" -- dictionary["task_plan"] -- dictionary["initial_state"] -- dictionary["question"] -""" -Four keys exist in dictionary["task_plan"]. -""" -- dictionary["task_plan"]["action_sequence"] -- dictionary["task_plan"]["manipulated_objects"] -- dictionary["task_plan"]["behavior_tree"] -""" - -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/data/prompts/skeleton_refiner/prompt/refine_role.txt b/data/prompts/skeleton_refiner/prompt/refine_role.txt deleted file mode 100644 index 5dbb1793..00000000 --- a/data/prompts/skeleton_refiner/prompt/refine_role.txt +++ /dev/null @@ -1,4 +0,0 @@ -[user] -You are an excellent task planner for robotic assembly tasks. Given the world state and a rough plan, you break it down and refine the plan into a well formatted and executable behavior tree. -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/data/prompts/skeleton_refiner/prompt/refine_state.txt b/data/prompts/skeleton_refiner/prompt/refine_state.txt deleted file mode 100644 index dbce7c03..00000000 --- a/data/prompts/skeleton_refiner/prompt/refine_state.txt +++ /dev/null @@ -1,23 +0,0 @@ -[user] -The initial state in the input is explained as follows: - - "objects": the objects and their properties (as lists of strings) in the world. The properties can be changed by robot actions. - - "constraints": the constraint relations that are enforced and should not be changed. They are knowledge about the world or some constraints set by the user. - - "relations": the relations in the world. They can be changed by robot actions. -following is another example: -""" -{ - "objects": [ - {"name": "leg1", "properties": []}, # leg1 is an object with no properties - {"name": "seat", "properties": []}, # seat is an object with no properties - {"name": "hand", "properties": ["is_free"]}, # hand is an object with a property "is_free" - ], - "constraints": [ - {"source": "leg1", "name": "can_screw_to", "target": "seat"}, # means leg1 can be screwed to seat - ], - "relations": [ - {"source": "leg1", "name": "is_screwed_to", "target": "seat"}, # means leg1 is screwed to seat - ], -} -""" -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/data/prompts/skeleton_refiner/query/dup_query.txt b/data/prompts/skeleton_refiner/query/dup_query.txt deleted file mode 100644 index c1a7955e..00000000 --- a/data/prompts/skeleton_refiner/query/dup_query.txt +++ /dev/null @@ -1,16 +0,0 @@ -Start working. Resume from the input below. -""" -{"input": [PROBLEM]} -""" -The dictionary that you return should be formatted as json file. Follow these rules: -1. Always equip the proper tools before performing specific object manipulaion. -2. Make sure that each element of the ["action_instructions"] explains corresponding element of the ["action_sequence"]. Refer to the "ROBOT ACTION LIST" to understand the elements of ["action_sequence"]. -3, The length of the ["action_instructions"] list and the ["action_sequence"] list should be the same. -4. Never left ',' at the end of the list. Use json boolean values, "true" or "false", instead of "True" or "False". -5. Keep track of all items listed in the "objects" section of the "initial_state" field. -6. Use the "PREDICATE_LIST" for the initial states (:init) in the problem to generate ["initial_state"]. -7. All keys of the dictionary should be double-quoted. -8. Insert ``` at the beginning and the end of the dictionary to separate it from the rest of your response. Don't write any additional things. -9. Make sure that you output a consistent manipultation as a single arm robot. For example, grasping an object should not occur in successive steps. -10. When generating the behavior tree, make sure the tree has the asked structure. -Adhere to the output format I defined above. Follow the rules above. Think step by step. \ No newline at end of file diff --git a/data/prompts/skeleton_refiner/query/query.txt b/data/prompts/skeleton_refiner/query/query.txt deleted file mode 100644 index 861444e0..00000000 --- a/data/prompts/skeleton_refiner/query/query.txt +++ /dev/null @@ -1,14 +0,0 @@ -Start working. Resume from the input below. -""" -input: {input} -""" -The dictionary that you return should be formatted as json file. Follow these rules: -1. Only refine the skeleton behavior tree in the "behavior_tree" field. Refine the tree according to rules provided before. Replace the old behavior tree with the refined behavior tree. -2. Refer to the domain knowledge, namely "ROBOT_ACTION_LIST" and "PREDICATE_LIST", to understand and refine the skeleton behavior tree. -3. Do not change other fields of the dictionary and return the new dictionary as json file. -4. Never left ',' at the end of the list. -5. Use json boolean values, "true" or "false", instead of "True" or "False". -6. All keys of the dictionary should be double-quoted. -7. Insert ``` at the beginning and the end of the dictionary to separate it from the rest of your response. Don't write any additional things. -8. When generating the behavior tree, make sure the tree has the asked structure. -Adhere to the output format I defined above. Follow the rules above. Think step by step. \ No newline at end of file diff --git a/data/prompts/skeleton_refiner/system/system.txt b/data/prompts/skeleton_refiner/system/system.txt deleted file mode 100644 index 5bc3c5a6..00000000 --- a/data/prompts/skeleton_refiner/system/system.txt +++ /dev/null @@ -1 +0,0 @@ -You are an excellent robot behavior tree expert in robotic assembly tasks. Given the knowledge you need, the world state and a rough plan, you break it down and refine the task plan to make it well formatted and ready to be executed. \ No newline at end of file diff --git a/data/prompts/sot/action_sequence/COMPLETE b/data/prompts/sot/action_sequence/COMPLETE deleted file mode 100644 index e69de29b..00000000 diff --git a/data/prompts/sot/action_sequence/chain.txt b/data/prompts/sot/action_sequence/chain.txt deleted file mode 100644 index 9116e19d..00000000 --- a/data/prompts/sot/action_sequence/chain.txt +++ /dev/null @@ -1,5 +0,0 @@ -You follow the work flow below: -1. analyze the world state. describe the current state of the world, including the objects, the objects' properties, the constraints and the relationships. -2. you find the goal state. -3. you find the action steps that can be performed to reach the goal state. -4. you format the actions and put them in a sequence to form the output plan. \ No newline at end of file diff --git a/data/prompts/sot/action_sequence/domain.txt b/data/prompts/sot/action_sequence/domain.txt deleted file mode 100644 index b67bcb08..00000000 --- a/data/prompts/sot/action_sequence/domain.txt +++ /dev/null @@ -1,41 +0,0 @@ -Following is the domain knowledge you need, which includes: -- the object types in the world, -- the predicates for describing the states of the world, -- the actions that the robot can apply, with their precondtions and effects defined. - -The object types are explained as follows: -""" -thing: The base class for all objects in the world. Its subclasses are: hand, tool, part. -hand: The robot hand. A empty can only hold one at a time. -tool: The tool to manipulate . A empty can hold one at a time. -part: The part to be manipulated. -""" - -The precidates are explained as follows: -""" -"PREDICATE_LIST" -- is_empty(): is empty and can hold something. -- can_manipulate(, ): can be used to manipulate (manipulation constraint). This regulates the compatibility between and . -- can_insert_to(, ): can be inserted into (insertion assembly constraint) -- can_screw_to(, ): can be screwed into (screw assembly constraint) -- can_place_to(, ): can be placed into (placement assembly constraint) -- hold(, ): is holding . It can be holding or holding . -- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) -- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) -- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) -""" - -The actions are explained as follows: -""" -"ROBOT_ACTION_LIST" -- pick_up(, , ): use to pick up . As preconditions, should be holding and should be empty in order to hold the . The will be held by the as a result of this action. -- put_down(, , ): use to put down . As preconditions, should be holding and should be holding . The will be empty and will not hold the anymore as a result of this action. -- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding . -- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be placed into . -- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding . -- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be inserted into . -- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding . -- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be screwed into . -- change_tool(, , ): change the equipped tool from to in order to manipulate specific parts. As preconditions, should be holding and should be empty. -""" -the items in <> indicate the type of the instances needed for the predicates and actions. \ No newline at end of file diff --git a/data/prompts/sot/action_sequence/example.txt b/data/prompts/sot/action_sequence/example.txt deleted file mode 100644 index 2b58c241..00000000 --- a/data/prompts/sot/action_sequence/example.txt +++ /dev/null @@ -1,26 +0,0 @@ -Following is an input/output example: -INPUT: -start_world_state: -{ -"objects": [ - {"name": "outward_claw", "properties": ["is_empty"]}, - {"name": "shaft1", "properties": []}, - {"name": "left_hand", "properties": []}, - {"name": "parallel_box1", "properties": [is_empty]} -], -"constraints": [ - {"source": "outward_claw", "name": "can_manipulate", "target": "gear1"} -], -"relations": [ - {"source": "left_hand", "name": "hold", "target": "parallel_box1"} -] -} -target: "change the tool from the parallel_box1 to the outward_claw" - -OUTPUT: -{ - "explanation": "The left_hand is holding the parallel_box1 now and the parallel_box1 is empty, so the change_tool action can be conducted to change the tool in the hand from the parallel_box1 to outward_claw.", - "task_plan": [ - "change_tool(left_hand, parallel_box1, outward_claw)" - ] -} \ No newline at end of file diff --git a/data/prompts/sot/action_sequence/output_format.txt b/data/prompts/sot/action_sequence/output_format.txt deleted file mode 100644 index 05d8c8ee..00000000 --- a/data/prompts/sot/action_sequence/output_format.txt +++ /dev/null @@ -1,3 +0,0 @@ -The output json object has the following keys. -- "task_plan": Contains a list of robot actions in the defined form. Only the actions defined in the "ROBOT_ACTION LIST" will be used. The actions should be in pddl action format. -- "explanation": A string that explains your reasoning process briefly. \ No newline at end of file diff --git a/data/prompts/sot/action_sequence/readme.md b/data/prompts/sot/action_sequence/readme.md deleted file mode 100644 index 4b769667..00000000 --- a/data/prompts/sot/action_sequence/readme.md +++ /dev/null @@ -1,39 +0,0 @@ -what is the proper way to assemble a tree? - -the structure itself is a problem. though you have the unit trees and the action sequence, I still don't think it is a good idea to adopt the action sequence itself as the skeleton of the tree. It doesn't show a tree strucuture, and it does not present the dependency explicitly. - -In json format the following is an attempt. -```json -{ - "actions": - [ - { - "name": "change_tool(hand, tool1, tool2)", - "preconditions_and_related_actions": - { - "hold(hand, tool1)": null, - "empty(tool1)": null - } - }, - { - "name": "pick_up(hand, tool2, p1)", - "preconditions_and_related_actions": - { - "hold(hand, tool2)": "change_tool(hand, tool1, tool2)", - "empty(tool2)": null - } - }, - { - "name": "insert(hand, tool2, p1, p2)", - "preconditions_and_related_actions": - { - "hold(hand, tool2)": "change_tool(hand, tool1, tool2)", - "hold(tool, p1)": "pick_up(hand, tool, p1)" - } - }, - ] -} -``` -maybe it is a good idea to import causal graph to present this dependency and refine it later... ? - -Let's try this json format first. \ No newline at end of file diff --git a/data/prompts/sot/action_sequence/system.txt b/data/prompts/sot/action_sequence/system.txt deleted file mode 100644 index c0b37f9f..00000000 --- a/data/prompts/sot/action_sequence/system.txt +++ /dev/null @@ -1 +0,0 @@ -You are an expert in making plans for robotic assembly tasks. \ No newline at end of file diff --git a/data/prompts/sot/action_sequence/task.txt b/data/prompts/sot/action_sequence/task.txt deleted file mode 100644 index bd444795..00000000 --- a/data/prompts/sot/action_sequence/task.txt +++ /dev/null @@ -1,8 +0,0 @@ -The input are: -start_world_state: the world state when the behavior tree started. -target: the goal that the plan should achieve. - -You summarize the target into a goal world state. -You check if the target has been fulfilled. If so, return your explanation and an empty task plan in one json object. -If the target has not been fulfilled, you make a bullet list task plan to achieve it from the start_world_state. Use the definitions in the domain knowledge to construct actions. Do not add any superfluous steps. Then you return the task plan and your explanation in one json object. -You only output an json object, which is ready to be parsed directly. Any other superfluous output is forbidden. \ No newline at end of file diff --git a/data/prompts/sot/action_sequence/template.txt b/data/prompts/sot/action_sequence/template.txt deleted file mode 100644 index ccd1c78b..00000000 --- a/data/prompts/sot/action_sequence/template.txt +++ /dev/null @@ -1,3 +0,0 @@ -Now start to work with the following inputs: -start_world_state: {start_world_state} -target: {target} \ No newline at end of file diff --git a/data/prompts/sot/readme.md b/data/prompts/sot/readme.md deleted file mode 100644 index ebe46a0b..00000000 --- a/data/prompts/sot/readme.md +++ /dev/null @@ -1,11 +0,0 @@ -> THIS FILE HAS NOT BEEN FINISHED. - -This file shows an iterative way to generate an entire behavior tree. It includes the following steps: - -1. action sequence: generate an action sequence. -2. unit tree: generate unit trees for each action. -3. tree assembly: assembly the tree by replacing preconditions with unit trees according to the action sequence (dependency). -4. (optional)check: check the behavior tree with postorder traversal to check the logic. - - -+ new: relation_analysis: analyze the relation between the actions based on precondition definitions. \ No newline at end of file diff --git a/data/prompts/sot/relation_analysis/COMPLETE b/data/prompts/sot/relation_analysis/COMPLETE deleted file mode 100644 index e69de29b..00000000 diff --git a/data/prompts/sot/relation_analysis/NOTchain.txt b/data/prompts/sot/relation_analysis/NOTchain.txt deleted file mode 100644 index 9116e19d..00000000 --- a/data/prompts/sot/relation_analysis/NOTchain.txt +++ /dev/null @@ -1,5 +0,0 @@ -You follow the work flow below: -1. analyze the world state. describe the current state of the world, including the objects, the objects' properties, the constraints and the relationships. -2. you find the goal state. -3. you find the action steps that can be performed to reach the goal state. -4. you format the actions and put them in a sequence to form the output plan. \ No newline at end of file diff --git a/data/prompts/sot/relation_analysis/domain.txt b/data/prompts/sot/relation_analysis/domain.txt deleted file mode 100644 index b67bcb08..00000000 --- a/data/prompts/sot/relation_analysis/domain.txt +++ /dev/null @@ -1,41 +0,0 @@ -Following is the domain knowledge you need, which includes: -- the object types in the world, -- the predicates for describing the states of the world, -- the actions that the robot can apply, with their precondtions and effects defined. - -The object types are explained as follows: -""" -thing: The base class for all objects in the world. Its subclasses are: hand, tool, part. -hand: The robot hand. A empty can only hold one at a time. -tool: The tool to manipulate . A empty can hold one at a time. -part: The part to be manipulated. -""" - -The precidates are explained as follows: -""" -"PREDICATE_LIST" -- is_empty(): is empty and can hold something. -- can_manipulate(, ): can be used to manipulate (manipulation constraint). This regulates the compatibility between and . -- can_insert_to(, ): can be inserted into (insertion assembly constraint) -- can_screw_to(, ): can be screwed into (screw assembly constraint) -- can_place_to(, ): can be placed into (placement assembly constraint) -- hold(, ): is holding . It can be holding or holding . -- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) -- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) -- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) -""" - -The actions are explained as follows: -""" -"ROBOT_ACTION_LIST" -- pick_up(, , ): use to pick up . As preconditions, should be holding and should be empty in order to hold the . The will be held by the as a result of this action. -- put_down(, , ): use to put down . As preconditions, should be holding and should be holding . The will be empty and will not hold the anymore as a result of this action. -- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding . -- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be placed into . -- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding . -- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be inserted into . -- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding . -- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be screwed into . -- change_tool(, , ): change the equipped tool from to in order to manipulate specific parts. As preconditions, should be holding and should be empty. -""" -the items in <> indicate the type of the instances needed for the predicates and actions. \ No newline at end of file diff --git a/data/prompts/sot/relation_analysis/example.txt b/data/prompts/sot/relation_analysis/example.txt deleted file mode 100644 index 8407e367..00000000 --- a/data/prompts/sot/relation_analysis/example.txt +++ /dev/null @@ -1,42 +0,0 @@ -Following is an input/output example: -INPUT: -action_sequence: -[ - "change_tool(hand, tool1, tool2)", - "pick_up(hand, tool2, p1)", - "insert(hand, tool2, p1, p2)" -] -explanation: "after change_tool being executed, hand will hold tool2, which allows pick_up to be excuted. After pick_up, tool2 will hold p1, which allows insert to be executed." - - -OUTPUT: -{ - "actions": - [ - { - "name": "change_tool(hand, tool1, tool2)", - "preconditions_and_related_actions": - { - "hold(hand, tool1)": null, - "empty(tool1)": null - } - }, - { - "name": "pick_up(hand, tool2, p1)", - "preconditions_and_related_actions": - { - "hold(hand, tool2)": "change_tool(hand, tool1, tool2)", - "empty(tool2)": null - } - }, - { - "name": "insert(hand, tool2, p1, p2)", - "preconditions_and_related_actions": - { - "hold(hand, tool2)": "change_tool(hand, tool1, tool2)", - "hold(tool, p1)": "pick_up(hand, tool, p1)" - } - } - ], - "explanation": "after change_tool is executed, hand will hold tool2, which allows pick_up to be excuted. After pick_up, tool2 will hold p1, which allows insert to be executed." -} \ No newline at end of file diff --git a/data/prompts/sot/relation_analysis/output_format.txt b/data/prompts/sot/relation_analysis/output_format.txt deleted file mode 100644 index da3a8fc4..00000000 --- a/data/prompts/sot/relation_analysis/output_format.txt +++ /dev/null @@ -1,3 +0,0 @@ -The output json object has the following keys. -- "actions": Contains a list of robot actions in the defined form. -- "explanation": A string that explains your reasoning process briefly. \ No newline at end of file diff --git a/data/prompts/sot/relation_analysis/readme.md b/data/prompts/sot/relation_analysis/readme.md deleted file mode 100644 index 7b7411da..00000000 --- a/data/prompts/sot/relation_analysis/readme.md +++ /dev/null @@ -1,51 +0,0 @@ -what is the proper way to assemble a tree? - -the structure itself is a problem. though you have the unit trees and the action sequence, I still don't think it is a good idea to adopt the action sequence itself as the skeleton of the tree. It doesn't show a tree strucuture, and it does not present the dependency explicitly. - -In json format the following is an attempt. -```json -{ - "actions": - [ - { - "name": "change_tool(hand, tool1, tool2)", - "preconditions": - [ - "hold(hand, tool1)", - "empty(tool1)" - ], - "related_actions_and_fulfilled_preconditions": - { - } - }, - { - "name": "pick_up(hand, tool2, p1)", - "preconditions": - [ - "hold(hand, tool2)", - "empty(tool2)" - ], - "related_actions_and_the_fulfilled_preconditions": - { - "change_tool(hand, tool1, tool2)": "hold(hand, tool2)", - } - }, - { - "name": "insert(hand, tool2, p1, p2)", - "preconditions": - [ - "hold(hand, tool2)", - "hold(tool, p1)" - ], - "related_actions_and_fulfilled_preconditions": - { - "change_tool(hand, tool1, tool2)": "hold(hand, tool2)", - "pick_up(hand, tool, p1)": "hold(tool2, p1)", - } - }, - ] -} -``` -maybe it is a good idea to import causal graph to present this dependency and refine it later... ? - -Let's try this json format first. \ No newline at end of file diff --git a/data/prompts/sot/relation_analysis/system.txt b/data/prompts/sot/relation_analysis/system.txt deleted file mode 100644 index c0b37f9f..00000000 --- a/data/prompts/sot/relation_analysis/system.txt +++ /dev/null @@ -1 +0,0 @@ -You are an expert in making plans for robotic assembly tasks. \ No newline at end of file diff --git a/data/prompts/sot/relation_analysis/task.txt b/data/prompts/sot/relation_analysis/task.txt deleted file mode 100644 index dc7e8326..00000000 --- a/data/prompts/sot/relation_analysis/task.txt +++ /dev/null @@ -1,7 +0,0 @@ -The input are: -action_sequence: a list of actions in the order they should be executed. -explanation: a string explaining the action sequence. - -You analyze the relationships between the actions in the action sequence based on domain knowledge. -You generate the asked json object according to your analysis. -You only output an json object, which is ready to be parsed directly. Any other superfluous output is forbidden. \ No newline at end of file diff --git a/data/prompts/sot/relation_analysis/template.txt b/data/prompts/sot/relation_analysis/template.txt deleted file mode 100644 index 11806ea5..00000000 --- a/data/prompts/sot/relation_analysis/template.txt +++ /dev/null @@ -1,3 +0,0 @@ -Now start to work with the following inputs: -action_sequence: {action_sequence} -explanation: {explanation} diff --git a/data/prompts/sot/skeleton_refine/UNCOMPLETE b/data/prompts/sot/skeleton_refine/UNCOMPLETE deleted file mode 100644 index 07d3a15d..00000000 --- a/data/prompts/sot/skeleton_refine/UNCOMPLETE +++ /dev/null @@ -1,4 +0,0 @@ -noch nicht angefangen. - -Beispielen sind wahrscheinlich nicht notwendig. -Domain ebenso wahrscheinlich nicht. \ No newline at end of file diff --git a/data/prompts/sot/skeleton_refine/chain.txt b/data/prompts/sot/skeleton_refine/chain.txt deleted file mode 100644 index f2f7fb07..00000000 --- a/data/prompts/sot/skeleton_refine/chain.txt +++ /dev/null @@ -1,8 +0,0 @@ -You follow the work flow below: -1. For the - - -analyze the world state. describe the current state of the world, including the objects, the objects' properties, the constraints and the relationships. -2. you find the goal state. -3. you find the action steps that can be performed to reach the goal state. -4. you format the actions and put them in a sequence to form the output plan. \ No newline at end of file diff --git a/data/prompts/sot/skeleton_refine/domain.txt b/data/prompts/sot/skeleton_refine/domain.txt deleted file mode 100644 index 0b82606f..00000000 --- a/data/prompts/sot/skeleton_refine/domain.txt +++ /dev/null @@ -1,42 +0,0 @@ -/////////////////////////// NOT USED /////////////////////////// -Following is the domain knowledge you need, which includes: -- the object types in the world, -- the predicates for describing the states of the world, -- the actions that the robot can apply, with their precondtions and effects defined. - -The object types are explained as follows: -""" -thing: The base class for all objects in the world. Its subclasses are: hand, tool, part. -hand: The robot hand. A empty can only hold one at a time. -tool: The tool to manipulate . A empty can hold one at a time. -part: The part to be manipulated. -""" - -The precidates are explained as follows: -""" -"PREDICATE_LIST" -- is_empty(): is empty and can hold something. -- can_manipulate(, ): can be used to manipulate (manipulation constraint). This regulates the compatibility between and . -- can_insert_to(, ): can be inserted into (insertion assembly constraint) -- can_screw_to(, ): can be screwed into (screw assembly constraint) -- can_place_to(, ): can be placed into (placement assembly constraint) -- hold(, ): is holding . It can be holding or holding . -- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) -- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) -- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) -""" - -The actions are explained as follows: -""" -"ROBOT_ACTION_LIST" -- pick_up(, , ): use to pick up . As preconditions, should be holding and should be empty in order to hold the . The will be held by the as a result of this action. -- put_down(, , ): use to put down . As preconditions, should be holding and should be holding . The will be empty and will not hold the anymore as a result of this action. -- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding . -- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be placed into . -- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding . -- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be inserted into . -- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding . -- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be screwed into . -- change_tool(, , ): change the equipped tool from to in order to manipulate specific parts. As preconditions, should be holding and should be empty. -""" -the items in <> indicate the type of the instances needed for the predicates and actions. \ No newline at end of file diff --git a/data/prompts/sot/skeleton_refine/example.txt b/data/prompts/sot/skeleton_refine/example.txt deleted file mode 100644 index 2b58c241..00000000 --- a/data/prompts/sot/skeleton_refine/example.txt +++ /dev/null @@ -1,26 +0,0 @@ -Following is an input/output example: -INPUT: -start_world_state: -{ -"objects": [ - {"name": "outward_claw", "properties": ["is_empty"]}, - {"name": "shaft1", "properties": []}, - {"name": "left_hand", "properties": []}, - {"name": "parallel_box1", "properties": [is_empty]} -], -"constraints": [ - {"source": "outward_claw", "name": "can_manipulate", "target": "gear1"} -], -"relations": [ - {"source": "left_hand", "name": "hold", "target": "parallel_box1"} -] -} -target: "change the tool from the parallel_box1 to the outward_claw" - -OUTPUT: -{ - "explanation": "The left_hand is holding the parallel_box1 now and the parallel_box1 is empty, so the change_tool action can be conducted to change the tool in the hand from the parallel_box1 to outward_claw.", - "task_plan": [ - "change_tool(left_hand, parallel_box1, outward_claw)" - ] -} \ No newline at end of file diff --git a/data/prompts/sot/skeleton_refine/output_format.txt b/data/prompts/sot/skeleton_refine/output_format.txt deleted file mode 100644 index 05d8c8ee..00000000 --- a/data/prompts/sot/skeleton_refine/output_format.txt +++ /dev/null @@ -1,3 +0,0 @@ -The output json object has the following keys. -- "task_plan": Contains a list of robot actions in the defined form. Only the actions defined in the "ROBOT_ACTION LIST" will be used. The actions should be in pddl action format. -- "explanation": A string that explains your reasoning process briefly. \ No newline at end of file diff --git a/data/prompts/sot/skeleton_refine/system.txt b/data/prompts/sot/skeleton_refine/system.txt deleted file mode 100644 index c0b37f9f..00000000 --- a/data/prompts/sot/skeleton_refine/system.txt +++ /dev/null @@ -1 +0,0 @@ -You are an expert in making plans for robotic assembly tasks. \ No newline at end of file diff --git a/data/prompts/sot/skeleton_refine/task.txt b/data/prompts/sot/skeleton_refine/task.txt deleted file mode 100644 index b6ffd0d3..00000000 --- a/data/prompts/sot/skeleton_refine/task.txt +++ /dev/null @@ -1,8 +0,0 @@ -The input are: -explanation: the explanation of the action sequence. -action_sequence: the action sequence that was planned. -unit_trees: the corresponding unit trees of the actions in the action sequence. - -You assemble the unit trees into an entire tree based on the action sequence and the explanation. You return the result tree as a json object. - -You only output an json object, which is ready to be parsed directly. Any other superfluous output is forbidden. \ No newline at end of file diff --git a/data/prompts/sot/skeleton_refine/template.txt b/data/prompts/sot/skeleton_refine/template.txt deleted file mode 100644 index 637c192e..00000000 --- a/data/prompts/sot/skeleton_refine/template.txt +++ /dev/null @@ -1,4 +0,0 @@ -Now start to work with the following inputs: -explanation: {explanation} -action_sequence: {action_sequence} -unit_trees: {unit_trees} \ No newline at end of file diff --git a/data/prompts/sot/tree_assembly/NOTchain.txt b/data/prompts/sot/tree_assembly/NOTchain.txt deleted file mode 100644 index f2f7fb07..00000000 --- a/data/prompts/sot/tree_assembly/NOTchain.txt +++ /dev/null @@ -1,8 +0,0 @@ -You follow the work flow below: -1. For the - - -analyze the world state. describe the current state of the world, including the objects, the objects' properties, the constraints and the relationships. -2. you find the goal state. -3. you find the action steps that can be performed to reach the goal state. -4. you format the actions and put them in a sequence to form the output plan. \ No newline at end of file diff --git a/data/prompts/sot/tree_assembly/UNCOMPLETE b/data/prompts/sot/tree_assembly/UNCOMPLETE deleted file mode 100644 index 07d3a15d..00000000 --- a/data/prompts/sot/tree_assembly/UNCOMPLETE +++ /dev/null @@ -1,4 +0,0 @@ -noch nicht angefangen. - -Beispielen sind wahrscheinlich nicht notwendig. -Domain ebenso wahrscheinlich nicht. \ No newline at end of file diff --git a/data/prompts/sot/tree_assembly/domain.txt b/data/prompts/sot/tree_assembly/domain.txt deleted file mode 100644 index b67bcb08..00000000 --- a/data/prompts/sot/tree_assembly/domain.txt +++ /dev/null @@ -1,41 +0,0 @@ -Following is the domain knowledge you need, which includes: -- the object types in the world, -- the predicates for describing the states of the world, -- the actions that the robot can apply, with their precondtions and effects defined. - -The object types are explained as follows: -""" -thing: The base class for all objects in the world. Its subclasses are: hand, tool, part. -hand: The robot hand. A empty can only hold one at a time. -tool: The tool to manipulate . A empty can hold one at a time. -part: The part to be manipulated. -""" - -The precidates are explained as follows: -""" -"PREDICATE_LIST" -- is_empty(): is empty and can hold something. -- can_manipulate(, ): can be used to manipulate (manipulation constraint). This regulates the compatibility between and . -- can_insert_to(, ): can be inserted into (insertion assembly constraint) -- can_screw_to(, ): can be screwed into (screw assembly constraint) -- can_place_to(, ): can be placed into (placement assembly constraint) -- hold(, ): is holding . It can be holding or holding . -- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) -- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) -- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) -""" - -The actions are explained as follows: -""" -"ROBOT_ACTION_LIST" -- pick_up(, , ): use to pick up . As preconditions, should be holding and should be empty in order to hold the . The will be held by the as a result of this action. -- put_down(, , ): use to put down . As preconditions, should be holding and should be holding . The will be empty and will not hold the anymore as a result of this action. -- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding . -- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be placed into . -- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding . -- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be inserted into . -- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding . -- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be screwed into . -- change_tool(, , ): change the equipped tool from to in order to manipulate specific parts. As preconditions, should be holding and should be empty. -""" -the items in <> indicate the type of the instances needed for the predicates and actions. \ No newline at end of file diff --git a/data/prompts/sot/tree_assembly/example.txt b/data/prompts/sot/tree_assembly/example.txt deleted file mode 100644 index 21f3dfc2..00000000 --- a/data/prompts/sot/tree_assembly/example.txt +++ /dev/null @@ -1,36 +0,0 @@ -Following is an input/output example: -INPUT: -"actions": -[ - { - "name": "change_tool(hand, tool1, tool2)", - "preconditions_and_related_actions": - { - "hold(hand, tool1)": null, - "empty(tool1)": null - } - }, - { - "name": "pick_up(hand, tool2, p1)", - "preconditions_and_related_actions": - { - "hold(hand, tool2)": "change_tool(hand, tool1, tool2)", - "empty(tool2)": null - } - }, - { - "name": "insert(hand, tool2, p1, p2)", - "preconditions_and_related_actions": - { - "hold(hand, tool2)": "change_tool(hand, tool1, tool2)", - "hold(tool, p1)": "pick_up(hand, tool, p1)" - } - } -] -"explanation": "after change_tool is executed, hand will hold tool2, which allows pick_up to be excuted. After pick_up, tool2 will hold p1, which allows insert to be executed." - -OUTPUT: -{ - "summary": "a selector to insert p1 into p2 with the tool2 held by hand", - "children": -} diff --git a/data/prompts/sot/tree_assembly/output_format.txt b/data/prompts/sot/tree_assembly/output_format.txt deleted file mode 100644 index ae24db55..00000000 --- a/data/prompts/sot/tree_assembly/output_format.txt +++ /dev/null @@ -1 +0,0 @@ -The output json object is a behavior tree in json dictionary. \ No newline at end of file diff --git a/data/prompts/sot/tree_assembly/system.txt b/data/prompts/sot/tree_assembly/system.txt deleted file mode 100644 index c0b37f9f..00000000 --- a/data/prompts/sot/tree_assembly/system.txt +++ /dev/null @@ -1 +0,0 @@ -You are an expert in making plans for robotic assembly tasks. \ No newline at end of file diff --git a/data/prompts/sot/tree_assembly/task.txt b/data/prompts/sot/tree_assembly/task.txt deleted file mode 100644 index e7ab85b0..00000000 --- a/data/prompts/sot/tree_assembly/task.txt +++ /dev/null @@ -1,9 +0,0 @@ -The input are: -actions: an action sequence with the dependency information between the actions. -explanation: the explanation of the action sequence. - -You assemble a behavior tree according to the information provided and the domain knowledge. -You construct the behavior tree in the asked format. -You return the result tree as a json object. - -You only output an json object, which is ready to be parsed directly. Any other superfluous output is forbidden. \ No newline at end of file diff --git a/data/prompts/sot/tree_assembly/template.txt b/data/prompts/sot/tree_assembly/template.txt deleted file mode 100644 index 2df8cab5..00000000 --- a/data/prompts/sot/tree_assembly/template.txt +++ /dev/null @@ -1,3 +0,0 @@ -Now start to work with the following inputs: -actions: {actions} -explanation: {explanation} \ No newline at end of file diff --git a/data/prompts/sot/unit_tree/UNCOMPLETE b/data/prompts/sot/unit_tree/UNCOMPLETE deleted file mode 100644 index fc3361d4..00000000 --- a/data/prompts/sot/unit_tree/UNCOMPLETE +++ /dev/null @@ -1 +0,0 @@ -not only one tree. generate all and preserve both the action sequence and the explanation. \ No newline at end of file diff --git a/data/prompts/sot/unit_tree/domain.txt b/data/prompts/sot/unit_tree/domain.txt deleted file mode 100644 index b67bcb08..00000000 --- a/data/prompts/sot/unit_tree/domain.txt +++ /dev/null @@ -1,41 +0,0 @@ -Following is the domain knowledge you need, which includes: -- the object types in the world, -- the predicates for describing the states of the world, -- the actions that the robot can apply, with their precondtions and effects defined. - -The object types are explained as follows: -""" -thing: The base class for all objects in the world. Its subclasses are: hand, tool, part. -hand: The robot hand. A empty can only hold one at a time. -tool: The tool to manipulate . A empty can hold one at a time. -part: The part to be manipulated. -""" - -The precidates are explained as follows: -""" -"PREDICATE_LIST" -- is_empty(): is empty and can hold something. -- can_manipulate(, ): can be used to manipulate (manipulation constraint). This regulates the compatibility between and . -- can_insert_to(, ): can be inserted into (insertion assembly constraint) -- can_screw_to(, ): can be screwed into (screw assembly constraint) -- can_place_to(, ): can be placed into (placement assembly constraint) -- hold(, ): is holding . It can be holding or holding . -- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) -- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) -- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) -""" - -The actions are explained as follows: -""" -"ROBOT_ACTION_LIST" -- pick_up(, , ): use to pick up . As preconditions, should be holding and should be empty in order to hold the . The will be held by the as a result of this action. -- put_down(, , ): use to put down . As preconditions, should be holding and should be holding . The will be empty and will not hold the anymore as a result of this action. -- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding . -- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be placed into . -- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding . -- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be inserted into . -- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding . -- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be screwed into . -- change_tool(, , ): change the equipped tool from to in order to manipulate specific parts. As preconditions, should be holding and should be empty. -""" -the items in <> indicate the type of the instances needed for the predicates and actions. \ No newline at end of file diff --git a/data/prompts/sot/unit_tree/system.txt b/data/prompts/sot/unit_tree/system.txt deleted file mode 100644 index 770c1052..00000000 --- a/data/prompts/sot/unit_tree/system.txt +++ /dev/null @@ -1 +0,0 @@ -You are an excellent task planner for robotic assembly tasks. Given an action, you construct a unit subtree for the action based on its definition in the domain knowledge. \ No newline at end of file diff --git a/data/prompts/sot/unit_tree/task.txt b/data/prompts/sot/unit_tree/task.txt deleted file mode 100644 index 2a3a4062..00000000 --- a/data/prompts/sot/unit_tree/task.txt +++ /dev/null @@ -1,6 +0,0 @@ -The input are: -action: The action that you construct the unit subtree from. - -You analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it. -The unit subtree should follow the regulated format. -You only return the unit subtree as a json object. \ No newline at end of file diff --git a/data/prompts/sot/unit_tree/template.txt b/data/prompts/sot/unit_tree/template.txt deleted file mode 100644 index 2182ebce..00000000 --- a/data/prompts/sot/unit_tree/template.txt +++ /dev/null @@ -1,4 +0,0 @@ -Resume from the input below. ---------------- -action: {action} ---------------- diff --git a/data/prompts/sot/unit_tree/unit_tree.txt b/data/prompts/sot/unit_tree/unit_tree.txt deleted file mode 100644 index 1c44921e..00000000 --- a/data/prompts/sot/unit_tree/unit_tree.txt +++ /dev/null @@ -1,48 +0,0 @@ -The behavior tree should be in json dictionary format. -When building a unit subtree, follow these rules: -1. Every node are described by a summary and a name: - - "summary": a short interpretive description of the node. - - "name": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are "selector", "sequence", "condition" ("target", "precondition"), "action". -2. The control flow nodes, namely "selector" and "sequence", can have "children", which is a list of children nodes. -3. The "condition" nodes that are children of "selectors" are targets. Those that are children of "sequences" are preconditions. -4. All the node should always have their corresponding actions/precidates in the domain knowledge. -5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state. -6. The basic structure template of a unit subtree is: - - A selector as the root node. - - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied. - - A sequence node as the last child of the selector node to achieve the target condition node. - - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions. - - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before. - - The action should have effects that can fulfill the target condition node in the selector. -Following is an example: -action: change_tool(left_hand, outwardgripper, defaultgripper) -unit subtree: -{ - "summary": "Selector to change the tool in the left hand from outwardgripper to defaultgripper", - "name": "selector: change_tool(left_hand, outwardgripper, defaultgripper)" - "children": [ - { - "summary": "The target is to make the left hand hold the default gripper", - "name": "target: hold(left_hand, defaultgripper)" - }, - { - "summary": "Sequence to change the tool in the left hand from outwardgripper to defaultgripper", - "name": "sequence: change_tool(left_hand, outwardgripper, defaultgripper)", - "children": [ - { - "summary": "A precondition is that the left hand is holding the outwardgripper", - "name": "precondition: hold(left_hand, outwardgripper)" - }, - { - "summary": "A precondition is that the outwardgripper is empty", - "name": "precondition: is_empty(outwardgripper)" - }, - { - "summary": "Take the action to change the tool in the left hand from outwardgripper to defaultgripper", - "name": "action: change_tool(left_hand, outwardgripper, defaultgripper)" - } - ] - } - ] -} - diff --git a/data/prompts/spsk_refine/prompt/dicard_refine_bt.txt b/data/prompts/spsk_refine/prompt/dicard_refine_bt.txt deleted file mode 100644 index d7233761..00000000 --- a/data/prompts/spsk_refine/prompt/dicard_refine_bt.txt +++ /dev/null @@ -1,140 +0,0 @@ -[user] -You should refine the behavior tree skeleton into a full behavior tree, which is in json dictionary format and have a structure like the following example: -""" -{ - "summary": "selector to load inward_claw to left hand", - "name": "selector: load_tool(left_hand, inward_claw)", - "identifier": 3, - "type_name": "selector", - "children": [ - { - "summary": "check if left_hand is holding inward_claw", - "name": "target: hold(left_hand, inward_claw)", - "identifier": 4, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "inward_claw", - "status": true, - } - ], - }, - { - - "summary": "sequence to load inward_claw to left hand", - "name": "sequence: load_tool(left_hand, inward_claw)", - "identifier": 5, - "type_name": "sequence", - "children": [ - { - "summary": "check if inward_claw is equippable", - "name": "precondition: is_equippable(inward_claw)", - "identifier": 6, - "type_name": "condition", - "conditions": [ - { - "object_name": "inward_claw", - "property_name": "is_equippable", - "property_value": null, - "status": true, - } - ], - }, - { - "summary": "check if left hand is free", - "name": "precondition: is_free(left_hand)", - "identifier": 7, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "is_free", - "property_value": null, - "status": true, - } - ], - }, - { - "summary": "equip inward_claw to left hand", - "name": "action: load_tool(left_hand, inward_claw)", - "identifier": 8, - "type_name": "action", - "effects": [ - { - "object_name": "left_hand", - "property_name": "is_free", - "property_value": null, - "status": false, - }, - { - "object_name": "inward_claw", - "property_name": "is_equippable", - "property_value": null, - "status": false, - }, - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "inward_claw", - "status": true, - }, - ], - }, - ], - }, - ], -} -""" -The dictionary that you return should be formatted as json dictionary. Follow these rules: -1. Every node should have four basic keys: - - "summary": a short description of the node. - - "name": the name of the node. The name should follow the definitions in the domain knowledge. - - "identifier": the unique id of the node. - - "type_name": can be: "selector", "sequence", "condition", "action". -2. The control flow node, "selector" and "sequence" nodes, can have "children", which is a list of children nodes. -3. The "condition" nodes that are children of "selectors" are "targets". Those that are children of "sequences" are preconditions. -4. The "condition" nodes should have the key "conditions", which is a list of conditions to be checked. Each condition should have the following keys: - - "object_name": the object to check. - - "property_name": the property/relation of the object to be checked. - - "property_value": this should be the other object in the relation. For property it should be null. - - "status": true if the property/relation is expected to be true, false otherwise. -5. The "action" nodes should have a key "effects", which is a list of the to-be-exerted effects of the action. Each effect should have the following keys: - - "object_name": the affected object. - - "property_name": the affected property/relation of the object. - - "property_value": this should be the other object if this is a relation. For property it should be null. - - "status": true if the effect is to add this property/relation. false if is to remove it. -6. All the subtrees always have their corresponding action in ROBOT_ACTION_LIST. They are used to represent the actions in the action sequence. Its basic structure is: - - A selector as the root node. - - (Usually one) target condition node as the first child of the selector to check if the target of the action has been fulfilled. - - A sequence node as the last child of the selector node to achieve the target. - - (Usually a number of) precondition nodes as the front children of the sequence node to check the preconditions for executing the action. - - One action node as the last child of the sequence node, whose preconditions are checked before and will immediate start the job when being ticked. - - The action will exert its defined effects after finishing the job successfully. Those effects should fulfilled all the target conditions in the selector node. - A brief example: -{ - selector: load_tool selector, - children: [ - { - target: check if inward_claw is equipped by left hand, - }, - { - sequence: load_tool, - children: [ - precondition: check if inward_claw is equippable, - precondition: check if left hand is free, - {action: equip inward_claw to left hand, effects: [left hand not free, inward_claw not equippable, left_hand holds inward_claw]}, - ], - }, - ], -} - -7. When performing a new action to fulfill a unsatisfied precondition, you can construct a subtree to replace the precondition and take that unfulfilled precondition as the target of the subtree. Then you construct the sequence of the subtree according to the new action you want to perform. -8. The behavior tree should be constructed based on the action sequence you planned and the action definitions in ROBOT_ACTION_LIST. -9. All of the control flow nodes, namely selectors and sequences, are without memory. A new behavior tree tick will always start from the root and traverse all the nodes even if they are ticked in the last tick. - -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/data/prompts/spsk_refine/prompt/discard_cot_bt.txt b/data/prompts/spsk_refine/prompt/discard_cot_bt.txt deleted file mode 100644 index e2fc475c..00000000 --- a/data/prompts/spsk_refine/prompt/discard_cot_bt.txt +++ /dev/null @@ -1,68 +0,0 @@ -[user] -The behavior tree should be in json dictionary format and have a structure like the following example: -""" -{ - "summary": "selector to load inward_claw to left hand", - "children": [ - { - "summary": "condition node to check if inward_claw is held by left hand", - }, - { - "summary": "sequence to load inward_claw to left hand", - "children": [ - { - "summary": "condition node to check if inward_claw is equippable", - }, - { - "summary": "condition node to check if left hand is free", - }, - { - "summary": "action node to equip inward_claw to left hand", - "effects": [ - {"summary": "left hand will be not free anymore",}, - {"summary": "inward_claw will be not equippable anymore",}, - {"summary": "left_hand will hold inward_claw",}, - ], - }, - ], - }, - ], -} -""" -Follow these rules: -1. Every node are described by a summary, which is a short interpretive description of the node. It should include the type of the node and the purpose of the node. -2. The control flow nodes, "selector" and "sequence", can have "children", which is a list of children nodes. -3. The "condition" nodes that are children of "selectors" are "targets". Those that are children of "sequences" are preconditions. -4. The "action" nodes should have a key "effects" with a list of dictionaries, which describes the to-be-exerted effects of the action. The effects are shortly described under the key "summary". -5. All the subtrees always have their corresponding action in ROBOT_ACTION_LIST. They are used to represent the actions in the action sequence. The recommended structure is: - - A selector as the root node. - - (Usually one) target condition node as the first child of the selector to check if the target of the action has been fulfilled. - - A sequence node as the last child of the selector node to achieve the target. - - (Usually a number of) precondition nodes as the front children of the sequence node to check the preconditions for executing the action. - - One action node as the last child of the sequence node, whose preconditions are checked before and will immediate start the job when being ticked. - - The action will exert its defined effects after finishing the job successfully. Those effects should fulfilled all the target conditions in the selector node. - A brief example is as follows: -{ - selector: load_tool selector, - children: [ - { - target: check if inward_claw is equipped by left hand, - }, - { - sequence: load_tool, - children: [ - precondition: check if inward_claw is equippable, - precondition: check if left hand is free, - {action: equip inward_claw to left hand, effects: [left hand not free, inward_claw not equippable, left_hand holds inward_claw]}, - ], - }, - ], -} - -6. When performing a new action to fulfill a unsatisfied precondition, you can construct a subtree to replace the precondition and take that unfulfilled precondition as the target of the subtree. Then you construct the sequence of the subtree according to the new action you want to perform. -7. All of the control flow nodes, namely selectors and sequences, are without memory. A new behavior tree tick will always start from the root and traverse all the nodes even if they are ticked in the last tick. - -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/data/prompts/spsk_refine/prompt/discard_cot_example.txt b/data/prompts/spsk_refine/prompt/discard_cot_example.txt deleted file mode 100644 index 3d8d6d5b..00000000 --- a/data/prompts/spsk_refine/prompt/discard_cot_example.txt +++ /dev/null @@ -1,127 +0,0 @@ -[user] -I will give you some examples of the input and the output you will generate. -Example 1: -""" -- Input: -(define (problem robot_assembly_problem-problem) - (:domain robot_assembly_problem-domain) - (:objects - parallel_box1 parallel_box2 inward_claw outward_claw no_tool - tool - gear1 gear2 gear3 shaft1 shaft2 base - part - left_hand - hand - ) - (:init (can_manipulate parallel_box1 gear1) (can_manipulate outward_claw gear2) (can_manipulate inward_claw gear3) (can_manipulate parallel_box2 shaft1) (can_manipulate no_tool shaft2) (can_screw_to leg1 seat) (can_screw_to leg2 seat) (can_insert_to back seat) (can_screw_to nut1 seat) (can_screw_to nut2 seat) (can_screw_to blub base) (can_place_to lamp blub) (can_insert_to shaft1 base) (can_insert_to shaft2 base) (can_insert_to gear3 shaft2) (can_insert_to gear2 base) (can_insert_to gear1 shaft1) (is_inserted_to shaft1 base) (is_free left_hand) (is_free parallel_box1) (is_free parallel_box2) (is_free inward_claw) (is_free outward_claw) (is_free no_tool) (is_equippable parallel_box1) (is_equippable parallel_box2) (is_equippable inward_claw) (is_equippable outward_claw) (is_equippable no_tool)) - (:goal (and (is_inserted_to gear1 shaft1))) -) -- Output: -``` -{"task_plan": { - "action_sequence": [ - "load_tool(left_hand, parallel_box1)", - "pick_up(left_hand, parallel_box1, gear1)", - "insert(left_hand, parallel_box1, gear1, shaft1)" - ], - "action_instructions": [ - "equip the parallel_box1 on the left hand", - "pick up the gear1 with the parallel_box1 in the left hand", - "insert the gear1 into the shaft1 using the parallel_box1 in the left hand" - ], - "object_name": ["left_hand", "gear1", "parallel_box1", "shaft1"], - "behavior_tree": { - "summary": "selector to insert gear1 into shaft1", - "children": [ - { - "summary": "condition node to check if gear1 is inserted to shaft1", - }, - { - "summary": "sequence to insert gear1 into shaft1", - "children": [ - { - "summary": "selector to load_tool - "children": [ - { - "summary": "condition node to check if parallel_box1 is hold", - }, - { - "summary": "sequence to load_tool", - "children": [ - { - "summary": "condition node to check if parallel_box1 is equippable", - }, - { - "summary": "condition node to check if left hand is free", - }, - { - "summary": "action node to equip parallel_box1 to left hand", - "effects": [ - {"summary": "left_hand will be not free",}, - {"summary": "parallel_box1 will be not equippable",}, - {"summary": "left_hand will hold parallel_box1",}, - ], - }, - ], - }, - ], - }, - { - "summary": "selector to pick_up", - "children": [ - { - "summary": "condition node to check if parallel_box1 holds gear1", - }, - { - "summary": "sequence to pick_up", - "children": [ - { - "summary": "condition node to check if parallel_box1 is free", - }, - { - "summary": "condition node to check if left hand holds parallel_box1", - }, - { - "summary": "action node to pick up gear1 using parallel_box1", - "effects": [ - {"summary": "parallel_box1 will hold gear1",}, - {"summary": "parallel_box1 will be not free",}, - ], - }, - ], - }, - ], - }, - { - "summary": "condition node to check if gear1 can be inserted to shaft1", - }, - { - "summary": "action node to insert gear1 to shaft1", - "effects": [ - {"summary": "parallel_box1 will be not holding gear1",}, - {"summary": "parallel_box1 will be free",}, - {"summary": "gear1 will be inserted to shaft1",}, - ], - }, - ], - }, - ], -} -}, -"initial_state": {"objects": [ - {"name": "gear1", "properties": []}, - {"name": "shaft1", "properties": []}, - {"name": "parallel_box1", "properties": ["is_equippable", "is_free"]}, - {"name": "left_hand", "properties": ["is_free"]}, - ], - "constraints": [ - {"source": "gear1", "name": "can_insert_to", "target": "shaft1"}, - {"source": "parallel_box1", "name": "can_manipulate", "target": "gear1"}, - ], - "relations": [], -}, -"instruction_summary": "insert gear1 into shaft1", -"question":""} -``` -""" -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/data/prompts/spsk_refine/prompt/discard_refine_refine.txt b/data/prompts/spsk_refine/prompt/discard_refine_refine.txt deleted file mode 100644 index 777e8f26..00000000 --- a/data/prompts/spsk_refine/prompt/discard_refine_refine.txt +++ /dev/null @@ -1,247 +0,0 @@ -[user] -You should refine the behavior tree skeleton into a full behavior tree in json dictionary format. - -For each node, you should: -1. add an unique integer "identifier". -2. add its "type_name" accordingly. (precondition and target are both "condition") - -For control flow nodes, namely selectors and sequences, an example is shown as follows: -the skeleton: -""" -{ - "summary": "selector to load no_tool to hand", - "name": "selector: load_tool", - "children": [...], -} - -""" -after refinement: -""" -{ - "summary": "selector to load no_tool to hand", - "name": "selector: load_tool(left_hand, no_tool)", - "identifier": 3, - "type_name": "selector", - "children": [...], -} -""" - -For condition nodes, you should: -1. refine and add the key "conditions", which is a list of conditions to be checked. Each condition should have the following keys: - - "object_name": the object to check. - - "property_name": the property/relation of the object to be checked. - - "property_value": this should be the other object in the relation. For property it should be null. - - "status": true if the property/relation is expected to be true, false otherwise. -An example is shown below: -the skeleton: -""" -{ - "summary": "check if left_hand is holding parallel_box1", - "name": "target: hold(left_hand, parallel_box1)", -} -""" -after refinement: -""" -{ - "summary": "check if left_hand is holding parallel_box1", - "name": "target: hold(left_hand, parallel_box1)", - "identifier": 4, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "parallel_box1", - "status": true, - } - ], -}, -""" - -For action nodes, you should: -1. refine and replace the summaries in the key "effects", which is a list of the to-be-exerted effects of the action. Each effect should have the following keys: - - "object_name": the affected object. - - "property_name": the affected property/relation of the object. - - "property_value": this should be the other object if this is a relation. For property it should be null. - - "status": true if the effect is to add this property/relation. false if is to remove it. -An example is shown below: -the skeleton: -""" -{ - "summary": "left hand equips tool1", - "name": "action: load_tool(left_hand, tool1)", - "effects": [ - {"summary": "left hand will be not free anymore",}, - {"summary": "tool1 will be not equippable anymore",}, - {"summary": "left_hand will hold tool1",}, - ], -}, -""" -after refinement: -""" -{ - "summary": "left hand equips tool1", - "name": "action: load_tool(left_hand, tool1)", - "identifier": 8, - "type_name": "action", - "effects": [ - { - "object_name": "left_hand", - "property_name": "is_free", - "property_value": null, - "status": false, - }, - { - "object_name": "tool1", - "property_name": "is_equippable", - "property_value": null, - "status": false, - }, - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "tool1", - "status": true, - }, - ], -}, -""" - -""" -{ - "summary": "selector to load inward_claw to left hand", - "name": "selector: load_tool(left_hand, inward_claw)", - "identifier": 3, - "type_name": "selector", - "children": [ - { - "summary": "check if left_hand is holding inward_claw", - "name": "target: hold(left_hand, inward_claw)", - "identifier": 4, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "inward_claw", - "status": true, - } - ], - }, - { - - "summary": "sequence to load inward_claw to left hand", - "name": "sequence: load_tool(left_hand, inward_claw)", - "identifier": 5, - "type_name": "sequence", - "children": [ - { - "summary": "check if inward_claw is equippable", - "name": "precondition: is_equippable(inward_claw)", - "identifier": 6, - "type_name": "condition", - "conditions": [ - { - "object_name": "inward_claw", - "property_name": "is_equippable", - "property_value": null, - "status": true, - } - ], - }, - { - "summary": "check if left hand is free", - "name": "precondition: is_free(left_hand)", - "identifier": 7, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "is_free", - "property_value": null, - "status": true, - } - ], - }, - { - "summary": "equip inward_claw to left hand", - "name": "action: load_tool(left_hand, inward_claw)", - "identifier": 8, - "type_name": "action", - "effects": [ - { - "object_name": "left_hand", - "property_name": "is_free", - "property_value": null, - "status": false, - }, - { - "object_name": "inward_claw", - "property_name": "is_equippable", - "property_value": null, - "status": false, - }, - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "inward_claw", - "status": true, - }, - ], - }, - ], - }, - ], -} -""" -The dictionary that you return should be formatted as json dictionary. Follow these rules: -1. Every node should have four basic keys: - - "summary": a short description of the node. - - "name": the name of the node. The name should follow the definitions in the domain knowledge. - - "identifier": the unique id of the node. - - "type_name": can be: "selector", "sequence", "condition", "action". -2. The control flow node, "selector" and "sequence" nodes, can have "children", which is a list of children nodes. -3. The "condition" nodes that are children of "selectors" are "targets". Those that are children of "sequences" are preconditions. -4. The "condition" nodes should have the key "conditions", which is a list of conditions to be checked. Each condition should have the following keys: - - "object_name": the object to check. - - "property_name": the property/relation of the object to be checked. - - "property_value": this should be the other object in the relation. For property it should be null. - - "status": true if the property/relation is expected to be true, false otherwise. -5. The "action" nodes should have a key "effects", which is a list of the to-be-exerted effects of the action. Each effect should have the following keys: - - "object_name": the affected object. - - "property_name": the affected property/relation of the object. - - "property_value": this should be the other object if this is a relation. For property it should be null. - - "status": true if the effect is to add this property/relation. false if is to remove it. -6. All the subtrees always have their corresponding action in ROBOT_ACTION_LIST. They are used to represent the actions in the action sequence. Its basic structure is: - - A selector as the root node. - - (Usually one) target condition node as the first child of the selector to check if the target of the action has been fulfilled. - - A sequence node as the last child of the selector node to achieve the target. - - (Usually a number of) precondition nodes as the front children of the sequence node to check the preconditions for executing the action. - - One action node as the last child of the sequence node, whose preconditions are checked before and will immediate start the job when being ticked. - - The action will exert its defined effects after finishing the job successfully. Those effects should fulfilled all the target conditions in the selector node. - A brief example: -{ - selector: load_tool selector, - children: [ - { - target: check if inward_claw is equipped by left hand, - }, - { - sequence: load_tool, - children: [ - precondition: check if inward_claw is equippable, - precondition: check if left hand is free, - {action: equip inward_claw to left hand, effects: [left hand not free, inward_claw not equippable, left_hand holds inward_claw]}, - ], - }, - ], -} - -7. When performing a new action to fulfill a unsatisfied precondition, you can construct a subtree to replace the precondition and take that unfulfilled precondition as the target of the subtree. Then you construct the sequence of the subtree according to the new action you want to perform. -8. The behavior tree should be constructed based on the action sequence you planned and the action definitions in ROBOT_ACTION_LIST. -9. All of the control flow nodes, namely selectors and sequences, are without memory. A new behavior tree tick will always start from the root and traverse all the nodes even if they are ticked in the last tick. - -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/data/prompts/spsk_refine/prompt/refine_action.txt b/data/prompts/spsk_refine/prompt/refine_action.txt deleted file mode 100644 index 06e74ce2..00000000 --- a/data/prompts/spsk_refine/prompt/refine_action.txt +++ /dev/null @@ -1,54 +0,0 @@ -[user] -For action nodes, you should: -1. refine and replace the summaries in the key "effects", which is a list of the to-be-exerted effects of the action. Each effect should have the following keys: - - "object_name": the affected object. - - "property_name": the affected property/relation of the object. - - "property_value": this should be the other object if this is a relation. For property it should be null. - - "status": true if the effect is to add this property/relation. false if is to remove it. -An example is shown below: -the skeleton: -""" -{ - "summary": "left hand equips tool1", - "name": "action: load_tool(left_hand, tool1)", - "effects": [ - {"summary": "left hand will be not free anymore",}, - {"summary": "tool1 will be not equippable anymore",}, - {"summary": "left_hand will hold tool1",}, - ], -}, -""" -after refinement: -""" -{ - "summary": "left hand equips tool1", - "name": "action: load_tool(left_hand, tool1)", - "identifier": 8, - "type_name": "action", - "effects": [ - { - "object_name": "left_hand", - "property_name": "is_free", - "property_value": null, - "status": false, - }, - { - "object_name": "tool1", - "property_name": "is_equippable", - "property_value": null, - "status": false, - }, - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "tool1", - "status": true, - }, - ], -}, -""" - -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/data/prompts/spsk_refine/prompt/refine_chain.txt b/data/prompts/spsk_refine/prompt/refine_chain.txt deleted file mode 100644 index 8ab56a97..00000000 --- a/data/prompts/spsk_refine/prompt/refine_chain.txt +++ /dev/null @@ -1,15 +0,0 @@ -[user] -Follow this workflow to solve the task: - -1. Read the problem description. Parse and generate the "initial_state" in the output dictionary. -2. Interpret the goal of the problem as goal_state and write into the "instruction_summary" in the output dictionary. -3. Try to solve the problem, achieve the goal_state from the initial_state based on the domain knowledge provided. Generate a action sequence with the actions defined in and write it into dictionary["task_plan"]["action_sequence"] of the output. -4. If you cannot find a plan to solve the task, ask question with nature language in the "question" of the output dictionary. -5. Explain the actions in the action sequence and write the output dictionary["task_plan"]["action_instructions"]. -6. Collect the objects in the world that are effected by the action sequence, write into dictionary["task_plan"]["manipulated_objects"]. -7. Generate the behavior tree based on the action sequence and write into dictionary["task_plan"]["behavior_tree"]. - -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/data/prompts/spsk_refine/prompt/refine_condition.txt b/data/prompts/spsk_refine/prompt/refine_condition.txt deleted file mode 100644 index ff0f396c..00000000 --- a/data/prompts/spsk_refine/prompt/refine_condition.txt +++ /dev/null @@ -1,37 +0,0 @@ -[user] -For condition nodes, you should: -1. refine and add the key "conditions", which is a list of conditions to be checked. Each condition should have the following keys: - - "object_name": the object to check. - - "property_name": the property/relation of the object to be checked. - - "property_value": this should be the other object in the relation. For property it should be null. - - "status": true if the property/relation is expected to be true, false otherwise. -An example is shown below: -the skeleton: -""" -{ - "summary": "check if left_hand is holding parallel_box1", - "name": "target: hold(left_hand, parallel_box1)", -} -""" -after refinement: -""" -{ - "summary": "check if left_hand is holding parallel_box1", - "name": "target: hold(left_hand, parallel_box1)", - "identifier": 4, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "parallel_box1", - "status": true, - } - ], -}, -""" - -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/data/prompts/spsk_refine/prompt/refine_controlflow.txt b/data/prompts/spsk_refine/prompt/refine_controlflow.txt deleted file mode 100644 index 28828ac2..00000000 --- a/data/prompts/spsk_refine/prompt/refine_controlflow.txt +++ /dev/null @@ -1,26 +0,0 @@ -[user] -For control flow nodes, namely selectors and sequences, an example is shown as follows: -the skeleton: -""" -{ - "summary": "selector to load no_tool to hand", - "name": "selector: load_tool", - "children": [...], -} - -""" -after refinement: -""" -{ - "summary": "selector to load no_tool to hand", - "name": "selector: load_tool(left_hand, no_tool)", - "identifier": 3, - "type_name": "selector", - "children": [...], -} -""" - -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/data/prompts/spsk_refine/prompt/refine_domain.txt b/data/prompts/spsk_refine/prompt/refine_domain.txt deleted file mode 100644 index bf3c7859..00000000 --- a/data/prompts/spsk_refine/prompt/refine_domain.txt +++ /dev/null @@ -1,89 +0,0 @@ -[user] -Following is the domain knowledge you need in pddl domain format, which includes: -- the object types in the world, -- the predicates for describing the states of the world, -- the actions that the robot can apply, with their precondtions and effects defined. -""" -(define (domain robot_assembly_problem-domain) - (:requirements :strips :typing) - (:types - thing - object - tool part hand - thing - ) - (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part)) - (:action pick_up - :parameters ( ?hand - hand ?tool - tool ?part - part) - :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part)) - :effect (and (hold ?tool ?part) (not (is_free ?tool)))) - (:action put_down - :parameters ( ?hand - hand ?tool - tool ?part - part) - :precondition (and (hold ?tool ?part) (hold ?hand ?tool)) - :effect (and (not (hold ?hand ?part)) (is_free ?tool))) - (:action place - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2))) - (:action detach - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2)))) - (:action insert - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2))) - (:action pull - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2)))) - (:action screw - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2))) - (:action unscrew - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2)))) - (:action load_tool - :parameters ( ?hand - hand ?tool - tool) - :precondition (and (is_equippable ?tool) (is_free ?hand)) - :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool))) - (:action unload_tool - :parameters ( ?hand - hand ?tool - tool) - :precondition (and (hold ?hand ?tool) (is_free ?tool)) - :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool)))) -) -""" -The precidates are explained as follows: -""" -"PREDICATE_LIST" -- is_free( or ): or is free and can hold something. (default: true) -- is_equippable(): is ready to be eqquipped by a robot hand. (default: true) -- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false) -- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false) -- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false) -- can_place_to(, ): can be placed into (placement assembly constraint) (default: false) -- hold(, ): is holding (default: false) -- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false) -- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false) -- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false) -""" - -The actions are explained as follows: -""" -"ROBOT_ACTION_LIST" -- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something. -- put_down(, , ): use to put down . As preconditions, should be holding and should be holding . -- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding . -- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into . -- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding . -- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into . -- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding . -- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into . -- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free. -- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free. -""" -the items in <> indicate the type of the instances needed for the predicates and actions. -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. diff --git a/data/prompts/spsk_refine/prompt/refine_help.txt b/data/prompts/spsk_refine/prompt/refine_help.txt deleted file mode 100644 index c81cfca9..00000000 --- a/data/prompts/spsk_refine/prompt/refine_help.txt +++ /dev/null @@ -1,11 +0,0 @@ -[user] -You should refine the behavior tree skeleton into a full behavior tree in json dictionary format. - -1. Refer to "initial_state" of input to find the necessary objects, properties and relationships. -2. Refer to the domain knowledge to understand the actions (corresponds to "ROBOT_ACTION_LIST") and conditions (corresponds to "PREDICATE_LIST"). -3. The name of the node (if it has) also provides information for its refinement. - -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/data/prompts/spsk_refine/prompt/refine_input_format.txt b/data/prompts/spsk_refine/prompt/refine_input_format.txt deleted file mode 100644 index 27a2fdef..00000000 --- a/data/prompts/spsk_refine/prompt/refine_input_format.txt +++ /dev/null @@ -1,127 +0,0 @@ -[user] -The input you get and the output you return should be in json dictionary format. -The dictionary contains the following keys: -""" -- dictionary["task_plan"]: A dictionary containing the task plan. -- dictionary["initial_state"]: The initial world state before executing the plan. -- dictionary["instruction_summary"]: A brief summary of the task plan. -- dictionary["question"]: Question for more knowledge that are unknown yet but necessary for making the plan. Normally empty. -""" -Four keys exist in dictionary["task_plan"]. -""" -- dictionary["task_plan"]["action_sequence"]: Contains a list of robot actions. -- dictionary["task_plan"]["action_instructions"]: contains a list of instructions to explain the actions in dictionary["task_plan"]["action_sequence"]. -- dictionary["task_plan"]["manipulated_objects"]: The list of the objects that are related to the current task. -- dictionary["task_plan"]["behavior_tree"]: contains a dictionary of the behavior tree skeleton constructed according to the dictionary ["task_plan"]["action_sequence"] -""" -The following is an input example: -``` -{"task_plan": { - "action_sequence": [ - "load_tool(left_hand, parallel_box1)", - "pick_up(left_hand, parallel_box1, gear1)", - "insert(left_hand, parallel_box1, gear1, shaft1)" - ], - "action_instructions": [ - "equip the parallel_box1 on the left hand", - "pick up the gear1 with the parallel_box1 in the left hand", - "insert the gear1 into the shaft1 using the parallel_box1 in the left hand" - ], - "object_name": ["left_hand", "gear1", "parallel_box1", "shaft1"], - "behavior_tree": { - "summary": "selector to insert gear1 into shaft1", - "children": [ - { - "summary": "condition node to check if gear1 is inserted to shaft1", - }, - { - "summary": "sequence to insert gear1 into shaft1", - "children": [ - { - "summary": "selector to load_tool - "children": [ - { - "summary": "condition node to check if parallel_box1 is hold", - }, - { - "summary": "sequence to load_tool", - "children": [ - { - "summary": "condition node to check if parallel_box1 is equippable", - }, - { - "summary": "condition node to check if left hand is free", - }, - { - "summary": "action node to equip parallel_box1 to left hand", - "effects": [ - {"summary": "left_hand will be not free",}, - {"summary": "parallel_box1 will be not equippable",}, - {"summary": "left_hand will hold parallel_box1",}, - ], - }, - ], - }, - ], - }, - { - "summary": "selector to pick_up", - "children": [ - { - "summary": "condition node to check if parallel_box1 holds gear1", - }, - { - "summary": "sequence to pick_up", - "children": [ - { - "summary": "condition node to check if parallel_box1 is free", - }, - { - "summary": "condition node to check if left hand holds parallel_box1", - }, - { - "summary": "action node to pick up gear1 using parallel_box1", - "effects": [ - {"summary": "parallel_box1 will hold gear1",}, - {"summary": "parallel_box1 will be not free",}, - ], - }, - ], - }, - ], - }, - { - "summary": "condition node to check if gear1 can be inserted to shaft1", - }, - { - "summary": "action node to insert gear1 to shaft1", - "effects": [ - {"summary": "parallel_box1 will be not holding gear1",}, - {"summary": "parallel_box1 will be free",}, - {"summary": "gear1 will be inserted to shaft1",}, - ], - }, - ], - }, - ], -} -}, -"initial_state": {"objects": [ - {"name": "gear1", "properties": []}, - {"name": "shaft1", "properties": []}, - {"name": "parallel_box1", "properties": ["is_equippable", "is_free"]}, - {"name": "left_hand", "properties": ["is_free"]}, - ], - "constraints": [ - {"source": "gear1", "name": "can_insert_to", "target": "shaft1"}, - {"source": "parallel_box1", "name": "can_manipulate", "target": "gear1"}, - ], - "relations": [], -}, -"instruction_summary": "insert gear1 into shaft1", -"question":""} -``` -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/data/prompts/spsk_refine/prompt/refine_output_format.txt b/data/prompts/spsk_refine/prompt/refine_output_format.txt deleted file mode 100644 index d40e17ea..00000000 --- a/data/prompts/spsk_refine/prompt/refine_output_format.txt +++ /dev/null @@ -1,19 +0,0 @@ -[user] -The output dictionary has four keys. -""" -- dictionary["task_plan"] -- dictionary["initial_state"] -- dictionary["instruction_summary"] -- dictionary["question"] -""" -Four keys exist in dictionary["task_plan"]. -""" -- dictionary["task_plan"]["action_sequence"] -- dictionary["task_plan"]["action_instructions"] -- dictionary["task_plan"]["manipulated_objects"] -- dictionary["task_plan"]["behavior_tree"] -""" -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/data/prompts/spsk_refine/prompt/refine_role.txt b/data/prompts/spsk_refine/prompt/refine_role.txt deleted file mode 100644 index de12cb04..00000000 --- a/data/prompts/spsk_refine/prompt/refine_role.txt +++ /dev/null @@ -1,5 +0,0 @@ -[user] -You are an excellent task planner for robotic assembly tasks. Given the world state and a rough plan, you break it down and refine the plan into a well formatted and executable behavior tree. -Please do not start to work until I say "Start working." Instead, simply output the message "Waiting for next input." Understood? -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/data/prompts/spsk_refine/prompt/refine_state.txt b/data/prompts/spsk_refine/prompt/refine_state.txt deleted file mode 100644 index c6ce4d9d..00000000 --- a/data/prompts/spsk_refine/prompt/refine_state.txt +++ /dev/null @@ -1,27 +0,0 @@ -[user] -The initial state in the input is explained as follows: - - "objects": the objects and their properties (as lists of strings) in the world. The properties can be changed by robot actions. - - "constraints": the constraint relations that are enforced and should not be changed. They are knowledge about the world or some constraints set by the user. - - "relations": the relations in the world. They can be changed by robot actions. -following is another example: -""" -{ - "objects": [ - {"name": "leg1", "properties": []}, # leg1 is an object with no properties - {"name": "seat", "properties": []}, # seat is an object with no properties - {"name": "hand", "properties": ["is_free"]}, # hand is an object with a property "is_free" - ], - "constraints": [ - {"source": "leg1", "name": "can_screw_to", "target": "seat"}, # means leg1 can be screwed to seat - ], - "relations": [ - {"source": "leg1", "name": "is_screwed_to", "target": "seat"}, # means leg1 is screwed to seat - ], -} -""" -Only concentrate on the objects, constraints and relations that are related to the current task. - -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/data/prompts/spsk_refine/query/dup_query.txt b/data/prompts/spsk_refine/query/dup_query.txt deleted file mode 100644 index c1a7955e..00000000 --- a/data/prompts/spsk_refine/query/dup_query.txt +++ /dev/null @@ -1,16 +0,0 @@ -Start working. Resume from the input below. -""" -{"input": [PROBLEM]} -""" -The dictionary that you return should be formatted as json file. Follow these rules: -1. Always equip the proper tools before performing specific object manipulaion. -2. Make sure that each element of the ["action_instructions"] explains corresponding element of the ["action_sequence"]. Refer to the "ROBOT ACTION LIST" to understand the elements of ["action_sequence"]. -3, The length of the ["action_instructions"] list and the ["action_sequence"] list should be the same. -4. Never left ',' at the end of the list. Use json boolean values, "true" or "false", instead of "True" or "False". -5. Keep track of all items listed in the "objects" section of the "initial_state" field. -6. Use the "PREDICATE_LIST" for the initial states (:init) in the problem to generate ["initial_state"]. -7. All keys of the dictionary should be double-quoted. -8. Insert ``` at the beginning and the end of the dictionary to separate it from the rest of your response. Don't write any additional things. -9. Make sure that you output a consistent manipultation as a single arm robot. For example, grasping an object should not occur in successive steps. -10. When generating the behavior tree, make sure the tree has the asked structure. -Adhere to the output format I defined above. Follow the rules above. Think step by step. \ No newline at end of file diff --git a/data/prompts/spsk_refine/query/query.txt b/data/prompts/spsk_refine/query/query.txt deleted file mode 100644 index 7abaa278..00000000 --- a/data/prompts/spsk_refine/query/query.txt +++ /dev/null @@ -1,14 +0,0 @@ -Start working. Resume from the input below. -""" -{"input": [PROBLEM]} -""" -The dictionary that you return should be formatted as json file. Follow these rules: -1. Only refine the skeleton behavior tree in the "behavior_tree" field. Refine the tree according to rules provided before. Replace the old behavior tree with the refined behavior tree. -2. Refer to the domain knowledge, namely "ROBOT_ACTION_LIST" and "PREDICATE_LIST", to understand and refine the skeleton behavior tree. -3. Do not change other fields of the dictionary and return the new dictionary as json file. -4. Never left ',' at the end of the list. -5. Use json boolean values, "true" or "false", instead of "True" or "False". -6. All keys of the dictionary should be double-quoted. -7. Insert ``` at the beginning and the end of the dictionary to separate it from the rest of your response. Don't write any additional things. -8. When generating the behavior tree, make sure the tree has the asked structure. -Adhere to the output format I defined above. Follow the rules above. Think step by step. \ No newline at end of file diff --git a/data/prompts/spsk_refine/system/system.txt b/data/prompts/spsk_refine/system/system.txt deleted file mode 100644 index 5bc3c5a6..00000000 --- a/data/prompts/spsk_refine/system/system.txt +++ /dev/null @@ -1 +0,0 @@ -You are an excellent robot behavior tree expert in robotic assembly tasks. Given the knowledge you need, the world state and a rough plan, you break it down and refine the task plan to make it well formatted and ready to be executed. \ No newline at end of file diff --git a/data/prompts/state_est/domain.txt b/data/prompts/state_est/domain.txt deleted file mode 100644 index 7eec5ea4..00000000 --- a/data/prompts/state_est/domain.txt +++ /dev/null @@ -1,84 +0,0 @@ -Following is the domain knowledge you need in pddl domain format, which includes: -- the object types in the world, -- the predicates for describing the states of the world, -- the actions that the robot can apply, with their precondtions and effects defined. -""" -(define (domain robot_assembly_problem-domain) - (:requirements :strips :typing) - (:types - thing - object - tool part hand - thing - ) - (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part)) - (:action pick_up - :parameters ( ?hand - hand ?tool - tool ?part - part) - :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part)) - :effect (and (hold ?tool ?part) (not (is_free ?tool)))) - (:action put_down - :parameters ( ?hand - hand ?tool - tool ?part - part) - :precondition (and (hold ?tool ?part) (hold ?hand ?tool)) - :effect (and (not (hold ?hand ?part)) (is_free ?tool))) - (:action place - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2))) - (:action detach - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2)))) - (:action insert - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2))) - (:action pull - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2)))) - (:action screw - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2))) - (:action unscrew - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2)))) - (:action load_tool - :parameters ( ?hand - hand ?tool - tool) - :precondition (and (is_equippable ?tool) (is_free ?hand)) - :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool))) - (:action unload_tool - :parameters ( ?hand - hand ?tool - tool) - :precondition (and (hold ?hand ?tool) (is_free ?tool)) - :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool)))) -) -""" -The precidates are explained as follows: -""" -"PREDICATE_LIST" -- is_free( or ): or is free and can hold something. (default: true) -- is_equippable(): is ready to be eqquipped by a robot hand. (default: true) -- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false) -- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false) -- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false) -- can_place_to(, ): can be placed into (placement assembly constraint) (default: false) -- hold(, ): is holding (default: false) -- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false) -- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false) -- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false) -""" - -The actions are explained as follows: -""" -"ROBOT_ACTION_LIST" -- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something. -- put_down(, , ): use to put down . As preconditions, should be holding and should be holding . -- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding . -- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into . -- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding . -- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into . -- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding . -- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into . -- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free. -- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free. -""" -the items in <> indicate the type of the instances needed for the predicates and actions. \ No newline at end of file diff --git a/data/prompts/state_est/new_domain_nl.txt b/data/prompts/state_est/new_domain_nl.txt deleted file mode 100644 index b67bcb08..00000000 --- a/data/prompts/state_est/new_domain_nl.txt +++ /dev/null @@ -1,41 +0,0 @@ -Following is the domain knowledge you need, which includes: -- the object types in the world, -- the predicates for describing the states of the world, -- the actions that the robot can apply, with their precondtions and effects defined. - -The object types are explained as follows: -""" -thing: The base class for all objects in the world. Its subclasses are: hand, tool, part. -hand: The robot hand. A empty can only hold one at a time. -tool: The tool to manipulate . A empty can hold one at a time. -part: The part to be manipulated. -""" - -The precidates are explained as follows: -""" -"PREDICATE_LIST" -- is_empty(): is empty and can hold something. -- can_manipulate(, ): can be used to manipulate (manipulation constraint). This regulates the compatibility between and . -- can_insert_to(, ): can be inserted into (insertion assembly constraint) -- can_screw_to(, ): can be screwed into (screw assembly constraint) -- can_place_to(, ): can be placed into (placement assembly constraint) -- hold(, ): is holding . It can be holding or holding . -- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) -- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) -- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) -""" - -The actions are explained as follows: -""" -"ROBOT_ACTION_LIST" -- pick_up(, , ): use to pick up . As preconditions, should be holding and should be empty in order to hold the . The will be held by the as a result of this action. -- put_down(, , ): use to put down . As preconditions, should be holding and should be holding . The will be empty and will not hold the anymore as a result of this action. -- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding . -- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be placed into . -- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding . -- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be inserted into . -- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding . -- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be screwed into . -- change_tool(, , ): change the equipped tool from to in order to manipulate specific parts. As preconditions, should be holding and should be empty. -""" -the items in <> indicate the type of the instances needed for the predicates and actions. \ No newline at end of file diff --git a/data/prompts/state_est/new_example.txt b/data/prompts/state_est/new_example.txt deleted file mode 100644 index 6fb313a3..00000000 --- a/data/prompts/state_est/new_example.txt +++ /dev/null @@ -1,47 +0,0 @@ -Following is an input/output example: -input: -''' -start_world_state: -{ -"objects": [ - {"name": "outwardgripper", "properties": ["is_empty"]}, - {"name": "shaft1", "properties": []}, - {"name": "hole1", "properties": []}, - {"name": "defaultgripper", "properties": ["is_empty"]}, - {"name": "left_hand", "properties": []} -], -"constraints": [ - {"source": "outwardgripper", "name": "can_manipulate", "target": "shaft1"}, - {"source": "shaft1", "name": "can_insert_to", "target": "hole1"} -], -"relations": [ - {"source": "left_hand", "name": "hold", "target": "defaultgripper"} -] -} -action_plan: -[ - "change_tool(left_hand, defaultgripper, outwardgripper)", - "pick_up(left_hand, outwardgripper, shaft1)", - "insert(left_hand, outwardgripper, gear1, hole1)", -] -''' -output: -{ - "estimated_world_state": { - "objects": [ - {"name": "outwardgripper", "properties": ["is_empty"]}, - {"name": "shaft1", "properties": []}, - {"name": "hole1", "properties": []}, - {"name": "defaultgripper", "properties": ["is_empty"]}, - {"name": "left_hand", "properties": [""]} - ], - "constraints": [ - {"source": "outwardgripper", "name": "can_manipulate", "target": "shaft1"}, - {"source": "shaft1", "name": "can_insert_to", "target": "hole1"} - ], - "relations": [ - {"source": "left_hand", "name": "hold", "target": "outwardgripper"} - {"source": "shaft1", "name": "is_inserted_to", "target": "hole1"} - ] - } -} \ No newline at end of file diff --git a/data/prompts/state_est/output_format.txt b/data/prompts/state_est/output_format.txt deleted file mode 100644 index ccf11c35..00000000 --- a/data/prompts/state_est/output_format.txt +++ /dev/null @@ -1,2 +0,0 @@ -The output json object has the following keys. -- "estimated_world_state": The world state after executing the plan. diff --git a/data/prompts/state_est/state.txt b/data/prompts/state_est/state.txt deleted file mode 100644 index 15dd521a..00000000 --- a/data/prompts/state_est/state.txt +++ /dev/null @@ -1,21 +0,0 @@ -World states should be in json dictionary format. Example: -""" -{ - "objects": [ - {"name": "leg1", "properties": []}, # leg1 is an object with no properties - {"name": "seat", "properties": []}, # seat is an object with no properties - {"name": "tool1", "properties": ["is_empty"]} # hand is an object with a property "is_free" - ], - "constraints": [ - {"source": "leg1", "name": "can_screw_to", "target": "seat"} # means leg1 can be screwed to seat - ], - "relations": [ - {"source": "leg1", "name": "is_screwed_to", "target": "seat"} # means leg1 is screwed to seat - ] -} -""" -It contains three lists: - - "objects": the objects in the world and their properties. The properties can be changed by robot actions. - - "constraints": the constraint relations that are regulated and can not be changed. - - "relations": the relations in the world. They can be changed by robot actions. -The properties, constraints and relations should always align with the [PREDICATE_LIST] in the domain knowledge. \ No newline at end of file diff --git a/data/prompts/state_est/system.txt b/data/prompts/state_est/system.txt deleted file mode 100644 index 61593243..00000000 --- a/data/prompts/state_est/system.txt +++ /dev/null @@ -1 +0,0 @@ -You are an expert in estimating world state. \ No newline at end of file diff --git a/data/prompts/state_est/task.txt b/data/prompts/state_est/task.txt deleted file mode 100644 index 95ad05ad..00000000 --- a/data/prompts/state_est/task.txt +++ /dev/null @@ -1,6 +0,0 @@ -The input are: -start_world_state: the world state when the behavior tree started. -action_plan: a sequence of action to be performed. - -You estimate the world state after executing the task plan. -You return the estimated world state in a json object. diff --git a/data/prompts/state_est/template.txt b/data/prompts/state_est/template.txt deleted file mode 100644 index cf62bdac..00000000 --- a/data/prompts/state_est/template.txt +++ /dev/null @@ -1,3 +0,0 @@ -Now make a plan with the following information: -start_world_state: {start_world_state} -action_plan: {action_plan} \ No newline at end of file diff --git a/data/prompts/super_skeleton/prompt/backup b/data/prompts/super_skeleton/prompt/backup deleted file mode 100644 index c4a0f6f6..00000000 --- a/data/prompts/super_skeleton/prompt/backup +++ /dev/null @@ -1,43 +0,0 @@ - -There are also some counterexamples for you to refer to: -1. One sequence should never have multiple action nodes as its children like the behavior tree below. -{ - "summary": "selector to get the apple", - "name": "Get the apple", - "type_name": "selector", - "children": [ - { - "summary": "check if the apple is in hand", - "name": "target: in_hand(apple)", - "type_name": "condition", - }, - { - "summary": "sequence to get the apple", - "name": "sequence: get apple", - "type_name": "sequence", - "children": [ - { - "summary": "check if the apple is on the table", - "name": "precondition: on(apple, table)", - "type_name": "condition", - }, - { - "summary": "go to the table", - "name": "action: go_to(table)", - "type_name": "action", - }, - { - "summary": "check if the hand is free", - "name": "precondition: is_free(hand)", - "type_name": "condition", - }, - { - "summary": "pick up the apple", - "name": "action: pick_up(apple)", - "type_name": "action", - (effect omitted) - }, - ], - }, - ], -} \ No newline at end of file diff --git a/data/prompts/super_skeleton/prompt/prompt_bt_skeleton.txt b/data/prompts/super_skeleton/prompt/prompt_bt_skeleton.txt deleted file mode 100644 index 4b713321..00000000 --- a/data/prompts/super_skeleton/prompt/prompt_bt_skeleton.txt +++ /dev/null @@ -1,68 +0,0 @@ -[user] -The behavior tree should be in json dictionary format and have a structure like the following example: -""" -{ - "summary": "selector to load inward_claw to left hand", - "children": [ - { - "summary": "condition node to check if inward_claw is held by left hand", - }, - { - "summary": "sequence to load inward_claw to left hand", - "children": [ - { - "summary": "condition node to check if inward_claw is equippable", - }, - { - "summary": "condition node to check if left hand is free", - }, - { - "summary": "action node to equip inward_claw to left hand", - "effects": [ - {"summary": "left hand will be not free anymore",}, - {"summary": "inward_claw will be not equippable anymore",}, - {"summary": "left_hand will hold inward_claw",}, - ], - }, - ], - }, - ], -} -""" -Follow these rules: -1. Every node are described by a summary, which is a short interpretive description of the node. It should include the type of the node and the purpose of the node. -2. The control flow nodes, "selector" and "sequence", can have "children", which is a list of children nodes. -3. The "condition" nodes that are children of "selectors" are "targets". Those that are children of "sequences" are preconditions. -4. The "action" nodes should have a key "effects" with a list of dictionaries, which describes the to-be-exerted effects of the action. The effects are shortly described under the key "summary". -5. All the subtrees always have their corresponding action in ROBOT_ACTION_LIST. They are used to represent the actions in the action sequence. The recommended structure is: - - A selector as the root node. - - (Usually one) target condition node as the first child of the selector to check if the target of the action has been fulfilled. - - A sequence node as the last child of the selector node to achieve the target. - - (Usually a number of) precondition nodes as the front children of the sequence node to check the preconditions for executing the action. - - One action node as the last child of the sequence node, whose preconditions are checked before and will immediate start the job when being ticked. - - The action will exert its defined effects after finishing the job successfully. Those effects should fulfilled all the target conditions in the selector node. - A brief example is as follows: -{ - selector: load_tool selector, - children: [ - { - target: check if inward_claw is equipped by left hand, - }, - { - sequence: load_tool, - children: [ - precondition: check if inward_claw is equippable, - precondition: check if left hand is free, - {action: equip inward_claw to left hand, effects: [left hand not free, inward_claw not equippable, left_hand holds inward_claw]}, - ], - }, - ], -} - -6. When performing a new action to fulfill a unsatisfied precondition, you can construct a subtree to replace the precondition and take that precondition as the target of the subtree. Then you construct the sequence of the subtree according to the new action you want to perform. -7. All of the control flow nodes, namely selectors and sequences, are without memory. A new behavior tree tick will always start from the root and traverse all the nodes even if they are ticked in the last tick. - -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/data/prompts/super_skeleton/prompt/prompt_bt_skeleton_example.txt b/data/prompts/super_skeleton/prompt/prompt_bt_skeleton_example.txt deleted file mode 100644 index 6f832f67..00000000 --- a/data/prompts/super_skeleton/prompt/prompt_bt_skeleton_example.txt +++ /dev/null @@ -1,143 +0,0 @@ -[user] -I will give you some examples of the input and the output you will generate. -Example 1: -""" -- Input: -(define (problem robot_assembly_problem-problem) - (:domain robot_assembly_problem-domain) - (:objects - parallel_box1 parallel_box2 inward_claw outward_claw no_tool - tool - gear1 gear2 gear3 shaft1 shaft2 base - part - left_hand - hand - ) - (:init (can_manipulate parallel_box1 gear1) (can_manipulate outward_claw gear2) (can_manipulate inward_claw gear3) (can_manipulate parallel_box2 shaft1) (can_manipulate no_tool shaft2) (can_screw_to leg1 seat) (can_screw_to leg2 seat) (can_insert_to back seat) (can_screw_to nut1 seat) (can_screw_to nut2 seat) (can_screw_to blub base) (can_place_to lamp blub) (can_insert_to shaft1 base) (can_insert_to shaft2 base) (can_insert_to gear3 shaft2) (can_insert_to gear2 base) (can_insert_to gear1 shaft1) (is_inserted_to shaft1 base) (is_free left_hand) (is_free parallel_box1) (is_free parallel_box2) (is_free inward_claw) (is_free outward_claw) (is_free no_tool) (is_free gear1) (is_free gear2) (is_free gear3) (is_free shaft1) (is_free shaft2) (is_free base) (is_equippable parallel_box1) (is_equippable parallel_box2) (is_equippable inward_claw) (is_equippable outward_claw) (is_equippable no_tool)) - (:goal (and (is_inserted_to gear1 shaft1))) -) -- Output: -``` -{"task_plan": { - "action_sequence": [ - "load_tool(left_hand, parallel_box1)", - "pick_up(left_hand, parallel_box1, gear1)", - "insert(left_hand, parallel_box1, gear1, shaft1)" - ], - "action_instructions": [ - "equip the parallel_box1 on the left hand", - "pick up the gear1 with the parallel_box1 in the left hand", - "insert the gear1 into the shaft1 using the parallel_box1 in the left hand" - ], - "object_name": ["left_hand", "gear1", "parallel_box1", "shaft1"], - "behavior_tree": { - "summary": "selector to insert gear1 into shaft1", - "children": [ - { - "summary": "condition node to check if gear1 is inserted to shaft1", - }, - { - "summary": "sequence to insert gear1 into shaft1", - "children": [ - { - "summary": "selector to load_tool - "children": [ - { - "summary": "condition node to check if parallel_box1 is hold", - }, - { - "summary": "sequence to load_tool", - "children": [ - { - "summary": "condition node to check if parallel_box1 is equippable", - }, - { - "summary": "condition node to check if left hand is free", - }, - { - "summary": "action node to equip parallel_box1 to left hand", - "effects": [ - { - "summary": "left_hand will be not free", - }, - { - "summary": "parallel_box1 will be not equippable", - }, - { - "summary": "left_hand will hold parallel_box1", - }, - ], - }, - ], - }, - ], - }, - { - "summary": "selector to pick_up", - "children": [ - { - "summary": "condition node to check if parallel_box1 holds gear1", - }, - { - "summary": "sequence to pick_up", - "children": [ - { - "summary": "condition node to check if parallel_box1 is free", - }, - { - "summary": "condition node to check if left hand holds parallel_box1", - }, - { - "summary": "action node to pick up gear1 using parallel_box1", - "effects": [ - { - "summary": "parallel_box1 will hold gear1", - }, - { - "summary": "parallel_box1 will be not free", - }, - ], - }, - ], - }, - ], - }, - { - "summary": "condition node to check if gear1 can be inserted to shaft1", - }, - { - "summary": "action node to insert gear1 to shaft1", - "effects": [ - { - "summary": "parallel_box1 will be not holding gear1", - }, - { - "summary": "parallel_box1 will be free", - }, - { - "summary": "gear1 will be inserted to shaft1", - }, - ], - }, - ], - }, - ], -} -}, -"environment_before": {"objects": [ - {"name": "gear1", "properties": []}, - {"name": "shaft1", "properties": []}, - {"name": "parallel_box1", "properties": ["is_equippable", "is_free"]}, - {"name": "left_hand", "properties": ["is_free"]}, - ], - "constraints": [ - {"source": "gear1", "name": "can_insert_to", "target": "shaft1"}, - {"source": "parallel_box1", "name": "can_manipulate", "target": "gear1"}, - ], - "relations": [], -}, -"instruction_summary": "insert gear1 into shaft1", -"question":""} -``` -""" -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/data/prompts/super_skeleton/prompt/prompt_bt_validation.txt b/data/prompts/super_skeleton/prompt/prompt_bt_validation.txt deleted file mode 100644 index e69de29b..00000000 diff --git a/data/prompts/super_skeleton/prompt/prompt_domain.txt b/data/prompts/super_skeleton/prompt/prompt_domain.txt deleted file mode 100644 index bf3c7859..00000000 --- a/data/prompts/super_skeleton/prompt/prompt_domain.txt +++ /dev/null @@ -1,89 +0,0 @@ -[user] -Following is the domain knowledge you need in pddl domain format, which includes: -- the object types in the world, -- the predicates for describing the states of the world, -- the actions that the robot can apply, with their precondtions and effects defined. -""" -(define (domain robot_assembly_problem-domain) - (:requirements :strips :typing) - (:types - thing - object - tool part hand - thing - ) - (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part)) - (:action pick_up - :parameters ( ?hand - hand ?tool - tool ?part - part) - :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part)) - :effect (and (hold ?tool ?part) (not (is_free ?tool)))) - (:action put_down - :parameters ( ?hand - hand ?tool - tool ?part - part) - :precondition (and (hold ?tool ?part) (hold ?hand ?tool)) - :effect (and (not (hold ?hand ?part)) (is_free ?tool))) - (:action place - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2))) - (:action detach - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2)))) - (:action insert - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2))) - (:action pull - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2)))) - (:action screw - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2))) - (:action unscrew - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2)))) - (:action load_tool - :parameters ( ?hand - hand ?tool - tool) - :precondition (and (is_equippable ?tool) (is_free ?hand)) - :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool))) - (:action unload_tool - :parameters ( ?hand - hand ?tool - tool) - :precondition (and (hold ?hand ?tool) (is_free ?tool)) - :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool)))) -) -""" -The precidates are explained as follows: -""" -"PREDICATE_LIST" -- is_free( or ): or is free and can hold something. (default: true) -- is_equippable(): is ready to be eqquipped by a robot hand. (default: true) -- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false) -- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false) -- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false) -- can_place_to(, ): can be placed into (placement assembly constraint) (default: false) -- hold(, ): is holding (default: false) -- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false) -- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false) -- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false) -""" - -The actions are explained as follows: -""" -"ROBOT_ACTION_LIST" -- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something. -- put_down(, , ): use to put down . As preconditions, should be holding and should be holding . -- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding . -- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into . -- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding . -- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into . -- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding . -- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into . -- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free. -- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free. -""" -the items in <> indicate the type of the instances needed for the predicates and actions. -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. diff --git a/data/prompts/super_skeleton/prompt/prompt_environment.txt b/data/prompts/super_skeleton/prompt/prompt_environment.txt deleted file mode 100644 index 656fd60c..00000000 --- a/data/prompts/super_skeleton/prompt/prompt_environment.txt +++ /dev/null @@ -1,31 +0,0 @@ -[user] -Information about world states should be in json dictionary format. Example: -""" -{ - "objects": [ - {"name": "leg1", "properties": []}, # leg1 is an object with no properties - {"name": "seat", "properties": []}, # seat is an object with no properties - {"name": "cabinet", "properties": ["open"]}, # cabinet is an object with a property "open" - {"name": "parallel_box1", "properties": []}, # parallel_box1 is an object with no properties - {"name": "table", "properties": []}, # table is an object with no properties - ], - "constraints": [ - {"source": "leg1", "name": "can_screw_to", "target": "seat"}, # means leg1 can be screwed to seat - {"source": "parallel_box1", "name": "can_manipulate", "target": "leg1"}, # means parallel_box1 can manipulate leg1 - {"source": "cabinet", "name": "on_something", "target": "table"}, # means cabinet is on table - ], - "relations": [ - {"source": "leg1", "name": "is_screwed_to", "target": "seat"}, # means leg1 is screwed to seat - ], -} -""" -It contains three lists: - - "objects": the objects and their properties (as lists of strings) in the environment. The properties can be changed by robot actions. - - "constraints": the constraint relations that are enforced and should not be changed. They are knowledge about the world or some constraints set by the user. - - "relations": the relations in the environment. They can be changed by robot actions. -The properties, constraints and relations should always align with the [PREDICATE_LIST] in domain knowledge. -You should only concentrate on the objects, constraints and relations that are related to (or can be effected by) the current task. -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/data/prompts/super_skeleton/prompt/prompt_output_format.txt b/data/prompts/super_skeleton/prompt/prompt_output_format.txt deleted file mode 100644 index 83c5c6f7..00000000 --- a/data/prompts/super_skeleton/prompt/prompt_output_format.txt +++ /dev/null @@ -1,20 +0,0 @@ -[user] -You generate an action sequence to solve the task and contruct a behavior tree based on it in json format. -The output dictionary has five keys. -""" -- dictionary["task_plan"]: A dictionary containing information about the task plan. -- dictionary["environment_before"]: The environment state before executing the plan. This should be consistent with the initial states in the provided problem. -- dictionary["instruction_summary"]: A brief summary of the task plan. -- dictionary["question"]: If you don't have enough knowledge for solving the problem, you can ask questions about the knowledge you need. Leave this key empty if you can solve the task. -""" -Three keys exist in dictionary["task_plan"]. -""" -- dictionary["task_plan"]["action_sequence"]: Contains a list of robot actions. Only the actions defined in the "ROBOT_ACTION LIST" will be used. -- dictionary["task_plan"]["action_instructions"]: contains a list of instructions corresponding to dictionary["task_plan"]["action_sequence"] to explain the actions. -- dictionary["task_plan"]["manipulated_objects"]: The list of the names of the manipulated object. It only includes the objects that are related to the current task. -- dictionary["task_plan"]["behavior_tree"]: contains a dictionary of the behavior tree constructed according to the dictionary ["task_plan"]["action_sequence"] -""" -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/data/prompts/super_skeleton/prompt/prompt_problem.txt b/data/prompts/super_skeleton/prompt/prompt_problem.txt deleted file mode 100644 index 22734b26..00000000 --- a/data/prompts/super_skeleton/prompt/prompt_problem.txt +++ /dev/null @@ -1,23 +0,0 @@ -[user] -Information about the problem is given as a pddl problem. -It includes the following information: -- Objects in the world: all instances of the types defined in the domain knowledge. For example, "parallel_box1" is an instance of the type "tool", while "gear1" is an instance of the type "part". -- Initial states: the world states to start with, described with [PREDICATE_LIST] from domain knowledge. For example, "is_free hand" means the object "hand" is free and holds nothing. -- Goal states: the world states to achieve, described with [PREDICATE_LIST] from domain knowledge. -Example: -""" -(define (problem robot_assembly_problem-problem) - (:domain robot_assembly_problem-domain) - (:objects - parallel_box1 parallel_box2 inward_claw outward_claw no_tool - tool - gear1 gear2 gear3 shaft1 shaft2 base - part - left_hand - hand - ) - (:init (can_manipulate parallel_box1 gear1) (can_manipulate outward_claw gear2) (can_manipulate inward_claw gear3) (can_manipulate parallel_box2 shaft1) (can_manipulate no_tool shaft2) (can_screw_to leg1 seat) (can_screw_to leg2 seat) (can_insert_to back seat) (can_screw_to nut1 seat) (can_screw_to nut2 seat) (can_screw_to blub base) (can_place_to lamp blub) (can_insert_to shaft1 base) (can_insert_to shaft2 base) (can_insert_to gear3 shaft2) (can_insert_to gear2 base) (can_insert_to gear1 shaft1) (is_free left_hand) (is_free parallel_box1) (is_free parallel_box2) (is_free inward_claw) (is_free outward_claw) (is_free no_tool) (is_equippable parallel_box1) (is_equippable parallel_box2) (is_equippable inward_claw) (is_equippable outward_claw) (is_equippable no_tool)) - (:goal (and (is_inserted_to shaft1 base))) -) -""" -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/data/prompts/super_skeleton/prompt/prompt_role.txt b/data/prompts/super_skeleton/prompt/prompt_role.txt deleted file mode 100644 index 6ce736a0..00000000 --- a/data/prompts/super_skeleton/prompt/prompt_role.txt +++ /dev/null @@ -1,5 +0,0 @@ -[user] -You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you break it down and construct robot task plans represented by behaiovr trees. -Please do not begin working until I say "Start working." Instead, simply output the message "Waiting for next input." Understood? -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/data/prompts/super_skeleton/query/query.txt b/data/prompts/super_skeleton/query/query.txt deleted file mode 100644 index 674cfab0..00000000 --- a/data/prompts/super_skeleton/query/query.txt +++ /dev/null @@ -1,16 +0,0 @@ -Start working. Resume from the problem below. -""" -{"problem": [PROBLEM]} -""" -The dictionary that you return should be formatted as json file. Follow these rules: -1. Always equip the proper tools before performing specific object manipulaion. -2. Make sure that each element of the ["action_instructions"] explains corresponding element of the ["action_sequence"]. Refer to the "ROBOT ACTION LIST" to understand the elements of ["action_sequence"]. -3, The length of the ["action_instructions"] list and the ["action_sequence"] list should be the same. -4. Never left ',' at the end of the list. Use json boolean values, "true" or "false", instead of "True" or "False". -5. Keep track of all items listed in the "objects" section of the "environment_before" field. Please make sure you fill out the "objects" section for all listed items. -6. Use the "PREDICATE_LIST" for the initial states (:init) in the problem to generate ["environment_before"]. -7. All keys of the dictionary should be double-quoted. -8. Insert ``` at the beginning and the end of the dictionary to separate it from the rest of your response. Don't write any additional things. -9. Make sure that you output a consistent manipultation as a single arm robot. For example, grasping an object should not occur in successive steps. -10. When generating the behavior tree, make sure the tree has the asked structure. -Adhere to the output format I defined above. Follow the rules above. Think step by step. \ No newline at end of file diff --git a/data/prompts/super_skeleton/system/system.txt b/data/prompts/super_skeleton/system/system.txt deleted file mode 100644 index 26f2afe2..00000000 --- a/data/prompts/super_skeleton/system/system.txt +++ /dev/null @@ -1 +0,0 @@ -You are an excellent interpreter of instructions for robotic assembly tasks. Given the knowledge you need, the world states and some instructions, you break it down and construct robot task plans represented by behaiovr trees based on the action sequences. \ No newline at end of file diff --git a/data/prompts/vv/behaviortree.txt b/data/prompts/vv/behaviortree.txt deleted file mode 100644 index c0704a7d..00000000 --- a/data/prompts/vv/behaviortree.txt +++ /dev/null @@ -1,19 +0,0 @@ -About behavior tree: -1. Every node has: - - "summary": a short interpretive description of the node. - - "name": includes the node type and the formal name of the node according to the domain knowledge. -2. Selector nodes and sequence nodes can have a list "children" of child nodes. -3. Refer to the domain knowledge to understand the condition nodes and action nodes. -4. Make sure the preconditions of an action node are checked before the action is executed. -5. One condition node can only check one condition. -6. The child node of a sequence node will not be ticked unless all other child nodes in front of it return success. -7. The action node will execute the action when it is ticked again, even if it has been executed and returned success before. Use selector nodes with a condition node to avoid this. -8. Don't put two action nodes in one sequence node because in this case the first action node will be executed repeatedly and the second one will never get executed. - -Possible reasons for non-successful execution: -1. Nodes that conflict with the domain knowledge will never return success. For example, wrongly applying a predicate to an object of the wrong type. -2. A condition node is not fulfilled and the action to fulfill it doesn't exist. -3. Two action nodes are in one sequence node. The first action node is ticked repeatedly and the second one never gets ticked. -4. Live lock between two actions. -5. An action node failed and no alternative action is available. -6. The tree is not well-structured. \ No newline at end of file diff --git a/data/prompts/vv/domain.txt b/data/prompts/vv/domain.txt deleted file mode 100644 index 7eec5ea4..00000000 --- a/data/prompts/vv/domain.txt +++ /dev/null @@ -1,84 +0,0 @@ -Following is the domain knowledge you need in pddl domain format, which includes: -- the object types in the world, -- the predicates for describing the states of the world, -- the actions that the robot can apply, with their precondtions and effects defined. -""" -(define (domain robot_assembly_problem-domain) - (:requirements :strips :typing) - (:types - thing - object - tool part hand - thing - ) - (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part)) - (:action pick_up - :parameters ( ?hand - hand ?tool - tool ?part - part) - :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part)) - :effect (and (hold ?tool ?part) (not (is_free ?tool)))) - (:action put_down - :parameters ( ?hand - hand ?tool - tool ?part - part) - :precondition (and (hold ?tool ?part) (hold ?hand ?tool)) - :effect (and (not (hold ?hand ?part)) (is_free ?tool))) - (:action place - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2))) - (:action detach - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2)))) - (:action insert - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2))) - (:action pull - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2)))) - (:action screw - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2))) - (:action unscrew - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2)))) - (:action load_tool - :parameters ( ?hand - hand ?tool - tool) - :precondition (and (is_equippable ?tool) (is_free ?hand)) - :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool))) - (:action unload_tool - :parameters ( ?hand - hand ?tool - tool) - :precondition (and (hold ?hand ?tool) (is_free ?tool)) - :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool)))) -) -""" -The precidates are explained as follows: -""" -"PREDICATE_LIST" -- is_free( or ): or is free and can hold something. (default: true) -- is_equippable(): is ready to be eqquipped by a robot hand. (default: true) -- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false) -- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false) -- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false) -- can_place_to(, ): can be placed into (placement assembly constraint) (default: false) -- hold(, ): is holding (default: false) -- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false) -- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false) -- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false) -""" - -The actions are explained as follows: -""" -"ROBOT_ACTION_LIST" -- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something. -- put_down(, , ): use to put down . As preconditions, should be holding and should be holding . -- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding . -- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into . -- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding . -- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into . -- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding . -- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into . -- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free. -- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free. -""" -the items in <> indicate the type of the instances needed for the predicates and actions. \ No newline at end of file diff --git a/data/prompts/vv/example.txt b/data/prompts/vv/example.txt deleted file mode 100644 index 5fdda217..00000000 --- a/data/prompts/vv/example.txt +++ /dev/null @@ -1,189 +0,0 @@ -Following are some examples of the input and output. -Example 1: -""" -- Input: -objects: -{ - "tool": ["parallel_box1", "inward_claw", "outward_claw"], - "part": ["gear1", "shaft1", "gearbase_hole1", "gearbase_hole3"] - "hand": ["left_hand"] -} -runtime_world_state: -{ - "objects": [ - {"name": "parallel_box1", "properties": ["is_free", "is_equippable"]}, - {"name": "inward_claw", "properties": ["is_free", "is_equippable"]}, - {"name": "outward_claw", "properties": ["is_free"]}, - {"name": "gear1", "properties": []}, - {"name": "shaft1", "properties": []}, - {"name": "gearbase_hole1", "properties": []}, - {"name": "left_hand", "properties": []} - ], - "constraints": [ - {"source": "parallel_box1", "name": "can_manipulate", "target": "gear1"}, - {"source": "gear1", "name": "can_insert_to", "target": "shaft1"}, - ], - "relations": [ - {"source": "left_hand","name": "hold","target": "outward_claw"} - ] -} -last_behavior_tree: -{ - "summary": "selector to load parallel_box1", - "name": "selector: load_tool(left_hand, parallel_box1)", - "identifier": 0, - "children": [ - { - "summary": "condition node to check if left_hand holds parallel_box1", - "name": "target: hold(left_hand, parallel_box1)" - "identifier": 1, - }, - { - "summary": "sequence to load parallel_box1", - "name": "sequence: load_tool(left_hand, parallel_box1)", - "identifier": 2, - "children": [ - { - "summary": "condition node to check if parallel_box1 is equippable", - "name": "precondition: is_equippable(parallel_box1)" - "identifier": 3, - }, - { - "summary": "condition node to check if left_hand is free, - "name": "target: is_free(left_hand)" - "identifier": 4, - }, - { - "summary": "action node to equip parallel_box1 to left hand", - "name": "action: load_tool(left_hand, parallel_box1)", - "identifier": 5, - }, - ], - }, - ], -} -last_failed_node: -{ - "summary": "condition node to check if left_hand is free, - "name": "target: is_free(left_hand)" - "identifier": 4, -} -instructions: "insert gear1 into shaft1" - -- Output: -{ - "summary": "selector to load parallel_box1", - "name": "selector: load_tool(left_hand, parallel_box1)", - "identifier": 0, - "children": [ - { - "summary": "condition node to check if left_hand holds parallel_box1", - "name": "target: hold(left_hand, parallel_box1)" - "identifier": 1, - }, - { - "summary": "sequence to load parallel_box1", - "name": "sequence: load_tool(left_hand, parallel_box1)", - "identifier": 2, - "children": [ - { - "summary": "condition node to check if parallel_box1 is equippable", - "name": "precondition: is_equippable(parallel_box1)" - "identifier": 3, - }, - { - "summary": "selector to make left_hand unload outward_claw", - "name": "selector: unload_tool(left_hand, outward_claw)", - "children": [ - { - "summary": "condition node to check if left_hand is free, - "name": "target: is_free(left_hand)" - }, - { - "summary": "sequence to make left_hand unload outward_claw", - "name": "sequence: unload_tool(left_hand, outward_claw)", - "children": [ - { - "summary": "condition node to check if outward_claw is free", - "name": "precondition: is_free(outward_claw)" - }, - { - "summary": "condition node to check if left hand holds outward_claw", - "name": "precondition: hold(left_hand, outward_claw)" - }, - { - "summary": "action node to make left hand unload outward_claw", - "name": "action: unload_tool(left_hand, outward_claw)", - }, - ], - }, - ], - }, - { - "summary": "action node to equip parallel_box1 to left hand", - "name": "action: load_tool(left_hand, parallel_box1)", - "identifier": 5, - }, - ], - }, - ], -} -""" - -Example 2: -""" -- Input: -objects: -{ - "tool": ["inward_claw"], - "part": ["nut1", "bolt1"] - "hand": ["left_hand"] -} -runtime_world_state: -{ - "objects": [ - {"name": "inward_claw", "properties": ["is_free", "is_equippable"]}, - {"name": "nut1", "properties": []}, - {"name": "bolt1", "properties": []}, - {"name": "left_hand", "properties": ["is_free"]} - ], - "constraints": [ - {"source": "inward_claw", "name": "can_manipulate", "target": "nut1"}, - {"source": "nut1", "name": "can_screw_to", "target": "bolt1"}, - ], - "relations": [] -} -last_behavior_tree: None -last_failed_node: None -instructions: "screw nut1 into bolt1" - -- Output: -{ - "summary": "selector to use left_hand with inward_claw to screw nut1 into bolt1", - "name": "selector: screw(left_hand, inward_claw, nut1, bolt1)", - "children": [ - { - "summary": "condition node to check if nut1 is screwed into bolt1", - "name": "target: is_screwed_to(nut1, bolt1)" - }, - { - "summary": "sequence to use left_hand with inward_claw to screw nut1 into bolt1", - "name": "sequence: screw(left_hand, inward_claw, nut1, bolt1)", - "children": [ - { - "summary": "condition node to check if inward_claw holds nut1", - "name": "precondition: hold(inward_claw, nut1)" - }, - { - "summary": "condition node to check if nut1 can be screwed into bolt1", - "name": "precondition: can_screw_to(nut1, bolt1)" - }, - { - "summary": "action node to use left_hand with inward_claw to screw nut1 into bolt1", - "name": "action: screw(left_hand, inward_claw, nut1, bolt1)" - }, - ], - }, - ], -} -""" \ No newline at end of file diff --git a/data/prompts/vv/object.txt b/data/prompts/vv/object.txt deleted file mode 100644 index 14f247e1..00000000 --- a/data/prompts/vv/object.txt +++ /dev/null @@ -1,3 +0,0 @@ -objects are given in a dictionary. -Its keys are the types and the values are lists of instances of that type. -Refer to the domain knowledge to understand which predicates and actions are applicable to which types. \ No newline at end of file diff --git a/data/prompts/vv/state.txt b/data/prompts/vv/state.txt deleted file mode 100644 index 1f26f192..00000000 --- a/data/prompts/vv/state.txt +++ /dev/null @@ -1,6 +0,0 @@ -Information about world states is in json dictionary format. -It contains three lists: - - "objects": the objects in the world and their properties. The properties can be changed by robot actions. - - "constraints": the constraint relations that are enforced and should not be changed. - - "relations": the relations in the world. They can be changed by robot actions. -The properties, constraints and relations should always align with the [PREDICATE_LIST] in domain knowledge. \ No newline at end of file diff --git a/data/prompts/vv/system.txt b/data/prompts/vv/system.txt deleted file mode 100644 index 7ebac297..00000000 --- a/data/prompts/vv/system.txt +++ /dev/null @@ -1 +0,0 @@ -You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you construct robot task plans represented by behaiovr trees. \ No newline at end of file diff --git a/data/prompts/vv/task.txt b/data/prompts/vv/task.txt deleted file mode 100644 index ed900bb2..00000000 --- a/data/prompts/vv/task.txt +++ /dev/null @@ -1,8 +0,0 @@ -The input are: -objects: a list of objects in the world. -start_world_state: the world state when the behavior tree started. -runtime_world_state: the world state when the last behaior tree stopped. -last_behavior_tree: the last executed behavior tree. -last_failed_node: the last node that failed in the last behavior tree. - -You analyze the failure reason and summarize it briefly as your output. \ No newline at end of file diff --git a/data/prompts/vv/template.txt b/data/prompts/vv/template.txt deleted file mode 100644 index 0e1d741f..00000000 --- a/data/prompts/vv/template.txt +++ /dev/null @@ -1,13 +0,0 @@ -Start working. Resume from the input below. - -objects: {objects} - -runtime_world_state: {runtime_world_state} - -start_world_state: {start_world_state} - -last_behavior_tree: {last_behavior_tree} - -last_failed_node: {last_failed_node} - -summary: {summary} \ No newline at end of file diff --git a/data/router_utterances.json b/data/router_utterances.json deleted file mode 100644 index 9299619e..00000000 --- a/data/router_utterances.json +++ /dev/null @@ -1,34 +0,0 @@ -{ - "rectify": [ - "I think this part is wrong. I think it should be like this: ", - "The structure is not correct. ", - "This part is A, and you do it like B. That is the reason that the system is not working. You should do C so that the system returns D.", - "I think you should change this part. " - ], - "approve": [ - "I think it works. ", - "go ahead. ", - "Yes, I think this is correct. ", - "So far so good. ", - "Looks good. Let's try this out.", - "I think your plan is good. ", - "I don't find any mistake in your response. Let's run it." - ], - "disapprove": [ - "No I don't think this is correct. ", - "The plan does not perform correctly. ", - "I guess you need to do it again." - ], - "finish": [ - "Nothing else to do. bye. ", - "That's all. You did a good job. ", - ". " - ], - "instruction": - [ - "I need you to write a program that does the following", - "You should add pepper to the soup", - "Now please write a program that deletes the last element of the list", - "You do this, then you do that, finally you should finish with this" - ] -} \ No newline at end of file diff --git a/data/world_definition.json b/data/world_definition.json deleted file mode 100644 index 097ad7a9..00000000 --- a/data/world_definition.json +++ /dev/null @@ -1,159 +0,0 @@ -{ - "properties": [ - "is_empty", - "is_equippable" - ], - "constraints": [ - "can_manipulate", - "can_screw_to", - "can_insert_to", - "can_place_to" - ], - "relations": [ - "hold", - "is_inserted_to", - "is_screwed_to", - "is_placed_to" - ], - "actions": { - "pick_up": { - "precondition": [ - "is_empty ?tool", - "hold ?hand ?tool", - "can_manipulate ?tool ?part" - ], - "effect": [ - "hold ?tool ?part", - "(not (is_empty ?tool))" - ], - "template": "pick_up(?hand, ?tool, ?part)" - }, - "put_down": { - "precondition": [ - "hold ?tool ?part", - "hold ?hand ?tool" - ], - "effect": [ - "(not (hold ?tool ?part))", - "is_empty ?tool" - ], - "template": "put_down(?hand, ?tool, ?part)" - }, - "place": { - "precondition": [ - "hold ?hand ?tool", - "hold ?tool ?part1", - "can_place_to ?part1 ?part2" - ], - "effect": [ - "(not (hold ?tool ?part1))", - "is_empty ?tool", - "is_placed_to ?part1 ?part2" - ], - "template": "place(?hand, ?tool, ?part1, ?part2)" - }, - "detach": { - "precondition": [ - "hold ?hand ?tool", - "is_empty ?tool", - "can_manipulate ?tool ?part1", - "is_placed_to ?part1 ?part2" - ], - "effect": [ - "hold ?tool ?part1", - "(not (is_empty ?tool))", - "(not (is_placed_to ?part1 ?part2))" - ], - "template": "detach(?hand, ?tool, ?part1, ?part2)" - }, - "insert": { - "precondition": [ - "hold ?hand ?tool", - "hold ?tool ?part1", - "can_insert_to ?part1 ?part2" - ], - "effect": [ - "(not (hold ?tool ?part1))", - "is_empty ?tool", - "is_inserted_to ?part1 ?part2" - ], - "template": "insert(?hand, ?tool, ?part1, ?part2)" - }, - "pull": { - "precondition": [ - "hold ?hand ?tool", - "is_empty ?tool", - "is_inserted_to ?part1 ?part2", - "can_manipulate ?tool ?part1" - ], - "effect": [ - "hold ?tool ?part1", - "(not (is_empty ?tool))", - "(not (is_inserted_to ?part1 ?part2))" - ], - "template": "pull(?hand, ?tool, ?part1, ?part2)" - }, - "screw": { - "precondition": [ - "hold ?hand ?tool", - "hold ?tool ?part1", - "can_screw_to ?part1 ?part2" - ], - "effect": [ - "(not (hold ?tool ?part1))", - "is_empty ?tool", - "is_screwed_to ?part1 ?part2" - ], - "template": "screw(?hand, ?tool, ?part1, ?part2)" - }, - "unscrew": { - "precondition": [ - "hold ?hand ?tool", - "is_empty ?tool", - "is_screwed_to ?part1 ?part2", - "can_manipulate ?tool ?part1" - ], - "effect": [ - "hold ?tool ?part1", - "(not (is_empty ?tool))", - "(not (is_screwed_to ?part1 ?part2))" - ], - "template": "unscrew(?hand, ?tool, ?part1, ?part2)" - }, - "load_tool": { - "precondition": [ - "is_equippable ?tool", - "is_empty ?hand" - ], - "effect": [ - "(not (is_empty ?hand))", - "(not (is_equippable ?tool))", - "hold ?hand ?tool" - ], - "template": "load_tool(?hand, ?tool)" - }, - "unload_tool": { - "precondition": [ - "hold ?hand ?tool", - "is_empty ?tool" - ], - "effect": [ - "is_empty ?hand", - "is_equippable ?tool", - "(not (hold ?hand ?tool))" - ], - "template": "unload_tool(?hand, ?tool)" - }, - "change_tool":{ - "precondition":[ - "hold ?hand ?tool1", - "is_empty ?tool1" - ], - "effect":[ - "hold ?hand ?tool2", - "(not (hold ?hand ?tool1))" - ], - "template": "change_tool(?hand, ?tool1, ?tool2)" - } - } -} diff --git a/demo_dot_graphs_test.dot b/demo_dot_graphs_test.dot deleted file mode 100644 index 93d4c67f..00000000 --- a/demo_dot_graphs_test.dot +++ /dev/null @@ -1,25 +0,0 @@ -digraph pastafarianism { -ordering=out; -graph [fontname="times-roman"]; -node [fontname="times-roman"]; -edge [fontname="times-roman"]; -"Demo Dot Graphs TEST" [fillcolor=cyan, fontcolor=black, fontsize=9, label="Demo Dot Graphs TEST", shape=octagon, style=filled]; -"BlackBox 1" [fillcolor=orange, fontcolor=black, fontsize=9, label="Ⓜ BlackBox 1", shape=box, style=filled]; -"Demo Dot Graphs TEST" -> "BlackBox 1"; -Worker [fillcolor=gray, fontcolor=black, fontsize=9, label=Worker, shape=ellipse, style=filled]; -"BlackBox 1" -> Worker; -"Worker*" [fillcolor=gray, fontcolor=black, fontsize=9, label="Worker*", shape=ellipse, style=filled]; -"BlackBox 1" -> "Worker*"; -"Worker**" [fillcolor=gray, fontcolor=black, fontsize=9, label="Worker**", shape=ellipse, style=filled]; -"BlackBox 1" -> "Worker**"; -"Blackbox 3" [fillcolor=gray20, fontcolor=dodgerblue, fontsize=9, label="Ⓜ Blackbox 3", shape=box, style=filled]; -"BlackBox 1" -> "Blackbox 3"; -"Blackbox 2" [fillcolor=orange, fontcolor=black, fontsize=9, label="Ⓜ Blackbox 2", shape=box, style=filled]; -"Demo Dot Graphs TEST" -> "Blackbox 2"; -"Worker***" [fillcolor=gray, fontcolor=black, fontsize=9, label="Worker***", shape=ellipse, style=filled]; -"Blackbox 2" -> "Worker***"; -"Worker****" [fillcolor=gray, fontcolor=black, fontsize=9, label="Worker****", shape=ellipse, style=filled]; -"Blackbox 2" -> "Worker****"; -"Worker*****" [fillcolor=gray, fontcolor=black, fontsize=9, label="Worker*****", shape=ellipse, style=filled]; -"Blackbox 2" -> "Worker*****"; -} diff --git a/demo_dot_graphs_test.png b/demo_dot_graphs_test.png deleted file mode 100644 index 5c033675..00000000 Binary files a/demo_dot_graphs_test.png and /dev/null differ diff --git a/demo_dot_graphs_test.svg b/demo_dot_graphs_test.svg deleted file mode 100644 index 442652db..00000000 --- a/demo_dot_graphs_test.svg +++ /dev/null @@ -1,127 +0,0 @@ - - - - - - -pastafarianism - - - -Demo Dot Graphs TEST - -Demo Dot Graphs TEST - - - -BlackBox 1 - -Ⓜ BlackBox 1 - - - -Demo Dot Graphs TEST->BlackBox 1 - - - - - -Blackbox 2 - -Ⓜ Blackbox 2 - - - -Demo Dot Graphs TEST->Blackbox 2 - - - - - -Worker - -Worker - - - -BlackBox 1->Worker - - - - - -Worker* - -Worker* - - - -BlackBox 1->Worker* - - - - - -Worker** - -Worker** - - - -BlackBox 1->Worker** - - - - - -Blackbox 3 - -Ⓜ Blackbox 3 - - - -BlackBox 1->Blackbox 3 - - - - - -Worker*** - -Worker*** - - - -Blackbox 2->Worker*** - - - - - -Worker**** - -Worker**** - - - -Blackbox 2->Worker**** - - - - - -Worker***** - -Worker***** - - - -Blackbox 2->Worker***** - - - - - diff --git a/examples/chair/01/01.png b/examples/chair/01/01.png deleted file mode 100644 index bfa486f2..00000000 Binary files a/examples/chair/01/01.png and /dev/null differ diff --git a/examples/chair/01/02.png b/examples/chair/01/02.png deleted file mode 100644 index 632ccc2b..00000000 Binary files a/examples/chair/01/02.png and /dev/null differ diff --git a/examples/chair/01/03.png b/examples/chair/01/03.png deleted file mode 100644 index 0c611522..00000000 Binary files a/examples/chair/01/03.png and /dev/null differ diff --git a/examples/chair/01/04.png b/examples/chair/01/04.png deleted file mode 100644 index c93d8fb8..00000000 Binary files a/examples/chair/01/04.png and /dev/null differ diff --git a/examples/chair/01/05.png b/examples/chair/01/05.png deleted file mode 100644 index 371482b3..00000000 Binary files a/examples/chair/01/05.png and /dev/null differ diff --git a/experiments/chair/baseline.py b/experiments/chair/baseline.py deleted file mode 100644 index db9c69fa..00000000 --- a/experiments/chair/baseline.py +++ /dev/null @@ -1,213 +0,0 @@ -import json -import os -import re -from pprint import pprint -from typing import List, Tuple, Annotated, TypedDict -import operator -import logging -import asyncio - -from kios_bt.bt_stewardship import BehaviorTreeStewardship -from kios_scene.scene_factory import SceneFactory -from kios_bt.bt_factory import BehaviorTreeFactory -from kios_robot.robot_interface import RobotInterface -from kios_world.world_interface import WorldInterface - -from kios_agent.llm_supporter import KiosLLMSupporter -from kios_agent.data_types import KiosPromptSkeleton - -from kios_agent.kios_tools import ( - BehaviorTreeSimulatorTool, -) - -from dotenv import load_dotenv - -load_dotenv() - -from kios_utils.pybt_test import generate_bt_stewardship, render_dot_tree -from kios_utils.pddl_problem_parser import parse_problem_init, parse_problem_objects - - -def render_bt(bt_json: dict): - test_class = BehaviorTreeFactory() - bt = test_class.from_json_to_simple_bt(bt_json) - # bt = test_class.from_json_to_tree_root(bt_json) - bt_stewardship = generate_bt_stewardship(bt) - # bt_stewardship.setup(timeout=15) - render_dot_tree(bt_stewardship) - - -####################### dirs -current_dir = os.path.dirname(os.path.abspath(__file__)) -scene_path = os.path.join(current_dir, "scene.json") -# bt_json_file_path = os.path.join(current_dir, "behavior_tree.json") -world_state_path = os.path.join(current_dir, "world_state.json") -problem_path = os.path.join(current_dir, "gearset.problem") -domain_knowledge_path = os.path.join(current_dir, "domain_knowledge.txt") - -####################### problem -# with open(problem_path, "r") as file: -# problem = file.read() - -####################### scene -with open(scene_path, "r") as file: - scene_json_object = json.load(file) - -scene = SceneFactory().create_scene_from_json(scene_json_object) - -####################### world -world_interface = WorldInterface() -with open(world_state_path, "r") as file: - world_state_json = json.load(file) - world_interface.load_world_from_json(world_state_json) - - -pprint(world_state_json) - - -####################### robot -robot_interface = RobotInterface( - robot_address="127.0.0.1", - robot_port=12000, -) -robot_interface.setup_scene(scene) - -####################### bt_factory -bt_factory = BehaviorTreeFactory( - world_interface=world_interface, - robot_interface=robot_interface, -) - -####################### behavior_tree_stewardship -behavior_tree_stewardship = BehaviorTreeStewardship( - behaviortree_factory=bt_factory, - world_interface=world_interface, - robot_interface=robot_interface, -) - - -def baseline_run(tree_root: dict, world_state: dict): - - sk_json = tree_root - initial_world_state = world_state - - # * first sim run - from kios_plan.dynamic_planning import chair_ut_dict - - solultion = behavior_tree_stewardship.sk_baseline( - initial_world_state, sk_json, chair_ut_dict - ) - record = { - "target": sk_json["name"], - "initial_world_state": initial_world_state, - "result": solultion, - } - file_dir = os.path.join(current_dir, "baseline_result.jsonl") - with open(file_dir, "a") as file: - file.write(json.dumps(record) + "\n") - - -def match_type(node: dict) -> tuple[str, str]: - node_name = node["name"] - match = re.search( - r"(selector|sequence|action|precondition|condition|target):\s*(.+)", node_name - ) - if match: - node_type = match.group(1) - node_body = match.group(2) - return node_type, node_body - else: - raise ValueError(f"the node name {node_name} does not match any type.") - - -def expand_nodes( - node_list: list[dict], - start_state: dict, - overall_tree: list[dict] = None, -) -> dict: - """ - in order to monitor the tree generation, the overall tree and the node list should be the same variable when being passed in. - """ - pprint("----------check the entire tree:") - if overall_tree is not None: - render_bt(overall_tree[0]) - pprint("----------start to expand the node list:") - pprint(node_list) - pause = input("paused here! check the tree.") - - assert len(node_list) > 0 - state = start_state - - for i in range(len(node_list)): - type_name, body = match_type(node_list[i]) - # if match_type(node_list[i]) == "action": - if type_name == "action": - print(f"the node {node_list[i]['name']} is an action node. skip it.") - pause = input("paused here! check!") - # elif match_type(node_list[i]) == "precondition" or "target": - elif type_name in ["precondition", "target"]: - # goal = node_list[i]["name"] - goal = body - plan = make_plan(state, goal) - if len(plan) == 0: - logging.warning(f"No action should be performed for the goal {goal}.") - logging.warning(f'the node {node_list[i]["name"]} has been skipped.') - pause = input("paused here! check!") - else: - logging.info(f"Actions have been planed for the goal {goal}.") - pprint(f"the plan for the goal {goal} is {plan}") - pause = input("paused here! check!") - last_action = plan[-1] - unit_subtree = generate_unit_subtree(last_action) - # insert the subtree into the node_list - node_list[i] = unit_subtree - new_node_list = get_node_list_from_tree(unit_subtree) - expand_nodes( - node_list=new_node_list, - start_state=state, - overall_tree=overall_tree, - ) - state = estimate_state(state, plan) - - return node_list[0] - - -def test_expand_nodes(): - start_state = world_state_json - node_list = [ - { - "summary": "screw the lampbulb into the lampbase", - "name": "target: screw the lampbulb into the lampbase", - } - ] - # node_list = [ - # { - # "summary": "place the lampbulb onto the lampbulb", - # "name": "target: place the lampshade onto the lampbulb", - # } - # ] - expand_nodes(node_list, start_state, node_list) - pprint(node_list) - - -def test_baseline(): - sk = { - # "summary": "the target is to screw the chairnut1 into the chairseatbolt1", - # "name": "target: is_screwed_to(chairnut1, chairseatbolt1)", - # "summary": "the target is to screw the chairnut2 into the chairseatbolt2", - # "name": "target: is_screwed_to(chairnut2, chairseatbolt2)", - "summary": "the target is to screw the chairleg1 into the chairseatthread1", - "name": "target: is_screwed_to(chairleg1, chairseatthread1)", - # "summary": "the target is to screw the chairleg2 into the chairseatthread2", - # "name": "target: is_screwed_to(chairleg2, chairseatthread2)", - # "summary": "the target is to insert the chairback into the chairseatconnector", - # "name": "target: is_inserted_to(chairback, chairseatconnector)", - } - baseline_run(sk, world_state_json) - - -if __name__ == "__main__": - pass - - # test_expand_nodes() - test_baseline() diff --git a/experiments/chair/baseline_result.jsonl b/experiments/chair/baseline_result.jsonl deleted file mode 100644 index 31927ad9..00000000 --- a/experiments/chair/baseline_result.jsonl +++ /dev/null @@ -1,3 +0,0 @@ -{"target": "target: is_screwed_to(chairnut1, chairseatbolt1)", "initial_world_state": {"objects": [{"name": "clampgripper", "properties": ["is_empty"]}, {"name": "parallelgripper", "properties": ["is_empty"]}, {"name": "inwardgripper", "properties": ["is_empty"]}, {"name": "outwardgripper", "properties": ["is_empty"]}, {"name": "defaultgripper", "properties": []}, {"name": "chairnut1", "properties": []}, {"name": "chairnut2", "properties": []}, {"name": "chairseat", "properties": []}, {"name": "chairseatbolt1", "properties": []}, {"name": "chairseatbolt2", "properties": []}, {"name": "chairseatconnection", "properties": []}, {"name": "chairseatthread1", "properties": []}, {"name": "chairseatthread2", "properties": []}, {"name": "chairback", "properties": []}, {"name": "chairleg1", "properties": []}, {"name": "chairleg2", "properties": []}, {"name": "left_hand", "properties": []}], "constraints": [{"source": "clampgripper", "name": "can_manipulate", "target": "chairback"}, {"source": "inwardgripper", "name": "can_manipulate", "target": "chairnut1"}, {"source": "inwardgripper", "name": "can_manipulate", "target": "chairnut2"}, {"source": "defaultgripper", "name": "can_manipulate", "target": "chairleg1"}, {"source": "defaultgripper", "name": "can_manipulate", "target": "chairleg2"}, {"source": "defaultgripper", "name": "can_manipulate", "target": "chairseat"}, {"source": "chairnut1", "name": "can_screw_to", "target": "chairseatbolt1"}, {"source": "chairnut2", "name": "can_screw_to", "target": "chairseatbolt2"}, {"source": "chairback", "name": "can_insert_to", "target": "chairseatconnection"}, {"source": "chairleg1", "name": "can_screw_to", "target": "chairseatthread1"}, {"source": "chairleg2", "name": "can_screw_to", "target": "chairseatthread2"}], "relations": [{"source": "left_hand", "name": "hold", "target": "defaultgripper"}, {"source": "defaultgripper", "name": "hold", "target": "chairleg1"}]}, "result": {"summary": "selector to screw the chairnut1 into the chairseatbolt1 with the inwardgripper in the left_hand", "name": "selector: screw(left_hand, inwardgripper, chairnut1, chairseatbolt1)", "children": [{"summary": "the target is to make the chairnut1 be screwed into the chairseatbolt1", "name": "target: is_screwed_to(chairnut1, chairseatbolt1)"}, {"summary": "sequence to screw the chairnut1 into the chairseatbolt1 with the inwardgripper in the left_hand", "name": "sequence: screw(left_hand, inwardgripper, chairnut1, chairseatbolt1)", "children": [{"summary": "selector to change tool in left_hand from defaultgripper to inwardgripper", "name": "selector: change_tool(left_hand, defaultgripper, inwardgripper)", "children": [{"summary": "the target is to make left_hand hold inwardgripper", "name": "target: hold(left_hand, inwardgripper)"}, {"summary": "sequence to change tool in left_hand from defaultgripper to inwardgripper", "name": "sequence: change_tool(left_hand, defaultgripper, inwardgripper)", "children": [{"summary": "a precondition is left_hand is holding defaultgripper", "name": "precondition: hold(left_hand, defaultgripper)"}, {"summary": "selector to put down the chairleg1 held by the defaultgripper in the left_hand", "name": "selector: put_down(left_hand, defaultgripper, chairleg1)", "children": [{"summary": "the target is to make the defaultgripper empty", "name": "target: is_empty(defaultgripper)"}, {"summary": "sequence to put down the chairleg1 held by the defaultgripper in the left_hand", "name": "sequence: put_down(left_hand, defaultgripper, chairleg1)", "children": [{"summary": "a precondition is the left_hand is holding defaultgripper", "name": "precondition: hold(left_hand, defaultgripper)"}, {"summary": "a precondition is the defaultgripper is holding chairleg1", "name": "precondition: hold(defaultgripper, chairleg1)"}, {"summary": "the action to put down the chairleg1 held by the defaultgripper in the left_hand", "name": "action: put_down(left_hand, defaultgripper, chairleg1)"}]}]}, {"summary": "the action to change tool in left_hand from defaultgripper to inwardgripper", "name": "action: change_tool(left_hand, defaultgripper, inwardgripper)"}]}]}, {"summary": "selector to pick up the chairnut1 with the inwardgripper in the left_hand", "name": "selector: pick_up(left_hand, inwardgripper, chairnut1)", "children": [{"summary": "the target is to make the inwardgripper hold the chairnut1", "name": "target: hold(inwardgripper, chairnut1)"}, {"summary": "sequence to pick up the chairnut1 with the inwardgripper in the left_hand", "name": "sequence: pick_up(left_hand, inwardgripper, chairnut1)", "children": [{"summary": "a precondition is the inwardgripper is empty", "name": "precondition: is_empty(inwardgripper)"}, {"summary": "a precondition is the left_hand is holding inwardgripper", "name": "precondition: hold(left_hand, inwardgripper)"}, {"summary": "the action to pick up the chairnut1 with the inwardgripper in the left_hand", "name": "action: pick_up(left_hand, inwardgripper, chairnut1)"}]}]}, {"summary": "the action to screw the chairnut1 into the chairseatbolt1 with the inwardgripper in the left_hand", "name": "action: screw(left_hand, inwardgripper, chairnut1, chairseatbolt1)"}]}]}} -{"target": "target: is_screwed_to(chairnut1, chairseatbolt1)", "initial_world_state": {"objects": [{"name": "clampgripper", "properties": ["is_empty"]}, {"name": "parallelgripper", "properties": ["is_empty"]}, {"name": "inwardgripper", "properties": ["is_empty"]}, {"name": "outwardgripper", "properties": ["is_empty"]}, {"name": "defaultgripper", "properties": ["is_empty"]}, {"name": "chairnut1", "properties": []}, {"name": "chairnut2", "properties": []}, {"name": "chairseat", "properties": []}, {"name": "chairseatbolt1", "properties": []}, {"name": "chairseatbolt2", "properties": []}, {"name": "chairseatconnector", "properties": []}, {"name": "chairseatthread1", "properties": []}, {"name": "chairseatthread2", "properties": []}, {"name": "chairback", "properties": []}, {"name": "chairleg1", "properties": []}, {"name": "chairleg2", "properties": []}, {"name": "left_hand", "properties": []}], "constraints": [{"source": "clampgripper", "name": "can_manipulate", "target": "chairback"}, {"source": "inwardgripper", "name": "can_manipulate", "target": "chairnut1"}, {"source": "inwardgripper", "name": "can_manipulate", "target": "chairnut2"}, {"source": "defaultgripper", "name": "can_manipulate", "target": "chairleg1"}, {"source": "defaultgripper", "name": "can_manipulate", "target": "chairleg2"}, {"source": "defaultgripper", "name": "can_manipulate", "target": "chairseat"}, {"source": "chairnut1", "name": "can_screw_to", "target": "chairseatbolt1"}, {"source": "chairnut2", "name": "can_screw_to", "target": "chairseatbolt2"}, {"source": "chairback", "name": "can_insert_to", "target": "chairseatconnector"}, {"source": "chairleg1", "name": "can_screw_to", "target": "chairseatthread1"}, {"source": "chairleg2", "name": "can_screw_to", "target": "chairseatthread2"}], "relations": [{"source": "left_hand", "name": "hold", "target": "clampgripper"}]}, "result": {"summary": "selector to screw the chairnut1 into the chairseatbolt1 with the inwardgripper in the left_hand", "name": "selector: screw(left_hand, inwardgripper, chairnut1, chairseatbolt1)", "children": [{"summary": "the target is to make the chairnut1 be screwed into the chairseatbolt1", "name": "target: is_screwed_to(chairnut1, chairseatbolt1)"}, {"summary": "sequence to screw the chairnut1 into the chairseatbolt1 with the inwardgripper in the left_hand", "name": "sequence: screw(left_hand, inwardgripper, chairnut1, chairseatbolt1)", "children": [{"summary": "selector to change tool in left_hand from clampgripper to inwardgripper", "name": "selector: change_tool(left_hand, clampgripper, inwardgripper)", "children": [{"summary": "the target is to make left_hand hold inwardgripper", "name": "target: hold(left_hand, inwardgripper)"}, {"summary": "sequence to change tool in left_hand from clampgripper to inwardgripper", "name": "sequence: change_tool(left_hand, clampgripper, inwardgripper)", "children": [{"summary": "a precondition is left_hand is holding clampgripper", "name": "precondition: hold(left_hand, clampgripper)"}, {"summary": "a precondition is clampgripper is empty", "name": "precondition: is_empty(clampgripper)"}, {"summary": "the action to change tool in left_hand from clampgripper to inwardgripper", "name": "action: change_tool(left_hand, clampgripper, inwardgripper)"}]}]}, {"summary": "selector to pick up the chairnut1 with the inwardgripper in the left_hand", "name": "selector: pick_up(left_hand, inwardgripper, chairnut1)", "children": [{"summary": "the target is to make the inwardgripper hold the chairnut1", "name": "target: hold(inwardgripper, chairnut1)"}, {"summary": "sequence to pick up the chairnut1 with the inwardgripper in the left_hand", "name": "sequence: pick_up(left_hand, inwardgripper, chairnut1)", "children": [{"summary": "a precondition is the inwardgripper is empty", "name": "precondition: is_empty(inwardgripper)"}, {"summary": "a precondition is the left_hand is holding inwardgripper", "name": "precondition: hold(left_hand, inwardgripper)"}, {"summary": "the action to pick up the chairnut1 with the inwardgripper in the left_hand", "name": "action: pick_up(left_hand, inwardgripper, chairnut1)"}]}]}, {"summary": "the action to screw the chairnut1 into the chairseatbolt1 with the inwardgripper in the left_hand", "name": "action: screw(left_hand, inwardgripper, chairnut1, chairseatbolt1)"}]}]}} -{"target": "target: is_screwed_to(chairleg1, chairseatthread1)", "initial_world_state": {"objects": [{"name": "clampgripper", "properties": ["is_empty"]}, {"name": "parallelgripper", "properties": ["is_empty"]}, {"name": "inwardgripper", "properties": ["is_empty"]}, {"name": "outwardgripper", "properties": ["is_empty"]}, {"name": "defaultgripper", "properties": ["is_empty"]}, {"name": "chairnut1", "properties": []}, {"name": "chairnut2", "properties": []}, {"name": "chairseat", "properties": []}, {"name": "chairseatbolt1", "properties": []}, {"name": "chairseatbolt2", "properties": []}, {"name": "chairseatconnector", "properties": []}, {"name": "chairseatthread1", "properties": []}, {"name": "chairseatthread2", "properties": []}, {"name": "chairback", "properties": []}, {"name": "chairleg1", "properties": []}, {"name": "chairleg2", "properties": []}, {"name": "left_hand", "properties": []}], "constraints": [{"source": "clampgripper", "name": "can_manipulate", "target": "chairback"}, {"source": "inwardgripper", "name": "can_manipulate", "target": "chairnut1"}, {"source": "inwardgripper", "name": "can_manipulate", "target": "chairnut2"}, {"source": "defaultgripper", "name": "can_manipulate", "target": "chairleg1"}, {"source": "defaultgripper", "name": "can_manipulate", "target": "chairleg2"}, {"source": "defaultgripper", "name": "can_manipulate", "target": "chairseat"}, {"source": "chairnut1", "name": "can_screw_to", "target": "chairseatbolt1"}, {"source": "chairnut2", "name": "can_screw_to", "target": "chairseatbolt2"}, {"source": "chairback", "name": "can_insert_to", "target": "chairseatconnector"}, {"source": "chairleg1", "name": "can_screw_to", "target": "chairseatthread1"}, {"source": "chairleg2", "name": "can_screw_to", "target": "chairseatthread2"}], "relations": [{"source": "left_hand", "name": "hold", "target": "defaultgripper"}]}, "result": {"summary": "selector to screw the chairleg1 into the chairseatthread1 with the defaultgripper in the left_hand", "name": "selector: screw(left_hand, defaultgripper, chairleg1, chairseatthread1)", "children": [{"summary": "the target is to make the chairleg1 be screwed into the chairseatthread1", "name": "target: is_screwed_to(chairleg1, chairseatthread1)"}, {"summary": "sequence to screw the chairleg1 into the chairseatthread1 with the defaultgripper in the left_hand", "name": "sequence: screw(left_hand, defaultgripper, chairleg1, chairseatthread1)", "children": [{"summary": "a precondition is the left_hand is holding defaultgripper", "name": "precondition: hold(left_hand, defaultgripper)"}, {"summary": "selector to pick up the chairleg1 with the defaultgripper in the left_hand", "name": "selector: pick_up(left_hand, defaultgripper, chairleg1)", "children": [{"summary": "the target is to make the defaultgripper hold the chairleg1", "name": "target: hold(defaultgripper, chairleg1)"}, {"summary": "sequence to pick up the chairleg1 with the defaultgripper in the left_hand", "name": "sequence: pick_up(left_hand, defaultgripper, chairleg1)", "children": [{"summary": "a precondition is the defaultgripper is empty", "name": "precondition: is_empty(defaultgripper)"}, {"summary": "a precondition is the left_hand is holding defaultgripper", "name": "precondition: hold(left_hand, defaultgripper)"}, {"summary": "the action to pick up the chairleg1 with the defaultgripper in the left_hand", "name": "action: pick_up(left_hand, defaultgripper, chairleg1)"}]}]}, {"summary": "the action to screw the chairleg1 into the chairseatthread1 with the defaultgripper in the left_hand", "name": "action: screw(left_hand, defaultgripper, chairleg1, chairseatthread1)"}]}]}} diff --git a/experiments/chair/baseline_result_handle.py b/experiments/chair/baseline_result_handle.py deleted file mode 100644 index 17d76784..00000000 --- a/experiments/chair/baseline_result_handle.py +++ /dev/null @@ -1,22 +0,0 @@ -import json -import os -import json -import os - -file_dir = os.path.dirname(os.path.abspath(__file__)) -result_dir = os.path.join(file_dir, "baseline_result.jsonl") - -if not os.path.exists(os.path.join(file_dir, "problem_set")): - os.makedirs(os.path.join(file_dir, "problem_set")) - -with open(result_dir, "r") as file: - results = file.read() - data = [json.loads(line) for line in results.splitlines()] - -i = 0 -for item in data: - i_str = str(i).zfill(3) - entity_path = os.path.join(file_dir, "problem_set", f"problem_{i_str}.json") - with open(entity_path, "w") as file: - file.write(json.dumps(item, indent=4)) - i += 1 diff --git a/experiments/chair/behavior_tree.json b/experiments/chair/behavior_tree.json deleted file mode 100644 index ba968b94..00000000 --- a/experiments/chair/behavior_tree.json +++ /dev/null @@ -1,232 +0,0 @@ -{ - "summary": "selector to insert shaft1 into gearbase_hole1", - "name": "selector: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)", - "identifier": 1, - "type_name": "selector", - "children": [ - { - "summary": "condition node to check if shaft1 is inserted to gearbase_hole1", - "name": "target: is_inserted_to(shaft1, gearbase_hole1)", - "identifier": 2, - "type_name": "condition", - "conditions": [ - { - "object_name": "shaft1", - "property_name": "is_inserted_to", - "property_value": "gearbase_hole1", - "status": true - } - ] - }, - { - "summary": "sequence to insert shaft1 into gearbase_hole1", - "name": "sequence: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)", - "identifier": 3, - "type_name": "sequence", - "children": [ - { - "summary": "selector to load parallel_box1", - "name": "selector: load_tool(left_hand, parallel_box1)", - "identifier": 4, - "type_name": "selector", - "children": [ - { - "summary": "condition node to check if left_hand holds parallel_box1", - "name": "target: hold(left_hand, parallel_box1)", - "identifier": 5, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "parallel_box1", - "status": true - } - ] - }, - { - "summary": "sequence to load parallel_box1", - "name": "sequence: load_tool(left_hand, parallel_box1)", - "identifier": 6, - "type_name": "sequence", - "children": [ - { - "summary": "condition node to check if parallel_box1 is equippable", - "name": "precondition: is_equippable(parallel_box1)", - "identifier": 7, - "type_name": "condition", - "conditions": [ - { - "object_name": "parallel_box1", - "property_name": "is_equippable", - "property_value": null, - "status": true - } - ] - }, - { - "summary": "condition node to check if left hand is free", - "name": "precondition: is_free(left_hand)", - "identifier": 8, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "is_free", - "property_value": null, - "status": true - } - ] - }, - { - "summary": "action node to equip parallel_box1 to left hand", - "name": "action: load_tool(left_hand, parallel_box1)", - "identifier": 9, - "type_name": "action", - "effects": [ - { - "object_name": "left_hand", - "property_name": "is_free", - "property_value": null, - "status": false - }, - { - "object_name": "parallel_box1", - "property_name": "is_equippable", - "property_value": null, - "status": false - }, - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "parallel_box1", - "status": true - } - ] - } - ] - } - ] - }, - { - "summary": "selector to use left_hand with parallel_box1 to pick up shaft1", - "name": "selector: pick_up(left_hand, parallel_box1, shaft1)", - "identifier": 10, - "type_name": "selector", - "children": [ - { - "summary": "condition node to check if parallel_box1 holds shaft1", - "name": "target: hold(parallel_box1, shaft1)", - "identifier": 11, - "type_name": "condition", - "conditions": [ - { - "object_name": "parallel_box1", - "property_name": "hold", - "property_value": "shaft1", - "status": true - } - ] - }, - { - "summary": "sequence to use left hand with parallel_box1 to pick up shaft1", - "name": "sequence: pick_up(left_hand, parallel_box1, shaft1)", - "identifier": 12, - "type_name": "sequence", - "children": [ - { - "summary": "condition node to check if parallel_box1 is free", - "name": "precondition: is_free(parallel_box1)", - "identifier": 13, - "type_name": "condition", - "conditions": [ - { - "object_name": "parallel_box1", - "property_name": "is_free", - "property_value": null, - "status": true - } - ] - }, - { - "summary": "condition node to check if left hand holds parallel_box1", - "name": "precondition: hold(left_hand, parallel_box1)", - "identifier": 14, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "parallel_box1", - "status": true - } - ] - }, - { - "summary": "action node to use left hand with parallel_box1 to pick up shaft1", - "name": "action: pick_up(left_hand, parallel_box1, shaft1)", - "identifier": 15, - "type_name": "action", - "effects": [ - { - "object_name": "parallel_box1", - "property_name": "hold", - "property_value": "shaft1", - "status": true - }, - { - "object_name": "parallel_box1", - "property_name": "is_free", - "property_value": null, - "status": false - } - ] - } - ] - } - ] - }, - { - "summary": "condition node to check if shaft1 can be inserted to gearbase_hole1", - "name": "precondition: can_insert_to(shaft1, gearbase_hole1)", - "identifier": 16, - "type_name": "condition", - "conditions": [ - { - "object_name": "shaft1", - "property_name": "can_insert_to", - "property_value": "gearbase_hole1", - "status": true - } - ] - }, - { - "summary": "action node to use left_hand with parallel_box1 to insert shaft1 to gearbase_hole1", - "name": "action: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)", - "identifier": 17, - "type_name": "action", - "effects": [ - { - "object_name": "parallel_box1", - "property_name": "hold", - "property_value": "shaft1", - "status": false - }, - { - "object_name": "parallel_box1", - "property_name": "is_free", - "property_value": null, - "status": true - }, - { - "object_name": "shaft1", - "property_name": "is_inserted_to", - "property_value": "gearbase_hole1", - "status": true - } - ] - } - ] - } - ] - } \ No newline at end of file diff --git a/experiments/chair/domain_knowledge.txt b/experiments/chair/domain_knowledge.txt deleted file mode 100644 index ba5c088b..00000000 --- a/experiments/chair/domain_knowledge.txt +++ /dev/null @@ -1,88 +0,0 @@ -[user] -Following is the domain knowledge you need in pddl domain format, which includes: -- the object types in the world, -- the predicates for describing the states of the world, -- the actions that the robot can apply, with their precondtions and effects defined. -""" -(define (domain robot_assembly_problem-domain) - (:requirements :strips :typing) - (:types - thing - object - tool part hand - thing - ) - (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part)) - (:action pick_up - :parameters ( ?hand - hand ?tool - tool ?part - part) - :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part)) - :effect (and (hold ?tool ?part) (not (is_free ?tool)))) - (:action put_down - :parameters ( ?hand - hand ?tool - tool ?part - part) - :precondition (and (hold ?tool ?part) (hold ?hand ?tool)) - :effect (and (not (hold ?hand ?part)) (is_free ?tool))) - (:action place - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2))) - (:action detach - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2)))) - (:action insert - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2))) - (:action pull - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2)))) - (:action screw - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2))) - (:action unscrew - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2)))) - (:action load_tool - :parameters ( ?hand - hand ?tool - tool) - :precondition (and (is_equippable ?tool) (is_free ?hand)) - :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool))) - (:action unload_tool - :parameters ( ?hand - hand ?tool - tool) - :precondition (and (hold ?hand ?tool) (is_free ?tool)) - :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool)))) -) -""" -The precidates are explained as follows: -""" -"PREDICATE_LIST" -- is_free( or ): or is free and can hold something. (default: true) -- is_equippable(): is ready to be eqquipped by a robot hand. (default: true) -- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false) -- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false) -- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false) -- can_place_to(, ): can be placed into (placement assembly constraint) (default: false) -- hold(, ): is holding (default: false) -- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false) -- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false) -- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false) -""" - -The actions are explained as follows: -""" -"ROBOT_ACTION_LIST" -- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something. -- put_down(, , ): use to put down . As preconditions, should be holding and should be holding . -- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding . -- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into . -- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding . -- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into . -- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding . -- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into . -- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free. -- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free. -""" -the items in <> indicate the type of the instances needed for the predicates and actions. - -[assistant] -Understood. I will wait for further instructions before starting to work. diff --git a/experiments/chair/gearset.domain b/experiments/chair/gearset.domain deleted file mode 100644 index 39c677b0..00000000 --- a/experiments/chair/gearset.domain +++ /dev/null @@ -1,48 +0,0 @@ -(define (domain robot_assembly_problem-domain) - (:requirements :strips :typing) - (:types - thing - object - tool part hand - thing - ) - (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part)) - (:action pick_up - :parameters ( ?hand - hand ?tool - tool ?part - part) - :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part)) - :effect (and (hold ?tool ?part) (not (is_free ?tool)))) - (:action put_down - :parameters ( ?hand - hand ?tool - tool ?part - part) - :precondition (and (hold ?tool ?part) (hold ?hand ?tool)) - :effect (and (not (hold ?hand ?part)) (is_free ?tool))) - (:action place - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2))) - (:action detach - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2)))) - (:action insert - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2))) - (:action pull - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2)))) - (:action screw - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2))) - (:action unscrew - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2)))) - (:action load_tool - :parameters ( ?hand - hand ?tool - tool) - :precondition (and (is_equippable ?tool) (is_free ?hand)) - :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool))) - (:action unload_tool - :parameters ( ?hand - hand ?tool - tool) - :precondition (and (hold ?hand ?tool) (is_free ?tool)) - :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool)))) -) \ No newline at end of file diff --git a/experiments/chair/gearset1 copy.py b/experiments/chair/gearset1 copy.py deleted file mode 100644 index cd823828..00000000 --- a/experiments/chair/gearset1 copy.py +++ /dev/null @@ -1,89 +0,0 @@ -import json -import os - -from kios_bt.bt_stewardship import BehaviorTreeStewardship -from kios_scene.scene_factory import SceneFactory -from kios_bt.bt_factory import BehaviorTreeFactory -from kios_robot.robot_interface import RobotInterface -from kios_world.world_interface import WorldInterface -from kios_utils.pybt_test import ( - generate_bt_stewardship, - render_dot_tree, - tick_loop_test, - tick_1000HZ_test, - tick_frequency_test, -) - - -def core_loop(): - ####################### dirs - current_dir = os.path.dirname(os.path.abspath(__file__)) - scene_json_file_path = os.path.join(current_dir, "gearset_scene.json") - bt_json_file_path = os.path.join(current_dir, "behavior_tree.json") - world_state_json_file_path = os.path.join(current_dir, "world_state.json") - - ####################### scene - with open(scene_json_file_path, "r") as file: - json_object = json.load(file) - - gearset_scene = SceneFactory().create_scene_from_json(json_object) - - ####################### world - world_interface = WorldInterface() - with open(world_state_json_file_path, "r") as file: - json_object = json.load(file) - world_interface.load_world_from_json(json_object) - - ####################### robot - robot_interface = RobotInterface( - robot_address="127.0.0.1", - robot_port=12000, - ) - robot_interface.setup_scene(gearset_scene) - - ####################### agent - # * use the generated plan instead - - ####################### bt - bt_factory = BehaviorTreeFactory( - world_interface=world_interface, - robot_interface=robot_interface, - ) - - ####################### * old code - # behavior_tree = None - # with open(bt_json_file_path, "r") as file: - # json_object = json.load(file) - # behavior_tree = bt_factory.from_json_to_bt(json_object) - - # bt_steward = generate_bt_stewardship(behavior_tree) - # render_dot_tree(bt_steward) - # # tick_loop_test(bt_steward) - # tick_frequency_test(bt_steward) - - # * new code - behavior_tree_stewardship = BehaviorTreeStewardship( - behaviortree_factory=bt_factory, - world_interface=world_interface, - robot_interface=robot_interface, - ) - - with open(bt_json_file_path, "r") as file: - json_object = json.load(file) - behavior_tree_stewardship.load_bt_json(json_object) - - behavior_tree_stewardship.generate_behavior_tree() - behavior_tree_stewardship.setup_behavior_tree() - behavior_tree_stewardship.render_dot_tree() - - # ! return the tree result - behavior_tree_stewardship.tick_loop_test() - - # sim_result = behavior_tree_stewardship.simulate_behavior_tree() - # print(sim_result.to_json()) - - # ! world state has problem. gear3 is not free! - - -if __name__ == "__main__": - core_loop() diff --git a/experiments/chair/gearset1_graph copy.py b/experiments/chair/gearset1_graph copy.py deleted file mode 100644 index 04f9d5b5..00000000 --- a/experiments/chair/gearset1_graph copy.py +++ /dev/null @@ -1,522 +0,0 @@ -import json -import os - -from kios_bt.bt_stewardship import BehaviorTreeStewardship -from kios_scene.scene_factory import SceneFactory -from kios_bt.bt_factory import BehaviorTreeFactory -from kios_robot.robot_interface import RobotInterface -from kios_world.world_interface import WorldInterface - -from kios_agent.llm_supporter import KiosLLMSupporter -from kios_agent.data_types import KiosPromptSkeleton - -from kios_agent.kios_tools import ( - BehaviorTreeExecutorTool, - BehaviorTreeSimulatorTool, - WorldStateQueryTool, -) - -from langchain_core.pydantic_v1 import BaseModel, Field -from typing import List, Tuple, Annotated, TypedDict -import operator - -from dotenv import load_dotenv - -from langchain import hub -from langchain.agents import create_openai_functions_agent, AgentExecutor -from langchain_openai import ChatOpenAI -from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder -from langchain_core.output_parsers import JsonOutputParser, StrOutputParser - -from langchain.chains.openai_functions import ( - create_structured_output_runnable, - create_openai_fn_runnable, -) -from langchain_core.utils.function_calling import convert_to_openai_function - -from langchain.agents.format_scratchpad import format_to_openai_function_messages - -from langchain_core.agents import AgentActionMessageLog, AgentFinish - -from langgraph.graph import StateGraph, END - -from langchain_core.messages import HumanMessage - - -load_dotenv() - -from langsmith import traceable - -os.environ["LANGCHAIN_TRACING_V2"] = "true" -os.environ["LANGCHAIN_ENDPOINT"] = "https://api.smith.langchain.com" -os.environ["LANGCHAIN_PROJECT"] = "kios_agent" - - -####################### dirs -current_dir = os.path.dirname(os.path.abspath(__file__)) -scene_path = os.path.join(current_dir, "scene.json") -# bt_json_file_path = os.path.join(current_dir, "behavior_tree.json") -world_state_path = os.path.join(current_dir, "world_state.json") -problem_path = os.path.join(current_dir, "gearset.problem") - - -####################### problem -with open(problem_path, "r") as file: - problem = file.read() - -####################### scene -with open(scene_path, "r") as file: - scene_json_object = json.load(file) - -scene = SceneFactory().create_scene_from_json(scene_json_object) - -####################### world -world_interface = WorldInterface() -with open(world_state_path, "r") as file: - world_state_json_object = json.load(file) - world_interface.load_world_from_json(world_state_json_object) - -####################### robot -robot_interface = RobotInterface( - robot_address="127.0.0.1", - robot_port=12000, -) -robot_interface.setup_scene(scene) - -####################### bt_factory -bt_factory = BehaviorTreeFactory( - world_interface=world_interface, - robot_interface=robot_interface, -) - -####################### behavior_tree_stewardship -behavior_tree_stewardship = BehaviorTreeStewardship( - behaviortree_factory=bt_factory, - world_interface=world_interface, - robot_interface=robot_interface, -) - -####################### * tools -# bt_executor_tool = BehaviorTreeExecutorTool(behavior_tree_stewardship) - -bt_sim_tool = BehaviorTreeSimulatorTool(metadata={"bt_stw": behavior_tree_stewardship}) - -# world_state_query_tool = WorldStateQueryTool(behavior_tree_stewardship) - -executor_tools = [bt_sim_tool] - -# * kios data prompt skeleton dir -data_dir = os.environ.get("KIOS_DATA_DIR").format(username=os.getlogin()) -print(data_dir) -prompt_sk_dir = os.path.join(data_dir, "prompt_skeletons") - -# * refiner chain -llm_spt_re_sk = KiosLLMSupporter() -with open(os.path.join(prompt_sk_dir, "skeleton_refiner.json"), "r") as f: - re_sk_pptsk_json = json.load(f) - re_sk_pptsk = KiosPromptSkeleton.from_json(re_sk_pptsk_json) - llm_spt_re_sk.initialize_from_prompt_skeleton(re_sk_pptsk) -re_sk_ppt = llm_spt_re_sk.create_prompt() -re_sk_llm = ChatOpenAI(model_name=llm_spt_re_sk.model_name, temperature=0) - -re_sk_chain = re_sk_ppt | re_sk_llm | JsonOutputParser() - - -# * skeleton generator chain -llm_spt_skeleton_generator = KiosLLMSupporter() -with open(os.path.join(prompt_sk_dir, "skeleton_generator.json"), "r") as f: - skeleton_generator_pptsk_json = json.load(f) - skeleton_generator_pptsk = KiosPromptSkeleton.from_json( - skeleton_generator_pptsk_json - ) - llm_spt_skeleton_generator.initialize_from_prompt_skeleton(skeleton_generator_pptsk) -skeleton_generator_ppt = llm_spt_skeleton_generator.create_prompt() -skeleton_generator_llm = ChatOpenAI( - model_name=llm_spt_skeleton_generator.model_name, temperature=0 -) - -skeleton_generator_chain = ( - skeleton_generator_ppt - | skeleton_generator_llm - # | StrOutputParser() - | JsonOutputParser() -) - -# ! DISCARDED behavior tree generator chain -# bt_gen_chain = skeleton_generator_chain | re_sk_chain - -############################################### * behavior tree executor agent -# ! NOW SUBSTITUTED BY THE FAKE EXECUTOR FUNCTION -# bt_exe_ppt = hub.pull("hwchase17/openai-functions-agent") -# bt_exe_ppt = ChatPromptTemplate.from_messages( -# [ -# ("system", "You are a helpful assistant"), -# ("user", "The behavior tree is: {behavior_tree}"), -# ("user", "The world state is: {world_state}"), -# MessagesPlaceholder(variable_name="agent_scratchpad"), -# ] -# ) - -# bt_exe_llm = ChatOpenAI(model="gpt-3.5-turbo", temperature=0) - - -# def executor_agent_parse(output): -# # If no function was invoked, return to user -# if "function_call" not in output.additional_kwargs: -# return AgentFinish(return_values={"output": output.content}, log=output.content) - -# # Parse out the function call -# function_call = output.additional_kwargs["function_call"] -# name = function_call["name"] -# inputs = json.loads(function_call["arguments"]) - -# # If the Response function was invoked, return to the user with the function inputs -# if name == "ExecutorResponse": -# return AgentFinish(return_values=inputs, log=str(function_call)) -# # Otherwise, return an agent action -# else: -# return AgentActionMessageLog( -# tool=name, tool_input=inputs, log="", message_log=[output] -# ) - - -# class ExecutorResponse(BaseModel): -# """the result of the behavior tree execution""" - -# hasSucceeded: bool = Field(description="If the behavior tree has succeeded or not") -# world_state: dict[str, list[dict[str, str]]] = Field( -# description="The final world state after the execution" -# ) - - -# bt_exe_llm_with_tools = bt_exe_llm.bind_functions( -# [convert_to_openai_function(t) for t in executor_tools] -# ) - - -# ! BUG -# bt_exe_agent = ( -# { -# "input": lambda x: x["input"], -# # Format agent scratchpad from intermediate steps -# "agent_scratchpad": lambda x: format_to_openai_function_messages( -# x["intermediate_steps"] -# ), -# } -# | bt_exe_ppt -# | bt_exe_llm_with_tools -# | executor_agent_parse -# ) - -# bt_exe_agent_executor = AgentExecutor( -# tools=executor_tools, agent=bt_exe_agent, verbose=True -# ) - - -# * the graph state -class PlanExecuteState(TypedDict): - plan: List[str] - behavior_tree_skeleton: dict # ! not sure if use this or not. - behavior_tree: dict - world_state: Annotated[ - List[dict], operator.add - ] # ! to add, you need to make world_state a list of dict - # = Field(default=None) - past_steps: Annotated[List[Tuple], operator.add] - response: str # ! not sure if use this or not. - # to_user: bool - user_input: str - problem: str - - -##################################################### * planner -# * output schema of the planner -class Plan(BaseModel): - """Plan to follow in future""" - - steps: List[str] = Field( - description="a list of different steps to follow, should be in sorted order" - ) - - -planner_prompt = ChatPromptTemplate.from_template( - """ - The user input should be an instruction (or several instructions) of the robot assembly task. \ - For the given user input, summarize and come up with a bullet list task plan. \ -This plan should involve individual assembly step(s) of robot assembly task.\ -Do not add any superfluous steps. \ -Make sure that each step has all the information needed - do not skip steps.\ - -You should only use assembly actions as the action in the steps. -The available assembly actions are:\ -- insert something into something\ -- place something onto something\ -- screw something into something\ - -Example:\ -user_input: "you should insert the red block into the blue hole"\ -plan: "insert the red block into the blue hole"\ - -user_input: "intert the blub into the base, then put the cover onto it."\ -plan: ["insert the blub into the base", "put the cover onto the blub"]\ - -world_state: {world_state}\ -user_input: {user_input}\ -""" -) -planner = create_structured_output_runnable( - Plan, ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), planner_prompt -) - - -##################################################### * plan_updaterner - - -class UpdaterResponse(BaseModel): - """used to response to the user for asking for more inputs.""" - - response: str - - -plan_updater_prompt = ChatPromptTemplate.from_template( - """For the given objective, come up with a simple step by step plan. \ -This plan should involve individual tasks, that if executed correctly will yield the correct answer. Do not add any superfluous steps. \ -The result of the final step should be the final answer. Make sure that each step has all the information needed - do not skip steps. - -Your objective was this: -{user_input} - -Your original plan was this: -{plan} - -You have currently done the follow steps: -{past_steps} - -The world state is: -{world_state} - -Read the world state. Check the original plan from the first step and update your plan with the steps need to be conducted next according to the world state.\ -If no more steps are needed to achieve the objective defined by user input, respond to the user for more input. Otherwise, fill out the plan.\ -Only add steps to the plan that still NEED to be done. Do not return previously done steps as part of the plan.""" -) - -plan_updater = create_openai_fn_runnable( - [Plan, UpdaterResponse], # * here two schemas are used - ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), - plan_updater_prompt, -) - - -##################################################### * graph node functions -async def user_input_step(state: PlanExecuteState): - """ - get the input from the user - """ - print(f"-----user_input_step-----") - if state["response"] is not None: - user_input = input(state["response"]) - state["response"] = None - else: - user_input = input("Your next instruction: ") - - return { - "user_input": user_input, - } - - -def user_input_should_end(state: PlanExecuteState): - """ - if the user input is empty, then end - """ - print(f"-----user_input_should_end-----") - - # print(f'user_input: {state["user_input"]}') - # print(f'world_state: {state["world_state"]}') - - if not state["user_input"] or state["user_input"] == "": - return True - else: - return False - - -async def behavior_tree_generate_step(state: PlanExecuteState): - """ - generate the behavior tree based on the instruction - """ - print(f"-----behavior_tree_generate_step-----") - - instruction = state["plan"][0] - latest_world_state = state["world_state"][-1] - - skeleton = await skeleton_generator_chain.ainvoke( - { - "problem": state["problem"], - "world_state": latest_world_state, - "instructions": instruction, - } - ) - - response = await re_sk_chain.ainvoke( - { - "input": skeleton, - } - ) - - behavior_tree = response.get("task_plan").get("behavior_tree") - - init_world_state = [response.get("initial_state")] - - return { - "behavior_tree": behavior_tree, - "world_state": init_world_state, # * this is necessary because the constraints in problem - } - - -async def behavior_tree_execute_step(state: PlanExecuteState): - """ - execute the first step of the plan, append the result to the past steps - """ - print(f"-----behavior_tree_execute_step-----") - - behavior_tree = state["behavior_tree"] - latest_world_state = state["world_state"][-1] - - # exe_input = { - # "behavior_tree": behavior_tree, - # "world_state": latest_world_state, - # } - - # agent_response = await bt_exe_agent_executor.ainvoke( - # { - # "input": exe_input, - # } - # ) - - response = behavior_tree_stewardship.fake_run( - world_state=latest_world_state, bt_json=behavior_tree - ) - - # hasSucceeded = response["hasSucceeded"] - # new_world_state = [response["world_state"]] - hasSucceeded = response.result - new_world_state = [response.world_state] - the_step = state["plan"][0] - - return { - "past_steps": (the_step, hasSucceeded), # ! why here not a list of tuple? - "world_state": new_world_state, - } - - -async def plan_step(state: PlanExecuteState): - """ - plan the steps based on user input and world state - """ - print(f"-----plan_step-----") - - plan = await planner.ainvoke( - { - "user_input": state["user_input"], - "world_state": state["world_state"], - } - ) - return {"plan": plan.steps} - - -async def plan_updater_step(state: PlanExecuteState): - """ - if return a response, then success, response the use, end. - otherwise, return the updated newplan (normally the same as the old plan with the first step popped out. - """ - print(f"-----plan_updater_step-----") - - output = await plan_updater.ainvoke(state) - if isinstance( - output, UpdaterResponse - ): # * determine if it is time to response and end - return { - "response": "Your last instruction has been finished." - } # * Don't need to update. - else: - return {"plan": output.steps} # * update the plan - - -def plan_updater_should_end(state: PlanExecuteState): - """ - end router - """ - print(f"-----plan_updater_should_end-----") - - if state["response"]: - return True - else: - return False - - -##################################################### * construct the graph - -workflow = StateGraph(PlanExecuteState) - -workflow.add_node("planner", plan_step) - -workflow.add_node("behavior_tree_generator", behavior_tree_generate_step) - -workflow.add_node("behavior_tree_executor", behavior_tree_execute_step) - -workflow.add_node("plan_updater", plan_updater_step) - -workflow.add_node("user_input_node", user_input_step) - -workflow.set_entry_point("user_input_node") - -# workflow.add_edge("user_input_node", "planner") - -workflow.add_edge( - "planner", "behavior_tree_generator" -) # ! may need to add a condition here - -workflow.add_edge("behavior_tree_generator", "behavior_tree_executor") - -workflow.add_edge("behavior_tree_executor", "plan_updater") - -workflow.add_conditional_edges( - "plan_updater", - plan_updater_should_end, - { - True: "user_input_node", - False: "behavior_tree_generator", - }, -) - -workflow.add_conditional_edges( - "user_input_node", - user_input_should_end, - { - True: END, - False: "planner", - }, -) - -app = workflow.compile() - - -config = {"recursion_limit": 50} - - -inputs = { - "world_state": [world_state_json_object], - "problem": problem, -} - - -async def core_run(): - async for event in app.astream(inputs, config=config): - for k, v in event.items(): - if k != "__end__": - print(v) - - -if __name__ == "__main__": - import asyncio - - asyncio.run(core_run()) diff --git a/experiments/chair/gearset1_graph.py b/experiments/chair/gearset1_graph.py deleted file mode 100644 index a877a619..00000000 --- a/experiments/chair/gearset1_graph.py +++ /dev/null @@ -1,581 +0,0 @@ -import json -import os - -os.environ["LANGCHAIN_TRACING_V2"] = "true" -os.environ["LANGCHAIN_ENDPOINT"] = "https://api.smith.langchain.com" -os.environ["LANGCHAIN_PROJECT"] = "kios_agent" - -from kios_bt.bt_stewardship import BehaviorTreeStewardship -from kios_scene.scene_factory import SceneFactory -from kios_bt.bt_factory import BehaviorTreeFactory -from kios_robot.robot_interface import RobotInterface -from kios_world.world_interface import WorldInterface - -from kios_agent.llm_supporter import KiosLLMSupporter -from kios_agent.data_types import KiosPromptSkeleton - -from kios_agent.kios_tools import ( - BehaviorTreeExecutorTool, - BehaviorTreeSimulatorTool, - WorldStateQueryTool, -) - -from typing import List, Tuple, Annotated, TypedDict -import operator - -from dotenv import load_dotenv - -from langchain import hub - -from langchain_openai import ChatOpenAI - -from langchain.chains.openai_functions import ( - create_structured_output_runnable, - create_openai_fn_runnable, -) -from langchain_core.utils.function_calling import convert_to_openai_function -from langchain_core.agents import AgentActionMessageLog, AgentFinish -from langchain_core.messages import HumanMessage -from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder -from langchain_core.output_parsers import JsonOutputParser, StrOutputParser -from langchain_core.pydantic_v1 import BaseModel, Field - -from langchain.agents.format_scratchpad import format_to_openai_function_messages -from langchain.agents import create_openai_functions_agent, AgentExecutor - -from langchain.prompts.pipeline import PipelinePromptTemplate -from langchain.prompts.prompt import PromptTemplate - -from langgraph.graph import StateGraph, END - -from langsmith import traceable - -load_dotenv() - -from kios_utils.pybt_test import generate_bt_stewardship, render_dot_tree - - -def test_bt(bt_json: json): - test_class = BehaviorTreeFactory() - bt = test_class.from_json_to_simple_bt(bt_json) - # bt = test_class.from_json_to_tree_root(bt_json) - bt_stewardship = generate_bt_stewardship(bt) - # bt_stewardship.setup(timeout=15) - render_dot_tree(bt_stewardship) - - -####################### dirs -current_dir = os.path.dirname(os.path.abspath(__file__)) -scene_path = os.path.join(current_dir, "scene.json") -# bt_json_file_path = os.path.join(current_dir, "behavior_tree.json") -world_state_path = os.path.join(current_dir, "world_state.json") -problem_path = os.path.join(current_dir, "gearset.problem") - - -####################### problem -with open(problem_path, "r") as file: - problem = file.read() - -####################### scene -with open(scene_path, "r") as file: - scene_json_object = json.load(file) - -scene = SceneFactory().create_scene_from_json(scene_json_object) - -####################### world -world_interface = WorldInterface() -with open(world_state_path, "r") as file: - world_state_json_object = json.load(file) - world_interface.load_world_from_json(world_state_json_object) - -####################### robot -robot_interface = RobotInterface( - robot_address="127.0.0.1", - robot_port=12000, -) -robot_interface.setup_scene(scene) - -####################### bt_factory -bt_factory = BehaviorTreeFactory( - world_interface=world_interface, - robot_interface=robot_interface, -) - -####################### behavior_tree_stewardship -behavior_tree_stewardship = BehaviorTreeStewardship( - behaviortree_factory=bt_factory, - world_interface=world_interface, - robot_interface=robot_interface, -) - -####################### * tools -# bt_executor_tool = BehaviorTreeExecutorTool(behavior_tree_stewardship) - -bt_sim_tool = BehaviorTreeSimulatorTool(metadata={"bt_stw": behavior_tree_stewardship}) - -# world_state_query_tool = WorldStateQueryTool(behavior_tree_stewardship) - -executor_tools = [bt_sim_tool] - -# * kios data prompt skeleton dir -data_dir = os.environ.get("KIOS_DATA_DIR").format(username=os.getlogin()) -print(data_dir) -prompt_sk_dir = os.path.join(data_dir, "prompt_skeletons") -prompt_dir = os.path.join(data_dir, "prompts") - -# * refiner chain -llm_spt_re_sk = KiosLLMSupporter() -with open(os.path.join(prompt_sk_dir, "skeleton_refiner.json"), "r") as f: - re_sk_pptsk_json = json.load(f) - re_sk_pptsk = KiosPromptSkeleton.from_json(re_sk_pptsk_json) - llm_spt_re_sk.initialize_from_prompt_skeleton(re_sk_pptsk) -re_sk_ppt = llm_spt_re_sk.create_prompt() -re_sk_llm = ChatOpenAI(model_name=llm_spt_re_sk.model_name, temperature=0) - -re_sk_chain = re_sk_ppt | re_sk_llm | JsonOutputParser() - - -# * skeleton generator chain -llm_spt_skeleton_generator = KiosLLMSupporter() -with open(os.path.join(prompt_sk_dir, "skeleton_generator.json"), "r") as f: - skeleton_generator_pptsk_json = json.load(f) - skeleton_generator_pptsk = KiosPromptSkeleton.from_json( - skeleton_generator_pptsk_json - ) - llm_spt_skeleton_generator.initialize_from_prompt_skeleton(skeleton_generator_pptsk) -skeleton_generator_ppt = llm_spt_skeleton_generator.create_prompt() -skeleton_generator_llm = ChatOpenAI( - model_name=llm_spt_skeleton_generator.model_name, temperature=0 -) - -skeleton_generator_chain = ( - skeleton_generator_ppt - | skeleton_generator_llm - # | StrOutputParser() - | JsonOutputParser() -) - -# ! DISCARDED behavior tree generator chain -# bt_gen_chain = skeleton_generator_chain | re_sk_chain - -############################################### * behavior tree executor agent -# ! NOW SUBSTITUTED BY THE FAKE EXECUTOR FUNCTION -# bt_exe_ppt = hub.pull("hwchase17/openai-functions-agent") -# bt_exe_ppt = ChatPromptTemplate.from_messages( -# [ -# ("system", "You are a helpful assistant"), -# ("user", "The behavior tree is: {behavior_tree}"), -# ("user", "The world state is: {world_state}"), -# MessagesPlaceholder(variable_name="agent_scratchpad"), -# ] -# ) - -# bt_exe_llm = ChatOpenAI(model="gpt-3.5-turbo", temperature=0) - - -# def executor_agent_parse(output): -# # If no function was invoked, return to user -# if "function_call" not in output.additional_kwargs: -# return AgentFinish(return_values={"output": output.content}, log=output.content) - -# # Parse out the function call -# function_call = output.additional_kwargs["function_call"] -# name = function_call["name"] -# inputs = json.loads(function_call["arguments"]) - -# # If the Response function was invoked, return to the user with the function inputs -# if name == "ExecutorResponse": -# return AgentFinish(return_values=inputs, log=str(function_call)) -# # Otherwise, return an agent action -# else: -# return AgentActionMessageLog( -# tool=name, tool_input=inputs, log="", message_log=[output] -# ) - - -# class ExecutorResponse(BaseModel): -# """the result of the behavior tree execution""" - -# hasSucceeded: bool = Field(description="If the behavior tree has succeeded or not") -# world_state: dict[str, list[dict[str, str]]] = Field( -# description="The final world state after the execution" -# ) - - -# bt_exe_llm_with_tools = bt_exe_llm.bind_functions( -# [convert_to_openai_function(t) for t in executor_tools] -# ) - - -# ! BUG -# bt_exe_agent = ( -# { -# "input": lambda x: x["input"], -# # Format agent scratchpad from intermediate steps -# "agent_scratchpad": lambda x: format_to_openai_function_messages( -# x["intermediate_steps"] -# ), -# } -# | bt_exe_ppt -# | bt_exe_llm_with_tools -# | executor_agent_parse -# ) - -# bt_exe_agent_executor = AgentExecutor( -# tools=executor_tools, agent=bt_exe_agent, verbose=True -# ) - - -# * the graph state -class PlanExecuteState(TypedDict): - plan: List[str] - behavior_tree_skeleton: dict # ! not sure if use this or not. - behavior_tree: dict - world_state: Annotated[ - List[dict], operator.add - ] # ! to add, you need to make world_state a list of dict - # = Field(default=None) - past_steps: Annotated[List[Tuple], operator.add] - response: str # ! not sure if use this or not. - # to_user: bool - user_input: str - problem: str - - -##################################################### * planner -# * output schema of the planner -class Plan(BaseModel): - """Plan to follow in future""" - - steps: List[str] = Field( - description="a list of different steps to follow, should be in sorted order" - ) - - -template_file = os.path.join(prompt_dir, "planner/template.txt") -task_file = os.path.join(prompt_dir, "planner/task.txt") -system_file = os.path.join(prompt_dir, "planner/system.txt") -domain_file = os.path.join(prompt_dir, "planner/domain.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {template} - """ -) - -planner_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ], -) - -planner = create_structured_output_runnable( - Plan, ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), planner_ppt_ppl -) - -##################################################### * plan_updaterner - - -class UpdaterResponse(BaseModel): - """used to response to the user for asking for more inputs.""" - - response: str - - -template_file = os.path.join(prompt_dir, "plan_updater/template.txt") -task_file = os.path.join(prompt_dir, "plan_updater/task.txt") -system_file = os.path.join(prompt_dir, "plan_updater/system.txt") -domain_file = os.path.join(prompt_dir, "plan_updater/domain.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {template} - """ -) - -plan_updater_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ], -) - -plan_updater = create_openai_fn_runnable( - [Plan, UpdaterResponse], # * here two schemas are used - ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), - plan_updater_ppt_ppl, -) - - -##################################################### * graph node functions -@traceable(name="user_input_node_step") -async def user_input_step(state: PlanExecuteState): - """ - get the input from the user - """ - print(f"-----user_input_step-----") - if state["response"] is not None: - user_input = input(state["response"]) - state["response"] = None - else: - user_input = input("Your next instruction: ") - - return { - "user_input": user_input, - } - - -@traceable(name="behavior_tree_generate_step") -async def behavior_tree_generate_step(state: PlanExecuteState): - """ - generate the behavior tree based on the instruction - """ - print(f"-----behavior_tree_generate_step-----") - - instruction = state["plan"][0] - latest_world_state = state["world_state"][-1] - - skeleton = await skeleton_generator_chain.ainvoke( - { - "problem": state["problem"], - "world_state": latest_world_state, - "instructions": instruction, - } - ) - - # sk_json = JsonOutputParser().invoke(skeleton) - sk_json = skeleton - - behavior_tree_sk = sk_json["task_plan"]["behavior_tree"] - - test_bt(behavior_tree_sk) - - pause = input("paused here") - - # ! OBJECT CONCENTRATION - - response = await re_sk_chain.ainvoke( - { - "input": skeleton, - } - ) - - behavior_tree = response.get("task_plan").get("behavior_tree") - - init_world_state = [response.get("initial_state")] - - return { - "behavior_tree": behavior_tree, - "world_state": init_world_state, # ! this is necessary because the constraints in problem - } - - -# ! BBWARN INITIALIZE THE CONSTRAINT IN THE PROMPT OR IN THE INITIALIZATION NODE - - -@traceable(name="behavior_tree_execute_step") -async def behavior_tree_execute_step(state: PlanExecuteState): - """ - execute the first step of the plan, append the result to the past steps - """ - print(f"-----behavior_tree_execute_step-----") - - behavior_tree = state["behavior_tree"] - latest_world_state = state["world_state"][-1] - - # exe_input = { - # "behavior_tree": behavior_tree, - # "world_state": latest_world_state, - # } - - # agent_response = await bt_exe_agent_executor.ainvoke( - # { - # "input": exe_input, - # } - # ) - - response = behavior_tree_stewardship.fake_run( - world_state=latest_world_state, bt_json=behavior_tree - ) - - # hasSucceeded = response["hasSucceeded"] - # new_world_state = [response["world_state"]] - hasSucceeded = response.result - new_world_state = [response.world_state] - the_step = state["plan"][0] - - return { - "past_steps": ( - the_step, - hasSucceeded, - ), # * only one because new plan will be generated and old steps are all removed - "world_state": new_world_state, - } - - -@traceable(name="planner_step") -async def planner_step(state: PlanExecuteState): - """ - plan the steps based on user input and world state - """ - print(f"-----plan_step-----") - - plan = await planner.ainvoke( - { - "user_input": state["user_input"], - "world_state": state["world_state"], - } - ) - return {"plan": plan.steps} - - -@traceable(name="plan_updater_step") -async def plan_updater_step(state: PlanExecuteState): - """ - if return a response, then success, response the use, end. - otherwise, return the updated newplan (normally the same as the old plan with the first step popped out. - """ - print(f"-----plan_updater_step-----") - - output = await plan_updater.ainvoke(state) - if isinstance( - output, UpdaterResponse - ): # * determine if it is time to response and end - return { - "response": "Your last instruction has been finished." - } # * Don't need to update. - else: - return {"plan": output.steps} # * update the plan - - -##################################################### * construct the graph - -workflow = StateGraph(PlanExecuteState) - -workflow.add_node("planner", planner_step) - -workflow.add_node("behavior_tree_generator", behavior_tree_generate_step) - -workflow.add_node("behavior_tree_executor", behavior_tree_execute_step) - -workflow.add_node("plan_updater", plan_updater_step) - -workflow.add_node("user_input_node", user_input_step) - -workflow.set_entry_point("user_input_node") - -# workflow.add_edge("user_input_node", "planner") - -workflow.add_edge( - "planner", "behavior_tree_generator" -) # ! may need to add a condition here - -workflow.add_edge("behavior_tree_generator", "behavior_tree_executor") - -workflow.add_edge("behavior_tree_executor", "plan_updater") - - -def plan_updater_should_end(state: PlanExecuteState): - """ - end router - """ - print(f"-----plan_updater_should_end-----") - - if state["response"]: - return True - else: - return False - - -workflow.add_conditional_edges( - "plan_updater", - plan_updater_should_end, - { - True: "user_input_node", - False: "behavior_tree_generator", - }, -) - - -def user_input_should_end(state: PlanExecuteState): - """ - if the user input is empty, then end - """ - print(f"-----user_input_should_end-----") - - # print(f'user_input: {state["user_input"]}') - # print(f'world_state: {state["world_state"]}') - - if not state["user_input"] or state["user_input"] == "": - return True - else: - return False - - -workflow.add_conditional_edges( - "user_input_node", - user_input_should_end, - { - True: END, - False: "planner", - }, -) - -app = workflow.compile() - - -# config = {"recursion_limit": 50} - -inputs = { - "world_state": [world_state_json_object], - "problem": problem, -} - - -async def core_run(): - async for event in app.astream( - inputs, - # config=config, - ): - for k, v in event.items(): - if k != "__end__": - print(v) - - -if __name__ == "__main__": - import asyncio - - asyncio.run(core_run()) diff --git a/experiments/chair/gearset1_graph_bu_txt.py b/experiments/chair/gearset1_graph_bu_txt.py deleted file mode 100644 index f226fd54..00000000 --- a/experiments/chair/gearset1_graph_bu_txt.py +++ /dev/null @@ -1,545 +0,0 @@ -import json -import os - -os.environ["LANGCHAIN_TRACING_V2"] = "true" -os.environ["LANGCHAIN_ENDPOINT"] = "https://api.smith.langchain.com" -os.environ["LANGCHAIN_PROJECT"] = "kios_agent" - -from kios_bt.bt_stewardship import BehaviorTreeStewardship -from kios_scene.scene_factory import SceneFactory -from kios_bt.bt_factory import BehaviorTreeFactory -from kios_robot.robot_interface import RobotInterface -from kios_world.world_interface import WorldInterface - -from kios_agent.llm_supporter import KiosLLMSupporter -from kios_agent.data_types import KiosPromptSkeleton - -from kios_agent.kios_tools import ( - BehaviorTreeExecutorTool, - BehaviorTreeSimulatorTool, - WorldStateQueryTool, -) - -from typing import List, Tuple, Annotated, TypedDict -import operator - -from dotenv import load_dotenv - -from langchain import hub - -from langchain_openai import ChatOpenAI - -from langchain.chains.openai_functions import ( - create_structured_output_runnable, - create_openai_fn_runnable, -) -from langchain_core.utils.function_calling import convert_to_openai_function -from langchain_core.agents import AgentActionMessageLog, AgentFinish -from langchain_core.messages import HumanMessage -from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder -from langchain_core.output_parsers import JsonOutputParser, StrOutputParser -from langchain_core.pydantic_v1 import BaseModel, Field - -from langchain.agents.format_scratchpad import format_to_openai_function_messages -from langchain.agents import create_openai_functions_agent, AgentExecutor - -from langchain.prompts.pipeline import PipelinePromptTemplate -from langchain.prompts.prompt import PromptTemplate - -from langgraph.graph import StateGraph, END - -from langsmith import traceable - -load_dotenv() - - -####################### dirs -current_dir = os.path.dirname(os.path.abspath(__file__)) -scene_path = os.path.join(current_dir, "scene.json") -# bt_json_file_path = os.path.join(current_dir, "behavior_tree.json") -world_state_path = os.path.join(current_dir, "world_state.json") -problem_path = os.path.join(current_dir, "gearset.problem") - - -####################### problem -with open(problem_path, "r") as file: - problem = file.read() - -####################### scene -with open(scene_path, "r") as file: - scene_json_object = json.load(file) - -scene = SceneFactory().create_scene_from_json(scene_json_object) - -####################### world -world_interface = WorldInterface() -with open(world_state_path, "r") as file: - world_state_json_object = json.load(file) - world_interface.load_world_from_json(world_state_json_object) - -####################### robot -robot_interface = RobotInterface( - robot_address="127.0.0.1", - robot_port=12000, -) -robot_interface.setup_scene(scene) - -####################### bt_factory -bt_factory = BehaviorTreeFactory( - world_interface=world_interface, - robot_interface=robot_interface, -) - -####################### behavior_tree_stewardship -behavior_tree_stewardship = BehaviorTreeStewardship( - behaviortree_factory=bt_factory, - world_interface=world_interface, - robot_interface=robot_interface, -) - -####################### * tools -# bt_executor_tool = BehaviorTreeExecutorTool(behavior_tree_stewardship) - -bt_sim_tool = BehaviorTreeSimulatorTool(metadata={"bt_stw": behavior_tree_stewardship}) - -# world_state_query_tool = WorldStateQueryTool(behavior_tree_stewardship) - -executor_tools = [bt_sim_tool] - -# * kios data prompt skeleton dir -data_dir = os.environ.get("KIOS_DATA_DIR").format(username=os.getlogin()) -print(data_dir) -prompt_sk_dir = os.path.join(data_dir, "prompt_skeletons") -prompt_dir = os.path.join(data_dir, "prompts") - -# * refiner chain -llm_spt_re_sk = KiosLLMSupporter() -with open(os.path.join(prompt_sk_dir, "skeleton_refiner.json"), "r") as f: - re_sk_pptsk_json = json.load(f) - re_sk_pptsk = KiosPromptSkeleton.from_json(re_sk_pptsk_json) - llm_spt_re_sk.initialize_from_prompt_skeleton(re_sk_pptsk) -re_sk_ppt = llm_spt_re_sk.create_prompt() -re_sk_llm = ChatOpenAI(model_name=llm_spt_re_sk.model_name, temperature=0) - -re_sk_chain = re_sk_ppt | re_sk_llm | JsonOutputParser() - - -# * skeleton generator chain -llm_spt_skeleton_generator = KiosLLMSupporter() -with open(os.path.join(prompt_sk_dir, "skeleton_generator.json"), "r") as f: - skeleton_generator_pptsk_json = json.load(f) - skeleton_generator_pptsk = KiosPromptSkeleton.from_json( - skeleton_generator_pptsk_json - ) - llm_spt_skeleton_generator.initialize_from_prompt_skeleton(skeleton_generator_pptsk) -skeleton_generator_ppt = llm_spt_skeleton_generator.create_prompt() -skeleton_generator_llm = ChatOpenAI( - model_name=llm_spt_skeleton_generator.model_name, temperature=0 -) - -skeleton_generator_chain = ( - skeleton_generator_ppt - | skeleton_generator_llm - # | StrOutputParser() - | JsonOutputParser() -) - -# ! DISCARDED behavior tree generator chain -# bt_gen_chain = skeleton_generator_chain | re_sk_chain - -############################################### * behavior tree executor agent -# ! NOW SUBSTITUTED BY THE FAKE EXECUTOR FUNCTION -# bt_exe_ppt = hub.pull("hwchase17/openai-functions-agent") -# bt_exe_ppt = ChatPromptTemplate.from_messages( -# [ -# ("system", "You are a helpful assistant"), -# ("user", "The behavior tree is: {behavior_tree}"), -# ("user", "The world state is: {world_state}"), -# MessagesPlaceholder(variable_name="agent_scratchpad"), -# ] -# ) - -# bt_exe_llm = ChatOpenAI(model="gpt-3.5-turbo", temperature=0) - - -# def executor_agent_parse(output): -# # If no function was invoked, return to user -# if "function_call" not in output.additional_kwargs: -# return AgentFinish(return_values={"output": output.content}, log=output.content) - -# # Parse out the function call -# function_call = output.additional_kwargs["function_call"] -# name = function_call["name"] -# inputs = json.loads(function_call["arguments"]) - -# # If the Response function was invoked, return to the user with the function inputs -# if name == "ExecutorResponse": -# return AgentFinish(return_values=inputs, log=str(function_call)) -# # Otherwise, return an agent action -# else: -# return AgentActionMessageLog( -# tool=name, tool_input=inputs, log="", message_log=[output] -# ) - - -# class ExecutorResponse(BaseModel): -# """the result of the behavior tree execution""" - -# hasSucceeded: bool = Field(description="If the behavior tree has succeeded or not") -# world_state: dict[str, list[dict[str, str]]] = Field( -# description="The final world state after the execution" -# ) - - -# bt_exe_llm_with_tools = bt_exe_llm.bind_functions( -# [convert_to_openai_function(t) for t in executor_tools] -# ) - - -# ! BUG -# bt_exe_agent = ( -# { -# "input": lambda x: x["input"], -# # Format agent scratchpad from intermediate steps -# "agent_scratchpad": lambda x: format_to_openai_function_messages( -# x["intermediate_steps"] -# ), -# } -# | bt_exe_ppt -# | bt_exe_llm_with_tools -# | executor_agent_parse -# ) - -# bt_exe_agent_executor = AgentExecutor( -# tools=executor_tools, agent=bt_exe_agent, verbose=True -# ) - - -# * the graph state -class PlanExecuteState(TypedDict): - plan: List[str] - behavior_tree_skeleton: dict # ! not sure if use this or not. - behavior_tree: dict - world_state: Annotated[ - List[dict], operator.add - ] # ! to add, you need to make world_state a list of dict - # = Field(default=None) - past_steps: Annotated[List[Tuple], operator.add] - response: str # ! not sure if use this or not. - # to_user: bool - user_input: str - problem: str - - -##################################################### * planner -# * output schema of the planner -class Plan(BaseModel): - """Plan to follow in future""" - - steps: List[str] = Field( - description="a list of different steps to follow, should be in sorted order" - ) - - -# ! now in data_dir -# planner_prompt = ChatPromptTemplate.from_template( -# """ -# The user input should be an instruction (or several instructions) of the robot assembly task. \ -# For the given user input, summarize and come up with a bullet list task plan. \ -# This plan should involve individual assembly step(s) of robot assembly task.\ -# Do not add any superfluous steps. \ -# Make sure that each step has all the information needed - do not skip steps.\ - -# You should only use assembly actions as the action in the steps. -# The available assembly actions are:\ -# - insert something into something\ -# - place something onto something\ -# - screw something into something\ - -# Example:\ -# user_input: "you should insert the red block into the blue hole"\ -# plan: "insert the red block into the blue hole"\ - -# user_input: "intert the blub into the base, then put the cover onto it."\ -# plan: ["insert the blub into the base", "put the cover onto the blub"]\ - -# world_state: {world_state}\ -# user_input: {user_input}\ -# """ -# ) - -planner_prompt_file = os.path.join(prompt_dir, "planner/template.txt") - -with open(planner_prompt_file, "w") as f: - planner_prompt = f.read() - -planner = create_structured_output_runnable( - Plan, ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), planner_prompt -) - - -##################################################### * plan_updaterner - - -class UpdaterResponse(BaseModel): - """used to response to the user for asking for more inputs.""" - - response: str - - -plan_updater_prompt = ChatPromptTemplate.from_template( - """For the given objective, come up with a simple step by step plan. \ -This plan should involve individual tasks, that if executed correctly will yield the correct answer. Do not add any superfluous steps. \ -The result of the final step should be the final answer. Make sure that each step has all the information needed - do not skip steps. - -Your objective was this: -{user_input} - -Your original plan was this: -{plan} - -You have currently done the follow steps: -{past_steps} - -The world state is: -{world_state} - -Read the world state. Check the original plan from the first step and update your plan with the steps need to be conducted next according to the world state.\ -If no more steps are needed to achieve the objective defined by user input, respond to the user for more input. Otherwise, fill out the plan.\ -Only add steps to the plan that still NEED to be done. Do not return previously done steps as part of the plan.""" -) - -plan_updater = create_openai_fn_runnable( - [Plan, UpdaterResponse], # * here two schemas are used - ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), - plan_updater_prompt, -) - - -##################################################### * graph node functions -@traceable(name="user_input_node_step") -async def user_input_step(state: PlanExecuteState): - """ - get the input from the user - """ - print(f"-----user_input_step-----") - if state["response"] is not None: - user_input = input(state["response"]) - state["response"] = None - else: - user_input = input("Your next instruction: ") - - return { - "user_input": user_input, - } - - -@traceable(name="behavior_tree_generate_step") -async def behavior_tree_generate_step(state: PlanExecuteState): - """ - generate the behavior tree based on the instruction - """ - print(f"-----behavior_tree_generate_step-----") - - instruction = state["plan"][0] - latest_world_state = state["world_state"][-1] - - skeleton = await skeleton_generator_chain.ainvoke( - { - "problem": state["problem"], - "world_state": latest_world_state, - "instructions": instruction, - } - ) - - # ! OBJECT CONCENTRATION - - response = await re_sk_chain.ainvoke( - { - "input": skeleton, - } - ) - - behavior_tree = response.get("task_plan").get("behavior_tree") - - init_world_state = [response.get("initial_state")] - - return { - "behavior_tree": behavior_tree, - "world_state": init_world_state, # ! this is necessary because the constraints in problem - } - - -# ! BBWARN INITIALIZE THE CONSTRAINT IN THE PROMPT OR IN THE INITIALIZATION NODE - - -@traceable(name="behavior_tree_execute_step") -async def behavior_tree_execute_step(state: PlanExecuteState): - """ - execute the first step of the plan, append the result to the past steps - """ - print(f"-----behavior_tree_execute_step-----") - - behavior_tree = state["behavior_tree"] - latest_world_state = state["world_state"][-1] - - # exe_input = { - # "behavior_tree": behavior_tree, - # "world_state": latest_world_state, - # } - - # agent_response = await bt_exe_agent_executor.ainvoke( - # { - # "input": exe_input, - # } - # ) - - response = behavior_tree_stewardship.fake_run( - world_state=latest_world_state, bt_json=behavior_tree - ) - - # hasSucceeded = response["hasSucceeded"] - # new_world_state = [response["world_state"]] - hasSucceeded = response.result - new_world_state = [response.world_state] - the_step = state["plan"][0] - - return { - "past_steps": (the_step, hasSucceeded), # ! why here not a list of tuple? - "world_state": new_world_state, - } - - -@traceable(name="planner_step") -async def planner_step(state: PlanExecuteState): - """ - plan the steps based on user input and world state - """ - print(f"-----plan_step-----") - - plan = await planner.ainvoke( - { - "user_input": state["user_input"], - "world_state": state["world_state"], - } - ) - return {"plan": plan.steps} - - -@traceable(name="plan_updater_step") -async def plan_updater_step(state: PlanExecuteState): - """ - if return a response, then success, response the use, end. - otherwise, return the updated newplan (normally the same as the old plan with the first step popped out. - """ - print(f"-----plan_updater_step-----") - - output = await plan_updater.ainvoke(state) - if isinstance( - output, UpdaterResponse - ): # * determine if it is time to response and end - return { - "response": "Your last instruction has been finished." - } # * Don't need to update. - else: - return {"plan": output.steps} # * update the plan - - -##################################################### * construct the graph - -workflow = StateGraph(PlanExecuteState) - -workflow.add_node("planner", planner_step) - -workflow.add_node("behavior_tree_generator", behavior_tree_generate_step) - -workflow.add_node("behavior_tree_executor", behavior_tree_execute_step) - -workflow.add_node("plan_updater", plan_updater_step) - -workflow.add_node("user_input_node", user_input_step) - -workflow.set_entry_point("user_input_node") - -# workflow.add_edge("user_input_node", "planner") - -workflow.add_edge( - "planner", "behavior_tree_generator" -) # ! may need to add a condition here - -workflow.add_edge("behavior_tree_generator", "behavior_tree_executor") - -workflow.add_edge("behavior_tree_executor", "plan_updater") - - -def plan_updater_should_end(state: PlanExecuteState): - """ - end router - """ - print(f"-----plan_updater_should_end-----") - - if state["response"]: - return True - else: - return False - - -workflow.add_conditional_edges( - "plan_updater", - plan_updater_should_end, - { - True: "user_input_node", - False: "behavior_tree_generator", - }, -) - - -def user_input_should_end(state: PlanExecuteState): - """ - if the user input is empty, then end - """ - print(f"-----user_input_should_end-----") - - # print(f'user_input: {state["user_input"]}') - # print(f'world_state: {state["world_state"]}') - - if not state["user_input"] or state["user_input"] == "": - return True - else: - return False - - -workflow.add_conditional_edges( - "user_input_node", - user_input_should_end, - { - True: END, - False: "planner", - }, -) - -app = workflow.compile() - - -# config = {"recursion_limit": 50} - -inputs = { - "world_state": [world_state_json_object], - "problem": problem, -} - - -async def core_run(): - async for event in app.astream( - inputs, - # config=config, - ): - for k, v in event.items(): - if k != "__end__": - print(v) - - -if __name__ == "__main__": - import asyncio - - asyncio.run(core_run()) diff --git a/experiments/chair/graph.py b/experiments/chair/graph.py deleted file mode 100644 index d0ff8d17..00000000 --- a/experiments/chair/graph.py +++ /dev/null @@ -1,180 +0,0 @@ -from langchain_core.pydantic_v1 import BaseModel, Field -from typing import List, Tuple, Annotated, TypedDict -import operator - -from langchain_openai import ChatOpenAI - -import os - -os.environ["LANGCHAIN_TRACING_V2"] = "true" -os.environ["LANGCHAIN_PROJECT"] = "kios_agent" - -##################################################### * tool -from langchain_community.tools.tavily_search import TavilySearchResults -from kios_agent.kios_tools import BehaviorTreeExecutorTool, BehaviorTreeSimulatorTool - -# tools = [TavilySearchResults(max_results=3)] -tools = [BehaviorTreeSimulatorTool()] # ! need stw - -##################################################### * execution - -from langchain import hub -from langchain.agents import create_openai_functions_agent -from langchain_openai import ChatOpenAI - -prompt = hub.pull("hwchase17/openai-functions-agent") - -# Choose the LLM that will drive the agent -llm = ChatOpenAI(model="gpt-4-turbo-preview") -# Construct the OpenAI Functions agent -agent_runnable = create_openai_functions_agent(llm, tools, prompt) - -from langgraph.prebuilt import create_agent_executor - -agent_executor = create_agent_executor(agent_runnable, tools) - - -# * the graph state -class PlanExecuteState(TypedDict): - input: str - plan: List[str] - world_state: List[dict] - past_steps: Annotated[List[Tuple], operator.add] - response: str - - -##################################################### * planner -# * output schema of the planner -class Plan(BaseModel): - """Plan to follow in future""" - - steps: List[str] = Field( - description="different steps to follow, should be in sorted order" - ) - - -from langchain.chains.openai_functions import create_structured_output_runnable -from langchain_core.prompts import ChatPromptTemplate - -planner_prompt = ChatPromptTemplate.from_template( - """For the given objective, come up with a simple step by step plan. \ -This plan should involve individual tasks, that if executed correctly will yield the correct answer. Do not add any superfluous steps. \ -The result of the final step should be the final answer. Make sure that each step has all the information needed - do not skip steps. - -{objective}""" -) -planner = create_structured_output_runnable( - Plan, ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), planner_prompt -) - -##################################################### * plan_updaterner -from langchain.chains.openai_functions import create_openai_fn_runnable - - -class Response(BaseModel): - """Response to user.""" - - response: str - - -plan_updater_prompt = ChatPromptTemplate.from_template( - """For the given objective, come up with a simple step by step plan. \ -This plan should involve individual tasks, that if executed correctly will yield the correct answer. Do not add any superfluous steps. \ -The result of the final step should be the final answer. Make sure that each step has all the information needed - do not skip steps. - -Your objective was this: -{input} - -Your original plan was this: -{plan} - -You have currently done the follow steps: -{past_steps} - -Update your plan accordingly. If no more steps are needed and you can return to the user, then respond with that. Otherwise, fill out the plan. Only add steps to the plan that still NEED to be done. Do not return previously done steps as part of the plan.""" -) - -plan_updater = create_openai_fn_runnable( - [Plan, Response], # * here two schemas are used - ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), - plan_updater_prompt, -) - - -##################################################### * graph -async def execute_step(state: PlanExecuteState): - """ - execute the first step of the plan, append the result to the past steps - """ - task = state["plan"][0] - agent_response = await agent_executor.ainvoke({"input": task, "chat_history": []}) - return { - "past_steps": (task, agent_response["agent_outcome"].return_values["output"]) - } - - -async def plan_step(state: PlanExecuteState): - """ - plan the steps based on user input - """ - plan = await planner.ainvoke({"objective": state["input"]}) - return {"plan": plan.steps} - - -async def plan_updater_step(state: PlanExecuteState): - """ - if return a response, then success, response the use, end. - otherwise, return the updated newplan (normally the same as the old plan with the first step popped out. - """ - output = await plan_updater.ainvoke(state) - if isinstance(output, Response): # * determine if it is time to response and end - return {"response": output.response} - else: - return {"plan": output.steps} - - -def should_end(state: PlanExecuteState): - """ - end router - """ - if state["response"]: - return True - else: - return False - - -##################################################### * construct the graph -from langgraph.graph import StateGraph, END - -workflow = StateGraph(PlanExecuteState) - -# Add the plan node -workflow.add_node("planner", plan_step) - -# Add the execution step -workflow.add_node("agent", execute_step) - -# Add a plan_updater node -workflow.add_node("plan_updater", plan_updater_step) - -workflow.set_entry_point("planner") - -# From plan we go to agent -workflow.add_edge("planner", "agent") - -# From agent, we plan_updater -workflow.add_edge("agent", "plan_updater") - -workflow.add_conditional_edges( - "plan_updater", - # Next, we pass in the function that will determine which node is called next. - should_end, - { - # If `tools`, then we call the tool node. - True: END, - False: "agent", - }, -) - -# compiles it into a LangChain Runnable, -app = workflow.compile() diff --git a/experiments/chair/problem.py b/experiments/chair/problem.py deleted file mode 100644 index 3350099f..00000000 --- a/experiments/chair/problem.py +++ /dev/null @@ -1,106 +0,0 @@ -import re - - -def parse_init(problem): - init_pattern = r"\(:init\s+(.*?)\s+\)" - element_pattern = r"\((.*?)\)" - - init_match = re.search(init_pattern, problem, re.DOTALL) - if init_match: - init_content = init_match.group(1) - elements = re.findall(element_pattern, init_content) - - parsed_init = [] - for element in elements: - parts = element.split() - name = parts[0] - args = parts[1:] - parsed_init.append({"name": name, "args": args}) - - return parsed_init - else: - return [] - - -properties = [ - "is_free", - "is_equippable", -] - -constraints = [ - "can_manipulate", - "can_screw_to", - "can_insert_to", - "can_place_to", -] - -relations = [ - "hold", - "is_inserted_to", - "is_screwed_to", - "is_placed_to", -] - - -def classify_item(item_name): - if item_name in properties: - return "property" - elif item_name in constraints: - return "constraint" - elif item_name in relations: - return "relation" - else: - raise ValueError(f"Unknown item name: {item_name}") - - -# Usage example: -problem = """ -(define (problem robot_assembly_problem-problem) - (:domain robot_assembly_problem-domain) - (:objects - parallel_box1 parallel_box2 inward_claw outward_claw no_tool - tool - gear1 gear2 gear3 shaft1 shaft2 shaft3 gearbase gearbase_hole1 gearbase_hole3 - part - left_hand - hand - ) - (:init (can_manipulate parallel_box2 gear1) (can_manipulate outward_claw gear2) (can_manipulate outward_claw gear3) (can_manipulate no_tool shaft3) (can_manipulate parallel_box1 shaft1) (can_screw_to leg1 seat) (can_screw_to leg2 seat) (can_insert_to back seat) (can_screw_to nut1 seat) (can_screw_to nut2 seat) (can_screw_to blub lampbase) (can_place_to lamp blub) (can_insert_to shaft1 gearbase_hole1) (can_insert_to shaft3 gearbase_hole3) (can_insert_to gear3 shaft3) (can_insert_to gear2 shaft2) (can_insert_to gear1 shaft1)) - (:goal (and )) - )""" - -parsed_init = parse_init(problem) - -for item in parsed_init: - item_name = item.get("name") - item_type = classify_item(item_name) - - world_state = { - "objects": [], - "constraints": [], - "relations": [], - } - - if item_type == "property": - world_state["objects"].append( - {"name": item["args"][0], "properties": [item_name]} # ! list or item? - ) - - elif item_type == "constraint": - world_state["constraints"].append( - { - "source": item["args"][0], - "name": item_name, - "target": item["args"][1], - } - ) - elif item_type == "relation": - world_state["relations"].append( - { - "source": item["args"][0], - "name": item_name, - "target": item["args"][1], - } - ) - else: - pass - - -print(parsed_init) diff --git a/experiments/chair/problem_set/problem_000.json b/experiments/chair/problem_set/problem_000.json deleted file mode 100644 index 5a118c6f..00000000 --- a/experiments/chair/problem_set/problem_000.json +++ /dev/null @@ -1,252 +0,0 @@ -{ - "target": "target: is_screwed_to(chairnut1, chairseatbolt1)", - "initial_world_state": { - "objects": [ - { - "name": "clampgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "parallelgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "inwardgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "outwardgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "defaultgripper", - "properties": [] - }, - { - "name": "chairnut1", - "properties": [] - }, - { - "name": "chairnut2", - "properties": [] - }, - { - "name": "chairseat", - "properties": [] - }, - { - "name": "chairseatbolt1", - "properties": [] - }, - { - "name": "chairseatbolt2", - "properties": [] - }, - { - "name": "chairseatconnection", - "properties": [] - }, - { - "name": "chairseatthread1", - "properties": [] - }, - { - "name": "chairseatthread2", - "properties": [] - }, - { - "name": "chairback", - "properties": [] - }, - { - "name": "chairleg1", - "properties": [] - }, - { - "name": "chairleg2", - "properties": [] - }, - { - "name": "left_hand", - "properties": [] - } - ], - "constraints": [ - { - "source": "clampgripper", - "name": "can_manipulate", - "target": "chairback" - }, - { - "source": "inwardgripper", - "name": "can_manipulate", - "target": "chairnut1" - }, - { - "source": "inwardgripper", - "name": "can_manipulate", - "target": "chairnut2" - }, - { - "source": "defaultgripper", - "name": "can_manipulate", - "target": "chairleg1" - }, - { - "source": "defaultgripper", - "name": "can_manipulate", - "target": "chairleg2" - }, - { - "source": "defaultgripper", - "name": "can_manipulate", - "target": "chairseat" - }, - { - "source": "chairnut1", - "name": "can_screw_to", - "target": "chairseatbolt1" - }, - { - "source": "chairnut2", - "name": "can_screw_to", - "target": "chairseatbolt2" - }, - { - "source": "chairback", - "name": "can_insert_to", - "target": "chairseatconnection" - }, - { - "source": "chairleg1", - "name": "can_screw_to", - "target": "chairseatthread1" - }, - { - "source": "chairleg2", - "name": "can_screw_to", - "target": "chairseatthread2" - } - ], - "relations": [ - { - "source": "left_hand", - "name": "hold", - "target": "defaultgripper" - }, - { - "source": "defaultgripper", - "name": "hold", - "target": "chairleg1" - } - ] - }, - "result": { - "summary": "selector to screw the chairnut1 into the chairseatbolt1 with the inwardgripper in the left_hand", - "name": "selector: screw(left_hand, inwardgripper, chairnut1, chairseatbolt1)", - "children": [ - { - "summary": "the target is to make the chairnut1 be screwed into the chairseatbolt1", - "name": "target: is_screwed_to(chairnut1, chairseatbolt1)" - }, - { - "summary": "sequence to screw the chairnut1 into the chairseatbolt1 with the inwardgripper in the left_hand", - "name": "sequence: screw(left_hand, inwardgripper, chairnut1, chairseatbolt1)", - "children": [ - { - "summary": "selector to change tool in left_hand from defaultgripper to inwardgripper", - "name": "selector: change_tool(left_hand, defaultgripper, inwardgripper)", - "children": [ - { - "summary": "the target is to make left_hand hold inwardgripper", - "name": "target: hold(left_hand, inwardgripper)" - }, - { - "summary": "sequence to change tool in left_hand from defaultgripper to inwardgripper", - "name": "sequence: change_tool(left_hand, defaultgripper, inwardgripper)", - "children": [ - { - "summary": "a precondition is left_hand is holding defaultgripper", - "name": "precondition: hold(left_hand, defaultgripper)" - }, - { - "summary": "selector to put down the chairleg1 held by the defaultgripper in the left_hand", - "name": "selector: put_down(left_hand, defaultgripper, chairleg1)", - "children": [ - { - "summary": "the target is to make the defaultgripper empty", - "name": "target: is_empty(defaultgripper)" - }, - { - "summary": "sequence to put down the chairleg1 held by the defaultgripper in the left_hand", - "name": "sequence: put_down(left_hand, defaultgripper, chairleg1)", - "children": [ - { - "summary": "a precondition is the left_hand is holding defaultgripper", - "name": "precondition: hold(left_hand, defaultgripper)" - }, - { - "summary": "a precondition is the defaultgripper is holding chairleg1", - "name": "precondition: hold(defaultgripper, chairleg1)" - }, - { - "summary": "the action to put down the chairleg1 held by the defaultgripper in the left_hand", - "name": "action: put_down(left_hand, defaultgripper, chairleg1)" - } - ] - } - ] - }, - { - "summary": "the action to change tool in left_hand from defaultgripper to inwardgripper", - "name": "action: change_tool(left_hand, defaultgripper, inwardgripper)" - } - ] - } - ] - }, - { - "summary": "selector to pick up the chairnut1 with the inwardgripper in the left_hand", - "name": "selector: pick_up(left_hand, inwardgripper, chairnut1)", - "children": [ - { - "summary": "the target is to make the inwardgripper hold the chairnut1", - "name": "target: hold(inwardgripper, chairnut1)" - }, - { - "summary": "sequence to pick up the chairnut1 with the inwardgripper in the left_hand", - "name": "sequence: pick_up(left_hand, inwardgripper, chairnut1)", - "children": [ - { - "summary": "a precondition is the inwardgripper is empty", - "name": "precondition: is_empty(inwardgripper)" - }, - { - "summary": "a precondition is the left_hand is holding inwardgripper", - "name": "precondition: hold(left_hand, inwardgripper)" - }, - { - "summary": "the action to pick up the chairnut1 with the inwardgripper in the left_hand", - "name": "action: pick_up(left_hand, inwardgripper, chairnut1)" - } - ] - } - ] - }, - { - "summary": "the action to screw the chairnut1 into the chairseatbolt1 with the inwardgripper in the left_hand", - "name": "action: screw(left_hand, inwardgripper, chairnut1, chairseatbolt1)" - } - ] - } - ] - } -} \ No newline at end of file diff --git a/experiments/chair/problem_set/problem_001.json b/experiments/chair/problem_set/problem_001.json deleted file mode 100644 index 2837418f..00000000 --- a/experiments/chair/problem_set/problem_001.json +++ /dev/null @@ -1,225 +0,0 @@ -{ - "target": "target: is_screwed_to(chairnut1, chairseatbolt1)", - "initial_world_state": { - "objects": [ - { - "name": "clampgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "parallelgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "inwardgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "outwardgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "defaultgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "chairnut1", - "properties": [] - }, - { - "name": "chairnut2", - "properties": [] - }, - { - "name": "chairseat", - "properties": [] - }, - { - "name": "chairseatbolt1", - "properties": [] - }, - { - "name": "chairseatbolt2", - "properties": [] - }, - { - "name": "chairseatconnector", - "properties": [] - }, - { - "name": "chairseatthread1", - "properties": [] - }, - { - "name": "chairseatthread2", - "properties": [] - }, - { - "name": "chairback", - "properties": [] - }, - { - "name": "chairleg1", - "properties": [] - }, - { - "name": "chairleg2", - "properties": [] - }, - { - "name": "left_hand", - "properties": [] - } - ], - "constraints": [ - { - "source": "clampgripper", - "name": "can_manipulate", - "target": "chairback" - }, - { - "source": "inwardgripper", - "name": "can_manipulate", - "target": "chairnut1" - }, - { - "source": "inwardgripper", - "name": "can_manipulate", - "target": "chairnut2" - }, - { - "source": "defaultgripper", - "name": "can_manipulate", - "target": "chairleg1" - }, - { - "source": "defaultgripper", - "name": "can_manipulate", - "target": "chairleg2" - }, - { - "source": "defaultgripper", - "name": "can_manipulate", - "target": "chairseat" - }, - { - "source": "chairnut1", - "name": "can_screw_to", - "target": "chairseatbolt1" - }, - { - "source": "chairnut2", - "name": "can_screw_to", - "target": "chairseatbolt2" - }, - { - "source": "chairback", - "name": "can_insert_to", - "target": "chairseatconnector" - }, - { - "source": "chairleg1", - "name": "can_screw_to", - "target": "chairseatthread1" - }, - { - "source": "chairleg2", - "name": "can_screw_to", - "target": "chairseatthread2" - } - ], - "relations": [ - { - "source": "left_hand", - "name": "hold", - "target": "clampgripper" - } - ] - }, - "result": { - "summary": "selector to screw the chairnut1 into the chairseatbolt1 with the inwardgripper in the left_hand", - "name": "selector: screw(left_hand, inwardgripper, chairnut1, chairseatbolt1)", - "children": [ - { - "summary": "the target is to make the chairnut1 be screwed into the chairseatbolt1", - "name": "target: is_screwed_to(chairnut1, chairseatbolt1)" - }, - { - "summary": "sequence to screw the chairnut1 into the chairseatbolt1 with the inwardgripper in the left_hand", - "name": "sequence: screw(left_hand, inwardgripper, chairnut1, chairseatbolt1)", - "children": [ - { - "summary": "selector to change tool in left_hand from clampgripper to inwardgripper", - "name": "selector: change_tool(left_hand, clampgripper, inwardgripper)", - "children": [ - { - "summary": "the target is to make left_hand hold inwardgripper", - "name": "target: hold(left_hand, inwardgripper)" - }, - { - "summary": "sequence to change tool in left_hand from clampgripper to inwardgripper", - "name": "sequence: change_tool(left_hand, clampgripper, inwardgripper)", - "children": [ - { - "summary": "a precondition is left_hand is holding clampgripper", - "name": "precondition: hold(left_hand, clampgripper)" - }, - { - "summary": "a precondition is clampgripper is empty", - "name": "precondition: is_empty(clampgripper)" - }, - { - "summary": "the action to change tool in left_hand from clampgripper to inwardgripper", - "name": "action: change_tool(left_hand, clampgripper, inwardgripper)" - } - ] - } - ] - }, - { - "summary": "selector to pick up the chairnut1 with the inwardgripper in the left_hand", - "name": "selector: pick_up(left_hand, inwardgripper, chairnut1)", - "children": [ - { - "summary": "the target is to make the inwardgripper hold the chairnut1", - "name": "target: hold(inwardgripper, chairnut1)" - }, - { - "summary": "sequence to pick up the chairnut1 with the inwardgripper in the left_hand", - "name": "sequence: pick_up(left_hand, inwardgripper, chairnut1)", - "children": [ - { - "summary": "a precondition is the inwardgripper is empty", - "name": "precondition: is_empty(inwardgripper)" - }, - { - "summary": "a precondition is the left_hand is holding inwardgripper", - "name": "precondition: hold(left_hand, inwardgripper)" - }, - { - "summary": "the action to pick up the chairnut1 with the inwardgripper in the left_hand", - "name": "action: pick_up(left_hand, inwardgripper, chairnut1)" - } - ] - } - ] - }, - { - "summary": "the action to screw the chairnut1 into the chairseatbolt1 with the inwardgripper in the left_hand", - "name": "action: screw(left_hand, inwardgripper, chairnut1, chairseatbolt1)" - } - ] - } - ] - } -} \ No newline at end of file diff --git a/experiments/chair/problem_set/problem_002.json b/experiments/chair/problem_set/problem_002.json deleted file mode 100644 index 09ca6449..00000000 --- a/experiments/chair/problem_set/problem_002.json +++ /dev/null @@ -1,201 +0,0 @@ -{ - "target": "target: is_screwed_to(chairleg1, chairseatthread1)", - "initial_world_state": { - "objects": [ - { - "name": "clampgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "parallelgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "inwardgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "outwardgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "defaultgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "chairnut1", - "properties": [] - }, - { - "name": "chairnut2", - "properties": [] - }, - { - "name": "chairseat", - "properties": [] - }, - { - "name": "chairseatbolt1", - "properties": [] - }, - { - "name": "chairseatbolt2", - "properties": [] - }, - { - "name": "chairseatconnector", - "properties": [] - }, - { - "name": "chairseatthread1", - "properties": [] - }, - { - "name": "chairseatthread2", - "properties": [] - }, - { - "name": "chairback", - "properties": [] - }, - { - "name": "chairleg1", - "properties": [] - }, - { - "name": "chairleg2", - "properties": [] - }, - { - "name": "left_hand", - "properties": [] - } - ], - "constraints": [ - { - "source": "clampgripper", - "name": "can_manipulate", - "target": "chairback" - }, - { - "source": "inwardgripper", - "name": "can_manipulate", - "target": "chairnut1" - }, - { - "source": "inwardgripper", - "name": "can_manipulate", - "target": "chairnut2" - }, - { - "source": "defaultgripper", - "name": "can_manipulate", - "target": "chairleg1" - }, - { - "source": "defaultgripper", - "name": "can_manipulate", - "target": "chairleg2" - }, - { - "source": "defaultgripper", - "name": "can_manipulate", - "target": "chairseat" - }, - { - "source": "chairnut1", - "name": "can_screw_to", - "target": "chairseatbolt1" - }, - { - "source": "chairnut2", - "name": "can_screw_to", - "target": "chairseatbolt2" - }, - { - "source": "chairback", - "name": "can_insert_to", - "target": "chairseatconnector" - }, - { - "source": "chairleg1", - "name": "can_screw_to", - "target": "chairseatthread1" - }, - { - "source": "chairleg2", - "name": "can_screw_to", - "target": "chairseatthread2" - } - ], - "relations": [ - { - "source": "left_hand", - "name": "hold", - "target": "defaultgripper" - } - ] - }, - "result": { - "summary": "selector to screw the chairleg1 into the chairseatthread1 with the defaultgripper in the left_hand", - "name": "selector: screw(left_hand, defaultgripper, chairleg1, chairseatthread1)", - "children": [ - { - "summary": "the target is to make the chairleg1 be screwed into the chairseatthread1", - "name": "target: is_screwed_to(chairleg1, chairseatthread1)" - }, - { - "summary": "sequence to screw the chairleg1 into the chairseatthread1 with the defaultgripper in the left_hand", - "name": "sequence: screw(left_hand, defaultgripper, chairleg1, chairseatthread1)", - "children": [ - { - "summary": "a precondition is the left_hand is holding defaultgripper", - "name": "precondition: hold(left_hand, defaultgripper)" - }, - { - "summary": "selector to pick up the chairleg1 with the defaultgripper in the left_hand", - "name": "selector: pick_up(left_hand, defaultgripper, chairleg1)", - "children": [ - { - "summary": "the target is to make the defaultgripper hold the chairleg1", - "name": "target: hold(defaultgripper, chairleg1)" - }, - { - "summary": "sequence to pick up the chairleg1 with the defaultgripper in the left_hand", - "name": "sequence: pick_up(left_hand, defaultgripper, chairleg1)", - "children": [ - { - "summary": "a precondition is the defaultgripper is empty", - "name": "precondition: is_empty(defaultgripper)" - }, - { - "summary": "a precondition is the left_hand is holding defaultgripper", - "name": "precondition: hold(left_hand, defaultgripper)" - }, - { - "summary": "the action to pick up the chairleg1 with the defaultgripper in the left_hand", - "name": "action: pick_up(left_hand, defaultgripper, chairleg1)" - } - ] - } - ] - }, - { - "summary": "the action to screw the chairleg1 into the chairseatthread1 with the defaultgripper in the left_hand", - "name": "action: screw(left_hand, defaultgripper, chairleg1, chairseatthread1)" - } - ] - } - ] - } -} \ No newline at end of file diff --git a/experiments/chair/rec_sk_gen.py b/experiments/chair/rec_sk_gen.py deleted file mode 100644 index 2c4d7ac9..00000000 --- a/experiments/chair/rec_sk_gen.py +++ /dev/null @@ -1,738 +0,0 @@ -import json -import os -from pprint import pprint - -os.environ["LANGCHAIN_TRACING_V2"] = "true" -os.environ["LANGCHAIN_ENDPOINT"] = "https://api.smith.langchain.com" -os.environ["LANGCHAIN_PROJECT"] = "kios_agent" - -from kios_bt.bt_stewardship import BehaviorTreeStewardship -from kios_scene.scene_factory import SceneFactory -from kios_bt.bt_factory import BehaviorTreeFactory -from kios_robot.robot_interface import RobotInterface -from kios_world.world_interface import WorldInterface - -from kios_agent.llm_supporter import KiosLLMSupporter -from kios_agent.data_types import KiosPromptSkeleton - -from kios_agent.kios_tools import ( - BehaviorTreeExecutorTool, - BehaviorTreeSimulatorTool, - WorldStateQueryTool, -) - -from typing import List, Tuple, Annotated, TypedDict -import operator - -from dotenv import load_dotenv - -from langchain import hub - -from langchain_openai import ChatOpenAI - -from langchain.chains.openai_functions import ( - create_structured_output_runnable, - create_openai_fn_runnable, -) -from langchain_core.utils.function_calling import convert_to_openai_function -from langchain_core.agents import AgentActionMessageLog, AgentFinish -from langchain_core.messages import HumanMessage -from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder -from langchain_core.output_parsers import JsonOutputParser, StrOutputParser -from langchain_core.pydantic_v1 import BaseModel, Field - -from langchain.agents.format_scratchpad import format_to_openai_function_messages -from langchain.agents import create_openai_functions_agent, AgentExecutor - -from langchain.prompts.pipeline import PipelinePromptTemplate -from langchain.prompts.prompt import PromptTemplate - -from langchain_community.document_loaders import TextLoader -from langchain_openai import OpenAIEmbeddings - -from langchain_community.vectorstores import FAISS -from langchain.text_splitter import ( - RecursiveCharacterTextSplitter, - CharacterTextSplitter, -) - -from langgraph.graph import StateGraph, END - -from langsmith import traceable - -load_dotenv() - -from kios_utils.pybt_test import generate_bt_stewardship, render_dot_tree -from kios_utils.pddl_problem_parser import parse_problem_init, parse_problem_objects - - -def render_bt(bt_json: json): - test_class = BehaviorTreeFactory() - bt = test_class.from_json_to_simple_bt(bt_json) - # bt = test_class.from_json_to_tree_root(bt_json) - bt_stewardship = generate_bt_stewardship(bt) - # bt_stewardship.setup(timeout=15) - render_dot_tree(bt_stewardship) - - -####################### dirs -current_dir = os.path.dirname(os.path.abspath(__file__)) -scene_path = os.path.join(current_dir, "scene.json") -# bt_json_file_path = os.path.join(current_dir, "behavior_tree.json") -world_state_path = os.path.join(current_dir, "world_state.json") -problem_path = os.path.join(current_dir, "gearset.problem") -domain_knowledge_path = os.path.join(current_dir, "domain_knowledge.txt") - -####################### problem -with open(problem_path, "r") as file: - problem = file.read() - -####################### scene -with open(scene_path, "r") as file: - scene_json_object = json.load(file) - -scene = SceneFactory().create_scene_from_json(scene_json_object) - -####################### world -world_interface = WorldInterface() -with open(world_state_path, "r") as file: - world_state_json_object = json.load(file) - world_interface.load_world_from_json(world_state_json_object) - -from_problem = parse_problem_init(problem=problem) -objects = parse_problem_objects(problem=problem) -world_interface.update_world(from_problem) - -world_state_json = world_interface.get_world_to_json() -pprint(from_problem) -pprint(world_state_json) -pprint(objects) - - -####################### robot -robot_interface = RobotInterface( - robot_address="127.0.0.1", - robot_port=12000, -) -robot_interface.setup_scene(scene) - -####################### bt_factory -bt_factory = BehaviorTreeFactory( - world_interface=world_interface, - robot_interface=robot_interface, -) - -####################### behavior_tree_stewardship -behavior_tree_stewardship = BehaviorTreeStewardship( - behaviortree_factory=bt_factory, - world_interface=world_interface, - robot_interface=robot_interface, -) - -####################### * tools -# bt_executor_tool = BehaviorTreeExecutorTool(behavior_tree_stewardship) - -bt_sim_tool = BehaviorTreeSimulatorTool(metadata={"bt_stw": behavior_tree_stewardship}) - -# world_state_query_tool = WorldStateQueryTool(behavior_tree_stewardship) - -executor_tools = [bt_sim_tool] - -# * kios data prompt skeleton dir -data_dir = os.environ.get("KIOS_DATA_DIR").format(username=os.getlogin()) -print(data_dir) -prompt_sk_dir = os.path.join(data_dir, "prompt_skeletons") -prompt_dir = os.path.join(data_dir, "prompts") - -# * retriever chain -# from langchain_experimental.text_splitter import SemanticChunker - -# # with open(domain_knowledge_path, "r") as f: -# # domain_knowledge = f.read() - -# # text_splitter = SemanticChunker(OpenAIEmbeddings()) - -# # docs = text_splitter.create_documents([domain_knowledge]) - - -# dk_loader = TextLoader(domain_knowledge_path) -# domain_knowledge_doc = dk_loader.load() - -# embeddings = OpenAIEmbeddings() - -# text_splitter = RecursiveCharacterTextSplitter( -# chunk_size=300, -# chunk_overlap=10, -# length_function=len, -# is_separator_regex=False, -# ) -# # text_splitter = CharacterTextSplitter() -# docs = text_splitter.split_documents(domain_knowledge_doc) -# vector_store = FAISS.from_documents(docs, embeddings) - -# retriever = vector_store.as_retriever() - -# docs = retriever.get_relevant_documents( -# "how to fulfill the condition 'is_free'?", -# k=3, -# threshold=0.5, -# ) - -# pprint(docs) - - -# * the graph state -class PlanExecuteState(TypedDict): - plan: List[str] - behavior_tree: dict - world_state: Annotated[ - List[dict], operator.add - ] # ! to add, you need to make world_state a list of dict - past_steps: Annotated[List[Tuple], operator.add] - user_input: str - objects: dict[str, list[str]] - last_behavior_tree: dict - last_failed_node: dict - runtime_world_state: dict - behavior_tree_execution_summary: str - BTExecutionHasSucceeded: bool - - -########## verifier - -behaviortree_file = os.path.join(prompt_dir, "vv/behaviortree.txt") -template_file = os.path.join(prompt_dir, "vv/template.txt") -task_file = os.path.join(prompt_dir, "vv/task.txt") -system_file = os.path.join(prompt_dir, "vv/system.txt") -object_file = os.path.join(prompt_dir, "vv/object.txt") -domain_file = os.path.join(prompt_dir, "vv/domain.txt") -state_file = os.path.join(prompt_dir, "vv/state.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) -with open(object_file, "r") as f: - object_ppt = PromptTemplate.from_template(f.read()) -with open(behaviortree_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - behaviortree_ppt = ppt_tmp.partial(input=f.read()) - -with open(state_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - state_ppt = ppt_tmp.partial(input=f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {state} - - {object} - - {behaviortree} - - {template} - """ -) - -verifier_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ("object", object_ppt), - ("behaviortree", behaviortree_ppt), - ("state", state_ppt), - ], -) - -verifier_chain = ( - verifier_ppt_ppl - | ChatOpenAI(model="gpt-4-turbo-preview", temperature=0) - | StrOutputParser() -) - - -# * output schema of the recursive_sk_generator -class SolutionBehaviortree(BaseModel): - """the fixed behavior tree""" - - behavior_tree: dict = Field( - description="the behavior tree that adheres the required format." - ) - - -behaviortree_file = os.path.join(prompt_dir, "rec_sk_gen/behaviortree.txt") -template_file = os.path.join(prompt_dir, "rec_sk_gen/template.txt") -task_file = os.path.join(prompt_dir, "rec_sk_gen/task.txt") -system_file = os.path.join(prompt_dir, "rec_sk_gen/system.txt") -object_file = os.path.join(prompt_dir, "rec_sk_gen/object.txt") -domain_file = os.path.join(prompt_dir, "rec_sk_gen/domain.txt") -example_file = os.path.join(prompt_dir, "rec_sk_gen/example.txt") -state_file = os.path.join(prompt_dir, "rec_sk_gen/state.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) -with open(object_file, "r") as f: - object_ppt = PromptTemplate.from_template(f.read()) -with open(behaviortree_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - behaviortree_ppt = ppt_tmp.partial(input=f.read()) - -with open(state_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - state_ppt = ppt_tmp.partial(input=f.read()) - -with open(example_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - example_ppt = ppt_tmp.partial(input=f.read()) -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {state} - - {object} - - {behaviortree} - - {example} - - {template} - """ -) - -re_sk_gen_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ("object", object_ppt), - ("behaviortree", behaviortree_ppt), - ("example", example_ppt), - ("state", state_ppt), - ], -) - -rec_sk_gen_chain = ( - re_sk_gen_ppt_ppl - | ChatOpenAI(model="gpt-4-turbo-preview", temperature=0) - | JsonOutputParser() -) - -# recursive_sk_generator = create_structured_output_runnable( -# SolutionBehaviortree, -# ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), -# re_sk_gen_ppt_ppl, -# mode="openai-json", -# ) - - -##################################################### * planner -# * output schema of the planner -class Plan(BaseModel): - """Plan to follow in future""" - - steps: List[str] = Field( - description="a list of different steps to follow, should be in sorted order" - ) - - -template_file = os.path.join(prompt_dir, "planner/template.txt") -task_file = os.path.join(prompt_dir, "planner/task.txt") -system_file = os.path.join(prompt_dir, "planner/system.txt") -domain_file = os.path.join(prompt_dir, "planner/domain.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {template} - """ -) - -planner_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ], -) - -planner = create_structured_output_runnable( - Plan, ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), planner_ppt_ppl -) - -##################################################### * plan_updaterner - -template_file = os.path.join(prompt_dir, "plan_updater/template.txt") -task_file = os.path.join(prompt_dir, "plan_updater/task.txt") -system_file = os.path.join(prompt_dir, "plan_updater/system.txt") -domain_file = os.path.join(prompt_dir, "plan_updater/domain.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {template} - """ -) - -plan_updater_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ], -) - -plan_updater = create_structured_output_runnable( - Plan, ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), plan_updater_ppt_ppl -) -# ! create openai fn runnable has bug - - -##################################################### * graph node functions -@traceable(name="user_input_node_step") -async def user_input_step(state: PlanExecuteState): - """ - get the input from the user - """ - print(f"-----user_input_step-----") - - user_input = input("Your next instruction: ") - - return { - "user_input": user_input, - } - - -@traceable(name="behavior_tree_generate_step") -async def behavior_tree_generate_step(state: PlanExecuteState): - """ - generate the behavior tree based on the instruction - """ - print(f"-----behavior_tree_generate_step-----") - - instruction = state["plan"][0] - if state["runtime_world_state"] is None: - runtime_world_state = state["world_state"][-1] - else: - runtime_world_state = state["runtime_world_state"] - objects = state["objects"] - - # solution_bt = await recursive_sk_generator.ainvoke( - # { - # "objects": objects, - # "runtime_world_state": runtime_world_state, - # "last_behavior_tree": state["last_behavior_tree"], - # "last_failed_node": state["last_failed_node"], - # "instructions": instruction, - # } - # ) - - bt_skeleton = await rec_sk_gen_chain.ainvoke( - { - "objects": objects, - "runtime_world_state": runtime_world_state, - "last_behavior_tree": state["last_behavior_tree"], - "last_failed_node": state["last_failed_node"], - "summary": state["behavior_tree_execution_summary"], - "instructions": instruction, - } - ) - - # print(solution_bt) - # pause = input("paused here") - - # bt_skeleton = solution_bt.behavior_tree - - render_bt(bt_skeleton) - - pause = input("paused here") - - return { - "last_behavior_tree": bt_skeleton, - "BTExecutionHasSucceeded": False, - } - - -@traceable(name="behavior_tree_execute_step") -async def behavior_tree_execute_step(state: PlanExecuteState): - """ - execute the first step of the plan, append the result to the past steps - """ - print(f"-----behavior_tree_execute_step-----") - - this_step = state["plan"][0] - behavior_tree_skeleton = state["last_behavior_tree"] - latest_world_state = state["world_state"][-1] - - # * first sim run - tree_result, skeleton_json = behavior_tree_stewardship.sk_sim_run( - world_state=latest_world_state, skeleton_json=behavior_tree_skeleton - ) - - pprint(tree_result.to_json()) - pause = input("paused here") - - # * check result - if tree_result.result == "success": - - # * second fake run - tree_result, skeleton_json = behavior_tree_stewardship.sk_fake_run( - world_state=latest_world_state, skeleton_json=behavior_tree_skeleton - ) - - pprint(tree_result.to_json()) - pause = input("paused here") - - if tree_result.result == "success": - return { - "BTExecutionHasSucceeded": True, - "past_steps": ( - this_step, - tree_result.result, - ), # * only one because new plan will be generated and old steps are all removed - "world_state": [tree_result.world_state], - # * clear all - "last_behavior_tree": None, - "last_failed_node": None, - "runtime_world_state": None, - "behavior_tree_execution_summary": None, - } - else: - return { - "world_state": [tree_result.world_state], # * real world state - "last_behavior_tree": skeleton_json, - "last_failed_node": { - "summary": tree_result.final_node.get("summary"), - "name": tree_result.final_node.get("name"), - "identifier": tree_result.final_node.get("identifier"), - }, - "runtime_world_state": tree_result.world_state, - "behavior_tree_execution_summary": tree_result.summary, - } - - else: - return { - # ! do not change world state - "last_behavior_tree": skeleton_json, - "last_failed_node": { - "summary": tree_result.final_node.get("summary"), - "name": tree_result.final_node.get("name"), - "identifier": tree_result.final_node.get("identifier"), - }, - "runtime_world_state": tree_result.world_state, - "behavior_tree_execution_summary": tree_result.summary, - } - - -@traceable(name="planner_step") -async def planner_step(state: PlanExecuteState): - """ - plan the steps based on user input and world state - """ - print(f"-----plan_step-----") - - plan = await planner.ainvoke( - { - "user_input": state["user_input"], - "world_state": state["world_state"], - } - ) - return {"plan": plan.steps} - - -@traceable(name="plan_updater_step") -async def plan_updater_step(state: PlanExecuteState): - """ - if return a response, then success, response the use, end. - otherwise, return the updated newplan (normally the same as the old plan with the first step popped out. - """ - print(f"-----plan_updater_step-----") - - # ! BOOOM! BUG! - output = await plan_updater.ainvoke( - { - "user_input": state["user_input"], - "plan": state["plan"], - "world_state": state["world_state"], - "past_steps": state["past_steps"], - } - ) - # if isinstance( - # output, UpdaterResponse - # ): # * determine if it is time to response and end - # return { - # "response": "Your last instruction has been finished." - # } # * Don't need to update. - # else: - return {"plan": output.steps} # * update the plan - - -##################################################### * construct the graph - -workflow = StateGraph(PlanExecuteState) - -workflow.add_node("planner", planner_step) - -workflow.add_node("behavior_tree_generator", behavior_tree_generate_step) - -workflow.add_node("behavior_tree_executor", behavior_tree_execute_step) - -workflow.add_node("plan_updater", plan_updater_step) - -workflow.add_node("user_input_node", user_input_step) - -workflow.set_entry_point("user_input_node") - -workflow.add_edge( - "planner", "behavior_tree_generator" -) # ! may need to add a condition here - -workflow.add_edge("behavior_tree_generator", "behavior_tree_executor") - - -def executor_should_end(state: PlanExecuteState): - """ - end router - """ - print(f"-----executor_should_end-----") - - if state["BTExecutionHasSucceeded"] == True: - state["BTExecutionHasSucceeded"] = False - return True - else: - return False - - -workflow.add_conditional_edges( - "behavior_tree_executor", - executor_should_end, - { - True: "plan_updater", - False: "behavior_tree_generator", - }, -) - - -def plan_updater_should_end(state: PlanExecuteState): - """ - end router - """ - print(f"-----plan_updater_should_end-----") - - # if state["response"]: - # return True - # else: - # return False - if state["plan"] == [] or len(state["plan"]) == 0: - return True - else: - return False - - -workflow.add_conditional_edges( - "plan_updater", - plan_updater_should_end, - { - True: "user_input_node", - False: "behavior_tree_generator", - }, -) - - -def user_input_should_end(state: PlanExecuteState): - """ - if the user input is empty, then end - """ - print(f"-----user_input_should_end-----") - - # print(f'user_input: {state["user_input"]}') - # print(f'world_state: {state["world_state"]}') - - if not state["user_input"] or state["user_input"] == "": - return True - else: - return False - - -workflow.add_conditional_edges( - "user_input_node", - user_input_should_end, - { - True: END, - False: "planner", - }, -) - -app = workflow.compile() - -config = {"recursion_limit": 500} - -inputs = { - "world_state": [world_state_json], - "objects": objects, -} - - -async def core_run(): - async for event in app.astream( - inputs, - config=config, - ): - for k, v in event.items(): - if k != "__end__": - print(v) - - -if __name__ == "__main__": - import asyncio - - asyncio.run(core_run()) diff --git a/experiments/chair/rec_sk_seq.py b/experiments/chair/rec_sk_seq.py deleted file mode 100644 index 65c5a20e..00000000 --- a/experiments/chair/rec_sk_seq.py +++ /dev/null @@ -1,682 +0,0 @@ -import json -import os -from pprint import pprint -from typing import List, Tuple, Annotated, TypedDict -import operator - -""" -recursive generation + sequence guidance + simulation -""" - -os.environ["LANGCHAIN_TRACING_V2"] = "true" -os.environ["LANGCHAIN_ENDPOINT"] = "https://api.smith.langchain.com" -os.environ["LANGCHAIN_PROJECT"] = "kios_agent" - -from kios_bt.bt_stewardship import BehaviorTreeStewardship -from kios_scene.scene_factory import SceneFactory -from kios_bt.bt_factory import BehaviorTreeFactory -from kios_robot.robot_interface import RobotInterface -from kios_world.world_interface import WorldInterface - -from kios_agent.llm_supporter import KiosLLMSupporter -from kios_agent.data_types import KiosPromptSkeleton - -from kios_agent.kios_tools import ( - BehaviorTreeExecutorTool, - BehaviorTreeSimulatorTool, - WorldStateQueryTool, -) - -from dotenv import load_dotenv - -from langchain import hub - -from langchain_openai import ChatOpenAI - -from langchain.chains.openai_functions import ( - create_structured_output_runnable, - create_openai_fn_runnable, -) -from langchain_core.utils.function_calling import convert_to_openai_function -from langchain_core.agents import AgentActionMessageLog, AgentFinish -from langchain_core.messages import HumanMessage -from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder -from langchain_core.output_parsers import JsonOutputParser, StrOutputParser -from langchain_core.pydantic_v1 import BaseModel, Field - -from langchain.agents.format_scratchpad import format_to_openai_function_messages -from langchain.agents import create_openai_functions_agent, AgentExecutor - -from langchain.prompts.pipeline import PipelinePromptTemplate -from langchain.prompts.prompt import PromptTemplate - -from langchain_community.document_loaders import TextLoader -from langchain_openai import OpenAIEmbeddings - -from langchain_community.vectorstores import FAISS -from langchain.text_splitter import ( - RecursiveCharacterTextSplitter, - CharacterTextSplitter, -) - -from langgraph.graph import StateGraph, END - -from langsmith import traceable - -load_dotenv() - -from kios_utils.pybt_test import generate_bt_stewardship, render_dot_tree -from kios_utils.pddl_problem_parser import parse_problem_init, parse_problem_objects - - -def render_bt(bt_json: json): - test_class = BehaviorTreeFactory() - bt = test_class.from_json_to_simple_bt(bt_json) - # bt = test_class.from_json_to_tree_root(bt_json) - bt_stewardship = generate_bt_stewardship(bt) - # bt_stewardship.setup(timeout=15) - render_dot_tree(bt_stewardship) - - -####################### dirs -current_dir = os.path.dirname(os.path.abspath(__file__)) -scene_path = os.path.join(current_dir, "scene.json") -# bt_json_file_path = os.path.join(current_dir, "behavior_tree.json") -world_state_path = os.path.join(current_dir, "world_state.json") -problem_path = os.path.join(current_dir, "gearset.problem") -domain_knowledge_path = os.path.join(current_dir, "domain_knowledge.txt") - -####################### problem -with open(problem_path, "r") as file: - problem = file.read() - -####################### scene -with open(scene_path, "r") as file: - scene_json_object = json.load(file) - -scene = SceneFactory().create_scene_from_json(scene_json_object) - -####################### world -world_interface = WorldInterface() -with open(world_state_path, "r") as file: - world_state_json_object = json.load(file) - world_interface.load_world_from_json(world_state_json_object) - -from_problem = parse_problem_init(problem=problem) -objects = parse_problem_objects(problem=problem) -world_interface.update_world(from_problem) - -world_state_json = world_interface.get_world_to_json() -pprint(from_problem) -pprint(world_state_json) -pprint(objects) - - -####################### robot -robot_interface = RobotInterface( - robot_address="127.0.0.1", - robot_port=12000, -) -robot_interface.setup_scene(scene) - -####################### bt_factory -bt_factory = BehaviorTreeFactory( - world_interface=world_interface, - robot_interface=robot_interface, -) - -####################### behavior_tree_stewardship -behavior_tree_stewardship = BehaviorTreeStewardship( - behaviortree_factory=bt_factory, - world_interface=world_interface, - robot_interface=robot_interface, -) - -####################### * tools -# bt_executor_tool = BehaviorTreeExecutorTool(behavior_tree_stewardship) - -bt_sim_tool = BehaviorTreeSimulatorTool(metadata={"bt_stw": behavior_tree_stewardship}) - -# world_state_query_tool = WorldStateQueryTool(behavior_tree_stewardship) - -executor_tools = [bt_sim_tool] - -# * kios data prompt skeleton dir -data_dir = os.environ.get("KIOS_DATA_DIR").format(username=os.getlogin()) -print(data_dir) -prompt_sk_dir = os.path.join(data_dir, "prompt_skeletons") -prompt_dir = os.path.join(data_dir, "prompts") - - -# * the graph state -class PlanExecuteState(TypedDict): - plan: List[str] - behavior_tree: dict - world_state: Annotated[ - List[dict], operator.add - ] # ! to add, you need to make world_state a list of dict - past_steps: Annotated[List[Tuple], operator.add] - user_input: str - objects: dict[str, list[str]] - last_behavior_tree: dict - last_failed_node: dict - runtime_world_state: dict - behavior_tree_execution_summary: str - BTExecutionHasSucceeded: bool - - action_sequence: List[str] - - -########## verifier - -template_file = os.path.join(prompt_dir, "seq_planner/template.txt") -task_file = os.path.join(prompt_dir, "seq_planner/task.txt") -system_file = os.path.join(prompt_dir, "seq_planner/system.txt") -domain_file = os.path.join(prompt_dir, "seq_planner/domain.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {template} - """ -) - -seq_planner_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ], -) - -seq_planner_chain = ( - seq_planner_ppt_ppl - | ChatOpenAI(model="gpt-4-turbo-preview", temperature=0) - | StrOutputParser() -) - - -# * output schema of the recursive_sk_generator -class SolutionBehaviortree(BaseModel): - """the fixed behavior tree""" - - behavior_tree: dict = Field( - description="the behavior tree that adheres the required format." - ) - - -behaviortree_file = os.path.join(prompt_dir, "rec_sk_gen/behaviortree.txt") -template_file = os.path.join(prompt_dir, "rec_sk_gen/template.txt") -task_file = os.path.join(prompt_dir, "rec_sk_gen/task.txt") -system_file = os.path.join(prompt_dir, "rec_sk_gen/system.txt") -object_file = os.path.join(prompt_dir, "rec_sk_gen/object.txt") -domain_file = os.path.join(prompt_dir, "rec_sk_gen/domain.txt") -example_file = os.path.join(prompt_dir, "rec_sk_gen/example.txt") -state_file = os.path.join(prompt_dir, "rec_sk_gen/state.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) -with open(object_file, "r") as f: - object_ppt = PromptTemplate.from_template(f.read()) -with open(behaviortree_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - behaviortree_ppt = ppt_tmp.partial(input=f.read()) - -with open(state_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - state_ppt = ppt_tmp.partial(input=f.read()) - -with open(example_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - example_ppt = ppt_tmp.partial(input=f.read()) -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {state} - - {object} - - {behaviortree} - - {example} - - {template} - """ -) -# * recureive_sk_generator -re_sk_gen_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ("object", object_ppt), - ("behaviortree", behaviortree_ppt), - ("example", example_ppt), - ("state", state_ppt), - ], -) - -rec_sk_gen_chain = ( - re_sk_gen_ppt_ppl - | ChatOpenAI(model="gpt-4-turbo-preview", temperature=0) - | JsonOutputParser() -) - -# recursive_sk_generator = create_structured_output_runnable( -# SolutionBehaviortree, -# ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), -# re_sk_gen_ppt_ppl, -# mode="openai-json", -# ) - - -##################################################### * planner -# * output schema of the planner -class Plan(BaseModel): - """Plan to follow in future""" - - steps: List[str] = Field( - description="a list of different steps to follow, should be in sorted order" - ) - - -template_file = os.path.join(prompt_dir, "planner/template.txt") -task_file = os.path.join(prompt_dir, "planner/task.txt") -system_file = os.path.join(prompt_dir, "planner/system.txt") -domain_file = os.path.join(prompt_dir, "planner/domain.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {template} - """ -) - -planner_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ], -) - -planner = create_structured_output_runnable( - Plan, ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), planner_ppt_ppl -) - -##################################################### * plan_updaterner - -template_file = os.path.join(prompt_dir, "plan_updater/template.txt") -task_file = os.path.join(prompt_dir, "plan_updater/task.txt") -system_file = os.path.join(prompt_dir, "plan_updater/system.txt") -domain_file = os.path.join(prompt_dir, "plan_updater/domain.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {template} - """ -) - -plan_updater_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ], -) - -plan_updater = create_structured_output_runnable( - Plan, ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), plan_updater_ppt_ppl -) -# ! create openai fn runnable has bug - - -##################################################### * graph node functions -@traceable(name="user_input_node_step") -async def user_input_step(state: PlanExecuteState): - """ - get the input from the user - """ - print(f"-----user_input_step-----") - - user_input = input("Your next instruction: ") - - return { - "user_input": user_input, - } - - -@traceable(name="sequence_generate_step") -async def sequence_generate_step(state: PlanExecuteState): - """ - generate the sequence based on the instruction - """ - print(f"-----sequence_generate_step-----") - - instruction = state["plan"][0] - start_world_state = state["world_state"][-1] - objects = state["objects"] - - action_sequence = await seq_planner_chain.ainvoke( - { - "objects": objects, - "start_world_state": start_world_state, - "user_instruction": instruction, - } - ) - - return { - "action_sequence": action_sequence, - } - - -@traceable(name="behavior_tree_generate_step") -async def behavior_tree_generate_step(state: PlanExecuteState): - """ - generate the behavior tree based on the instruction - """ - print(f"-----behavior_tree_generate_step-----") - - instruction = state["plan"][0] - if state["runtime_world_state"] is None: - runtime_world_state = state["world_state"][-1] - else: - runtime_world_state = state["runtime_world_state"] - objects = state["objects"] - - bt_skeleton = await rec_sk_gen_chain.ainvoke( - { - "objects": objects, - "runtime_world_state": runtime_world_state, - "last_behavior_tree": state["last_behavior_tree"], - "last_failed_node": state["last_failed_node"], - "summary": state["behavior_tree_execution_summary"], - "action_sequence": state["action_sequence"], - } - ) - - render_bt(bt_skeleton) - - pause = input("paused here") - - return { - "last_behavior_tree": bt_skeleton, - "BTExecutionHasSucceeded": False, - } - - -@traceable(name="behavior_tree_execute_step") -async def behavior_tree_execute_step(state: PlanExecuteState): - """ - execute the first step of the plan, append the result to the past steps - """ - print(f"-----behavior_tree_execute_step-----") - - this_step = state["plan"][0] - behavior_tree_skeleton = state["last_behavior_tree"] - latest_world_state = state["world_state"][-1] - - # * first sim run - tree_result, skeleton_json = behavior_tree_stewardship.sk_sim_run( - world_state=latest_world_state, skeleton_json=behavior_tree_skeleton - ) - - pprint(tree_result.to_json()) - pause = input("paused here") - - # * check result - if tree_result.result == "success": - - # * second fake run - tree_result, skeleton_json = behavior_tree_stewardship.sk_fake_run( - world_state=latest_world_state, skeleton_json=behavior_tree_skeleton - ) - - pprint(tree_result.to_json()) - pause = input("paused here") - - if tree_result.result == "success": - return { - "BTExecutionHasSucceeded": True, - "past_steps": ( - this_step, - tree_result.result, - ), # * only one because new plan will be generated and old steps are all removed - "world_state": [tree_result.world_state], - # * clear all - "last_behavior_tree": None, - "last_failed_node": None, - "runtime_world_state": None, - "behavior_tree_execution_summary": None, - } - else: - return { - "world_state": [tree_result.world_state], # * real world state - "last_behavior_tree": skeleton_json, - "last_failed_node": { - "summary": tree_result.final_node.get("summary"), - "name": tree_result.final_node.get("name"), - "identifier": tree_result.final_node.get("identifier"), - }, - "runtime_world_state": tree_result.world_state, - "behavior_tree_execution_summary": tree_result.summary, - } - - else: - return { - # ! do not change world state - "last_behavior_tree": skeleton_json, - "last_failed_node": { - "summary": tree_result.final_node.get("summary"), - "name": tree_result.final_node.get("name"), - "identifier": tree_result.final_node.get("identifier"), - }, - "runtime_world_state": tree_result.world_state, - "behavior_tree_execution_summary": tree_result.summary, - } - - -@traceable(name="planner_step") -async def planner_step(state: PlanExecuteState): - """ - plan the steps based on user input and world state - """ - print(f"-----plan_step-----") - - plan = await planner.ainvoke( - { - "user_input": state["user_input"], - "world_state": state["world_state"], - } - ) - return {"plan": plan.steps} - - -@traceable(name="plan_updater_step") -async def plan_updater_step(state: PlanExecuteState): - """ - if return a response, then success, response the use, end. - otherwise, return the updated newplan (normally the same as the old plan with the first step popped out. - """ - print(f"-----plan_updater_step-----") - - # ! BOOOM! BUG! - output = await plan_updater.ainvoke( - { - "user_input": state["user_input"], - "plan": state["plan"], - "world_state": state["world_state"], - "past_steps": state["past_steps"], - } - ) - # if isinstance( - # output, UpdaterResponse - # ): # * determine if it is time to response and end - # return { - # "response": "Your last instruction has been finished." - # } # * Don't need to update. - # else: - return {"plan": output.steps} # * update the plan - - -##################################################### * construct the graph - -workflow = StateGraph(PlanExecuteState) - -workflow.add_node("planner", planner_step) -workflow.add_node("sequence_generator", sequence_generate_step) -workflow.add_node("behavior_tree_generator", behavior_tree_generate_step) -workflow.add_node("behavior_tree_executor", behavior_tree_execute_step) -workflow.add_node("plan_updater", plan_updater_step) -workflow.add_node("user_input_node", user_input_step) -workflow.set_entry_point("user_input_node") -workflow.add_edge("planner", "sequence_generator") -workflow.add_edge("sequence_generator", "behavior_tree_generator") -workflow.add_edge("behavior_tree_generator", "behavior_tree_executor") - - -def executor_should_end(state: PlanExecuteState): - """ - end router - """ - print(f"-----executor_should_end-----") - - if state["BTExecutionHasSucceeded"] == True: - state["BTExecutionHasSucceeded"] = False - return True - else: - return False - - -workflow.add_conditional_edges( - "behavior_tree_executor", - executor_should_end, - { - True: "plan_updater", - False: "behavior_tree_generator", - }, -) - - -def plan_updater_should_end(state: PlanExecuteState): - """ - end router - """ - print(f"-----plan_updater_should_end-----") - - if state["plan"] == [] or len(state["plan"]) == 0: - return True - else: - return False - - -workflow.add_conditional_edges( - "plan_updater", - plan_updater_should_end, - { - True: "user_input_node", - False: "behavior_tree_generator", - }, -) - - -def user_input_should_end(state: PlanExecuteState): - """ - if the user input is empty, then end - """ - print(f"-----user_input_should_end-----") - - if not state["user_input"] or state["user_input"] == "": - return True - else: - return False - - -workflow.add_conditional_edges( - "user_input_node", - user_input_should_end, - { - True: END, - False: "planner", - }, -) - -app = workflow.compile() - -config = {"recursion_limit": 500} - -inputs = { - "world_state": [world_state_json], - "objects": objects, -} - - -async def core_run(): - async for event in app.astream( - inputs, - config=config, - ): - for k, v in event.items(): - if k != "__end__": - print(v) - - -if __name__ == "__main__": - # import asyncio - - # asyncio.run(core_run()) - pass diff --git a/experiments/chair/rec_sk_unit.py b/experiments/chair/rec_sk_unit.py deleted file mode 100644 index 8808bae4..00000000 --- a/experiments/chair/rec_sk_unit.py +++ /dev/null @@ -1,677 +0,0 @@ -import json -import os -from pprint import pprint -from typing import List, Tuple, Annotated, TypedDict -import operator - -os.environ["LANGCHAIN_TRACING_V2"] = "true" -os.environ["LANGCHAIN_ENDPOINT"] = "https://api.smith.langchain.com" -os.environ["LANGCHAIN_PROJECT"] = "kios_agent" - -from kios_bt.bt_stewardship import BehaviorTreeStewardship -from kios_scene.scene_factory import SceneFactory -from kios_bt.bt_factory import BehaviorTreeFactory -from kios_robot.robot_interface import RobotInterface -from kios_world.world_interface import WorldInterface - -from kios_agent.llm_supporter import KiosLLMSupporter -from kios_agent.data_types import KiosPromptSkeleton - -from kios_agent.kios_tools import ( - BehaviorTreeExecutorTool, - BehaviorTreeSimulatorTool, - WorldStateQueryTool, -) - -from dotenv import load_dotenv - -from langchain import hub - -from langchain_openai import ChatOpenAI - -from langchain.chains.openai_functions import ( - create_structured_output_runnable, - create_openai_fn_runnable, -) -from langchain_core.utils.function_calling import convert_to_openai_function -from langchain_core.agents import AgentActionMessageLog, AgentFinish -from langchain_core.messages import HumanMessage -from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder -from langchain_core.output_parsers import JsonOutputParser, StrOutputParser -from langchain_core.pydantic_v1 import BaseModel, Field - -from langchain.agents.format_scratchpad import format_to_openai_function_messages -from langchain.agents import create_openai_functions_agent, AgentExecutor - -from langchain.prompts.pipeline import PipelinePromptTemplate -from langchain.prompts.prompt import PromptTemplate - -from langchain_community.document_loaders import TextLoader -from langchain_openai import OpenAIEmbeddings - -from langchain_community.vectorstores import FAISS -from langchain.text_splitter import ( - RecursiveCharacterTextSplitter, - CharacterTextSplitter, -) - -from langgraph.graph import StateGraph, END - -from langsmith import traceable - -load_dotenv() - -from kios_utils.pybt_test import generate_bt_stewardship, render_dot_tree -from kios_utils.pddl_problem_parser import parse_problem_init, parse_problem_objects - - -def render_bt(bt_json: json): - test_class = BehaviorTreeFactory() - bt = test_class.from_json_to_simple_bt(bt_json) - # bt = test_class.from_json_to_tree_root(bt_json) - bt_stewardship = generate_bt_stewardship(bt) - # bt_stewardship.setup(timeout=15) - render_dot_tree(bt_stewardship) - - -####################### dirs -current_dir = os.path.dirname(os.path.abspath(__file__)) -scene_path = os.path.join(current_dir, "scene.json") -# bt_json_file_path = os.path.join(current_dir, "behavior_tree.json") -world_state_path = os.path.join(current_dir, "world_state.json") -problem_path = os.path.join(current_dir, "gearset.problem") -domain_knowledge_path = os.path.join(current_dir, "domain_knowledge.txt") - -####################### problem -with open(problem_path, "r") as file: - problem = file.read() - -####################### scene -with open(scene_path, "r") as file: - scene_json_object = json.load(file) - -scene = SceneFactory().create_scene_from_json(scene_json_object) - -####################### world -world_interface = WorldInterface() -with open(world_state_path, "r") as file: - world_state_json_object = json.load(file) - world_interface.load_world_from_json(world_state_json_object) - -from_problem = parse_problem_init(problem=problem) -objects = parse_problem_objects(problem=problem) -world_interface.update_world(from_problem) - -world_state_json = world_interface.get_world_to_json() -pprint(from_problem) -pprint(world_state_json) -pprint(objects) - - -####################### robot -robot_interface = RobotInterface( - robot_address="127.0.0.1", - robot_port=12000, -) -robot_interface.setup_scene(scene) - -####################### bt_factory -bt_factory = BehaviorTreeFactory( - world_interface=world_interface, - robot_interface=robot_interface, -) - -####################### behavior_tree_stewardship -behavior_tree_stewardship = BehaviorTreeStewardship( - behaviortree_factory=bt_factory, - world_interface=world_interface, - robot_interface=robot_interface, -) - -####################### * tools -# bt_executor_tool = BehaviorTreeExecutorTool(behavior_tree_stewardship) - -bt_sim_tool = BehaviorTreeSimulatorTool(metadata={"bt_stw": behavior_tree_stewardship}) - -# world_state_query_tool = WorldStateQueryTool(behavior_tree_stewardship) - -executor_tools = [bt_sim_tool] - -# * kios data prompt skeleton dir -data_dir = os.environ.get("KIOS_DATA_DIR").format(username=os.getlogin()) -print(data_dir) -prompt_sk_dir = os.path.join(data_dir, "prompt_skeletons") -prompt_dir = os.path.join(data_dir, "prompts") - - -# * the graph state -class PlanExecuteState(TypedDict): - plan: List[str] - behavior_tree: dict - world_state: Annotated[ - List[dict], operator.add - ] # ! to add, you need to make world_state a list of dict - past_steps: Annotated[List[Tuple], operator.add] - user_input: str - objects: dict[str, list[str]] - last_behavior_tree: dict - last_failed_node: dict - runtime_world_state: dict - behavior_tree_execution_summary: str - BTExecutionHasSucceeded: bool - - action_sequence: List[str] - - -########## verifier - -template_file = os.path.join(prompt_dir, "seq_planner/template.txt") -task_file = os.path.join(prompt_dir, "seq_planner/task.txt") -system_file = os.path.join(prompt_dir, "seq_planner/system.txt") -domain_file = os.path.join(prompt_dir, "seq_planner/domain.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {template} - """ -) - -seq_planner_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ], -) - -seq_planner_chain = ( - seq_planner_ppt_ppl - | ChatOpenAI(model="gpt-4-turbo-preview", temperature=0) - | StrOutputParser() -) - - -# * output schema of the recursive_sk_generator -class SolutionBehaviortree(BaseModel): - """the fixed behavior tree""" - - behavior_tree: dict = Field( - description="the behavior tree that adheres the required format." - ) - - -behaviortree_file = os.path.join(prompt_dir, "rec_sk_gen/behaviortree.txt") -template_file = os.path.join(prompt_dir, "rec_sk_gen/template.txt") -task_file = os.path.join(prompt_dir, "rec_sk_gen/task.txt") -system_file = os.path.join(prompt_dir, "rec_sk_gen/system.txt") -object_file = os.path.join(prompt_dir, "rec_sk_gen/object.txt") -domain_file = os.path.join(prompt_dir, "rec_sk_gen/domain.txt") -example_file = os.path.join(prompt_dir, "rec_sk_gen/example.txt") -state_file = os.path.join(prompt_dir, "rec_sk_gen/state.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) -with open(object_file, "r") as f: - object_ppt = PromptTemplate.from_template(f.read()) -with open(behaviortree_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - behaviortree_ppt = ppt_tmp.partial(input=f.read()) - -with open(state_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - state_ppt = ppt_tmp.partial(input=f.read()) - -with open(example_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - example_ppt = ppt_tmp.partial(input=f.read()) -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {state} - - {object} - - {behaviortree} - - {example} - - {template} - """ -) - -re_sk_gen_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ("object", object_ppt), - ("behaviortree", behaviortree_ppt), - ("example", example_ppt), - ("state", state_ppt), - ], -) - -rec_sk_gen_chain = ( - re_sk_gen_ppt_ppl - | ChatOpenAI(model="gpt-4-turbo-preview", temperature=0) - | JsonOutputParser() -) - -# recursive_sk_generator = create_structured_output_runnable( -# SolutionBehaviortree, -# ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), -# re_sk_gen_ppt_ppl, -# mode="openai-json", -# ) - - -##################################################### * planner -# * output schema of the planner -class Plan(BaseModel): - """Plan to follow in future""" - - steps: List[str] = Field( - description="a list of different steps to follow, should be in sorted order" - ) - - -template_file = os.path.join(prompt_dir, "planner/template.txt") -task_file = os.path.join(prompt_dir, "planner/task.txt") -system_file = os.path.join(prompt_dir, "planner/system.txt") -domain_file = os.path.join(prompt_dir, "planner/domain.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {template} - """ -) - -planner_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ], -) - -planner = create_structured_output_runnable( - Plan, ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), planner_ppt_ppl -) - -##################################################### * plan_updaterner - -template_file = os.path.join(prompt_dir, "plan_updater/template.txt") -task_file = os.path.join(prompt_dir, "plan_updater/task.txt") -system_file = os.path.join(prompt_dir, "plan_updater/system.txt") -domain_file = os.path.join(prompt_dir, "plan_updater/domain.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {template} - """ -) - -plan_updater_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ], -) - -plan_updater = create_structured_output_runnable( - Plan, ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), plan_updater_ppt_ppl -) -# ! create openai fn runnable has bug - - -##################################################### * graph node functions -@traceable(name="user_input_node_step") -async def user_input_step(state: PlanExecuteState): - """ - get the input from the user - """ - print(f"-----user_input_step-----") - - user_input = input("Your next instruction: ") - - return { - "user_input": user_input, - } - - -@traceable(name="sequence_generate_step") -async def sequence_generate_step(state: PlanExecuteState): - """ - generate the sequence based on the instruction - """ - print(f"-----sequence_generate_step-----") - - instruction = state["plan"][0] - start_world_state = state["world_state"][-1] - objects = state["objects"] - - action_sequence = await seq_planner_chain.ainvoke( - { - "objects": objects, - "start_world_state": start_world_state, - "user_instruction": instruction, - } - ) - - return { - "action_sequence": action_sequence, - } - - -@traceable(name="behavior_tree_generate_step") -async def behavior_tree_generate_step(state: PlanExecuteState): - """ - generate the behavior tree based on the instruction - """ - print(f"-----behavior_tree_generate_step-----") - - instruction = state["plan"][0] - if state["runtime_world_state"] is None: - runtime_world_state = state["world_state"][-1] - else: - runtime_world_state = state["runtime_world_state"] - objects = state["objects"] - - bt_skeleton = await rec_sk_gen_chain.ainvoke( - { - "objects": objects, - "runtime_world_state": runtime_world_state, - "last_behavior_tree": state["last_behavior_tree"], - "last_failed_node": state["last_failed_node"], - "summary": state["behavior_tree_execution_summary"], - "action_sequence": state["action_sequence"], - } - ) - - render_bt(bt_skeleton) - - pause = input("paused here") - - return { - "last_behavior_tree": bt_skeleton, - "BTExecutionHasSucceeded": False, - } - - -@traceable(name="behavior_tree_execute_step") -async def behavior_tree_execute_step(state: PlanExecuteState): - """ - execute the first step of the plan, append the result to the past steps - """ - print(f"-----behavior_tree_execute_step-----") - - this_step = state["plan"][0] - behavior_tree_skeleton = state["last_behavior_tree"] - latest_world_state = state["world_state"][-1] - - # * first sim run - tree_result, skeleton_json = behavior_tree_stewardship.sk_sim_run( - world_state=latest_world_state, skeleton_json=behavior_tree_skeleton - ) - - pprint(tree_result.to_json()) - pause = input("paused here") - - # * check result - if tree_result.result == "success": - - # * second fake run - tree_result, skeleton_json = behavior_tree_stewardship.sk_fake_run( - world_state=latest_world_state, skeleton_json=behavior_tree_skeleton - ) - - pprint(tree_result.to_json()) - pause = input("paused here") - - if tree_result.result == "success": - return { - "BTExecutionHasSucceeded": True, - "past_steps": ( - this_step, - tree_result.result, - ), # * only one because new plan will be generated and old steps are all removed - "world_state": [tree_result.world_state], - # * clear all - "last_behavior_tree": None, - "last_failed_node": None, - "runtime_world_state": None, - "behavior_tree_execution_summary": None, - } - else: - return { - "world_state": [tree_result.world_state], # * real world state - "last_behavior_tree": skeleton_json, - "last_failed_node": { - "summary": tree_result.final_node.get("summary"), - "name": tree_result.final_node.get("name"), - "identifier": tree_result.final_node.get("identifier"), - }, - "runtime_world_state": tree_result.world_state, - "behavior_tree_execution_summary": tree_result.summary, - } - - else: - return { - # ! do not change world state - "last_behavior_tree": skeleton_json, - "last_failed_node": { - "summary": tree_result.final_node.get("summary"), - "name": tree_result.final_node.get("name"), - "identifier": tree_result.final_node.get("identifier"), - }, - "runtime_world_state": tree_result.world_state, - "behavior_tree_execution_summary": tree_result.summary, - } - - -@traceable(name="planner_step") -async def planner_step(state: PlanExecuteState): - """ - plan the steps based on user input and world state - """ - print(f"-----plan_step-----") - - plan = await planner.ainvoke( - { - "user_input": state["user_input"], - "world_state": state["world_state"], - } - ) - return {"plan": plan.steps} - - -@traceable(name="plan_updater_step") -async def plan_updater_step(state: PlanExecuteState): - """ - if return a response, then success, response the use, end. - otherwise, return the updated newplan (normally the same as the old plan with the first step popped out. - """ - print(f"-----plan_updater_step-----") - - # ! BOOOM! BUG! - output = await plan_updater.ainvoke( - { - "user_input": state["user_input"], - "plan": state["plan"], - "world_state": state["world_state"], - "past_steps": state["past_steps"], - } - ) - # if isinstance( - # output, UpdaterResponse - # ): # * determine if it is time to response and end - # return { - # "response": "Your last instruction has been finished." - # } # * Don't need to update. - # else: - return {"plan": output.steps} # * update the plan - - -##################################################### * construct the graph - -workflow = StateGraph(PlanExecuteState) - -workflow.add_node("planner", planner_step) -workflow.add_node("sequence_generator", sequence_generate_step) -workflow.add_node("behavior_tree_generator", behavior_tree_generate_step) -workflow.add_node("behavior_tree_executor", behavior_tree_execute_step) -workflow.add_node("plan_updater", plan_updater_step) -workflow.add_node("user_input_node", user_input_step) -workflow.set_entry_point("user_input_node") -workflow.add_edge("planner", "sequence_generator") -workflow.add_edge("sequence_generator", "behavior_tree_generator") -workflow.add_edge("behavior_tree_generator", "behavior_tree_executor") - - -def executor_should_end(state: PlanExecuteState): - """ - end router - """ - print(f"-----executor_should_end-----") - - if state["BTExecutionHasSucceeded"] == True: - state["BTExecutionHasSucceeded"] = False - return True - else: - return False - - -workflow.add_conditional_edges( - "behavior_tree_executor", - executor_should_end, - { - True: "plan_updater", - False: "behavior_tree_generator", - }, -) - - -def plan_updater_should_end(state: PlanExecuteState): - """ - end router - """ - print(f"-----plan_updater_should_end-----") - - if state["plan"] == [] or len(state["plan"]) == 0: - return True - else: - return False - - -workflow.add_conditional_edges( - "plan_updater", - plan_updater_should_end, - { - True: "user_input_node", - False: "behavior_tree_generator", - }, -) - - -def user_input_should_end(state: PlanExecuteState): - """ - if the user input is empty, then end - """ - print(f"-----user_input_should_end-----") - - if not state["user_input"] or state["user_input"] == "": - return True - else: - return False - - -workflow.add_conditional_edges( - "user_input_node", - user_input_should_end, - { - True: END, - False: "planner", - }, -) - -app = workflow.compile() - -config = {"recursion_limit": 500} - -inputs = { - "world_state": [world_state_json], - "objects": objects, -} - - -async def core_run(): - async for event in app.astream( - inputs, - config=config, - ): - for k, v in event.items(): - if k != "__end__": - print(v) - - -if __name__ == "__main__": - import asyncio - - asyncio.run(core_run()) diff --git a/experiments/chair/rec_vv.py b/experiments/chair/rec_vv.py deleted file mode 100644 index ad534551..00000000 --- a/experiments/chair/rec_vv.py +++ /dev/null @@ -1,732 +0,0 @@ -import json -import os -from pprint import pprint - -os.environ["LANGCHAIN_TRACING_V2"] = "true" -os.environ["LANGCHAIN_ENDPOINT"] = "https://api.smith.langchain.com" -os.environ["LANGCHAIN_PROJECT"] = "kios_agent" - -from kios_bt.bt_stewardship import BehaviorTreeStewardship -from kios_scene.scene_factory import SceneFactory -from kios_bt.bt_factory import BehaviorTreeFactory -from kios_robot.robot_interface import RobotInterface -from kios_world.world_interface import WorldInterface - -from kios_agent.llm_supporter import KiosLLMSupporter -from kios_agent.data_types import KiosPromptSkeleton - -from kios_agent.kios_tools import ( - BehaviorTreeExecutorTool, - BehaviorTreeSimulatorTool, - WorldStateQueryTool, -) - -from typing import List, Tuple, Annotated, TypedDict -import operator - -from dotenv import load_dotenv - -from langchain import hub - -from langchain_openai import ChatOpenAI - -from langchain.chains.openai_functions import ( - create_structured_output_runnable, - create_openai_fn_runnable, -) -from langchain_core.utils.function_calling import convert_to_openai_function -from langchain_core.agents import AgentActionMessageLog, AgentFinish -from langchain_core.messages import HumanMessage -from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder -from langchain_core.output_parsers import JsonOutputParser, StrOutputParser -from langchain_core.pydantic_v1 import BaseModel, Field - -from langchain.agents.format_scratchpad import format_to_openai_function_messages -from langchain.agents import create_openai_functions_agent, AgentExecutor - -from langchain.prompts.pipeline import PipelinePromptTemplate -from langchain.prompts.prompt import PromptTemplate - -from langchain_community.document_loaders import TextLoader -from langchain_openai import OpenAIEmbeddings - -from langchain_community.vectorstores import FAISS -from langchain.text_splitter import ( - RecursiveCharacterTextSplitter, - CharacterTextSplitter, -) - -from langgraph.graph import StateGraph, END - -from langsmith import traceable - -load_dotenv() - -from kios_utils.pybt_test import generate_bt_stewardship, render_dot_tree -from kios_utils.pddl_problem_parser import parse_problem_init, parse_problem_objects - - -def render_bt(bt_json: json): - test_class = BehaviorTreeFactory() - bt = test_class.from_json_to_simple_bt(bt_json) - # bt = test_class.from_json_to_tree_root(bt_json) - bt_stewardship = generate_bt_stewardship(bt) - # bt_stewardship.setup(timeout=15) - render_dot_tree(bt_stewardship) - - -####################### dirs -current_dir = os.path.dirname(os.path.abspath(__file__)) -scene_path = os.path.join(current_dir, "scene.json") -# bt_json_file_path = os.path.join(current_dir, "behavior_tree.json") -world_state_path = os.path.join(current_dir, "world_state.json") -problem_path = os.path.join(current_dir, "gearset.problem") -domain_knowledge_path = os.path.join(current_dir, "domain_knowledge.txt") - -####################### problem -with open(problem_path, "r") as file: - problem = file.read() - -####################### scene -with open(scene_path, "r") as file: - scene_json_object = json.load(file) - -scene = SceneFactory().create_scene_from_json(scene_json_object) - -####################### world -world_interface = WorldInterface() -with open(world_state_path, "r") as file: - world_state_json_object = json.load(file) - world_interface.load_world_from_json(world_state_json_object) - -from_problem = parse_problem_init(problem=problem) -objects = parse_problem_objects(problem=problem) -world_interface.update_world(from_problem) - -world_state_json = world_interface.get_world_to_json() -pprint(from_problem) -pprint(world_state_json) -pprint(objects) - - -####################### robot -robot_interface = RobotInterface( - robot_address="127.0.0.1", - robot_port=12000, -) -robot_interface.setup_scene(scene) - -####################### bt_factory -bt_factory = BehaviorTreeFactory( - world_interface=world_interface, - robot_interface=robot_interface, -) - -####################### behavior_tree_stewardship -behavior_tree_stewardship = BehaviorTreeStewardship( - behaviortree_factory=bt_factory, - world_interface=world_interface, - robot_interface=robot_interface, -) - -####################### * tools -# bt_executor_tool = BehaviorTreeExecutorTool(behavior_tree_stewardship) - -bt_sim_tool = BehaviorTreeSimulatorTool(metadata={"bt_stw": behavior_tree_stewardship}) - -# world_state_query_tool = WorldStateQueryTool(behavior_tree_stewardship) - -executor_tools = [bt_sim_tool] - -# * kios data prompt skeleton dir -data_dir = os.environ.get("KIOS_DATA_DIR").format(username=os.getlogin()) -print(data_dir) -prompt_sk_dir = os.path.join(data_dir, "prompt_skeletons") -prompt_dir = os.path.join(data_dir, "prompts") - - -# * the graph state -class PlanExecuteState(TypedDict): - plan: List[str] - behavior_tree: dict - world_state: Annotated[ - List[dict], operator.add - ] # ! to add, you need to make world_state a list of dict - past_steps: Annotated[List[Tuple], operator.add] - user_input: str - objects: dict[str, list[str]] - last_behavior_tree: dict - last_failed_node: dict - runtime_world_state: dict - behavior_tree_execution_summary: str - BTExecutionHasSucceeded: bool - - -########## verifier - -behaviortree_file = os.path.join(prompt_dir, "vv/behaviortree.txt") -template_file = os.path.join(prompt_dir, "vv/template.txt") -task_file = os.path.join(prompt_dir, "vv/task.txt") -system_file = os.path.join(prompt_dir, "vv/system.txt") -object_file = os.path.join(prompt_dir, "vv/object.txt") -domain_file = os.path.join(prompt_dir, "vv/domain.txt") -state_file = os.path.join(prompt_dir, "vv/state.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) -with open(object_file, "r") as f: - object_ppt = PromptTemplate.from_template(f.read()) -with open(behaviortree_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - behaviortree_ppt = ppt_tmp.partial(input=f.read()) - -with open(state_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - state_ppt = ppt_tmp.partial(input=f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {state} - - {object} - - {behaviortree} - - {template} - """ -) - -verifier_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ("object", object_ppt), - ("behaviortree", behaviortree_ppt), - ("state", state_ppt), - ], -) - -verifier_chain = ( - verifier_ppt_ppl - | ChatOpenAI(model="gpt-4-turbo-preview", temperature=0) - | StrOutputParser() -) - - -# * output schema of the recursive_sk_generator -class SolutionBehaviortree(BaseModel): - """the fixed behavior tree""" - - behavior_tree: dict = Field( - description="the behavior tree that adheres the required format." - ) - - -behaviortree_file = os.path.join(prompt_dir, "rec_sk_gen/behaviortree.txt") -template_file = os.path.join(prompt_dir, "rec_sk_gen/template.txt") -task_file = os.path.join(prompt_dir, "rec_sk_gen/task.txt") -system_file = os.path.join(prompt_dir, "rec_sk_gen/system.txt") -object_file = os.path.join(prompt_dir, "rec_sk_gen/object.txt") -domain_file = os.path.join(prompt_dir, "rec_sk_gen/domain.txt") -example_file = os.path.join(prompt_dir, "rec_sk_gen/example.txt") -state_file = os.path.join(prompt_dir, "rec_sk_gen/state.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) -with open(object_file, "r") as f: - object_ppt = PromptTemplate.from_template(f.read()) -with open(behaviortree_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - behaviortree_ppt = ppt_tmp.partial(input=f.read()) - -with open(state_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - state_ppt = ppt_tmp.partial(input=f.read()) - -with open(example_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - example_ppt = ppt_tmp.partial(input=f.read()) -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {state} - - {object} - - {behaviortree} - - {example} - - {template} - """ -) - -re_sk_gen_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ("object", object_ppt), - ("behaviortree", behaviortree_ppt), - ("example", example_ppt), - ("state", state_ppt), - ], -) - -rec_sk_gen_chain = ( - re_sk_gen_ppt_ppl - | ChatOpenAI(model="gpt-4-turbo-preview", temperature=0) - | JsonOutputParser() -) - -# recursive_sk_generator = create_structured_output_runnable( -# SolutionBehaviortree, -# ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), -# re_sk_gen_ppt_ppl, -# mode="openai-json", -# ) - - -##################################################### * planner -# * output schema of the planner -class Plan(BaseModel): - """Plan to follow in future""" - - steps: List[str] = Field( - description="a list of different steps to follow, should be in sorted order" - ) - - -template_file = os.path.join(prompt_dir, "planner/template.txt") -task_file = os.path.join(prompt_dir, "planner/task.txt") -system_file = os.path.join(prompt_dir, "planner/system.txt") -domain_file = os.path.join(prompt_dir, "planner/domain.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {template} - """ -) - -planner_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ], -) - -planner = create_structured_output_runnable( - Plan, ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), planner_ppt_ppl -) - -##################################################### * plan_updaterner - -template_file = os.path.join(prompt_dir, "plan_updater/template.txt") -task_file = os.path.join(prompt_dir, "plan_updater/task.txt") -system_file = os.path.join(prompt_dir, "plan_updater/system.txt") -domain_file = os.path.join(prompt_dir, "plan_updater/domain.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {template} - """ -) - -plan_updater_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ], -) - -plan_updater = create_structured_output_runnable( - Plan, ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), plan_updater_ppt_ppl -) -# ! create openai fn runnable has bug - - -##################################################### * graph node functions -@traceable(name="user_input_node_step") -async def user_input_step(state: PlanExecuteState): - """ - get the input from the user - """ - print(f"-----user_input_step-----") - - user_input = input("Your next instruction: ") - - return { - "user_input": user_input, - } - - -@traceable(name="behavior_tree_generate_step") -async def behavior_tree_generate_step(state: PlanExecuteState): - """ - generate the behavior tree based on the instruction - """ - print(f"-----behavior_tree_generate_step-----") - - instruction = state["plan"][0] - if state["runtime_world_state"] is None: - runtime_world_state = state["world_state"][-1] - else: - runtime_world_state = state["runtime_world_state"] - objects = state["objects"] - - bt_skeleton = await rec_sk_gen_chain.ainvoke( - { - "objects": objects, - "runtime_world_state": runtime_world_state, - "last_behavior_tree": state["last_behavior_tree"], - "last_failed_node": state["last_failed_node"], - "summary": state["behavior_tree_execution_summary"], - "instructions": instruction, - } - ) - - render_bt(bt_skeleton) - - pause = input("paused here") - - return { - "last_behavior_tree": bt_skeleton, - "BTExecutionHasSucceeded": False, - } - - -@traceable(name="behavior_tree_execute_step") -async def behavior_tree_execute_step(state: PlanExecuteState): - """ - execute the first step of the plan, append the result to the past steps - """ - print(f"-----behavior_tree_execute_step-----") - - this_step = state["plan"][0] - behavior_tree_skeleton = state["last_behavior_tree"] - latest_world_state = state["world_state"][-1] - - # * first sim run - tree_result, skeleton_json = behavior_tree_stewardship.sk_sim_run( - world_state=latest_world_state, skeleton_json=behavior_tree_skeleton - ) - - pprint(tree_result.to_json()) - pause = input("paused here") - - # * check result - if tree_result.result == "success": - - # * second fake run - tree_result, skeleton_json = behavior_tree_stewardship.sk_fake_run( - world_state=latest_world_state, skeleton_json=behavior_tree_skeleton - ) - - pprint(tree_result.to_json()) - pause = input("paused here") - - if tree_result.result == "success": - return { - "BTExecutionHasSucceeded": True, - "past_steps": ( - this_step, - tree_result.result, - ), # * only one because new plan will be generated and old steps are all removed - "world_state": [tree_result.world_state], - # * clear all - "last_behavior_tree": None, - "last_failed_node": None, - "runtime_world_state": None, - "behavior_tree_execution_summary": None, - } - else: - return { - "world_state": [tree_result.world_state], # * real world state - "last_behavior_tree": skeleton_json, - "last_failed_node": { - "summary": tree_result.final_node.get("summary"), - "name": tree_result.final_node.get("name"), - "identifier": tree_result.final_node.get("identifier"), - }, - "runtime_world_state": tree_result.world_state, - "behavior_tree_execution_summary": tree_result.summary, - } - - else: - return { - # ! do not change world state - "last_behavior_tree": skeleton_json, - "last_failed_node": { - "summary": tree_result.final_node.get("summary"), - "name": tree_result.final_node.get("name"), - "identifier": tree_result.final_node.get("identifier"), - }, - "runtime_world_state": tree_result.world_state, - "behavior_tree_execution_summary": tree_result.summary, - } - - -@traceable(name="verifier_step") -async def verifier_step(state: PlanExecuteState): - """ - verify the behavior tree - """ - print(f"-----verifier_step-----") - start_world_state = state["world_state"][-1] - - # * get the objects - objects = state["objects"] - - # * get the last behavior tree - last_behavior_tree = state["last_behavior_tree"] - - # * get the last failed node - last_failed_node = state["last_failed_node"] - - # * get the runtime world state - runtime_world_state = state["runtime_world_state"] - - # * get the behavior tree execution summary - behavior_tree_execution_summary = state["behavior_tree_execution_summary"] - - # * get the verifier response - verifier_response = await verifier_chain.ainvoke( - { - "objects": objects, - "start_world_state": start_world_state, - "last_behavior_tree": last_behavior_tree, - "last_failed_node": last_failed_node, - "runtime_world_state": runtime_world_state, - "summary": behavior_tree_execution_summary, - } - ) - - # * return the verifier response - return { - "behavior_tree_execution_summary": verifier_response, - } - - -@traceable(name="planner_step") -async def planner_step(state: PlanExecuteState): - """ - plan the steps based on user input and world state - """ - print(f"-----plan_step-----") - - plan = await planner.ainvoke( - { - "user_input": state["user_input"], - "world_state": state["world_state"], - } - ) - return {"plan": plan.steps} - - -@traceable(name="plan_updater_step") -async def plan_updater_step(state: PlanExecuteState): - """ - if return a response, then success, response the use, end. - otherwise, return the updated newplan (normally the same as the old plan with the first step popped out. - """ - print(f"-----plan_updater_step-----") - - # ! BOOOM! BUG! - output = await plan_updater.ainvoke( - { - "user_input": state["user_input"], - "plan": state["plan"], - "world_state": state["world_state"], - "past_steps": state["past_steps"], - } - ) - # if isinstance( - # output, UpdaterResponse - # ): # * determine if it is time to response and end - # return { - # "response": "Your last instruction has been finished." - # } # * Don't need to update. - # else: - return {"plan": output.steps} # * update the plan - - -##################################################### * construct the graph - -workflow = StateGraph(PlanExecuteState) - -workflow.add_node("planner", planner_step) - -workflow.add_node("behavior_tree_generator", behavior_tree_generate_step) - -workflow.add_node("behavior_tree_executor", behavior_tree_execute_step) - -workflow.add_node("verifier", verifier_step) - -workflow.add_node("plan_updater", plan_updater_step) - -workflow.add_node("user_input_node", user_input_step) - -workflow.set_entry_point("user_input_node") - -workflow.add_edge( - "planner", "behavior_tree_generator" -) # ! may need to add a condition here - -workflow.add_edge("behavior_tree_generator", "behavior_tree_executor") - -workflow.add_edge("verifier", "behavior_tree_generator") - - -def executor_should_end(state: PlanExecuteState): - """ - end router - """ - print(f"-----executor_should_end-----") - - if state["BTExecutionHasSucceeded"] == True: - state["BTExecutionHasSucceeded"] = False - return True - else: - return False - - -workflow.add_conditional_edges( - "behavior_tree_executor", - executor_should_end, - { - True: "plan_updater", - False: "verifier", - }, -) - - -def plan_updater_should_end(state: PlanExecuteState): - """ - end router - """ - print(f"-----plan_updater_should_end-----") - - # if state["response"]: - # return True - # else: - # return False - if state["plan"] == [] or len(state["plan"]) == 0: - return True - else: - return False - - -workflow.add_conditional_edges( - "plan_updater", - plan_updater_should_end, - { - True: "user_input_node", - False: "behavior_tree_generator", - }, -) - - -def user_input_should_end(state: PlanExecuteState): - """ - if the user input is empty, then end - """ - print(f"-----user_input_should_end-----") - - # print(f'user_input: {state["user_input"]}') - # print(f'world_state: {state["world_state"]}') - - if not state["user_input"] or state["user_input"] == "": - return True - else: - return False - - -workflow.add_conditional_edges( - "user_input_node", - user_input_should_end, - { - True: END, - False: "planner", - }, -) - -app = workflow.compile() - -config = {"recursion_limit": 500} - -inputs = { - "world_state": [world_state_json], - "objects": objects, -} - - -async def core_run(): - async for event in app.astream( - inputs, - config=config, - ): - for k, v in event.items(): - if k != "__end__": - print(v) - - -if __name__ == "__main__": - import asyncio - - asyncio.run(core_run()) diff --git a/experiments/chair/scene.json b/experiments/chair/scene.json deleted file mode 100644 index 65f4d8c0..00000000 --- a/experiments/chair/scene.json +++ /dev/null @@ -1,147 +0,0 @@ -{ - "reference_objects": - [ - { - "object_name": "test_box1", - "source": "pre-defined", - "O_T_TCP": [ - [1, 0, 0, 0.0], - [0, 1, 0, 0.0], - [0, 0, 1, -0.1], - [0, 0, 0, 1] - ] - }, - { - "object_name": "TestObject1", - "source": "mios" - }, - { - "object_name": "inwardgripper", - "source": "mios" - }, - { - "object_name": "outwardgripper", - "source": "mios" - }, - { - "object_name": "clampgripper", - "source": "mios" - }, - { - "object_name": "parallelgripper", - "source": "mios" - }, - { - "object_name": "defaultgripper", - "source": "mios" - }, - { - "object_name": "chairback", - "source": "mios" - }, - { - "object_name": "chairleg1", - "source": "mios" - }, - { - "object_name": "chairleg2", - "source": "mios" - }, - { - "object_name": "chairnut1", - "source": "mios" - }, - { - "object_name": "chairnut2", - "source": "mios" - }, - { - "object_name": "chairseat", - "source": "mios" - }, - { - "object_name": "chairseatthread1", - "source": "mios" - }, - { - "object_name":"chairseatthread2", - "source": "mios" - }, - { - "object_name": "chairseatbolt1", - "source": "mios" - }, - { - "object_name": "chairseatbolt2", - "source": "mios" - } - ], - "tools": - [ - { - "tool_name": "clampgripper", - "EE_T_TCP": [ - [1, 0, 0, 0.0], - [0, 1, 0, 0.0], - [0, 0, 1, 0.085], - [0, 0, 0, 1] - ], - "EE_finger_width_max": 0.042, - "EE_finger_width_min": 0.01, - "load_width": 0.042, - "unload_width": 0.08 - }, - { - "tool_name": "parallelgripper", - "EE_T_TCP": [ - [1, 0, 0, 0.0], - [0, 1, 0, 0.0], - [0, 0, 1, 0.07], - [0, 0, 0, 1] - ], - "EE_finger_width_max": 0.042, - "EE_finger_width_min": 0.01, - "load_width": 0.042, - "unload_width": 0.08 - }, - { - "tool_name": "inwardgripper", - "EE_T_TCP": [ - [1, 0, 0, 0.0], - [0, 1, 0, 0.0], - [0, 0, 1, 0.095], - [0, 0, 0, 1] - ], - "EE_finger_width_max": 0.042, - "EE_finger_width_min": 0.01, - "load_width": 0.042, - "unload_width": 0.08 - }, - { - "tool_name": "outwardgripper", - "EE_T_TCP": [ - [1, 0, 0, 0.0], - [0, 1, 0, 0.0], - [0, 0, 1, 0.125], - [0, 0, 0, 1] - ], - "EE_finger_width_max": 0.042, - "EE_finger_width_min": 0.01, - "load_width": 0.042, - "unload_width": 0.08 - }, - { - "tool_name": "defaultgripper", - "EE_T_TCP": [ - [1, 0, 0, 0.0], - [0, 1, 0, 0.0], - [0, 0, 1, 0.025], - [0, 0, 0, 1] - ], - "EE_finger_width_max": 0.08, - "EE_finger_width_min": 0.01, - "load_width": 0.08, - "unload_width": 0.08 - } - ] -} diff --git a/experiments/chair/show.py b/experiments/chair/show.py deleted file mode 100644 index ab90d92e..00000000 --- a/experiments/chair/show.py +++ /dev/null @@ -1,1077 +0,0 @@ -result_1 = { - "behavior_tree": { - "summary": "selector to insert gear2 into shaft2", - "name": "selector: insert(left_hand, outward_claw, gear2, shaft2)", - "identifier": 1, - "type_name": "selector", - "children": [ - { - "summary": "condition node to check if gear2 is inserted to shaft2", - "name": "target: is_inserted_to(gear2, shaft2)", - "identifier": 2, - "type_name": "condition", - "conditions": [ - { - "object_name": "gear2", - "property_name": "is_inserted_to", - "property_value": "shaft2", - "status": True, - } - ], - }, - { - "summary": "sequence to insert gear2 into shaft2", - "name": "sequence: insert(left_hand, outward_claw, gear2, shaft2)", - "identifier": 3, - "type_name": "sequence", - "children": [ - { - "summary": "selector to load outward_claw", - "name": "selector: load_tool(left_hand, outward_claw)", - "identifier": 4, - "type_name": "selector", - "children": [ - { - "summary": "condition node to check if left_hand holds outward_claw", - "name": "target: hold(left_hand, outward_claw)", - "identifier": 5, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "outward_claw", - "status": True, - } - ], - }, - { - "summary": "sequence to load outward_claw", - "name": "sequence: load_tool(left_hand, outward_claw)", - "identifier": 6, - "type_name": "sequence", - "children": [ - { - "summary": "condition node to check if outward_claw is equippable", - "name": "precondition: is_equippable(outward_claw)", - "identifier": 7, - "type_name": "condition", - "conditions": [ - { - "object_name": "outward_claw", - "property_name": "is_equippable", - "property_value": None, - "status": True, - } - ], - }, - { - "summary": "condition node to check if left hand is free", - "name": "precondition: is_free(left_hand)", - "identifier": 8, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "is_free", - "property_value": None, - "status": True, - } - ], - }, - { - "summary": "action node to equip outward_claw to left hand", - "name": "action: load_tool(left_hand, outward_claw)", - "identifier": 9, - "type_name": "action", - "effects": [ - { - "object_name": "left_hand", - "property_name": "is_free", - "property_value": None, - "status": False, - }, - { - "object_name": "outward_claw", - "property_name": "is_equippable", - "property_value": None, - "status": False, - }, - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "outward_claw", - "status": True, - }, - ], - }, - ], - }, - ], - }, - { - "summary": "selector to use left_hand with outward_claw to pick up gear2", - "name": "selector: pick_up(left_hand, outward_claw, gear2)", - "identifier": 10, - "type_name": "selector", - "children": [ - { - "summary": "condition node to check if outward_claw holds gear2", - "name": "target: hold(outward_claw, gear2)", - "identifier": 11, - "type_name": "condition", - "conditions": [ - { - "object_name": "outward_claw", - "property_name": "hold", - "property_value": "gear2", - "status": True, - } - ], - }, - { - "summary": "sequence to use left hand with outward_claw to pick up gear2", - "name": "sequence: pick_up(left_hand, outward_claw, gear2)", - "identifier": 12, - "type_name": "sequence", - "children": [ - { - "summary": "condition node to check if outward_claw is free", - "name": "precondition: is_free(outward_claw)", - "identifier": 13, - "type_name": "condition", - "conditions": [ - { - "object_name": "outward_claw", - "property_name": "is_free", - "property_value": None, - "status": True, - } - ], - }, - { - "summary": "condition node to check if left hand holds outward_claw", - "name": "precondition: hold(left_hand, outward_claw)", - "identifier": 14, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "outward_claw", - "status": True, - } - ], - }, - { - "summary": "action node to use left hand with outward_claw to pick up gear2", - "name": "action: pick_up(left_hand, outward_claw, gear2)", - "identifier": 15, - "type_name": "action", - "effects": [ - { - "object_name": "outward_claw", - "property_name": "hold", - "property_value": "gear2", - "status": True, - }, - { - "object_name": "outward_claw", - "property_name": "is_free", - "property_value": None, - "status": False, - }, - ], - }, - ], - }, - ], - }, - { - "summary": "condition node to check if gear2 can be inserted to shaft2", - "name": "precondition: can_insert_to(gear2, shaft2)", - "identifier": 16, - "type_name": "condition", - "conditions": [ - { - "object_name": "gear2", - "property_name": "can_insert_to", - "property_value": "shaft2", - "status": True, - } - ], - }, - { - "summary": "action node to use left_hand with outward_claw to insert gear2 to shaft2", - "name": "action: insert(left_hand, outward_claw, gear2, shaft2)", - "identifier": 17, - "type_name": "action", - "effects": [ - { - "object_name": "outward_claw", - "property_name": "hold", - "property_value": "gear2", - "status": False, - }, - { - "object_name": "outward_claw", - "property_name": "is_free", - "property_value": None, - "status": True, - }, - { - "object_name": "gear2", - "property_name": "is_inserted_to", - "property_value": "shaft2", - "status": True, - }, - ], - }, - ], - }, - ], - }, - "world_state": [ - { - "objects": [ - {"name": "parallel_box1", "properties": ["is_free"]}, - {"name": "shaft1", "properties": []}, - {"name": "gearbase_hole1", "properties": []}, - {"name": "left_hand", "properties": []}, - ], - "constraints": [ - { - "source": "parallel_box1", - "name": "can_manipulate", - "target": "shaft1", - }, - { - "source": "shaft1", - "name": "can_insert_to", - "target": "gearbase_hole1", - }, - ], - "relations": [ - {"source": "left_hand", "name": "hold", "target": "parallel_box1"}, - { - "source": "shaft1", - "name": "is_inserted_to", - "target": "gearbase_hole1", - }, - ], - } - ], -} -# -------------------------------------------------------- - -result = { - "behavior_tree": { - "summary": "selector to insert shaft1 into gearbase_hole1", - "name": "selector: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)", - "identifier": 1, - "type_name": "selector", - "children": [ - { - "summary": "condition node to check if shaft1 is inserted to gearbase_hole1", - "name": "target: is_inserted_to(shaft1, gearbase_hole1)", - "identifier": 2, - "type_name": "condition", - "conditions": [ - { - "object_name": "shaft1", - "property_name": "is_inserted_to", - "property_value": "gearbase_hole1", - "status": True, - } - ], - }, - { - "summary": "sequence to insert shaft1 into gearbase_hole1", - "name": "sequence: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)", - "identifier": 3, - "type_name": "sequence", - "children": [ - { - "summary": "selector to unload outward_claw", - "name": "selector: unload_tool(left_hand, outward_claw)", - "identifier": 4, - "type_name": "selector", - "children": [ - { - "summary": "condition node to check if left_hand is free", - "name": "target: is_free(left_hand)", - "identifier": 5, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "is_free", - "property_value": None, - "status": True, - } - ], - }, - { - "summary": "sequence to unload outward_claw", - "name": "sequence: unload_tool(left_hand, outward_claw)", - "identifier": 6, - "type_name": "sequence", - "children": [ - { - "summary": "condition node to check if left hand holds outward_claw", - "name": "precondition: hold(left_hand, outward_claw)", - "identifier": 7, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "outward_claw", - "status": True, - } - ], - }, - { - "summary": "action node to unload outward_claw from left hand", - "name": "action: unload_tool(left_hand, outward_claw)", - "identifier": 8, - "type_name": "action", - "effects": [ - { - "object_name": "left_hand", - "property_name": "is_free", - "property_value": None, - "status": True, - }, - { - "object_name": "outward_claw", - "property_name": "is_equippable", - "property_value": None, - "status": True, - }, - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "outward_claw", - "status": False, - }, - ], - }, - ], - }, - ], - }, - { - "summary": "selector to load parallel_box1", - "name": "selector: load_tool(left_hand, parallel_box1)", - "identifier": 9, - "type_name": "selector", - "children": [ - { - "summary": "condition node to check if left_hand holds parallel_box1", - "name": "target: hold(left_hand, parallel_box1)", - "identifier": 10, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "parallel_box1", - "status": True, - } - ], - }, - { - "summary": "sequence to load parallel_box1", - "name": "sequence: load_tool(left_hand, parallel_box1)", - "identifier": 11, - "type_name": "sequence", - "children": [ - { - "summary": "condition node to check if parallel_box1 is equippable", - "name": "precondition: is_equippable(parallel_box1)", - "identifier": 12, - "type_name": "condition", - "conditions": [ - { - "object_name": "parallel_box1", - "property_name": "is_equippable", - "property_value": None, - "status": True, - } - ], - }, - { - "summary": "condition node to check if left hand is free", - "name": "precondition: is_free(left_hand)", - "identifier": 13, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "is_free", - "property_value": None, - "status": True, - } - ], - }, - { - "summary": "action node to equip parallel_box1 to left hand", - "name": "action: load_tool(left_hand, parallel_box1)", - "identifier": 14, - "type_name": "action", - "effects": [ - { - "object_name": "left_hand", - "property_name": "is_free", - "property_value": None, - "status": False, - }, - { - "object_name": "parallel_box1", - "property_name": "is_equippable", - "property_value": None, - "status": False, - }, - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "parallel_box1", - "status": True, - }, - ], - }, - ], - }, - ], - }, - { - "summary": "selector to pick up shaft1 with parallel_box1", - "name": "selector: pick_up(left_hand, parallel_box1, shaft1)", - "identifier": 15, - "type_name": "selector", - "children": [ - { - "summary": "condition node to check if parallel_box1 holds shaft1", - "name": "target: hold(parallel_box1, shaft1)", - "identifier": 16, - "type_name": "condition", - "conditions": [ - { - "object_name": "parallel_box1", - "property_name": "hold", - "property_value": "shaft1", - "status": True, - } - ], - }, - { - "summary": "sequence to pick up shaft1 with parallel_box1", - "name": "sequence: pick_up(left_hand, parallel_box1, shaft1)", - "identifier": 17, - "type_name": "sequence", - "children": [ - { - "summary": "condition node to check if parallel_box1 is free", - "name": "precondition: is_free(parallel_box1)", - "identifier": 18, - "type_name": "condition", - "conditions": [ - { - "object_name": "parallel_box1", - "property_name": "is_free", - "property_value": None, - "status": True, - } - ], - }, - { - "summary": "condition node to check if left hand holds parallel_box1", - "name": "precondition: hold(left_hand, parallel_box1)", - "identifier": 19, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "parallel_box1", - "status": True, - } - ], - }, - { - "summary": "action node to pick up shaft1 with parallel_box1", - "name": "action: pick_up(left_hand, parallel_box1, shaft1)", - "identifier": 20, - "type_name": "action", - "effects": [ - { - "object_name": "parallel_box1", - "property_name": "hold", - "property_value": "shaft1", - "status": True, - }, - { - "object_name": "parallel_box1", - "property_name": "is_free", - "property_value": None, - "status": False, - }, - ], - }, - ], - }, - ], - }, - { - "summary": "condition node to check if shaft1 can be inserted to gearbase_hole1", - "name": "precondition: can_insert_to(shaft1, gearbase_hole1)", - "identifier": 21, - "type_name": "condition", - "conditions": [ - { - "object_name": "shaft1", - "property_name": "can_insert_to", - "property_value": "gearbase_hole1", - "status": True, - } - ], - }, - { - "summary": "action node to insert shaft1 into gearbase_hole1", - "name": "action: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)", - "identifier": 22, - "type_name": "action", - "effects": [ - { - "object_name": "parallel_box1", - "property_name": "hold", - "property_value": "shaft1", - "status": False, - }, - { - "object_name": "parallel_box1", - "property_name": "is_free", - "property_value": None, - "status": True, - }, - { - "object_name": "shaft1", - "property_name": "is_inserted_to", - "property_value": "gearbase_hole1", - "status": True, - }, - ], - }, - ], - }, - ], - }, - "world_state": [ - { - "objects": [ - {"name": "parallel_box1", "properties": ["is_free", "is_equippable"]}, - {"name": "parallel_box2", "properties": ["is_free", "is_equippable"]}, - {"name": "inward_claw", "properties": ["is_free", "is_equippable"]}, - {"name": "outward_claw", "properties": ["is_free"]}, - {"name": "no_tool", "properties": ["is_free", "is_equippable"]}, - {"name": "gear1", "properties": []}, - {"name": "gear2", "properties": []}, - {"name": "gear3", "properties": []}, - {"name": "shaft1", "properties": []}, - {"name": "shaft2", "properties": []}, - {"name": "shaft3", "properties": []}, - {"name": "gearbase", "properties": []}, - {"name": "gearbase_hole1", "properties": []}, - {"name": "gearbase_hole3", "properties": []}, - {"name": "left_hand", "properties": []}, - ], - "constraints": [ - { - "source": "parallel_box1", - "name": "can_manipulate", - "target": "shaft1", - }, - { - "source": "parallel_box2", - "name": "can_manipulate", - "target": "gear1", - }, - {"source": "outward_claw", "name": "can_manipulate", "target": "gear2"}, - {"source": "outward_claw", "name": "can_manipulate", "target": "gear3"}, - {"source": "no_tool", "name": "can_manipulate", "target": "shaft3"}, - { - "source": "shaft1", - "name": "can_insert_to", - "target": "gearbase_hole1", - }, - { - "source": "shaft3", - "name": "can_insert_to", - "target": "gearbase_hole3", - }, - {"source": "gear3", "name": "can_insert_to", "target": "shaft3"}, - ], - "relations": [ - {"source": "left_hand", "name": "hold", "target": "outward_claw"} - ], - } - ], -} - -import json -from kios_bt.bt_factory import BehaviorTreeFactory -from kios_utils.pybt_test import generate_bt_stewardship, render_dot_tree - -example1 = { - "summary": "selector to load left_hand with parallel_box1", - "name": "selector: load_tool(left_hand, parallel_box1)", - "children": [ - { - "summary": "condition node to check if left_hand holds parallel_box1", - "name": "target: hold(left_hand, parallel_box1)", - }, - { - "summary": "sequence to load left_hand with parallel_box1", - "name": "sequence: load_tool(left_hand, parallel_box1)", - "children": [ - { - "summary": "selector to make left_hand unload outward_claw", - "name": "selector: unload_tool(left_hand, outward_claw)", - "children": [ - { - "summary": "condition node to check if left_hand is free", - "name": "target: is_free(left_hand)", - }, - { - "summary": "sequence to make left_hand unload outward_claw", - "name": "sequence: unload_tool(left_hand, outward_claw)", - "children": [ - { - "summary": "condition node to check if outward_claw is free", - "name": "precondition: is_free(outward_claw)", - }, - { - "summary": "condition node to check if left hand holds outward_claw", - "name": "precondition: hold(left_hand, outward_claw)", - }, - { - "summary": "action node to make left hand unload outward_claw", - "name": "action: unload_tool(left_hand, outward_claw)", - "effects": [ - { - "summary": "left_hand will be free", - }, - { - "summary": "outward_claw will be equippable", - }, - { - "summary": "left_hand will not hold outward_claw", - }, - ], - }, - ], - }, - ], - }, - { - "summary": "condition node to check if parallel_box1 is equippable", - "name": "precondition: is_equippable(parallel_box1)", - }, - { - "summary": "action node to equip left hand with parallel_box1", - "name": "action: load_tool(left_hand, parallel_box1)", - "effects": [ - { - "summary": "left_hand will be not free", - }, - { - "summary": "parallel_box1 will be not equippable", - }, - { - "summary": "left_hand will hold parallel_box1", - }, - ], - }, - ], - }, - ], -} - -result3 = { - "summary": "selector to insert shaft1 into gearbase_hole1", - "name": "selector: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)", - "children": [ - { - "summary": "condition node to check if shaft1 is inserted into gearbase_hole1", - "name": "target: is_inserted_to(shaft1, gearbase_hole1)", - }, - { - "summary": "sequence to insert shaft1 into gearbase_hole1", - "name": "sequence: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)", - "children": [ - { - "summary": "selector to unload outward_claw from left_hand", - "name": "selector: unload_tool(left_hand, outward_claw)", - "children": [ - { - "summary": "condition node to check if left_hand is free", - "name": "target: is_free(left_hand)", - }, - { - "summary": "sequence to unload outward_claw from left_hand", - "name": "sequence: unload_tool(left_hand, outward_claw)", - "children": [ - { - "summary": "condition node to check if left hand holds outward_claw", - "name": "precondition: hold(left_hand, outward_claw)", - }, - { - "summary": "action node to unload outward_claw from left hand", - "name": "action: unload_tool(left_hand, outward_claw)", - "effects": [ - { - "summary": "left_hand will be free", - "name": "effect: is_free(left_hand)", - }, - { - "summary": "outward_claw will be equippable", - "name": "effect: is_equippable(outward_claw)", - }, - { - "summary": "left_hand will not hold outward_claw", - "name": "effect: not hold(left_hand, outward_claw)", - }, - ], - }, - ], - }, - ], - }, - { - "summary": "selector to load parallel_box1 on left_hand", - "name": "selector: load_tool(left_hand, parallel_box1)", - "children": [ - { - "summary": "condition node to check if left_hand holds parallel_box1", - "name": "target: hold(left_hand, parallel_box1)", - }, - { - "summary": "sequence to load parallel_box1 on left_hand", - "name": "sequence: load_tool(left_hand, parallel_box1)", - "children": [ - { - "summary": "condition node to check if parallel_box1 is equippable", - "name": "precondition: is_equippable(parallel_box1)", - }, - { - "summary": "condition node to check if left hand is free", - "name": "precondition: is_free(left_hand)", - }, - { - "summary": "action node to load parallel_box1 on left hand", - "name": "action: load_tool(left_hand, parallel_box1)", - "effects": [ - { - "summary": "left_hand will not be free", - "name": "effect: not is_free(left_hand)", - }, - { - "summary": "parallel_box1 will not be equippable", - "name": "effect: not is_equippable(parallel_box1)", - }, - { - "summary": "left_hand will hold parallel_box1", - "name": "effect: hold(left_hand, parallel_box1)", - }, - ], - }, - ], - }, - ], - }, - { - "summary": "condition node to check if parallel_box1 can manipulate shaft1", - "name": "precondition: can_manipulate(parallel_box1, shaft1)", - }, - { - "summary": "condition node to check if shaft1 can be inserted to gearbase_hole1", - "name": "precondition: can_insert_to(shaft1, gearbase_hole1)", - }, - { - "summary": "action node to insert shaft1 into gearbase_hole1", - "name": "action: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)", - "effects": [ - { - "summary": "parallel_box1 will not hold shaft1", - "name": "effect: not hold(parallel_box1, shaft1)", - }, - { - "summary": "parallel_box1 will be free", - "name": "effect: is_free(parallel_box1)", - }, - { - "summary": "shaft1 will be inserted into gearbase_hole1", - "name": "effect: is_inserted_to(shaft1, gearbase_hole1)", - }, - ], - }, - ], - }, - ], -} - -example2 = { - "summary": "selector to insert gear1 into shaft1", - "name": "selector: insert(left_hand, parallel_box1, gear1, shaft1)", - "children": [ - { - "summary": "condition node to check if gear1 is inserted to shaft1", - "name": "target: is_inserted_to(gear1, shaft1)", - }, - { - "summary": "sequence to insert gear1 into shaft1", - "name": "sequence: insert(left_hand, parallel_box1, gear1, shaft1)", - "children": [ - { - "summary": "selector to load parallel_box1", - "name": "selector: load_tool(left_hand, parallel_box1)", - "children": [ - { - "summary": "condition node to check if left_hand holds parallel_box1", - "name": "target: hold(left_hand, parallel_box1)", - }, - { - "summary": "sequence to load parallel_box1", - "name": "sequence: load_tool(left_hand, parallel_box1)", - "children": [ - { - "summary": "condition node to check if parallel_box1 is equippable", - "name": "precondition: is_equippable(parallel_box1)", - }, - { - "summary": "selector to make left_hand unload outward_claw", - "name": "selector: unload_tool(left_hand, outward_claw)", - "children": [ - { - "summary": "condition node to check if left_hand is free", - "name": "target: is_free(left_hand)", - }, - { - "summary": "sequence to make left_hand unload outward_claw", - "name": "sequence: unload_tool(left_hand, outward_claw)", - "children": [ - { - "summary": "condition node to check if outward_claw is free", - "name": "precondition: is_free(outward_claw)", - }, - { - "summary": "condition node to check if left hand holds outward_claw", - "name": "precondition: hold(left_hand, outward_claw)", - }, - { - "summary": "action node to make left hand unload outward_claw", - "name": "action: unload_tool(left_hand, outward_claw)", - "effects": [ - { - "summary": "left_hand will be free", - }, - { - "summary": "outward_claw will be equippable", - }, - { - "summary": "left_hand will not hold outward_claw", - }, - ], - }, - ], - }, - ], - }, - { - "summary": "action node to equip parallel_box1 to left hand", - "name": "action: load_tool(left_hand, parallel_box1)", - "effects": [ - { - "summary": "left_hand will be not free", - }, - { - "summary": "parallel_box1 will be not equippable", - }, - { - "summary": "left_hand will hold parallel_box1", - }, - ], - }, - ], - }, - ], - }, - { - "summary": "selector to use left_hand with parallel_box1 to pick up gear1", - "name": "selector: pick_up(left_hand, parallel_box1, gear1)", - "children": [ - { - "summary": "condition node to check if parallel_box1 holds gear1", - "name": "target: hold(parallel_box1, gear1)", - }, - { - "summary": "sequence to use left hand with parallel_box1 to pick up gear1", - "name": "sequence: pick_up(left_hand, parallel_box1, gear1)", - "children": [ - { - "summary": "condition node to check if parallel_box1 is free", - "name": "precondition: is_free(parallel_box1)", - }, - { - "summary": "condition node to check if left hand holds parallel_box1", - "name": "precondition: hold(left_hand, parallel_box1)", - }, - { - "summary": "action node to use left hand with parallel_box1 to pick up gear1", - "name": "action: pick_up(left_hand, parallel_box1, gear1)", - "effects": [ - { - "summary": "parallel_box1 will hold gear1", - }, - { - "summary": "parallel_box1 will be not free", - }, - ], - }, - ], - }, - ], - }, - { - "summary": "condition node to check if gear1 can be inserted to shaft1", - "name": "precondition: can_insert_to(gear1, shaft1)", - }, - { - "summary": "action node to use left_hand with parallel_box1 to insert gear1 to shaft1", - "name": "action: insert(left_hand, parallel_box1, gear1, shaft1)", - "effects": [ - { - "summary": "parallel_box1 will be not holding gear1", - }, - { - "summary": "parallel_box1 will be free", - }, - { - "summary": "gear1 will be inserted to shaft1", - }, - ], - }, - ], - }, - ], -} - -result = { - "summary": "Selector to insert shaft1 into gearbase_hole1 using parallel_box1 in the left hand", - "name": "selector: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)", - "children": [ - { - "summary": "Check the target that shaft1 is inserted into gearbase_hole1", - "name": "target: is_inserted_to(shaft1, gearbase_hole1)", - }, - { - "summary": "Sequence to insert shaft1 into gearbase_hole1 using parallel_box1 in the left hand", - "name": "sequence: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)", - "children": [ - { - "summary": "Selector to load the parallel_box1 in the left hand", - "name": "selector: load_tool(left_hand, parallel_box1)", - "children": [ - { - "summary": "Check the target that the left hand is free", - "name": "target: is_free(left_hand)", - }, - { - "summary": "Sequence to load the tool in the left hand", - "name": "sequence: load_tool(left_hand, parallel_box1)", - "children": [ - { - "summary": "Check the precondition that the parallel_box1 is equippable", - "name": "precondition: is_equippable(parallel_box1)", - }, - { - "summary": "Selector to unload the outward_claw in the left hand", - "name": "selector: unload_tool(left_hand, outward_claw)", - "children": [ - { - "summary": "Check the target that the left hand is free", - "name": "target: is_free(left_hand)", - }, - { - "summary": "Sequence to unload the tool in the left hand", - "name": "sequence: unload_tool(left_hand, outward_claw)", - "children": [ - { - "summary": "Check the precondition that the left hand is holding a outward_claw", - "name": "precondition: hold(left_hand, outward_claw)", - }, - { - "summary": "Unload the outward_claw in the left hand", - "name": "action: unload_tool(left_hand, outward_claw)", - }, - ], - }, - ], - }, - { - "summary": "Load the parallel_box1 in the left hand", - "name": "action: load_tool(left_hand, parallel_box1)", - }, - ], - }, - ], - }, - { - "summary": "Selector to pick up the shaft1 with the parallel_box1 in the left hand", - "name": "selector: pick_up(left_hand, parallel_box1, shaft1)", - "children": [ - { - "summary": "Check the target that the parallel_box1 is holding the shaft1", - "name": "target: hold(parallel_box1, shaft1)", - }, - { - "summary": "Sequence to pick up the shaft1 with the parallel_box1 in the left hand", - "name": "sequence: pick_up(left_hand, parallel_box1, shaft1)", - "children": [ - { - "summary": "Check the precondition that the left hand is holding the parallel_box1", - "name": "precondition: hold(left_hand, parallel_box1)", - }, - { - "summary": "Pick up the shaft1 with the parallel_box1 in the left hand", - "name": "action: pick_up(left_hand, parallel_box1, shaft1)", - }, - ], - }, - ], - }, - { - "summary": "Insert shaft1 into gearbase_hole1 using parallel_box1", - "name": "action: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)", - }, - ], - }, - ], -} - - -def test_bt(bt_json: json): - test_class = BehaviorTreeFactory() - bt = test_class.from_json_to_simple_bt(bt_json) - # bt = test_class.from_json_to_tree_root(bt_json) - bt_stewardship = generate_bt_stewardship(bt) - # bt_stewardship.setup(timeout=15) - render_dot_tree(bt_stewardship) - - -# test_bt(result_1["behavior_tree"]) -# test_bt(example1) -# test_bt(result3) -# test_bt(example2) -test_bt(result) diff --git a/experiments/chair/sk_gen_test.py b/experiments/chair/sk_gen_test.py deleted file mode 100644 index ba4dfd8e..00000000 --- a/experiments/chair/sk_gen_test.py +++ /dev/null @@ -1,662 +0,0 @@ -import json -import os -from pprint import pprint - -os.environ["LANGCHAIN_TRACING_V2"] = "true" -os.environ["LANGCHAIN_ENDPOINT"] = "https://api.smith.langchain.com" -os.environ["LANGCHAIN_PROJECT"] = "kios_agent" - -from kios_bt.bt_stewardship import BehaviorTreeStewardship -from kios_scene.scene_factory import SceneFactory -from kios_bt.bt_factory import BehaviorTreeFactory -from kios_robot.robot_interface import RobotInterface -from kios_world.world_interface import WorldInterface - -from kios_agent.llm_supporter import KiosLLMSupporter -from kios_agent.data_types import KiosPromptSkeleton - -from kios_agent.kios_tools import ( - BehaviorTreeExecutorTool, - BehaviorTreeSimulatorTool, - WorldStateQueryTool, -) - -from typing import List, Tuple, Annotated, TypedDict -import operator - -from dotenv import load_dotenv - -from langchain import hub - -from langchain_openai import ChatOpenAI - -from langchain.chains.openai_functions import ( - create_structured_output_runnable, - create_openai_fn_runnable, -) -from langchain_core.utils.function_calling import convert_to_openai_function -from langchain_core.agents import AgentActionMessageLog, AgentFinish -from langchain_core.messages import HumanMessage -from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder -from langchain_core.output_parsers import JsonOutputParser, StrOutputParser -from langchain_core.pydantic_v1 import BaseModel, Field - -from langchain.agents.format_scratchpad import format_to_openai_function_messages -from langchain.agents import create_openai_functions_agent, AgentExecutor - -from langchain.prompts.pipeline import PipelinePromptTemplate -from langchain.prompts.prompt import PromptTemplate - -from langchain_community.document_loaders import TextLoader -from langchain_openai import OpenAIEmbeddings - -from langchain_community.vectorstores import FAISS -from langchain.text_splitter import ( - RecursiveCharacterTextSplitter, - CharacterTextSplitter, -) - -from langgraph.graph import StateGraph, END - -from langsmith import traceable - -load_dotenv() - -from kios_utils.pybt_test import generate_bt_stewardship, render_dot_tree -from kios_utils.pddl_problem_parser import parse_problem_init, parse_problem_objects - - -def render_bt(bt_json: json): - test_class = BehaviorTreeFactory() - bt = test_class.from_json_to_simple_bt(bt_json) - # bt = test_class.from_json_to_tree_root(bt_json) - bt_stewardship = generate_bt_stewardship(bt) - # bt_stewardship.setup(timeout=15) - render_dot_tree(bt_stewardship) - - -####################### dirs -current_dir = os.path.dirname(os.path.abspath(__file__)) -scene_path = os.path.join(current_dir, "scene.json") -# bt_json_file_path = os.path.join(current_dir, "behavior_tree.json") -world_state_path = os.path.join(current_dir, "world_state.json") -problem_path = os.path.join(current_dir, "gearset.problem") -domain_knowledge_path = os.path.join(current_dir, "domain_knowledge.txt") - -####################### problem -with open(problem_path, "r") as file: - problem = file.read() - -####################### scene -with open(scene_path, "r") as file: - scene_json_object = json.load(file) - -scene = SceneFactory().create_scene_from_json(scene_json_object) - -####################### world -world_interface = WorldInterface() -with open(world_state_path, "r") as file: - world_state_json_object = json.load(file) - world_interface.load_world_from_json(world_state_json_object) - -from_problem = parse_problem_init(problem=problem) -objects = parse_problem_objects(problem=problem) -world_interface.update_world(from_problem) - -world_state_json = world_interface.get_world_to_json() -pprint(from_problem) -pprint(world_state_json) -pprint(objects) - - -####################### robot -robot_interface = RobotInterface( - robot_address="127.0.0.1", - robot_port=12000, -) -robot_interface.setup_scene(scene) - -####################### bt_factory -bt_factory = BehaviorTreeFactory( - world_interface=world_interface, - robot_interface=robot_interface, -) - -####################### behavior_tree_stewardship -behavior_tree_stewardship = BehaviorTreeStewardship( - behaviortree_factory=bt_factory, - world_interface=world_interface, - robot_interface=robot_interface, -) - -####################### * tools -# bt_executor_tool = BehaviorTreeExecutorTool(behavior_tree_stewardship) - -bt_sim_tool = BehaviorTreeSimulatorTool(metadata={"bt_stw": behavior_tree_stewardship}) - -# world_state_query_tool = WorldStateQueryTool(behavior_tree_stewardship) - -executor_tools = [bt_sim_tool] - -# * kios data prompt skeleton dir -data_dir = os.environ.get("KIOS_DATA_DIR").format(username=os.getlogin()) -print(data_dir) -prompt_sk_dir = os.path.join(data_dir, "prompt_skeletons") -prompt_dir = os.path.join(data_dir, "prompts") - -# * retriever chain -# from langchain_experimental.text_splitter import SemanticChunker - -# # with open(domain_knowledge_path, "r") as f: -# # domain_knowledge = f.read() - -# # text_splitter = SemanticChunker(OpenAIEmbeddings()) - -# # docs = text_splitter.create_documents([domain_knowledge]) - - -# dk_loader = TextLoader(domain_knowledge_path) -# domain_knowledge_doc = dk_loader.load() - -# embeddings = OpenAIEmbeddings() - -# text_splitter = RecursiveCharacterTextSplitter( -# chunk_size=300, -# chunk_overlap=10, -# length_function=len, -# is_separator_regex=False, -# ) -# # text_splitter = CharacterTextSplitter() -# docs = text_splitter.split_documents(domain_knowledge_doc) -# vector_store = FAISS.from_documents(docs, embeddings) - -# retriever = vector_store.as_retriever() - -# docs = retriever.get_relevant_documents( -# "how to fulfill the condition 'is_free'?", -# k=3, -# threshold=0.5, -# ) - -# pprint(docs) - -# * refiner chain -llm_spt_re_sk = KiosLLMSupporter() -with open(os.path.join(prompt_sk_dir, "skeleton_refiner.json"), "r") as f: - re_sk_pptsk_json = json.load(f) - re_sk_pptsk = KiosPromptSkeleton.from_json(re_sk_pptsk_json) - llm_spt_re_sk.initialize_from_prompt_skeleton(re_sk_pptsk) -re_sk_ppt = llm_spt_re_sk.create_prompt() -re_sk_llm = ChatOpenAI(model_name=llm_spt_re_sk.model_name, temperature=0) - -re_sk_chain = re_sk_ppt | re_sk_llm | JsonOutputParser() - - -# * skeleton generator chain -llm_spt_skeleton_generator = KiosLLMSupporter() -with open(os.path.join(prompt_sk_dir, "skeleton_generator.json"), "r") as f: - skeleton_generator_pptsk_json = json.load(f) - skeleton_generator_pptsk = KiosPromptSkeleton.from_json( - skeleton_generator_pptsk_json - ) - llm_spt_skeleton_generator.initialize_from_prompt_skeleton(skeleton_generator_pptsk) -skeleton_generator_ppt = llm_spt_skeleton_generator.create_prompt() -skeleton_generator_llm = ChatOpenAI( - model_name=llm_spt_skeleton_generator.model_name, temperature=0 -) - -skeleton_generator_chain = ( - skeleton_generator_ppt - | skeleton_generator_llm - # | StrOutputParser() - | JsonOutputParser() -) - - -# * the graph state -class PlanExecuteState(TypedDict): - plan: List[str] - behavior_tree: dict - world_state: Annotated[ - List[dict], operator.add - ] # ! to add, you need to make world_state a list of dict - # = Field(default=None) - past_steps: Annotated[List[Tuple], operator.add] - response: str # ! not sure if use this or not. - user_input: str - problem: str - objects: dict[str, list[str]] - - -# * output schema of the planner -class SolutionSubtree(BaseModel): - """solution subtree that will be used to replace the failed condition node in the last behavior tree""" - - subtree: dict = Field( - description="a subtree that adheres the required format, which will be used to replace the failed condition node in the last behavior tree" - ) - - -behaviortree_file = os.path.join(prompt_dir, "rec_sk_gen/behaviortree.txt") -template_file = os.path.join(prompt_dir, "rec_sk_gen/template.txt") -task_file = os.path.join(prompt_dir, "rec_sk_gen/task.txt") -system_file = os.path.join(prompt_dir, "rec_sk_gen/system.txt") -object_file = os.path.join(prompt_dir, "rec_sk_gen/object.txt") -domain_file = os.path.join(prompt_dir, "rec_sk_gen/domain.txt") -example_file = os.path.join(prompt_dir, "rec_sk_gen/example.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {behaviortree} - - {domain} - - {example} - - {template} - """ -) - -planner_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ], -) - -planner = create_structured_output_runnable( - Plan, ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), planner_ppt_ppl -) - - -##################################################### * planner -# * output schema of the planner -class Plan(BaseModel): - """Plan to follow in future""" - - steps: List[str] = Field( - description="a list of different steps to follow, should be in sorted order" - ) - - -template_file = os.path.join(prompt_dir, "planner/template.txt") -task_file = os.path.join(prompt_dir, "planner/task.txt") -system_file = os.path.join(prompt_dir, "planner/system.txt") -domain_file = os.path.join(prompt_dir, "planner/domain.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {template} - """ -) - -planner_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ], -) - -planner = create_structured_output_runnable( - Plan, ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), planner_ppt_ppl -) - -##################################################### * plan_updaterner - -template_file = os.path.join(prompt_dir, "plan_updater/template.txt") -task_file = os.path.join(prompt_dir, "plan_updater/task.txt") -system_file = os.path.join(prompt_dir, "plan_updater/system.txt") -domain_file = os.path.join(prompt_dir, "plan_updater/domain.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {template} - """ -) - -plan_updater_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ], -) - -plan_updater = create_structured_output_runnable( - Plan, ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), plan_updater_ppt_ppl -) -# ! create openai fn runnable has bug - - -##################################################### * graph node functions -@traceable(name="user_input_node_step") -async def user_input_step(state: PlanExecuteState): - """ - get the input from the user - """ - print(f"-----user_input_step-----") - # if state["response"] is not None: - # user_input = input(state["response"]) - # state["response"] = None - # else: - user_input = input("Your next instruction: ") - - return { - "user_input": user_input, - } - - -@traceable(name="behavior_tree_generate_step") -async def behavior_tree_generate_step(state: PlanExecuteState): - """ - generate the behavior tree based on the instruction - """ - print(f"-----behavior_tree_generate_step-----") - - instruction = state["plan"][0] - latest_world_state = state["world_state"][-1] - objects = state["objects"] - - skeleton = await skeleton_generator_chain.ainvoke( - { - # "problem": state["problem"], - "objects": objects, - "world_state": latest_world_state, - "instructions": instruction, - } - ) - - # sk_json = JsonOutputParser().invoke(skeleton) - sk_json = skeleton - - behavior_tree_sk = sk_json["task_plan"]["behavior_tree"] - - render_bt(behavior_tree_sk) - - pause = input("paused here") - - # ! OBJECT CONCENTRATION - - # response = await re_sk_chain.ainvoke( - # { - # "input": skeleton, - # } - # ) - - behavior_tree = skeleton.get("task_plan").get("behavior_tree") - - # init_world_state = [skeleton.get("initial_state")] - - return { - "behavior_tree": behavior_tree, - # "world_state": init_world_state, # ! this is necessary because the constraints in problem - } - - -@traceable(name="behavior_tree_sim_step") -async def behavior_tree_sim_step(state: PlanExecuteState): - """ - simulate the behavior tree based on the world state - """ - print(f"-----behavior_tree_sim_step-----") - - behavior_tree = state["behavior_tree"] - latest_world_state = state["world_state"][-1] - - # exe_input = { - # "behavior_tree": behavior_tree, - # "world_state": latest_world_state, - # } - - # agent_response = await bt_sim_tool.ainvoke( - # { - # "input": exe_input, - # } - # ) - - response = behavior_tree_stewardship.sk_fake_run( - world_state=latest_world_state, bt_json=behavior_tree - ) - - # hasSucceeded = response["hasSucceeded"] - # new_world_state = [response["world_state"]] - hasSucceeded = response.result - new_world_state = [response.world_state] - the_step = state["plan"][0] - - return { - "past_steps": ( - the_step, - hasSucceeded, - ), # * only one because new plan will be generated and old steps are all removed - "world_state": new_world_state, - } - - -@traceable(name="behavior_tree_execute_step") -async def behavior_tree_execute_step(state: PlanExecuteState): - """ - execute the first step of the plan, append the result to the past steps - """ - print(f"-----behavior_tree_execute_step-----") - - behavior_tree = state["behavior_tree"] - latest_world_state = state["world_state"][-1] - - # exe_input = { - # "behavior_tree": behavior_tree, - # "world_state": latest_world_state, - # } - - # agent_response = await bt_exe_agent_executor.ainvoke( - # { - # "input": exe_input, - # } - # ) - - response = behavior_tree_stewardship.sk_fake_run( - world_state=latest_world_state, bt_json=behavior_tree - ) - - # hasSucceeded = response["hasSucceeded"] - # new_world_state = [response["world_state"]] - hasSucceeded = response.result - new_world_state = [response.world_state] - the_step = state["plan"][0] - - return { - "past_steps": ( - the_step, - hasSucceeded, - ), # * only one because new plan will be generated and old steps are all removed - "world_state": new_world_state, - } - - -@traceable(name="planner_step") -async def planner_step(state: PlanExecuteState): - """ - plan the steps based on user input and world state - """ - print(f"-----plan_step-----") - - plan = await planner.ainvoke( - { - "user_input": state["user_input"], - "world_state": state["world_state"], - } - ) - return {"plan": plan.steps} - - -@traceable(name="plan_updater_step") -async def plan_updater_step(state: PlanExecuteState): - """ - if return a response, then success, response the use, end. - otherwise, return the updated newplan (normally the same as the old plan with the first step popped out. - """ - print(f"-----plan_updater_step-----") - - # ! BOOOM! BUG! - output = await plan_updater.ainvoke( - { - "user_input": state["user_input"], - "plan": state["plan"], - "world_state": state["world_state"], - "past_steps": state["past_steps"], - } - ) - # if isinstance( - # output, UpdaterResponse - # ): # * determine if it is time to response and end - # return { - # "response": "Your last instruction has been finished." - # } # * Don't need to update. - # else: - return {"plan": output.steps} # * update the plan - - -##################################################### * construct the graph - -workflow = StateGraph(PlanExecuteState) - -workflow.add_node("planner", planner_step) - -workflow.add_node("behavior_tree_generator", behavior_tree_generate_step) - -workflow.add_node("behavior_tree_executor", behavior_tree_execute_step) - -workflow.add_node("plan_updater", plan_updater_step) - -workflow.add_node("user_input_node", user_input_step) - -workflow.set_entry_point("user_input_node") - -workflow.add_edge( - "planner", "behavior_tree_generator" -) # ! may need to add a condition here - -workflow.add_edge("behavior_tree_generator", "behavior_tree_executor") - -workflow.add_edge("behavior_tree_executor", "plan_updater") - - -def plan_updater_should_end(state: PlanExecuteState): - """ - end router - """ - print(f"-----plan_updater_should_end-----") - - # if state["response"]: - # return True - # else: - # return False - if state["plan"] == [] or len(state["plan"]) == 0: - return True - else: - return False - - -workflow.add_conditional_edges( - "plan_updater", - plan_updater_should_end, - { - True: "user_input_node", - False: "behavior_tree_generator", - }, -) - - -def user_input_should_end(state: PlanExecuteState): - """ - if the user input is empty, then end - """ - print(f"-----user_input_should_end-----") - - # print(f'user_input: {state["user_input"]}') - # print(f'world_state: {state["world_state"]}') - - if not state["user_input"] or state["user_input"] == "": - return True - else: - return False - - -workflow.add_conditional_edges( - "user_input_node", - user_input_should_end, - { - True: END, - False: "planner", - }, -) - -app = workflow.compile() - -config = {"recursion_limit": 500} - -inputs = { - "world_state": [world_state_json], - "objects": objects, -} - - -async def core_run(): - async for event in app.astream( - inputs, - config=config, - ): - for k, v in event.items(): - if k != "__end__": - print(v) - - -if __name__ == "__main__": - import asyncio - - asyncio.run(core_run()) diff --git a/experiments/chair/tasks.txt b/experiments/chair/tasks.txt deleted file mode 100644 index e69de29b..00000000 diff --git a/experiments/chair/test_graph.py b/experiments/chair/test_graph.py deleted file mode 100644 index 60e01153..00000000 --- a/experiments/chair/test_graph.py +++ /dev/null @@ -1,74 +0,0 @@ -import os -from typing import List, Tuple, Annotated, TypedDict -import operator - -os.environ["LANGCHAIN_TRACING_V2"] = "true" -os.environ["LANGCHAIN_ENDPOINT"] = "https://api.smith.langchain.com" -os.environ["LANGCHAIN_PROJECT"] = "kios_agent" - -from langgraph.graph import StateGraph, END - -from langsmith import traceable - - -class PlanExecuteState(TypedDict): - plan: List[str] - behavior_tree_skeleton: dict # ! not sure if use this or not. - behavior_tree: dict - world_state: Annotated[ - List[dict], operator.add - ] # ! to add, you need to make world_state a list of dict - # = Field(default=None) - past_steps: Annotated[List[Tuple], operator.add] - response: str # ! not sure if use this or not. - # to_user: bool - user_input: str - problem: str - - -@traceable(name="user_input_node_step") -async def user_input_step(state: PlanExecuteState): - user_input = input("Enter your next step: ") - return { - "user_input": user_input, - } - - -@traceable(name="user_input_conditional_edge") -def user_input_should_continue(state: PlanExecuteState): - - if state["user_input"] == "gogogo": - return True - else: - return False - - -workflow = StateGraph(PlanExecuteState) - -workflow.add_node("user_input_node", user_input_step) - -workflow.set_entry_point("user_input_node") - -workflow.add_conditional_edges( - "user_input_node", - user_input_should_continue, - { - True: "user_input_node", - False: END, - }, -) - -app = workflow.compile() - - -async def core_run(): - async for event in app.astream(input={}): - for k, v in event.items(): - if k != "__end__": - print(v) - - -if __name__ == "__main__": - import asyncio - - asyncio.run(core_run()) diff --git a/experiments/chair/ut_test.py b/experiments/chair/ut_test.py deleted file mode 100644 index b4226a72..00000000 --- a/experiments/chair/ut_test.py +++ /dev/null @@ -1,1208 +0,0 @@ -import json -import os -import re -from pprint import pprint -from typing import List, Tuple, Annotated, TypedDict -import operator -import logging -import asyncio - -""" -unit tree generation -""" - -os.environ["LANGCHAIN_TRACING_V2"] = "true" -os.environ["LANGCHAIN_ENDPOINT"] = "https://api.smith.langchain.com" -os.environ["LANGCHAIN_PROJECT"] = "kios_agent" - -from kios_bt.bt_stewardship import BehaviorTreeStewardship -from kios_scene.scene_factory import SceneFactory -from kios_bt.bt_factory import BehaviorTreeFactory -from kios_robot.robot_interface import RobotInterface -from kios_world.world_interface import WorldInterface - -from kios_agent.llm_supporter import KiosLLMSupporter -from kios_agent.data_types import KiosPromptSkeleton - -from kios_agent.kios_tools import ( - BehaviorTreeExecutorTool, - BehaviorTreeSimulatorTool, - WorldStateQueryTool, -) - -from dotenv import load_dotenv - -from langchain import hub - -from langchain_openai import ChatOpenAI - -from langchain.chains.openai_functions import ( - create_structured_output_runnable, - create_openai_fn_runnable, -) -from langchain_core.utils.function_calling import convert_to_openai_function -from langchain_core.agents import AgentActionMessageLog, AgentFinish -from langchain_core.messages import HumanMessage -from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder -from langchain_core.output_parsers import JsonOutputParser, StrOutputParser -from langchain_core.pydantic_v1 import BaseModel, Field - -from langchain.agents.format_scratchpad import format_to_openai_function_messages -from langchain.agents import create_openai_functions_agent, AgentExecutor - -from langchain.prompts.pipeline import PipelinePromptTemplate -from langchain.prompts.prompt import PromptTemplate - -from langchain_community.document_loaders import TextLoader -from langchain_openai import OpenAIEmbeddings - -from langchain_community.vectorstores import FAISS -from langchain.text_splitter import ( - RecursiveCharacterTextSplitter, - CharacterTextSplitter, -) - -from langgraph.graph import StateGraph, END - -from langsmith import traceable - -load_dotenv() - -from kios_utils.pybt_test import generate_bt_stewardship, render_dot_tree -from kios_utils.pddl_problem_parser import parse_problem_init, parse_problem_objects - - -def render_bt(bt_json: dict): - test_class = BehaviorTreeFactory() - bt = test_class.from_json_to_simple_bt(bt_json) - # bt = test_class.from_json_to_tree_root(bt_json) - bt_stewardship = generate_bt_stewardship(bt) - # bt_stewardship.setup(timeout=15) - render_dot_tree(bt_stewardship) - - -####################### dirs -current_dir = os.path.dirname(os.path.abspath(__file__)) -scene_path = os.path.join(current_dir, "scene.json") -# bt_json_file_path = os.path.join(current_dir, "behavior_tree.json") -world_state_path = os.path.join(current_dir, "world_state.json") -problem_path = os.path.join(current_dir, "gearset.problem") -domain_knowledge_path = os.path.join(current_dir, "domain_knowledge.txt") - -####################### problem -with open(problem_path, "r") as file: - problem = file.read() - -####################### scene -with open(scene_path, "r") as file: - scene_json_object = json.load(file) - -scene = SceneFactory().create_scene_from_json(scene_json_object) - -####################### world -world_interface = WorldInterface() -with open(world_state_path, "r") as file: - world_state_json_object = json.load(file) - pprint(world_state_json_object) - world_interface.load_world_from_json(world_state_json_object) - -# from_problem = parse_problem_init(problem=problem) -# objects = parse_problem_objects(problem=problem) -# world_interface.update_world(from_problem) - -# world_state_json = world_interface.get_world_to_json() -# pprint(from_problem) -# pprint(world_state_json) -# pprint(objects) - - -####################### robot -robot_interface = RobotInterface( - robot_address="127.0.0.1", - robot_port=12000, -) -robot_interface.setup_scene(scene) - -####################### bt_factory -bt_factory = BehaviorTreeFactory( - world_interface=world_interface, - robot_interface=robot_interface, -) - -####################### behavior_tree_stewardship -behavior_tree_stewardship = BehaviorTreeStewardship( - behaviortree_factory=bt_factory, - world_interface=world_interface, - robot_interface=robot_interface, -) - -####################### * tools -# bt_executor_tool = BehaviorTreeExecutorTool(behavior_tree_stewardship) - -bt_sim_tool = BehaviorTreeSimulatorTool(metadata={"bt_stw": behavior_tree_stewardship}) - -# world_state_query_tool = WorldStateQueryTool(behavior_tree_stewardship) - -executor_tools = [bt_sim_tool] - -# * kios data prompt skeleton dir -data_dir = os.environ.get("KIOS_DATA_DIR").format(username=os.getlogin()) -print(data_dir) -prompt_sk_dir = os.path.join(data_dir, "prompt_skeletons") -prompt_dir = os.path.join(data_dir, "prompts") - - -# * the graph state -class PlanExecuteState(TypedDict): - plan: List[str] - behavior_tree: dict - world_state: Annotated[ - List[dict], operator.add - ] # ! to add, you need to make world_state a list of dict - past_steps: Annotated[List[Tuple], operator.add] - user_input: str - objects: dict[str, list[str]] - last_behavior_tree: dict - last_failed_node: dict - runtime_world_state: dict - behavior_tree_execution_summary: str - BTExecutionHasSucceeded: bool - - action_sequence: List[str] - - -########## verifier - -template_file = os.path.join(prompt_dir, "seq_planner/template.txt") -task_file = os.path.join(prompt_dir, "seq_planner/task.txt") -system_file = os.path.join(prompt_dir, "seq_planner/system.txt") -domain_file = os.path.join(prompt_dir, "seq_planner/domain.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {template} - """ -) - -seq_planner_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ], -) - -seq_planner_chain = ( - seq_planner_ppt_ppl - | ChatOpenAI(model="gpt-4-turbo-preview", temperature=0) - | StrOutputParser() -) - - -# * output schema of the recursive_sk_generator -class SolutionBehaviortree(BaseModel): - """the fixed behavior tree""" - - behavior_tree: dict = Field( - description="the behavior tree that adheres the required format." - ) - - -behaviortree_file = os.path.join(prompt_dir, "rec_sk_gen/behaviortree.txt") -template_file = os.path.join(prompt_dir, "rec_sk_gen/template.txt") -task_file = os.path.join(prompt_dir, "rec_sk_gen/task.txt") -system_file = os.path.join(prompt_dir, "rec_sk_gen/system.txt") -object_file = os.path.join(prompt_dir, "rec_sk_gen/object.txt") -domain_file = os.path.join(prompt_dir, "rec_sk_gen/domain.txt") -example_file = os.path.join(prompt_dir, "rec_sk_gen/example.txt") -state_file = os.path.join(prompt_dir, "rec_sk_gen/state.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) -with open(object_file, "r") as f: - object_ppt = PromptTemplate.from_template(f.read()) -with open(behaviortree_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - behaviortree_ppt = ppt_tmp.partial(input=f.read()) - -with open(state_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - state_ppt = ppt_tmp.partial(input=f.read()) - -with open(example_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - example_ppt = ppt_tmp.partial(input=f.read()) -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {state} - - {object} - - {behaviortree} - - {example} - - {template} - """ -) - -re_sk_gen_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ("object", object_ppt), - ("behaviortree", behaviortree_ppt), - ("example", example_ppt), - ("state", state_ppt), - ], -) - -rec_sk_gen_chain = ( - re_sk_gen_ppt_ppl - | ChatOpenAI(model="gpt-4-turbo-preview", temperature=0) - | JsonOutputParser() -) - -# recursive_sk_generator = create_structured_output_runnable( -# SolutionBehaviortree, -# ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), -# re_sk_gen_ppt_ppl, -# mode="openai-json", -# ) - - -##################################################### * planner -# * output schema of the planner -class Plan(BaseModel): - """Plan to follow in future""" - - steps: List[str] = Field( - description="a list of different steps to follow, should be in sorted order" - ) - - -template_file = os.path.join(prompt_dir, "planner/template.txt") -task_file = os.path.join(prompt_dir, "planner/task.txt") -system_file = os.path.join(prompt_dir, "planner/system.txt") -domain_file = os.path.join(prompt_dir, "planner/domain.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {template} - """ -) - -planner_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ], -) - -planner = create_structured_output_runnable( - Plan, ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), planner_ppt_ppl -) - -##################################################### * plan_updaterner - -template_file = os.path.join(prompt_dir, "plan_updater/template.txt") -task_file = os.path.join(prompt_dir, "plan_updater/task.txt") -system_file = os.path.join(prompt_dir, "plan_updater/system.txt") -domain_file = os.path.join(prompt_dir, "plan_updater/domain.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {template} - """ -) - -plan_updater_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ], -) - -plan_updater = create_structured_output_runnable( - Plan, ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), plan_updater_ppt_ppl -) -# ! create openai fn runnable has bug - - -##################################################### * graph node functions -@traceable(name="user_input_node_step") -async def user_input_step(state: PlanExecuteState): - """ - get the input from the user - """ - print(f"-----user_input_step-----") - - user_input = input("Your next instruction: ") - - return { - "user_input": user_input, - } - - -@traceable(name="sequence_generate_step") -async def sequence_generate_step(state: PlanExecuteState): - """ - generate the sequence based on the instruction - """ - print(f"-----sequence_generate_step-----") - - instruction = state["plan"][0] - start_world_state = state["world_state"][-1] - objects = state["objects"] - - action_sequence = await seq_planner_chain.ainvoke( - { - "objects": objects, - "start_world_state": start_world_state, - "user_instruction": instruction, - } - ) - - return { - "action_sequence": action_sequence, - } - - -@traceable(name="behavior_tree_generate_step") -async def behavior_tree_generate_step(state: PlanExecuteState): - """ - generate the behavior tree based on the instruction - """ - print(f"-----behavior_tree_generate_step-----") - - instruction = state["plan"][0] - if state["runtime_world_state"] is None: - runtime_world_state = state["world_state"][-1] - else: - runtime_world_state = state["runtime_world_state"] - objects = state["objects"] - - bt_skeleton = await rec_sk_gen_chain.ainvoke( - { - "objects": objects, - "runtime_world_state": runtime_world_state, - "last_behavior_tree": state["last_behavior_tree"], - "last_failed_node": state["last_failed_node"], - "summary": state["behavior_tree_execution_summary"], - "action_sequence": state["action_sequence"], - } - ) - - render_bt(bt_skeleton) - - pause = input("paused here") - - return { - "last_behavior_tree": bt_skeleton, - "BTExecutionHasSucceeded": False, - } - - -@traceable(name="behavior_tree_execute_step") -async def behavior_tree_execute_step(state: PlanExecuteState): - """ - execute the first step of the plan, append the result to the past steps - """ - print(f"-----behavior_tree_execute_step-----") - - this_step = state["plan"][0] - behavior_tree_skeleton = state["last_behavior_tree"] - latest_world_state = state["world_state"][-1] - - # * first sim run - tree_result, skeleton_json = behavior_tree_stewardship.sk_sim_run( - world_state=latest_world_state, skeleton_json=behavior_tree_skeleton - ) - - pprint(tree_result.to_json()) - pause = input("paused here") - - # * check result - if tree_result.result == "success": - - # * second fake run - tree_result, skeleton_json = behavior_tree_stewardship.sk_fake_run( - world_state=latest_world_state, skeleton_json=behavior_tree_skeleton - ) - - pprint(tree_result.to_json()) - pause = input("paused here") - - if tree_result.result == "success": - return { - "BTExecutionHasSucceeded": True, - "past_steps": ( - this_step, - tree_result.result, - ), # * only one because new plan will be generated and old steps are all removed - "world_state": [tree_result.world_state], - # * clear all - "last_behavior_tree": None, - "last_failed_node": None, - "runtime_world_state": None, - "behavior_tree_execution_summary": None, - } - else: - return { - "world_state": [tree_result.world_state], # * real world state - "last_behavior_tree": skeleton_json, - "last_failed_node": { - "summary": tree_result.final_node.get("summary"), - "name": tree_result.final_node.get("name"), - "identifier": tree_result.final_node.get("identifier"), - }, - "runtime_world_state": tree_result.world_state, - "behavior_tree_execution_summary": tree_result.summary, - } - - else: - return { - # ! do not change world state - "last_behavior_tree": skeleton_json, - "last_failed_node": { - "summary": tree_result.final_node.get("summary"), - "name": tree_result.final_node.get("name"), - "identifier": tree_result.final_node.get("identifier"), - }, - "runtime_world_state": tree_result.world_state, - "behavior_tree_execution_summary": tree_result.summary, - } - - -@traceable(name="planner_step") -async def planner_step(state: PlanExecuteState): - """ - plan the steps based on user input and world state - """ - print(f"-----plan_step-----") - - plan = await planner.ainvoke( - { - "user_input": state["user_input"], - "world_state": state["world_state"], - } - ) - return {"plan": plan.steps} - - -@traceable(name="plan_updater_step") -async def plan_updater_step(state: PlanExecuteState): - """ - if return a response, then success, response the use, end. - otherwise, return the updated newplan (normally the same as the old plan with the first step popped out. - """ - print(f"-----plan_updater_step-----") - - # ! BOOOM! BUG! - output = await plan_updater.ainvoke( - { - "user_input": state["user_input"], - "plan": state["plan"], - "world_state": state["world_state"], - "past_steps": state["past_steps"], - } - ) - # if isinstance( - # output, UpdaterResponse - # ): # * determine if it is time to response and end - # return { - # "response": "Your last instruction has been finished." - # } # * Don't need to update. - # else: - return {"plan": output.steps} # * update the plan - - -##################################################### * construct the graph - -workflow = StateGraph(PlanExecuteState) - -workflow.add_node("planner", planner_step) -workflow.add_node("sequence_generator", sequence_generate_step) -workflow.add_node("behavior_tree_generator", behavior_tree_generate_step) -workflow.add_node("behavior_tree_executor", behavior_tree_execute_step) -workflow.add_node("plan_updater", plan_updater_step) -workflow.add_node("user_input_node", user_input_step) -workflow.set_entry_point("user_input_node") -workflow.add_edge("planner", "sequence_generator") -workflow.add_edge("sequence_generator", "behavior_tree_generator") -workflow.add_edge("behavior_tree_generator", "behavior_tree_executor") - - -def executor_should_end(state: PlanExecuteState): - """ - end router - """ - print(f"-----executor_should_end-----") - - if state["BTExecutionHasSucceeded"] == True: - state["BTExecutionHasSucceeded"] = False - return True - else: - return False - - -workflow.add_conditional_edges( - "behavior_tree_executor", - executor_should_end, - { - True: "plan_updater", - False: "behavior_tree_generator", - }, -) - - -def plan_updater_should_end(state: PlanExecuteState): - """ - end router - """ - print(f"-----plan_updater_should_end-----") - - if state["plan"] == [] or len(state["plan"]) == 0: - return True - else: - return False - - -workflow.add_conditional_edges( - "plan_updater", - plan_updater_should_end, - { - True: "user_input_node", - False: "behavior_tree_generator", - }, -) - - -def user_input_should_end(state: PlanExecuteState): - """ - if the user input is empty, then end - """ - print(f"-----user_input_should_end-----") - - if not state["user_input"] or state["user_input"] == "": - return True - else: - return False - - -workflow.add_conditional_edges( - "user_input_node", - user_input_should_end, - { - True: END, - False: "planner", - }, -) - -app = workflow.compile() - -config = {"recursion_limit": 500} - -inputs = { - "world_state": [world_state_json_object], - # "objects": objects, # ! warning! -} - -# * unit tree generator ppl -system_file = os.path.join(prompt_dir, "new/ut_gen/system.txt") -task_file = os.path.join(prompt_dir, "new/ut_gen/task.txt") -domain_file = os.path.join(prompt_dir, "new/ut_gen/new_domain_nl.txt") -behaviortree_file = os.path.join(prompt_dir, "new/ut_gen/new_behaviortree.txt") -template_file = os.path.join(prompt_dir, "new/ut_gen/template.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) -with open(behaviortree_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - behaviortree_ppt = ppt_tmp.partial(input=f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {behaviortree} - - {template} - - {format_instructions} - """ -) - -parser = JsonOutputParser() - -format_instructions = PromptTemplate.from_template("""{input}""").partial( - input=parser.get_format_instructions() -) - -ut_gen_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ("behaviortree", behaviortree_ppt), - ("format_instructions", format_instructions), - ], -) - -ut_gen_chain = ( - ut_gen_ppt_ppl - # | ChatOpenAI(model="gpt-3.5-turbo-0125", temperature=0) - # | ChatOpenAI(model="ft:gpt-3.5-turbo-0125:kifabrik-mirmi::8y1cXwVw", temperature=0) - | ChatOpenAI( - model="ft:gpt-3.5-turbo-0125:kifabrik-mirmi:kios-ut-gen-v2:8z2KbPsr", - temperature=0, - ) - # | ChatOpenAI(model="gpt-4", temperature=0) - | JsonOutputParser() -) - -# ! sequence planner estimation ppl -system_file = os.path.join(prompt_dir, "seq_planner_est/system.txt") -task_file = os.path.join(prompt_dir, "seq_planner_est/task.txt") -domain_file = os.path.join(prompt_dir, "seq_planner_est/domain.txt") -state_file = os.path.join(prompt_dir, "seq_planner_est/state.txt") -output_format_file = os.path.join(prompt_dir, "seq_planner_est/output_format.txt") -template_file = os.path.join(prompt_dir, "seq_planner_est/template.txt") -example_file = os.path.join(prompt_dir, "seq_planner_est/example.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) -with open(output_format_file, "r") as f: - output_format_ppt = PromptTemplate.from_template(f.read()) -with open(example_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - example_ppt = ppt_tmp.partial(input=f.read()) -with open(state_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - state_ppt = ppt_tmp.partial(input=f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {state} - - {output_format} - - {example} - - {template} - - {format_instructions} - """ -) - -parser = JsonOutputParser() - -format_instructions = PromptTemplate.from_template("""{input}""").partial( - input=parser.get_format_instructions() -) - -seq_planner_est_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ("state", state_ppt), - ("output_format", output_format_ppt), - ("format_instructions", format_instructions), - ("example", example_ppt), - ], -) - -seq_planner_est_chain = ( - seq_planner_est_ppt_ppl - # | ChatOpenAI(model="gpt-4", temperature=0) - | ChatOpenAI(model="gpt-3.5-turbo-0125", temperature=0) - | JsonOutputParser() -) - - -async def core_run(): - async for event in app.astream( - inputs, - config=config, - ): - for k, v in event.items(): - if k != "__end__": - print(v) - - -def seq_planner_est_test(): - return seq_planner_est_chain.invoke( - { - "start_world_state": world_state_json_object, - # "target": "is_inserted_to(shaft1, gearbase_hole1)", - "target": "hold(left_hand, outward_claw)", - } - ) - - -# * sequence action planner -system_file = os.path.join(prompt_dir, "new/seq_plan/system.txt") -task_file = os.path.join(prompt_dir, "new/seq_plan/task.txt") -domain_file = os.path.join(prompt_dir, "new/seq_plan/new_domain_nl.txt") -state_file = os.path.join(prompt_dir, "new/seq_plan/state.txt") -output_format_file = os.path.join(prompt_dir, "new/seq_plan/output_format.txt") -template_file = os.path.join(prompt_dir, "new/seq_plan/template.txt") -example_file = os.path.join(prompt_dir, "new/seq_plan/new_example.txt") -chain_file = os.path.join(prompt_dir, "new/seq_plan/chain.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) -with open(output_format_file, "r") as f: - output_format_ppt = PromptTemplate.from_template(f.read()) -with open(example_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - example_ppt = ppt_tmp.partial(input=f.read()) -with open(state_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - state_ppt = ppt_tmp.partial(input=f.read()) -# # * this is for gpt 3.5 -# with open(chain_file, "r") as f: -# chain_ppt = PromptTemplate.from_template(f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {state} - - {output_format} - - {example} - - {template} - - {format_instructions} - """ -) - -parser = JsonOutputParser() - -format_instructions = PromptTemplate.from_template("""{input}""").partial( - input=parser.get_format_instructions() -) - -seq_ac_pl_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ("state", state_ppt), - ("output_format", output_format_ppt), - ("format_instructions", format_instructions), - ("example", example_ppt), - # ("chain", chain_ppt), - ], -) - -seq_ac_pl_chain = ( - seq_ac_pl_ppt_ppl - | ChatOpenAI(model="gpt-4", temperature=0) - # | ChatOpenAI(model="gpt-3.5-turbo-0125", temperature=0) - | JsonOutputParser() -) - -# * state estimater -system_file = os.path.join(prompt_dir, "new/state_est/system.txt") -task_file = os.path.join(prompt_dir, "new/state_est/task.txt") -domain_file = os.path.join(prompt_dir, "new/state_est/new_domain_nl.txt") -state_file = os.path.join(prompt_dir, "new/state_est/state.txt") -output_format_file = os.path.join(prompt_dir, "new/state_est/output_format.txt") -template_file = os.path.join(prompt_dir, "new/state_est/template.txt") -example_file = os.path.join(prompt_dir, "new/state_est/new_example.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) -with open(output_format_file, "r") as f: - output_format_ppt = PromptTemplate.from_template(f.read()) -with open(example_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - example_ppt = ppt_tmp.partial(input=f.read()) -with open(state_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - state_ppt = ppt_tmp.partial(input=f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {state} - - {output_format} - - {example} - - {template} - - {format_instructions} - """ -) - -parser = JsonOutputParser() - -format_instructions = PromptTemplate.from_template("""{input}""").partial( - input=parser.get_format_instructions() -) - -state_est_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ("state", state_ppt), - ("output_format", output_format_ppt), - ("format_instructions", format_instructions), - ("example", example_ppt), - ], -) - - -state_est_ppt_ppl_chain = ( - state_est_ppt_ppl - # | ChatOpenAI(model="gpt-3.5-turbo", temperature=0) - | ChatOpenAI(model="gpt-4", temperature=0) - | JsonOutputParser() -) - - -async def core_run(): - raise NotImplementedError - async for event in app.astream( - inputs, - config=config, - ): - for k, v in event.items(): - if k != "__end__": - print(v) - - -def ut_gen_test(): - ut = ut_gen_chain.invoke( - { - # "action": "insert(left_hand, defaultgripper, gear1, shaft1)", - # "action": "insert(left_hand, parallel_box1, shaft1, gearbase_hole1)", - # "action": "change_tool(left_hand, parallel_box1, defaultgripper)", - "action": "insert(left_hand, clampgripper, gear2, shaft2)", - } - ) - - render_bt(ut) - - -def seq_planner_est_test(): - return seq_planner_est_chain.invoke( - { - "start_world_state": world_state_json_object, - "target": "is_inserted_to(shaft1, gearbase_hole1)", - # "target": "hold(left_hand, outward_claw)", - } - ) - - -def seq_action_plan_test(): - return seq_ac_pl_chain.invoke( - { - "start_world_state": world_state_json_object, - "target": "is_inserted_to(shaft1, gearbase_hole1)", - # "target": "hold(left_hand, outward_claw)", - } - ) - - -def state_est_test(): - raise NotImplementedError - return state_est_ppt_ppl_chain.invoke( - { - "start_world_state": world_state_json_object, - "action_plan": [ - "pick_up(left_hand, parallel_box1, shaft1)", - "insert(left_hand, parallel_box1, shaft1, gearbase_hole1)", - ], - } - ) - - -##################################### * speed imp -@traceable(name="make_plan") -def make_plan(state: dict, goal: str) -> list[str]: - print(f"----------start to make plan for the goal {goal}") - response = seq_ac_pl_chain.invoke( - { - "start_world_state": state, - "target": goal, - } - ) - print(f"finished making plan for the goal {goal}.") - pprint(f'LLM thought flow: {response["explanation"]}') - return response["task_plan"] - - -@traceable(name="estimate_state") -def estimate_state(start_world_state: dict, action_plan: list[str]) -> dict: - print("----------start to estimate the state after the action plan:") - pprint(action_plan) - response = state_est_ppt_ppl_chain.invoke( - { - "start_world_state": start_world_state, - "action_plan": action_plan, - } - ) - print(f"finished estimating the state after the action plan {action_plan}.") - return response["estimated_world_state"] - - -@traceable(name="generate_unit_subtree") -def generate_unit_subtree(action: str) -> dict: - print("----------start to generate the unit subtree for the action") - pprint(action) - response = ut_gen_chain.invoke( - { - "action": action, - } - ) - print(f"finished generating the unit subtree for the action {action}.") - return response - - -def get_node_list_from_tree(unit_subtree: dict) -> list[dict]: - children = unit_subtree["children"][1][ - "children" - ] # * the second child is a sequence - return children - - -def extract_goal(node: dict) -> str: - name = node["name"] - - -def match_type(node: dict) -> tuple[str, str]: - node_name = node["name"] - match = re.search( - r"(selector|sequence|action|precondition|condition|target):\s*(.+)", node_name - ) - if match: - node_type = match.group(1) - node_body = match.group(2) - return node_type, node_body - else: - raise ValueError(f"the node name {node_name} does not match any type.") - - -def expand_nodes( - node_list: list[dict], - start_state: dict, - overall_tree: list[dict] = None, -) -> dict: - """ - in order to monitor the tree generation, the overall tree and the node list should be the same variable when being passed in. - """ - pprint("----------check the entire tree:") - if overall_tree is not None: - render_bt(overall_tree[0]) - pprint("----------start to expand the node list:") - pprint(node_list) - pause = input("paused here! check the tree.") - - assert len(node_list) > 0 - state = start_state - - for i in range(len(node_list)): - type_name, body = match_type(node_list[i]) - # if match_type(node_list[i]) == "action": - if type_name == "action": - print(f"the node {node_list[i]['name']} is an action node. skip it.") - pause = input("paused here! check!") - # elif match_type(node_list[i]) == "precondition" or "target": - elif type_name in ["precondition", "target"]: - # goal = node_list[i]["name"] - goal = body - plan = make_plan(state, goal) - if len(plan) == 0: - logging.warning(f"No action should be performed for the goal {goal}.") - logging.warning(f'the node {node_list[i]["name"]} has been skipped.') - pause = input("paused here! check!") - else: - logging.info(f"Actions have been planed for the goal {goal}.") - pprint(f"the plan for the goal {goal} is {plan}") - pause = input("paused here! check!") - last_action = plan[-1] - unit_subtree = generate_unit_subtree(last_action) - # insert the subtree into the node_list - node_list[i] = unit_subtree - new_node_list = get_node_list_from_tree(unit_subtree) - expand_nodes( - node_list=new_node_list, - start_state=state, - overall_tree=overall_tree, - ) - state = estimate_state(state, plan) - - return node_list[0] - - -def embed_ut_nl(unit_subtree: dict) -> str: - """ - embed the unit subtree into the overall tree - """ - selector_children = unit_subtree["children"] - target = "" - # * target - for node in selector_children: - if match_type(node)[0] == "target": - target += node["summary"] - - sequence_children = selector_children[1]["children"] - preconditions = [] - action = "" - for node in sequence_children: - if match_type(node)[0] == "precondition": - preconditions.append(node["summary"]) - if match_type(node)[0] == "action": - action = node["summary"] - - embedding = f"if {' and '.join(preconditions)} then {action}, {target}" - - return embedding - - -def test_embedding_nl(): - unit_tree = ut_gen_test() - pprint(unit_tree) - nl = embed_ut_nl(unit_tree) - print(nl) - - -def test_expand_nodes(): - start_state = world_state_json_object - node_list = [ - # { - # "summary": "assemble the chairback to the seat of the chair", - # "name": "target: assemble the chairback to the seat of the chair", - # }, - { - "summary": "assemble the chairbut1 to the seat of the chair", - "name": "target: assemble the chairnut1 to the seat of the chair", - }, - ] - expand_nodes(node_list, start_state, node_list) - pprint(node_list) - - -if __name__ == "__main__": - pass - - # asyncio.run(core_run()) - # pprint(ut_gen_test()) - # pprint(seq_action_plan_test()) - # pprint(state_est_test()) - - test_expand_nodes() - - # test_embedding_nl() diff --git a/experiments/chair/world_state.json b/experiments/chair/world_state.json deleted file mode 100644 index ce460322..00000000 --- a/experiments/chair/world_state.json +++ /dev/null @@ -1,89 +0,0 @@ -{ - "objects": [ - { - "name": "clampgripper", - "properties": ["is_empty"] - }, - { - "name": "parallelgripper", - "properties": ["is_empty"] - }, - { - "name": "inwardgripper", - "properties": ["is_empty"] - }, - { - "name": "outwardgripper", - "properties": ["is_empty"] - }, - { - "name": "defaultgripper", - "properties": ["is_empty"] - }, - { - "name": "chairnut1", - "properties": [] - }, - { - "name": "chairnut2", - "properties": [] - }, - { - "name": "chairseat", - "properties": [] - }, - { - "name": "chairseatbolt1", - "properties": [] - }, - { - "name": "chairseatbolt2", - "properties": [] - }, - { - "name": "chairseatconnector", - "properties": [] - }, - { - "name": "chairseatthread1", - "properties": [] - }, - { - "name": "chairseatthread2", - "properties": [] - }, - { - "name": "chairback", - "properties": [] - }, - { - "name": "chairleg1", - "properties": [] - }, - { - "name": "chairleg2", - "properties": [] - }, - { - "name": "left_hand", - "properties": [] - } - ], - "constraints": [ - {"source": "clampgripper", "name": "can_manipulate", "target": "chairback"}, - {"source": "inwardgripper", "name": "can_manipulate", "target": "chairnut1"}, - {"source": "inwardgripper", "name": "can_manipulate", "target": "chairnut2"}, - {"source": "defaultgripper", "name": "can_manipulate", "target": "chairleg1"}, - {"source": "defaultgripper", "name": "can_manipulate", "target": "chairleg2"}, - {"source": "defaultgripper", "name": "can_manipulate", "target": "chairseat"}, - {"source": "chairnut1", "name": "can_screw_to", "target": "chairseatbolt1"}, - {"source": "chairnut2", "name": "can_screw_to", "target": "chairseatbolt2"}, - {"source": "chairback", "name": "can_insert_to", "target": "chairseatconnector"}, - {"source": "chairleg1", "name": "can_screw_to", "target": "chairseatthread1"}, - {"source": "chairleg2", "name": "can_screw_to", "target": "chairseatthread2"} - ], - "relations": [ - {"source": "left_hand", "name": "hold", "target": "defaultgripper"} - ] -} - diff --git a/experiments/cook/baseline.py b/experiments/cook/baseline.py deleted file mode 100644 index b5c8c6cc..00000000 --- a/experiments/cook/baseline.py +++ /dev/null @@ -1,137 +0,0 @@ -import json -import os -import re -from pprint import pprint -from typing import List, Tuple, Annotated, TypedDict - -from kios_bt.bt_stewardship import BehaviorTreeStewardship -from kios_scene.scene_factory import SceneFactory -from kios_bt.bt_factory import BehaviorTreeFactory -from kios_robot.robot_interface import RobotInterface -from kios_world.world_interface import WorldInterface - -from dotenv import load_dotenv - -load_dotenv() - -from kios_utils.pybt_test import generate_bt_stewardship, render_dot_tree - - -def render_bt(bt_json: dict): - test_class = BehaviorTreeFactory() - bt = test_class.from_json_to_simple_bt(bt_json) - # bt = test_class.from_json_to_tree_root(bt_json) - bt_stewardship = generate_bt_stewardship(bt) - # bt_stewardship.setup(timeout=15) - render_dot_tree(bt_stewardship) - - -####################### dirs -current_dir = os.path.dirname(os.path.abspath(__file__)) -scene_path = os.path.join(current_dir, "scene.json") -# bt_json_file_path = os.path.join(current_dir, "behavior_tree.json") -world_state_path = os.path.join(current_dir, "world_state.json") -problem_path = os.path.join(current_dir, "gearset.problem") -domain_knowledge_path = os.path.join(current_dir, "domain_knowledge.txt") - -####################### problem -with open(problem_path, "r") as file: - problem = file.read() - -####################### scene -with open(scene_path, "r") as file: - scene_json_object = json.load(file) - -scene = SceneFactory().create_scene_from_json(scene_json_object) - -####################### world -world_interface = WorldInterface() -with open(world_state_path, "r") as file: - world_state_json = json.load(file) - world_interface.load_world_from_json(world_state_json) - -pprint(world_state_json) - - -####################### robot -robot_interface = RobotInterface( - robot_address="127.0.0.1", - robot_port=12000, -) -robot_interface.setup_scene(scene) - -####################### bt_factory -bt_factory = BehaviorTreeFactory( - world_interface=world_interface, - robot_interface=robot_interface, -) - -####################### behavior_tree_stewardship -behavior_tree_stewardship = BehaviorTreeStewardship( - behaviortree_factory=bt_factory, - world_interface=world_interface, - robot_interface=robot_interface, -) - - -def baseline_run(tree_root: dict, world_state: dict): - - sk_json = tree_root - initial_world_state = world_state - - from kios_plan.dynamic_planning import gearset_ut_dict - - # * first sim run - solultion = behavior_tree_stewardship.sk_baseline( - initial_world_state, - sk_json, - gearset_ut_dict, - ) - record = { - "target": sk_json["name"], - "initial_world_state": initial_world_state, - "result": solultion, - } - file_dir = os.path.join(current_dir, "baseline_result.jsonl") - with open(file_dir, "a") as file: - file.write(json.dumps(record) + "\n") - - -def baseline_plan(): - sk = { - # "summary": "the target is to insert the shaft1 into the gearbase_hole1", - # "name": "target: is_inserted_to(shaft1, gearbase_hole1)", - # "summary": "the target is to insert the shaft3 into the gearbase_hole3", - # "name": "target: is_inserted_to(shaft3, gearbase_hole3)", - # "summary": "the target is to insert the gear2 into the shaft2", - # "name": "target: is_inserted_to(gear2, shaft2)", - # "summary": "the target is to insert gear3 into shaft3", - # "name": "target: is_inserted_to(gear3, shaft3)", - "summary": "the target is to insert gear1 to shaft1", - "name": "target: is_inserted_to(gear1, shaft1)", - } - baseline_run(sk, world_state_json) - - -def test_result(problem_id: int): - """ - test the plan from the baseline_result.jsonl - """ - file_dir = os.path.join(current_dir, "baseline_result.jsonl") - with open(file_dir, "r") as file: - results = file.read() - data = [json.loads(line) for line in results.splitlines()] - problem = data[problem_id] - initial_world_state = problem["initial_world_state"] - sk = problem["result"] - result, node = behavior_tree_stewardship.sk_sim_run( - world_state=initial_world_state, skeleton_json=sk - ) - pprint(result.result) - pprint(result.summary) - - -if __name__ == "__main__": - pass - # baseline_plan() - test_result(15) diff --git a/experiments/cook/baseline_result.jsonl b/experiments/cook/baseline_result.jsonl deleted file mode 100644 index c4d9d3af..00000000 --- a/experiments/cook/baseline_result.jsonl +++ /dev/null @@ -1,17 +0,0 @@ -{"target": "target: is_inserted_to(shaft1, gearbase_hole1)", "initial_world_state": {"objects": [{"name": "clampgripper", "properties": ["is_empty"]}, {"name": "parallelgripper", "properties": ["is_empty"]}, {"name": "inwardgripper", "properties": ["is_empty"]}, {"name": "outwardgripper", "properties": []}, {"name": "defaultgripper", "properties": ["is_empty"]}, {"name": "gear1", "properties": []}, {"name": "gear2", "properties": []}, {"name": "gear3", "properties": []}, {"name": "shaft1", "properties": []}, {"name": "shaft2", "properties": []}, {"name": "shaft3", "properties": []}, {"name": "gearbase", "properties": []}, {"name": "gearbase_hole1", "properties": []}, {"name": "gearbase_hole3", "properties": []}, {"name": "left_hand", "properties": []}], "constraints": [{"source": "gear3", "name": "can_insert_to", "target": "shaft3"}, {"source": "shaft1", "name": "can_insert_to", "target": "gearbase_hole1"}, {"source": "parallelgripper", "name": "can_manipulate", "target": "gear1"}, {"source": "gear2", "name": "can_insert_to", "target": "shaft2"}, {"source": "outwardgripper", "name": "can_manipulate", "target": "gear3"}, {"source": "gear1", "name": "can_insert_to", "target": "shaft1"}, {"source": "defaultgripper", "name": "can_manipulate", "target": "shaft3"}, {"source": "outwardgripper", "name": "can_manipulate", "target": "gear2"}, {"source": "shaft3", "name": "can_insert_to", "target": "gearbase_hole3"}, {"source": "clampgripper", "name": "can_manipulate", "target": "shaft1"}], "relations": [{"source": "outwardgripper", "name": "hold", "target": "gear2"}, {"source": "left_hand", "name": "hold", "target": "outwardgripper"}]}, "result": {"summary": "selector to insert the shaft1 into the gearbase_hole1 with the clampgripper in the left_hand", "name": "selector: insert(left_hand, clampgripper, shaft1, gearbase_hole1)", "children": [{"summary": "the target is to make the shaft1 be inserted into the gearbase_hole1", "name": "target: is_inserted_to(shaft1, gearbase_hole1)"}, {"summary": "sequence to insert the shaft1 into the gearbase_hole1 with the clampgripper in the left_hand", "name": "sequence: insert(left_hand, clampgripper, shaft1, gearbase_hole1)", "children": [{"summary": "selector to change tool in left_hand from outwardgripper to clampgripper", "name": "selector: change_tool(left_hand, outwardgripper, clampgripper)", "children": [{"summary": "the target is to make left_hand hold clampgripper", "name": "target: hold(left_hand, clampgripper)"}, {"summary": "sequence to change tool in left_hand from outwardgripper to clampgripper", "name": "sequence: change_tool(left_hand, outwardgripper, clampgripper)", "children": [{"summary": "a precondition is left_hand is holding outwardgripper", "name": "precondition: hold(left_hand, outwardgripper)"}, {"summary": "selector to put down the gear2 held by the outwardgripper in the left_hand", "name": "selector: put_down(left_hand, outwardgripper, gear2)", "children": [{"summary": "the target is to make the outwardgripper empty", "name": "target: is_empty(outwardgripper)"}, {"summary": "sequence to put down the gear2 held by the outwardgripper in the left_hand", "name": "sequence: put_down(left_hand, outwardgripper, gear2)", "children": [{"summary": "a precondition is the left_hand is holding outwardgripper", "name": "precondition: hold(left_hand, outwardgripper)"}, {"summary": "a precondition is the outwardgripper is holding gear2", "name": "precondition: hold(outwardgripper, gear2)"}, {"summary": "the action to put down the gear2 held by the outwardgripper in the left_hand", "name": "action: put_down(left_hand, outwardgripper, gear2)"}]}]}, {"summary": "the action to change tool in left_hand from outwardgripper to clampgripper", "name": "action: change_tool(left_hand, outwardgripper, clampgripper)"}]}]}, {"summary": "selector to pick up the shaft1 with the clampgripper in the left_hand", "name": "selector: pick_up(left_hand, clampgripper, shaft1)", "children": [{"summary": "the target is to make the clampgripper hold the shaft1", "name": "target: hold(clampgripper, shaft1)"}, {"summary": "sequence to pick up the shaft1 with the clampgripper in the left_hand", "name": "sequence: pick_up(left_hand, clampgripper, shaft1)", "children": [{"summary": "a precondition is the clampgripper is empty", "name": "precondition: is_empty(clampgripper)"}, {"summary": "a precondition is the left_hand is holding clampgripper", "name": "precondition: hold(left_hand, clampgripper)"}, {"summary": "the action to pick up the shaft1 with the clampgripper in the left_hand", "name": "action: pick_up(left_hand, clampgripper, shaft1)"}]}]}, {"summary": "the action to insert the shaft1 into the gearbase_hole1 with the clampgripper in the left_hand", "name": "action: insert(left_hand, clampgripper, shaft1, gearbase_hole1)"}]}]}} -{"target": "target: is_inserted_to(shaft1, gearbase_hole1)", "initial_world_state": {"objects": [{"name": "clampgripper", "properties": ["is_empty"]}, {"name": "parallelgripper", "properties": ["is_empty"]}, {"name": "inwardgripper", "properties": ["is_empty"]}, {"name": "outwardgripper", "properties": ["is_empty"]}, {"name": "defaultgripper", "properties": ["is_empty"]}, {"name": "gear1", "properties": []}, {"name": "gear2", "properties": []}, {"name": "gear3", "properties": []}, {"name": "shaft1", "properties": []}, {"name": "shaft2", "properties": []}, {"name": "shaft3", "properties": []}, {"name": "gearbase", "properties": []}, {"name": "gearbase_hole1", "properties": []}, {"name": "gearbase_hole3", "properties": []}, {"name": "left_hand", "properties": []}], "constraints": [{"source": "defaultgripper", "name": "can_manipulate", "target": "shaft3"}, {"source": "gear2", "name": "can_insert_to", "target": "shaft2"}, {"source": "shaft1", "name": "can_insert_to", "target": "gearbase_hole1"}, {"source": "clampgripper", "name": "can_manipulate", "target": "shaft1"}, {"source": "parallelgripper", "name": "can_manipulate", "target": "gear1"}, {"source": "outwardgripper", "name": "can_manipulate", "target": "gear3"}, {"source": "gear1", "name": "can_insert_to", "target": "shaft1"}, {"source": "shaft3", "name": "can_insert_to", "target": "gearbase_hole3"}, {"source": "outwardgripper", "name": "can_manipulate", "target": "gear2"}, {"source": "gear3", "name": "can_insert_to", "target": "shaft3"}], "relations": [{"source": "left_hand", "name": "hold", "target": "outwardgripper"}]}, "result": {"summary": "selector to insert the shaft1 into the gearbase_hole1 with the clampgripper in the left_hand", "name": "selector: insert(left_hand, clampgripper, shaft1, gearbase_hole1)", "children": [{"summary": "the target is to make the shaft1 be inserted into the gearbase_hole1", "name": "target: is_inserted_to(shaft1, gearbase_hole1)"}, {"summary": "sequence to insert the shaft1 into the gearbase_hole1 with the clampgripper in the left_hand", "name": "sequence: insert(left_hand, clampgripper, shaft1, gearbase_hole1)", "children": [{"summary": "selector to change tool in left_hand from outwardgripper to clampgripper", "name": "selector: change_tool(left_hand, outwardgripper, clampgripper)", "children": [{"summary": "the target is to make left_hand hold clampgripper", "name": "target: hold(left_hand, clampgripper)"}, {"summary": "sequence to change tool in left_hand from outwardgripper to clampgripper", "name": "sequence: change_tool(left_hand, outwardgripper, clampgripper)", "children": [{"summary": "a precondition is left_hand is holding outwardgripper", "name": "precondition: hold(left_hand, outwardgripper)"}, {"summary": "a precondition is outwardgripper is empty", "name": "precondition: is_empty(outwardgripper)"}, {"summary": "the action to change tool in left_hand from outwardgripper to clampgripper", "name": "action: change_tool(left_hand, outwardgripper, clampgripper)"}]}]}, {"summary": "selector to pick up the shaft1 with the clampgripper in the left_hand", "name": "selector: pick_up(left_hand, clampgripper, shaft1)", "children": [{"summary": "the target is to make the clampgripper hold the shaft1", "name": "target: hold(clampgripper, shaft1)"}, {"summary": "sequence to pick up the shaft1 with the clampgripper in the left_hand", "name": "sequence: pick_up(left_hand, clampgripper, shaft1)", "children": [{"summary": "a precondition is the clampgripper is empty", "name": "precondition: is_empty(clampgripper)"}, {"summary": "a precondition is the left_hand is holding clampgripper", "name": "precondition: hold(left_hand, clampgripper)"}, {"summary": "the action to pick up the shaft1 with the clampgripper in the left_hand", "name": "action: pick_up(left_hand, clampgripper, shaft1)"}]}]}, {"summary": "the action to insert the shaft1 into the gearbase_hole1 with the clampgripper in the left_hand", "name": "action: insert(left_hand, clampgripper, shaft1, gearbase_hole1)"}]}]}} -{"target": "target: is_inserted_to(gear2, shaft2)", "initial_world_state": {"objects": [{"name": "clampgripper", "properties": ["is_empty"]}, {"name": "parallelgripper", "properties": ["is_empty"]}, {"name": "inwardgripper", "properties": ["is_empty"]}, {"name": "outwardgripper", "properties": ["is_empty"]}, {"name": "defaultgripper", "properties": ["is_empty"]}, {"name": "gear1", "properties": []}, {"name": "gear2", "properties": []}, {"name": "gear3", "properties": []}, {"name": "shaft1", "properties": []}, {"name": "shaft2", "properties": []}, {"name": "shaft3", "properties": []}, {"name": "gearbase", "properties": []}, {"name": "gearbase_hole1", "properties": []}, {"name": "gearbase_hole3", "properties": []}, {"name": "left_hand", "properties": []}], "constraints": [{"source": "shaft1", "name": "can_insert_to", "target": "gearbase_hole1"}, {"source": "clampgripper", "name": "can_manipulate", "target": "shaft1"}, {"source": "outwardgripper", "name": "can_manipulate", "target": "gear3"}, {"source": "gear3", "name": "can_insert_to", "target": "shaft3"}, {"source": "defaultgripper", "name": "can_manipulate", "target": "shaft3"}, {"source": "gear2", "name": "can_insert_to", "target": "shaft2"}, {"source": "shaft3", "name": "can_insert_to", "target": "gearbase_hole3"}, {"source": "gear1", "name": "can_insert_to", "target": "shaft1"}, {"source": "outwardgripper", "name": "can_manipulate", "target": "gear2"}, {"source": "parallelgripper", "name": "can_manipulate", "target": "gear1"}], "relations": [{"source": "left_hand", "name": "hold", "target": "outwardgripper"}]}, "result": {"summary": "selector to insert the gear2 into the shaft2 with the outwardgripper in the left_hand", "name": "selector: insert(left_hand, outwardgripper, gear2, shaft2)", "children": [{"summary": "the target is to make the gear2 be inserted into the shaft2", "name": "target: is_inserted_to(gear2, shaft2)"}, {"summary": "sequence to insert the gear2 into the shaft2 with the outwardgripper in the left_hand", "name": "sequence: insert(left_hand, outwardgripper, gear2, shaft2)", "children": [{"summary": "a precondition is the left_hand is holding outwardgripper", "name": "precondition: hold(left_hand, outwardgripper)"}, {"summary": "selector to pick up the gear2 with the outwardgripper in the left_hand", "name": "selector: pick_up(left_hand, outwardgripper, gear2)", "children": [{"summary": "the target is to make the outwardgripper hold the gear2", "name": "target: hold(outwardgripper, gear2)"}, {"summary": "sequence to pick up the gear2 with the outwardgripper in the left_hand", "name": "sequence: pick_up(left_hand, outwardgripper, gear2)", "children": [{"summary": "a precondition is the outwardgripper is empty", "name": "precondition: is_empty(outwardgripper)"}, {"summary": "a precondition is the left_hand is holding outwardgripper", "name": "precondition: hold(left_hand, outwardgripper)"}, {"summary": "the action to pick up the gear2 with the outwardgripper in the left_hand", "name": "action: pick_up(left_hand, outwardgripper, gear2)"}]}]}, {"summary": "the action to insert the gear2 into the shaft2 with the outwardgripper in the left_hand", "name": "action: insert(left_hand, outwardgripper, gear2, shaft2)"}]}]}} -{"target": "target: is_inserted_to(gear2, shaft2)", "initial_world_state": {"objects": [{"name": "clampgripper", "properties": []}, {"name": "parallelgripper", "properties": ["is_empty"]}, {"name": "inwardgripper", "properties": ["is_empty"]}, {"name": "outwardgripper", "properties": ["is_empty"]}, {"name": "defaultgripper", "properties": ["is_empty"]}, {"name": "gear1", "properties": []}, {"name": "gear2", "properties": []}, {"name": "gear3", "properties": []}, {"name": "shaft1", "properties": []}, {"name": "shaft2", "properties": []}, {"name": "shaft3", "properties": []}, {"name": "gearbase", "properties": []}, {"name": "gearbase_hole1", "properties": []}, {"name": "gearbase_hole3", "properties": []}, {"name": "left_hand", "properties": []}], "constraints": [{"source": "gear1", "name": "can_insert_to", "target": "shaft1"}, {"source": "defaultgripper", "name": "can_manipulate", "target": "shaft3"}, {"source": "outwardgripper", "name": "can_manipulate", "target": "gear2"}, {"source": "shaft1", "name": "can_insert_to", "target": "gearbase_hole1"}, {"source": "gear2", "name": "can_insert_to", "target": "shaft2"}, {"source": "gear3", "name": "can_insert_to", "target": "shaft3"}, {"source": "shaft3", "name": "can_insert_to", "target": "gearbase_hole3"}, {"source": "outwardgripper", "name": "can_manipulate", "target": "gear3"}, {"source": "parallelgripper", "name": "can_manipulate", "target": "gear1"}, {"source": "clampgripper", "name": "can_manipulate", "target": "shaft1"}], "relations": [{"source": "clampgripper", "name": "hold", "target": "shaft1"}, {"source": "left_hand", "name": "hold", "target": "clampgripper"}]}, "result": {"summary": "selector to insert the gear2 into the shaft2 with the outwardgripper in the left_hand", "name": "selector: insert(left_hand, outwardgripper, gear2, shaft2)", "children": [{"summary": "the target is to make the gear2 be inserted into the shaft2", "name": "target: is_inserted_to(gear2, shaft2)"}, {"summary": "sequence to insert the gear2 into the shaft2 with the outwardgripper in the left_hand", "name": "sequence: insert(left_hand, outwardgripper, gear2, shaft2)", "children": [{"summary": "selector to change tool in left_hand from clampgripper to outwardgripper", "name": "selector: change_tool(left_hand, clampgripper, outwardgripper)", "children": [{"summary": "the target is to make left_hand hold outwardgripper", "name": "target: hold(left_hand, outwardgripper)"}, {"summary": "sequence to change tool in left_hand from clampgripper to outwardgripper", "name": "sequence: change_tool(left_hand, clampgripper, outwardgripper)", "children": [{"summary": "a precondition is left_hand is holding clampgripper", "name": "precondition: hold(left_hand, clampgripper)"}, {"summary": "selector to put down the shaft1 held by the clampgripper in the left_hand", "name": "selector: put_down(left_hand, clampgripper, shaft1)", "children": [{"summary": "the target is to make the clampgripper empty", "name": "target: is_empty(clampgripper)"}, {"summary": "sequence to put down the shaft1 held by the clampgripper in the left_hand", "name": "sequence: put_down(left_hand, clampgripper, shaft1)", "children": [{"summary": "a precondition is the left_hand is holding clampgripper", "name": "precondition: hold(left_hand, clampgripper)"}, {"summary": "a precondition is the clampgripper is holding shaft1", "name": "precondition: hold(clampgripper, shaft1)"}, {"summary": "the action to put down the shaft1 held by the clampgripper in the left_hand", "name": "action: put_down(left_hand, clampgripper, shaft1)"}]}]}, {"summary": "the action to change tool in left_hand from clampgripper to outwardgripper", "name": "action: change_tool(left_hand, clampgripper, outwardgripper)"}]}]}, {"summary": "selector to pick up the gear2 with the outwardgripper in the left_hand", "name": "selector: pick_up(left_hand, outwardgripper, gear2)", "children": [{"summary": "the target is to make the outwardgripper hold the gear2", "name": "target: hold(outwardgripper, gear2)"}, {"summary": "sequence to pick up the gear2 with the outwardgripper in the left_hand", "name": "sequence: pick_up(left_hand, outwardgripper, gear2)", "children": [{"summary": "a precondition is the outwardgripper is empty", "name": "precondition: is_empty(outwardgripper)"}, {"summary": "a precondition is the left_hand is holding outwardgripper", "name": "precondition: hold(left_hand, outwardgripper)"}, {"summary": "the action to pick up the gear2 with the outwardgripper in the left_hand", "name": "action: pick_up(left_hand, outwardgripper, gear2)"}]}]}, {"summary": "the action to insert the gear2 into the shaft2 with the outwardgripper in the left_hand", "name": "action: insert(left_hand, outwardgripper, gear2, shaft2)"}]}]}} -{"target": "target: is_inserted_to(gear3, shaft3)", "initial_world_state": {"objects": [{"name": "clampgripper", "properties": []}, {"name": "parallelgripper", "properties": ["is_empty"]}, {"name": "inwardgripper", "properties": ["is_empty"]}, {"name": "outwardgripper", "properties": ["is_empty"]}, {"name": "defaultgripper", "properties": ["is_empty"]}, {"name": "gear1", "properties": []}, {"name": "gear2", "properties": []}, {"name": "gear3", "properties": []}, {"name": "shaft1", "properties": []}, {"name": "shaft2", "properties": []}, {"name": "shaft3", "properties": []}, {"name": "gearbase", "properties": []}, {"name": "gearbase_hole1", "properties": []}, {"name": "gearbase_hole3", "properties": []}, {"name": "left_hand", "properties": []}], "constraints": [{"source": "clampgripper", "name": "can_manipulate", "target": "shaft1"}, {"source": "gear3", "name": "can_insert_to", "target": "shaft3"}, {"source": "outwardgripper", "name": "can_manipulate", "target": "gear3"}, {"source": "shaft3", "name": "can_insert_to", "target": "gearbase_hole3"}, {"source": "gear1", "name": "can_insert_to", "target": "shaft1"}, {"source": "parallelgripper", "name": "can_manipulate", "target": "gear1"}, {"source": "defaultgripper", "name": "can_manipulate", "target": "shaft3"}, {"source": "gear2", "name": "can_insert_to", "target": "shaft2"}, {"source": "shaft1", "name": "can_insert_to", "target": "gearbase_hole1"}, {"source": "outwardgripper", "name": "can_manipulate", "target": "gear2"}], "relations": [{"source": "left_hand", "name": "hold", "target": "clampgripper"}, {"source": "clampgripper", "name": "hold", "target": "shaft1"}]}, "result": {"summary": "selector to insert the gear3 into the shaft3 with the outwardgripper in the left_hand", "name": "selector: insert(left_hand, outwardgripper, gear3, shaft3)", "children": [{"summary": "the target is to make the gear3 be inserted into the shaft3", "name": "target: is_inserted_to(gear3, shaft3)"}, {"summary": "sequence to insert the gear3 into the shaft3 with the outwardgripper in the left_hand", "name": "sequence: insert(left_hand, outwardgripper, gear3, shaft3)", "children": [{"summary": "selector to change tool in left_hand from clampgripper to outwardgripper", "name": "selector: change_tool(left_hand, clampgripper, outwardgripper)", "children": [{"summary": "the target is to make left_hand hold outwardgripper", "name": "target: hold(left_hand, outwardgripper)"}, {"summary": "sequence to change tool in left_hand from clampgripper to outwardgripper", "name": "sequence: change_tool(left_hand, clampgripper, outwardgripper)", "children": [{"summary": "a precondition is left_hand is holding clampgripper", "name": "precondition: hold(left_hand, clampgripper)"}, {"summary": "selector to put down the shaft1 held by the clampgripper in the left_hand", "name": "selector: put_down(left_hand, clampgripper, shaft1)", "children": [{"summary": "the target is to make the clampgripper empty", "name": "target: is_empty(clampgripper)"}, {"summary": "sequence to put down the shaft1 held by the clampgripper in the left_hand", "name": "sequence: put_down(left_hand, clampgripper, shaft1)", "children": [{"summary": "a precondition is the left_hand is holding clampgripper", "name": "precondition: hold(left_hand, clampgripper)"}, {"summary": "a precondition is the clampgripper is holding shaft1", "name": "precondition: hold(clampgripper, shaft1)"}, {"summary": "the action to put down the shaft1 held by the clampgripper in the left_hand", "name": "action: put_down(left_hand, clampgripper, shaft1)"}]}]}, {"summary": "the action to change tool in left_hand from clampgripper to outwardgripper", "name": "action: change_tool(left_hand, clampgripper, outwardgripper)"}]}]}, {"summary": "selector to pick up the gear3 with the outwardgripper in the left_hand", "name": "selector: pick_up(left_hand, outwardgripper, gear3)", "children": [{"summary": "the target is to make the outwardgripper hold the gear3", "name": "target: hold(outwardgripper, gear3)"}, {"summary": "sequence to pick up the gear3 with the outwardgripper in the left_hand", "name": "sequence: pick_up(left_hand, outwardgripper, gear3)", "children": [{"summary": "a precondition is the outwardgripper is empty", "name": "precondition: is_empty(outwardgripper)"}, {"summary": "a precondition is the left_hand is holding outwardgripper", "name": "precondition: hold(left_hand, outwardgripper)"}, {"summary": "the action to pick up the gear3 with the outwardgripper in the left_hand", "name": "action: pick_up(left_hand, outwardgripper, gear3)"}]}]}, {"summary": "the action to insert the gear3 into the shaft3 with the outwardgripper in the left_hand", "name": "action: insert(left_hand, outwardgripper, gear3, shaft3)"}]}]}} -{"target": "target: is_inserted_to(gear3, shaft3)", "initial_world_state": {"objects": [{"name": "clampgripper", "properties": ["is_empty"]}, {"name": "parallelgripper", "properties": ["is_empty"]}, {"name": "inwardgripper", "properties": ["is_empty"]}, {"name": "outwardgripper", "properties": ["is_empty"]}, {"name": "defaultgripper", "properties": ["is_empty"]}, {"name": "gear1", "properties": []}, {"name": "gear2", "properties": []}, {"name": "gear3", "properties": []}, {"name": "shaft1", "properties": []}, {"name": "shaft2", "properties": []}, {"name": "shaft3", "properties": []}, {"name": "gearbase", "properties": []}, {"name": "gearbase_hole1", "properties": []}, {"name": "gearbase_hole3", "properties": []}, {"name": "left_hand", "properties": []}], "constraints": [{"source": "gear2", "name": "can_insert_to", "target": "shaft2"}, {"source": "outwardgripper", "name": "can_manipulate", "target": "gear2"}, {"source": "shaft3", "name": "can_insert_to", "target": "gearbase_hole3"}, {"source": "clampgripper", "name": "can_manipulate", "target": "shaft1"}, {"source": "shaft1", "name": "can_insert_to", "target": "gearbase_hole1"}, {"source": "outwardgripper", "name": "can_manipulate", "target": "gear3"}, {"source": "gear1", "name": "can_insert_to", "target": "shaft1"}, {"source": "defaultgripper", "name": "can_manipulate", "target": "shaft3"}, {"source": "gear3", "name": "can_insert_to", "target": "shaft3"}, {"source": "parallelgripper", "name": "can_manipulate", "target": "gear1"}], "relations": [{"source": "left_hand", "name": "hold", "target": "defaultgripper"}]}, "result": {"summary": "selector to insert the gear3 into the shaft3 with the outwardgripper in the left_hand", "name": "selector: insert(left_hand, outwardgripper, gear3, shaft3)", "children": [{"summary": "the target is to make the gear3 be inserted into the shaft3", "name": "target: is_inserted_to(gear3, shaft3)"}, {"summary": "sequence to insert the gear3 into the shaft3 with the outwardgripper in the left_hand", "name": "sequence: insert(left_hand, outwardgripper, gear3, shaft3)", "children": [{"summary": "selector to change tool in left_hand from clampgripper to outwardgripper", "name": "selector: change_tool(left_hand, clampgripper, outwardgripper)", "children": [{"summary": "the target is to make left_hand hold outwardgripper", "name": "target: hold(left_hand, outwardgripper)"}, {"summary": "sequence to change tool in left_hand from clampgripper to outwardgripper", "name": "sequence: change_tool(left_hand, clampgripper, outwardgripper)", "children": [{"summary": "selector to change tool in left_hand from defaultgripper to clampgripper", "name": "selector: change_tool(left_hand, defaultgripper, clampgripper)", "children": [{"summary": "the target is to make left_hand hold clampgripper", "name": "target: hold(left_hand, clampgripper)"}, {"summary": "sequence to change tool in left_hand from defaultgripper to clampgripper", "name": "sequence: change_tool(left_hand, defaultgripper, clampgripper)", "children": [{"summary": "a precondition is left_hand is holding defaultgripper", "name": "precondition: hold(left_hand, defaultgripper)"}, {"summary": "a precondition is defaultgripper is empty", "name": "precondition: is_empty(defaultgripper)"}, {"summary": "the action to change tool in left_hand from defaultgripper to clampgripper", "name": "action: change_tool(left_hand, defaultgripper, clampgripper)"}]}]}, {"summary": "a precondition is clampgripper is empty", "name": "precondition: is_empty(clampgripper)"}, {"summary": "the action to change tool in left_hand from clampgripper to outwardgripper", "name": "action: change_tool(left_hand, clampgripper, outwardgripper)"}]}]}, {"summary": "selector to pick up the gear3 with the outwardgripper in the left_hand", "name": "selector: pick_up(left_hand, outwardgripper, gear3)", "children": [{"summary": "the target is to make the outwardgripper hold the gear3", "name": "target: hold(outwardgripper, gear3)"}, {"summary": "sequence to pick up the gear3 with the outwardgripper in the left_hand", "name": "sequence: pick_up(left_hand, outwardgripper, gear3)", "children": [{"summary": "a precondition is the outwardgripper is empty", "name": "precondition: is_empty(outwardgripper)"}, {"summary": "a precondition is the left_hand is holding outwardgripper", "name": "precondition: hold(left_hand, outwardgripper)"}, {"summary": "the action to pick up the gear3 with the outwardgripper in the left_hand", "name": "action: pick_up(left_hand, outwardgripper, gear3)"}]}]}, {"summary": "the action to insert the gear3 into the shaft3 with the outwardgripper in the left_hand", "name": "action: insert(left_hand, outwardgripper, gear3, shaft3)"}]}]}} -{"target": "target: is_inserted_to(gear3, shaft3)", "initial_world_state": {"objects": [{"name": "clampgripper", "properties": ["is_empty"]}, {"name": "parallelgripper", "properties": ["is_empty"]}, {"name": "inwardgripper", "properties": ["is_empty"]}, {"name": "outwardgripper", "properties": ["is_empty"]}, {"name": "defaultgripper", "properties": ["is_empty"]}, {"name": "gear1", "properties": []}, {"name": "gear2", "properties": []}, {"name": "gear3", "properties": []}, {"name": "shaft1", "properties": []}, {"name": "shaft2", "properties": []}, {"name": "shaft3", "properties": []}, {"name": "gearbase", "properties": []}, {"name": "gearbase_hole1", "properties": []}, {"name": "gearbase_hole3", "properties": []}, {"name": "left_hand", "properties": []}], "constraints": [{"source": "outwardgripper", "name": "can_manipulate", "target": "gear2"}, {"source": "clampgripper", "name": "can_manipulate", "target": "shaft1"}, {"source": "defaultgripper", "name": "can_manipulate", "target": "shaft3"}, {"source": "gear2", "name": "can_insert_to", "target": "shaft2"}, {"source": "shaft1", "name": "can_insert_to", "target": "gearbase_hole1"}, {"source": "gear3", "name": "can_insert_to", "target": "shaft3"}, {"source": "shaft3", "name": "can_insert_to", "target": "gearbase_hole3"}, {"source": "parallelgripper", "name": "can_manipulate", "target": "gear1"}, {"source": "outwardgripper", "name": "can_manipulate", "target": "gear3"}, {"source": "gear1", "name": "can_insert_to", "target": "shaft1"}], "relations": [{"source": "left_hand", "name": "hold", "target": "outwardgripper"}]}, "result": {"summary": "selector to insert the gear3 into the shaft3 with the outwardgripper in the left_hand", "name": "selector: insert(left_hand, outwardgripper, gear3, shaft3)", "children": [{"summary": "the target is to make the gear3 be inserted into the shaft3", "name": "target: is_inserted_to(gear3, shaft3)"}, {"summary": "sequence to insert the gear3 into the shaft3 with the outwardgripper in the left_hand", "name": "sequence: insert(left_hand, outwardgripper, gear3, shaft3)", "children": [{"summary": "a precondition is the left_hand is holding outwardgripper", "name": "precondition: hold(left_hand, outwardgripper)"}, {"summary": "selector to pick up the gear3 with the outwardgripper in the left_hand", "name": "selector: pick_up(left_hand, outwardgripper, gear3)", "children": [{"summary": "the target is to make the outwardgripper hold the gear3", "name": "target: hold(outwardgripper, gear3)"}, {"summary": "sequence to pick up the gear3 with the outwardgripper in the left_hand", "name": "sequence: pick_up(left_hand, outwardgripper, gear3)", "children": [{"summary": "a precondition is the outwardgripper is empty", "name": "precondition: is_empty(outwardgripper)"}, {"summary": "a precondition is the left_hand is holding outwardgripper", "name": "precondition: hold(left_hand, outwardgripper)"}, {"summary": "the action to pick up the gear3 with the outwardgripper in the left_hand", "name": "action: pick_up(left_hand, outwardgripper, gear3)"}]}]}, {"summary": "the action to insert the gear3 into the shaft3 with the outwardgripper in the left_hand", "name": "action: insert(left_hand, outwardgripper, gear3, shaft3)"}]}]}} -{"target": "target: is_inserted_to(gear1, shaft1)", "initial_world_state": {"objects": [{"name": "clampgripper", "properties": ["is_empty"]}, {"name": "parallelgripper", "properties": ["is_empty"]}, {"name": "inwardgripper", "properties": ["is_empty"]}, {"name": "outwardgripper", "properties": ["is_empty"]}, {"name": "defaultgripper", "properties": ["is_empty"]}, {"name": "gear1", "properties": []}, {"name": "gear2", "properties": []}, {"name": "gear3", "properties": []}, {"name": "shaft1", "properties": []}, {"name": "shaft2", "properties": []}, {"name": "shaft3", "properties": []}, {"name": "gearbase", "properties": []}, {"name": "gearbase_hole1", "properties": []}, {"name": "gearbase_hole3", "properties": []}, {"name": "left_hand", "properties": []}], "constraints": [{"source": "clampgripper", "name": "can_manipulate", "target": "shaft1"}, {"source": "outwardgripper", "name": "can_manipulate", "target": "gear2"}, {"source": "parallelgripper", "name": "can_manipulate", "target": "gear1"}, {"source": "gear1", "name": "can_insert_to", "target": "shaft1"}, {"source": "shaft1", "name": "can_insert_to", "target": "gearbase_hole1"}, {"source": "gear2", "name": "can_insert_to", "target": "shaft2"}, {"source": "outwardgripper", "name": "can_manipulate", "target": "gear3"}, {"source": "gear3", "name": "can_insert_to", "target": "shaft3"}, {"source": "shaft3", "name": "can_insert_to", "target": "gearbase_hole3"}, {"source": "defaultgripper", "name": "can_manipulate", "target": "shaft3"}], "relations": [{"source": "shaft1", "name": "is_inserted_to", "target": "gearbase_hole1"}, {"source": "left_hand", "name": "hold", "target": "outwardgripper"}]}, "result": {"summary": "selector to insert the gear1 into the shaft1 with the parallelgripper in the left_hand", "name": "selector: insert(left_hand, parallelgripper, gear1, shaft1)", "children": [{"summary": "the target is to make the gear1 be inserted into the shaft1", "name": "target: is_inserted_to(gear1, shaft1)"}, {"summary": "sequence to insert the gear1 into the shaft1 with the parallelgripper in the left_hand", "name": "sequence: insert(left_hand, parallelgripper, gear1, shaft1)", "children": [{"summary": "selector to change tool in left_hand from outwardgripper to parallelgripper", "name": "selector: change_tool(left_hand, outwardgripper, parallelgripper)", "children": [{"summary": "the target is to make left_hand hold parallelgripper", "name": "target: hold(left_hand, parallelgripper)"}, {"summary": "sequence to change tool in left_hand from outwardgripper to parallelgripper", "name": "sequence: change_tool(left_hand, outwardgripper, parallelgripper)", "children": [{"summary": "a precondition is left_hand is holding outwardgripper", "name": "precondition: hold(left_hand, outwardgripper)"}, {"summary": "a precondition is outwardgripper is empty", "name": "precondition: is_empty(outwardgripper)"}, {"summary": "the action to change tool in left_hand from outwardgripper to parallelgripper", "name": "action: change_tool(left_hand, outwardgripper, parallelgripper)"}]}]}, {"summary": "selector to pick up the gear1 with the parallelgripper in the left_hand", "name": "selector: pick_up(left_hand, parallelgripper, gear1)", "children": [{"summary": "the target is to make the parallelgripper hold the gear1", "name": "target: hold(parallelgripper, gear1)"}, {"summary": "sequence to pick up the gear1 with the parallelgripper in the left_hand", "name": "sequence: pick_up(left_hand, parallelgripper, gear1)", "children": [{"summary": "a precondition is the parallelgripper is empty", "name": "precondition: is_empty(parallelgripper)"}, {"summary": "a precondition is the left_hand is holding parallelgripper", "name": "precondition: hold(left_hand, parallelgripper)"}, {"summary": "the action to pick up the gear1 with the parallelgripper in the left_hand", "name": "action: pick_up(left_hand, parallelgripper, gear1)"}]}]}, {"summary": "the action to insert the gear1 into the shaft1 with the parallelgripper in the left_hand", "name": "action: insert(left_hand, parallelgripper, gear1, shaft1)"}]}]}} -{"target": "target: is_inserted_to(gear1, shaft1)", "initial_world_state": {"objects": [{"name": "clampgripper", "properties": ["is_empty"]}, {"name": "parallelgripper", "properties": ["is_empty"]}, {"name": "inwardgripper", "properties": ["is_empty"]}, {"name": "outwardgripper", "properties": ["is_empty"]}, {"name": "defaultgripper", "properties": ["is_empty"]}, {"name": "gear1", "properties": []}, {"name": "gear2", "properties": []}, {"name": "gear3", "properties": []}, {"name": "shaft1", "properties": []}, {"name": "shaft2", "properties": []}, {"name": "shaft3", "properties": []}, {"name": "gearbase", "properties": []}, {"name": "gearbase_hole1", "properties": []}, {"name": "gearbase_hole3", "properties": []}, {"name": "left_hand", "properties": []}], "constraints": [{"source": "parallelgripper", "name": "can_manipulate", "target": "gear1"}, {"source": "outwardgripper", "name": "can_manipulate", "target": "gear3"}, {"source": "gear2", "name": "can_insert_to", "target": "shaft2"}, {"source": "defaultgripper", "name": "can_manipulate", "target": "shaft3"}, {"source": "outwardgripper", "name": "can_manipulate", "target": "gear2"}, {"source": "gear3", "name": "can_insert_to", "target": "shaft3"}, {"source": "clampgripper", "name": "can_manipulate", "target": "shaft1"}, {"source": "shaft3", "name": "can_insert_to", "target": "gearbase_hole3"}, {"source": "shaft1", "name": "can_insert_to", "target": "gearbase_hole1"}, {"source": "gear1", "name": "can_insert_to", "target": "shaft1"}], "relations": [{"source": "shaft1", "name": "is_inserted_to", "target": "gearbase_hole1"}, {"source": "left_hand", "name": "hold", "target": "outwardgripper"}, {"source": "gear1", "name": "is_inserted_to", "target": "shaft1"}]}, "result": {"summary": "the target is to insert gear1 to shaft1", "name": "target: is_inserted_to(gear1, shaft1)"}} -{"target": "target: is_inserted_to(gear3, shaft3)", "initial_world_state": {"objects": [{"name": "clampgripper", "properties": ["is_empty"]}, {"name": "parallelgripper", "properties": ["is_empty"]}, {"name": "inwardgripper", "properties": ["is_empty"]}, {"name": "outwardgripper", "properties": ["is_empty"]}, {"name": "defaultgripper", "properties": ["is_empty"]}, {"name": "gear1", "properties": []}, {"name": "gear2", "properties": []}, {"name": "gear3", "properties": []}, {"name": "shaft1", "properties": []}, {"name": "shaft2", "properties": []}, {"name": "shaft3", "properties": []}, {"name": "gearbase", "properties": []}, {"name": "gearbase_hole1", "properties": []}, {"name": "gearbase_hole3", "properties": []}, {"name": "left_hand", "properties": []}], "constraints": [{"source": "gear2", "name": "can_insert_to", "target": "shaft2"}, {"source": "parallelgripper", "name": "can_manipulate", "target": "gear1"}, {"source": "gear3", "name": "can_insert_to", "target": "shaft3"}, {"source": "defaultgripper", "name": "can_manipulate", "target": "shaft3"}, {"source": "outwardgripper", "name": "can_manipulate", "target": "gear2"}, {"source": "gear1", "name": "can_insert_to", "target": "shaft1"}, {"source": "outwardgripper", "name": "can_manipulate", "target": "gear3"}, {"source": "shaft1", "name": "can_insert_to", "target": "gearbase_hole1"}, {"source": "clampgripper", "name": "can_manipulate", "target": "shaft1"}, {"source": "shaft3", "name": "can_insert_to", "target": "gearbase_hole3"}], "relations": [{"source": "gear1", "name": "is_inserted_to", "target": "shaft1"}, {"source": "shaft1", "name": "is_inserted_to", "target": "gearbase_hole1"}, {"source": "shaft3", "name": "is_inserted_to", "target": "gearbase_hole3"}, {"source": "left_hand", "name": "hold", "target": "defaultgripper"}]}, "result": {"summary": "selector to insert the gear3 into the shaft3 with the outwardgripper in the left_hand", "name": "selector: insert(left_hand, outwardgripper, gear3, shaft3)", "children": [{"summary": "the target is to make the gear3 be inserted into the shaft3", "name": "target: is_inserted_to(gear3, shaft3)"}, {"summary": "sequence to insert the gear3 into the shaft3 with the outwardgripper in the left_hand", "name": "sequence: insert(left_hand, outwardgripper, gear3, shaft3)", "children": [{"summary": "selector to change tool in left_hand from defaultgripper to outwardgripper", "name": "selector: change_tool(left_hand, defaultgripper, outwardgripper)", "children": [{"summary": "the target is to make left_hand hold outwardgripper", "name": "target: hold(left_hand, outwardgripper)"}, {"summary": "sequence to change tool in left_hand from defaultgripper to outwardgripper", "name": "sequence: change_tool(left_hand, defaultgripper, outwardgripper)", "children": [{"summary": "a precondition is left_hand is holding defaultgripper", "name": "precondition: hold(left_hand, defaultgripper)"}, {"summary": "a precondition is defaultgripper is empty", "name": "precondition: is_empty(defaultgripper)"}, {"summary": "the action to change tool in left_hand from defaultgripper to outwardgripper", "name": "action: change_tool(left_hand, defaultgripper, outwardgripper)"}]}]}, {"summary": "selector to pick up the gear3 with the outwardgripper in the left_hand", "name": "selector: pick_up(left_hand, outwardgripper, gear3)", "children": [{"summary": "the target is to make the outwardgripper hold the gear3", "name": "target: hold(outwardgripper, gear3)"}, {"summary": "sequence to pick up the gear3 with the outwardgripper in the left_hand", "name": "sequence: pick_up(left_hand, outwardgripper, gear3)", "children": [{"summary": "a precondition is the outwardgripper is empty", "name": "precondition: is_empty(outwardgripper)"}, {"summary": "a precondition is the left_hand is holding outwardgripper", "name": "precondition: hold(left_hand, outwardgripper)"}, {"summary": "the action to pick up the gear3 with the outwardgripper in the left_hand", "name": "action: pick_up(left_hand, outwardgripper, gear3)"}]}]}, {"summary": "the action to insert the gear3 into the shaft3 with the outwardgripper in the left_hand", "name": "action: insert(left_hand, outwardgripper, gear3, shaft3)"}]}]}} -{"target": "target: is_inserted_to(shaft1, gearbase_hole1)", "initial_world_state": {"objects": [{"name": "clampgripper", "properties": ["is_empty"]}, {"name": "parallelgripper", "properties": ["is_empty"]}, {"name": "inwardgripper", "properties": ["is_empty"]}, {"name": "outwardgripper", "properties": ["is_empty"]}, {"name": "defaultgripper", "properties": []}, {"name": "gear1", "properties": []}, {"name": "gear2", "properties": []}, {"name": "gear3", "properties": []}, {"name": "shaft1", "properties": []}, {"name": "shaft2", "properties": []}, {"name": "shaft3", "properties": []}, {"name": "gearbase", "properties": []}, {"name": "gearbase_hole1", "properties": []}, {"name": "gearbase_hole3", "properties": []}, {"name": "left_hand", "properties": []}], "constraints": [{"source": "shaft3", "name": "can_insert_to", "target": "gearbase_hole3"}, {"source": "defaultgripper", "name": "can_manipulate", "target": "shaft3"}, {"source": "gear2", "name": "can_insert_to", "target": "shaft2"}, {"source": "gear1", "name": "can_insert_to", "target": "shaft1"}, {"source": "outwardgripper", "name": "can_manipulate", "target": "gear2"}, {"source": "shaft1", "name": "can_insert_to", "target": "gearbase_hole1"}, {"source": "outwardgripper", "name": "can_manipulate", "target": "gear3"}, {"source": "clampgripper", "name": "can_manipulate", "target": "shaft1"}, {"source": "parallelgripper", "name": "can_manipulate", "target": "gear1"}, {"source": "gear3", "name": "can_insert_to", "target": "shaft3"}], "relations": [{"source": "left_hand", "name": "hold", "target": "defaultgripper"}, {"source": "defaultgripper", "name": "hold", "target": "shaft3"}]}, "result": {"summary": "selector to insert the shaft1 into the gearbase_hole1 with the clampgripper in the left_hand", "name": "selector: insert(left_hand, clampgripper, shaft1, gearbase_hole1)", "children": [{"summary": "the target is to make the shaft1 be inserted into the gearbase_hole1", "name": "target: is_inserted_to(shaft1, gearbase_hole1)"}, {"summary": "sequence to insert the shaft1 into the gearbase_hole1 with the clampgripper in the left_hand", "name": "sequence: insert(left_hand, clampgripper, shaft1, gearbase_hole1)", "children": [{"summary": "selector to change tool in left_hand from defaultgripper to clampgripper", "name": "selector: change_tool(left_hand, defaultgripper, clampgripper)", "children": [{"summary": "the target is to make left_hand hold clampgripper", "name": "target: hold(left_hand, clampgripper)"}, {"summary": "sequence to change tool in left_hand from defaultgripper to clampgripper", "name": "sequence: change_tool(left_hand, defaultgripper, clampgripper)", "children": [{"summary": "a precondition is left_hand is holding defaultgripper", "name": "precondition: hold(left_hand, defaultgripper)"}, {"summary": "selector to put down the shaft3 held by the defaultgripper in the left_hand", "name": "selector: put_down(left_hand, defaultgripper, shaft3)", "children": [{"summary": "the target is to make the defaultgripper empty", "name": "target: is_empty(defaultgripper)"}, {"summary": "sequence to put down the shaft3 held by the defaultgripper in the left_hand", "name": "sequence: put_down(left_hand, defaultgripper, shaft3)", "children": [{"summary": "a precondition is the left_hand is holding defaultgripper", "name": "precondition: hold(left_hand, defaultgripper)"}, {"summary": "a precondition is the defaultgripper is holding shaft3", "name": "precondition: hold(defaultgripper, shaft3)"}, {"summary": "the action to put down the shaft3 held by the defaultgripper in the left_hand", "name": "action: put_down(left_hand, defaultgripper, shaft3)"}]}]}, {"summary": "the action to change tool in left_hand from defaultgripper to clampgripper", "name": "action: change_tool(left_hand, defaultgripper, clampgripper)"}]}]}, {"summary": "selector to pick up the shaft1 with the clampgripper in the left_hand", "name": "selector: pick_up(left_hand, clampgripper, shaft1)", "children": [{"summary": "the target is to make the clampgripper hold the shaft1", "name": "target: hold(clampgripper, shaft1)"}, {"summary": "sequence to pick up the shaft1 with the clampgripper in the left_hand", "name": "sequence: pick_up(left_hand, clampgripper, shaft1)", "children": [{"summary": "a precondition is the clampgripper is empty", "name": "precondition: is_empty(clampgripper)"}, {"summary": "a precondition is the left_hand is holding clampgripper", "name": "precondition: hold(left_hand, clampgripper)"}, {"summary": "the action to pick up the shaft1 with the clampgripper in the left_hand", "name": "action: pick_up(left_hand, clampgripper, shaft1)"}]}]}, {"summary": "the action to insert the shaft1 into the gearbase_hole1 with the clampgripper in the left_hand", "name": "action: insert(left_hand, clampgripper, shaft1, gearbase_hole1)"}]}]}} -{"target": "target: is_inserted_to(gear2, shaft2)", "initial_world_state": {"objects": [{"name": "clampgripper", "properties": ["is_empty"]}, {"name": "parallelgripper", "properties": ["is_empty"]}, {"name": "inwardgripper", "properties": ["is_empty"]}, {"name": "outwardgripper", "properties": ["is_empty"]}, {"name": "defaultgripper", "properties": ["is_empty"]}, {"name": "gear1", "properties": []}, {"name": "gear2", "properties": []}, {"name": "gear3", "properties": []}, {"name": "shaft1", "properties": []}, {"name": "shaft2", "properties": []}, {"name": "shaft3", "properties": []}, {"name": "gearbase", "properties": []}, {"name": "gearbase_hole1", "properties": []}, {"name": "gearbase_hole3", "properties": []}, {"name": "left_hand", "properties": []}], "constraints": [{"source": "shaft1", "name": "can_insert_to", "target": "gearbase_hole1"}, {"source": "shaft3", "name": "can_insert_to", "target": "gearbase_hole3"}, {"source": "parallelgripper", "name": "can_manipulate", "target": "gear1"}, {"source": "clampgripper", "name": "can_manipulate", "target": "shaft1"}, {"source": "defaultgripper", "name": "can_manipulate", "target": "shaft3"}, {"source": "outwardgripper", "name": "can_manipulate", "target": "gear2"}, {"source": "gear3", "name": "can_insert_to", "target": "shaft3"}, {"source": "outwardgripper", "name": "can_manipulate", "target": "gear3"}, {"source": "gear1", "name": "can_insert_to", "target": "shaft1"}, {"source": "gear2", "name": "can_insert_to", "target": "shaft2"}], "relations": [{"source": "left_hand", "name": "hold", "target": "defaultgripper"}]}, "result": {"summary": "selector to insert the gear2 into the shaft2 with the outwardgripper in the left_hand", "name": "selector: insert(left_hand, outwardgripper, gear2, shaft2)", "children": [{"summary": "the target is to make the gear2 be inserted into the shaft2", "name": "target: is_inserted_to(gear2, shaft2)"}, {"summary": "sequence to insert the gear2 into the shaft2 with the outwardgripper in the left_hand", "name": "sequence: insert(left_hand, outwardgripper, gear2, shaft2)", "children": [{"summary": "selector to change tool in left_hand from defaultgripper to outwardgripper", "name": "selector: change_tool(left_hand, defaultgripper, outwardgripper)", "children": [{"summary": "the target is to make left_hand hold outwardgripper", "name": "target: hold(left_hand, outwardgripper)"}, {"summary": "sequence to change tool in left_hand from defaultgripper to outwardgripper", "name": "sequence: change_tool(left_hand, defaultgripper, outwardgripper)", "children": [{"summary": "a precondition is left_hand is holding defaultgripper", "name": "precondition: hold(left_hand, defaultgripper)"}, {"summary": "a precondition is defaultgripper is empty", "name": "precondition: is_empty(defaultgripper)"}, {"summary": "the action to change tool in left_hand from defaultgripper to outwardgripper", "name": "action: change_tool(left_hand, defaultgripper, outwardgripper)"}]}]}, {"summary": "selector to pick up the gear2 with the outwardgripper in the left_hand", "name": "selector: pick_up(left_hand, outwardgripper, gear2)", "children": [{"summary": "the target is to make the outwardgripper hold the gear2", "name": "target: hold(outwardgripper, gear2)"}, {"summary": "sequence to pick up the gear2 with the outwardgripper in the left_hand", "name": "sequence: pick_up(left_hand, outwardgripper, gear2)", "children": [{"summary": "a precondition is the outwardgripper is empty", "name": "precondition: is_empty(outwardgripper)"}, {"summary": "a precondition is the left_hand is holding outwardgripper", "name": "precondition: hold(left_hand, outwardgripper)"}, {"summary": "the action to pick up the gear2 with the outwardgripper in the left_hand", "name": "action: pick_up(left_hand, outwardgripper, gear2)"}]}]}, {"summary": "the action to insert the gear2 into the shaft2 with the outwardgripper in the left_hand", "name": "action: insert(left_hand, outwardgripper, gear2, shaft2)"}]}]}} -{"target": "target: is_inserted_to(shaft3, gearbase_hole3)", "initial_world_state": {"objects": [{"name": "clampgripper", "properties": ["is_empty"]}, {"name": "parallelgripper", "properties": ["is_empty"]}, {"name": "inwardgripper", "properties": ["is_empty"]}, {"name": "outwardgripper", "properties": ["is_empty"]}, {"name": "defaultgripper", "properties": ["is_empty"]}, {"name": "gear1", "properties": []}, {"name": "gear2", "properties": []}, {"name": "gear3", "properties": []}, {"name": "shaft1", "properties": []}, {"name": "shaft2", "properties": []}, {"name": "shaft3", "properties": []}, {"name": "gearbase", "properties": []}, {"name": "gearbase_hole1", "properties": []}, {"name": "gearbase_hole3", "properties": []}, {"name": "left_hand", "properties": []}], "constraints": [{"source": "outwardgripper", "name": "can_manipulate", "target": "gear2"}, {"source": "shaft1", "name": "can_insert_to", "target": "gearbase_hole1"}, {"source": "clampgripper", "name": "can_manipulate", "target": "shaft1"}, {"source": "defaultgripper", "name": "can_manipulate", "target": "shaft3"}, {"source": "outwardgripper", "name": "can_manipulate", "target": "gear3"}, {"source": "gear2", "name": "can_insert_to", "target": "shaft2"}, {"source": "parallelgripper", "name": "can_manipulate", "target": "gear1"}, {"source": "gear1", "name": "can_insert_to", "target": "shaft1"}, {"source": "gear3", "name": "can_insert_to", "target": "shaft3"}, {"source": "shaft3", "name": "can_insert_to", "target": "gearbase_hole3"}], "relations": [{"source": "left_hand", "name": "hold", "target": "inwardgripper"}]}, "result": {"summary": "selector to insert the shaft3 into the gearbase_hole3 with the defaultgripper in the left_hand", "name": "selector: insert(left_hand, defaultgripper, shaft3, gearbase_hole3)", "children": [{"summary": "the target is to make the shaft3 be inserted into the gearbase_hole3", "name": "target: is_inserted_to(shaft3, gearbase_hole3)"}, {"summary": "sequence to insert the shaft3 into the gearbase_hole3 with the defaultgripper in the left_hand", "name": "sequence: insert(left_hand, defaultgripper, shaft3, gearbase_hole3)", "children": [{"summary": "selector to change tool in left_hand from inwardgripper to defaultgripper", "name": "selector: change_tool(left_hand, inwardgripper, defaultgripper)", "children": [{"summary": "the target is to make left_hand hold defaultgripper", "name": "target: hold(left_hand, defaultgripper)"}, {"summary": "sequence to change tool in left_hand from inwardgripper to defaultgripper", "name": "sequence: change_tool(left_hand, inwardgripper, defaultgripper)", "children": [{"summary": "a precondition is left_hand is holding inwardgripper", "name": "precondition: hold(left_hand, inwardgripper)"}, {"summary": "a precondition is inwardgripper is empty", "name": "precondition: is_empty(inwardgripper)"}, {"summary": "the action to change tool in left_hand from inwardgripper to defaultgripper", "name": "action: change_tool(left_hand, inwardgripper, defaultgripper)"}]}]}, {"summary": "selector to pick up the shaft3 with the defaultgripper in the left_hand", "name": "selector: pick_up(left_hand, defaultgripper, shaft3)", "children": [{"summary": "the target is to make the defaultgripper hold the shaft3", "name": "target: hold(defaultgripper, shaft3)"}, {"summary": "sequence to pick up the shaft3 with the defaultgripper in the left_hand", "name": "sequence: pick_up(left_hand, defaultgripper, shaft3)", "children": [{"summary": "a precondition is the defaultgripper is empty", "name": "precondition: is_empty(defaultgripper)"}, {"summary": "a precondition is the left_hand is holding defaultgripper", "name": "precondition: hold(left_hand, defaultgripper)"}, {"summary": "the action to pick up the shaft3 with the defaultgripper in the left_hand", "name": "action: pick_up(left_hand, defaultgripper, shaft3)"}]}]}, {"summary": "the action to insert the shaft3 into the gearbase_hole3 with the defaultgripper in the left_hand", "name": "action: insert(left_hand, defaultgripper, shaft3, gearbase_hole3)"}]}]}} -{"target": "target: is_inserted_to(shaft3, gearbase_hole3)", "initial_world_state": {"objects": [{"name": "clampgripper", "properties": ["is_empty"]}, {"name": "parallelgripper", "properties": []}, {"name": "inwardgripper", "properties": ["is_empty"]}, {"name": "outwardgripper", "properties": ["is_empty"]}, {"name": "defaultgripper", "properties": ["is_empty"]}, {"name": "gear1", "properties": []}, {"name": "gear2", "properties": []}, {"name": "gear3", "properties": []}, {"name": "shaft1", "properties": []}, {"name": "shaft2", "properties": []}, {"name": "shaft3", "properties": []}, {"name": "gearbase", "properties": []}, {"name": "gearbase_hole1", "properties": []}, {"name": "gearbase_hole3", "properties": []}, {"name": "left_hand", "properties": []}], "constraints": [{"source": "shaft3", "name": "can_insert_to", "target": "gearbase_hole3"}, {"source": "gear3", "name": "can_insert_to", "target": "shaft3"}, {"source": "defaultgripper", "name": "can_manipulate", "target": "shaft3"}, {"source": "shaft1", "name": "can_insert_to", "target": "gearbase_hole1"}, {"source": "outwardgripper", "name": "can_manipulate", "target": "gear2"}, {"source": "gear1", "name": "can_insert_to", "target": "shaft1"}, {"source": "clampgripper", "name": "can_manipulate", "target": "shaft1"}, {"source": "outwardgripper", "name": "can_manipulate", "target": "gear3"}, {"source": "gear2", "name": "can_insert_to", "target": "shaft2"}, {"source": "parallelgripper", "name": "can_manipulate", "target": "gear1"}], "relations": [{"source": "left_hand", "name": "hold", "target": "parallelgripper"}, {"source": "parallelgripper", "name": "hold", "target": "gear1"}]}, "result": {"summary": "selector to insert the shaft3 into the gearbase_hole3 with the defaultgripper in the left_hand", "name": "selector: insert(left_hand, defaultgripper, shaft3, gearbase_hole3)", "children": [{"summary": "the target is to make the shaft3 be inserted into the gearbase_hole3", "name": "target: is_inserted_to(shaft3, gearbase_hole3)"}, {"summary": "sequence to insert the shaft3 into the gearbase_hole3 with the defaultgripper in the left_hand", "name": "sequence: insert(left_hand, defaultgripper, shaft3, gearbase_hole3)", "children": [{"summary": "selector to change tool in left_hand from parallelgripper to defaultgripper", "name": "selector: change_tool(left_hand, parallelgripper, defaultgripper)", "children": [{"summary": "the target is to make left_hand hold defaultgripper", "name": "target: hold(left_hand, defaultgripper)"}, {"summary": "sequence to change tool in left_hand from parallelgripper to defaultgripper", "name": "sequence: change_tool(left_hand, parallelgripper, defaultgripper)", "children": [{"summary": "a precondition is left_hand is holding parallelgripper", "name": "precondition: hold(left_hand, parallelgripper)"}, {"summary": "selector to put down the gear1 held by the parallelgripper in the left_hand", "name": "selector: put_down(left_hand, parallelgripper, gear1)", "children": [{"summary": "the target is to make the parallelgripper empty", "name": "target: is_empty(parallelgripper)"}, {"summary": "sequence to put down the gear1 held by the parallelgripper in the left_hand", "name": "sequence: put_down(left_hand, parallelgripper, gear1)", "children": [{"summary": "a precondition is the left_hand is holding parallelgripper", "name": "precondition: hold(left_hand, parallelgripper)"}, {"summary": "a precondition is the parallelgripper is holding gear1", "name": "precondition: hold(parallelgripper, gear1)"}, {"summary": "the action to put down the gear1 held by the parallelgripper in the left_hand", "name": "action: put_down(left_hand, parallelgripper, gear1)"}]}]}, {"summary": "the action to change tool in left_hand from parallelgripper to defaultgripper", "name": "action: change_tool(left_hand, parallelgripper, defaultgripper)"}]}]}, {"summary": "selector to pick up the shaft3 with the defaultgripper in the left_hand", "name": "selector: pick_up(left_hand, defaultgripper, shaft3)", "children": [{"summary": "the target is to make the defaultgripper hold the shaft3", "name": "target: hold(defaultgripper, shaft3)"}, {"summary": "sequence to pick up the shaft3 with the defaultgripper in the left_hand", "name": "sequence: pick_up(left_hand, defaultgripper, shaft3)", "children": [{"summary": "a precondition is the defaultgripper is empty", "name": "precondition: is_empty(defaultgripper)"}, {"summary": "a precondition is the left_hand is holding defaultgripper", "name": "precondition: hold(left_hand, defaultgripper)"}, {"summary": "the action to pick up the shaft3 with the defaultgripper in the left_hand", "name": "action: pick_up(left_hand, defaultgripper, shaft3)"}]}]}, {"summary": "the action to insert the shaft3 into the gearbase_hole3 with the defaultgripper in the left_hand", "name": "action: insert(left_hand, defaultgripper, shaft3, gearbase_hole3)"}]}]}} -{"target": "target: is_inserted_to(gear2, shaft2)", "initial_world_state": {"objects": [{"name": "clampgripper", "properties": ["is_empty"]}, {"name": "parallelgripper", "properties": []}, {"name": "inwardgripper", "properties": ["is_empty"]}, {"name": "outwardgripper", "properties": ["is_empty"]}, {"name": "defaultgripper", "properties": ["is_empty"]}, {"name": "gear1", "properties": []}, {"name": "gear2", "properties": []}, {"name": "gear3", "properties": []}, {"name": "shaft1", "properties": []}, {"name": "shaft2", "properties": []}, {"name": "shaft3", "properties": []}, {"name": "gearbase", "properties": []}, {"name": "gearbase_hole1", "properties": []}, {"name": "gearbase_hole3", "properties": []}, {"name": "lampbase", "properties": []}, {"name": "chair_leg1", "properties": []}, {"name": "left_hand", "properties": []}], "constraints": [{"source": "clampgripper", "name": "can_manipulate", "target": "shaft1"}, {"source": "outwardgripper", "name": "can_manipulate", "target": "gear3"}, {"source": "shaft3", "name": "can_insert_to", "target": "gearbase_hole3"}, {"source": "parallelgripper", "name": "can_manipulate", "target": "gear1"}, {"source": "gear3", "name": "can_insert_to", "target": "shaft3"}, {"source": "outwardgripper", "name": "can_manipulate", "target": "gear2"}, {"source": "shaft1", "name": "can_insert_to", "target": "gearbase_hole1"}, {"source": "defaultgripper", "name": "can_manipulate", "target": "shaft3"}, {"source": "gear1", "name": "can_insert_to", "target": "shaft1"}, {"source": "gear2", "name": "can_insert_to", "target": "shaft2"}], "relations": [{"source": "gear2", "name": "is_inserted_to", "target": "shaft2"}, {"source": "parallelgripper", "name": "hold", "target": "gear1"}, {"source": "left_hand", "name": "hold", "target": "parallelgripper"}]}, "result": {"summary": "the target is to insert the gear2 into the shaft2", "name": "target: is_inserted_to(gear2, shaft2)"}} -{"target": "target: is_inserted_to(gear3, shaft3)", "initial_world_state": {"objects": [{"name": "clampgripper", "properties": ["is_empty"]}, {"name": "parallelgripper", "properties": []}, {"name": "inwardgripper", "properties": ["is_empty"]}, {"name": "outwardgripper", "properties": ["is_empty"]}, {"name": "defaultgripper", "properties": ["is_empty"]}, {"name": "gear1", "properties": []}, {"name": "gear2", "properties": []}, {"name": "gear3", "properties": []}, {"name": "shaft1", "properties": []}, {"name": "shaft2", "properties": []}, {"name": "shaft3", "properties": []}, {"name": "gearbase", "properties": []}, {"name": "gearbase_hole1", "properties": []}, {"name": "gearbase_hole3", "properties": []}, {"name": "lampbase", "properties": []}, {"name": "chair_leg1", "properties": []}, {"name": "left_hand", "properties": []}], "constraints": [{"source": "outwardgripper", "name": "can_manipulate", "target": "gear2"}, {"source": "shaft1", "name": "can_insert_to", "target": "gearbase_hole1"}, {"source": "outwardgripper", "name": "can_manipulate", "target": "gear3"}, {"source": "parallelgripper", "name": "can_manipulate", "target": "gear1"}, {"source": "defaultgripper", "name": "can_manipulate", "target": "shaft3"}, {"source": "clampgripper", "name": "can_manipulate", "target": "shaft1"}, {"source": "gear3", "name": "can_insert_to", "target": "shaft3"}, {"source": "shaft3", "name": "can_insert_to", "target": "gearbase_hole3"}, {"source": "gear1", "name": "can_insert_to", "target": "shaft1"}, {"source": "gear2", "name": "can_insert_to", "target": "shaft2"}], "relations": [{"source": "shaft1", "name": "is_inserted_to", "target": "gearbase_hole1"}, {"source": "left_hand", "name": "hold", "target": "parallelgripper"}, {"source": "gear3", "name": "is_inserted_to", "target": "shaft3"}, {"source": "shaft3", "name": "is_inserted_to", "target": "gearbase_hole3"}, {"source": "gear2", "name": "is_inserted_to", "target": "shaft2"}]}, "result": {"summary": "the target is to insert gear3 into shaft3", "name": "target: is_inserted_to(gear3, shaft3)"}} -{"target": "target: is_inserted_to(gear1, shaft1)", "initial_world_state": {"objects": [{"name": "clampgripper", "properties": ["is_empty"]}, {"name": "parallelgripper", "properties": ["is_empty"]}, {"name": "inwardgripper", "properties": ["is_empty"]}, {"name": "outwardgripper", "properties": ["is_empty"]}, {"name": "defaultgripper", "properties": ["is_empty"]}, {"name": "gear1", "properties": []}, {"name": "gear2", "properties": []}, {"name": "gear3", "properties": []}, {"name": "shaft1", "properties": []}, {"name": "shaft2", "properties": []}, {"name": "shaft3", "properties": []}, {"name": "gearbase", "properties": []}, {"name": "gearbase_hole1", "properties": []}, {"name": "gearbase_hole3", "properties": []}, {"name": "lampbase", "properties": []}, {"name": "chair_leg1", "properties": []}, {"name": "left_hand", "properties": []}], "constraints": [{"source": "gear2", "name": "can_insert_to", "target": "shaft2"}, {"source": "outwardgripper", "name": "can_manipulate", "target": "gear3"}, {"source": "outwardgripper", "name": "can_manipulate", "target": "gear2"}, {"source": "gear3", "name": "can_insert_to", "target": "shaft3"}, {"source": "shaft3", "name": "can_insert_to", "target": "gearbase_hole3"}, {"source": "shaft1", "name": "can_insert_to", "target": "gearbase_hole1"}, {"source": "parallelgripper", "name": "can_manipulate", "target": "gear1"}, {"source": "clampgripper", "name": "can_manipulate", "target": "shaft1"}, {"source": "gear1", "name": "can_insert_to", "target": "shaft1"}, {"source": "defaultgripper", "name": "can_manipulate", "target": "shaft3"}], "relations": [{"source": "shaft3", "name": "is_inserted_to", "target": "gearbase_hole3"}, {"source": "shaft1", "name": "is_inserted_to", "target": "gearbase_hole1"}, {"source": "left_hand", "name": "hold", "target": "inwardgripper"}, {"source": "gear2", "name": "is_inserted_to", "target": "shaft2"}]}, "result": {"summary": "selector to insert the gear1 into the shaft1 with the parallelgripper in the left_hand", "name": "selector: insert(left_hand, parallelgripper, gear1, shaft1)", "children": [{"summary": "the target is to make the gear1 be inserted into the shaft1", "name": "target: is_inserted_to(gear1, shaft1)"}, {"summary": "sequence to insert the gear1 into the shaft1 with the parallelgripper in the left_hand", "name": "sequence: insert(left_hand, parallelgripper, gear1, shaft1)", "children": [{"summary": "selector to change tool in left_hand from inwardgripper to parallelgripper", "name": "selector: change_tool(left_hand, inwardgripper, parallelgripper)", "children": [{"summary": "the target is to make left_hand hold parallelgripper", "name": "target: hold(left_hand, parallelgripper)"}, {"summary": "sequence to change tool in left_hand from inwardgripper to parallelgripper", "name": "sequence: change_tool(left_hand, inwardgripper, parallelgripper)", "children": [{"summary": "a precondition is left_hand is holding inwardgripper", "name": "precondition: hold(left_hand, inwardgripper)"}, {"summary": "a precondition is inwardgripper is empty", "name": "precondition: is_empty(inwardgripper)"}, {"summary": "the action to change tool in left_hand from inwardgripper to parallelgripper", "name": "action: change_tool(left_hand, inwardgripper, parallelgripper)"}]}]}, {"summary": "selector to pick up the gear1 with the parallelgripper in the left_hand", "name": "selector: pick_up(left_hand, parallelgripper, gear1)", "children": [{"summary": "the target is to make the parallelgripper hold the gear1", "name": "target: hold(parallelgripper, gear1)"}, {"summary": "sequence to pick up the gear1 with the parallelgripper in the left_hand", "name": "sequence: pick_up(left_hand, parallelgripper, gear1)", "children": [{"summary": "a precondition is the parallelgripper is empty", "name": "precondition: is_empty(parallelgripper)"}, {"summary": "a precondition is the left_hand is holding parallelgripper", "name": "precondition: hold(left_hand, parallelgripper)"}, {"summary": "the action to pick up the gear1 with the parallelgripper in the left_hand", "name": "action: pick_up(left_hand, parallelgripper, gear1)"}]}]}, {"summary": "the action to insert the gear1 into the shaft1 with the parallelgripper in the left_hand", "name": "action: insert(left_hand, parallelgripper, gear1, shaft1)"}]}]}} diff --git a/experiments/cook/baseline_result_handle.py b/experiments/cook/baseline_result_handle.py deleted file mode 100644 index 43099b1d..00000000 --- a/experiments/cook/baseline_result_handle.py +++ /dev/null @@ -1,84 +0,0 @@ -import json -import os -import json -import os - -""" -methods to handle the results of the baseline method. -use this to generate the problem set from the baseline_result.jsonl file -""" - - -def remove_entities(): - file_dir = os.path.dirname(os.path.abspath(__file__)) - result_dir = os.path.join(file_dir, "baseline_result.jsonl") - - with open(result_dir, "r") as file: - results = file.read() - data = [json.loads(line) for line in results.splitlines()] - - for item in data: - ws = item["initial_world_state"] - objs: list[dict] = ws["objects"] - cstrs = ws["constraints"] - for obj in objs: - if obj["name"] in [ - "parallel_box1", - "parallel_box2", - "outward_claw", - "inward_claw", - "no_tool", - ]: - print(f"Removing {obj['name']}") - objs.remove(obj) - - for cstr in cstrs: - if cstr["source"] in [ - "parallel_box1", - "parallel_box2", - "outward_claw", - "inward_claw", - "no_tool", - ]: - cstrs.remove(cstr) - elif cstr["target"] in [ - "parallel_box1", - "parallel_box2", - "outward_claw", - "inward_claw", - "no_tool", - ]: - cstrs.remove(cstr) - else: - pass - - with open(result_dir, "w") as file: - for item in data: - file.write(json.dumps(item)) - file.write("\n") - - -def handle_baseline_results(): - file_dir = os.path.dirname(os.path.abspath(__file__)) - result_dir = os.path.join(file_dir, "baseline_result.jsonl") - - if not os.path.exists(os.path.join(file_dir, "problem_set")): - os.makedirs(os.path.join(file_dir, "problem_set")) - - with open(result_dir, "r") as file: - results = file.read() - data = [json.loads(line) for line in results.splitlines()] - - i = 0 - for item in data: - i_str = str(i).zfill(3) - entity_path = os.path.join(file_dir, "problem_set", f"problem_{i_str}.json") - with open(entity_path, "w") as file: - file.write(json.dumps(item, indent=4)) - i += 1 - - -if __name__ == "__main__": - handle_baseline_results() - # remove_entities() - pass diff --git a/experiments/cook/behavior_tree.json b/experiments/cook/behavior_tree.json deleted file mode 100644 index ba968b94..00000000 --- a/experiments/cook/behavior_tree.json +++ /dev/null @@ -1,232 +0,0 @@ -{ - "summary": "selector to insert shaft1 into gearbase_hole1", - "name": "selector: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)", - "identifier": 1, - "type_name": "selector", - "children": [ - { - "summary": "condition node to check if shaft1 is inserted to gearbase_hole1", - "name": "target: is_inserted_to(shaft1, gearbase_hole1)", - "identifier": 2, - "type_name": "condition", - "conditions": [ - { - "object_name": "shaft1", - "property_name": "is_inserted_to", - "property_value": "gearbase_hole1", - "status": true - } - ] - }, - { - "summary": "sequence to insert shaft1 into gearbase_hole1", - "name": "sequence: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)", - "identifier": 3, - "type_name": "sequence", - "children": [ - { - "summary": "selector to load parallel_box1", - "name": "selector: load_tool(left_hand, parallel_box1)", - "identifier": 4, - "type_name": "selector", - "children": [ - { - "summary": "condition node to check if left_hand holds parallel_box1", - "name": "target: hold(left_hand, parallel_box1)", - "identifier": 5, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "parallel_box1", - "status": true - } - ] - }, - { - "summary": "sequence to load parallel_box1", - "name": "sequence: load_tool(left_hand, parallel_box1)", - "identifier": 6, - "type_name": "sequence", - "children": [ - { - "summary": "condition node to check if parallel_box1 is equippable", - "name": "precondition: is_equippable(parallel_box1)", - "identifier": 7, - "type_name": "condition", - "conditions": [ - { - "object_name": "parallel_box1", - "property_name": "is_equippable", - "property_value": null, - "status": true - } - ] - }, - { - "summary": "condition node to check if left hand is free", - "name": "precondition: is_free(left_hand)", - "identifier": 8, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "is_free", - "property_value": null, - "status": true - } - ] - }, - { - "summary": "action node to equip parallel_box1 to left hand", - "name": "action: load_tool(left_hand, parallel_box1)", - "identifier": 9, - "type_name": "action", - "effects": [ - { - "object_name": "left_hand", - "property_name": "is_free", - "property_value": null, - "status": false - }, - { - "object_name": "parallel_box1", - "property_name": "is_equippable", - "property_value": null, - "status": false - }, - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "parallel_box1", - "status": true - } - ] - } - ] - } - ] - }, - { - "summary": "selector to use left_hand with parallel_box1 to pick up shaft1", - "name": "selector: pick_up(left_hand, parallel_box1, shaft1)", - "identifier": 10, - "type_name": "selector", - "children": [ - { - "summary": "condition node to check if parallel_box1 holds shaft1", - "name": "target: hold(parallel_box1, shaft1)", - "identifier": 11, - "type_name": "condition", - "conditions": [ - { - "object_name": "parallel_box1", - "property_name": "hold", - "property_value": "shaft1", - "status": true - } - ] - }, - { - "summary": "sequence to use left hand with parallel_box1 to pick up shaft1", - "name": "sequence: pick_up(left_hand, parallel_box1, shaft1)", - "identifier": 12, - "type_name": "sequence", - "children": [ - { - "summary": "condition node to check if parallel_box1 is free", - "name": "precondition: is_free(parallel_box1)", - "identifier": 13, - "type_name": "condition", - "conditions": [ - { - "object_name": "parallel_box1", - "property_name": "is_free", - "property_value": null, - "status": true - } - ] - }, - { - "summary": "condition node to check if left hand holds parallel_box1", - "name": "precondition: hold(left_hand, parallel_box1)", - "identifier": 14, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "parallel_box1", - "status": true - } - ] - }, - { - "summary": "action node to use left hand with parallel_box1 to pick up shaft1", - "name": "action: pick_up(left_hand, parallel_box1, shaft1)", - "identifier": 15, - "type_name": "action", - "effects": [ - { - "object_name": "parallel_box1", - "property_name": "hold", - "property_value": "shaft1", - "status": true - }, - { - "object_name": "parallel_box1", - "property_name": "is_free", - "property_value": null, - "status": false - } - ] - } - ] - } - ] - }, - { - "summary": "condition node to check if shaft1 can be inserted to gearbase_hole1", - "name": "precondition: can_insert_to(shaft1, gearbase_hole1)", - "identifier": 16, - "type_name": "condition", - "conditions": [ - { - "object_name": "shaft1", - "property_name": "can_insert_to", - "property_value": "gearbase_hole1", - "status": true - } - ] - }, - { - "summary": "action node to use left_hand with parallel_box1 to insert shaft1 to gearbase_hole1", - "name": "action: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)", - "identifier": 17, - "type_name": "action", - "effects": [ - { - "object_name": "parallel_box1", - "property_name": "hold", - "property_value": "shaft1", - "status": false - }, - { - "object_name": "parallel_box1", - "property_name": "is_free", - "property_value": null, - "status": true - }, - { - "object_name": "shaft1", - "property_name": "is_inserted_to", - "property_value": "gearbase_hole1", - "status": true - } - ] - } - ] - } - ] - } \ No newline at end of file diff --git a/experiments/cook/domain_knowledge.txt b/experiments/cook/domain_knowledge.txt deleted file mode 100644 index ba5c088b..00000000 --- a/experiments/cook/domain_knowledge.txt +++ /dev/null @@ -1,88 +0,0 @@ -[user] -Following is the domain knowledge you need in pddl domain format, which includes: -- the object types in the world, -- the predicates for describing the states of the world, -- the actions that the robot can apply, with their precondtions and effects defined. -""" -(define (domain robot_assembly_problem-domain) - (:requirements :strips :typing) - (:types - thing - object - tool part hand - thing - ) - (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part)) - (:action pick_up - :parameters ( ?hand - hand ?tool - tool ?part - part) - :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part)) - :effect (and (hold ?tool ?part) (not (is_free ?tool)))) - (:action put_down - :parameters ( ?hand - hand ?tool - tool ?part - part) - :precondition (and (hold ?tool ?part) (hold ?hand ?tool)) - :effect (and (not (hold ?hand ?part)) (is_free ?tool))) - (:action place - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2))) - (:action detach - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2)))) - (:action insert - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2))) - (:action pull - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2)))) - (:action screw - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2))) - (:action unscrew - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2)))) - (:action load_tool - :parameters ( ?hand - hand ?tool - tool) - :precondition (and (is_equippable ?tool) (is_free ?hand)) - :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool))) - (:action unload_tool - :parameters ( ?hand - hand ?tool - tool) - :precondition (and (hold ?hand ?tool) (is_free ?tool)) - :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool)))) -) -""" -The precidates are explained as follows: -""" -"PREDICATE_LIST" -- is_free( or ): or is free and can hold something. (default: true) -- is_equippable(): is ready to be eqquipped by a robot hand. (default: true) -- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false) -- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false) -- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false) -- can_place_to(, ): can be placed into (placement assembly constraint) (default: false) -- hold(, ): is holding (default: false) -- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false) -- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false) -- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false) -""" - -The actions are explained as follows: -""" -"ROBOT_ACTION_LIST" -- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something. -- put_down(, , ): use to put down . As preconditions, should be holding and should be holding . -- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding . -- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into . -- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding . -- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into . -- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding . -- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into . -- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free. -- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free. -""" -the items in <> indicate the type of the instances needed for the predicates and actions. - -[assistant] -Understood. I will wait for further instructions before starting to work. diff --git a/experiments/cook/end2end_test.py b/experiments/cook/end2end_test.py deleted file mode 100644 index 5cbf08d8..00000000 --- a/experiments/cook/end2end_test.py +++ /dev/null @@ -1,227 +0,0 @@ -import json -import os -from pprint import pprint -from typing import List, Tuple, Annotated, TypedDict - - -""" -unit tree generation -""" - -os.environ["LANGCHAIN_TRACING_V2"] = "true" -os.environ["LANGCHAIN_ENDPOINT"] = "https://api.smith.langchain.com" -os.environ["LANGCHAIN_PROJECT"] = "kios_e2e" - -from kios_bt.bt_stewardship import BehaviorTreeStewardship -from kios_scene.scene_factory import SceneFactory -from kios_bt.bt_factory import BehaviorTreeFactory -from kios_robot.robot_interface import RobotInterface -from kios_world.world_interface import WorldInterface - -from kios_agent.llm_supporter import KiosLLMSupporter -from kios_agent.data_types import KiosPromptSkeleton - -from dotenv import load_dotenv - -from langchain_openai import ChatOpenAI - -from langchain.chains.openai_functions import ( - create_structured_output_runnable, - create_openai_fn_runnable, -) -from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder -from langchain_core.output_parsers import JsonOutputParser, StrOutputParser - -from langchain.prompts.pipeline import PipelinePromptTemplate -from langchain.prompts.prompt import PromptTemplate - -from langchain_community.document_loaders import TextLoader - -from langsmith import traceable - -load_dotenv() - -from kios_utils.pybt_test import generate_bt_stewardship, render_dot_tree -from kios_utils.pddl_problem_parser import parse_problem_init, parse_problem_objects - - -def render_bt(bt_json: dict): - test_class = BehaviorTreeFactory() - bt = test_class.from_json_to_simple_bt(bt_json) - # bt = test_class.from_json_to_tree_root(bt_json) - bt_stewardship = generate_bt_stewardship(bt) - # bt_stewardship.setup(timeout=15) - render_dot_tree(bt_stewardship) - - -def get_problem(problem_id: int) -> dict: - file_dir = os.path.dirname(os.path.abspath(__file__)) - problem_dir = os.path.join( - file_dir, "problem_set", f"problem_{str(problem_id).zfill(3)}.json" - ) - with open(problem_dir, "r") as file: - return json.load(file) - - -####################### dirs -current_dir = os.path.dirname(os.path.abspath(__file__)) -scene_path = os.path.join(current_dir, "scene.json") -# bt_json_file_path = os.path.join(current_dir, "behavior_tree.json") -world_state_path = os.path.join(current_dir, "world_state.json") -problem_path = os.path.join(current_dir, "gearset.problem") -domain_knowledge_path = os.path.join(current_dir, "domain_knowledge.txt") - -####################### problem -with open(problem_path, "r") as file: - problem = file.read() - -####################### scene -with open(scene_path, "r") as file: - scene_json_object = json.load(file) - -scene = SceneFactory().create_scene_from_json(scene_json_object) - -####################### world -world_interface = WorldInterface() -with open(world_state_path, "r") as file: - world_state_json = json.load(file) - world_interface.load_world_from_json(world_state_json) - -####################### robot -robot_interface = RobotInterface( - robot_address="127.0.0.1", - robot_port=12000, -) -robot_interface.setup_scene(scene) - -####################### bt_factory -bt_factory = BehaviorTreeFactory( - world_interface=world_interface, - robot_interface=robot_interface, -) - -####################### behavior_tree_stewardship -behavior_tree_stewardship = BehaviorTreeStewardship( - behaviortree_factory=bt_factory, - world_interface=world_interface, - robot_interface=robot_interface, -) - -# * kios data prompt skeleton dir -data_dir = os.environ.get("KIOS_DATA_DIR").format(username=os.getlogin()) -print(data_dir) -prompt_sk_dir = os.path.join(data_dir, "prompt_skeletons") -prompt_dir = os.path.join(data_dir, "prompts") - -# * end 2 end bt generation ppl -system_file = os.path.join(prompt_dir, "end_to_end_v3/system.txt") -task_file = os.path.join(prompt_dir, "end_to_end_v3/task.txt") -domain_file = os.path.join(prompt_dir, "end_to_end_v3/domain.txt") -behaviortree_file = os.path.join(prompt_dir, "end_to_end_v3/behaviortree.txt") -example_file = os.path.join(prompt_dir, "end_to_end_v3/example.txt") -output_format_file = os.path.join(prompt_dir, "end_to_end_v3/output_format.txt") -state_file = os.path.join(prompt_dir, "end_to_end_v3/state.txt") -template_file = os.path.join(prompt_dir, "end_to_end_v3/template.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) -with open(state_file, "r") as f: - state_ppt = PromptTemplate.from_template(f.read()) -with open(output_format_file, "r") as f: - output_format_ppt = PromptTemplate.from_template(f.read()) -with open(behaviortree_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - behaviortree_ppt = ppt_tmp.partial(input=f.read()) -with open(example_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - example_ppt = ppt_tmp.partial(input=f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {state} - - {behaviortree} - - {output_format} - - {example} - - {template} - - {format_instructions} - """ -) - -parser = JsonOutputParser() - -format_instructions = PromptTemplate.from_template("""{input}""").partial( - input=parser.get_format_instructions() -) - -e2e_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ("behaviortree", behaviortree_ppt), - ("output_format", output_format_ppt), - ("example", example_ppt), - ("state", state_ppt), - ("format_instructions", format_instructions), - ], -) - -e2e_chain = ( - e2e_ppt_ppl - # | ChatOpenAI(model="gpt-3.5-turbo-0125", temperature=0) - # | ChatOpenAI(model="ft:gpt-3.5-turbo-0125:kifabrik-mirmi::8y1cXwVw", temperature=0) - # | ChatOpenAI( - # model="ft:gpt-3.5-turbo-0125:kifabrik-mirmi:kios-ut-gen-v2:8z2KbPsr", - # temperature=0, - # ) - | ChatOpenAI(model="gpt-4", temperature=0) - | JsonOutputParser() -) - - -@traceable(name="e2e_test_baselines") -def e2e_test(problem_id: int): - task = get_problem(problem_id) - bt = e2e_chain.invoke( - { - "target": task["target"], - "initial_state": task["initial_world_state"], - } - ) - - result, node = behavior_tree_stewardship.sk_sim_run( - task["initial_world_state"], - bt.get("behavior_tree"), - ) - - pprint(result.to_json()) - pprint(f'LLM thought flow: {bt["thought"]}') - pprint(f'the action sequence: {bt["action_sequence"]}') - render_bt(bt.get("behavior_tree")) - - -# def e2e_test_baselines(): -# for i in range(17): -# e2e_test(i) - - -if __name__ == "__main__": - pass - # e2e_test_baselines() diff --git a/experiments/cook/gearset.domain b/experiments/cook/gearset.domain deleted file mode 100644 index 39c677b0..00000000 --- a/experiments/cook/gearset.domain +++ /dev/null @@ -1,48 +0,0 @@ -(define (domain robot_assembly_problem-domain) - (:requirements :strips :typing) - (:types - thing - object - tool part hand - thing - ) - (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part)) - (:action pick_up - :parameters ( ?hand - hand ?tool - tool ?part - part) - :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part)) - :effect (and (hold ?tool ?part) (not (is_free ?tool)))) - (:action put_down - :parameters ( ?hand - hand ?tool - tool ?part - part) - :precondition (and (hold ?tool ?part) (hold ?hand ?tool)) - :effect (and (not (hold ?hand ?part)) (is_free ?tool))) - (:action place - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2))) - (:action detach - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2)))) - (:action insert - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2))) - (:action pull - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2)))) - (:action screw - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2))) - (:action unscrew - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2)))) - (:action load_tool - :parameters ( ?hand - hand ?tool - tool) - :precondition (and (is_equippable ?tool) (is_free ?hand)) - :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool))) - (:action unload_tool - :parameters ( ?hand - hand ?tool - tool) - :precondition (and (hold ?hand ?tool) (is_free ?tool)) - :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool)))) -) \ No newline at end of file diff --git a/experiments/cook/gearset.problem b/experiments/cook/gearset.problem deleted file mode 100644 index 7e1cecc9..00000000 --- a/experiments/cook/gearset.problem +++ /dev/null @@ -1,10 +0,0 @@ -(define (problem robot_assembly_problem-problem) - (:domain robot_assembly_problem-domain) - (:objects - parallel_box1 parallel_box2 inward_claw outward_claw no_tool - tool - gear1 gear2 gear3 shaft1 shaft2 shaft3 gearbase gearbase_hole1 gearbase_hole3 - part - left_hand - hand - ) - (:init (can_manipulate parallel_box2 gear1) (can_manipulate outward_claw gear2) (can_manipulate outward_claw gear3) (can_manipulate no_tool shaft3) (can_manipulate parallel_box1 shaft1) (can_insert_to shaft1 gearbase_hole1) (can_insert_to shaft3 gearbase_hole3) (can_insert_to gear3 shaft3) (can_insert_to gear2 shaft2) (can_insert_to gear1 shaft1)) - (:goal (and )) - ) \ No newline at end of file diff --git a/experiments/cook/gearset1_graph copy.py b/experiments/cook/gearset1_graph copy.py deleted file mode 100644 index 71cbe082..00000000 --- a/experiments/cook/gearset1_graph copy.py +++ /dev/null @@ -1,526 +0,0 @@ -import json -import os - -""" -don't use this. it is just a backup. -""" - -from kios_bt.bt_stewardship import BehaviorTreeStewardship -from kios_scene.scene_factory import SceneFactory -from kios_bt.bt_factory import BehaviorTreeFactory -from kios_robot.robot_interface import RobotInterface -from kios_world.world_interface import WorldInterface - -from kios_agent.llm_supporter import KiosLLMSupporter -from kios_agent.data_types import KiosPromptSkeleton - -from kios_agent.kios_tools import ( - BehaviorTreeExecutorTool, - BehaviorTreeSimulatorTool, - WorldStateQueryTool, -) - -from langchain_core.pydantic_v1 import BaseModel, Field -from typing import List, Tuple, Annotated, TypedDict -import operator - -from dotenv import load_dotenv - -from langchain import hub -from langchain.agents import create_openai_functions_agent, AgentExecutor -from langchain_openai import ChatOpenAI -from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder -from langchain_core.output_parsers import JsonOutputParser, StrOutputParser - -from langchain.chains.openai_functions import ( - create_structured_output_runnable, - create_openai_fn_runnable, -) -from langchain_core.utils.function_calling import convert_to_openai_function - -from langchain.agents.format_scratchpad import format_to_openai_function_messages - -from langchain_core.agents import AgentActionMessageLog, AgentFinish - -from langgraph.graph import StateGraph, END - -from langchain_core.messages import HumanMessage - - -load_dotenv() - -from langsmith import traceable - -os.environ["LANGCHAIN_TRACING_V2"] = "true" -os.environ["LANGCHAIN_ENDPOINT"] = "https://api.smith.langchain.com" -os.environ["LANGCHAIN_PROJECT"] = "kios_agent" - - -####################### dirs -current_dir = os.path.dirname(os.path.abspath(__file__)) -scene_path = os.path.join(current_dir, "scene.json") -# bt_json_file_path = os.path.join(current_dir, "behavior_tree.json") -world_state_path = os.path.join(current_dir, "world_state.json") -problem_path = os.path.join(current_dir, "gearset.problem") - - -####################### problem -with open(problem_path, "r") as file: - problem = file.read() - -####################### scene -with open(scene_path, "r") as file: - scene_json_object = json.load(file) - -scene = SceneFactory().create_scene_from_json(scene_json_object) - -####################### world -world_interface = WorldInterface() -with open(world_state_path, "r") as file: - world_state_json_object = json.load(file) - world_interface.load_world_from_json(world_state_json_object) - -####################### robot -robot_interface = RobotInterface( - robot_address="127.0.0.1", - robot_port=12000, -) -robot_interface.setup_scene(scene) - -####################### bt_factory -bt_factory = BehaviorTreeFactory( - world_interface=world_interface, - robot_interface=robot_interface, -) - -####################### behavior_tree_stewardship -behavior_tree_stewardship = BehaviorTreeStewardship( - behaviortree_factory=bt_factory, - world_interface=world_interface, - robot_interface=robot_interface, -) - -####################### * tools -# bt_executor_tool = BehaviorTreeExecutorTool(behavior_tree_stewardship) - -bt_sim_tool = BehaviorTreeSimulatorTool(metadata={"bt_stw": behavior_tree_stewardship}) - -# world_state_query_tool = WorldStateQueryTool(behavior_tree_stewardship) - -executor_tools = [bt_sim_tool] - -# * kios data prompt skeleton dir -data_dir = os.environ.get("KIOS_DATA_DIR").format(username=os.getlogin()) -print(data_dir) -prompt_sk_dir = os.path.join(data_dir, "prompt_skeletons") - -# * refiner chain -llm_spt_re_sk = KiosLLMSupporter() -with open(os.path.join(prompt_sk_dir, "skeleton_refiner.json"), "r") as f: - re_sk_pptsk_json = json.load(f) - re_sk_pptsk = KiosPromptSkeleton.from_json(re_sk_pptsk_json) - llm_spt_re_sk.initialize_from_prompt_skeleton(re_sk_pptsk) -re_sk_ppt = llm_spt_re_sk.create_prompt() -re_sk_llm = ChatOpenAI(model_name=llm_spt_re_sk.model_name, temperature=0) - -re_sk_chain = re_sk_ppt | re_sk_llm | JsonOutputParser() - - -# * skeleton generator chain -llm_spt_skeleton_generator = KiosLLMSupporter() -with open(os.path.join(prompt_sk_dir, "skeleton_generator.json"), "r") as f: - skeleton_generator_pptsk_json = json.load(f) - skeleton_generator_pptsk = KiosPromptSkeleton.from_json( - skeleton_generator_pptsk_json - ) - llm_spt_skeleton_generator.initialize_from_prompt_skeleton(skeleton_generator_pptsk) -skeleton_generator_ppt = llm_spt_skeleton_generator.create_prompt() -skeleton_generator_llm = ChatOpenAI( - model_name=llm_spt_skeleton_generator.model_name, temperature=0 -) - -skeleton_generator_chain = ( - skeleton_generator_ppt - | skeleton_generator_llm - # | StrOutputParser() - | JsonOutputParser() -) - -# ! DISCARDED behavior tree generator chain -# bt_gen_chain = skeleton_generator_chain | re_sk_chain - -############################################### * behavior tree executor agent -# ! NOW SUBSTITUTED BY THE FAKE EXECUTOR FUNCTION -# bt_exe_ppt = hub.pull("hwchase17/openai-functions-agent") -# bt_exe_ppt = ChatPromptTemplate.from_messages( -# [ -# ("system", "You are a helpful assistant"), -# ("user", "The behavior tree is: {behavior_tree}"), -# ("user", "The world state is: {world_state}"), -# MessagesPlaceholder(variable_name="agent_scratchpad"), -# ] -# ) - -# bt_exe_llm = ChatOpenAI(model="gpt-3.5-turbo", temperature=0) - - -# def executor_agent_parse(output): -# # If no function was invoked, return to user -# if "function_call" not in output.additional_kwargs: -# return AgentFinish(return_values={"output": output.content}, log=output.content) - -# # Parse out the function call -# function_call = output.additional_kwargs["function_call"] -# name = function_call["name"] -# inputs = json.loads(function_call["arguments"]) - -# # If the Response function was invoked, return to the user with the function inputs -# if name == "ExecutorResponse": -# return AgentFinish(return_values=inputs, log=str(function_call)) -# # Otherwise, return an agent action -# else: -# return AgentActionMessageLog( -# tool=name, tool_input=inputs, log="", message_log=[output] -# ) - - -# class ExecutorResponse(BaseModel): -# """the result of the behavior tree execution""" - -# hasSucceeded: bool = Field(description="If the behavior tree has succeeded or not") -# world_state: dict[str, list[dict[str, str]]] = Field( -# description="The final world state after the execution" -# ) - - -# bt_exe_llm_with_tools = bt_exe_llm.bind_functions( -# [convert_to_openai_function(t) for t in executor_tools] -# ) - - -# ! BUG -# bt_exe_agent = ( -# { -# "input": lambda x: x["input"], -# # Format agent scratchpad from intermediate steps -# "agent_scratchpad": lambda x: format_to_openai_function_messages( -# x["intermediate_steps"] -# ), -# } -# | bt_exe_ppt -# | bt_exe_llm_with_tools -# | executor_agent_parse -# ) - -# bt_exe_agent_executor = AgentExecutor( -# tools=executor_tools, agent=bt_exe_agent, verbose=True -# ) - - -# * the graph state -class PlanExecuteState(TypedDict): - plan: List[str] - behavior_tree_skeleton: dict # ! not sure if use this or not. - behavior_tree: dict - world_state: Annotated[ - List[dict], operator.add - ] # ! to add, you need to make world_state a list of dict - # = Field(default=None) - past_steps: Annotated[List[Tuple], operator.add] - response: str # ! not sure if use this or not. - # to_user: bool - user_input: str - problem: str - - -##################################################### * planner -# * output schema of the planner -class Plan(BaseModel): - """Plan to follow in future""" - - steps: List[str] = Field( - description="a list of different steps to follow, should be in sorted order" - ) - - -planner_prompt = ChatPromptTemplate.from_template( - """ - The user input should be an instruction (or several instructions) of the robot assembly task. \ - For the given user input, summarize and come up with a bullet list task plan. \ -This plan should involve individual assembly step(s) of robot assembly task.\ -Do not add any superfluous steps. \ -Make sure that each step has all the information needed - do not skip steps.\ - -You should only use assembly actions as the action in the steps. -The available assembly actions are:\ -- insert something into something\ -- place something onto something\ -- screw something into something\ - -Example:\ -user_input: "you should insert the red block into the blue hole"\ -plan: "insert the red block into the blue hole"\ - -user_input: "intert the blub into the base, then put the cover onto it."\ -plan: ["insert the blub into the base", "put the cover onto the blub"]\ - -world_state: {world_state}\ -user_input: {user_input}\ -""" -) -planner = create_structured_output_runnable( - Plan, ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), planner_prompt -) - - -##################################################### * plan_updaterner - - -class UpdaterResponse(BaseModel): - """used to response to the user for asking for more inputs.""" - - response: str - - -plan_updater_prompt = ChatPromptTemplate.from_template( - """For the given objective, come up with a simple step by step plan. \ -This plan should involve individual tasks, that if executed correctly will yield the correct answer. Do not add any superfluous steps. \ -The result of the final step should be the final answer. Make sure that each step has all the information needed - do not skip steps. - -Your objective was this: -{user_input} - -Your original plan was this: -{plan} - -You have currently done the follow steps: -{past_steps} - -The world state is: -{world_state} - -Read the world state. Check the original plan from the first step and update your plan with the steps need to be conducted next according to the world state.\ -If no more steps are needed to achieve the objective defined by user input, respond to the user for more input. Otherwise, fill out the plan.\ -Only add steps to the plan that still NEED to be done. Do not return previously done steps as part of the plan.""" -) - -plan_updater = create_openai_fn_runnable( - [Plan, UpdaterResponse], # * here two schemas are used - ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), - plan_updater_prompt, -) - - -##################################################### * graph node functions -async def user_input_step(state: PlanExecuteState): - """ - get the input from the user - """ - print(f"-----user_input_step-----") - if state["response"] is not None: - user_input = input(state["response"]) - state["response"] = None - else: - user_input = input("Your next instruction: ") - - return { - "user_input": user_input, - } - - -def user_input_should_end(state: PlanExecuteState): - """ - if the user input is empty, then end - """ - print(f"-----user_input_should_end-----") - - # print(f'user_input: {state["user_input"]}') - # print(f'world_state: {state["world_state"]}') - - if not state["user_input"] or state["user_input"] == "": - return True - else: - return False - - -async def behavior_tree_generate_step(state: PlanExecuteState): - """ - generate the behavior tree based on the instruction - """ - print(f"-----behavior_tree_generate_step-----") - - instruction = state["plan"][0] - latest_world_state = state["world_state"][-1] - - skeleton = await skeleton_generator_chain.ainvoke( - { - "problem": state["problem"], - "world_state": latest_world_state, - "instructions": instruction, - } - ) - - response = await re_sk_chain.ainvoke( - { - "input": skeleton, - } - ) - - behavior_tree = response.get("task_plan").get("behavior_tree") - - init_world_state = [response.get("initial_state")] - - return { - "behavior_tree": behavior_tree, - "world_state": init_world_state, # * this is necessary because the constraints in problem - } - - -async def behavior_tree_execute_step(state: PlanExecuteState): - """ - execute the first step of the plan, append the result to the past steps - """ - print(f"-----behavior_tree_execute_step-----") - - behavior_tree = state["behavior_tree"] - latest_world_state = state["world_state"][-1] - - # exe_input = { - # "behavior_tree": behavior_tree, - # "world_state": latest_world_state, - # } - - # agent_response = await bt_exe_agent_executor.ainvoke( - # { - # "input": exe_input, - # } - # ) - - response = behavior_tree_stewardship.fake_run( - world_state=latest_world_state, bt_json=behavior_tree - ) - - # hasSucceeded = response["hasSucceeded"] - # new_world_state = [response["world_state"]] - hasSucceeded = response.result - new_world_state = [response.world_state] - the_step = state["plan"][0] - - return { - "past_steps": (the_step, hasSucceeded), # ! why here not a list of tuple? - "world_state": new_world_state, - } - - -async def plan_step(state: PlanExecuteState): - """ - plan the steps based on user input and world state - """ - print(f"-----plan_step-----") - - plan = await planner.ainvoke( - { - "user_input": state["user_input"], - "world_state": state["world_state"], - } - ) - return {"plan": plan.steps} - - -async def plan_updater_step(state: PlanExecuteState): - """ - if return a response, then success, response the use, end. - otherwise, return the updated newplan (normally the same as the old plan with the first step popped out. - """ - print(f"-----plan_updater_step-----") - - output = await plan_updater.ainvoke(state) - if isinstance( - output, UpdaterResponse - ): # * determine if it is time to response and end - return { - "response": "Your last instruction has been finished." - } # * Don't need to update. - else: - return {"plan": output.steps} # * update the plan - - -def plan_updater_should_end(state: PlanExecuteState): - """ - end router - """ - print(f"-----plan_updater_should_end-----") - - if state["response"]: - return True - else: - return False - - -##################################################### * construct the graph - -workflow = StateGraph(PlanExecuteState) - -workflow.add_node("planner", plan_step) - -workflow.add_node("behavior_tree_generator", behavior_tree_generate_step) - -workflow.add_node("behavior_tree_executor", behavior_tree_execute_step) - -workflow.add_node("plan_updater", plan_updater_step) - -workflow.add_node("user_input_node", user_input_step) - -workflow.set_entry_point("user_input_node") - -# workflow.add_edge("user_input_node", "planner") - -workflow.add_edge( - "planner", "behavior_tree_generator" -) # ! may need to add a condition here - -workflow.add_edge("behavior_tree_generator", "behavior_tree_executor") - -workflow.add_edge("behavior_tree_executor", "plan_updater") - -workflow.add_conditional_edges( - "plan_updater", - plan_updater_should_end, - { - True: "user_input_node", - False: "behavior_tree_generator", - }, -) - -workflow.add_conditional_edges( - "user_input_node", - user_input_should_end, - { - True: END, - False: "planner", - }, -) - -app = workflow.compile() - - -config = {"recursion_limit": 50} - - -inputs = { - "world_state": [world_state_json_object], - "problem": problem, -} - - -async def core_run(): - async for event in app.astream(inputs, config=config): - for k, v in event.items(): - if k != "__end__": - print(v) - - -if __name__ == "__main__": - import asyncio - - asyncio.run(core_run()) diff --git a/experiments/cook/gearset1_graph.py b/experiments/cook/gearset1_graph.py deleted file mode 100644 index b2d08f69..00000000 --- a/experiments/cook/gearset1_graph.py +++ /dev/null @@ -1,509 +0,0 @@ -import json -import os - -os.environ["LANGCHAIN_TRACING_V2"] = "true" -os.environ["LANGCHAIN_ENDPOINT"] = "https://api.smith.langchain.com" -os.environ["LANGCHAIN_PROJECT"] = "kios_agent" - -from kios_bt.bt_stewardship import BehaviorTreeStewardship -from kios_scene.scene_factory import SceneFactory -from kios_bt.bt_factory import BehaviorTreeFactory -from kios_robot.robot_interface import RobotInterface -from kios_world.world_interface import WorldInterface - -from kios_agent.llm_supporter import KiosLLMSupporter -from kios_agent.data_types import KiosPromptSkeleton - -from kios_agent.kios_tools import ( - BehaviorTreeExecutorTool, - BehaviorTreeSimulatorTool, - WorldStateQueryTool, -) - -from typing import List, Tuple, Annotated, TypedDict -import operator - -from dotenv import load_dotenv - -from langchain import hub - -from langchain_openai import ChatOpenAI - -from langchain.chains.openai_functions import ( - create_structured_output_runnable, - create_openai_fn_runnable, -) -from langchain_core.utils.function_calling import convert_to_openai_function -from langchain_core.agents import AgentActionMessageLog, AgentFinish -from langchain_core.messages import HumanMessage -from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder -from langchain_core.output_parsers import JsonOutputParser, StrOutputParser -from langchain_core.pydantic_v1 import BaseModel, Field - -from langchain.agents.format_scratchpad import format_to_openai_function_messages -from langchain.agents import create_openai_functions_agent, AgentExecutor - -from langchain.prompts.pipeline import PipelinePromptTemplate -from langchain.prompts.prompt import PromptTemplate - -from langgraph.graph import StateGraph, END - -from langsmith import traceable - -load_dotenv() - -from kios_utils.pybt_test import generate_bt_stewardship, render_dot_tree - - -def test_bt(bt_json: json): - test_class = BehaviorTreeFactory() - bt = test_class.from_json_to_simple_bt(bt_json) - # bt = test_class.from_json_to_tree_root(bt_json) - bt_stewardship = generate_bt_stewardship(bt) - # bt_stewardship.setup(timeout=15) - render_dot_tree(bt_stewardship) - - -####################### dirs -current_dir = os.path.dirname(os.path.abspath(__file__)) -scene_path = os.path.join(current_dir, "scene.json") -# bt_json_file_path = os.path.join(current_dir, "behavior_tree.json") -world_state_path = os.path.join(current_dir, "world_state.json") -problem_path = os.path.join(current_dir, "gearset.problem") - - -####################### problem -with open(problem_path, "r") as file: - problem = file.read() - -####################### scene -with open(scene_path, "r") as file: - scene_json_object = json.load(file) - -scene = SceneFactory().create_scene_from_json(scene_json_object) - -####################### world -world_interface = WorldInterface() -with open(world_state_path, "r") as file: - world_state_json_object = json.load(file) - world_interface.load_world_from_json(world_state_json_object) - -####################### robot -robot_interface = RobotInterface( - robot_address="127.0.0.1", - robot_port=12000, -) -robot_interface.setup_scene(scene) - -####################### bt_factory -bt_factory = BehaviorTreeFactory( - world_interface=world_interface, - robot_interface=robot_interface, -) - -####################### behavior_tree_stewardship -behavior_tree_stewardship = BehaviorTreeStewardship( - behaviortree_factory=bt_factory, - world_interface=world_interface, - robot_interface=robot_interface, -) - -####################### * tools -# bt_executor_tool = BehaviorTreeExecutorTool(behavior_tree_stewardship) - -bt_sim_tool = BehaviorTreeSimulatorTool(metadata={"bt_stw": behavior_tree_stewardship}) - -# world_state_query_tool = WorldStateQueryTool(behavior_tree_stewardship) - -executor_tools = [bt_sim_tool] - -# * kios data prompt skeleton dir -data_dir = os.environ.get("KIOS_DATA_DIR").format(username=os.getlogin()) -print(data_dir) -prompt_sk_dir = os.path.join(data_dir, "prompt_skeletons") -prompt_dir = os.path.join(data_dir, "prompts") - -# * refiner chain -llm_spt_re_sk = KiosLLMSupporter() -with open(os.path.join(prompt_sk_dir, "skeleton_refiner.json"), "r") as f: - re_sk_pptsk_json = json.load(f) - re_sk_pptsk = KiosPromptSkeleton.from_json(re_sk_pptsk_json) - llm_spt_re_sk.initialize_from_prompt_skeleton(re_sk_pptsk) -re_sk_ppt = llm_spt_re_sk.create_prompt() -re_sk_llm = ChatOpenAI(model_name=llm_spt_re_sk.model_name, temperature=0) - -re_sk_chain = re_sk_ppt | re_sk_llm | JsonOutputParser() - - -# * skeleton generator chain -llm_spt_skeleton_generator = KiosLLMSupporter() -with open(os.path.join(prompt_sk_dir, "skeleton_generator.json"), "r") as f: - skeleton_generator_pptsk_json = json.load(f) - skeleton_generator_pptsk = KiosPromptSkeleton.from_json( - skeleton_generator_pptsk_json - ) - llm_spt_skeleton_generator.initialize_from_prompt_skeleton(skeleton_generator_pptsk) -skeleton_generator_ppt = llm_spt_skeleton_generator.create_prompt() -skeleton_generator_llm = ChatOpenAI( - model_name=llm_spt_skeleton_generator.model_name, temperature=0 -) - -skeleton_generator_chain = ( - skeleton_generator_ppt - | skeleton_generator_llm - # | StrOutputParser() - | JsonOutputParser() -) - - -# * the graph state -class PlanExecuteState(TypedDict): - plan: List[str] - behavior_tree_skeleton: dict # ! not sure if use this or not. - behavior_tree: dict - world_state: Annotated[ - List[dict], operator.add - ] # ! to add, you need to make world_state a list of dict - # = Field(default=None) - past_steps: Annotated[List[Tuple], operator.add] - response: str # ! not sure if use this or not. - # to_user: bool - user_input: str - problem: str - - -##################################################### * planner -# * output schema of the planner -class Plan(BaseModel): - """Plan to follow in future""" - - steps: List[str] = Field( - description="a list of different steps to follow, should be in sorted order" - ) - - -template_file = os.path.join(prompt_dir, "planner/template.txt") -task_file = os.path.join(prompt_dir, "planner/task.txt") -system_file = os.path.join(prompt_dir, "planner/system.txt") -domain_file = os.path.join(prompt_dir, "planner/domain.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {template} - """ -) - -planner_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ], -) - -planner = create_structured_output_runnable( - Plan, ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), planner_ppt_ppl -) - -##################################################### * plan_updaterner -class UpdaterResponse(BaseModel): - """used to response to the user for asking for more inputs.""" - - response: str - - -template_file = os.path.join(prompt_dir, "plan_updater/template.txt") -task_file = os.path.join(prompt_dir, "plan_updater/task.txt") -system_file = os.path.join(prompt_dir, "plan_updater/system.txt") -domain_file = os.path.join(prompt_dir, "plan_updater/domain.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {template} - """ -) - -plan_updater_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ], -) - -plan_updater = create_openai_fn_runnable( - [Plan, UpdaterResponse], # * here two schemas are used - ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), - plan_updater_ppt_ppl, -) - - -##################################################### * graph node functions -@traceable(name="user_input_node_step") -async def user_input_step(state: PlanExecuteState): - """ - get the input from the user - """ - print(f"-----user_input_step-----") - if state["response"] is not None: - user_input = input(state["response"]) - state["response"] = None - else: - user_input = input("Your next instruction: ") - - return { - "user_input": user_input, - } - - -@traceable(name="behavior_tree_generate_step") -async def behavior_tree_generate_step(state: PlanExecuteState): - """ - generate the behavior tree based on the instruction - """ - print(f"-----behavior_tree_generate_step-----") - - instruction = state["plan"][0] - latest_world_state = state["world_state"][-1] - - skeleton = await skeleton_generator_chain.ainvoke( - { - "problem": state["problem"], - "world_state": latest_world_state, - "instructions": instruction, - } - ) - - # sk_json = JsonOutputParser().invoke(skeleton) - sk_json = skeleton - - behavior_tree_sk = sk_json["task_plan"]["behavior_tree"] - - test_bt(behavior_tree_sk) - - pause = input("paused here") - - # ! OBJECT CONCENTRATION - - response = await re_sk_chain.ainvoke( - { - "input": skeleton, - } - ) - - behavior_tree = response.get("task_plan").get("behavior_tree") - - init_world_state = [response.get("initial_state")] - - return { - "behavior_tree": behavior_tree, - "world_state": init_world_state, # ! this is necessary because the constraints in problem - } - - -# ! BBWARN INITIALIZE THE CONSTRAINT IN THE PROMPT OR IN THE INITIALIZATION NODE - - -@traceable(name="behavior_tree_execute_step") -async def behavior_tree_execute_step(state: PlanExecuteState): - """ - execute the first step of the plan, append the result to the past steps - """ - print(f"-----behavior_tree_execute_step-----") - - behavior_tree = state["behavior_tree"] - latest_world_state = state["world_state"][-1] - - # exe_input = { - # "behavior_tree": behavior_tree, - # "world_state": latest_world_state, - # } - - # agent_response = await bt_exe_agent_executor.ainvoke( - # { - # "input": exe_input, - # } - # ) - - response = behavior_tree_stewardship.fake_run( - world_state=latest_world_state, bt_json=behavior_tree - ) - - # hasSucceeded = response["hasSucceeded"] - # new_world_state = [response["world_state"]] - hasSucceeded = response.result - new_world_state = [response.world_state] - the_step = state["plan"][0] - - return { - "past_steps": ( - the_step, - hasSucceeded, - ), # * only one because new plan will be generated and old steps are all removed - "world_state": new_world_state, - } - - -@traceable(name="planner_step") -async def planner_step(state: PlanExecuteState): - """ - plan the steps based on user input and world state - """ - print(f"-----plan_step-----") - - plan = await planner.ainvoke( - { - "user_input": state["user_input"], - "world_state": state["world_state"], - } - ) - return {"plan": plan.steps} - - -@traceable(name="plan_updater_step") -async def plan_updater_step(state: PlanExecuteState): - """ - if return a response, then success, response the use, end. - otherwise, return the updated newplan (normally the same as the old plan with the first step popped out. - """ - print(f"-----plan_updater_step-----") - - output = await plan_updater.ainvoke(state) - if isinstance( - output, UpdaterResponse - ): # * determine if it is time to response and end - return { - "response": "Your last instruction has been finished." - } # * Don't need to update. - else: - return {"plan": output.steps} # * update the plan - - -##################################################### * construct the graph - -workflow = StateGraph(PlanExecuteState) - -workflow.add_node("planner", planner_step) - -workflow.add_node("behavior_tree_generator", behavior_tree_generate_step) - -workflow.add_node("behavior_tree_executor", behavior_tree_execute_step) - -workflow.add_node("plan_updater", plan_updater_step) - -workflow.add_node("user_input_node", user_input_step) - -workflow.set_entry_point("user_input_node") - -# workflow.add_edge("user_input_node", "planner") - -workflow.add_edge( - "planner", "behavior_tree_generator" -) # ! may need to add a condition here - -workflow.add_edge("behavior_tree_generator", "behavior_tree_executor") - -workflow.add_edge("behavior_tree_executor", "plan_updater") - - -def plan_updater_should_end(state: PlanExecuteState): - """ - end router - """ - print(f"-----plan_updater_should_end-----") - - if state["response"]: - return True - else: - return False - - -workflow.add_conditional_edges( - "plan_updater", - plan_updater_should_end, - { - True: "user_input_node", - False: "behavior_tree_generator", - }, -) - - -def user_input_should_end(state: PlanExecuteState): - """ - if the user input is empty, then end - """ - print(f"-----user_input_should_end-----") - - # print(f'user_input: {state["user_input"]}') - # print(f'world_state: {state["world_state"]}') - - if not state["user_input"] or state["user_input"] == "": - return True - else: - return False - - -workflow.add_conditional_edges( - "user_input_node", - user_input_should_end, - { - True: END, - False: "planner", - }, -) - -app = workflow.compile() - - -# config = {"recursion_limit": 50} - -inputs = { - "world_state": [world_state_json_object], - "problem": problem, -} - - -async def core_run(): - async for event in app.astream( - inputs, - # config=config, - ): - for k, v in event.items(): - if k != "__end__": - print(v) - - -if __name__ == "__main__": - import asyncio - - asyncio.run(core_run()) diff --git a/experiments/cook/gearset1_graph_bu_txt.py b/experiments/cook/gearset1_graph_bu_txt.py deleted file mode 100644 index 9d78ee99..00000000 --- a/experiments/cook/gearset1_graph_bu_txt.py +++ /dev/null @@ -1,418 +0,0 @@ -import json -import os - -os.environ["LANGCHAIN_TRACING_V2"] = "true" -os.environ["LANGCHAIN_ENDPOINT"] = "https://api.smith.langchain.com" -os.environ["LANGCHAIN_PROJECT"] = "kios_agent" - -from kios_bt.bt_stewardship import BehaviorTreeStewardship -from kios_scene.scene_factory import SceneFactory -from kios_bt.bt_factory import BehaviorTreeFactory -from kios_robot.robot_interface import RobotInterface -from kios_world.world_interface import WorldInterface - -from kios_agent.llm_supporter import KiosLLMSupporter -from kios_agent.data_types import KiosPromptSkeleton - -from typing import List, Tuple, Annotated, TypedDict -import operator - -from dotenv import load_dotenv - -from langchain_openai import ChatOpenAI - -from langchain.chains.openai_functions import ( - create_structured_output_runnable, - create_openai_fn_runnable, -) - -from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder -from langchain_core.output_parsers import JsonOutputParser, StrOutputParser -from langchain_core.pydantic_v1 import BaseModel, Field - -from langchain.agents.format_scratchpad import format_to_openai_function_messages -from langchain.agents import create_openai_functions_agent, AgentExecutor - -from langchain.prompts.pipeline import PipelinePromptTemplate -from langchain.prompts.prompt import PromptTemplate - -from langgraph.graph import StateGraph, END - -from langsmith import traceable - -load_dotenv() - - -####################### dirs -current_dir = os.path.dirname(os.path.abspath(__file__)) -scene_path = os.path.join(current_dir, "scene.json") -# bt_json_file_path = os.path.join(current_dir, "behavior_tree.json") -world_state_path = os.path.join(current_dir, "world_state.json") -problem_path = os.path.join(current_dir, "gearset.problem") - - -####################### problem -with open(problem_path, "r") as file: - problem = file.read() - -####################### scene -with open(scene_path, "r") as file: - scene_json_object = json.load(file) - -scene = SceneFactory().create_scene_from_json(scene_json_object) - -####################### world -world_interface = WorldInterface() -with open(world_state_path, "r") as file: - world_state_json_object = json.load(file) - world_interface.load_world_from_json(world_state_json_object) - -####################### robot -robot_interface = RobotInterface( - robot_address="127.0.0.1", - robot_port=12000, -) -robot_interface.setup_scene(scene) - -####################### bt_factory -bt_factory = BehaviorTreeFactory( - world_interface=world_interface, - robot_interface=robot_interface, -) - -####################### behavior_tree_stewardship -behavior_tree_stewardship = BehaviorTreeStewardship( - behaviortree_factory=bt_factory, - world_interface=world_interface, - robot_interface=robot_interface, -) - -####################### * tools - -# * kios data prompt skeleton dir -data_dir = os.environ.get("KIOS_DATA_DIR").format(username=os.getlogin()) -print(data_dir) -prompt_sk_dir = os.path.join(data_dir, "prompt_skeletons") -prompt_dir = os.path.join(data_dir, "prompts") - -# * refiner chain -llm_spt_re_sk = KiosLLMSupporter() -with open(os.path.join(prompt_sk_dir, "skeleton_refiner.json"), "r") as f: - re_sk_pptsk_json = json.load(f) - re_sk_pptsk = KiosPromptSkeleton.from_json(re_sk_pptsk_json) - llm_spt_re_sk.initialize_from_prompt_skeleton(re_sk_pptsk) -re_sk_ppt = llm_spt_re_sk.create_prompt() -re_sk_llm = ChatOpenAI(model_name=llm_spt_re_sk.model_name, temperature=0) - -re_sk_chain = re_sk_ppt | re_sk_llm | JsonOutputParser() - - -# * skeleton generator chain -llm_spt_skeleton_generator = KiosLLMSupporter() -with open(os.path.join(prompt_sk_dir, "skeleton_generator.json"), "r") as f: - skeleton_generator_pptsk_json = json.load(f) - skeleton_generator_pptsk = KiosPromptSkeleton.from_json( - skeleton_generator_pptsk_json - ) - llm_spt_skeleton_generator.initialize_from_prompt_skeleton(skeleton_generator_pptsk) -skeleton_generator_ppt = llm_spt_skeleton_generator.create_prompt() -skeleton_generator_llm = ChatOpenAI( - model_name=llm_spt_skeleton_generator.model_name, temperature=0 -) - -skeleton_generator_chain = ( - skeleton_generator_ppt - | skeleton_generator_llm - # | StrOutputParser() - | JsonOutputParser() -) - - -# * the graph state -class PlanExecuteState(TypedDict): - plan: List[str] - behavior_tree_skeleton: dict # ! not sure if use this or not. - behavior_tree: dict - world_state: Annotated[ - List[dict], operator.add - ] # ! to add, you need to make world_state a list of dict - # = Field(default=None) - past_steps: Annotated[List[Tuple], operator.add] - response: str # ! not sure if use this or not. - # to_user: bool - user_input: str - problem: str - - -##################################################### * planner -# * output schema of the planner -class Plan(BaseModel): - """Plan to follow in future""" - - steps: List[str] = Field( - description="a list of different steps to follow, should be in sorted order" - ) - - -planner_prompt_file = os.path.join(prompt_dir, "planner/template.txt") - -with open(planner_prompt_file, "w") as f: - planner_prompt = f.read() - -planner = create_structured_output_runnable( - Plan, ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), planner_prompt -) - - -##################################################### * plan_updaterner - - -class UpdaterResponse(BaseModel): - """used to response to the user for asking for more inputs.""" - - response: str - - -plan_updater_prompt = ChatPromptTemplate.from_template( - """For the given objective, come up with a simple step by step plan. \ -This plan should involve individual tasks, that if executed correctly will yield the correct answer. Do not add any superfluous steps. \ -The result of the final step should be the final answer. Make sure that each step has all the information needed - do not skip steps. - -Your objective was this: -{user_input} - -Your original plan was this: -{plan} - -You have currently done the follow steps: -{past_steps} - -The world state is: -{world_state} - -Read the world state. Check the original plan from the first step and update your plan with the steps need to be conducted next according to the world state.\ -If no more steps are needed to achieve the objective defined by user input, respond to the user for more input. Otherwise, fill out the plan.\ -Only add steps to the plan that still NEED to be done. Do not return previously done steps as part of the plan.""" -) - -plan_updater = create_openai_fn_runnable( - [Plan, UpdaterResponse], # * here two schemas are used - ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), - plan_updater_prompt, -) - - -##################################################### * graph node functions -@traceable(name="user_input_node_step") -async def user_input_step(state: PlanExecuteState): - """ - get the input from the user - """ - print(f"-----user_input_step-----") - if state["response"] is not None: - user_input = input(state["response"]) - state["response"] = None - else: - user_input = input("Your next instruction: ") - - return { - "user_input": user_input, - } - - -@traceable(name="behavior_tree_generate_step") -async def behavior_tree_generate_step(state: PlanExecuteState): - """ - generate the behavior tree based on the instruction - """ - print(f"-----behavior_tree_generate_step-----") - - instruction = state["plan"][0] - latest_world_state = state["world_state"][-1] - - skeleton = await skeleton_generator_chain.ainvoke( - { - "problem": state["problem"], - "world_state": latest_world_state, - "instructions": instruction, - } - ) - - # ! OBJECT CONCENTRATION - - response = await re_sk_chain.ainvoke( - { - "input": skeleton, - } - ) - - behavior_tree = response.get("task_plan").get("behavior_tree") - - init_world_state = [response.get("initial_state")] - - return { - "behavior_tree": behavior_tree, - "world_state": init_world_state, # ! this is necessary because the constraints in problem - } - - -# ! BBWARN INITIALIZE THE CONSTRAINT IN THE PROMPT OR IN THE INITIALIZATION NODE - - -@traceable(name="behavior_tree_execute_step") -async def behavior_tree_execute_step(state: PlanExecuteState): - """ - execute the first step of the plan, append the result to the past steps - """ - print(f"-----behavior_tree_execute_step-----") - - behavior_tree = state["behavior_tree"] - latest_world_state = state["world_state"][-1] - - response = behavior_tree_stewardship.fake_run( - world_state=latest_world_state, bt_json=behavior_tree - ) - - hasSucceeded = response.result - new_world_state = [response.world_state] - the_step = state["plan"][0] - - return { - "past_steps": (the_step, hasSucceeded), - "world_state": new_world_state, - } - - -@traceable(name="planner_step") -async def planner_step(state: PlanExecuteState): - """ - plan the steps based on user input and world state - """ - print(f"-----plan_step-----") - - plan = await planner.ainvoke( - { - "user_input": state["user_input"], - "world_state": state["world_state"], - } - ) - return {"plan": plan.steps} - - -@traceable(name="plan_updater_step") -async def plan_updater_step(state: PlanExecuteState): - """ - if return a response, then success, response the use, end. - otherwise, return the updated newplan (normally the same as the old plan with the first step popped out. - """ - print(f"-----plan_updater_step-----") - - output = await plan_updater.ainvoke(state) - if isinstance( - output, UpdaterResponse - ): # * determine if it is time to response and end - return { - "response": "Your last instruction has been finished." - } # * Don't need to update. - else: - return {"plan": output.steps} # * update the plan - - -##################################################### * construct the graph - -workflow = StateGraph(PlanExecuteState) - -workflow.add_node("planner", planner_step) - -workflow.add_node("behavior_tree_generator", behavior_tree_generate_step) - -workflow.add_node("behavior_tree_executor", behavior_tree_execute_step) - -workflow.add_node("plan_updater", plan_updater_step) - -workflow.add_node("user_input_node", user_input_step) - -workflow.set_entry_point("user_input_node") - -# workflow.add_edge("user_input_node", "planner") - -workflow.add_edge( - "planner", "behavior_tree_generator" -) # ! may need to add a condition here - -workflow.add_edge("behavior_tree_generator", "behavior_tree_executor") - -workflow.add_edge("behavior_tree_executor", "plan_updater") - - -def plan_updater_should_end(state: PlanExecuteState): - """ - end router - """ - print(f"-----plan_updater_should_end-----") - - if state["response"]: - return True - else: - return False - - -workflow.add_conditional_edges( - "plan_updater", - plan_updater_should_end, - { - True: "user_input_node", - False: "behavior_tree_generator", - }, -) - - -def user_input_should_end(state: PlanExecuteState): - """ - if the user input is empty, then end - """ - print(f"-----user_input_should_end-----") - - # print(f'user_input: {state["user_input"]}') - # print(f'world_state: {state["world_state"]}') - - if not state["user_input"] or state["user_input"] == "": - return True - else: - return False - - -workflow.add_conditional_edges( - "user_input_node", - user_input_should_end, - { - True: END, - False: "planner", - }, -) - -app = workflow.compile() - - -# config = {"recursion_limit": 50} - -inputs = { - "world_state": [world_state_json_object], - "problem": problem, -} - - -async def core_run(): - async for event in app.astream( - inputs, - # config=config, - ): - for k, v in event.items(): - if k != "__end__": - print(v) - - -if __name__ == "__main__": - import asyncio - - asyncio.run(core_run()) diff --git a/experiments/cook/problem.py b/experiments/cook/problem.py deleted file mode 100644 index 2233e3f7..00000000 --- a/experiments/cook/problem.py +++ /dev/null @@ -1,111 +0,0 @@ -import re - -""" -script to parse the initial state from a problem file. -out-of-date. now pddl part is removed and this part should be no longer in use. -""" - - -def parse_init(problem): - init_pattern = r"\(:init\s+(.*?)\s+\)" - element_pattern = r"\((.*?)\)" - - init_match = re.search(init_pattern, problem, re.DOTALL) - if init_match: - init_content = init_match.group(1) - elements = re.findall(element_pattern, init_content) - - parsed_init = [] - for element in elements: - parts = element.split() - name = parts[0] - args = parts[1:] - parsed_init.append({"name": name, "args": args}) - - return parsed_init - else: - return [] - - -properties = [ - "is_free", - "is_equippable", -] - -constraints = [ - "can_manipulate", - "can_screw_to", - "can_insert_to", - "can_place_to", -] - -relations = [ - "hold", - "is_inserted_to", - "is_screwed_to", - "is_placed_to", -] - - -def classify_item(item_name): - if item_name in properties: - return "property" - elif item_name in constraints: - return "constraint" - elif item_name in relations: - return "relation" - else: - raise ValueError(f"Unknown item name: {item_name}") - - -# Usage example: -problem = """ -(define (problem robot_assembly_problem-problem) - (:domain robot_assembly_problem-domain) - (:objects - parallel_box1 parallel_box2 inward_claw outward_claw no_tool - tool - gear1 gear2 gear3 shaft1 shaft2 shaft3 gearbase gearbase_hole1 gearbase_hole3 - part - left_hand - hand - ) - (:init (can_manipulate parallel_box2 gear1) (can_manipulate outward_claw gear2) (can_manipulate outward_claw gear3) (can_manipulate no_tool shaft3) (can_manipulate parallel_box1 shaft1) (can_screw_to leg1 seat) (can_screw_to leg2 seat) (can_insert_to back seat) (can_screw_to nut1 seat) (can_screw_to nut2 seat) (can_screw_to blub lampbase) (can_place_to lamp blub) (can_insert_to shaft1 gearbase_hole1) (can_insert_to shaft3 gearbase_hole3) (can_insert_to gear3 shaft3) (can_insert_to gear2 shaft2) (can_insert_to gear1 shaft1)) - (:goal (and )) - )""" - -parsed_init = parse_init(problem) - -for item in parsed_init: - item_name = item.get("name") - item_type = classify_item(item_name) - - world_state = { - "objects": [], - "constraints": [], - "relations": [], - } - - if item_type == "property": - world_state["objects"].append( - {"name": item["args"][0], "properties": [item_name]} # ! list or item? - ) - - elif item_type == "constraint": - world_state["constraints"].append( - { - "source": item["args"][0], - "name": item_name, - "target": item["args"][1], - } - ) - elif item_type == "relation": - world_state["relations"].append( - { - "source": item["args"][0], - "name": item_name, - "target": item["args"][1], - } - ) - else: - pass - - -print(parsed_init) diff --git a/experiments/cook/rec_sk_gen.py b/experiments/cook/rec_sk_gen.py deleted file mode 100644 index 15a852bc..00000000 --- a/experiments/cook/rec_sk_gen.py +++ /dev/null @@ -1,706 +0,0 @@ -import json -import os -from pprint import pprint - -""" -generate the behavior tree iteratively. -""" - -os.environ["LANGCHAIN_TRACING_V2"] = "true" -os.environ["LANGCHAIN_ENDPOINT"] = "https://api.smith.langchain.com" -os.environ["LANGCHAIN_PROJECT"] = "kios_agent" - -from kios_bt.bt_stewardship import BehaviorTreeStewardship -from kios_scene.scene_factory import SceneFactory -from kios_bt.bt_factory import BehaviorTreeFactory -from kios_robot.robot_interface import RobotInterface -from kios_world.world_interface import WorldInterface - -from kios_agent.llm_supporter import KiosLLMSupporter -from kios_agent.data_types import KiosPromptSkeleton - -from kios_agent.kios_tools import ( - BehaviorTreeExecutorTool, - BehaviorTreeSimulatorTool, - WorldStateQueryTool, -) - -from typing import List, Tuple, Annotated, TypedDict -import operator - -from dotenv import load_dotenv - -from langchain import hub - -from langchain_openai import ChatOpenAI - -from langchain.chains.openai_functions import ( - create_structured_output_runnable, - create_openai_fn_runnable, -) -from langchain_core.utils.function_calling import convert_to_openai_function -from langchain_core.agents import AgentActionMessageLog, AgentFinish -from langchain_core.messages import HumanMessage -from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder -from langchain_core.output_parsers import JsonOutputParser, StrOutputParser -from langchain_core.pydantic_v1 import BaseModel, Field - -from langchain.agents.format_scratchpad import format_to_openai_function_messages -from langchain.agents import create_openai_functions_agent, AgentExecutor - -from langchain.prompts.pipeline import PipelinePromptTemplate -from langchain.prompts.prompt import PromptTemplate - -from langchain_community.document_loaders import TextLoader -from langchain_openai import OpenAIEmbeddings - -from langchain_community.vectorstores import FAISS -from langchain.text_splitter import ( - RecursiveCharacterTextSplitter, - CharacterTextSplitter, -) - -from langgraph.graph import StateGraph, END - -from langsmith import traceable - -load_dotenv() - -from kios_utils.pybt_test import generate_bt_stewardship, render_dot_tree -from kios_utils.pddl_problem_parser import parse_problem_init, parse_problem_objects - - -def render_bt(bt_json: json): - test_class = BehaviorTreeFactory() - bt = test_class.from_json_to_simple_bt(bt_json) - # bt = test_class.from_json_to_tree_root(bt_json) - bt_stewardship = generate_bt_stewardship(bt) - # bt_stewardship.setup(timeout=15) - render_dot_tree(bt_stewardship) - - -####################### dirs -current_dir = os.path.dirname(os.path.abspath(__file__)) -scene_path = os.path.join(current_dir, "scene.json") -# bt_json_file_path = os.path.join(current_dir, "behavior_tree.json") -world_state_path = os.path.join(current_dir, "world_state.json") -problem_path = os.path.join(current_dir, "gearset.problem") -domain_knowledge_path = os.path.join(current_dir, "domain_knowledge.txt") - -####################### problem -with open(problem_path, "r") as file: - problem = file.read() - -####################### scene -with open(scene_path, "r") as file: - scene_json_object = json.load(file) - -scene = SceneFactory().create_scene_from_json(scene_json_object) - -####################### world -world_interface = WorldInterface() -with open(world_state_path, "r") as file: - world_state_json_object = json.load(file) - world_interface.load_world_from_json(world_state_json_object) - -from_problem = parse_problem_init(problem=problem) -objects = parse_problem_objects(problem=problem) -world_interface.update_world(from_problem) - -world_state_json = world_interface.get_world_to_json() -pprint(from_problem) -pprint(world_state_json) -pprint(objects) - - -####################### robot -robot_interface = RobotInterface( - robot_address="127.0.0.1", - robot_port=12000, -) -robot_interface.setup_scene(scene) - -####################### bt_factory -bt_factory = BehaviorTreeFactory( - world_interface=world_interface, - robot_interface=robot_interface, -) - -####################### behavior_tree_stewardship -behavior_tree_stewardship = BehaviorTreeStewardship( - behaviortree_factory=bt_factory, - world_interface=world_interface, - robot_interface=robot_interface, -) - -####################### * tools -# bt_executor_tool = BehaviorTreeExecutorTool(behavior_tree_stewardship) - -bt_sim_tool = BehaviorTreeSimulatorTool(metadata={"bt_stw": behavior_tree_stewardship}) - -# world_state_query_tool = WorldStateQueryTool(behavior_tree_stewardship) - -executor_tools = [bt_sim_tool] - -# * kios data prompt skeleton dir -data_dir = os.environ.get("KIOS_DATA_DIR").format(username=os.getlogin()) -print(data_dir) -prompt_sk_dir = os.path.join(data_dir, "prompt_skeletons") -prompt_dir = os.path.join(data_dir, "prompts") - - -# * the graph state -class PlanExecuteState(TypedDict): - plan: List[str] - behavior_tree: dict - world_state: Annotated[ - List[dict], operator.add - ] # ! to add, you need to make world_state a list of dict - past_steps: Annotated[List[Tuple], operator.add] - user_input: str - objects: dict[str, list[str]] - last_behavior_tree: dict - last_failed_node: dict - runtime_world_state: dict - behavior_tree_execution_summary: str - BTExecutionHasSucceeded: bool - - -########## verifier - -behaviortree_file = os.path.join(prompt_dir, "vv/behaviortree.txt") -template_file = os.path.join(prompt_dir, "vv/template.txt") -task_file = os.path.join(prompt_dir, "vv/task.txt") -system_file = os.path.join(prompt_dir, "vv/system.txt") -object_file = os.path.join(prompt_dir, "vv/object.txt") -domain_file = os.path.join(prompt_dir, "vv/domain.txt") -state_file = os.path.join(prompt_dir, "vv/state.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) -with open(object_file, "r") as f: - object_ppt = PromptTemplate.from_template(f.read()) -with open(behaviortree_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - behaviortree_ppt = ppt_tmp.partial(input=f.read()) - -with open(state_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - state_ppt = ppt_tmp.partial(input=f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {state} - - {object} - - {behaviortree} - - {template} - """ -) - -verifier_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ("object", object_ppt), - ("behaviortree", behaviortree_ppt), - ("state", state_ppt), - ], -) - -verifier_chain = ( - verifier_ppt_ppl - | ChatOpenAI(model="gpt-4-turbo-preview", temperature=0) - | StrOutputParser() -) - - -# * output schema of the recursive_sk_generator -class SolutionBehaviortree(BaseModel): - """the fixed behavior tree""" - - behavior_tree: dict = Field( - description="the behavior tree that adheres the required format." - ) - - -behaviortree_file = os.path.join(prompt_dir, "rec_sk_gen/behaviortree.txt") -template_file = os.path.join(prompt_dir, "rec_sk_gen/template.txt") -task_file = os.path.join(prompt_dir, "rec_sk_gen/task.txt") -system_file = os.path.join(prompt_dir, "rec_sk_gen/system.txt") -object_file = os.path.join(prompt_dir, "rec_sk_gen/object.txt") -domain_file = os.path.join(prompt_dir, "rec_sk_gen/domain.txt") -example_file = os.path.join(prompt_dir, "rec_sk_gen/example.txt") -state_file = os.path.join(prompt_dir, "rec_sk_gen/state.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) -with open(object_file, "r") as f: - object_ppt = PromptTemplate.from_template(f.read()) -with open(behaviortree_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - behaviortree_ppt = ppt_tmp.partial(input=f.read()) - -with open(state_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - state_ppt = ppt_tmp.partial(input=f.read()) - -with open(example_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - example_ppt = ppt_tmp.partial(input=f.read()) -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {state} - - {object} - - {behaviortree} - - {example} - - {template} - """ -) - -re_sk_gen_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ("object", object_ppt), - ("behaviortree", behaviortree_ppt), - ("example", example_ppt), - ("state", state_ppt), - ], -) - -rec_sk_gen_chain = ( - re_sk_gen_ppt_ppl - | ChatOpenAI(model="gpt-4-turbo-preview", temperature=0) - | JsonOutputParser() -) - -# recursive_sk_generator = create_structured_output_runnable( -# SolutionBehaviortree, -# ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), -# re_sk_gen_ppt_ppl, -# mode="openai-json", -# ) - - -##################################################### * planner -# * output schema of the planner -class Plan(BaseModel): - """Plan to follow in future""" - - steps: List[str] = Field( - description="a list of different steps to follow, should be in sorted order" - ) - - -template_file = os.path.join(prompt_dir, "planner/template.txt") -task_file = os.path.join(prompt_dir, "planner/task.txt") -system_file = os.path.join(prompt_dir, "planner/system.txt") -domain_file = os.path.join(prompt_dir, "planner/domain.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {template} - """ -) - -planner_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ], -) - -planner = create_structured_output_runnable( - Plan, ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), planner_ppt_ppl -) - -##################################################### * plan_updaterner - -template_file = os.path.join(prompt_dir, "plan_updater/template.txt") -task_file = os.path.join(prompt_dir, "plan_updater/task.txt") -system_file = os.path.join(prompt_dir, "plan_updater/system.txt") -domain_file = os.path.join(prompt_dir, "plan_updater/domain.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {template} - """ -) - -plan_updater_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ], -) - -plan_updater = create_structured_output_runnable( - Plan, ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), plan_updater_ppt_ppl -) -# ! create openai fn runnable has bug - - -##################################################### * graph node functions -@traceable(name="user_input_node_step") -async def user_input_step(state: PlanExecuteState): - """ - get the input from the user - """ - print(f"-----user_input_step-----") - - user_input = input("Your next instruction: ") - - return { - "user_input": user_input, - } - - -@traceable(name="behavior_tree_generate_step") -async def behavior_tree_generate_step(state: PlanExecuteState): - """ - generate the behavior tree based on the instruction - """ - print(f"-----behavior_tree_generate_step-----") - - instruction = state["plan"][0] - if state["runtime_world_state"] is None: - runtime_world_state = state["world_state"][-1] - else: - runtime_world_state = state["runtime_world_state"] - objects = state["objects"] - - # solution_bt = await recursive_sk_generator.ainvoke( - # { - # "objects": objects, - # "runtime_world_state": runtime_world_state, - # "last_behavior_tree": state["last_behavior_tree"], - # "last_failed_node": state["last_failed_node"], - # "instructions": instruction, - # } - # ) - - bt_skeleton = await rec_sk_gen_chain.ainvoke( - { - "objects": objects, - "runtime_world_state": runtime_world_state, - "last_behavior_tree": state["last_behavior_tree"], - "last_failed_node": state["last_failed_node"], - "summary": state["behavior_tree_execution_summary"], - "instructions": instruction, - } - ) - - # print(solution_bt) - # pause = input("paused here") - - # bt_skeleton = solution_bt.behavior_tree - - render_bt(bt_skeleton) - - pause = input("paused here") - - return { - "last_behavior_tree": bt_skeleton, - "BTExecutionHasSucceeded": False, - } - - -@traceable(name="behavior_tree_execute_step") -async def behavior_tree_execute_step(state: PlanExecuteState): - """ - execute the first step of the plan, append the result to the past steps - """ - print(f"-----behavior_tree_execute_step-----") - - this_step = state["plan"][0] - behavior_tree_skeleton = state["last_behavior_tree"] - latest_world_state = state["world_state"][-1] - - # * first sim run - tree_result, skeleton_json = behavior_tree_stewardship.sk_sim_run( - world_state=latest_world_state, skeleton_json=behavior_tree_skeleton - ) - - pprint(tree_result.to_json()) - pause = input("paused here") - - # * check result - if tree_result.result == "success": - - # * second fake run - tree_result, skeleton_json = behavior_tree_stewardship.sk_fake_run( - world_state=latest_world_state, skeleton_json=behavior_tree_skeleton - ) - - pprint(tree_result.to_json()) - pause = input("paused here") - - if tree_result.result == "success": - return { - "BTExecutionHasSucceeded": True, - "past_steps": ( - this_step, - tree_result.result, - ), # * only one because new plan will be generated and old steps are all removed - "world_state": [tree_result.world_state], - # * clear all - "last_behavior_tree": None, - "last_failed_node": None, - "runtime_world_state": None, - "behavior_tree_execution_summary": None, - } - else: - return { - "world_state": [tree_result.world_state], # * real world state - "last_behavior_tree": skeleton_json, - "last_failed_node": { - "summary": tree_result.final_node.get("summary"), - "name": tree_result.final_node.get("name"), - "identifier": tree_result.final_node.get("identifier"), - }, - "runtime_world_state": tree_result.world_state, - "behavior_tree_execution_summary": tree_result.summary, - } - - else: - return { - # ! do not change world state - "last_behavior_tree": skeleton_json, - "last_failed_node": { - "summary": tree_result.final_node.get("summary"), - "name": tree_result.final_node.get("name"), - "identifier": tree_result.final_node.get("identifier"), - }, - "runtime_world_state": tree_result.world_state, - "behavior_tree_execution_summary": tree_result.summary, - } - - -@traceable(name="planner_step") -async def planner_step(state: PlanExecuteState): - """ - plan the steps based on user input and world state - """ - print(f"-----plan_step-----") - - plan = await planner.ainvoke( - { - "user_input": state["user_input"], - "world_state": state["world_state"], - } - ) - return {"plan": plan.steps} - - -@traceable(name="plan_updater_step") -async def plan_updater_step(state: PlanExecuteState): - """ - if return a response, then success, response the use, end. - otherwise, return the updated newplan (normally the same as the old plan with the first step popped out. - """ - print(f"-----plan_updater_step-----") - - # ! BOOOM! BUG! - output = await plan_updater.ainvoke( - { - "user_input": state["user_input"], - "plan": state["plan"], - "world_state": state["world_state"], - "past_steps": state["past_steps"], - } - ) - # if isinstance( - # output, UpdaterResponse - # ): # * determine if it is time to response and end - # return { - # "response": "Your last instruction has been finished." - # } # * Don't need to update. - # else: - return {"plan": output.steps} # * update the plan - - -##################################################### * construct the graph - -workflow = StateGraph(PlanExecuteState) - -workflow.add_node("planner", planner_step) - -workflow.add_node("behavior_tree_generator", behavior_tree_generate_step) - -workflow.add_node("behavior_tree_executor", behavior_tree_execute_step) - -workflow.add_node("plan_updater", plan_updater_step) - -workflow.add_node("user_input_node", user_input_step) - -workflow.set_entry_point("user_input_node") - -workflow.add_edge( - "planner", "behavior_tree_generator" -) # ! may need to add a condition here - -workflow.add_edge("behavior_tree_generator", "behavior_tree_executor") - - -def executor_should_end(state: PlanExecuteState): - """ - end router - """ - print(f"-----executor_should_end-----") - - if state["BTExecutionHasSucceeded"] == True: - state["BTExecutionHasSucceeded"] = False - return True - else: - return False - - -workflow.add_conditional_edges( - "behavior_tree_executor", - executor_should_end, - { - True: "plan_updater", - False: "behavior_tree_generator", - }, -) - - -def plan_updater_should_end(state: PlanExecuteState): - """ - end router - """ - print(f"-----plan_updater_should_end-----") - - # if state["response"]: - # return True - # else: - # return False - if state["plan"] == [] or len(state["plan"]) == 0: - return True - else: - return False - - -workflow.add_conditional_edges( - "plan_updater", - plan_updater_should_end, - { - True: "user_input_node", - False: "behavior_tree_generator", - }, -) - - -def user_input_should_end(state: PlanExecuteState): - """ - if the user input is empty, then end - """ - print(f"-----user_input_should_end-----") - - # print(f'user_input: {state["user_input"]}') - # print(f'world_state: {state["world_state"]}') - - if not state["user_input"] or state["user_input"] == "": - return True - else: - return False - - -workflow.add_conditional_edges( - "user_input_node", - user_input_should_end, - { - True: END, - False: "planner", - }, -) - -app = workflow.compile() - -config = {"recursion_limit": 500} - -inputs = { - "world_state": [world_state_json], - "objects": objects, -} - - -async def core_run(): - async for event in app.astream( - inputs, - config=config, - ): - for k, v in event.items(): - if k != "__end__": - print(v) - - -if __name__ == "__main__": - import asyncio - - asyncio.run(core_run()) diff --git a/experiments/cook/rec_sk_seq.py b/experiments/cook/rec_sk_seq.py deleted file mode 100644 index e95a9fe4..00000000 --- a/experiments/cook/rec_sk_seq.py +++ /dev/null @@ -1,659 +0,0 @@ -import json -import os -from pprint import pprint -from typing import List, Tuple, Annotated, TypedDict -import operator - -""" -recursive generation + sequence guidance + simulation -""" - -os.environ["LANGCHAIN_TRACING_V2"] = "true" -os.environ["LANGCHAIN_ENDPOINT"] = "https://api.smith.langchain.com" -os.environ["LANGCHAIN_PROJECT"] = "kios_agent" - -from kios_bt.bt_stewardship import BehaviorTreeStewardship -from kios_scene.scene_factory import SceneFactory -from kios_bt.bt_factory import BehaviorTreeFactory -from kios_robot.robot_interface import RobotInterface -from kios_world.world_interface import WorldInterface - -from kios_agent.llm_supporter import KiosLLMSupporter -from kios_agent.data_types import KiosPromptSkeleton - -from kios_agent.kios_tools import ( - BehaviorTreeExecutorTool, - BehaviorTreeSimulatorTool, - WorldStateQueryTool, -) - -from dotenv import load_dotenv - -from langchain import hub - -from langchain_openai import ChatOpenAI - -from langchain.chains.openai_functions import ( - create_structured_output_runnable, - create_openai_fn_runnable, -) -from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder -from langchain_core.output_parsers import JsonOutputParser, StrOutputParser -from langchain_core.pydantic_v1 import BaseModel, Field - -from langchain.prompts.pipeline import PipelinePromptTemplate -from langchain.prompts.prompt import PromptTemplate - -from langgraph.graph import StateGraph, END - -from langsmith import traceable - -load_dotenv() - -from kios_utils.pybt_test import generate_bt_stewardship, render_dot_tree -from kios_utils.pddl_problem_parser import parse_problem_init, parse_problem_objects - - -def render_bt(bt_json: json): - test_class = BehaviorTreeFactory() - bt = test_class.from_json_to_simple_bt(bt_json) - # bt = test_class.from_json_to_tree_root(bt_json) - bt_stewardship = generate_bt_stewardship(bt) - # bt_stewardship.setup(timeout=15) - render_dot_tree(bt_stewardship) - - -####################### dirs -current_dir = os.path.dirname(os.path.abspath(__file__)) -scene_path = os.path.join(current_dir, "scene.json") -# bt_json_file_path = os.path.join(current_dir, "behavior_tree.json") -world_state_path = os.path.join(current_dir, "world_state.json") -problem_path = os.path.join(current_dir, "gearset.problem") -domain_knowledge_path = os.path.join(current_dir, "domain_knowledge.txt") - -####################### problem -with open(problem_path, "r") as file: - problem = file.read() - -####################### scene -with open(scene_path, "r") as file: - scene_json_object = json.load(file) - -scene = SceneFactory().create_scene_from_json(scene_json_object) - -####################### world -world_interface = WorldInterface() -with open(world_state_path, "r") as file: - world_state_json_object = json.load(file) - world_interface.load_world_from_json(world_state_json_object) - -from_problem = parse_problem_init(problem=problem) -objects = parse_problem_objects(problem=problem) -world_interface.update_world(from_problem) - -world_state_json = world_interface.get_world_to_json() -pprint(from_problem) -pprint(world_state_json) -pprint(objects) - - -####################### robot -robot_interface = RobotInterface( - robot_address="127.0.0.1", - robot_port=12000, -) -robot_interface.setup_scene(scene) - -####################### bt_factory -bt_factory = BehaviorTreeFactory( - world_interface=world_interface, - robot_interface=robot_interface, -) - -####################### behavior_tree_stewardship -behavior_tree_stewardship = BehaviorTreeStewardship( - behaviortree_factory=bt_factory, - world_interface=world_interface, - robot_interface=robot_interface, -) - -####################### * tools -# bt_executor_tool = BehaviorTreeExecutorTool(behavior_tree_stewardship) - -bt_sim_tool = BehaviorTreeSimulatorTool(metadata={"bt_stw": behavior_tree_stewardship}) - -# world_state_query_tool = WorldStateQueryTool(behavior_tree_stewardship) - -executor_tools = [bt_sim_tool] - -# * kios data prompt skeleton dir -data_dir = os.environ.get("KIOS_DATA_DIR").format(username=os.getlogin()) -print(data_dir) -prompt_sk_dir = os.path.join(data_dir, "prompt_skeletons") -prompt_dir = os.path.join(data_dir, "prompts") - - -# * the graph state -class PlanExecuteState(TypedDict): - plan: List[str] - behavior_tree: dict - world_state: Annotated[ - List[dict], operator.add - ] # ! to add, you need to make world_state a list of dict - past_steps: Annotated[List[Tuple], operator.add] - user_input: str - objects: dict[str, list[str]] - last_behavior_tree: dict - last_failed_node: dict - runtime_world_state: dict - behavior_tree_execution_summary: str - BTExecutionHasSucceeded: bool - - action_sequence: List[str] - - -########## verifier - -template_file = os.path.join(prompt_dir, "seq_planner/template.txt") -task_file = os.path.join(prompt_dir, "seq_planner/task.txt") -system_file = os.path.join(prompt_dir, "seq_planner/system.txt") -domain_file = os.path.join(prompt_dir, "seq_planner/domain.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {template} - """ -) - -seq_planner_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ], -) - -seq_planner_chain = ( - seq_planner_ppt_ppl - | ChatOpenAI(model="gpt-4-turbo-preview", temperature=0) - | StrOutputParser() -) - - -# * output schema of the recursive_sk_generator -class SolutionBehaviortree(BaseModel): - """the fixed behavior tree""" - - behavior_tree: dict = Field( - description="the behavior tree that adheres the required format." - ) - - -behaviortree_file = os.path.join(prompt_dir, "rec_sk_gen/behaviortree.txt") -template_file = os.path.join(prompt_dir, "rec_sk_gen/template.txt") -task_file = os.path.join(prompt_dir, "rec_sk_gen/task.txt") -system_file = os.path.join(prompt_dir, "rec_sk_gen/system.txt") -object_file = os.path.join(prompt_dir, "rec_sk_gen/object.txt") -domain_file = os.path.join(prompt_dir, "rec_sk_gen/domain.txt") -example_file = os.path.join(prompt_dir, "rec_sk_gen/example.txt") -state_file = os.path.join(prompt_dir, "rec_sk_gen/state.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) -with open(object_file, "r") as f: - object_ppt = PromptTemplate.from_template(f.read()) -with open(behaviortree_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - behaviortree_ppt = ppt_tmp.partial(input=f.read()) - -with open(state_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - state_ppt = ppt_tmp.partial(input=f.read()) - -with open(example_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - example_ppt = ppt_tmp.partial(input=f.read()) -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {state} - - {object} - - {behaviortree} - - {example} - - {template} - """ -) -# * recureive_sk_generator -re_sk_gen_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ("object", object_ppt), - ("behaviortree", behaviortree_ppt), - ("example", example_ppt), - ("state", state_ppt), - ], -) - -rec_sk_gen_chain = ( - re_sk_gen_ppt_ppl - | ChatOpenAI(model="gpt-4-turbo-preview", temperature=0) - | JsonOutputParser() -) - -##################################################### * planner -# * output schema of the planner -class Plan(BaseModel): - """Plan to follow in future""" - - steps: List[str] = Field( - description="a list of different steps to follow, should be in sorted order" - ) - - -template_file = os.path.join(prompt_dir, "planner/template.txt") -task_file = os.path.join(prompt_dir, "planner/task.txt") -system_file = os.path.join(prompt_dir, "planner/system.txt") -domain_file = os.path.join(prompt_dir, "planner/domain.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {template} - """ -) - -planner_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ], -) - -planner = create_structured_output_runnable( - Plan, ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), planner_ppt_ppl -) - -##################################################### * plan_updaterner - -template_file = os.path.join(prompt_dir, "plan_updater/template.txt") -task_file = os.path.join(prompt_dir, "plan_updater/task.txt") -system_file = os.path.join(prompt_dir, "plan_updater/system.txt") -domain_file = os.path.join(prompt_dir, "plan_updater/domain.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {template} - """ -) - -plan_updater_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ], -) - -plan_updater = create_structured_output_runnable( - Plan, ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), plan_updater_ppt_ppl -) -# ! create openai fn runnable has bug - - -##################################################### * graph node functions -@traceable(name="user_input_node_step") -async def user_input_step(state: PlanExecuteState): - """ - get the input from the user - """ - print(f"-----user_input_step-----") - - user_input = input("Your next instruction: ") - - return { - "user_input": user_input, - } - - -@traceable(name="sequence_generate_step") -async def sequence_generate_step(state: PlanExecuteState): - """ - generate the sequence based on the instruction - """ - print(f"-----sequence_generate_step-----") - - instruction = state["plan"][0] - start_world_state = state["world_state"][-1] - objects = state["objects"] - - action_sequence = await seq_planner_chain.ainvoke( - { - "objects": objects, - "start_world_state": start_world_state, - "user_instruction": instruction, - } - ) - - return { - "action_sequence": action_sequence, - } - - -@traceable(name="behavior_tree_generate_step") -async def behavior_tree_generate_step(state: PlanExecuteState): - """ - generate the behavior tree based on the instruction - """ - print(f"-----behavior_tree_generate_step-----") - - instruction = state["plan"][0] - if state["runtime_world_state"] is None: - runtime_world_state = state["world_state"][-1] - else: - runtime_world_state = state["runtime_world_state"] - objects = state["objects"] - - bt_skeleton = await rec_sk_gen_chain.ainvoke( - { - "objects": objects, - "runtime_world_state": runtime_world_state, - "last_behavior_tree": state["last_behavior_tree"], - "last_failed_node": state["last_failed_node"], - "summary": state["behavior_tree_execution_summary"], - "action_sequence": state["action_sequence"], - } - ) - - render_bt(bt_skeleton) - - pause = input("paused here") - - return { - "last_behavior_tree": bt_skeleton, - "BTExecutionHasSucceeded": False, - } - - -@traceable(name="behavior_tree_execute_step") -async def behavior_tree_execute_step(state: PlanExecuteState): - """ - execute the first step of the plan, append the result to the past steps - """ - print(f"-----behavior_tree_execute_step-----") - - this_step = state["plan"][0] - behavior_tree_skeleton = state["last_behavior_tree"] - latest_world_state = state["world_state"][-1] - - # * first sim run - tree_result, skeleton_json = behavior_tree_stewardship.sk_sim_run( - world_state=latest_world_state, skeleton_json=behavior_tree_skeleton - ) - - pprint(tree_result.to_json()) - pause = input("paused here") - - # * check result - if tree_result.result == "success": - - # * second fake run - tree_result, skeleton_json = behavior_tree_stewardship.sk_fake_run( - world_state=latest_world_state, skeleton_json=behavior_tree_skeleton - ) - - pprint(tree_result.to_json()) - pause = input("paused here") - - if tree_result.result == "success": - return { - "BTExecutionHasSucceeded": True, - "past_steps": ( - this_step, - tree_result.result, - ), # * only one because new plan will be generated and old steps are all removed - "world_state": [tree_result.world_state], - # * clear all - "last_behavior_tree": None, - "last_failed_node": None, - "runtime_world_state": None, - "behavior_tree_execution_summary": None, - } - else: - return { - "world_state": [tree_result.world_state], # * real world state - "last_behavior_tree": skeleton_json, - "last_failed_node": { - "summary": tree_result.final_node.get("summary"), - "name": tree_result.final_node.get("name"), - "identifier": tree_result.final_node.get("identifier"), - }, - "runtime_world_state": tree_result.world_state, - "behavior_tree_execution_summary": tree_result.summary, - } - - else: - return { - # ! do not change world state - "last_behavior_tree": skeleton_json, - "last_failed_node": { - "summary": tree_result.final_node.get("summary"), - "name": tree_result.final_node.get("name"), - "identifier": tree_result.final_node.get("identifier"), - }, - "runtime_world_state": tree_result.world_state, - "behavior_tree_execution_summary": tree_result.summary, - } - - -@traceable(name="planner_step") -async def planner_step(state: PlanExecuteState): - """ - plan the steps based on user input and world state - """ - print(f"-----plan_step-----") - - plan = await planner.ainvoke( - { - "user_input": state["user_input"], - "world_state": state["world_state"], - } - ) - return {"plan": plan.steps} - - -@traceable(name="plan_updater_step") -async def plan_updater_step(state: PlanExecuteState): - """ - if return a response, then success, response the use, end. - otherwise, return the updated newplan (normally the same as the old plan with the first step popped out. - """ - print(f"-----plan_updater_step-----") - - # ! BOOOM! BUG! - output = await plan_updater.ainvoke( - { - "user_input": state["user_input"], - "plan": state["plan"], - "world_state": state["world_state"], - "past_steps": state["past_steps"], - } - ) - # if isinstance( - # output, UpdaterResponse - # ): # * determine if it is time to response and end - # return { - # "response": "Your last instruction has been finished." - # } # * Don't need to update. - # else: - return {"plan": output.steps} # * update the plan - - -##################################################### * construct the graph - -workflow = StateGraph(PlanExecuteState) - -workflow.add_node("planner", planner_step) -workflow.add_node("sequence_generator", sequence_generate_step) -workflow.add_node("behavior_tree_generator", behavior_tree_generate_step) -workflow.add_node("behavior_tree_executor", behavior_tree_execute_step) -workflow.add_node("plan_updater", plan_updater_step) -workflow.add_node("user_input_node", user_input_step) -workflow.set_entry_point("user_input_node") -workflow.add_edge("planner", "sequence_generator") -workflow.add_edge("sequence_generator", "behavior_tree_generator") -workflow.add_edge("behavior_tree_generator", "behavior_tree_executor") - - -def executor_should_end(state: PlanExecuteState): - """ - end router - """ - print(f"-----executor_should_end-----") - - if state["BTExecutionHasSucceeded"] == True: - state["BTExecutionHasSucceeded"] = False - return True - else: - return False - - -workflow.add_conditional_edges( - "behavior_tree_executor", - executor_should_end, - { - True: "plan_updater", - False: "behavior_tree_generator", - }, -) - - -def plan_updater_should_end(state: PlanExecuteState): - """ - end router - """ - print(f"-----plan_updater_should_end-----") - - if state["plan"] == [] or len(state["plan"]) == 0: - return True - else: - return False - - -workflow.add_conditional_edges( - "plan_updater", - plan_updater_should_end, - { - True: "user_input_node", - False: "behavior_tree_generator", - }, -) - - -def user_input_should_end(state: PlanExecuteState): - """ - if the user input is empty, then end - """ - print(f"-----user_input_should_end-----") - - if not state["user_input"] or state["user_input"] == "": - return True - else: - return False - - -workflow.add_conditional_edges( - "user_input_node", - user_input_should_end, - { - True: END, - False: "planner", - }, -) - -app = workflow.compile() - -config = {"recursion_limit": 500} - -inputs = { - "world_state": [world_state_json], - "objects": objects, -} - - -async def core_run(): - async for event in app.astream( - inputs, - config=config, - ): - for k, v in event.items(): - if k != "__end__": - print(v) - - -if __name__ == "__main__": - # import asyncio - - # asyncio.run(core_run()) - pass diff --git a/experiments/cook/rec_sk_unit.py b/experiments/cook/rec_sk_unit.py deleted file mode 100644 index bcc923d6..00000000 --- a/experiments/cook/rec_sk_unit.py +++ /dev/null @@ -1,663 +0,0 @@ -import json -import os -from pprint import pprint -from typing import List, Tuple, Annotated, TypedDict -import operator - -os.environ["LANGCHAIN_TRACING_V2"] = "true" -os.environ["LANGCHAIN_ENDPOINT"] = "https://api.smith.langchain.com" -os.environ["LANGCHAIN_PROJECT"] = "kios_agent" - -from kios_bt.bt_stewardship import BehaviorTreeStewardship -from kios_scene.scene_factory import SceneFactory -from kios_bt.bt_factory import BehaviorTreeFactory -from kios_robot.robot_interface import RobotInterface -from kios_world.world_interface import WorldInterface - -from kios_agent.llm_supporter import KiosLLMSupporter -from kios_agent.data_types import KiosPromptSkeleton - -from kios_agent.kios_tools import ( - BehaviorTreeExecutorTool, - BehaviorTreeSimulatorTool, - WorldStateQueryTool, -) - -from dotenv import load_dotenv - -from langchain import hub - -from langchain_openai import ChatOpenAI - -from langchain.chains.openai_functions import ( - create_structured_output_runnable, - create_openai_fn_runnable, -) - -from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder -from langchain_core.output_parsers import JsonOutputParser, StrOutputParser -from langchain_core.pydantic_v1 import BaseModel, Field - -from langchain.prompts.pipeline import PipelinePromptTemplate -from langchain.prompts.prompt import PromptTemplate - -from langgraph.graph import StateGraph, END - -from langsmith import traceable - -load_dotenv() - -from kios_utils.pybt_test import generate_bt_stewardship, render_dot_tree -from kios_utils.pddl_problem_parser import parse_problem_init, parse_problem_objects - - -def render_bt(bt_json: json): - test_class = BehaviorTreeFactory() - bt = test_class.from_json_to_simple_bt(bt_json) - # bt = test_class.from_json_to_tree_root(bt_json) - bt_stewardship = generate_bt_stewardship(bt) - # bt_stewardship.setup(timeout=15) - render_dot_tree(bt_stewardship) - - -####################### dirs -current_dir = os.path.dirname(os.path.abspath(__file__)) -scene_path = os.path.join(current_dir, "scene.json") -# bt_json_file_path = os.path.join(current_dir, "behavior_tree.json") -world_state_path = os.path.join(current_dir, "world_state.json") -problem_path = os.path.join(current_dir, "gearset.problem") -domain_knowledge_path = os.path.join(current_dir, "domain_knowledge.txt") - -####################### problem -with open(problem_path, "r") as file: - problem = file.read() - -####################### scene -with open(scene_path, "r") as file: - scene_json_object = json.load(file) - -scene = SceneFactory().create_scene_from_json(scene_json_object) - -####################### world -world_interface = WorldInterface() -with open(world_state_path, "r") as file: - world_state_json_object = json.load(file) - world_interface.load_world_from_json(world_state_json_object) - -from_problem = parse_problem_init(problem=problem) -objects = parse_problem_objects(problem=problem) -world_interface.update_world(from_problem) - -world_state_json = world_interface.get_world_to_json() -pprint(from_problem) -pprint(world_state_json) -pprint(objects) - - -####################### robot -robot_interface = RobotInterface( - robot_address="127.0.0.1", - robot_port=12000, -) -robot_interface.setup_scene(scene) - -####################### bt_factory -bt_factory = BehaviorTreeFactory( - world_interface=world_interface, - robot_interface=robot_interface, -) - -####################### behavior_tree_stewardship -behavior_tree_stewardship = BehaviorTreeStewardship( - behaviortree_factory=bt_factory, - world_interface=world_interface, - robot_interface=robot_interface, -) - -####################### * tools -# bt_executor_tool = BehaviorTreeExecutorTool(behavior_tree_stewardship) - -bt_sim_tool = BehaviorTreeSimulatorTool(metadata={"bt_stw": behavior_tree_stewardship}) - -# world_state_query_tool = WorldStateQueryTool(behavior_tree_stewardship) - -executor_tools = [bt_sim_tool] - -# * kios data prompt skeleton dir -data_dir = os.environ.get("KIOS_DATA_DIR").format(username=os.getlogin()) -print(data_dir) -prompt_sk_dir = os.path.join(data_dir, "prompt_skeletons") -prompt_dir = os.path.join(data_dir, "prompts") - - -# * the graph state -class PlanExecuteState(TypedDict): - plan: List[str] - behavior_tree: dict - world_state: Annotated[ - List[dict], operator.add - ] # ! to add, you need to make world_state a list of dict - past_steps: Annotated[List[Tuple], operator.add] - user_input: str - objects: dict[str, list[str]] - last_behavior_tree: dict - last_failed_node: dict - runtime_world_state: dict - behavior_tree_execution_summary: str - BTExecutionHasSucceeded: bool - - action_sequence: List[str] - - -########## verifier - -template_file = os.path.join(prompt_dir, "seq_planner/template.txt") -task_file = os.path.join(prompt_dir, "seq_planner/task.txt") -system_file = os.path.join(prompt_dir, "seq_planner/system.txt") -domain_file = os.path.join(prompt_dir, "seq_planner/domain.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {template} - """ -) - -seq_planner_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ], -) - -seq_planner_chain = ( - seq_planner_ppt_ppl - | ChatOpenAI(model="gpt-4-turbo-preview", temperature=0) - | StrOutputParser() -) - - -# * output schema of the recursive_sk_generator -class SolutionBehaviortree(BaseModel): - """the fixed behavior tree""" - - behavior_tree: dict = Field( - description="the behavior tree that adheres the required format." - ) - - -behaviortree_file = os.path.join(prompt_dir, "rec_sk_gen/behaviortree.txt") -template_file = os.path.join(prompt_dir, "rec_sk_gen/template.txt") -task_file = os.path.join(prompt_dir, "rec_sk_gen/task.txt") -system_file = os.path.join(prompt_dir, "rec_sk_gen/system.txt") -object_file = os.path.join(prompt_dir, "rec_sk_gen/object.txt") -domain_file = os.path.join(prompt_dir, "rec_sk_gen/domain.txt") -example_file = os.path.join(prompt_dir, "rec_sk_gen/example.txt") -state_file = os.path.join(prompt_dir, "rec_sk_gen/state.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) -with open(object_file, "r") as f: - object_ppt = PromptTemplate.from_template(f.read()) -with open(behaviortree_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - behaviortree_ppt = ppt_tmp.partial(input=f.read()) - -with open(state_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - state_ppt = ppt_tmp.partial(input=f.read()) - -with open(example_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - example_ppt = ppt_tmp.partial(input=f.read()) -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {state} - - {object} - - {behaviortree} - - {example} - - {template} - """ -) - -re_sk_gen_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ("object", object_ppt), - ("behaviortree", behaviortree_ppt), - ("example", example_ppt), - ("state", state_ppt), - ], -) - -rec_sk_gen_chain = ( - re_sk_gen_ppt_ppl - | ChatOpenAI(model="gpt-4-turbo-preview", temperature=0) - | JsonOutputParser() -) - -# recursive_sk_generator = create_structured_output_runnable( -# SolutionBehaviortree, -# ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), -# re_sk_gen_ppt_ppl, -# mode="openai-json", -# ) - - -##################################################### * planner -# * output schema of the planner -class Plan(BaseModel): - """Plan to follow in future""" - - steps: List[str] = Field( - description="a list of different steps to follow, should be in sorted order" - ) - - -template_file = os.path.join(prompt_dir, "planner/template.txt") -task_file = os.path.join(prompt_dir, "planner/task.txt") -system_file = os.path.join(prompt_dir, "planner/system.txt") -domain_file = os.path.join(prompt_dir, "planner/domain.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {template} - """ -) - -planner_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ], -) - -planner = create_structured_output_runnable( - Plan, ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), planner_ppt_ppl -) - -##################################################### * plan_updaterner - -template_file = os.path.join(prompt_dir, "plan_updater/template.txt") -task_file = os.path.join(prompt_dir, "plan_updater/task.txt") -system_file = os.path.join(prompt_dir, "plan_updater/system.txt") -domain_file = os.path.join(prompt_dir, "plan_updater/domain.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {template} - """ -) - -plan_updater_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ], -) - -plan_updater = create_structured_output_runnable( - Plan, ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), plan_updater_ppt_ppl -) -# ! create openai fn runnable has bug - - -##################################################### * graph node functions -@traceable(name="user_input_node_step") -async def user_input_step(state: PlanExecuteState): - """ - get the input from the user - """ - print(f"-----user_input_step-----") - - user_input = input("Your next instruction: ") - - return { - "user_input": user_input, - } - - -@traceable(name="sequence_generate_step") -async def sequence_generate_step(state: PlanExecuteState): - """ - generate the sequence based on the instruction - """ - print(f"-----sequence_generate_step-----") - - instruction = state["plan"][0] - start_world_state = state["world_state"][-1] - objects = state["objects"] - - action_sequence = await seq_planner_chain.ainvoke( - { - "objects": objects, - "start_world_state": start_world_state, - "user_instruction": instruction, - } - ) - - return { - "action_sequence": action_sequence, - } - - -@traceable(name="behavior_tree_generate_step") -async def behavior_tree_generate_step(state: PlanExecuteState): - """ - generate the behavior tree based on the instruction - """ - print(f"-----behavior_tree_generate_step-----") - - instruction = state["plan"][0] - if state["runtime_world_state"] is None: - runtime_world_state = state["world_state"][-1] - else: - runtime_world_state = state["runtime_world_state"] - objects = state["objects"] - - bt_skeleton = await rec_sk_gen_chain.ainvoke( - { - "objects": objects, - "runtime_world_state": runtime_world_state, - "last_behavior_tree": state["last_behavior_tree"], - "last_failed_node": state["last_failed_node"], - "summary": state["behavior_tree_execution_summary"], - "action_sequence": state["action_sequence"], - } - ) - - render_bt(bt_skeleton) - - pause = input("paused here") - - return { - "last_behavior_tree": bt_skeleton, - "BTExecutionHasSucceeded": False, - } - - -@traceable(name="behavior_tree_execute_step") -async def behavior_tree_execute_step(state: PlanExecuteState): - """ - execute the first step of the plan, append the result to the past steps - """ - print(f"-----behavior_tree_execute_step-----") - - this_step = state["plan"][0] - behavior_tree_skeleton = state["last_behavior_tree"] - latest_world_state = state["world_state"][-1] - - # * first sim run - tree_result, skeleton_json = behavior_tree_stewardship.sk_sim_run( - world_state=latest_world_state, skeleton_json=behavior_tree_skeleton - ) - - pprint(tree_result.to_json()) - pause = input("paused here") - - # * check result - if tree_result.result == "success": - - # * second fake run - tree_result, skeleton_json = behavior_tree_stewardship.sk_fake_run( - world_state=latest_world_state, skeleton_json=behavior_tree_skeleton - ) - - pprint(tree_result.to_json()) - pause = input("paused here") - - if tree_result.result == "success": - return { - "BTExecutionHasSucceeded": True, - "past_steps": ( - this_step, - tree_result.result, - ), # * only one because new plan will be generated and old steps are all removed - "world_state": [tree_result.world_state], - # * clear all - "last_behavior_tree": None, - "last_failed_node": None, - "runtime_world_state": None, - "behavior_tree_execution_summary": None, - } - else: - return { - "world_state": [tree_result.world_state], # * real world state - "last_behavior_tree": skeleton_json, - "last_failed_node": { - "summary": tree_result.final_node.get("summary"), - "name": tree_result.final_node.get("name"), - "identifier": tree_result.final_node.get("identifier"), - }, - "runtime_world_state": tree_result.world_state, - "behavior_tree_execution_summary": tree_result.summary, - } - - else: - return { - # ! do not change world state - "last_behavior_tree": skeleton_json, - "last_failed_node": { - "summary": tree_result.final_node.get("summary"), - "name": tree_result.final_node.get("name"), - "identifier": tree_result.final_node.get("identifier"), - }, - "runtime_world_state": tree_result.world_state, - "behavior_tree_execution_summary": tree_result.summary, - } - - -@traceable(name="planner_step") -async def planner_step(state: PlanExecuteState): - """ - plan the steps based on user input and world state - """ - print(f"-----plan_step-----") - - plan = await planner.ainvoke( - { - "user_input": state["user_input"], - "world_state": state["world_state"], - } - ) - return {"plan": plan.steps} - - -@traceable(name="plan_updater_step") -async def plan_updater_step(state: PlanExecuteState): - """ - if return a response, then success, response the use, end. - otherwise, return the updated newplan (normally the same as the old plan with the first step popped out. - """ - print(f"-----plan_updater_step-----") - - # ! BOOOM! BUG! - output = await plan_updater.ainvoke( - { - "user_input": state["user_input"], - "plan": state["plan"], - "world_state": state["world_state"], - "past_steps": state["past_steps"], - } - ) - # if isinstance( - # output, UpdaterResponse - # ): # * determine if it is time to response and end - # return { - # "response": "Your last instruction has been finished." - # } # * Don't need to update. - # else: - return {"plan": output.steps} # * update the plan - - -##################################################### * construct the graph - -workflow = StateGraph(PlanExecuteState) - -workflow.add_node("planner", planner_step) -workflow.add_node("sequence_generator", sequence_generate_step) -workflow.add_node("behavior_tree_generator", behavior_tree_generate_step) -workflow.add_node("behavior_tree_executor", behavior_tree_execute_step) -workflow.add_node("plan_updater", plan_updater_step) -workflow.add_node("user_input_node", user_input_step) -workflow.set_entry_point("user_input_node") -workflow.add_edge("planner", "sequence_generator") -workflow.add_edge("sequence_generator", "behavior_tree_generator") -workflow.add_edge("behavior_tree_generator", "behavior_tree_executor") - - -def executor_should_end(state: PlanExecuteState): - """ - end router - """ - print(f"-----executor_should_end-----") - - if state["BTExecutionHasSucceeded"] == True: - state["BTExecutionHasSucceeded"] = False - return True - else: - return False - - -workflow.add_conditional_edges( - "behavior_tree_executor", - executor_should_end, - { - True: "plan_updater", - False: "behavior_tree_generator", - }, -) - - -def plan_updater_should_end(state: PlanExecuteState): - """ - end router - """ - print(f"-----plan_updater_should_end-----") - - if state["plan"] == [] or len(state["plan"]) == 0: - return True - else: - return False - - -workflow.add_conditional_edges( - "plan_updater", - plan_updater_should_end, - { - True: "user_input_node", - False: "behavior_tree_generator", - }, -) - - -def user_input_should_end(state: PlanExecuteState): - """ - if the user input is empty, then end - """ - print(f"-----user_input_should_end-----") - - if not state["user_input"] or state["user_input"] == "": - return True - else: - return False - - -workflow.add_conditional_edges( - "user_input_node", - user_input_should_end, - { - True: END, - False: "planner", - }, -) - -app = workflow.compile() - -config = {"recursion_limit": 500} - -inputs = { - "world_state": [world_state_json], - "objects": objects, -} - - -async def core_run(): - async for event in app.astream( - inputs, - config=config, - ): - for k, v in event.items(): - if k != "__end__": - print(v) - - -if __name__ == "__main__": - import asyncio - - asyncio.run(core_run()) diff --git a/experiments/cook/rec_vv.py b/experiments/cook/rec_vv.py deleted file mode 100644 index dfd72be2..00000000 --- a/experiments/cook/rec_vv.py +++ /dev/null @@ -1,737 +0,0 @@ -import json -import os -from pprint import pprint - -""" -iterative generation of behavior tree with a verifier and validator. -a good try but works not well. -""" - -os.environ["LANGCHAIN_TRACING_V2"] = "true" -os.environ["LANGCHAIN_ENDPOINT"] = "https://api.smith.langchain.com" -os.environ["LANGCHAIN_PROJECT"] = "kios_agent" - -from kios_bt.bt_stewardship import BehaviorTreeStewardship -from kios_scene.scene_factory import SceneFactory -from kios_bt.bt_factory import BehaviorTreeFactory -from kios_robot.robot_interface import RobotInterface -from kios_world.world_interface import WorldInterface - -from kios_agent.llm_supporter import KiosLLMSupporter -from kios_agent.data_types import KiosPromptSkeleton - -from kios_agent.kios_tools import ( - BehaviorTreeExecutorTool, - BehaviorTreeSimulatorTool, - WorldStateQueryTool, -) - -from typing import List, Tuple, Annotated, TypedDict -import operator - -from dotenv import load_dotenv - -from langchain import hub - -from langchain_openai import ChatOpenAI - -from langchain.chains.openai_functions import ( - create_structured_output_runnable, - create_openai_fn_runnable, -) -from langchain_core.utils.function_calling import convert_to_openai_function -from langchain_core.agents import AgentActionMessageLog, AgentFinish -from langchain_core.messages import HumanMessage -from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder -from langchain_core.output_parsers import JsonOutputParser, StrOutputParser -from langchain_core.pydantic_v1 import BaseModel, Field - -from langchain.agents.format_scratchpad import format_to_openai_function_messages -from langchain.agents import create_openai_functions_agent, AgentExecutor - -from langchain.prompts.pipeline import PipelinePromptTemplate -from langchain.prompts.prompt import PromptTemplate - -from langchain_community.document_loaders import TextLoader -from langchain_openai import OpenAIEmbeddings - -from langchain_community.vectorstores import FAISS -from langchain.text_splitter import ( - RecursiveCharacterTextSplitter, - CharacterTextSplitter, -) - -from langgraph.graph import StateGraph, END - -from langsmith import traceable - -load_dotenv() - -from kios_utils.pybt_test import generate_bt_stewardship, render_dot_tree -from kios_utils.pddl_problem_parser import parse_problem_init, parse_problem_objects - - -def render_bt(bt_json: json): - test_class = BehaviorTreeFactory() - bt = test_class.from_json_to_simple_bt(bt_json) - # bt = test_class.from_json_to_tree_root(bt_json) - bt_stewardship = generate_bt_stewardship(bt) - # bt_stewardship.setup(timeout=15) - render_dot_tree(bt_stewardship) - - -####################### dirs -current_dir = os.path.dirname(os.path.abspath(__file__)) -scene_path = os.path.join(current_dir, "scene.json") -# bt_json_file_path = os.path.join(current_dir, "behavior_tree.json") -world_state_path = os.path.join(current_dir, "world_state.json") -problem_path = os.path.join(current_dir, "gearset.problem") -domain_knowledge_path = os.path.join(current_dir, "domain_knowledge.txt") - -####################### problem -with open(problem_path, "r") as file: - problem = file.read() - -####################### scene -with open(scene_path, "r") as file: - scene_json_object = json.load(file) - -scene = SceneFactory().create_scene_from_json(scene_json_object) - -####################### world -world_interface = WorldInterface() -with open(world_state_path, "r") as file: - world_state_json_object = json.load(file) - world_interface.load_world_from_json(world_state_json_object) - -from_problem = parse_problem_init(problem=problem) -objects = parse_problem_objects(problem=problem) -world_interface.update_world(from_problem) - -world_state_json = world_interface.get_world_to_json() -pprint(from_problem) -pprint(world_state_json) -pprint(objects) - - -####################### robot -robot_interface = RobotInterface( - robot_address="127.0.0.1", - robot_port=12000, -) -robot_interface.setup_scene(scene) - -####################### bt_factory -bt_factory = BehaviorTreeFactory( - world_interface=world_interface, - robot_interface=robot_interface, -) - -####################### behavior_tree_stewardship -behavior_tree_stewardship = BehaviorTreeStewardship( - behaviortree_factory=bt_factory, - world_interface=world_interface, - robot_interface=robot_interface, -) - -####################### * tools -# bt_executor_tool = BehaviorTreeExecutorTool(behavior_tree_stewardship) - -bt_sim_tool = BehaviorTreeSimulatorTool(metadata={"bt_stw": behavior_tree_stewardship}) - -# world_state_query_tool = WorldStateQueryTool(behavior_tree_stewardship) - -executor_tools = [bt_sim_tool] - -# * kios data prompt skeleton dir -data_dir = os.environ.get("KIOS_DATA_DIR").format(username=os.getlogin()) -print(data_dir) -prompt_sk_dir = os.path.join(data_dir, "prompt_skeletons") -prompt_dir = os.path.join(data_dir, "prompts") - - -# * the graph state -class PlanExecuteState(TypedDict): - plan: List[str] - behavior_tree: dict - world_state: Annotated[ - List[dict], operator.add - ] # ! to add, you need to make world_state a list of dict - past_steps: Annotated[List[Tuple], operator.add] - user_input: str - objects: dict[str, list[str]] - last_behavior_tree: dict - last_failed_node: dict - runtime_world_state: dict - behavior_tree_execution_summary: str - BTExecutionHasSucceeded: bool - - -########## verifier - -behaviortree_file = os.path.join(prompt_dir, "vv/behaviortree.txt") -template_file = os.path.join(prompt_dir, "vv/template.txt") -task_file = os.path.join(prompt_dir, "vv/task.txt") -system_file = os.path.join(prompt_dir, "vv/system.txt") -object_file = os.path.join(prompt_dir, "vv/object.txt") -domain_file = os.path.join(prompt_dir, "vv/domain.txt") -state_file = os.path.join(prompt_dir, "vv/state.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) -with open(object_file, "r") as f: - object_ppt = PromptTemplate.from_template(f.read()) -with open(behaviortree_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - behaviortree_ppt = ppt_tmp.partial(input=f.read()) - -with open(state_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - state_ppt = ppt_tmp.partial(input=f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {state} - - {object} - - {behaviortree} - - {template} - """ -) - -verifier_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ("object", object_ppt), - ("behaviortree", behaviortree_ppt), - ("state", state_ppt), - ], -) - -verifier_chain = ( - verifier_ppt_ppl - | ChatOpenAI(model="gpt-4-turbo-preview", temperature=0) - | StrOutputParser() -) - - -# * output schema of the recursive_sk_generator -class SolutionBehaviortree(BaseModel): - """the fixed behavior tree""" - - behavior_tree: dict = Field( - description="the behavior tree that adheres the required format." - ) - - -behaviortree_file = os.path.join(prompt_dir, "rec_sk_gen/behaviortree.txt") -template_file = os.path.join(prompt_dir, "rec_sk_gen/template.txt") -task_file = os.path.join(prompt_dir, "rec_sk_gen/task.txt") -system_file = os.path.join(prompt_dir, "rec_sk_gen/system.txt") -object_file = os.path.join(prompt_dir, "rec_sk_gen/object.txt") -domain_file = os.path.join(prompt_dir, "rec_sk_gen/domain.txt") -example_file = os.path.join(prompt_dir, "rec_sk_gen/example.txt") -state_file = os.path.join(prompt_dir, "rec_sk_gen/state.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) -with open(object_file, "r") as f: - object_ppt = PromptTemplate.from_template(f.read()) -with open(behaviortree_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - behaviortree_ppt = ppt_tmp.partial(input=f.read()) - -with open(state_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - state_ppt = ppt_tmp.partial(input=f.read()) - -with open(example_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - example_ppt = ppt_tmp.partial(input=f.read()) -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {state} - - {object} - - {behaviortree} - - {example} - - {template} - """ -) - -re_sk_gen_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ("object", object_ppt), - ("behaviortree", behaviortree_ppt), - ("example", example_ppt), - ("state", state_ppt), - ], -) - -rec_sk_gen_chain = ( - re_sk_gen_ppt_ppl - | ChatOpenAI(model="gpt-4-turbo-preview", temperature=0) - | JsonOutputParser() -) - -# recursive_sk_generator = create_structured_output_runnable( -# SolutionBehaviortree, -# ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), -# re_sk_gen_ppt_ppl, -# mode="openai-json", -# ) - - -##################################################### * planner -# * output schema of the planner -class Plan(BaseModel): - """Plan to follow in future""" - - steps: List[str] = Field( - description="a list of different steps to follow, should be in sorted order" - ) - - -template_file = os.path.join(prompt_dir, "planner/template.txt") -task_file = os.path.join(prompt_dir, "planner/task.txt") -system_file = os.path.join(prompt_dir, "planner/system.txt") -domain_file = os.path.join(prompt_dir, "planner/domain.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {template} - """ -) - -planner_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ], -) - -planner = create_structured_output_runnable( - Plan, ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), planner_ppt_ppl -) - -##################################################### * plan_updaterner - -template_file = os.path.join(prompt_dir, "plan_updater/template.txt") -task_file = os.path.join(prompt_dir, "plan_updater/task.txt") -system_file = os.path.join(prompt_dir, "plan_updater/system.txt") -domain_file = os.path.join(prompt_dir, "plan_updater/domain.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {template} - """ -) - -plan_updater_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ], -) - -plan_updater = create_structured_output_runnable( - Plan, ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), plan_updater_ppt_ppl -) -# ! create openai fn runnable has bug - - -##################################################### * graph node functions -@traceable(name="user_input_node_step") -async def user_input_step(state: PlanExecuteState): - """ - get the input from the user - """ - print(f"-----user_input_step-----") - - user_input = input("Your next instruction: ") - - return { - "user_input": user_input, - } - - -@traceable(name="behavior_tree_generate_step") -async def behavior_tree_generate_step(state: PlanExecuteState): - """ - generate the behavior tree based on the instruction - """ - print(f"-----behavior_tree_generate_step-----") - - instruction = state["plan"][0] - if state["runtime_world_state"] is None: - runtime_world_state = state["world_state"][-1] - else: - runtime_world_state = state["runtime_world_state"] - objects = state["objects"] - - bt_skeleton = await rec_sk_gen_chain.ainvoke( - { - "objects": objects, - "runtime_world_state": runtime_world_state, - "last_behavior_tree": state["last_behavior_tree"], - "last_failed_node": state["last_failed_node"], - "summary": state["behavior_tree_execution_summary"], - "instructions": instruction, - } - ) - - render_bt(bt_skeleton) - - pause = input("paused here") - - return { - "last_behavior_tree": bt_skeleton, - "BTExecutionHasSucceeded": False, - } - - -@traceable(name="behavior_tree_execute_step") -async def behavior_tree_execute_step(state: PlanExecuteState): - """ - execute the first step of the plan, append the result to the past steps - """ - print(f"-----behavior_tree_execute_step-----") - - this_step = state["plan"][0] - behavior_tree_skeleton = state["last_behavior_tree"] - latest_world_state = state["world_state"][-1] - - # * first sim run - tree_result, skeleton_json = behavior_tree_stewardship.sk_sim_run( - world_state=latest_world_state, skeleton_json=behavior_tree_skeleton - ) - - pprint(tree_result.to_json()) - pause = input("paused here") - - # * check result - if tree_result.result == "success": - - # * second fake run - tree_result, skeleton_json = behavior_tree_stewardship.sk_fake_run( - world_state=latest_world_state, skeleton_json=behavior_tree_skeleton - ) - - pprint(tree_result.to_json()) - pause = input("paused here") - - if tree_result.result == "success": - return { - "BTExecutionHasSucceeded": True, - "past_steps": ( - this_step, - tree_result.result, - ), # * only one because new plan will be generated and old steps are all removed - "world_state": [tree_result.world_state], - # * clear all - "last_behavior_tree": None, - "last_failed_node": None, - "runtime_world_state": None, - "behavior_tree_execution_summary": None, - } - else: - return { - "world_state": [tree_result.world_state], # * real world state - "last_behavior_tree": skeleton_json, - "last_failed_node": { - "summary": tree_result.final_node.get("summary"), - "name": tree_result.final_node.get("name"), - "identifier": tree_result.final_node.get("identifier"), - }, - "runtime_world_state": tree_result.world_state, - "behavior_tree_execution_summary": tree_result.summary, - } - - else: - return { - # ! do not change world state - "last_behavior_tree": skeleton_json, - "last_failed_node": { - "summary": tree_result.final_node.get("summary"), - "name": tree_result.final_node.get("name"), - "identifier": tree_result.final_node.get("identifier"), - }, - "runtime_world_state": tree_result.world_state, - "behavior_tree_execution_summary": tree_result.summary, - } - - -@traceable(name="verifier_step") -async def verifier_step(state: PlanExecuteState): - """ - verify the behavior tree - """ - print(f"-----verifier_step-----") - start_world_state = state["world_state"][-1] - - # * get the objects - objects = state["objects"] - - # * get the last behavior tree - last_behavior_tree = state["last_behavior_tree"] - - # * get the last failed node - last_failed_node = state["last_failed_node"] - - # * get the runtime world state - runtime_world_state = state["runtime_world_state"] - - # * get the behavior tree execution summary - behavior_tree_execution_summary = state["behavior_tree_execution_summary"] - - # * get the verifier response - verifier_response = await verifier_chain.ainvoke( - { - "objects": objects, - "start_world_state": start_world_state, - "last_behavior_tree": last_behavior_tree, - "last_failed_node": last_failed_node, - "runtime_world_state": runtime_world_state, - "summary": behavior_tree_execution_summary, - } - ) - - # * return the verifier response - return { - "behavior_tree_execution_summary": verifier_response, - } - - -@traceable(name="planner_step") -async def planner_step(state: PlanExecuteState): - """ - plan the steps based on user input and world state - """ - print(f"-----plan_step-----") - - plan = await planner.ainvoke( - { - "user_input": state["user_input"], - "world_state": state["world_state"], - } - ) - return {"plan": plan.steps} - - -@traceable(name="plan_updater_step") -async def plan_updater_step(state: PlanExecuteState): - """ - if return a response, then success, response the use, end. - otherwise, return the updated newplan (normally the same as the old plan with the first step popped out. - """ - print(f"-----plan_updater_step-----") - - # ! BOOOM! BUG! - output = await plan_updater.ainvoke( - { - "user_input": state["user_input"], - "plan": state["plan"], - "world_state": state["world_state"], - "past_steps": state["past_steps"], - } - ) - # if isinstance( - # output, UpdaterResponse - # ): # * determine if it is time to response and end - # return { - # "response": "Your last instruction has been finished." - # } # * Don't need to update. - # else: - return {"plan": output.steps} # * update the plan - - -##################################################### * construct the graph - -workflow = StateGraph(PlanExecuteState) - -workflow.add_node("planner", planner_step) - -workflow.add_node("behavior_tree_generator", behavior_tree_generate_step) - -workflow.add_node("behavior_tree_executor", behavior_tree_execute_step) - -workflow.add_node("verifier", verifier_step) - -workflow.add_node("plan_updater", plan_updater_step) - -workflow.add_node("user_input_node", user_input_step) - -workflow.set_entry_point("user_input_node") - -workflow.add_edge( - "planner", "behavior_tree_generator" -) # ! may need to add a condition here - -workflow.add_edge("behavior_tree_generator", "behavior_tree_executor") - -workflow.add_edge("verifier", "behavior_tree_generator") - - -def executor_should_end(state: PlanExecuteState): - """ - end router - """ - print(f"-----executor_should_end-----") - - if state["BTExecutionHasSucceeded"] == True: - state["BTExecutionHasSucceeded"] = False - return True - else: - return False - - -workflow.add_conditional_edges( - "behavior_tree_executor", - executor_should_end, - { - True: "plan_updater", - False: "verifier", - }, -) - - -def plan_updater_should_end(state: PlanExecuteState): - """ - end router - """ - print(f"-----plan_updater_should_end-----") - - # if state["response"]: - # return True - # else: - # return False - if state["plan"] == [] or len(state["plan"]) == 0: - return True - else: - return False - - -workflow.add_conditional_edges( - "plan_updater", - plan_updater_should_end, - { - True: "user_input_node", - False: "behavior_tree_generator", - }, -) - - -def user_input_should_end(state: PlanExecuteState): - """ - if the user input is empty, then end - """ - print(f"-----user_input_should_end-----") - - # print(f'user_input: {state["user_input"]}') - # print(f'world_state: {state["world_state"]}') - - if not state["user_input"] or state["user_input"] == "": - return True - else: - return False - - -workflow.add_conditional_edges( - "user_input_node", - user_input_should_end, - { - True: END, - False: "planner", - }, -) - -app = workflow.compile() - -config = {"recursion_limit": 500} - -inputs = { - "world_state": [world_state_json], - "objects": objects, -} - - -async def core_run(): - async for event in app.astream( - inputs, - config=config, - ): - for k, v in event.items(): - if k != "__end__": - print(v) - - -if __name__ == "__main__": - import asyncio - - asyncio.run(core_run()) diff --git a/experiments/cook/scene.json b/experiments/cook/scene.json deleted file mode 100644 index bf74d64d..00000000 --- a/experiments/cook/scene.json +++ /dev/null @@ -1,143 +0,0 @@ -{ - "reference_objects": - [ - { - "object_name": "test_box1", - "source": "pre-defined", - "O_T_TCP": [ - [1, 0, 0, 0.0], - [0, 1, 0, 0.0], - [0, 0, 1, -0.1], - [0, 0, 0, 1] - ] - }, - { - "object_name": "TestObject1", - "source": "mios" - }, - { - "object_name": "inwardgripper", - "source": "mios" - }, - { - "object_name": "outwardgripper", - "source": "mios" - }, - { - "object_name": "clampgripper", - "source": "mios" - }, - { - "object_name": "parallelgripper", - "source": "mios" - }, - { - "object_name": "defaultgripper", - "source": "mios" - }, - { - "object_name": "gear1", - "source": "mios" - }, - { - "object_name": "gear2", - "source": "mios" - }, - { - "object_name": "gear3", - "source": "mios" - }, - { - "object_name": "shaft1", - "source": "mios" - }, - { - "object_name": "shaft2", - "source": "mios" - }, - { - "object_name": "shaft3", - "source": "mios" - }, - { - "object_name": "gearbase", - "source": "mios" - }, - { - "object_name": "gearbase_hole1", - "source": "mios" - }, - { - "object_name": "gearbase_hole3", - "source": "mios" - } - ], - "tools": - [ - { - "tool_name": "clampgripper", - "EE_T_TCP": [ - [1, 0, 0, 0.0], - [0, 1, 0, 0.0], - [0, 0, 1, 0.085], - [0, 0, 0, 1] - ], - "EE_finger_width_max": 0.042, - "EE_finger_width_min": 0.01, - "load_width": 0.042, - "unload_width": 0.08 - }, - { - "tool_name": "parallelgripper", - "EE_T_TCP": [ - [1, 0, 0, 0.0], - [0, 1, 0, 0.0], - [0, 0, 1, 0.07], - [0, 0, 0, 1] - ], - "EE_finger_width_max": 0.042, - "EE_finger_width_min": 0.01, - "load_width": 0.042, - "unload_width": 0.08 - }, - { - "tool_name": "inwardgripper", - "EE_T_TCP": [ - [1, 0, 0, 0.0], - [0, 1, 0, 0.0], - [0, 0, 1, 0.095], - [0, 0, 0, 1] - ], - "EE_finger_width_max": 0.042, - "EE_finger_width_min": 0.01, - "load_width": 0.042, - "unload_width": 0.08 - }, - { - "tool_name": "outwardgripper", - "EE_T_TCP": [ - [1, 0, 0, 0.0], - [0, 1, 0, 0.0], - [0, 0, 1, 0.125], - [0, 0, 0, 1] - ], - "EE_finger_width_max": 0.042, - "EE_finger_width_min": 0.01, - "load_width": 0.042, - "unload_width": 0.08 - }, - { - "tool_name": "defaultgripper", - "EE_T_TCP": [ - [1, 0, 0, 0.0], - [0, 1, 0, 0.0], - [0, 0, 1, 0.0], - [0, 0, 0, 1] - ], - "EE_finger_width_max": 0.08, - "EE_finger_width_min": 0.01, - "load_width": 0.08, - "unload_width": 0.08 - } - ] -} diff --git a/experiments/cook/sk_gen_test.py b/experiments/cook/sk_gen_test.py deleted file mode 100644 index 72dba16e..00000000 --- a/experiments/cook/sk_gen_test.py +++ /dev/null @@ -1,621 +0,0 @@ -import json -import os -from pprint import pprint - -""" -test skeleton behavior tree generation. -the btw data structure should be updated. -""" - -os.environ["LANGCHAIN_TRACING_V2"] = "true" -os.environ["LANGCHAIN_ENDPOINT"] = "https://api.smith.langchain.com" -os.environ["LANGCHAIN_PROJECT"] = "default" - -from kios_bt.bt_stewardship import BehaviorTreeStewardship -from kios_scene.scene_factory import SceneFactory -from kios_bt.bt_factory import BehaviorTreeFactory -from kios_robot.robot_interface import RobotInterface -from kios_world.world_interface import WorldInterface - -from kios_agent.llm_supporter import KiosLLMSupporter -from kios_agent.data_types import KiosPromptSkeleton - -from kios_agent.kios_tools import ( - BehaviorTreeExecutorTool, - BehaviorTreeSimulatorTool, - WorldStateQueryTool, -) - -from typing import List, Tuple, Annotated, TypedDict -import operator - -from dotenv import load_dotenv - -from langchain import hub - -from langchain_openai import ChatOpenAI - -from langchain.chains.openai_functions import ( - create_structured_output_runnable, - create_openai_fn_runnable, -) -from langchain_core.utils.function_calling import convert_to_openai_function -from langchain_core.agents import AgentActionMessageLog, AgentFinish -from langchain_core.messages import HumanMessage -from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder -from langchain_core.output_parsers import JsonOutputParser, StrOutputParser -from langchain_core.pydantic_v1 import BaseModel, Field - -from langchain.prompts.pipeline import PipelinePromptTemplate -from langchain.prompts.prompt import PromptTemplate - -from langgraph.graph import StateGraph, END - -from langsmith import traceable - -load_dotenv() - -from kios_utils.pybt_test import generate_bt_stewardship, render_dot_tree -from kios_utils.pddl_problem_parser import parse_problem_init, parse_problem_objects - - -def render_bt(bt_json: json): - test_class = BehaviorTreeFactory() - bt = test_class.from_json_to_simple_bt(bt_json) - # bt = test_class.from_json_to_tree_root(bt_json) - bt_stewardship = generate_bt_stewardship(bt) - # bt_stewardship.setup(timeout=15) - render_dot_tree(bt_stewardship) - - -####################### dirs -current_dir = os.path.dirname(os.path.abspath(__file__)) -scene_path = os.path.join(current_dir, "scene.json") -# bt_json_file_path = os.path.join(current_dir, "behavior_tree.json") -world_state_path = os.path.join(current_dir, "world_state.json") -problem_path = os.path.join(current_dir, "gearset.problem") -domain_knowledge_path = os.path.join(current_dir, "domain_knowledge.txt") - -####################### problem -with open(problem_path, "r") as file: - problem = file.read() - -####################### scene -with open(scene_path, "r") as file: - scene_json_object = json.load(file) - -scene = SceneFactory().create_scene_from_json(scene_json_object) - -####################### world -world_interface = WorldInterface() -with open(world_state_path, "r") as file: - world_state_json_object = json.load(file) - world_interface.load_world_from_json(world_state_json_object) - -from_problem = parse_problem_init(problem=problem) -objects = parse_problem_objects(problem=problem) -world_interface.update_world(from_problem) - -world_state_json = world_interface.get_world_to_json() -pprint(from_problem) -pprint(world_state_json) -pprint(objects) - - -####################### robot -robot_interface = RobotInterface( - robot_address="127.0.0.1", - robot_port=12000, -) -robot_interface.setup_scene(scene) - -####################### bt_factory -bt_factory = BehaviorTreeFactory( - world_interface=world_interface, - robot_interface=robot_interface, -) - -####################### behavior_tree_stewardship -behavior_tree_stewardship = BehaviorTreeStewardship( - behaviortree_factory=bt_factory, - world_interface=world_interface, - robot_interface=robot_interface, -) - -####################### * tools -# bt_executor_tool = BehaviorTreeExecutorTool(behavior_tree_stewardship) - -bt_sim_tool = BehaviorTreeSimulatorTool(metadata={"bt_stw": behavior_tree_stewardship}) - -# world_state_query_tool = WorldStateQueryTool(behavior_tree_stewardship) - -executor_tools = [bt_sim_tool] - -# * kios data prompt skeleton dir -data_dir = os.environ.get("KIOS_DATA_DIR").format(username=os.getlogin()) -print(data_dir) -prompt_sk_dir = os.path.join(data_dir, "prompt_skeletons") -prompt_dir = os.path.join(data_dir, "prompts") - -# * refiner chain -llm_spt_re_sk = KiosLLMSupporter() -with open(os.path.join(prompt_sk_dir, "skeleton_refiner.json"), "r") as f: - re_sk_pptsk_json = json.load(f) - re_sk_pptsk = KiosPromptSkeleton.from_json(re_sk_pptsk_json) - llm_spt_re_sk.initialize_from_prompt_skeleton(re_sk_pptsk) -re_sk_ppt = llm_spt_re_sk.create_prompt() -re_sk_llm = ChatOpenAI(model_name=llm_spt_re_sk.model_name, temperature=0) - -re_sk_chain = re_sk_ppt | re_sk_llm | JsonOutputParser() - - -# * skeleton generator chain -llm_spt_skeleton_generator = KiosLLMSupporter() -with open(os.path.join(prompt_sk_dir, "skeleton_generator.json"), "r") as f: - skeleton_generator_pptsk_json = json.load(f) - skeleton_generator_pptsk = KiosPromptSkeleton.from_json( - skeleton_generator_pptsk_json - ) - llm_spt_skeleton_generator.initialize_from_prompt_skeleton(skeleton_generator_pptsk) -skeleton_generator_ppt = llm_spt_skeleton_generator.create_prompt() -skeleton_generator_llm = ChatOpenAI( - model_name=llm_spt_skeleton_generator.model_name, temperature=0 -) - -skeleton_generator_chain = ( - skeleton_generator_ppt - | skeleton_generator_llm - # | StrOutputParser() - | JsonOutputParser() -) - - -# * the graph state -class PlanExecuteState(TypedDict): - plan: List[str] - behavior_tree: dict - world_state: Annotated[ - List[dict], operator.add - ] # ! to add, you need to make world_state a list of dict - # = Field(default=None) - past_steps: Annotated[List[Tuple], operator.add] - response: str # ! not sure if use this or not. - user_input: str - problem: str - objects: dict[str, list[str]] - - -# * output schema of the planner -class SolutionSubtree(BaseModel): - """solution subtree that will be used to replace the failed condition node in the last behavior tree""" - - subtree: dict = Field( - description="a subtree that adheres the required format, which will be used to replace the failed condition node in the last behavior tree" - ) - - -behaviortree_file = os.path.join(prompt_dir, "rec_sk_gen/behaviortree.txt") -template_file = os.path.join(prompt_dir, "rec_sk_gen/template.txt") -task_file = os.path.join(prompt_dir, "rec_sk_gen/task.txt") -system_file = os.path.join(prompt_dir, "rec_sk_gen/system.txt") -object_file = os.path.join(prompt_dir, "rec_sk_gen/object.txt") -domain_file = os.path.join(prompt_dir, "rec_sk_gen/domain.txt") -example_file = os.path.join(prompt_dir, "rec_sk_gen/example.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {behaviortree} - - {domain} - - {example} - - {template} - """ -) - -planner_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ], -) - -planner = create_structured_output_runnable( - PlanExecuteState, - ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), - planner_ppt_ppl, -) - - -##################################################### * planner -# * output schema of the planner -class Plan(BaseModel): - """Plan to follow in future""" - - steps: List[str] = Field( - description="a list of different steps to follow, should be in sorted order" - ) - - -template_file = os.path.join(prompt_dir, "planner/template.txt") -task_file = os.path.join(prompt_dir, "planner/task.txt") -system_file = os.path.join(prompt_dir, "planner/system.txt") -domain_file = os.path.join(prompt_dir, "planner/domain.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {template} - """ -) - -planner_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ], -) - -planner = create_structured_output_runnable( - Plan, ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), planner_ppt_ppl -) - -##################################################### * plan_updaterner - -template_file = os.path.join(prompt_dir, "plan_updater/template.txt") -task_file = os.path.join(prompt_dir, "plan_updater/task.txt") -system_file = os.path.join(prompt_dir, "plan_updater/system.txt") -domain_file = os.path.join(prompt_dir, "plan_updater/domain.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {template} - """ -) - -plan_updater_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ], -) - -plan_updater = create_structured_output_runnable( - Plan, ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), plan_updater_ppt_ppl -) -# ! create openai fn runnable has bug - - -##################################################### * graph node functions -@traceable(name="user_input_node_step") -async def user_input_step(state: PlanExecuteState): - """ - get the input from the user - """ - print(f"-----user_input_step-----") - # if state["response"] is not None: - # user_input = input(state["response"]) - # state["response"] = None - # else: - user_input = input("Your next instruction: ") - - return { - "user_input": user_input, - } - - -@traceable(name="behavior_tree_generate_step") -async def behavior_tree_generate_step(state: PlanExecuteState): - """ - generate the behavior tree based on the instruction - """ - print(f"-----behavior_tree_generate_step-----") - - instruction = state["plan"][0] - latest_world_state = state["world_state"][-1] - objects = state["objects"] - - skeleton = await skeleton_generator_chain.ainvoke( - { - # "problem": state["problem"], - "objects": objects, - "world_state": latest_world_state, - "instructions": instruction, - } - ) - - # sk_json = JsonOutputParser().invoke(skeleton) - sk_json = skeleton - - behavior_tree_sk = sk_json["task_plan"]["behavior_tree"] - - render_bt(behavior_tree_sk) - - pause = input("paused here") - - # ! OBJECT CONCENTRATION - - # response = await re_sk_chain.ainvoke( - # { - # "input": skeleton, - # } - # ) - - behavior_tree = skeleton.get("task_plan").get("behavior_tree") - - # init_world_state = [skeleton.get("initial_state")] - - return { - "behavior_tree": behavior_tree, - # "world_state": init_world_state, # ! this is necessary because the constraints in problem - } - - -@traceable(name="behavior_tree_sim_step") -async def behavior_tree_sim_step(state: PlanExecuteState): - """ - simulate the behavior tree based on the world state - """ - print(f"-----behavior_tree_sim_step-----") - - behavior_tree = state["behavior_tree"] - latest_world_state = state["world_state"][-1] - - # exe_input = { - # "behavior_tree": behavior_tree, - # "world_state": latest_world_state, - # } - - # agent_response = await bt_sim_tool.ainvoke( - # { - # "input": exe_input, - # } - # ) - - response = behavior_tree_stewardship.sk_fake_run( - world_state=latest_world_state, bt_json=behavior_tree - ) - - # hasSucceeded = response["hasSucceeded"] - # new_world_state = [response["world_state"]] - hasSucceeded = response.result - new_world_state = [response.world_state] - the_step = state["plan"][0] - - return { - "past_steps": ( - the_step, - hasSucceeded, - ), # * only one because new plan will be generated and old steps are all removed - "world_state": new_world_state, - } - - -@traceable(name="behavior_tree_execute_step") -async def behavior_tree_execute_step(state: PlanExecuteState): - """ - execute the first step of the plan, append the result to the past steps - """ - print(f"-----behavior_tree_execute_step-----") - - behavior_tree = state["behavior_tree"] - latest_world_state = state["world_state"][-1] - - # exe_input = { - # "behavior_tree": behavior_tree, - # "world_state": latest_world_state, - # } - - # agent_response = await bt_exe_agent_executor.ainvoke( - # { - # "input": exe_input, - # } - # ) - - response = behavior_tree_stewardship.sk_fake_run( - world_state=latest_world_state, bt_json=behavior_tree - ) - - # hasSucceeded = response["hasSucceeded"] - # new_world_state = [response["world_state"]] - hasSucceeded = response.result - new_world_state = [response.world_state] - the_step = state["plan"][0] - - return { - "past_steps": ( - the_step, - hasSucceeded, - ), # * only one because new plan will be generated and old steps are all removed - "world_state": new_world_state, - } - - -@traceable(name="planner_step") -async def planner_step(state: PlanExecuteState): - """ - plan the steps based on user input and world state - """ - print(f"-----plan_step-----") - - plan = await planner.ainvoke( - { - "user_input": state["user_input"], - "world_state": state["world_state"], - } - ) - return {"plan": plan.steps} - - -@traceable(name="plan_updater_step") -async def plan_updater_step(state: PlanExecuteState): - """ - if return a response, then success, response the use, end. - otherwise, return the updated newplan (normally the same as the old plan with the first step popped out. - """ - print(f"-----plan_updater_step-----") - - # ! BOOOM! BUG! - output = await plan_updater.ainvoke( - { - "user_input": state["user_input"], - "plan": state["plan"], - "world_state": state["world_state"], - "past_steps": state["past_steps"], - } - ) - # if isinstance( - # output, UpdaterResponse - # ): # * determine if it is time to response and end - # return { - # "response": "Your last instruction has been finished." - # } # * Don't need to update. - # else: - return {"plan": output.steps} # * update the plan - - -##################################################### * construct the graph - -workflow = StateGraph(PlanExecuteState) - -workflow.add_node("planner", planner_step) - -workflow.add_node("behavior_tree_generator", behavior_tree_generate_step) - -workflow.add_node("behavior_tree_executor", behavior_tree_execute_step) - -workflow.add_node("plan_updater", plan_updater_step) - -workflow.add_node("user_input_node", user_input_step) - -workflow.set_entry_point("user_input_node") - -workflow.add_edge( - "planner", "behavior_tree_generator" -) # ! may need to add a condition here - -workflow.add_edge("behavior_tree_generator", "behavior_tree_executor") - -workflow.add_edge("behavior_tree_executor", "plan_updater") - - -def plan_updater_should_end(state: PlanExecuteState): - """ - end router - """ - print(f"-----plan_updater_should_end-----") - - # if state["response"]: - # return True - # else: - # return False - if state["plan"] == [] or len(state["plan"]) == 0: - return True - else: - return False - - -workflow.add_conditional_edges( - "plan_updater", - plan_updater_should_end, - { - True: "user_input_node", - False: "behavior_tree_generator", - }, -) - - -def user_input_should_end(state: PlanExecuteState): - """ - if the user input is empty, then end - """ - print(f"-----user_input_should_end-----") - - # print(f'user_input: {state["user_input"]}') - # print(f'world_state: {state["world_state"]}') - - if not state["user_input"] or state["user_input"] == "": - return True - else: - return False - - -workflow.add_conditional_edges( - "user_input_node", - user_input_should_end, - { - True: END, - False: "planner", - }, -) - -app = workflow.compile() - -config = {"recursion_limit": 500} - -inputs = { - "world_state": [world_state_json], - "objects": objects, -} - - -async def core_run(): - async for event in app.astream( - inputs, - config=config, - ): - for k, v in event.items(): - if k != "__end__": - print(v) - - -if __name__ == "__main__": - import asyncio - - asyncio.run(core_run()) diff --git a/experiments/cook/test_graph.py b/experiments/cook/test_graph.py deleted file mode 100644 index faf6dff7..00000000 --- a/experiments/cook/test_graph.py +++ /dev/null @@ -1,78 +0,0 @@ -import os -from typing import List, Tuple, Annotated, TypedDict -import operator - -""" -a test script for the functionality of langgraph -""" - -os.environ["LANGCHAIN_TRACING_V2"] = "true" -os.environ["LANGCHAIN_ENDPOINT"] = "https://api.smith.langchain.com" -os.environ["LANGCHAIN_PROJECT"] = "kios_agent" - -from langgraph.graph import StateGraph, END - -from langsmith import traceable - - -class PlanExecuteState(TypedDict): - plan: List[str] - behavior_tree_skeleton: dict # ! not sure if use this or not. - behavior_tree: dict - world_state: Annotated[ - List[dict], operator.add - ] # ! to add, you need to make world_state a list of dict - # = Field(default=None) - past_steps: Annotated[List[Tuple], operator.add] - response: str # ! not sure if use this or not. - # to_user: bool - user_input: str - problem: str - - -@traceable(name="user_input_node_step") -async def user_input_step(state: PlanExecuteState): - user_input = input("Enter your next step: ") - return { - "user_input": user_input, - } - - -@traceable(name="user_input_conditional_edge") -def user_input_should_continue(state: PlanExecuteState): - - if state["user_input"] == "gogogo": - return True - else: - return False - - -workflow = StateGraph(PlanExecuteState) - -workflow.add_node("user_input_node", user_input_step) - -workflow.set_entry_point("user_input_node") - -workflow.add_conditional_edges( - "user_input_node", - user_input_should_continue, - { - True: "user_input_node", - False: END, - }, -) - -app = workflow.compile() - - -async def core_run(): - async for event in app.astream(input={}): - for k, v in event.items(): - if k != "__end__": - print(v) - - -if __name__ == "__main__": - import asyncio - - asyncio.run(core_run()) diff --git a/experiments/cook/ut_test.py b/experiments/cook/ut_test.py deleted file mode 100644 index dc8e0e93..00000000 --- a/experiments/cook/ut_test.py +++ /dev/null @@ -1,1207 +0,0 @@ -import json -import os -import re -from pprint import pprint -from typing import List, Tuple, Annotated, TypedDict -import operator -import logging -import asyncio - -""" -node expanding method script. -run the algo to generate the tree for current goal, forecast the future world state, and find the new goal to expand. -currently not a langgraph. -""" - -os.environ["LANGCHAIN_TRACING_V2"] = "true" -os.environ["LANGCHAIN_ENDPOINT"] = "https://api.smith.langchain.com" -os.environ["LANGCHAIN_PROJECT"] = "kios_agent" - -from kios_bt.bt_stewardship import BehaviorTreeStewardship -from kios_scene.scene_factory import SceneFactory -from kios_bt.bt_factory import BehaviorTreeFactory -from kios_robot.robot_interface import RobotInterface -from kios_world.world_interface import WorldInterface - -from kios_agent.llm_supporter import KiosLLMSupporter -from kios_agent.data_types import KiosPromptSkeleton - -from kios_agent.kios_tools import ( - BehaviorTreeExecutorTool, - BehaviorTreeSimulatorTool, - WorldStateQueryTool, -) - -from dotenv import load_dotenv - -from langchain import hub - -from langchain_openai import ChatOpenAI - -from langchain.chains.openai_functions import ( - create_structured_output_runnable, - create_openai_fn_runnable, -) -from langchain_core.utils.function_calling import convert_to_openai_function -from langchain_core.agents import AgentActionMessageLog, AgentFinish -from langchain_core.messages import HumanMessage -from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder -from langchain_core.output_parsers import JsonOutputParser, StrOutputParser -from langchain_core.pydantic_v1 import BaseModel, Field - -from langchain.agents.format_scratchpad import format_to_openai_function_messages -from langchain.agents import create_openai_functions_agent, AgentExecutor - -from langchain.prompts.pipeline import PipelinePromptTemplate -from langchain.prompts.prompt import PromptTemplate - -from langchain_community.document_loaders import TextLoader -from langchain_openai import OpenAIEmbeddings - -from langchain_community.vectorstores import FAISS -from langchain.text_splitter import ( - RecursiveCharacterTextSplitter, - CharacterTextSplitter, -) - -from langgraph.graph import StateGraph, END - -from langsmith import traceable - -load_dotenv() - -from kios_utils.pybt_test import generate_bt_stewardship, render_dot_tree -from kios_utils.pddl_problem_parser import parse_problem_init, parse_problem_objects - - -def render_bt(bt_json: dict): - test_class = BehaviorTreeFactory() - bt = test_class.from_json_to_simple_bt(bt_json) - # bt = test_class.from_json_to_tree_root(bt_json) - bt_stewardship = generate_bt_stewardship(bt) - # bt_stewardship.setup(timeout=15) - render_dot_tree(bt_stewardship) - - -####################### dirs -current_dir = os.path.dirname(os.path.abspath(__file__)) -scene_path = os.path.join(current_dir, "scene.json") -# bt_json_file_path = os.path.join(current_dir, "behavior_tree.json") -world_state_path = os.path.join(current_dir, "world_state.json") -problem_path = os.path.join(current_dir, "gearset.problem") -domain_knowledge_path = os.path.join(current_dir, "domain_knowledge.txt") - -####################### problem -with open(problem_path, "r") as file: - problem = file.read() - -####################### scene -with open(scene_path, "r") as file: - scene_json_object = json.load(file) - -scene = SceneFactory().create_scene_from_json(scene_json_object) - -####################### world -world_interface = WorldInterface() -with open(world_state_path, "r") as file: - world_state_json_object = json.load(file) - world_interface.load_world_from_json(world_state_json_object) - -from_problem = parse_problem_init(problem=problem) -objects = parse_problem_objects(problem=problem) -world_interface.update_world(from_problem) - -world_state_json = world_interface.get_world_to_json() -pprint(from_problem) -pprint(world_state_json) -pprint(objects) - - -####################### robot -robot_interface = RobotInterface( - robot_address="127.0.0.1", - robot_port=12000, -) -robot_interface.setup_scene(scene) - -####################### bt_factory -bt_factory = BehaviorTreeFactory( - world_interface=world_interface, - robot_interface=robot_interface, -) - -####################### behavior_tree_stewardship -behavior_tree_stewardship = BehaviorTreeStewardship( - behaviortree_factory=bt_factory, - world_interface=world_interface, - robot_interface=robot_interface, -) - -####################### * tools -# bt_executor_tool = BehaviorTreeExecutorTool(behavior_tree_stewardship) - -bt_sim_tool = BehaviorTreeSimulatorTool(metadata={"bt_stw": behavior_tree_stewardship}) - -# world_state_query_tool = WorldStateQueryTool(behavior_tree_stewardship) - -executor_tools = [bt_sim_tool] - -# * kios data prompt skeleton dir -data_dir = os.environ.get("KIOS_DATA_DIR").format(username=os.getlogin()) -print(data_dir) -prompt_sk_dir = os.path.join(data_dir, "prompt_skeletons") -prompt_dir = os.path.join(data_dir, "prompts") - - -# * the graph state -class PlanExecuteState(TypedDict): - plan: List[str] - behavior_tree: dict - world_state: Annotated[ - List[dict], operator.add - ] # ! to add, you need to make world_state a list of dict - past_steps: Annotated[List[Tuple], operator.add] - user_input: str - objects: dict[str, list[str]] - last_behavior_tree: dict - last_failed_node: dict - runtime_world_state: dict - behavior_tree_execution_summary: str - BTExecutionHasSucceeded: bool - - action_sequence: List[str] - - -########## verifier - -template_file = os.path.join(prompt_dir, "seq_planner/template.txt") -task_file = os.path.join(prompt_dir, "seq_planner/task.txt") -system_file = os.path.join(prompt_dir, "seq_planner/system.txt") -domain_file = os.path.join(prompt_dir, "seq_planner/domain.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {template} - """ -) - -seq_planner_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ], -) - -seq_planner_chain = ( - seq_planner_ppt_ppl - | ChatOpenAI(model="gpt-4-turbo-preview", temperature=0) - | StrOutputParser() -) - - -# * output schema of the recursive_sk_generator -class SolutionBehaviortree(BaseModel): - """the fixed behavior tree""" - - behavior_tree: dict = Field( - description="the behavior tree that adheres the required format." - ) - - -behaviortree_file = os.path.join(prompt_dir, "rec_sk_gen/behaviortree.txt") -template_file = os.path.join(prompt_dir, "rec_sk_gen/template.txt") -task_file = os.path.join(prompt_dir, "rec_sk_gen/task.txt") -system_file = os.path.join(prompt_dir, "rec_sk_gen/system.txt") -object_file = os.path.join(prompt_dir, "rec_sk_gen/object.txt") -domain_file = os.path.join(prompt_dir, "rec_sk_gen/domain.txt") -example_file = os.path.join(prompt_dir, "rec_sk_gen/example.txt") -state_file = os.path.join(prompt_dir, "rec_sk_gen/state.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) -with open(object_file, "r") as f: - object_ppt = PromptTemplate.from_template(f.read()) -with open(behaviortree_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - behaviortree_ppt = ppt_tmp.partial(input=f.read()) - -with open(state_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - state_ppt = ppt_tmp.partial(input=f.read()) - -with open(example_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - example_ppt = ppt_tmp.partial(input=f.read()) -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {state} - - {object} - - {behaviortree} - - {example} - - {template} - """ -) - -re_sk_gen_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ("object", object_ppt), - ("behaviortree", behaviortree_ppt), - ("example", example_ppt), - ("state", state_ppt), - ], -) - -rec_sk_gen_chain = ( - re_sk_gen_ppt_ppl - | ChatOpenAI(model="gpt-4-turbo-preview", temperature=0) - | JsonOutputParser() -) - -# recursive_sk_generator = create_structured_output_runnable( -# SolutionBehaviortree, -# ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), -# re_sk_gen_ppt_ppl, -# mode="openai-json", -# ) - - -##################################################### * planner -# * output schema of the planner -class Plan(BaseModel): - """Plan to follow in future""" - - steps: List[str] = Field( - description="a list of different steps to follow, should be in sorted order" - ) - - -template_file = os.path.join(prompt_dir, "planner/template.txt") -task_file = os.path.join(prompt_dir, "planner/task.txt") -system_file = os.path.join(prompt_dir, "planner/system.txt") -domain_file = os.path.join(prompt_dir, "planner/domain.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {template} - """ -) - -planner_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ], -) - -planner = create_structured_output_runnable( - Plan, ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), planner_ppt_ppl -) - -##################################################### * plan_updaterner - -template_file = os.path.join(prompt_dir, "plan_updater/template.txt") -task_file = os.path.join(prompt_dir, "plan_updater/task.txt") -system_file = os.path.join(prompt_dir, "plan_updater/system.txt") -domain_file = os.path.join(prompt_dir, "plan_updater/domain.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {template} - """ -) - -plan_updater_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ], -) - -plan_updater = create_structured_output_runnable( - Plan, ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), plan_updater_ppt_ppl -) -# ! create openai fn runnable has bug - - -##################################################### * graph node functions -@traceable(name="user_input_node_step") -async def user_input_step(state: PlanExecuteState): - """ - get the input from the user - """ - print(f"-----user_input_step-----") - - user_input = input("Your next instruction: ") - - return { - "user_input": user_input, - } - - -@traceable(name="sequence_generate_step") -async def sequence_generate_step(state: PlanExecuteState): - """ - generate the sequence based on the instruction - """ - print(f"-----sequence_generate_step-----") - - instruction = state["plan"][0] - start_world_state = state["world_state"][-1] - objects = state["objects"] - - action_sequence = await seq_planner_chain.ainvoke( - { - "objects": objects, - "start_world_state": start_world_state, - "user_instruction": instruction, - } - ) - - return { - "action_sequence": action_sequence, - } - - -@traceable(name="behavior_tree_generate_step") -async def behavior_tree_generate_step(state: PlanExecuteState): - """ - generate the behavior tree based on the instruction - """ - print(f"-----behavior_tree_generate_step-----") - - instruction = state["plan"][0] - if state["runtime_world_state"] is None: - runtime_world_state = state["world_state"][-1] - else: - runtime_world_state = state["runtime_world_state"] - objects = state["objects"] - - bt_skeleton = await rec_sk_gen_chain.ainvoke( - { - "objects": objects, - "runtime_world_state": runtime_world_state, - "last_behavior_tree": state["last_behavior_tree"], - "last_failed_node": state["last_failed_node"], - "summary": state["behavior_tree_execution_summary"], - "action_sequence": state["action_sequence"], - } - ) - - render_bt(bt_skeleton) - - pause = input("paused here") - - return { - "last_behavior_tree": bt_skeleton, - "BTExecutionHasSucceeded": False, - } - - -@traceable(name="behavior_tree_execute_step") -async def behavior_tree_execute_step(state: PlanExecuteState): - """ - execute the first step of the plan, append the result to the past steps - """ - print(f"-----behavior_tree_execute_step-----") - - this_step = state["plan"][0] - behavior_tree_skeleton = state["last_behavior_tree"] - latest_world_state = state["world_state"][-1] - - # * first sim run - tree_result, skeleton_json = behavior_tree_stewardship.sk_sim_run( - world_state=latest_world_state, skeleton_json=behavior_tree_skeleton - ) - - pprint(tree_result.to_json()) - pause = input("paused here") - - # * check result - if tree_result.result == "success": - - # * second fake run - tree_result, skeleton_json = behavior_tree_stewardship.sk_fake_run( - world_state=latest_world_state, skeleton_json=behavior_tree_skeleton - ) - - pprint(tree_result.to_json()) - pause = input("paused here") - - if tree_result.result == "success": - return { - "BTExecutionHasSucceeded": True, - "past_steps": ( - this_step, - tree_result.result, - ), # * only one because new plan will be generated and old steps are all removed - "world_state": [tree_result.world_state], - # * clear all - "last_behavior_tree": None, - "last_failed_node": None, - "runtime_world_state": None, - "behavior_tree_execution_summary": None, - } - else: - return { - "world_state": [tree_result.world_state], # * real world state - "last_behavior_tree": skeleton_json, - "last_failed_node": { - "summary": tree_result.final_node.get("summary"), - "name": tree_result.final_node.get("name"), - "identifier": tree_result.final_node.get("identifier"), - }, - "runtime_world_state": tree_result.world_state, - "behavior_tree_execution_summary": tree_result.summary, - } - - else: - return { - # ! do not change world state - "last_behavior_tree": skeleton_json, - "last_failed_node": { - "summary": tree_result.final_node.get("summary"), - "name": tree_result.final_node.get("name"), - "identifier": tree_result.final_node.get("identifier"), - }, - "runtime_world_state": tree_result.world_state, - "behavior_tree_execution_summary": tree_result.summary, - } - - -@traceable(name="planner_step") -async def planner_step(state: PlanExecuteState): - """ - plan the steps based on user input and world state - """ - print(f"-----plan_step-----") - - plan = await planner.ainvoke( - { - "user_input": state["user_input"], - "world_state": state["world_state"], - } - ) - return {"plan": plan.steps} - - -@traceable(name="plan_updater_step") -async def plan_updater_step(state: PlanExecuteState): - """ - if return a response, then success, response the use, end. - otherwise, return the updated newplan (normally the same as the old plan with the first step popped out. - """ - print(f"-----plan_updater_step-----") - - # ! BOOOM! BUG! - output = await plan_updater.ainvoke( - { - "user_input": state["user_input"], - "plan": state["plan"], - "world_state": state["world_state"], - "past_steps": state["past_steps"], - } - ) - # if isinstance( - # output, UpdaterResponse - # ): # * determine if it is time to response and end - # return { - # "response": "Your last instruction has been finished." - # } # * Don't need to update. - # else: - return {"plan": output.steps} # * update the plan - - -##################################################### * construct the graph - -workflow = StateGraph(PlanExecuteState) - -workflow.add_node("planner", planner_step) -workflow.add_node("sequence_generator", sequence_generate_step) -workflow.add_node("behavior_tree_generator", behavior_tree_generate_step) -workflow.add_node("behavior_tree_executor", behavior_tree_execute_step) -workflow.add_node("plan_updater", plan_updater_step) -workflow.add_node("user_input_node", user_input_step) -workflow.set_entry_point("user_input_node") -workflow.add_edge("planner", "sequence_generator") -workflow.add_edge("sequence_generator", "behavior_tree_generator") -workflow.add_edge("behavior_tree_generator", "behavior_tree_executor") - - -def executor_should_end(state: PlanExecuteState): - """ - end router - """ - print(f"-----executor_should_end-----") - - if state["BTExecutionHasSucceeded"] == True: - state["BTExecutionHasSucceeded"] = False - return True - else: - return False - - -workflow.add_conditional_edges( - "behavior_tree_executor", - executor_should_end, - { - True: "plan_updater", - False: "behavior_tree_generator", - }, -) - - -def plan_updater_should_end(state: PlanExecuteState): - """ - end router - """ - print(f"-----plan_updater_should_end-----") - - if state["plan"] == [] or len(state["plan"]) == 0: - return True - else: - return False - - -workflow.add_conditional_edges( - "plan_updater", - plan_updater_should_end, - { - True: "user_input_node", - False: "behavior_tree_generator", - }, -) - - -def user_input_should_end(state: PlanExecuteState): - """ - if the user input is empty, then end - """ - print(f"-----user_input_should_end-----") - - if not state["user_input"] or state["user_input"] == "": - return True - else: - return False - - -workflow.add_conditional_edges( - "user_input_node", - user_input_should_end, - { - True: END, - False: "planner", - }, -) - -app = workflow.compile() - -config = {"recursion_limit": 500} - -inputs = { - "world_state": [world_state_json], - "objects": objects, -} - -# * unit tree generator ppl -system_file = os.path.join(prompt_dir, "new/ut_gen/system.txt") -task_file = os.path.join(prompt_dir, "new/ut_gen/task.txt") -domain_file = os.path.join(prompt_dir, "new/ut_gen/new_domain_nl.txt") -behaviortree_file = os.path.join(prompt_dir, "new/ut_gen/new_behaviortree.txt") -template_file = os.path.join(prompt_dir, "new/ut_gen/template.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) -with open(behaviortree_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - behaviortree_ppt = ppt_tmp.partial(input=f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {behaviortree} - - {template} - - {format_instructions} - """ -) - -parser = JsonOutputParser() - -format_instructions = PromptTemplate.from_template("""{input}""").partial( - input=parser.get_format_instructions() -) - -ut_gen_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ("behaviortree", behaviortree_ppt), - ("format_instructions", format_instructions), - ], -) - -ut_gen_chain = ( - ut_gen_ppt_ppl - # | ChatOpenAI(model="gpt-3.5-turbo-0125", temperature=0) - # | ChatOpenAI(model="ft:gpt-3.5-turbo-0125:kifabrik-mirmi::8y1cXwVw", temperature=0) - | ChatOpenAI( - model="ft:gpt-3.5-turbo-0125:kifabrik-mirmi:kios-ut-gen-v2:8z2KbPsr", - temperature=0, - ) - # | ChatOpenAI(model="gpt-4", temperature=0) - | JsonOutputParser() -) - -# ! sequence planner estimation ppl -system_file = os.path.join(prompt_dir, "seq_planner_est/system.txt") -task_file = os.path.join(prompt_dir, "seq_planner_est/task.txt") -domain_file = os.path.join(prompt_dir, "seq_planner_est/domain.txt") -state_file = os.path.join(prompt_dir, "seq_planner_est/state.txt") -output_format_file = os.path.join(prompt_dir, "seq_planner_est/output_format.txt") -template_file = os.path.join(prompt_dir, "seq_planner_est/template.txt") -example_file = os.path.join(prompt_dir, "seq_planner_est/example.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) -with open(output_format_file, "r") as f: - output_format_ppt = PromptTemplate.from_template(f.read()) -with open(example_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - example_ppt = ppt_tmp.partial(input=f.read()) -with open(state_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - state_ppt = ppt_tmp.partial(input=f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {state} - - {output_format} - - {example} - - {template} - - {format_instructions} - """ -) - -parser = JsonOutputParser() - -format_instructions = PromptTemplate.from_template("""{input}""").partial( - input=parser.get_format_instructions() -) - -seq_planner_est_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ("state", state_ppt), - ("output_format", output_format_ppt), - ("format_instructions", format_instructions), - ("example", example_ppt), - ], -) - -seq_planner_est_chain = ( - seq_planner_est_ppt_ppl - # | ChatOpenAI(model="gpt-4", temperature=0) - | ChatOpenAI(model="gpt-3.5-turbo-0125", temperature=0) - | JsonOutputParser() -) - - -def seq_planner_est_test(): - return seq_planner_est_chain.invoke( - { - "start_world_state": world_state_json, - # "target": "is_inserted_to(shaft1, gearbase_hole1)", - "target": "hold(left_hand, outward_claw)", - } - ) - - -# * sequence action planner -system_file = os.path.join(prompt_dir, "new/seq_plan/system.txt") -task_file = os.path.join(prompt_dir, "new/seq_plan/task.txt") -domain_file = os.path.join(prompt_dir, "new/seq_plan/new_domain_nl.txt") -state_file = os.path.join(prompt_dir, "new/seq_plan/state.txt") -output_format_file = os.path.join(prompt_dir, "new/seq_plan/output_format.txt") -template_file = os.path.join(prompt_dir, "new/seq_plan/template.txt") -example_file = os.path.join(prompt_dir, "new/seq_plan/new_example.txt") -chain_file = os.path.join(prompt_dir, "new/seq_plan/chain.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) -with open(output_format_file, "r") as f: - output_format_ppt = PromptTemplate.from_template(f.read()) -with open(example_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - example_ppt = ppt_tmp.partial(input=f.read()) -with open(state_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - state_ppt = ppt_tmp.partial(input=f.read()) -# # * this is for gpt 3.5 -# with open(chain_file, "r") as f: -# chain_ppt = PromptTemplate.from_template(f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {state} - - {output_format} - - {example} - - {template} - - {format_instructions} - """ -) - -parser = JsonOutputParser() - -format_instructions = PromptTemplate.from_template("""{input}""").partial( - input=parser.get_format_instructions() -) - -seq_ac_pl_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ("state", state_ppt), - ("output_format", output_format_ppt), - ("format_instructions", format_instructions), - ("example", example_ppt), - # ("chain", chain_ppt), - ], -) - -seq_ac_pl_chain = ( - seq_ac_pl_ppt_ppl - | ChatOpenAI(model="gpt-4", temperature=0) - # | ChatOpenAI(model="gpt-3.5-turbo-0125", temperature=0) - | JsonOutputParser() -) - -# * state estimater -system_file = os.path.join(prompt_dir, "new/state_est/system.txt") -task_file = os.path.join(prompt_dir, "new/state_est/task.txt") -domain_file = os.path.join(prompt_dir, "new/state_est/new_domain_nl.txt") -state_file = os.path.join(prompt_dir, "new/state_est/state.txt") -output_format_file = os.path.join(prompt_dir, "new/state_est/output_format.txt") -template_file = os.path.join(prompt_dir, "new/state_est/template.txt") -example_file = os.path.join(prompt_dir, "new/state_est/new_example.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) -with open(output_format_file, "r") as f: - output_format_ppt = PromptTemplate.from_template(f.read()) -with open(example_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - example_ppt = ppt_tmp.partial(input=f.read()) -with open(state_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - state_ppt = ppt_tmp.partial(input=f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {state} - - {output_format} - - {example} - - {template} - - {format_instructions} - """ -) - -parser = JsonOutputParser() - -format_instructions = PromptTemplate.from_template("""{input}""").partial( - input=parser.get_format_instructions() -) - -state_est_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ("state", state_ppt), - ("output_format", output_format_ppt), - ("format_instructions", format_instructions), - ("example", example_ppt), - ], -) - - -state_est_ppt_ppl_chain = ( - state_est_ppt_ppl - # | ChatOpenAI(model="gpt-3.5-turbo", temperature=0) - | ChatOpenAI(model="gpt-4", temperature=0) - | JsonOutputParser() -) - - -async def core_run(): - async for event in app.astream( - inputs, - config=config, - ): - for k, v in event.items(): - if k != "__end__": - print(v) - - -def ut_gen_test(): - ut = ut_gen_chain.invoke( - { - # "action": "insert(left_hand, defaultgripper, gear1, shaft1)", - # "action": "insert(left_hand, parallel_box1, shaft1, gearbase_hole1)", - # "action": "change_tool(left_hand, parallel_box1, defaultgripper)", - "action": "insert(left_hand, clampgripper, gear2, shaft2)", - } - ) - - render_bt(ut) - - -def seq_planner_est_test(): - return seq_planner_est_chain.invoke( - { - "start_world_state": world_state_json, - "target": "is_inserted_to(shaft1, gearbase_hole1)", - # "target": "hold(left_hand, outward_claw)", - } - ) - - -def seq_action_plan_test(): - return seq_ac_pl_chain.invoke( - { - "start_world_state": world_state_json, - "target": "is_inserted_to(shaft1, gearbase_hole1)", - # "target": "hold(left_hand, outward_claw)", - } - ) - - -def state_est_test(): - return state_est_ppt_ppl_chain.invoke( - { - "start_world_state": world_state_json, - "action_plan": [ - "unload_tool(left_hand, outward_claw)", - "load_tool(left_hand, parallel_box1)", - "pick_up(left_hand, parallel_box1, shaft1)", - "insert(left_hand, parallel_box1, shaft1, gearbase_hole1)", - ], - } - ) - - -##################################### * speed imp -@traceable(name="make_plan") -def make_plan(state: dict, goal: str) -> list[str]: - print(f"----------start to make plan for the goal {goal}") - response = seq_ac_pl_chain.invoke( - { - "start_world_state": state, - "target": goal, - } - ) - print(f"finished making plan for the goal {goal}.") - pprint(f'LLM thought flow: {response["explanation"]}') - return response["task_plan"] - - -@traceable(name="estimate_state") -def estimate_state(start_world_state: dict, action_plan: list[str]) -> dict: - print("----------start to estimate the state after the action plan:") - pprint(action_plan) - response = state_est_ppt_ppl_chain.invoke( - { - "start_world_state": start_world_state, - "action_plan": action_plan, - } - ) - print(f"finished estimating the state after the action plan {action_plan}.") - return response["estimated_world_state"] - - -@traceable(name="generate_unit_subtree") -def generate_unit_subtree(action: str) -> dict: - print("----------start to generate the unit subtree for the action") - pprint(action) - response = ut_gen_chain.invoke( - { - "action": action, - } - ) - print(f"finished generating the unit subtree for the action {action}.") - return response - - -def get_node_list_from_tree(unit_subtree: dict) -> list[dict]: - children = unit_subtree["children"][1][ - "children" - ] # * the second child is a sequence - return children - - -def extract_goal(node: dict) -> str: - name = node["name"] - - -def match_type(node: dict) -> tuple[str, str]: - node_name = node["name"] - match = re.search( - r"(selector|sequence|action|precondition|condition|target):\s*(.+)", node_name - ) - if match: - node_type = match.group(1) - node_body = match.group(2) - return node_type, node_body - else: - raise ValueError(f"the node name {node_name} does not match any type.") - - -def expand_nodes( - node_list: list[dict], - start_state: dict, - overall_tree: list[dict] = None, -) -> dict: - """ - in order to monitor the tree generation, the overall tree and the node list should be the same variable when being passed in. - """ - pprint("----------check the entire tree:") - if overall_tree is not None: - render_bt(overall_tree[0]) - pprint("----------start to expand the node list:") - pprint(node_list) - pause = input("paused here! check the tree.") - - assert len(node_list) > 0 - state = start_state - - for i in range(len(node_list)): - type_name, body = match_type(node_list[i]) - # if match_type(node_list[i]) == "action": - if type_name == "action": - print(f"the node {node_list[i]['name']} is an action node. skip it.") - pause = input("paused here! check!") - # elif match_type(node_list[i]) == "precondition" or "target": - elif type_name in ["precondition", "target"]: - # goal = node_list[i]["name"] - goal = body - plan = make_plan(state, goal) - if len(plan) == 0: - logging.warning(f"No action should be performed for the goal {goal}.") - logging.warning(f'the node {node_list[i]["name"]} has been skipped.') - pause = input("paused here! check!") - else: - logging.info(f"Actions have been planed for the goal {goal}.") - pprint(f"the plan for the goal {goal} is {plan}") - pause = input("paused here! check!") - last_action = plan[-1] - unit_subtree = generate_unit_subtree(last_action) - # insert the subtree into the node_list - node_list[i] = unit_subtree - new_node_list = get_node_list_from_tree(unit_subtree) - expand_nodes( - node_list=new_node_list, - start_state=state, - overall_tree=overall_tree, - ) - state = estimate_state(state, plan) - - return node_list[0] - - -def embed_ut_nl(unit_subtree: dict) -> str: - """ - embed the unit subtree into the overall tree - """ - selector_children = unit_subtree["children"] - target = "" - # * target - for node in selector_children: - if match_type(node)[0] == "target": - target += node["summary"] - - sequence_children = selector_children[1]["children"] - preconditions = [] - action = "" - for node in sequence_children: - if match_type(node)[0] == "precondition": - preconditions.append(node["summary"]) - if match_type(node)[0] == "action": - action = node["summary"] - - embedding = f"if {' and '.join(preconditions)} then {action}, {target}" - - return embedding - - -def test_embedding_nl(): - unit_tree = ut_gen_test() - pprint(unit_tree) - nl = embed_ut_nl(unit_tree) - print(nl) - - -def test_expand_nodes(): - start_state = world_state_json - node_list = [ - { - "summary": "insert shaft1 into gearbase hole1", - "name": "target: insert shaft1 into gearbase hole1", - } - ] - # node_list = [ - # { - # "summary": "insert gear2 into shaft2", - # "name": "target: insert gear2 into shaft2", - # } - # ] - # node_list = [ - # { - # "summary": "pick up the shaft1", - # "name": "target: pick up the shaft1", - # }, - # ] - expand_nodes(node_list, start_state, node_list) - pprint(node_list) - - -if __name__ == "__main__": - pass - - # asyncio.run(core_run()) - # pprint(ut_gen_test()) - # pprint(seq_action_plan_test()) - # pprint(state_est_test()) - - test_expand_nodes() - - # test_embedding_nl() diff --git a/experiments/cook/visualize_cheat.py b/experiments/cook/visualize_cheat.py deleted file mode 100644 index 0e840594..00000000 --- a/experiments/cook/visualize_cheat.py +++ /dev/null @@ -1,1080 +0,0 @@ -# -''' -script to visualize the tree. -a good place to go if you don't know how to do the right thing with the btw yourself. -''' -# * this is a full tree example. -result_1 = { - "behavior_tree": { - "summary": "selector to insert gear2 into shaft2", - "name": "selector: insert(left_hand, outward_claw, gear2, shaft2)", - "identifier": 1, - "type_name": "selector", - "children": [ - { - "summary": "condition node to check if gear2 is inserted to shaft2", - "name": "target: is_inserted_to(gear2, shaft2)", - "identifier": 2, - "type_name": "condition", - "conditions": [ - { - "object_name": "gear2", - "property_name": "is_inserted_to", - "property_value": "shaft2", - "status": True, - } - ], - }, - { - "summary": "sequence to insert gear2 into shaft2", - "name": "sequence: insert(left_hand, outward_claw, gear2, shaft2)", - "identifier": 3, - "type_name": "sequence", - "children": [ - { - "summary": "selector to load outward_claw", - "name": "selector: load_tool(left_hand, outward_claw)", - "identifier": 4, - "type_name": "selector", - "children": [ - { - "summary": "condition node to check if left_hand holds outward_claw", - "name": "target: hold(left_hand, outward_claw)", - "identifier": 5, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "outward_claw", - "status": True, - } - ], - }, - { - "summary": "sequence to load outward_claw", - "name": "sequence: load_tool(left_hand, outward_claw)", - "identifier": 6, - "type_name": "sequence", - "children": [ - { - "summary": "condition node to check if outward_claw is equippable", - "name": "precondition: is_equippable(outward_claw)", - "identifier": 7, - "type_name": "condition", - "conditions": [ - { - "object_name": "outward_claw", - "property_name": "is_equippable", - "property_value": None, - "status": True, - } - ], - }, - { - "summary": "condition node to check if left hand is free", - "name": "precondition: is_free(left_hand)", - "identifier": 8, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "is_free", - "property_value": None, - "status": True, - } - ], - }, - { - "summary": "action node to equip outward_claw to left hand", - "name": "action: load_tool(left_hand, outward_claw)", - "identifier": 9, - "type_name": "action", - "effects": [ - { - "object_name": "left_hand", - "property_name": "is_free", - "property_value": None, - "status": False, - }, - { - "object_name": "outward_claw", - "property_name": "is_equippable", - "property_value": None, - "status": False, - }, - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "outward_claw", - "status": True, - }, - ], - }, - ], - }, - ], - }, - { - "summary": "selector to use left_hand with outward_claw to pick up gear2", - "name": "selector: pick_up(left_hand, outward_claw, gear2)", - "identifier": 10, - "type_name": "selector", - "children": [ - { - "summary": "condition node to check if outward_claw holds gear2", - "name": "target: hold(outward_claw, gear2)", - "identifier": 11, - "type_name": "condition", - "conditions": [ - { - "object_name": "outward_claw", - "property_name": "hold", - "property_value": "gear2", - "status": True, - } - ], - }, - { - "summary": "sequence to use left hand with outward_claw to pick up gear2", - "name": "sequence: pick_up(left_hand, outward_claw, gear2)", - "identifier": 12, - "type_name": "sequence", - "children": [ - { - "summary": "condition node to check if outward_claw is free", - "name": "precondition: is_free(outward_claw)", - "identifier": 13, - "type_name": "condition", - "conditions": [ - { - "object_name": "outward_claw", - "property_name": "is_free", - "property_value": None, - "status": True, - } - ], - }, - { - "summary": "condition node to check if left hand holds outward_claw", - "name": "precondition: hold(left_hand, outward_claw)", - "identifier": 14, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "outward_claw", - "status": True, - } - ], - }, - { - "summary": "action node to use left hand with outward_claw to pick up gear2", - "name": "action: pick_up(left_hand, outward_claw, gear2)", - "identifier": 15, - "type_name": "action", - "effects": [ - { - "object_name": "outward_claw", - "property_name": "hold", - "property_value": "gear2", - "status": True, - }, - { - "object_name": "outward_claw", - "property_name": "is_free", - "property_value": None, - "status": False, - }, - ], - }, - ], - }, - ], - }, - { - "summary": "condition node to check if gear2 can be inserted to shaft2", - "name": "precondition: can_insert_to(gear2, shaft2)", - "identifier": 16, - "type_name": "condition", - "conditions": [ - { - "object_name": "gear2", - "property_name": "can_insert_to", - "property_value": "shaft2", - "status": True, - } - ], - }, - { - "summary": "action node to use left_hand with outward_claw to insert gear2 to shaft2", - "name": "action: insert(left_hand, outward_claw, gear2, shaft2)", - "identifier": 17, - "type_name": "action", - "effects": [ - { - "object_name": "outward_claw", - "property_name": "hold", - "property_value": "gear2", - "status": False, - }, - { - "object_name": "outward_claw", - "property_name": "is_free", - "property_value": None, - "status": True, - }, - { - "object_name": "gear2", - "property_name": "is_inserted_to", - "property_value": "shaft2", - "status": True, - }, - ], - }, - ], - }, - ], - }, - "world_state": [ - { - "objects": [ - {"name": "parallel_box1", "properties": ["is_free"]}, - {"name": "shaft1", "properties": []}, - {"name": "gearbase_hole1", "properties": []}, - {"name": "left_hand", "properties": []}, - ], - "constraints": [ - { - "source": "parallel_box1", - "name": "can_manipulate", - "target": "shaft1", - }, - { - "source": "shaft1", - "name": "can_insert_to", - "target": "gearbase_hole1", - }, - ], - "relations": [ - {"source": "left_hand", "name": "hold", "target": "parallel_box1"}, - { - "source": "shaft1", - "name": "is_inserted_to", - "target": "gearbase_hole1", - }, - ], - } - ], -} -# -------------------------------------------------------- -# * this is a full tree example. -result = { - "behavior_tree": { - "summary": "selector to insert shaft1 into gearbase_hole1", - "name": "selector: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)", - "identifier": 1, - "type_name": "selector", - "children": [ - { - "summary": "condition node to check if shaft1 is inserted to gearbase_hole1", - "name": "target: is_inserted_to(shaft1, gearbase_hole1)", - "identifier": 2, - "type_name": "condition", - "conditions": [ - { - "object_name": "shaft1", - "property_name": "is_inserted_to", - "property_value": "gearbase_hole1", - "status": True, - } - ], - }, - { - "summary": "sequence to insert shaft1 into gearbase_hole1", - "name": "sequence: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)", - "identifier": 3, - "type_name": "sequence", - "children": [ - { - "summary": "selector to unload outward_claw", - "name": "selector: unload_tool(left_hand, outward_claw)", - "identifier": 4, - "type_name": "selector", - "children": [ - { - "summary": "condition node to check if left_hand is free", - "name": "target: is_free(left_hand)", - "identifier": 5, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "is_free", - "property_value": None, - "status": True, - } - ], - }, - { - "summary": "sequence to unload outward_claw", - "name": "sequence: unload_tool(left_hand, outward_claw)", - "identifier": 6, - "type_name": "sequence", - "children": [ - { - "summary": "condition node to check if left hand holds outward_claw", - "name": "precondition: hold(left_hand, outward_claw)", - "identifier": 7, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "outward_claw", - "status": True, - } - ], - }, - { - "summary": "action node to unload outward_claw from left hand", - "name": "action: unload_tool(left_hand, outward_claw)", - "identifier": 8, - "type_name": "action", - "effects": [ - { - "object_name": "left_hand", - "property_name": "is_free", - "property_value": None, - "status": True, - }, - { - "object_name": "outward_claw", - "property_name": "is_equippable", - "property_value": None, - "status": True, - }, - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "outward_claw", - "status": False, - }, - ], - }, - ], - }, - ], - }, - { - "summary": "selector to load parallel_box1", - "name": "selector: load_tool(left_hand, parallel_box1)", - "identifier": 9, - "type_name": "selector", - "children": [ - { - "summary": "condition node to check if left_hand holds parallel_box1", - "name": "target: hold(left_hand, parallel_box1)", - "identifier": 10, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "parallel_box1", - "status": True, - } - ], - }, - { - "summary": "sequence to load parallel_box1", - "name": "sequence: load_tool(left_hand, parallel_box1)", - "identifier": 11, - "type_name": "sequence", - "children": [ - { - "summary": "condition node to check if parallel_box1 is equippable", - "name": "precondition: is_equippable(parallel_box1)", - "identifier": 12, - "type_name": "condition", - "conditions": [ - { - "object_name": "parallel_box1", - "property_name": "is_equippable", - "property_value": None, - "status": True, - } - ], - }, - { - "summary": "condition node to check if left hand is free", - "name": "precondition: is_free(left_hand)", - "identifier": 13, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "is_free", - "property_value": None, - "status": True, - } - ], - }, - { - "summary": "action node to equip parallel_box1 to left hand", - "name": "action: load_tool(left_hand, parallel_box1)", - "identifier": 14, - "type_name": "action", - "effects": [ - { - "object_name": "left_hand", - "property_name": "is_free", - "property_value": None, - "status": False, - }, - { - "object_name": "parallel_box1", - "property_name": "is_equippable", - "property_value": None, - "status": False, - }, - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "parallel_box1", - "status": True, - }, - ], - }, - ], - }, - ], - }, - { - "summary": "selector to pick up shaft1 with parallel_box1", - "name": "selector: pick_up(left_hand, parallel_box1, shaft1)", - "identifier": 15, - "type_name": "selector", - "children": [ - { - "summary": "condition node to check if parallel_box1 holds shaft1", - "name": "target: hold(parallel_box1, shaft1)", - "identifier": 16, - "type_name": "condition", - "conditions": [ - { - "object_name": "parallel_box1", - "property_name": "hold", - "property_value": "shaft1", - "status": True, - } - ], - }, - { - "summary": "sequence to pick up shaft1 with parallel_box1", - "name": "sequence: pick_up(left_hand, parallel_box1, shaft1)", - "identifier": 17, - "type_name": "sequence", - "children": [ - { - "summary": "condition node to check if parallel_box1 is free", - "name": "precondition: is_free(parallel_box1)", - "identifier": 18, - "type_name": "condition", - "conditions": [ - { - "object_name": "parallel_box1", - "property_name": "is_free", - "property_value": None, - "status": True, - } - ], - }, - { - "summary": "condition node to check if left hand holds parallel_box1", - "name": "precondition: hold(left_hand, parallel_box1)", - "identifier": 19, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "parallel_box1", - "status": True, - } - ], - }, - { - "summary": "action node to pick up shaft1 with parallel_box1", - "name": "action: pick_up(left_hand, parallel_box1, shaft1)", - "identifier": 20, - "type_name": "action", - "effects": [ - { - "object_name": "parallel_box1", - "property_name": "hold", - "property_value": "shaft1", - "status": True, - }, - { - "object_name": "parallel_box1", - "property_name": "is_free", - "property_value": None, - "status": False, - }, - ], - }, - ], - }, - ], - }, - { - "summary": "condition node to check if shaft1 can be inserted to gearbase_hole1", - "name": "precondition: can_insert_to(shaft1, gearbase_hole1)", - "identifier": 21, - "type_name": "condition", - "conditions": [ - { - "object_name": "shaft1", - "property_name": "can_insert_to", - "property_value": "gearbase_hole1", - "status": True, - } - ], - }, - { - "summary": "action node to insert shaft1 into gearbase_hole1", - "name": "action: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)", - "identifier": 22, - "type_name": "action", - "effects": [ - { - "object_name": "parallel_box1", - "property_name": "hold", - "property_value": "shaft1", - "status": False, - }, - { - "object_name": "parallel_box1", - "property_name": "is_free", - "property_value": None, - "status": True, - }, - { - "object_name": "shaft1", - "property_name": "is_inserted_to", - "property_value": "gearbase_hole1", - "status": True, - }, - ], - }, - ], - }, - ], - }, - "world_state": [ - { - "objects": [ - {"name": "parallel_box1", "properties": ["is_free", "is_equippable"]}, - {"name": "parallel_box2", "properties": ["is_free", "is_equippable"]}, - {"name": "inward_claw", "properties": ["is_free", "is_equippable"]}, - {"name": "outward_claw", "properties": ["is_free"]}, - {"name": "no_tool", "properties": ["is_free", "is_equippable"]}, - {"name": "gear1", "properties": []}, - {"name": "gear2", "properties": []}, - {"name": "gear3", "properties": []}, - {"name": "shaft1", "properties": []}, - {"name": "shaft2", "properties": []}, - {"name": "shaft3", "properties": []}, - {"name": "gearbase", "properties": []}, - {"name": "gearbase_hole1", "properties": []}, - {"name": "gearbase_hole3", "properties": []}, - {"name": "left_hand", "properties": []}, - ], - "constraints": [ - { - "source": "parallel_box1", - "name": "can_manipulate", - "target": "shaft1", - }, - { - "source": "parallel_box2", - "name": "can_manipulate", - "target": "gear1", - }, - {"source": "outward_claw", "name": "can_manipulate", "target": "gear2"}, - {"source": "outward_claw", "name": "can_manipulate", "target": "gear3"}, - {"source": "no_tool", "name": "can_manipulate", "target": "shaft3"}, - { - "source": "shaft1", - "name": "can_insert_to", - "target": "gearbase_hole1", - }, - { - "source": "shaft3", - "name": "can_insert_to", - "target": "gearbase_hole3", - }, - {"source": "gear3", "name": "can_insert_to", "target": "shaft3"}, - ], - "relations": [ - {"source": "left_hand", "name": "hold", "target": "outward_claw"} - ], - } - ], -} - -import json -from kios_bt.bt_factory import BehaviorTreeFactory -from kios_utils.pybt_test import generate_bt_stewardship, render_dot_tree - -# * this is a skeleton tree example. -example1 = { - "summary": "selector to load left_hand with parallel_box1", - "name": "selector: load_tool(left_hand, parallel_box1)", - "children": [ - { - "summary": "condition node to check if left_hand holds parallel_box1", - "name": "target: hold(left_hand, parallel_box1)", - }, - { - "summary": "sequence to load left_hand with parallel_box1", - "name": "sequence: load_tool(left_hand, parallel_box1)", - "children": [ - { - "summary": "selector to make left_hand unload outward_claw", - "name": "selector: unload_tool(left_hand, outward_claw)", - "children": [ - { - "summary": "condition node to check if left_hand is free", - "name": "target: is_free(left_hand)", - }, - { - "summary": "sequence to make left_hand unload outward_claw", - "name": "sequence: unload_tool(left_hand, outward_claw)", - "children": [ - { - "summary": "condition node to check if outward_claw is free", - "name": "precondition: is_free(outward_claw)", - }, - { - "summary": "condition node to check if left hand holds outward_claw", - "name": "precondition: hold(left_hand, outward_claw)", - }, - { - "summary": "action node to make left hand unload outward_claw", - "name": "action: unload_tool(left_hand, outward_claw)", - "effects": [ - { - "summary": "left_hand will be free", - }, - { - "summary": "outward_claw will be equippable", - }, - { - "summary": "left_hand will not hold outward_claw", - }, - ], - }, - ], - }, - ], - }, - { - "summary": "condition node to check if parallel_box1 is equippable", - "name": "precondition: is_equippable(parallel_box1)", - }, - { - "summary": "action node to equip left hand with parallel_box1", - "name": "action: load_tool(left_hand, parallel_box1)", - "effects": [ - { - "summary": "left_hand will be not free", - }, - { - "summary": "parallel_box1 will be not equippable", - }, - { - "summary": "left_hand will hold parallel_box1", - }, - ], - }, - ], - }, - ], -} - -result3 = { - "summary": "selector to insert shaft1 into gearbase_hole1", - "name": "selector: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)", - "children": [ - { - "summary": "condition node to check if shaft1 is inserted into gearbase_hole1", - "name": "target: is_inserted_to(shaft1, gearbase_hole1)", - }, - { - "summary": "sequence to insert shaft1 into gearbase_hole1", - "name": "sequence: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)", - "children": [ - { - "summary": "selector to unload outward_claw from left_hand", - "name": "selector: unload_tool(left_hand, outward_claw)", - "children": [ - { - "summary": "condition node to check if left_hand is free", - "name": "target: is_free(left_hand)", - }, - { - "summary": "sequence to unload outward_claw from left_hand", - "name": "sequence: unload_tool(left_hand, outward_claw)", - "children": [ - { - "summary": "condition node to check if left hand holds outward_claw", - "name": "precondition: hold(left_hand, outward_claw)", - }, - { - "summary": "action node to unload outward_claw from left hand", - "name": "action: unload_tool(left_hand, outward_claw)", - "effects": [ - { - "summary": "left_hand will be free", - "name": "effect: is_free(left_hand)", - }, - { - "summary": "outward_claw will be equippable", - "name": "effect: is_equippable(outward_claw)", - }, - { - "summary": "left_hand will not hold outward_claw", - "name": "effect: not hold(left_hand, outward_claw)", - }, - ], - }, - ], - }, - ], - }, - { - "summary": "selector to load parallel_box1 on left_hand", - "name": "selector: load_tool(left_hand, parallel_box1)", - "children": [ - { - "summary": "condition node to check if left_hand holds parallel_box1", - "name": "target: hold(left_hand, parallel_box1)", - }, - { - "summary": "sequence to load parallel_box1 on left_hand", - "name": "sequence: load_tool(left_hand, parallel_box1)", - "children": [ - { - "summary": "condition node to check if parallel_box1 is equippable", - "name": "precondition: is_equippable(parallel_box1)", - }, - { - "summary": "condition node to check if left hand is free", - "name": "precondition: is_free(left_hand)", - }, - { - "summary": "action node to load parallel_box1 on left hand", - "name": "action: load_tool(left_hand, parallel_box1)", - "effects": [ - { - "summary": "left_hand will not be free", - "name": "effect: not is_free(left_hand)", - }, - { - "summary": "parallel_box1 will not be equippable", - "name": "effect: not is_equippable(parallel_box1)", - }, - { - "summary": "left_hand will hold parallel_box1", - "name": "effect: hold(left_hand, parallel_box1)", - }, - ], - }, - ], - }, - ], - }, - { - "summary": "condition node to check if parallel_box1 can manipulate shaft1", - "name": "precondition: can_manipulate(parallel_box1, shaft1)", - }, - { - "summary": "condition node to check if shaft1 can be inserted to gearbase_hole1", - "name": "precondition: can_insert_to(shaft1, gearbase_hole1)", - }, - { - "summary": "action node to insert shaft1 into gearbase_hole1", - "name": "action: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)", - "effects": [ - { - "summary": "parallel_box1 will not hold shaft1", - "name": "effect: not hold(parallel_box1, shaft1)", - }, - { - "summary": "parallel_box1 will be free", - "name": "effect: is_free(parallel_box1)", - }, - { - "summary": "shaft1 will be inserted into gearbase_hole1", - "name": "effect: is_inserted_to(shaft1, gearbase_hole1)", - }, - ], - }, - ], - }, - ], -} - -example2 = { - "summary": "selector to insert gear1 into shaft1", - "name": "selector: insert(left_hand, parallel_box1, gear1, shaft1)", - "children": [ - { - "summary": "condition node to check if gear1 is inserted to shaft1", - "name": "target: is_inserted_to(gear1, shaft1)", - }, - { - "summary": "sequence to insert gear1 into shaft1", - "name": "sequence: insert(left_hand, parallel_box1, gear1, shaft1)", - "children": [ - { - "summary": "selector to load parallel_box1", - "name": "selector: load_tool(left_hand, parallel_box1)", - "children": [ - { - "summary": "condition node to check if left_hand holds parallel_box1", - "name": "target: hold(left_hand, parallel_box1)", - }, - { - "summary": "sequence to load parallel_box1", - "name": "sequence: load_tool(left_hand, parallel_box1)", - "children": [ - { - "summary": "condition node to check if parallel_box1 is equippable", - "name": "precondition: is_equippable(parallel_box1)", - }, - { - "summary": "selector to make left_hand unload outward_claw", - "name": "selector: unload_tool(left_hand, outward_claw)", - "children": [ - { - "summary": "condition node to check if left_hand is free", - "name": "target: is_free(left_hand)", - }, - { - "summary": "sequence to make left_hand unload outward_claw", - "name": "sequence: unload_tool(left_hand, outward_claw)", - "children": [ - { - "summary": "condition node to check if outward_claw is free", - "name": "precondition: is_free(outward_claw)", - }, - { - "summary": "condition node to check if left hand holds outward_claw", - "name": "precondition: hold(left_hand, outward_claw)", - }, - { - "summary": "action node to make left hand unload outward_claw", - "name": "action: unload_tool(left_hand, outward_claw)", - "effects": [ - { - "summary": "left_hand will be free", - }, - { - "summary": "outward_claw will be equippable", - }, - { - "summary": "left_hand will not hold outward_claw", - }, - ], - }, - ], - }, - ], - }, - { - "summary": "action node to equip parallel_box1 to left hand", - "name": "action: load_tool(left_hand, parallel_box1)", - "effects": [ - { - "summary": "left_hand will be not free", - }, - { - "summary": "parallel_box1 will be not equippable", - }, - { - "summary": "left_hand will hold parallel_box1", - }, - ], - }, - ], - }, - ], - }, - { - "summary": "selector to use left_hand with parallel_box1 to pick up gear1", - "name": "selector: pick_up(left_hand, parallel_box1, gear1)", - "children": [ - { - "summary": "condition node to check if parallel_box1 holds gear1", - "name": "target: hold(parallel_box1, gear1)", - }, - { - "summary": "sequence to use left hand with parallel_box1 to pick up gear1", - "name": "sequence: pick_up(left_hand, parallel_box1, gear1)", - "children": [ - { - "summary": "condition node to check if parallel_box1 is free", - "name": "precondition: is_free(parallel_box1)", - }, - { - "summary": "condition node to check if left hand holds parallel_box1", - "name": "precondition: hold(left_hand, parallel_box1)", - }, - { - "summary": "action node to use left hand with parallel_box1 to pick up gear1", - "name": "action: pick_up(left_hand, parallel_box1, gear1)", - "effects": [ - { - "summary": "parallel_box1 will hold gear1", - }, - { - "summary": "parallel_box1 will be not free", - }, - ], - }, - ], - }, - ], - }, - { - "summary": "condition node to check if gear1 can be inserted to shaft1", - "name": "precondition: can_insert_to(gear1, shaft1)", - }, - { - "summary": "action node to use left_hand with parallel_box1 to insert gear1 to shaft1", - "name": "action: insert(left_hand, parallel_box1, gear1, shaft1)", - "effects": [ - { - "summary": "parallel_box1 will be not holding gear1", - }, - { - "summary": "parallel_box1 will be free", - }, - { - "summary": "gear1 will be inserted to shaft1", - }, - ], - }, - ], - }, - ], -} - -result = { - "summary": "Selector to insert shaft1 into gearbase_hole1 using parallel_box1 in the left hand", - "name": "selector: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)", - "children": [ - { - "summary": "Check the target that shaft1 is inserted into gearbase_hole1", - "name": "target: is_inserted_to(shaft1, gearbase_hole1)", - }, - { - "summary": "Sequence to insert shaft1 into gearbase_hole1 using parallel_box1 in the left hand", - "name": "sequence: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)", - "children": [ - { - "summary": "Selector to load the parallel_box1 in the left hand", - "name": "selector: load_tool(left_hand, parallel_box1)", - "children": [ - { - "summary": "Check the target that the left hand is free", - "name": "target: is_free(left_hand)", - }, - { - "summary": "Sequence to load the tool in the left hand", - "name": "sequence: load_tool(left_hand, parallel_box1)", - "children": [ - { - "summary": "Check the precondition that the parallel_box1 is equippable", - "name": "precondition: is_equippable(parallel_box1)", - }, - { - "summary": "Selector to unload the outward_claw in the left hand", - "name": "selector: unload_tool(left_hand, outward_claw)", - "children": [ - { - "summary": "Check the target that the left hand is free", - "name": "target: is_free(left_hand)", - }, - { - "summary": "Sequence to unload the tool in the left hand", - "name": "sequence: unload_tool(left_hand, outward_claw)", - "children": [ - { - "summary": "Check the precondition that the left hand is holding a outward_claw", - "name": "precondition: hold(left_hand, outward_claw)", - }, - { - "summary": "Unload the outward_claw in the left hand", - "name": "action: unload_tool(left_hand, outward_claw)", - }, - ], - }, - ], - }, - { - "summary": "Load the parallel_box1 in the left hand", - "name": "action: load_tool(left_hand, parallel_box1)", - }, - ], - }, - ], - }, - { - "summary": "Selector to pick up the shaft1 with the parallel_box1 in the left hand", - "name": "selector: pick_up(left_hand, parallel_box1, shaft1)", - "children": [ - { - "summary": "Check the target that the parallel_box1 is holding the shaft1", - "name": "target: hold(parallel_box1, shaft1)", - }, - { - "summary": "Sequence to pick up the shaft1 with the parallel_box1 in the left hand", - "name": "sequence: pick_up(left_hand, parallel_box1, shaft1)", - "children": [ - { - "summary": "Check the precondition that the left hand is holding the parallel_box1", - "name": "precondition: hold(left_hand, parallel_box1)", - }, - { - "summary": "Pick up the shaft1 with the parallel_box1 in the left hand", - "name": "action: pick_up(left_hand, parallel_box1, shaft1)", - }, - ], - }, - ], - }, - { - "summary": "Insert shaft1 into gearbase_hole1 using parallel_box1", - "name": "action: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)", - }, - ], - }, - ], -} - - -def render_bt(bt_json: json): - test_class = BehaviorTreeFactory() - bt = test_class.from_json_to_simple_bt(bt_json) # * use this for skeleton tree - # bt = test_class.from_json_to_tree_root(bt_json) # * use this for a full tree - bt_stewardship = generate_bt_stewardship(bt) - # bt_stewardship.setup(timeout=15) - render_dot_tree(bt_stewardship) - - -render_bt(result) diff --git a/experiments/cook/world_state.json b/experiments/cook/world_state.json deleted file mode 100644 index 08e303e3..00000000 --- a/experiments/cook/world_state.json +++ /dev/null @@ -1,19 +0,0 @@ -{ - "objects": [ - {"name": "clampgripper", "properties": ["is_empty"]}, - {"name": "whisk", "properties": []}, - {"name": "spatula", "properties": []}, - {"name": "pan", "properties": ["is_empty"]}, - {"name": "pot", "properties": ["is_empty"]}, - {"name": "bowl", "properties": ["is_empty"]}, - {"name": "plate", "properties": ["is_empty"]}, - {"name": "left_hand", "properties": []} - ], - "constraints": [ - {"source": "gear1", "name": "can_insert_to", "target": "shaft1"} - ], - "relations": [ - {"source": "left_hand", "name": "hold", "target": "outwardgripper"} - ] -} - diff --git a/experiments/gearbox/experiment.py b/experiments/gearbox/experiment.py deleted file mode 100644 index bdf926cd..00000000 --- a/experiments/gearbox/experiment.py +++ /dev/null @@ -1,117 +0,0 @@ -import socket -from pprint import pprint -from runtime_script import * -from plan_parser import parse_plan, PlanStep - - -HOST = "10.157.175.108" # The server's hostname or IP address -PORT = 65431 # The port used by the server - -connected = False -# while not connected: -# try: -# with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s: -# s.connect((HOST, PORT)) -# data = s.recv(4096) -# connected = True -# if connected: -# print("Connected to the server") -# s.close() - -# except socket.error as e: -# print(f"Connection failed: {e}") -# Add any necessary delay before retrying the connection - - -# print(f"Received {data!r}") - - -order_dict: dict[str, object] = { - "outerring": insert_outerring, - "cone": insert_cone, - "outputshaftandgearstage2": insert_outputshaftandgearstage2, - "ringgear": insert_ringgear, - "designring": insert_designring, - "gearstage1": insert_gearstage1, - "driveflange": insert_driveflange, - "inputpinion": insert_inputpinion, -} - -# workflow -# 1. receive plan - -# 2. parse plan -# task_plan = parse_plan(data.decode()) -task_plan = parse_plan( - """ -0.0000: (initialize defaultgripper) [d:10.0000; c:1.0000] -0.0003: (change_gripper defaultgripper outwardgripper) [d:10.0000; c:1.0000] -10.0005: (initialize outwardgripper) [d:10.0000; c:1.0000] -20.0007: (insert outwardgripper outerring housing) [d:23.0000; c:1.0000] -43.0010: (insert outwardgripper cone outerring) [d:19.0000; c:1.0000] -62.0013: (change_gripper outwardgripper defaultgripper) [d:10.0000; c:1.0000] -72.0015: (initialize defaultgripper) [d:1.0000; c:1.0000] -73.0017: (insert defaultgripper outputshaftandgearstage2 cone) [d:40.0000; c:1.0000] -113.0020: (change_gripper defaultgripper parallelgripper) [d:10.0000; c:1.0000] -123.0023: (initialize parallelgripper) [d:10.0000; c:1.0000] -133.0025: (insert parallelgripper ringgear outputshaftandgearstage2) [d:25.0000; c:1.0000] -158.0027: (change_gripper parallelgripper inwardgripper) [d:25.0000; c:1.0000] -183.0030: (initialize inwardgripper) [d:10.0000; c:1.0000] -193.0033: (insert inwardgripper gearstage1 ringgear) [d:25.0000; c:1.0000] -218.0035: (change_gripper inwardgripper clampgripper) [d:30.0000; c:1.0000] -248.0038: (initialize clampgripper) [d:10.0000; c:1.0000] -258.0040: (insert clampgripper designring ringgear) [d:13.0000; c:1.0000] -271.0042: (change_gripper clampgripper defaultgripper) [d:15.0000; c:1.0000] -286.0045: (initialize defaultgripper) [d:1.0000; c:1.0000] -287.0048: (insert defaultgripper driveflange gearstage1) [d:25.0000; c:1.0000] -312.0050: (insert defaultgripper inputpinion driveflange) [d:25.0000; c:1.0000] -""" -) -# 3. execute plan -set_tool("defaultgripper") - - -def result_montage(plan_step: PlanStep, execution_time: float, last_result: str) -> str: - print(type(last_result)) - last_result += f"{plan_step.raw_action}, {execution_time}\n" - return last_result - - -result = "" -for plan_step in task_plan: - pprint(f"Executing {plan_step}") - action_name = plan_step.action_name - action_params = plan_step.action_params - execution_time = 0 - if action_name == "change_gripper": - execution_time = change_gripper(action_params[0], action_params[1]) - elif action_name == "initialize": - execution_time = initialize_gripper() - elif action_name == "insert": - part_name = action_params[1] - execution_time = order_dict[part_name]() - else: - raise ValueError(f"Unknown action name: {action_name}") - - result = result_montage(plan_step, execution_time, result) - -pprint(result) -file_path = os.path.join(os.path.dirname(__file__), "result.txt") -with open(file_path, "w") as f: - f.write(result) - f.close() - -# 4. send execution time back - - -def test_montage(): - result = "" - for plan_step in task_plan: - result = result_montage(plan_step, 1, result) - - pprint(result) - - -if __name__ == "__main__": - # test_montage() - pass diff --git a/experiments/gearbox/mios_memory_backups/gearbox.json b/experiments/gearbox/mios_memory_backups/gearbox.json deleted file mode 100644 index 9219778e..00000000 --- a/experiments/gearbox/mios_memory_backups/gearbox.json +++ /dev/null @@ -1,5302 +0,0 @@ -[ - { - "_id": "643d40f9dd8c607f760e5af8", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "NullObject", - "q": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ] - }, - { - "_id": "643d40f9dd8c607f760e5af9", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "NoneObject", - "q": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ] - }, - { - "_id": "643d40f9dd8c607f760e5afa", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.9451230787549627, - 0.32322187014216935, - -0.047441917833785346, - 0.0, - 0.326656439478547, - -0.9331005912790161, - 0.15033164671205046, - 0.0, - 0.004322477630794235, - -0.1575821506811295, - -0.9874964212460929, - 0.0, - 0.4023718149133433, - 0.135438661042489, - 0.3261623632093963, - 1.0 - ], - "O_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "EndEffector", - "q": [ - -0.26602681190215505, - -0.4142026782548806, - 0.6038659832617761, - -2.428547883716245, - 0.08718072785271538, - 2.0376345478921305, - 0.7037023221689134 - ] - }, - { - "_id": "6491ad548829c1097f0151a9", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "ring", - "q": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ] - }, - { - "_id": "64f8a17c8d1b98ac5f0c6b91", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "geometry": null, - "grasp_force": 1.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "TestObject1", - "q": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ] - }, - { - "_id": "6526b75ece654b001f026d61", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "gripper_connected", - "q": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ] - }, - { - "_id": "6526da7d86c098f7fe057f61", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "tool_load", - "q": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ] - }, - { - "_id": "6530168b631a09bd8e08d211", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "tool_grasp", - "q": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ] - }, - { - "_id": "65a8008494d3e83f7e035b11", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "kios_guidance", - "q": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ] - }, - { - "_id": "65b2664c677d37a9900fd7b1", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.020523632927385155, - 0.999779614758663, - 0.0004993052070315127, - 0.0, - 0.9997626802269471, - -0.020520362759945312, - -0.00585190899106819, - 0.0, - -0.0058404858401676345, - 0.000619301067554896, - -0.99998275244693, - 0.0, - 0.49407319098460234, - -0.3343870569274952, - 0.3249152265674699, - 1.0 - ], - "O_T_TCP": [ - 0.020523632927385155, - 0.999779614758663, - 0.0004993052070315127, - 0.0, - 0.9997626802269471, - -0.020520362759945312, - -0.00585190899106819, - 0.0, - -0.0058404858401676345, - 0.000619301067554896, - -0.99998275244693, - 0.0, - 0.49407319098460234, - -0.3343870569274952, - 0.3249152265674699, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "parallel_box1", - "q": [ - -0.4889897713452055, - 0.1520400121422783, - -0.11095685046074681, - -1.8813407051080338, - 0.015587054766497403, - 2.027237653652827, - -1.3705014213786446 - ] - }, - { - "_id": "65bb9aa7666d3392110788a1", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - -0.0055492434676263965, - 0.9999687095518451, - 0.003540157533654855, - 0.0, - 0.9994642785591136, - 0.005433245086445076, - 0.0319747186594265, - 0.0, - 0.03195509884729998, - 0.003715768034059008, - -0.9994823984070837, - 0.0, - 0.5073771436733974, - 0.2821340073022507, - 0.3273610300796691, - 1.0 - ], - "O_T_TCP": [ - -0.0055492434676263965, - 0.9999687095518451, - 0.003540157533654855, - 0.0, - 0.9994642785591136, - 0.005433245086445076, - 0.0319747186594265, - 0.0, - 0.03195509884729998, - 0.003715768034059008, - -0.9994823984070837, - 0.0, - 0.5073771436733974, - 0.2821340073022507, - 0.3273610300796691, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "inward_claw", - "q": [ - -0.3558367933193034, - 0.12986394129696988, - 0.8984244877664667, - -1.9720981870183556, - -0.1313234915672495, - 2.079114055554075, - -0.1914156752901866 - ] - }, - { - "_id": "65bbb462b8aab7a7d90d58c1", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.9949942224611444, - -0.05041086343049174, - 0.08617417837347448, - 0.0, - -0.04503393357509275, - -0.9969720659778388, - -0.06324074151875426, - 0.0, - 0.08910298453945457, - 0.05904454700105643, - -0.9942707878719971, - 0.0, - 0.6906053200978064, - -0.01614832324583123, - 0.2887581517923185, - 1.0 - ], - "O_T_TCP": [ - 0.9949942224611444, - -0.05041086343049174, - 0.08617417837347448, - 0.0, - -0.04503393357509275, - -0.9969720659778388, - -0.06324074151875426, - 0.0, - 0.08910298453945457, - 0.05904454700105643, - -0.9942707878719971, - 0.0, - 0.6990701036290545, - -0.01053909128073087, - 0.1943024269444788, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "test_object1", - "q": [ - -0.5258051569263356, - 0.4877585826129244, - 0.5891767306537179, - -1.6077540129764138, - -0.23719313260094485, - 2.1022328375975285, - 0.9414903357566682 - ] - }, - { - "_id": "65bbba7c5a0f2e8e04072891", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.005516821389947922, - -0.9949443596914403, - -0.10018000150304039, - 0.0, - -0.999821715401967, - -0.003733290448562814, - -0.017981847911249182, - 0.0, - 0.017517274375192557, - 0.10026327397364804, - -0.9948067254449727, - 0.0, - 0.643320166007396, - 0.11088726408849338, - 0.3035784369329164, - 1.0 - ], - "O_T_TCP": [ - 0.005516821389947922, - -0.9949443596914403, - -0.10018000150304039, - 0.0, - -0.999821715401967, - -0.003733290448562814, - -0.017981847911249182, - 0.0, - 0.017517274375192557, - 0.10026327397364804, - -0.9948067254449727, - 0.0, - 0.6455098253042951, - 0.12342017333519939, - 0.17922759625229484, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "gear3", - "q": [ - -0.4348329623676183, - 0.39139715203497827, - 0.6694900968644402, - -1.711178953283276, - -0.16270597490334937, - 2.05581780107816, - 2.617633598559432 - ] - }, - { - "_id": "65bbbab25a0f2e8e04072892", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.9980187907769038, - -0.05571650543342081, - -0.02889482963615323, - 0.0, - -0.05609811498518191, - -0.9983365128919355, - -0.012568031790528969, - 0.0, - -0.028147058566118602, - 0.014164350073066843, - -0.9995034338515715, - 0.0, - 0.507308310840537, - -0.04026469732735698, - 0.3761685279429274, - 1.0 - ], - "O_T_TCP": [ - 0.9980187907769038, - -0.05571650543342081, - -0.02889482963615323, - 0.0, - -0.05609811498518191, - -0.9983365128919355, - -0.012568031790528969, - 0.0, - -0.028147058566118602, - 0.014164350073066843, - -0.9995034338515715, - 0.0, - 0.5046343402767557, - -0.038919084070415624, - 0.2812157017270281, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "shaft2", - "q": [ - -0.04717889234179201, - -0.13185073856974885, - -0.034486469687599886, - -2.0883627749325773, - 0.0077889278013025046, - 1.9273695102532702, - 0.7572235297293566 - ] - }, - { - "_id": "65bfcf0f5a0e8797bb01e281", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.9437340865485336, - 0.06882246062716522, - 0.32343498527653963, - 0.0, - 0.050228524866912413, - -0.9965780243141402, - 0.06549873010240133, - 0.0, - 0.32684227512118086, - -0.045568599359231785, - -0.9439796766594333, - 0.0, - 0.4125253330001541, - -0.00839313893860622, - 0.6067596822698708, - 1.0 - ], - "O_T_TCP": [ - 0.9437393214724915, - 0.0688334428219026, - 0.3234173729983186, - 0.0, - 0.05023845593103052, - -0.9965770370070167, - 0.0655061352637274, - 0.0, - 0.32682563252160757, - -0.0455736036384628, - -0.9439851972241172, - 0.0, - 0.42069933942795634, - -0.009530327501703541, - 0.5831522581482022, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "test", - "q": [ - -0.057407226155201586, - -0.5933335052833447, - 0.037569229965147215, - -2.0232708176792724, - -0.017467192303803232, - 1.7643939062683323, - 0.7018759353094631 - ] - }, - { - "_id": "65ce303af045be1f60050f41", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - -0.00034130509165598697, - 0.9999088204921296, - 0.012766405683707687, - 0.0, - 0.9999006778257512, - 0.00017031581705472163, - 0.013392250644843597, - 0.0, - 0.01338911300712114, - 0.012769954401065334, - -0.9998288153066389, - 0.0, - 0.34586891787172797, - 0.28478778305517327, - 0.325633698814514, - 1.0 - ], - "O_T_TCP": [ - -0.00034130509165598697, - 0.9999088204921296, - 0.012766405683707687, - 0.0, - 0.9999006778257512, - 0.00017031581705472163, - 0.013392250644843597, - 0.0, - 0.01338911300712114, - 0.012769954401065334, - -0.9998288153066389, - 0.0, - 0.34586891787172797, - 0.28478778305517327, - 0.325633698814514, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "outward_claw", - "q": [ - -0.26322421671319607, - -0.48738894555267787, - 0.8759835133827272, - -2.392889458566437, - 0.4210987845394346, - 2.0469462890460046, - -0.42977081686632607 - ] - }, - { - "_id": "65ce3098f045be1f60050f42", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.016519502886665666, - 0.9997483636116822, - -0.014527986142256165, - 0.0, - 0.999314451509194, - -0.01603160985419312, - 0.033081133468638016, - 0.0, - 0.0328405343279425, - -0.015064800427202131, - -0.999347062382806, - 0.0, - 0.3334408867064976, - -0.33165373934959796, - 0.32422469388972297, - 1.0 - ], - "O_T_TCP": [ - 0.016519502886665666, - 0.9997483636116822, - -0.014527986142256165, - 0.0, - 0.999314451509194, - -0.01603160985419312, - 0.033081133468638016, - 0.0, - 0.0328405343279425, - -0.015064800427202131, - -0.999347062382806, - 0.0, - 0.3334408867064976, - -0.33165373934959796, - 0.32422469388972297, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "parallel_box2", - "q": [ - -0.49195081603365076, - -0.2536298767587599, - -0.28172348515657036, - -2.357436443128084, - -0.06903641326254598, - 2.1465019415994413, - -1.4987806172708702 - ] - }, - { - "_id": "65ce3113f045be1f60050f43", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.05033441300943016, - 0.998522325819256, - -0.020008963209219598, - 0.0, - 0.9980401159147424, - -0.049549287905040595, - 0.037967644043304945, - 0.0, - 0.036920821197481454, - -0.021881248318769652, - -0.9990786074849747, - 0.0, - 0.4951086360500897, - -0.1768782158810579, - 0.5007737812082008, - 1.0 - ], - "O_T_TCP": [ - 0.05033441300943016, - 0.998522325819256, - -0.020008963209219598, - 0.0, - 0.9980401159147424, - -0.049549287905040595, - 0.037967644043304945, - 0.0, - 0.036920821197481454, - -0.021881248318769652, - -0.9990786074849747, - 0.0, - 0.49824690585187564, - -0.1787381219881533, - 0.415852099571978, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "shaft1", - "q": [ - -0.6783767049908637, - -0.14173239019669984, - 0.31067318724331106, - -1.7888499834156932, - 0.03642009920908874, - 1.6961157420094577, - -1.1089101982976917 - ] - }, - { - "_id": "65ce315df045be1f60050f44", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.06323640929784359, - 0.9978521827830034, - 0.016520434483373037, - 0.0, - 0.9977655821240671, - -0.06356378921021756, - 0.020105589547900215, - 0.0, - 0.021112914320698742, - 0.015212408525423958, - -0.9996613563981266, - 0.0, - 0.3844454340627629, - -0.034519055467682386, - 0.3511666416459713, - 1.0 - ], - "O_T_TCP": [ - 0.06323640929784359, - 0.9978521827830034, - 0.016520434483373037, - 0.0, - 0.9977655821240671, - -0.06356378921021756, - 0.020105589547900215, - 0.0, - 0.021112914320698742, - 0.015212408525423958, - -0.9996613563981266, - 0.0, - 0.38624003178002225, - -0.03322600074302135, - 0.2661954263521305, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "gearbase_hole1", - "q": [ - -0.4779728950460752, - -0.5647525487445973, - 0.34118822538716465, - -2.533109467228605, - 0.2194305768719429, - 2.0038177083303617, - -0.9983835912605101 - ] - }, - { - "_id": "65ce31d5f045be1f60050f45", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.016773866766090845, - 0.998555668652971, - 0.050852344055508855, - 0.0, - 0.999651562629023, - -0.017761257817640888, - 0.019027295007331226, - 0.0, - 0.019903398084193856, - 0.05051643648500831, - -0.9985248842114842, - 0.0, - 0.6166632788352968, - -0.06722110520041134, - 0.29656367644075515, - 1.0 - ], - "O_T_TCP": [ - 0.016773866766090845, - 0.998555668652971, - 0.050852344055508855, - 0.0, - 0.999651562629023, - -0.017761257817640888, - 0.019027295007331226, - 0.0, - 0.019903398084193856, - 0.05051643648500831, - -0.9985248842114842, - 0.0, - 0.6171608637874016, - -0.06595819428828614, - 0.27160055433546804, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "shaft3", - "q": [ - -0.6255808761370809, - 0.2721908670313347, - 0.5482196540661509, - -1.844289931665387, - -0.10189347790347204, - 2.091038903571352, - -0.8129646023995416 - ] - }, - { - "_id": "65ce3207f045be1f60050f46", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - -0.15915497945225549, - 0.9860926026332786, - -0.04766359668224283, - 0.0, - 0.9850428396108329, - 0.16183700693102337, - 0.058992662427146104, - 0.0, - 0.06588723038902841, - -0.03756243183872672, - -0.9971198205762556, - 0.0, - 0.5043739432190215, - 0.08072733253032562, - 0.2875533813662191, - 1.0 - ], - "O_T_TCP": [ - -0.15915497945225549, - 0.9860926026332786, - -0.04766359668224283, - 0.0, - 0.9850428396108329, - 0.16183700693102337, - 0.058992662427146104, - 0.0, - 0.06588723038902841, - -0.03756243183872672, - -0.9971198205762556, - 0.0, - 0.5060211239787472, - 0.07978827173435744, - 0.2626253858518127, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "gearbase_hole3", - "q": [ - -0.6320993901689848, - -0.13105960240977546, - 0.7911574711811249, - -2.301617453056469, - 0.05951137099465102, - 2.2677469761391547, - -0.8261827120045797 - ] - }, - { - "_id": "65ce326bf045be1f60050f47", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.9982800059810064, - 0.0028567036015946868, - 0.058391915573158704, - 0.0, - 0.002665300689743662, - -0.9999811917035863, - 0.003355489907203343, - 0.0, - 0.05840152737118158, - -0.003194147969025999, - -0.9982880641475511, - 0.0, - 0.38329798339116805, - -0.1792311431159989, - 0.2652977941807128, - 1.0 - ], - "O_T_TCP": [ - 0.9982800059810064, - 0.0028567036015946868, - 0.058391915573158704, - 0.0, - 0.002665300689743662, - -0.9999811917035863, - 0.003355489907203343, - 0.0, - 0.05840152737118158, - -0.003194147969025999, - -0.9982880641475511, - 0.0, - 0.38738609030715077, - -0.1794547334738307, - 0.19541762969038423, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "gear1", - "q": [ - -0.743737426383209, - -0.3330958715231419, - 0.30138807209333335, - -2.598988366910377, - 0.16539736091956653, - 2.3272367282354502, - 0.21644120803806513 - ] - }, - { - "_id": "65ce3349f045be1f60050f48", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - -0.43529958087664555, - -0.8981800970094251, - 0.06138025029245873, - 0.0, - -0.9001402451487228, - 0.43304221718444524, - -0.04693318762651431, - 0.0, - 0.01557451519504121, - -0.07568228756712969, - -0.9970103438906971, - 0.0, - 0.32523152397741284, - 0.13156707585417038, - 0.31425307836568933, - 1.0 - ], - "O_T_TCP": [ - -0.43529958087664555, - -0.8981800970094251, - 0.06138025029245873, - 0.0, - -0.9001402451487228, - 0.43304221718444524, - -0.04693318762651431, - 0.0, - 0.01557451519504121, - -0.07568228756712969, - -0.9970103438906971, - 0.0, - 0.327178338376793, - 0.12210678990827917, - 0.18962678537935218, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "gear2", - "q": [ - -0.44017739813892465, - -0.8408044241896847, - 0.7812836869963553, - -2.6489709372327193, - 0.5117495865954292, - 1.9353651503056144, - 2.7550759355656957 - ] - }, - { - "_id": "65ce3392f045be1f60050f49", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - -0.19207017045850883, - -0.9745158408444792, - -0.11579582230989756, - 0.0, - -0.9619084037954584, - 0.21032850120116167, - -0.1745705908546879, - 0.0, - 0.19448071224163638, - 0.07785667044396119, - -0.9778116339220793, - 0.0, - 0.6668309353879216, - 0.1002329166290898, - 0.3839163655785476, - 1.0 - ], - "O_T_TCP": [ - -0.19207017045850883, - -0.9745158408444792, - -0.11579582230989756, - 0.0, - -0.9619084037954584, - 0.21032850120116167, - -0.1745705908546879, - 0.0, - 0.19448071224163638, - 0.07785667044396119, - -0.9778116339220793, - 0.0, - 0.6911410244181262, - 0.10996500043458494, - 0.2616899113382877, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "no_tool", - "q": [ - -0.449013091942916, - 0.2972115466353616, - 0.6671699467458255, - -1.6625072732841806, - -0.16246066116120655, - 2.102316742226921, - 2.8067302440068085 - ] - }, - { - "_id": "65ce3399f045be1f60050f4a", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.37121811244401426, - -0.911720063290249, - 0.1759101648249346, - 0.0, - -0.9285316648543275, - -0.3639617925463895, - 0.07308561986107809, - 0.0, - -0.00260909728174289, - -0.1904725312477871, - -0.9816890584352234, - 0.0, - 0.5482232455292271, - -0.09299434620256447, - 0.4961290551977721, - 1.0 - ], - "O_T_TCP": [ - 0.37121811244401426, - -0.911720063290249, - 0.1759101648249346, - 0.0, - -0.9285316648543275, - -0.3639617925463895, - 0.07308561986107809, - 0.0, - -0.00260909728174289, - -0.1904725312477871, - -0.9816890584352234, - 0.0, - 0.5478971083690092, - -0.11680341260853785, - 0.37341792289336917, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "gearbase", - "q": [ - -0.5710980864097402, - -0.030782228559778448, - 0.4631181385266316, - -1.681660839952254, - -0.1779492470121923, - 1.6707865634759267, - 1.8844083975966868 - ] - }, - { - "_id": "65cfa2c47f0205483b054d51", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.999127486937346, - 0.0020339066230885923, - 0.04148343008142645, - 0.0, - -0.0021570557684146043, - -0.9949015779177928, - 0.1007320419147068, - 0.0, - 0.041477608185149145, - -0.10073557343640634, - -0.994048264554432, - 0.0, - 0.3646566267993078, - -0.02254569907028298, - 0.40952082232522724, - 1.0 - ], - "O_T_TCP": [ - 0.9991272968724854, - 0.002032794737528359, - 0.04148806203046724, - 0.0, - -0.002158543289712084, - -0.9949017859475108, - 0.10072995537836842, - 0.0, - 0.041482108991960025, - -0.10073354124199559, - -0.9940482826816947, - 0.0, - 0.36465772773036376, - -0.02254865462649719, - 0.40952225264460873, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "test1", - "q": [ - -0.6494016965366246, - -0.753164953780274, - 0.5019667189581353, - -2.4719201877805506, - 0.2581255530648761, - 1.8439903862476348, - 0.4656479083630774 - ] - }, - { - "_id": "65cfa7905d75dfc96909caf1", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - -0.9935878908904953, - 0.08960520143219396, - -0.06880957675557518, - 0.0, - 0.09120083791165756, - 0.9956138318810523, - -0.02040225067357439, - 0.0, - 0.06668090241645759, - -0.026547431392950203, - -0.9974211202593233, - 0.0, - 0.5080013634448317, - 0.07744840851681808, - 0.310686758863888, - 1.0 - ], - "O_T_TCP": [ - -0.9935878908904953, - 0.08960520143219396, - -0.06880957675557518, - 0.0, - 0.09120083791165756, - 0.9956138318810523, - -0.02040225067357439, - 0.0, - 0.06668090241645759, - -0.026547431392950203, - -0.9974211202593233, - 0.0, - 0.5080013634448317, - 0.07744840851681808, - 0.310686758863888, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "test_object", - "q": [ - -0.6783759433123, - -0.16586847048823597, - 0.8175371052577723, - -2.261259556151273, - 0.1062502110335562, - 2.2074089546736957, - -2.190133549686696 - ] - }, - { - "_id": "65cfa8e660f0281a320a1ef1", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.9999648408880725, - 0.0002669773375134615, - 0.00714091114644764, - 0.0, - -4.124753500410915e-05, - -0.9990599232033038, - 0.04312789177385854, - 0.0, - 0.00714584989104384, - -0.0431275003184912, - -0.9990440208247148, - 0.0, - 0.5039719752053793, - 0.08282098192276433, - 0.4255527712058619, - 1.0 - ], - "O_T_TCP": [ - 0.9999648408880725, - 0.0002669773375134615, - 0.00714091114644764, - 0.0, - -4.124753500410915e-05, - -0.9990599232033038, - 0.04312789177385854, - 0.0, - 0.00714584989104384, - -0.0431275003184912, - -0.9990440208247148, - 0.0, - 0.5039719752053793, - 0.08282098192276433, - 0.4255527712058619, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "aaa", - "q": [ - -0.7009867876172065, - -0.24024575297265152, - 0.7995970885804996, - -1.9988235254230773, - 0.13235241008271123, - 1.8310430494149526, - 0.8356096653159838 - ] - }, - { - "_id": "65d8ab25365be5466b0fdc71", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.9021773910215635, - -0.038249869200695535, - 0.42964363085802043, - 0.0, - -0.03475251865213935, - -0.9992584430166064, - -0.015986663459536494, - 0.0, - 0.42994479117900936, - -0.0005084017524109786, - -0.9028550371270552, - 0.0, - 0.5071971613100603, - 0.04661989698448783, - 0.6363466027888695, - 1.0 - ], - "O_T_TCP": [ - 0.9021773910215635, - -0.038249869200695535, - 0.42964363085802043, - 0.0, - -0.03475251865213935, - -0.9992584430166064, - -0.015986663459536494, - 0.0, - 0.42994479117900936, - -0.0005084017524109786, - -0.9028550371270552, - 0.0, - 0.5071971613100603, - 0.04661989698448783, - 0.6363466027888695, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "yawgripper", - "q": [ - 0.11421029169640254, - -0.34931760987484234, - -0.004396662544784647, - -1.792835893163279, - -0.05164978933003214, - 1.8851889863806732, - 0.9384602965859719 - ] - }, - { - "_id": "65d8bc19d2850057440d7331", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.026665700373889236, - 0.9990310854222944, - -0.03473582710589628, - 0.0, - 0.9993871064511708, - -0.025869536002868367, - 0.023171652236316353, - 0.0, - 0.022251029556039845, - -0.03533310634903862, - -0.9991278513180502, - 0.0, - 0.34529017685396074, - 0.28611390088978156, - 0.32766119170850694, - 1.0 - ], - "O_T_TCP": [ - 0.026665700373889236, - 0.9990310854222944, - -0.03473582710589628, - 0.0, - 0.9993871064511708, - -0.025869536002868367, - 0.023171652236316353, - 0.0, - 0.022251029556039845, - -0.03533310634903862, - -0.9991278513180502, - 0.0, - 0.34529017685396074, - 0.28611390088978156, - 0.32766119170850694, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "outwardgripper", - "q": [ - 0.870870500266552, - -0.2957360618239955, - -0.1551851647736568, - -2.386680684173316, - -0.09917618758749594, - 2.0854945325421754, - 0.012588030038608445 - ] - }, - { - "_id": "65d8bc40d2850057440d7332", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.00497544968458589, - 0.9985565104873484, - -0.05329997340500955, - 0.0, - 0.9996668751570742, - -0.003637348545899981, - 0.025172511008136226, - 0.0, - 0.024942784397619912, - -0.05340849070064443, - -0.9982611835725023, - 0.0, - 0.5059302674034952, - 0.2832485707890151, - 0.3287370944613403, - 1.0 - ], - "O_T_TCP": [ - 0.00497544968458589, - 0.9985565104873484, - -0.05329997340500955, - 0.0, - 0.9996668751570742, - -0.003637348545899981, - 0.025172511008136226, - 0.0, - 0.024942784397619912, - -0.05340849070064443, - -0.9982611835725023, - 0.0, - 0.5066785509354238, - 0.2816463160679958, - 0.29878925895416525, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "inwardgripper", - "q": [ - 0.20320357527440056, - 0.10390104585999435, - 0.33527658712487163, - -1.9410412257221545, - -0.10363888059547315, - 2.0311467233498885, - -0.19773421850766457 - ] - }, - { - "_id": "65d8bc9cd2850057440d7333", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.9159565622373924, - 0.11163201311977883, - 0.38541226841175086, - 0.0, - 0.14128903439883572, - -0.9887250951913703, - -0.04940487626407901, - 0.0, - 0.3755588466496142, - 0.0997091675804246, - -0.9214193587089595, - 0.0, - 0.4479657579420528, - 0.04195357437032795, - 0.6367311153952495, - 1.0 - ], - "O_T_TCP": [ - 0.9159565622373924, - 0.11163201311977883, - 0.38541226841175086, - 0.0, - 0.14128903439883572, - -0.9887250951913703, - -0.04940487626407901, - 0.0, - 0.3755588466496142, - 0.0997091675804246, - -0.9214193587089595, - 0.0, - 0.4479657579420528, - 0.04195357437032795, - 0.6367311153952495, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "defaultgripper", - "q": [ - 0.021865537859103813, - -0.491120350155914, - 0.026315845489235032, - -1.8907418638905291, - 0.09657258354955249, - 1.7900419552325462, - 0.6985316049030074 - ] - }, - { - "_id": "65d8bcbbd2850057440d7334", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.02641181045508553, - 0.9985793852615791, - 0.04606923594221673, - 0.0, - 0.9996335248133997, - -0.02656782429594709, - 0.0027773517108995484, - 0.0, - 0.003997442493241836, - 0.04597988307865417, - -0.998934367616605, - 0.0, - 0.3336599900824048, - -0.3329920688645952, - 0.3250164621695375, - 1.0 - ], - "O_T_TCP": [ - 0.02641181045508553, - 0.9985793852615791, - 0.04606923594221673, - 0.0, - 0.9996335248133997, - -0.02656782429594709, - 0.0027773517108995484, - 0.0, - 0.003997442493241836, - 0.04597988307865417, - -0.998934367616605, - 0.0, - 0.3337799133572021, - -0.3316126723722356, - 0.29504843114103935, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "parallelgripper", - "q": [ - -0.3883849079065155, - -0.22314950738888212, - -0.38944405417958633, - -2.302035715802051, - -0.054835104276907666, - 2.0653319421609244, - -1.5000054941865528 - ] - }, - { - "_id": "65d8bd19d2850057440d7335", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.01657188384160618, - 0.9990552433374398, - 0.03993419996841205, - 0.0, - 0.9998158195093524, - -0.016213384129055507, - -0.009284402788233788, - 0.0, - -0.00862832888493567, - 0.04008147660504227, - -0.9991591600810217, - 0.0, - 0.4936055805433891, - -0.33471007413492543, - 0.3264931543876324, - 1.0 - ], - "O_T_TCP": [ - 0.01657188384160618, - 0.9990552433374398, - 0.03993419996841205, - 0.0, - 0.9998158195093524, - -0.016213384129055507, - -0.009284402788233788, - 0.0, - -0.00862832888493567, - 0.04008147660504227, - -0.9991591600810217, - 0.0, - 0.4936055805433891, - -0.33471007413492543, - 0.3264931543876324, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "clampgripper", - "q": [ - -0.23730045444296113, - 0.17615896566081463, - -0.3904649787620021, - -1.854275331395351, - 0.10299784066871676, - 1.9849424978146744, - -1.4353288235127737 - ] - }, - { - "_id": "65d8c5483d14aeb8d50802f1", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.9450272707317494, - 0.3268482835097846, - -0.008625777716608166, - 0.0, - 0.3269585226370145, - -0.944811193696208, - 0.020265232359282376, - 0.0, - -0.0015261043116654416, - -0.021971891793058175, - -0.9997574240667904, - 0.0, - 0.4457469370221283, - -0.16079376982416693, - 0.3110585447747644, - 1.0 - ], - "O_T_TCP": [ - 0.9450272707317494, - 0.3268482835097846, - -0.008625777716608166, - 0.0, - 0.3269585226370145, - -0.944811193696208, - 0.020265232359282376, - 0.0, - -0.0015261043116654416, - -0.021971891793058175, - -0.9997574240667904, - 0.0, - 0.44555617398317016, - -0.1635402562982992, - 0.18608886676641562, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "outerring", - "q": [ - -0.125661001253863, - -0.2077394962644658, - -0.20623197082469338, - -2.3448877243461714, - -0.0757377231783337, - 2.145753136721135, - 0.1656858233103826 - ] - }, - { - "_id": "65d8caf6d9a0e0f6b30af271", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.8554338023472642, - 0.5174638336014817, - 0.021093544447718618, - 0.0, - 0.5175700178929962, - -0.8556294640346566, - 0.0004937157258281831, - 0.0, - 0.018304090573053294, - 0.010495247122638255, - -0.9997773802482871, - 0.0, - 0.41513337774189596, - -0.023751420869901698, - 0.382635995099988, - 1.0 - ], - "O_T_TCP": [ - 0.8554338023472642, - 0.5174638336014817, - 0.021093544447718618, - 0.0, - 0.5175700178929962, - -0.8556294640346566, - 0.0004937157258281831, - 0.0, - 0.018304090573053294, - 0.010495247122638255, - -0.9997773802482871, - 0.0, - 0.4174213890635276, - -0.022439514979571917, - 0.2576638225689521, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "housingringhole", - "q": [ - 0.3282597493989426, - -0.4762393398495533, - -0.3451653095044588, - -2.4108050668383676, - -0.15829846423839866, - 1.9730392518838247, - 0.3199505141509904 - ] - }, - { - "_id": "65d8db2f6acb8ae6770b96c1", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.9143601578871972, - 0.4043119668520558, - -0.021402850976683194, - 0.0, - 0.40487805559894086, - -0.9130695248500423, - 0.048564902818258454, - 0.0, - 9.308219883339745e-05, - -0.05307237870414294, - -0.9985906638630205, - 0.0, - 0.3797952597363438, - -0.15738937435577127, - 0.3132205190879492, - 1.0 - ], - "O_T_TCP": [ - 0.9143601578871972, - 0.4043119668520558, - -0.021402850976683194, - 0.0, - 0.40487805559894086, - -0.9130695248500423, - 0.048564902818258454, - 0.0, - 9.308219883339745e-05, - -0.05307237870414294, - -0.9985906638630205, - 0.0, - 0.37980689501119796, - -0.16402342169378914, - 0.1883966861050716, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "cone", - "q": [ - -0.2032646288602205, - -0.4149487602153682, - -0.16325278544147923, - -2.5386192064467603, - -0.13731561666720551, - 2.1428268275003575, - 0.08865713309579425 - ] - }, - { - "_id": "65d8dba16acb8ae6770b96c2", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.999987906224758, - -0.0020268784904560297, - -0.000908883390595556, - 0.0, - -0.0020227675266590835, - -0.9999781961903642, - 0.004501383854620223, - 0.0, - -0.0009180050060284309, - -0.0044995775868994035, - -0.9999894554785809, - 0.0, - 0.5494106175501344, - 0.14342966321250666, - 0.3198477346620273, - 1.0 - ], - "O_T_TCP": [ - 0.999987906224758, - -0.0020268784904560297, - -0.000908883390595556, - 0.0, - -0.0020227675266590835, - -0.9999781961903642, - 0.004501383854620223, - 0.0, - -0.0009180050060284309, - -0.0044995775868994035, - -0.9999894554785809, - 0.0, - 0.5493830773999536, - 0.1432946758848997, - 0.2898480509976699, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "outputshaftandgearstage2", - "q": [ - 0.08933115612980294, - 0.0692455929224951, - 0.1706978926679542, - -2.003876755023256, - -0.018072482041186756, - 2.0700078295868973, - 1.0557316942269082 - ] - }, - { - "_id": "65d8dbd46acb8ae6770b96c3", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.6622518892013779, - 0.7492680325152687, - 0.0007730466431225671, - 0.0, - 0.7490190660730002, - -0.662057747688239, - 0.02511422478029191, - 0.0, - 0.019329459460671793, - -0.016053225206244647, - -0.9996842831401501, - 0.0, - 0.3913690827217789, - 0.12185252422573754, - 0.27162472763954526, - 1.0 - ], - "O_T_TCP": [ - 0.6622518892013779, - 0.7492680325152687, - 0.0007730466431225671, - 0.0, - 0.7490190660730002, - -0.662057747688239, - 0.02511422478029191, - 0.0, - 0.019329459460671793, - -0.016053225206244647, - -0.9996842831401501, - 0.0, - 0.393012086775936, - 0.12048800008320674, - 0.1866515635726325, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "ringgear", - "q": [ - -0.050023251361175095, - -0.37452352120583504, - 0.3575884737508338, - -2.6005898082766583, - 0.13921116405063205, - 2.2571393044553196, - 0.13618820093707001 - ] - }, - { - "_id": "65d8dc2a6acb8ae6770b96c4", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.9996779670050843, - 0.006263488697202127, - 0.02419665049402185, - 0.0, - 0.006070379334143311, - -0.9999395779731979, - 0.00804598028401658, - 0.0, - 0.024246051201543127, - -0.007896658400918596, - -0.9996748330268355, - 0.0, - 0.559343401988589, - -0.05948698017335339, - 0.2961342709605277, - 1.0 - ], - "O_T_TCP": [ - 0.9996779670050843, - 0.006263488697202127, - 0.02419665049402185, - 0.0, - 0.006070379334143311, - -0.9999395779731979, - 0.00804598028401658, - 0.0, - 0.024246051201543127, - -0.007896658400918596, - -0.9996748330268355, - 0.0, - 0.5616710229039371, - -0.06024505937984157, - 0.20016548698995149, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "designring", - "q": [ - -0.23980222428148423, - 0.06263345414970149, - 0.13727877759514645, - -2.0821014380886256, - -0.016858025526053484, - 2.169014367977778, - 0.6856549627714604 - ] - }, - { - "_id": "65d8dc9d6acb8ae6770b96c5", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.9917030439487258, - -0.12707497619338115, - -0.0189148076779041, - 0.0, - -0.12738495800105223, - -0.9917114008538792, - -0.01619619685960171, - 0.0, - -0.016700220619776197, - 0.01847163534392772, - -0.9996899025792809, - 0.0, - 0.663811885071489, - 0.02260876703652303, - 0.2829037052074508, - 1.0 - ], - "O_T_TCP": [ - 0.9917030439487258, - -0.12707497619338115, - -0.0189148076779041, - 0.0, - -0.12738495800105223, - -0.9917114008538792, - -0.01619619685960171, - 0.0, - -0.016700220619776197, - 0.01847163534392772, - -0.9996899025792809, - 0.0, - 0.662392366318808, - 0.024178856040756886, - 0.1979300634882119, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "gearstate1", - "q": [ - -0.08354264591032998, - 0.4030342974513543, - 0.1324892411447409, - -1.6324215784737326, - -0.0362645463593793, - 2.0160468011424646, - 0.9677353810403081 - ] - }, - { - "_id": "65d8dcaa6acb8ae6770b96c6", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - -0.21256157292254477, - 0.9763035678396146, - -0.04036914714903944, - 0.0, - 0.9769003641039072, - 0.21323922159381858, - 0.013246127317972726, - 0.0, - 0.02154094160493881, - -0.0366217219722513, - -0.999097010962679, - 0.0, - 0.6634628466452309, - 0.025898522896490285, - 0.32342715523209353, - 1.0 - ], - "O_T_TCP": [ - -0.21256157292254477, - 0.9763035678396146, - -0.04036914714903944, - 0.0, - 0.9769003641039072, - 0.21323922159381858, - 0.013246127317972726, - 0.0, - 0.02154094160493881, - -0.0366217219722513, - -0.999097010962679, - 0.0, - 0.6655092360977001, - 0.02241945930912641, - 0.22851293919063903, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "gearstage1", - "q": [ - -0.2310496118665634, - 0.36485854234517584, - 0.32882389615487156, - -1.6110621550563742, - -0.16700756684169124, - 1.9717454081062231, - -0.8593527560681966 - ] - }, - { - "_id": "65d8df7a6acb8ae6770b96c7", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.9983457491217146, - -0.05208751661697281, - -0.023945828958038713, - 0.0, - -0.05208269487800601, - -0.9986327946173754, - 0.0008254151343264248, - 0.0, - -0.023956545154051528, - 0.00042312175901195674, - -0.9997129112461532, - 0.0, - 0.6761475033614672, - -0.17466713886448487, - 0.2722830986146639, - 1.0 - ], - "O_T_TCP": [ - 0.9983457491217146, - -0.05208751661697281, - -0.023945828958038713, - 0.0, - -0.05208269487800601, - -0.9986327946173754, - 0.0008254151343264248, - 0.0, - -0.023956545154051528, - 0.00042312175901195674, - -0.9997129112461532, - 0.0, - 0.6755485897326159, - -0.17465656082050957, - 0.24729027583351004, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "driveflange", - "q": [ - -0.38208588488970274, - 0.5345268294201574, - 0.16745628726830958, - -1.4567519754275942, - -0.09753089586893726, - 1.9599688848904608, - 0.6390367531119816 - ] - }, - { - "_id": "65d8dfdc6acb8ae6770b96c8", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.9941529157822993, - 0.10660892842746389, - -0.016591061518444365, - 0.0, - 0.10667911326709052, - -0.9942780001076592, - 0.003401793008004694, - 0.0, - -0.016133776584101003, - -0.005151921359193992, - -0.9998565691935235, - 0.0, - 0.6690201696258397, - 0.14134393172722232, - 0.36692858415213564, - 1.0 - ], - "O_T_TCP": [ - 0.9941529157822993, - 0.10660892842746389, - -0.016591061518444365, - 0.0, - 0.10667911326709052, - -0.9942780001076592, - 0.003401793008004694, - 0.0, - -0.016133776584101003, - -0.005151921359193992, - -0.9998565691935235, - 0.0, - 0.667003447552827, - 0.14069994155732307, - 0.2419465130029452, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "inputpinion", - "q": [ - -0.36658148687450515, - 0.5480307648809332, - 0.7873751252802971, - -1.3581701436797236, - -0.3828242980717707, - 1.7342026976388893, - 1.0925998417091114 - ] - }, - { - "_id": "65db67fc47cc9af7370f5c21", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.9999838476755807, - 0.0007041745488701401, - -0.0035433638472598056, - 0.0, - 0.0007106663799625451, - -0.9999884442749796, - 0.0018311665710162486, - 0.0, - -0.003542101636660161, - -0.0018336904472012812, - -0.9999920455160327, - 0.0, - 0.41620961031018017, - -0.026741139813605378, - 0.36498461276282146, - 1.0 - ], - "O_T_TCP": [ - 0.9999838476755807, - 0.0007041745488701401, - -0.0035433638472598056, - 0.0, - 0.0007106663799625451, - -0.9999884442749796, - 0.0018311665710162486, - 0.0, - -0.003542101636660161, - -0.0018336904472012812, - -0.9999920455160327, - 0.0, - 0.41610334726108034, - -0.026796150527021417, - 0.33498485139734047, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "housinginternalgear", - "q": [ - -0.11159760705002565, - -0.42468266202550486, - 0.04340141112268784, - -2.4098499892757688, - 0.01723102298269882, - 1.9820651748871552, - 0.7056970563663375 - ] - }, - { - "_id": "65ddf561dd14b6fcb70b9181", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.838073489459574, - -0.545359567091919, - 0.014018407455063152, - 0.0, - -0.5455096437563031, - -0.838020349010329, - 0.011039479819545006, - 0.0, - 0.005727335041701339, - -0.016899397196790594, - -0.9998407913301524, - 0.0, - 0.41665775428811364, - -0.022347053933471497, - 0.40320964752376237, - 1.0 - ], - "O_T_TCP": [ - 0.838073489459574, - -0.545359567091919, - 0.014018407455063152, - 0.0, - -0.5455096437563031, - -0.838020349010329, - 0.011039479819545006, - 0.0, - 0.005727335041701339, - -0.016899397196790594, - -0.9998407913301524, - 0.0, - 0.41720185111707525, - -0.023952496667166603, - 0.30822477234739787, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "outputshaftandgearstage2push", - "q": [ - -0.10971178873380023, - -0.44959311833298, - 0.05445318151752114, - -2.334556603908351, - 0.007518621839784734, - 1.8922768517418738, - 1.2994310701356993 - ] - }, - { - "_id": "65de1054b0a8a8d89e01bae1", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.7778359413079095, - 0.6283849958747305, - 0.00918108219990433, - 0.0, - 0.6283421108618183, - -0.777890121068528, - 0.007341536822329491, - 0.0, - 0.011755411057927802, - 5.835048806805542e-05, - -0.9999309010657085, - 0.0, - 0.4139451520314652, - -0.02436540952955223, - 0.3932297425400695, - 1.0 - ], - "O_T_TCP": [ - 0.7778359413079095, - 0.6283849958747305, - 0.00918108219990433, - 0.0, - 0.6283421108618183, - -0.777890121068528, - 0.007341536822329491, - 0.0, - 0.011755411057927802, - 5.835048806805542e-05, - -0.9999309010657085, - 0.0, - 0.41494436197138906, - -0.024360449738066444, - 0.30823561594948423, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "housingringgearhole", - "q": [ - -0.6092992286082981, - -0.5185007247422704, - 0.4696629144434343, - -2.3612041679220486, - 0.24119912558131745, - 1.893441767613093, - -0.16335311901203906 - ] - }, - { - "_id": "65de2826ea2690710508e2a2", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.99938656451743, - -0.008789511194172067, - 0.033615265226152685, - 0.0, - -0.008553028984903552, - -0.9999280732004046, - -0.0071722396863144544, - 0.0, - 0.03367653624639689, - 0.006880460112365144, - -0.9994091005064379, - 0.0, - 0.407952755049321, - -0.007845277911615304, - 0.651871823910401, - 1.0 - ], - "O_T_TCP": [ - 0.99938656451743, - -0.008789511194172067, - 0.033615265226152685, - 0.0, - -0.008553028984903552, - -0.9999280732004046, - -0.0071722396863144544, - 0.0, - 0.03367653624639689, - 0.006880460112365144, - -0.9994091005064379, - 0.0, - 0.4121623220801206, - -0.006985220397569661, - 0.5269456863470963, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "initialposition", - "q": [ - -0.13408752005664926, - -0.33240807310321874, - 0.0789536473301419, - -1.5773980297958639, - 0.03611862126323087, - 1.2793651213177129, - 0.7460172530804181 - ] - }, - { - "_id": "65de2bf7ea2690710508e2a3", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.9877714483643791, - -0.15511337294124264, - 0.0151047751435027, - 0.0, - -0.15483615905495418, - -0.9877652457281556, - -0.018064608545818848, - 0.0, - 0.017722375504342, - 0.01550523770470364, - -0.9997227140613562, - 0.0, - 0.4151507328760942, - -0.02478245960516119, - 0.4001269305681699, - 1.0 - ], - "O_T_TCP": [ - 0.9877714483643791, - -0.15511337294124264, - 0.0151047751435027, - 0.0, - -0.15483615905495418, - -0.9877652457281556, - -0.018064608545818848, - 0.0, - 0.017722375504342, - 0.01550523770470364, - -0.9997227140613562, - 0.0, - 0.4156824041412245, - -0.024317302474020083, - 0.3701352491463292, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "inputpinionhole", - "q": [ - -0.1524973680711629, - -0.45994008938889563, - 0.07378913508395114, - -2.352641371128198, - 0.05270322115737649, - 1.9096883613938846, - 0.8378086123665174 - ] - }, - { - "_id": "65df7b0e33ffaa5ce60d6d91", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.9694218836090096, - 0.245216154306467, - -0.008425921214629994, - 0.0, - 0.2453607861926783, - -0.9688787985344701, - 0.03244541956148812, - 0.0, - -0.00020755940941075935, - -0.033521335788991205, - -0.9994379805500758, - 0.0, - 0.4155900381055534, - -0.019981089817033608, - 0.46728264954328513, - 1.0 - ], - "O_T_TCP": [ - 0.9694218836090096, - 0.245216154306467, - -0.008425921214629994, - 0.0, - 0.2453607861926783, - -0.9688787985344701, - 0.03244541956148812, - 0.0, - -0.00020755940941075935, - -0.033521335788991205, - -0.9994379805500758, - 0.0, - 0.41556409317937704, - -0.024171256790657507, - 0.34235290197452567, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "gearstage1hole", - "q": [ - -0.5033082389967483, - -0.49386459897873464, - 0.3720935603702197, - -2.171352911664728, - 0.14152964365679774, - 1.7106926437493268, - 0.34620421599679524 - ] - }, - { - "_id": "65df839833ffaa5ce60d6d92", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.9849884636012461, - -0.16474774592076277, - 0.051348356231651494, - 0.0, - -0.16610044520500197, - -0.9858245817773944, - 0.023265488761975345, - 0.0, - 0.046788435798120906, - -0.031445828290851165, - -0.9984097366105098, - 0.0, - 0.4109011111656137, - -0.018412318551761418, - 0.45990058464841843, - 1.0 - ], - "O_T_TCP": [ - 0.9849884636012461, - -0.16474774592076277, - 0.051348356231651494, - 0.0, - -0.16610044520500197, - -0.9858245817773944, - 0.023265488761975345, - 0.0, - 0.046788435798120906, - -0.031445828290851165, - -0.9984097366105098, - 0.0, - 0.4167496656403788, - -0.022343047088117813, - 0.3350993675721047, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "gearstage1push", - "q": [ - -0.5060363284882745, - -0.5448256020629615, - 0.37240348189998196, - -2.23849014419654, - 0.170629203718004, - 1.7741905958652497, - 0.7426301942090511 - ] - }, - { - "_id": "65df8e9835c7aacd3e08bf11", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.9988364672084291, - 0.010973019303475431, - 0.046755229901674425, - 0.0, - 0.009496371636274339, - -0.9994430591113914, - 0.03168812744574244, - 0.0, - 0.04707781083028177, - -0.03120785307914851, - -0.9984036005712411, - 0.0, - 0.41061644653585816, - -0.021354251407883586, - 0.43392191517524437, - 1.0 - ], - "O_T_TCP": [ - 0.9988364672084291, - 0.010973019303475431, - 0.046755229901674425, - 0.0, - 0.009496371636274339, - -0.9994430591113914, - 0.03168812744574244, - 0.0, - 0.04707781083028177, - -0.03120785307914851, - -0.9984036005712411, - 0.0, - 0.4151359163755652, - -0.024350205303481844, - 0.3380751695204052, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "designringhole", - "q": [ - -0.10978179252670522, - -0.5024384693346525, - 0.058291886597341296, - -2.3094043837387934, - -0.0003870843260342048, - 1.8565294750761316, - 0.7172597897123906 - ] - }, - { - "_id": "65df903e7c41a24c1a097331", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.9994981335228094, - -0.022569609705585393, - 0.021790839895834878, - 0.0, - -0.023064174087633997, - -0.9994662070350049, - 0.02271765326622348, - 0.0, - 0.021266888984666322, - -0.023209286613170076, - -0.999504401414932, - 0.0, - 0.4149244678428837, - -0.023597044494400223, - 0.38427019571613635, - 1.0 - ], - "O_T_TCP": [ - 0.9994981335228094, - -0.022569609705585393, - 0.021790839895834878, - 0.0, - -0.023064174087633997, - -0.9994662070350049, - 0.02271765326622348, - 0.0, - 0.021266888984666322, - -0.023209286613170076, - -0.999504401414932, - 0.0, - 0.41545614006750037, - -0.024177276659729473, - 0.35928258568076304, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "driveflangehole", - "q": [ - -0.36328720980882645, - -0.47589010498910833, - 0.275991911264888, - -2.391791089788313, - 0.11272685719860923, - 1.9531820533201192, - 0.6520902942570738 - ] - } -] \ No newline at end of file diff --git a/experiments/gearbox/mios_memory_backups/gearbox_exp.json b/experiments/gearbox/mios_memory_backups/gearbox_exp.json deleted file mode 100644 index f9877b40..00000000 --- a/experiments/gearbox/mios_memory_backups/gearbox_exp.json +++ /dev/null @@ -1,3502 +0,0 @@ -[ - { - "_id": "643d40f9dd8c607f760e5af8", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "NullObject", - "q": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ] - }, - { - "_id": "643d40f9dd8c607f760e5af9", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "NoneObject", - "q": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ] - }, - { - "_id": "643d40f9dd8c607f760e5afa", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.9964428031472694, - 0.08294013363142141, - 0.014262580075939998, - 0.0, - 0.08311109811989041, - -0.9964602553113516, - -0.01184279761501484, - 0.0, - 0.013230105687642353, - 0.012986299171246269, - -0.999828145401663, - 0.0, - 0.4431209230579011, - -0.166559761752491, - 0.4602278757574641, - 1.0 - ], - "O_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "EndEffector", - "q": [ - -0.42379051686692654, - -0.2828940049879727, - 0.04989039639235013, - -2.0382506847381587, - 0.03143752897779147, - 1.7631404176553087, - 0.32054617522995593 - ] - }, - { - "_id": "6491ad548829c1097f0151a9", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "ring", - "q": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ] - }, - { - "_id": "64f8a17c8d1b98ac5f0c6b91", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "geometry": null, - "grasp_force": 1.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "TestObject1", - "q": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ] - }, - { - "_id": "6526b75ece654b001f026d61", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "gripper_connected", - "q": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ] - }, - { - "_id": "6526da7d86c098f7fe057f61", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "tool_load", - "q": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ] - }, - { - "_id": "6530168b631a09bd8e08d211", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "tool_grasp", - "q": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ] - }, - { - "_id": "65a8008494d3e83f7e035b11", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "kios_guidance", - "q": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ] - }, - { - "_id": "65b2664c677d37a9900fd7b1", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.020523632927385155, - 0.999779614758663, - 0.0004993052070315127, - 0.0, - 0.9997626802269471, - -0.020520362759945312, - -0.00585190899106819, - 0.0, - -0.0058404858401676345, - 0.000619301067554896, - -0.99998275244693, - 0.0, - 0.49407319098460234, - -0.3343870569274952, - 0.3249152265674699, - 1.0 - ], - "O_T_TCP": [ - 0.020523632927385155, - 0.999779614758663, - 0.0004993052070315127, - 0.0, - 0.9997626802269471, - -0.020520362759945312, - -0.00585190899106819, - 0.0, - -0.0058404858401676345, - 0.000619301067554896, - -0.99998275244693, - 0.0, - 0.49407319098460234, - -0.3343870569274952, - 0.3249152265674699, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "parallel_box1", - "q": [ - -0.4889897713452055, - 0.1520400121422783, - -0.11095685046074681, - -1.8813407051080338, - 0.015587054766497403, - 2.027237653652827, - -1.3705014213786446 - ] - }, - { - "_id": "65bb9aa7666d3392110788a1", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - -0.0055492434676263965, - 0.9999687095518451, - 0.003540157533654855, - 0.0, - 0.9994642785591136, - 0.005433245086445076, - 0.0319747186594265, - 0.0, - 0.03195509884729998, - 0.003715768034059008, - -0.9994823984070837, - 0.0, - 0.5073771436733974, - 0.2821340073022507, - 0.3273610300796691, - 1.0 - ], - "O_T_TCP": [ - -0.0055492434676263965, - 0.9999687095518451, - 0.003540157533654855, - 0.0, - 0.9994642785591136, - 0.005433245086445076, - 0.0319747186594265, - 0.0, - 0.03195509884729998, - 0.003715768034059008, - -0.9994823984070837, - 0.0, - 0.5073771436733974, - 0.2821340073022507, - 0.3273610300796691, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "inward_claw", - "q": [ - -0.3558367933193034, - 0.12986394129696988, - 0.8984244877664667, - -1.9720981870183556, - -0.1313234915672495, - 2.079114055554075, - -0.1914156752901866 - ] - }, - { - "_id": "65bbb462b8aab7a7d90d58c1", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.9949942224611444, - -0.05041086343049174, - 0.08617417837347448, - 0.0, - -0.04503393357509275, - -0.9969720659778388, - -0.06324074151875426, - 0.0, - 0.08910298453945457, - 0.05904454700105643, - -0.9942707878719971, - 0.0, - 0.6906053200978064, - -0.01614832324583123, - 0.2887581517923185, - 1.0 - ], - "O_T_TCP": [ - 0.9949942224611444, - -0.05041086343049174, - 0.08617417837347448, - 0.0, - -0.04503393357509275, - -0.9969720659778388, - -0.06324074151875426, - 0.0, - 0.08910298453945457, - 0.05904454700105643, - -0.9942707878719971, - 0.0, - 0.6990701036290545, - -0.01053909128073087, - 0.1943024269444788, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "test_object1", - "q": [ - -0.5258051569263356, - 0.4877585826129244, - 0.5891767306537179, - -1.6077540129764138, - -0.23719313260094485, - 2.1022328375975285, - 0.9414903357566682 - ] - }, - { - "_id": "65bbba7c5a0f2e8e04072891", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.005516821389947922, - -0.9949443596914403, - -0.10018000150304039, - 0.0, - -0.999821715401967, - -0.003733290448562814, - -0.017981847911249182, - 0.0, - 0.017517274375192557, - 0.10026327397364804, - -0.9948067254449727, - 0.0, - 0.643320166007396, - 0.11088726408849338, - 0.3035784369329164, - 1.0 - ], - "O_T_TCP": [ - 0.005516821389947922, - -0.9949443596914403, - -0.10018000150304039, - 0.0, - -0.999821715401967, - -0.003733290448562814, - -0.017981847911249182, - 0.0, - 0.017517274375192557, - 0.10026327397364804, - -0.9948067254449727, - 0.0, - 0.6455098253042951, - 0.12342017333519939, - 0.17922759625229484, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "gear3", - "q": [ - -0.4348329623676183, - 0.39139715203497827, - 0.6694900968644402, - -1.711178953283276, - -0.16270597490334937, - 2.05581780107816, - 2.617633598559432 - ] - }, - { - "_id": "65bbbab25a0f2e8e04072892", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.9980187907769038, - -0.05571650543342081, - -0.02889482963615323, - 0.0, - -0.05609811498518191, - -0.9983365128919355, - -0.012568031790528969, - 0.0, - -0.028147058566118602, - 0.014164350073066843, - -0.9995034338515715, - 0.0, - 0.507308310840537, - -0.04026469732735698, - 0.3761685279429274, - 1.0 - ], - "O_T_TCP": [ - 0.9980187907769038, - -0.05571650543342081, - -0.02889482963615323, - 0.0, - -0.05609811498518191, - -0.9983365128919355, - -0.012568031790528969, - 0.0, - -0.028147058566118602, - 0.014164350073066843, - -0.9995034338515715, - 0.0, - 0.5046343402767557, - -0.038919084070415624, - 0.2812157017270281, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "shaft2", - "q": [ - -0.04717889234179201, - -0.13185073856974885, - -0.034486469687599886, - -2.0883627749325773, - 0.0077889278013025046, - 1.9273695102532702, - 0.7572235297293566 - ] - }, - { - "_id": "65bfcf0f5a0e8797bb01e281", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.9991077043349246, - -0.009049464669151653, - -0.041020107618180826, - 0.0, - -0.009843038297554691, - -0.9997578641005138, - -0.019185272149743374, - 0.0, - -0.0408373449804758, - 0.019572292532108137, - -0.9989740920664474, - 0.0, - 0.4292203229453998, - -0.033877595855194384, - 0.4236606084507305, - 1.0 - ], - "O_T_TCP": [ - 0.9991077043349246, - -0.009049464669151653, - -0.041020107618180826, - 0.0, - -0.009843038297554691, - -0.9997578641005138, - -0.019185272149743374, - 0.0, - -0.0408373449804758, - 0.019572292532108137, - -0.9989740920664474, - 0.0, - 0.4292203229453998, - -0.033877595855194384, - 0.4236606084507305, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "test", - "q": [ - -0.028742359890226728, - -0.3809770326088004, - -0.048948421164719876, - -2.2124784198559695, - -0.0017267249742586607, - 1.7897263017493257, - 0.7214576442114876 - ] - }, - { - "_id": "65ce303af045be1f60050f41", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - -0.00034130509165598697, - 0.9999088204921296, - 0.012766405683707687, - 0.0, - 0.9999006778257512, - 0.00017031581705472163, - 0.013392250644843597, - 0.0, - 0.01338911300712114, - 0.012769954401065334, - -0.9998288153066389, - 0.0, - 0.34586891787172797, - 0.28478778305517327, - 0.325633698814514, - 1.0 - ], - "O_T_TCP": [ - -0.00034130509165598697, - 0.9999088204921296, - 0.012766405683707687, - 0.0, - 0.9999006778257512, - 0.00017031581705472163, - 0.013392250644843597, - 0.0, - 0.01338911300712114, - 0.012769954401065334, - -0.9998288153066389, - 0.0, - 0.34586891787172797, - 0.28478778305517327, - 0.325633698814514, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "outward_claw", - "q": [ - -0.26322421671319607, - -0.48738894555267787, - 0.8759835133827272, - -2.392889458566437, - 0.4210987845394346, - 2.0469462890460046, - -0.42977081686632607 - ] - }, - { - "_id": "65ce3098f045be1f60050f42", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.016519502886665666, - 0.9997483636116822, - -0.014527986142256165, - 0.0, - 0.999314451509194, - -0.01603160985419312, - 0.033081133468638016, - 0.0, - 0.0328405343279425, - -0.015064800427202131, - -0.999347062382806, - 0.0, - 0.3334408867064976, - -0.33165373934959796, - 0.32422469388972297, - 1.0 - ], - "O_T_TCP": [ - 0.016519502886665666, - 0.9997483636116822, - -0.014527986142256165, - 0.0, - 0.999314451509194, - -0.01603160985419312, - 0.033081133468638016, - 0.0, - 0.0328405343279425, - -0.015064800427202131, - -0.999347062382806, - 0.0, - 0.3334408867064976, - -0.33165373934959796, - 0.32422469388972297, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "parallel_box2", - "q": [ - -0.49195081603365076, - -0.2536298767587599, - -0.28172348515657036, - -2.357436443128084, - -0.06903641326254598, - 2.1465019415994413, - -1.4987806172708702 - ] - }, - { - "_id": "65ce3113f045be1f60050f43", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.05033441300943016, - 0.998522325819256, - -0.020008963209219598, - 0.0, - 0.9980401159147424, - -0.049549287905040595, - 0.037967644043304945, - 0.0, - 0.036920821197481454, - -0.021881248318769652, - -0.9990786074849747, - 0.0, - 0.4951086360500897, - -0.1768782158810579, - 0.5007737812082008, - 1.0 - ], - "O_T_TCP": [ - 0.05033441300943016, - 0.998522325819256, - -0.020008963209219598, - 0.0, - 0.9980401159147424, - -0.049549287905040595, - 0.037967644043304945, - 0.0, - 0.036920821197481454, - -0.021881248318769652, - -0.9990786074849747, - 0.0, - 0.49824690585187564, - -0.1787381219881533, - 0.415852099571978, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "shaft1", - "q": [ - -0.6783767049908637, - -0.14173239019669984, - 0.31067318724331106, - -1.7888499834156932, - 0.03642009920908874, - 1.6961157420094577, - -1.1089101982976917 - ] - }, - { - "_id": "65ce315df045be1f60050f44", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.06323640929784359, - 0.9978521827830034, - 0.016520434483373037, - 0.0, - 0.9977655821240671, - -0.06356378921021756, - 0.020105589547900215, - 0.0, - 0.021112914320698742, - 0.015212408525423958, - -0.9996613563981266, - 0.0, - 0.3844454340627629, - -0.034519055467682386, - 0.3511666416459713, - 1.0 - ], - "O_T_TCP": [ - 0.06323640929784359, - 0.9978521827830034, - 0.016520434483373037, - 0.0, - 0.9977655821240671, - -0.06356378921021756, - 0.020105589547900215, - 0.0, - 0.021112914320698742, - 0.015212408525423958, - -0.9996613563981266, - 0.0, - 0.38624003178002225, - -0.03322600074302135, - 0.2661954263521305, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "gearbase_hole1", - "q": [ - -0.4779728950460752, - -0.5647525487445973, - 0.34118822538716465, - -2.533109467228605, - 0.2194305768719429, - 2.0038177083303617, - -0.9983835912605101 - ] - }, - { - "_id": "65ce31d5f045be1f60050f45", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.016773866766090845, - 0.998555668652971, - 0.050852344055508855, - 0.0, - 0.999651562629023, - -0.017761257817640888, - 0.019027295007331226, - 0.0, - 0.019903398084193856, - 0.05051643648500831, - -0.9985248842114842, - 0.0, - 0.6166632788352968, - -0.06722110520041134, - 0.29656367644075515, - 1.0 - ], - "O_T_TCP": [ - 0.016773866766090845, - 0.998555668652971, - 0.050852344055508855, - 0.0, - 0.999651562629023, - -0.017761257817640888, - 0.019027295007331226, - 0.0, - 0.019903398084193856, - 0.05051643648500831, - -0.9985248842114842, - 0.0, - 0.6171608637874016, - -0.06595819428828614, - 0.27160055433546804, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "shaft3", - "q": [ - -0.6255808761370809, - 0.2721908670313347, - 0.5482196540661509, - -1.844289931665387, - -0.10189347790347204, - 2.091038903571352, - -0.8129646023995416 - ] - }, - { - "_id": "65ce3207f045be1f60050f46", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - -0.15915497945225549, - 0.9860926026332786, - -0.04766359668224283, - 0.0, - 0.9850428396108329, - 0.16183700693102337, - 0.058992662427146104, - 0.0, - 0.06588723038902841, - -0.03756243183872672, - -0.9971198205762556, - 0.0, - 0.5043739432190215, - 0.08072733253032562, - 0.2875533813662191, - 1.0 - ], - "O_T_TCP": [ - -0.15915497945225549, - 0.9860926026332786, - -0.04766359668224283, - 0.0, - 0.9850428396108329, - 0.16183700693102337, - 0.058992662427146104, - 0.0, - 0.06588723038902841, - -0.03756243183872672, - -0.9971198205762556, - 0.0, - 0.5060211239787472, - 0.07978827173435744, - 0.2626253858518127, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "gearbase_hole3", - "q": [ - -0.6320993901689848, - -0.13105960240977546, - 0.7911574711811249, - -2.301617453056469, - 0.05951137099465102, - 2.2677469761391547, - -0.8261827120045797 - ] - }, - { - "_id": "65ce326bf045be1f60050f47", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.9982800059810064, - 0.0028567036015946868, - 0.058391915573158704, - 0.0, - 0.002665300689743662, - -0.9999811917035863, - 0.003355489907203343, - 0.0, - 0.05840152737118158, - -0.003194147969025999, - -0.9982880641475511, - 0.0, - 0.38329798339116805, - -0.1792311431159989, - 0.2652977941807128, - 1.0 - ], - "O_T_TCP": [ - 0.9982800059810064, - 0.0028567036015946868, - 0.058391915573158704, - 0.0, - 0.002665300689743662, - -0.9999811917035863, - 0.003355489907203343, - 0.0, - 0.05840152737118158, - -0.003194147969025999, - -0.9982880641475511, - 0.0, - 0.38738609030715077, - -0.1794547334738307, - 0.19541762969038423, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "gear1", - "q": [ - -0.743737426383209, - -0.3330958715231419, - 0.30138807209333335, - -2.598988366910377, - 0.16539736091956653, - 2.3272367282354502, - 0.21644120803806513 - ] - }, - { - "_id": "65ce3349f045be1f60050f48", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - -0.43529958087664555, - -0.8981800970094251, - 0.06138025029245873, - 0.0, - -0.9001402451487228, - 0.43304221718444524, - -0.04693318762651431, - 0.0, - 0.01557451519504121, - -0.07568228756712969, - -0.9970103438906971, - 0.0, - 0.32523152397741284, - 0.13156707585417038, - 0.31425307836568933, - 1.0 - ], - "O_T_TCP": [ - -0.43529958087664555, - -0.8981800970094251, - 0.06138025029245873, - 0.0, - -0.9001402451487228, - 0.43304221718444524, - -0.04693318762651431, - 0.0, - 0.01557451519504121, - -0.07568228756712969, - -0.9970103438906971, - 0.0, - 0.327178338376793, - 0.12210678990827917, - 0.18962678537935218, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "gear2", - "q": [ - -0.44017739813892465, - -0.8408044241896847, - 0.7812836869963553, - -2.6489709372327193, - 0.5117495865954292, - 1.9353651503056144, - 2.7550759355656957 - ] - }, - { - "_id": "65ce3392f045be1f60050f49", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - -0.19207017045850883, - -0.9745158408444792, - -0.11579582230989756, - 0.0, - -0.9619084037954584, - 0.21032850120116167, - -0.1745705908546879, - 0.0, - 0.19448071224163638, - 0.07785667044396119, - -0.9778116339220793, - 0.0, - 0.6668309353879216, - 0.1002329166290898, - 0.3839163655785476, - 1.0 - ], - "O_T_TCP": [ - -0.19207017045850883, - -0.9745158408444792, - -0.11579582230989756, - 0.0, - -0.9619084037954584, - 0.21032850120116167, - -0.1745705908546879, - 0.0, - 0.19448071224163638, - 0.07785667044396119, - -0.9778116339220793, - 0.0, - 0.6911410244181262, - 0.10996500043458494, - 0.2616899113382877, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "no_tool", - "q": [ - -0.449013091942916, - 0.2972115466353616, - 0.6671699467458255, - -1.6625072732841806, - -0.16246066116120655, - 2.102316742226921, - 2.8067302440068085 - ] - }, - { - "_id": "65ce3399f045be1f60050f4a", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.37121811244401426, - -0.911720063290249, - 0.1759101648249346, - 0.0, - -0.9285316648543275, - -0.3639617925463895, - 0.07308561986107809, - 0.0, - -0.00260909728174289, - -0.1904725312477871, - -0.9816890584352234, - 0.0, - 0.5482232455292271, - -0.09299434620256447, - 0.4961290551977721, - 1.0 - ], - "O_T_TCP": [ - 0.37121811244401426, - -0.911720063290249, - 0.1759101648249346, - 0.0, - -0.9285316648543275, - -0.3639617925463895, - 0.07308561986107809, - 0.0, - -0.00260909728174289, - -0.1904725312477871, - -0.9816890584352234, - 0.0, - 0.5478971083690092, - -0.11680341260853785, - 0.37341792289336917, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "gearbase", - "q": [ - -0.5710980864097402, - -0.030782228559778448, - 0.4631181385266316, - -1.681660839952254, - -0.1779492470121923, - 1.6707865634759267, - 1.8844083975966868 - ] - }, - { - "_id": "65cfa2c47f0205483b054d51", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.999127486937346, - 0.0020339066230885923, - 0.04148343008142645, - 0.0, - -0.0021570557684146043, - -0.9949015779177928, - 0.1007320419147068, - 0.0, - 0.041477608185149145, - -0.10073557343640634, - -0.994048264554432, - 0.0, - 0.3646566267993078, - -0.02254569907028298, - 0.40952082232522724, - 1.0 - ], - "O_T_TCP": [ - 0.9991272968724854, - 0.002032794737528359, - 0.04148806203046724, - 0.0, - -0.002158543289712084, - -0.9949017859475108, - 0.10072995537836842, - 0.0, - 0.041482108991960025, - -0.10073354124199559, - -0.9940482826816947, - 0.0, - 0.36465772773036376, - -0.02254865462649719, - 0.40952225264460873, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "test1", - "q": [ - -0.6494016965366246, - -0.753164953780274, - 0.5019667189581353, - -2.4719201877805506, - 0.2581255530648761, - 1.8439903862476348, - 0.4656479083630774 - ] - }, - { - "_id": "65cfa7905d75dfc96909caf1", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - -0.9935878908904953, - 0.08960520143219396, - -0.06880957675557518, - 0.0, - 0.09120083791165756, - 0.9956138318810523, - -0.02040225067357439, - 0.0, - 0.06668090241645759, - -0.026547431392950203, - -0.9974211202593233, - 0.0, - 0.5080013634448317, - 0.07744840851681808, - 0.310686758863888, - 1.0 - ], - "O_T_TCP": [ - -0.9935878908904953, - 0.08960520143219396, - -0.06880957675557518, - 0.0, - 0.09120083791165756, - 0.9956138318810523, - -0.02040225067357439, - 0.0, - 0.06668090241645759, - -0.026547431392950203, - -0.9974211202593233, - 0.0, - 0.5080013634448317, - 0.07744840851681808, - 0.310686758863888, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "test_object", - "q": [ - -0.6783759433123, - -0.16586847048823597, - 0.8175371052577723, - -2.261259556151273, - 0.1062502110335562, - 2.2074089546736957, - -2.190133549686696 - ] - }, - { - "_id": "65cfa8e660f0281a320a1ef1", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.9999648408880725, - 0.0002669773375134615, - 0.00714091114644764, - 0.0, - -4.124753500410915e-05, - -0.9990599232033038, - 0.04312789177385854, - 0.0, - 0.00714584989104384, - -0.0431275003184912, - -0.9990440208247148, - 0.0, - 0.5039719752053793, - 0.08282098192276433, - 0.4255527712058619, - 1.0 - ], - "O_T_TCP": [ - 0.9999648408880725, - 0.0002669773375134615, - 0.00714091114644764, - 0.0, - -4.124753500410915e-05, - -0.9990599232033038, - 0.04312789177385854, - 0.0, - 0.00714584989104384, - -0.0431275003184912, - -0.9990440208247148, - 0.0, - 0.5039719752053793, - 0.08282098192276433, - 0.4255527712058619, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "aaa", - "q": [ - -0.7009867876172065, - -0.24024575297265152, - 0.7995970885804996, - -1.9988235254230773, - 0.13235241008271123, - 1.8310430494149526, - 0.8356096653159838 - ] - }, - { - "_id": "65d8ab25365be5466b0fdc71", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.9021773910215635, - -0.038249869200695535, - 0.42964363085802043, - 0.0, - -0.03475251865213935, - -0.9992584430166064, - -0.015986663459536494, - 0.0, - 0.42994479117900936, - -0.0005084017524109786, - -0.9028550371270552, - 0.0, - 0.5071971613100603, - 0.04661989698448783, - 0.6363466027888695, - 1.0 - ], - "O_T_TCP": [ - 0.9021773910215635, - -0.038249869200695535, - 0.42964363085802043, - 0.0, - -0.03475251865213935, - -0.9992584430166064, - -0.015986663459536494, - 0.0, - 0.42994479117900936, - -0.0005084017524109786, - -0.9028550371270552, - 0.0, - 0.5071971613100603, - 0.04661989698448783, - 0.6363466027888695, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "yawgripper", - "q": [ - 0.11421029169640254, - -0.34931760987484234, - -0.004396662544784647, - -1.792835893163279, - -0.05164978933003214, - 1.8851889863806732, - 0.9384602965859719 - ] - }, - { - "_id": "65d8bc19d2850057440d7331", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.026665700373889236, - 0.9990310854222944, - -0.03473582710589628, - 0.0, - 0.9993871064511708, - -0.025869536002868367, - 0.023171652236316353, - 0.0, - 0.022251029556039845, - -0.03533310634903862, - -0.9991278513180502, - 0.0, - 0.34529017685396074, - 0.28611390088978156, - 0.32766119170850694, - 1.0 - ], - "O_T_TCP": [ - 0.026665700373889236, - 0.9990310854222944, - -0.03473582710589628, - 0.0, - 0.9993871064511708, - -0.025869536002868367, - 0.023171652236316353, - 0.0, - 0.022251029556039845, - -0.03533310634903862, - -0.9991278513180502, - 0.0, - 0.34529017685396074, - 0.28611390088978156, - 0.32766119170850694, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "outwardgripper", - "q": [ - 0.870870500266552, - -0.2957360618239955, - -0.1551851647736568, - -2.386680684173316, - -0.09917618758749594, - 2.0854945325421754, - 0.012588030038608445 - ] - }, - { - "_id": "65d8bc40d2850057440d7332", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.03396390773103101, - 0.9993034014499432, - -0.014829420872713595, - 0.0, - 0.998694979847861, - -0.033373237107500404, - 0.03840977965122197, - 0.0, - 0.037888847153174234, - -0.0161149246539099, - -0.999152012691163, - 0.0, - 0.5057945972945914, - 0.28251383833072985, - 0.32803008980848014, - 1.0 - ], - "O_T_TCP": [ - 0.03396390773103101, - 0.9993034014499432, - -0.014829420872713595, - 0.0, - 0.998694979847861, - -0.033373237107500404, - 0.03840977965122197, - 0.0, - 0.037888847153174234, - -0.0161149246539099, - -0.999152012691163, - 0.0, - 0.5057945972945914, - 0.28251383833072985, - 0.32803008980848014, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "inwardgripper", - "q": [ - 0.7132648979574978, - 0.0822906340856455, - -0.19983831917677083, - -1.9729896309966, - -0.018719173115160727, - 2.078648384739713, - -0.2285200706422465 - ] - }, - { - "_id": "65d8bc9cd2850057440d7333", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.9159565622373924, - 0.11163201311977883, - 0.38541226841175086, - 0.0, - 0.14128903439883572, - -0.9887250951913703, - -0.04940487626407901, - 0.0, - 0.3755588466496142, - 0.0997091675804246, - -0.9214193587089595, - 0.0, - 0.4479657579420528, - 0.04195357437032795, - 0.6367311153952495, - 1.0 - ], - "O_T_TCP": [ - 0.9159565622373924, - 0.11163201311977883, - 0.38541226841175086, - 0.0, - 0.14128903439883572, - -0.9887250951913703, - -0.04940487626407901, - 0.0, - 0.3755588466496142, - 0.0997091675804246, - -0.9214193587089595, - 0.0, - 0.4479657579420528, - 0.04195357437032795, - 0.6367311153952495, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "defaultgripper", - "q": [ - 0.021865537859103813, - -0.491120350155914, - 0.026315845489235032, - -1.8907418638905291, - 0.09657258354955249, - 1.7900419552325462, - 0.6985316049030074 - ] - }, - { - "_id": "65d8bcbbd2850057440d7334", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.024177210511319505, - 0.9975796811973359, - 0.06504605334162625, - 0.0, - 0.9996394690321734, - -0.024828988657792546, - 0.009230394168961284, - 0.0, - 0.010823289774918731, - 0.06480068465611978, - -0.9978395300180016, - 0.0, - 0.3348185254531909, - -0.33224274111487373, - 0.3257033634399661, - 1.0 - ], - "O_T_TCP": [ - 0.024177210511319505, - 0.9975796811973359, - 0.06504605334162625, - 0.0, - 0.9996394690321734, - -0.024828988657792546, - 0.009230394168961284, - 0.0, - 0.010823289774918731, - 0.06480068465611978, - -0.9978395300180016, - 0.0, - 0.3350891076975639, - -0.33062272399847076, - 0.30075737518951606, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "parallelgripper", - "q": [ - -0.5970329341689745, - -0.20463424487443924, - -0.19403221863983816, - -2.293547858754675, - 0.01611855673189336, - 2.05432398039057, - -1.5549974032722624 - ] - }, - { - "_id": "65d8bd19d2850057440d7335", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.01657188384160618, - 0.9990552433374398, - 0.03993419996841205, - 0.0, - 0.9998158195093524, - -0.016213384129055507, - -0.009284402788233788, - 0.0, - -0.00862832888493567, - 0.04008147660504227, - -0.9991591600810217, - 0.0, - 0.4936055805433891, - -0.33471007413492543, - 0.3264931543876324, - 1.0 - ], - "O_T_TCP": [ - 0.01657188384160618, - 0.9990552433374398, - 0.03993419996841205, - 0.0, - 0.9998158195093524, - -0.016213384129055507, - -0.009284402788233788, - 0.0, - -0.00862832888493567, - 0.04008147660504227, - -0.9991591600810217, - 0.0, - 0.4936055805433891, - -0.33471007413492543, - 0.3264931543876324, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "clampgripper", - "q": [ - -0.23730045444296113, - 0.17615896566081463, - -0.3904649787620021, - -1.854275331395351, - 0.10299784066871676, - 1.9849424978146744, - -1.4353288235127737 - ] - }, - { - "_id": "65d8c5483d14aeb8d50802f1", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.9973932098819581, - 0.07036323101665336, - 0.015380100892704794, - 0.0, - 0.07006619897038535, - -0.9973504557130216, - 0.01906680760061509, - 0.0, - 0.016681273988419575, - -0.017939824622862092, - -0.9996999038664673, - 0.0, - 0.44420899805383923, - -0.16125533487954755, - 0.3125141957624371, - 1.0 - ], - "O_T_TCP": [ - 0.9973932098819581, - 0.07036323101665336, - 0.015380100892704794, - 0.0, - 0.07006619897038535, - -0.9973504557130216, - 0.01906680760061509, - 0.0, - 0.016681273988419575, - -0.017939824622862092, - -0.9996999038664673, - 0.0, - 0.4462941573023917, - -0.16349781295740531, - 0.1875517077791287, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "outerring", - "q": [ - -0.425466590412938, - -0.2179346746508836, - 0.07799056342200245, - -2.3598260232021935, - 0.006869914273007049, - 2.1643380800088248, - 0.3622042840484207 - ] - } -] \ No newline at end of file diff --git a/experiments/gearbox/mios_memory_backups/gearbox_exp1.json b/experiments/gearbox/mios_memory_backups/gearbox_exp1.json deleted file mode 100644 index 5b04efd1..00000000 --- a/experiments/gearbox/mios_memory_backups/gearbox_exp1.json +++ /dev/null @@ -1,3602 +0,0 @@ -[ - { - "_id": "643d40f9dd8c607f760e5af8", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "NullObject", - "q": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ] - }, - { - "_id": "643d40f9dd8c607f760e5af9", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "NoneObject", - "q": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ] - }, - { - "_id": "643d40f9dd8c607f760e5afa", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.995807344231971, - 0.09038182927767592, - -0.013401679420805597, - 0.0, - 0.09056014012878236, - -0.9957917808908888, - 0.01335428886970798, - 0.0, - -0.012138530864960013, - -0.014512236305278915, - -0.9998210095141331, - 0.0, - 0.4186402258857764, - -0.0213018116400438, - 0.38213174742071804, - 1.0 - ], - "O_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "EndEffector", - "q": [ - 0.10379081284999846, - -0.42633626818015974, - -0.1307515779083688, - -2.3616170134324452, - -0.07320591049061881, - 1.9256821783758933, - 0.7056310605141853 - ] - }, - { - "_id": "6491ad548829c1097f0151a9", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "ring", - "q": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ] - }, - { - "_id": "64f8a17c8d1b98ac5f0c6b91", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "geometry": null, - "grasp_force": 1.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "TestObject1", - "q": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ] - }, - { - "_id": "6526b75ece654b001f026d61", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "gripper_connected", - "q": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ] - }, - { - "_id": "6526da7d86c098f7fe057f61", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "tool_load", - "q": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ] - }, - { - "_id": "6530168b631a09bd8e08d211", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "tool_grasp", - "q": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ] - }, - { - "_id": "65a8008494d3e83f7e035b11", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "kios_guidance", - "q": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ] - }, - { - "_id": "65b2664c677d37a9900fd7b1", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.020523632927385155, - 0.999779614758663, - 0.0004993052070315127, - 0.0, - 0.9997626802269471, - -0.020520362759945312, - -0.00585190899106819, - 0.0, - -0.0058404858401676345, - 0.000619301067554896, - -0.99998275244693, - 0.0, - 0.49407319098460234, - -0.3343870569274952, - 0.3249152265674699, - 1.0 - ], - "O_T_TCP": [ - 0.020523632927385155, - 0.999779614758663, - 0.0004993052070315127, - 0.0, - 0.9997626802269471, - -0.020520362759945312, - -0.00585190899106819, - 0.0, - -0.0058404858401676345, - 0.000619301067554896, - -0.99998275244693, - 0.0, - 0.49407319098460234, - -0.3343870569274952, - 0.3249152265674699, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "parallel_box1", - "q": [ - -0.4889897713452055, - 0.1520400121422783, - -0.11095685046074681, - -1.8813407051080338, - 0.015587054766497403, - 2.027237653652827, - -1.3705014213786446 - ] - }, - { - "_id": "65bb9aa7666d3392110788a1", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - -0.0055492434676263965, - 0.9999687095518451, - 0.003540157533654855, - 0.0, - 0.9994642785591136, - 0.005433245086445076, - 0.0319747186594265, - 0.0, - 0.03195509884729998, - 0.003715768034059008, - -0.9994823984070837, - 0.0, - 0.5073771436733974, - 0.2821340073022507, - 0.3273610300796691, - 1.0 - ], - "O_T_TCP": [ - -0.0055492434676263965, - 0.9999687095518451, - 0.003540157533654855, - 0.0, - 0.9994642785591136, - 0.005433245086445076, - 0.0319747186594265, - 0.0, - 0.03195509884729998, - 0.003715768034059008, - -0.9994823984070837, - 0.0, - 0.5073771436733974, - 0.2821340073022507, - 0.3273610300796691, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "inward_claw", - "q": [ - -0.3558367933193034, - 0.12986394129696988, - 0.8984244877664667, - -1.9720981870183556, - -0.1313234915672495, - 2.079114055554075, - -0.1914156752901866 - ] - }, - { - "_id": "65bbb462b8aab7a7d90d58c1", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.9949942224611444, - -0.05041086343049174, - 0.08617417837347448, - 0.0, - -0.04503393357509275, - -0.9969720659778388, - -0.06324074151875426, - 0.0, - 0.08910298453945457, - 0.05904454700105643, - -0.9942707878719971, - 0.0, - 0.6906053200978064, - -0.01614832324583123, - 0.2887581517923185, - 1.0 - ], - "O_T_TCP": [ - 0.9949942224611444, - -0.05041086343049174, - 0.08617417837347448, - 0.0, - -0.04503393357509275, - -0.9969720659778388, - -0.06324074151875426, - 0.0, - 0.08910298453945457, - 0.05904454700105643, - -0.9942707878719971, - 0.0, - 0.6990701036290545, - -0.01053909128073087, - 0.1943024269444788, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "test_object1", - "q": [ - -0.5258051569263356, - 0.4877585826129244, - 0.5891767306537179, - -1.6077540129764138, - -0.23719313260094485, - 2.1022328375975285, - 0.9414903357566682 - ] - }, - { - "_id": "65bbba7c5a0f2e8e04072891", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.005516821389947922, - -0.9949443596914403, - -0.10018000150304039, - 0.0, - -0.999821715401967, - -0.003733290448562814, - -0.017981847911249182, - 0.0, - 0.017517274375192557, - 0.10026327397364804, - -0.9948067254449727, - 0.0, - 0.643320166007396, - 0.11088726408849338, - 0.3035784369329164, - 1.0 - ], - "O_T_TCP": [ - 0.005516821389947922, - -0.9949443596914403, - -0.10018000150304039, - 0.0, - -0.999821715401967, - -0.003733290448562814, - -0.017981847911249182, - 0.0, - 0.017517274375192557, - 0.10026327397364804, - -0.9948067254449727, - 0.0, - 0.6455098253042951, - 0.12342017333519939, - 0.17922759625229484, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "gear3", - "q": [ - -0.4348329623676183, - 0.39139715203497827, - 0.6694900968644402, - -1.711178953283276, - -0.16270597490334937, - 2.05581780107816, - 2.617633598559432 - ] - }, - { - "_id": "65bbbab25a0f2e8e04072892", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.9980187907769038, - -0.05571650543342081, - -0.02889482963615323, - 0.0, - -0.05609811498518191, - -0.9983365128919355, - -0.012568031790528969, - 0.0, - -0.028147058566118602, - 0.014164350073066843, - -0.9995034338515715, - 0.0, - 0.507308310840537, - -0.04026469732735698, - 0.3761685279429274, - 1.0 - ], - "O_T_TCP": [ - 0.9980187907769038, - -0.05571650543342081, - -0.02889482963615323, - 0.0, - -0.05609811498518191, - -0.9983365128919355, - -0.012568031790528969, - 0.0, - -0.028147058566118602, - 0.014164350073066843, - -0.9995034338515715, - 0.0, - 0.5046343402767557, - -0.038919084070415624, - 0.2812157017270281, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "shaft2", - "q": [ - -0.04717889234179201, - -0.13185073856974885, - -0.034486469687599886, - -2.0883627749325773, - 0.0077889278013025046, - 1.9273695102532702, - 0.7572235297293566 - ] - }, - { - "_id": "65bfcf0f5a0e8797bb01e281", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.9991077043349246, - -0.009049464669151653, - -0.041020107618180826, - 0.0, - -0.009843038297554691, - -0.9997578641005138, - -0.019185272149743374, - 0.0, - -0.0408373449804758, - 0.019572292532108137, - -0.9989740920664474, - 0.0, - 0.4292203229453998, - -0.033877595855194384, - 0.4236606084507305, - 1.0 - ], - "O_T_TCP": [ - 0.9991077043349246, - -0.009049464669151653, - -0.041020107618180826, - 0.0, - -0.009843038297554691, - -0.9997578641005138, - -0.019185272149743374, - 0.0, - -0.0408373449804758, - 0.019572292532108137, - -0.9989740920664474, - 0.0, - 0.4292203229453998, - -0.033877595855194384, - 0.4236606084507305, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "test", - "q": [ - -0.028742359890226728, - -0.3809770326088004, - -0.048948421164719876, - -2.2124784198559695, - -0.0017267249742586607, - 1.7897263017493257, - 0.7214576442114876 - ] - }, - { - "_id": "65ce303af045be1f60050f41", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - -0.00034130509165598697, - 0.9999088204921296, - 0.012766405683707687, - 0.0, - 0.9999006778257512, - 0.00017031581705472163, - 0.013392250644843597, - 0.0, - 0.01338911300712114, - 0.012769954401065334, - -0.9998288153066389, - 0.0, - 0.34586891787172797, - 0.28478778305517327, - 0.325633698814514, - 1.0 - ], - "O_T_TCP": [ - -0.00034130509165598697, - 0.9999088204921296, - 0.012766405683707687, - 0.0, - 0.9999006778257512, - 0.00017031581705472163, - 0.013392250644843597, - 0.0, - 0.01338911300712114, - 0.012769954401065334, - -0.9998288153066389, - 0.0, - 0.34586891787172797, - 0.28478778305517327, - 0.325633698814514, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "outward_claw", - "q": [ - -0.26322421671319607, - -0.48738894555267787, - 0.8759835133827272, - -2.392889458566437, - 0.4210987845394346, - 2.0469462890460046, - -0.42977081686632607 - ] - }, - { - "_id": "65ce3098f045be1f60050f42", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.016519502886665666, - 0.9997483636116822, - -0.014527986142256165, - 0.0, - 0.999314451509194, - -0.01603160985419312, - 0.033081133468638016, - 0.0, - 0.0328405343279425, - -0.015064800427202131, - -0.999347062382806, - 0.0, - 0.3334408867064976, - -0.33165373934959796, - 0.32422469388972297, - 1.0 - ], - "O_T_TCP": [ - 0.016519502886665666, - 0.9997483636116822, - -0.014527986142256165, - 0.0, - 0.999314451509194, - -0.01603160985419312, - 0.033081133468638016, - 0.0, - 0.0328405343279425, - -0.015064800427202131, - -0.999347062382806, - 0.0, - 0.3334408867064976, - -0.33165373934959796, - 0.32422469388972297, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "parallel_box2", - "q": [ - -0.49195081603365076, - -0.2536298767587599, - -0.28172348515657036, - -2.357436443128084, - -0.06903641326254598, - 2.1465019415994413, - -1.4987806172708702 - ] - }, - { - "_id": "65ce3113f045be1f60050f43", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.05033441300943016, - 0.998522325819256, - -0.020008963209219598, - 0.0, - 0.9980401159147424, - -0.049549287905040595, - 0.037967644043304945, - 0.0, - 0.036920821197481454, - -0.021881248318769652, - -0.9990786074849747, - 0.0, - 0.4951086360500897, - -0.1768782158810579, - 0.5007737812082008, - 1.0 - ], - "O_T_TCP": [ - 0.05033441300943016, - 0.998522325819256, - -0.020008963209219598, - 0.0, - 0.9980401159147424, - -0.049549287905040595, - 0.037967644043304945, - 0.0, - 0.036920821197481454, - -0.021881248318769652, - -0.9990786074849747, - 0.0, - 0.49824690585187564, - -0.1787381219881533, - 0.415852099571978, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "shaft1", - "q": [ - -0.6783767049908637, - -0.14173239019669984, - 0.31067318724331106, - -1.7888499834156932, - 0.03642009920908874, - 1.6961157420094577, - -1.1089101982976917 - ] - }, - { - "_id": "65ce315df045be1f60050f44", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.06323640929784359, - 0.9978521827830034, - 0.016520434483373037, - 0.0, - 0.9977655821240671, - -0.06356378921021756, - 0.020105589547900215, - 0.0, - 0.021112914320698742, - 0.015212408525423958, - -0.9996613563981266, - 0.0, - 0.3844454340627629, - -0.034519055467682386, - 0.3511666416459713, - 1.0 - ], - "O_T_TCP": [ - 0.06323640929784359, - 0.9978521827830034, - 0.016520434483373037, - 0.0, - 0.9977655821240671, - -0.06356378921021756, - 0.020105589547900215, - 0.0, - 0.021112914320698742, - 0.015212408525423958, - -0.9996613563981266, - 0.0, - 0.38624003178002225, - -0.03322600074302135, - 0.2661954263521305, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "gearbase_hole1", - "q": [ - -0.4779728950460752, - -0.5647525487445973, - 0.34118822538716465, - -2.533109467228605, - 0.2194305768719429, - 2.0038177083303617, - -0.9983835912605101 - ] - }, - { - "_id": "65ce31d5f045be1f60050f45", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.016773866766090845, - 0.998555668652971, - 0.050852344055508855, - 0.0, - 0.999651562629023, - -0.017761257817640888, - 0.019027295007331226, - 0.0, - 0.019903398084193856, - 0.05051643648500831, - -0.9985248842114842, - 0.0, - 0.6166632788352968, - -0.06722110520041134, - 0.29656367644075515, - 1.0 - ], - "O_T_TCP": [ - 0.016773866766090845, - 0.998555668652971, - 0.050852344055508855, - 0.0, - 0.999651562629023, - -0.017761257817640888, - 0.019027295007331226, - 0.0, - 0.019903398084193856, - 0.05051643648500831, - -0.9985248842114842, - 0.0, - 0.6171608637874016, - -0.06595819428828614, - 0.27160055433546804, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "shaft3", - "q": [ - -0.6255808761370809, - 0.2721908670313347, - 0.5482196540661509, - -1.844289931665387, - -0.10189347790347204, - 2.091038903571352, - -0.8129646023995416 - ] - }, - { - "_id": "65ce3207f045be1f60050f46", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - -0.15915497945225549, - 0.9860926026332786, - -0.04766359668224283, - 0.0, - 0.9850428396108329, - 0.16183700693102337, - 0.058992662427146104, - 0.0, - 0.06588723038902841, - -0.03756243183872672, - -0.9971198205762556, - 0.0, - 0.5043739432190215, - 0.08072733253032562, - 0.2875533813662191, - 1.0 - ], - "O_T_TCP": [ - -0.15915497945225549, - 0.9860926026332786, - -0.04766359668224283, - 0.0, - 0.9850428396108329, - 0.16183700693102337, - 0.058992662427146104, - 0.0, - 0.06588723038902841, - -0.03756243183872672, - -0.9971198205762556, - 0.0, - 0.5060211239787472, - 0.07978827173435744, - 0.2626253858518127, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "gearbase_hole3", - "q": [ - -0.6320993901689848, - -0.13105960240977546, - 0.7911574711811249, - -2.301617453056469, - 0.05951137099465102, - 2.2677469761391547, - -0.8261827120045797 - ] - }, - { - "_id": "65ce326bf045be1f60050f47", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.9982800059810064, - 0.0028567036015946868, - 0.058391915573158704, - 0.0, - 0.002665300689743662, - -0.9999811917035863, - 0.003355489907203343, - 0.0, - 0.05840152737118158, - -0.003194147969025999, - -0.9982880641475511, - 0.0, - 0.38329798339116805, - -0.1792311431159989, - 0.2652977941807128, - 1.0 - ], - "O_T_TCP": [ - 0.9982800059810064, - 0.0028567036015946868, - 0.058391915573158704, - 0.0, - 0.002665300689743662, - -0.9999811917035863, - 0.003355489907203343, - 0.0, - 0.05840152737118158, - -0.003194147969025999, - -0.9982880641475511, - 0.0, - 0.38738609030715077, - -0.1794547334738307, - 0.19541762969038423, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "gear1", - "q": [ - -0.743737426383209, - -0.3330958715231419, - 0.30138807209333335, - -2.598988366910377, - 0.16539736091956653, - 2.3272367282354502, - 0.21644120803806513 - ] - }, - { - "_id": "65ce3349f045be1f60050f48", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - -0.43529958087664555, - -0.8981800970094251, - 0.06138025029245873, - 0.0, - -0.9001402451487228, - 0.43304221718444524, - -0.04693318762651431, - 0.0, - 0.01557451519504121, - -0.07568228756712969, - -0.9970103438906971, - 0.0, - 0.32523152397741284, - 0.13156707585417038, - 0.31425307836568933, - 1.0 - ], - "O_T_TCP": [ - -0.43529958087664555, - -0.8981800970094251, - 0.06138025029245873, - 0.0, - -0.9001402451487228, - 0.43304221718444524, - -0.04693318762651431, - 0.0, - 0.01557451519504121, - -0.07568228756712969, - -0.9970103438906971, - 0.0, - 0.327178338376793, - 0.12210678990827917, - 0.18962678537935218, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "gear2", - "q": [ - -0.44017739813892465, - -0.8408044241896847, - 0.7812836869963553, - -2.6489709372327193, - 0.5117495865954292, - 1.9353651503056144, - 2.7550759355656957 - ] - }, - { - "_id": "65ce3392f045be1f60050f49", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - -0.19207017045850883, - -0.9745158408444792, - -0.11579582230989756, - 0.0, - -0.9619084037954584, - 0.21032850120116167, - -0.1745705908546879, - 0.0, - 0.19448071224163638, - 0.07785667044396119, - -0.9778116339220793, - 0.0, - 0.6668309353879216, - 0.1002329166290898, - 0.3839163655785476, - 1.0 - ], - "O_T_TCP": [ - -0.19207017045850883, - -0.9745158408444792, - -0.11579582230989756, - 0.0, - -0.9619084037954584, - 0.21032850120116167, - -0.1745705908546879, - 0.0, - 0.19448071224163638, - 0.07785667044396119, - -0.9778116339220793, - 0.0, - 0.6911410244181262, - 0.10996500043458494, - 0.2616899113382877, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "no_tool", - "q": [ - -0.449013091942916, - 0.2972115466353616, - 0.6671699467458255, - -1.6625072732841806, - -0.16246066116120655, - 2.102316742226921, - 2.8067302440068085 - ] - }, - { - "_id": "65ce3399f045be1f60050f4a", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.37121811244401426, - -0.911720063290249, - 0.1759101648249346, - 0.0, - -0.9285316648543275, - -0.3639617925463895, - 0.07308561986107809, - 0.0, - -0.00260909728174289, - -0.1904725312477871, - -0.9816890584352234, - 0.0, - 0.5482232455292271, - -0.09299434620256447, - 0.4961290551977721, - 1.0 - ], - "O_T_TCP": [ - 0.37121811244401426, - -0.911720063290249, - 0.1759101648249346, - 0.0, - -0.9285316648543275, - -0.3639617925463895, - 0.07308561986107809, - 0.0, - -0.00260909728174289, - -0.1904725312477871, - -0.9816890584352234, - 0.0, - 0.5478971083690092, - -0.11680341260853785, - 0.37341792289336917, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "gearbase", - "q": [ - -0.5710980864097402, - -0.030782228559778448, - 0.4631181385266316, - -1.681660839952254, - -0.1779492470121923, - 1.6707865634759267, - 1.8844083975966868 - ] - }, - { - "_id": "65cfa2c47f0205483b054d51", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.999127486937346, - 0.0020339066230885923, - 0.04148343008142645, - 0.0, - -0.0021570557684146043, - -0.9949015779177928, - 0.1007320419147068, - 0.0, - 0.041477608185149145, - -0.10073557343640634, - -0.994048264554432, - 0.0, - 0.3646566267993078, - -0.02254569907028298, - 0.40952082232522724, - 1.0 - ], - "O_T_TCP": [ - 0.9991272968724854, - 0.002032794737528359, - 0.04148806203046724, - 0.0, - -0.002158543289712084, - -0.9949017859475108, - 0.10072995537836842, - 0.0, - 0.041482108991960025, - -0.10073354124199559, - -0.9940482826816947, - 0.0, - 0.36465772773036376, - -0.02254865462649719, - 0.40952225264460873, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "test1", - "q": [ - -0.6494016965366246, - -0.753164953780274, - 0.5019667189581353, - -2.4719201877805506, - 0.2581255530648761, - 1.8439903862476348, - 0.4656479083630774 - ] - }, - { - "_id": "65cfa7905d75dfc96909caf1", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - -0.9935878908904953, - 0.08960520143219396, - -0.06880957675557518, - 0.0, - 0.09120083791165756, - 0.9956138318810523, - -0.02040225067357439, - 0.0, - 0.06668090241645759, - -0.026547431392950203, - -0.9974211202593233, - 0.0, - 0.5080013634448317, - 0.07744840851681808, - 0.310686758863888, - 1.0 - ], - "O_T_TCP": [ - -0.9935878908904953, - 0.08960520143219396, - -0.06880957675557518, - 0.0, - 0.09120083791165756, - 0.9956138318810523, - -0.02040225067357439, - 0.0, - 0.06668090241645759, - -0.026547431392950203, - -0.9974211202593233, - 0.0, - 0.5080013634448317, - 0.07744840851681808, - 0.310686758863888, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "test_object", - "q": [ - -0.6783759433123, - -0.16586847048823597, - 0.8175371052577723, - -2.261259556151273, - 0.1062502110335562, - 2.2074089546736957, - -2.190133549686696 - ] - }, - { - "_id": "65cfa8e660f0281a320a1ef1", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.9999648408880725, - 0.0002669773375134615, - 0.00714091114644764, - 0.0, - -4.124753500410915e-05, - -0.9990599232033038, - 0.04312789177385854, - 0.0, - 0.00714584989104384, - -0.0431275003184912, - -0.9990440208247148, - 0.0, - 0.5039719752053793, - 0.08282098192276433, - 0.4255527712058619, - 1.0 - ], - "O_T_TCP": [ - 0.9999648408880725, - 0.0002669773375134615, - 0.00714091114644764, - 0.0, - -4.124753500410915e-05, - -0.9990599232033038, - 0.04312789177385854, - 0.0, - 0.00714584989104384, - -0.0431275003184912, - -0.9990440208247148, - 0.0, - 0.5039719752053793, - 0.08282098192276433, - 0.4255527712058619, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "aaa", - "q": [ - -0.7009867876172065, - -0.24024575297265152, - 0.7995970885804996, - -1.9988235254230773, - 0.13235241008271123, - 1.8310430494149526, - 0.8356096653159838 - ] - }, - { - "_id": "65d8ab25365be5466b0fdc71", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.9021773910215635, - -0.038249869200695535, - 0.42964363085802043, - 0.0, - -0.03475251865213935, - -0.9992584430166064, - -0.015986663459536494, - 0.0, - 0.42994479117900936, - -0.0005084017524109786, - -0.9028550371270552, - 0.0, - 0.5071971613100603, - 0.04661989698448783, - 0.6363466027888695, - 1.0 - ], - "O_T_TCP": [ - 0.9021773910215635, - -0.038249869200695535, - 0.42964363085802043, - 0.0, - -0.03475251865213935, - -0.9992584430166064, - -0.015986663459536494, - 0.0, - 0.42994479117900936, - -0.0005084017524109786, - -0.9028550371270552, - 0.0, - 0.5071971613100603, - 0.04661989698448783, - 0.6363466027888695, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "yawgripper", - "q": [ - 0.11421029169640254, - -0.34931760987484234, - -0.004396662544784647, - -1.792835893163279, - -0.05164978933003214, - 1.8851889863806732, - 0.9384602965859719 - ] - }, - { - "_id": "65d8bc19d2850057440d7331", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.026665700373889236, - 0.9990310854222944, - -0.03473582710589628, - 0.0, - 0.9993871064511708, - -0.025869536002868367, - 0.023171652236316353, - 0.0, - 0.022251029556039845, - -0.03533310634903862, - -0.9991278513180502, - 0.0, - 0.34529017685396074, - 0.28611390088978156, - 0.32766119170850694, - 1.0 - ], - "O_T_TCP": [ - 0.026665700373889236, - 0.9990310854222944, - -0.03473582710589628, - 0.0, - 0.9993871064511708, - -0.025869536002868367, - 0.023171652236316353, - 0.0, - 0.022251029556039845, - -0.03533310634903862, - -0.9991278513180502, - 0.0, - 0.34529017685396074, - 0.28611390088978156, - 0.32766119170850694, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "outwardgripper", - "q": [ - 0.870870500266552, - -0.2957360618239955, - -0.1551851647736568, - -2.386680684173316, - -0.09917618758749594, - 2.0854945325421754, - 0.012588030038608445 - ] - }, - { - "_id": "65d8bc40d2850057440d7332", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.03396390773103101, - 0.9993034014499432, - -0.014829420872713595, - 0.0, - 0.998694979847861, - -0.033373237107500404, - 0.03840977965122197, - 0.0, - 0.037888847153174234, - -0.0161149246539099, - -0.999152012691163, - 0.0, - 0.5057945972945914, - 0.28251383833072985, - 0.32803008980848014, - 1.0 - ], - "O_T_TCP": [ - 0.03396390773103101, - 0.9993034014499432, - -0.014829420872713595, - 0.0, - 0.998694979847861, - -0.033373237107500404, - 0.03840977965122197, - 0.0, - 0.037888847153174234, - -0.0161149246539099, - -0.999152012691163, - 0.0, - 0.5057945972945914, - 0.28251383833072985, - 0.32803008980848014, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "inwardgripper", - "q": [ - 0.7132648979574978, - 0.0822906340856455, - -0.19983831917677083, - -1.9729896309966, - -0.018719173115160727, - 2.078648384739713, - -0.2285200706422465 - ] - }, - { - "_id": "65d8bc9cd2850057440d7333", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.9159565622373924, - 0.11163201311977883, - 0.38541226841175086, - 0.0, - 0.14128903439883572, - -0.9887250951913703, - -0.04940487626407901, - 0.0, - 0.3755588466496142, - 0.0997091675804246, - -0.9214193587089595, - 0.0, - 0.4479657579420528, - 0.04195357437032795, - 0.6367311153952495, - 1.0 - ], - "O_T_TCP": [ - 0.9159565622373924, - 0.11163201311977883, - 0.38541226841175086, - 0.0, - 0.14128903439883572, - -0.9887250951913703, - -0.04940487626407901, - 0.0, - 0.3755588466496142, - 0.0997091675804246, - -0.9214193587089595, - 0.0, - 0.4479657579420528, - 0.04195357437032795, - 0.6367311153952495, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "defaultgripper", - "q": [ - 0.021865537859103813, - -0.491120350155914, - 0.026315845489235032, - -1.8907418638905291, - 0.09657258354955249, - 1.7900419552325462, - 0.6985316049030074 - ] - }, - { - "_id": "65d8bcbbd2850057440d7334", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.024177210511319505, - 0.9975796811973359, - 0.06504605334162625, - 0.0, - 0.9996394690321734, - -0.024828988657792546, - 0.009230394168961284, - 0.0, - 0.010823289774918731, - 0.06480068465611978, - -0.9978395300180016, - 0.0, - 0.3348185254531909, - -0.33224274111487373, - 0.3257033634399661, - 1.0 - ], - "O_T_TCP": [ - 0.024177210511319505, - 0.9975796811973359, - 0.06504605334162625, - 0.0, - 0.9996394690321734, - -0.024828988657792546, - 0.009230394168961284, - 0.0, - 0.010823289774918731, - 0.06480068465611978, - -0.9978395300180016, - 0.0, - 0.3350891076975639, - -0.33062272399847076, - 0.30075737518951606, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "parallelgripper", - "q": [ - -0.5970329341689745, - -0.20463424487443924, - -0.19403221863983816, - -2.293547858754675, - 0.01611855673189336, - 2.05432398039057, - -1.5549974032722624 - ] - }, - { - "_id": "65d8bd19d2850057440d7335", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.01657188384160618, - 0.9990552433374398, - 0.03993419996841205, - 0.0, - 0.9998158195093524, - -0.016213384129055507, - -0.009284402788233788, - 0.0, - -0.00862832888493567, - 0.04008147660504227, - -0.9991591600810217, - 0.0, - 0.4936055805433891, - -0.33471007413492543, - 0.3264931543876324, - 1.0 - ], - "O_T_TCP": [ - 0.01657188384160618, - 0.9990552433374398, - 0.03993419996841205, - 0.0, - 0.9998158195093524, - -0.016213384129055507, - -0.009284402788233788, - 0.0, - -0.00862832888493567, - 0.04008147660504227, - -0.9991591600810217, - 0.0, - 0.4936055805433891, - -0.33471007413492543, - 0.3264931543876324, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "clampgripper", - "q": [ - -0.23730045444296113, - 0.17615896566081463, - -0.3904649787620021, - -1.854275331395351, - 0.10299784066871676, - 1.9849424978146744, - -1.4353288235127737 - ] - }, - { - "_id": "65d8c5483d14aeb8d50802f1", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.933624932569012, - 0.35628124395277827, - 0.03726804789865669, - 0.0, - 0.35693730830490317, - -0.9340340742834403, - -0.012524093529490974, - 0.0, - 0.03034811129248953, - 0.024995643923285543, - -0.9992268060484761, - 0.0, - 0.44545936807828224, - -0.1661834851828791, - 0.31646079888985057, - 1.0 - ], - "O_T_TCP": [ - 0.933624932569012, - 0.35628124395277827, - 0.03726804789865669, - 0.0, - 0.35693730830490317, - -0.9340340742834403, - -0.012524093529490974, - 0.0, - 0.03034811129248953, - 0.024995643923285543, - -0.9992268060484761, - 0.0, - 0.4492528819898434, - -0.1630590296924684, - 0.19155744813379105, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "outerring", - "q": [ - 0.11351189522575914, - -0.23726041963016775, - -0.4624005758860121, - -2.342262844293015, - -0.08520184533463585, - 2.1478742948373153, - 0.12773431512433453 - ] - }, - { - "_id": "65d8caf6d9a0e0f6b30af271", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.9955314808092588, - 0.09232745936288851, - -0.019325057995431105, - 0.0, - 0.09255090807183364, - -0.9956374718301944, - 0.011004590127132072, - 0.0, - -0.018225076926981512, - -0.012744212936571426, - -0.9997526852215167, - 0.0, - 0.4189947061869506, - -0.022521929892301625, - 0.37325351738924123, - 1.0 - ], - "O_T_TCP": [ - 0.9955314808092588, - 0.09232745936288851, - -0.019325057995431105, - 0.0, - 0.09255090807183364, - -0.9956374718301944, - 0.011004590127132072, - 0.0, - -0.018225076926981512, - -0.012744212936571426, - -0.9997526852215167, - 0.0, - 0.41671657157107794, - -0.024114956509373054, - 0.24828443173655163, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "housingringhole", - "q": [ - 0.09830670809745788, - -0.4161818052843997, - -0.1303831775015659, - -2.3745934094219336, - -0.07053159491883385, - 1.9427183595498403, - 0.6984895182107446 - ] - } -] \ No newline at end of file diff --git a/experiments/gearbox/mios_memory_backups/gearbox_exp3.json b/experiments/gearbox/mios_memory_backups/gearbox_exp3.json deleted file mode 100644 index 98498cc3..00000000 --- a/experiments/gearbox/mios_memory_backups/gearbox_exp3.json +++ /dev/null @@ -1,3902 +0,0 @@ -[ - { - "_id": "643d40f9dd8c607f760e5af8", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "NullObject", - "q": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ] - }, - { - "_id": "643d40f9dd8c607f760e5af9", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "NoneObject", - "q": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ] - }, - { - "_id": "643d40f9dd8c607f760e5afa", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.995807344231971, - 0.09038182927767592, - -0.013401679420805597, - 0.0, - 0.09056014012878236, - -0.9957917808908888, - 0.01335428886970798, - 0.0, - -0.012138530864960013, - -0.014512236305278915, - -0.9998210095141331, - 0.0, - 0.4186402258857764, - -0.0213018116400438, - 0.38213174742071804, - 1.0 - ], - "O_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "EndEffector", - "q": [ - 0.10379081284999846, - -0.42633626818015974, - -0.1307515779083688, - -2.3616170134324452, - -0.07320591049061881, - 1.9256821783758933, - 0.7056310605141853 - ] - }, - { - "_id": "6491ad548829c1097f0151a9", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "ring", - "q": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ] - }, - { - "_id": "64f8a17c8d1b98ac5f0c6b91", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "geometry": null, - "grasp_force": 1.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "TestObject1", - "q": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ] - }, - { - "_id": "6526b75ece654b001f026d61", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "gripper_connected", - "q": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ] - }, - { - "_id": "6526da7d86c098f7fe057f61", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "tool_load", - "q": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ] - }, - { - "_id": "6530168b631a09bd8e08d211", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "tool_grasp", - "q": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ] - }, - { - "_id": "65a8008494d3e83f7e035b11", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "kios_guidance", - "q": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ] - }, - { - "_id": "65b2664c677d37a9900fd7b1", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.020523632927385155, - 0.999779614758663, - 0.0004993052070315127, - 0.0, - 0.9997626802269471, - -0.020520362759945312, - -0.00585190899106819, - 0.0, - -0.0058404858401676345, - 0.000619301067554896, - -0.99998275244693, - 0.0, - 0.49407319098460234, - -0.3343870569274952, - 0.3249152265674699, - 1.0 - ], - "O_T_TCP": [ - 0.020523632927385155, - 0.999779614758663, - 0.0004993052070315127, - 0.0, - 0.9997626802269471, - -0.020520362759945312, - -0.00585190899106819, - 0.0, - -0.0058404858401676345, - 0.000619301067554896, - -0.99998275244693, - 0.0, - 0.49407319098460234, - -0.3343870569274952, - 0.3249152265674699, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "parallel_box1", - "q": [ - -0.4889897713452055, - 0.1520400121422783, - -0.11095685046074681, - -1.8813407051080338, - 0.015587054766497403, - 2.027237653652827, - -1.3705014213786446 - ] - }, - { - "_id": "65bb9aa7666d3392110788a1", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - -0.0055492434676263965, - 0.9999687095518451, - 0.003540157533654855, - 0.0, - 0.9994642785591136, - 0.005433245086445076, - 0.0319747186594265, - 0.0, - 0.03195509884729998, - 0.003715768034059008, - -0.9994823984070837, - 0.0, - 0.5073771436733974, - 0.2821340073022507, - 0.3273610300796691, - 1.0 - ], - "O_T_TCP": [ - -0.0055492434676263965, - 0.9999687095518451, - 0.003540157533654855, - 0.0, - 0.9994642785591136, - 0.005433245086445076, - 0.0319747186594265, - 0.0, - 0.03195509884729998, - 0.003715768034059008, - -0.9994823984070837, - 0.0, - 0.5073771436733974, - 0.2821340073022507, - 0.3273610300796691, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "inward_claw", - "q": [ - -0.3558367933193034, - 0.12986394129696988, - 0.8984244877664667, - -1.9720981870183556, - -0.1313234915672495, - 2.079114055554075, - -0.1914156752901866 - ] - }, - { - "_id": "65bbb462b8aab7a7d90d58c1", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.9949942224611444, - -0.05041086343049174, - 0.08617417837347448, - 0.0, - -0.04503393357509275, - -0.9969720659778388, - -0.06324074151875426, - 0.0, - 0.08910298453945457, - 0.05904454700105643, - -0.9942707878719971, - 0.0, - 0.6906053200978064, - -0.01614832324583123, - 0.2887581517923185, - 1.0 - ], - "O_T_TCP": [ - 0.9949942224611444, - -0.05041086343049174, - 0.08617417837347448, - 0.0, - -0.04503393357509275, - -0.9969720659778388, - -0.06324074151875426, - 0.0, - 0.08910298453945457, - 0.05904454700105643, - -0.9942707878719971, - 0.0, - 0.6990701036290545, - -0.01053909128073087, - 0.1943024269444788, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "test_object1", - "q": [ - -0.5258051569263356, - 0.4877585826129244, - 0.5891767306537179, - -1.6077540129764138, - -0.23719313260094485, - 2.1022328375975285, - 0.9414903357566682 - ] - }, - { - "_id": "65bbba7c5a0f2e8e04072891", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.005516821389947922, - -0.9949443596914403, - -0.10018000150304039, - 0.0, - -0.999821715401967, - -0.003733290448562814, - -0.017981847911249182, - 0.0, - 0.017517274375192557, - 0.10026327397364804, - -0.9948067254449727, - 0.0, - 0.643320166007396, - 0.11088726408849338, - 0.3035784369329164, - 1.0 - ], - "O_T_TCP": [ - 0.005516821389947922, - -0.9949443596914403, - -0.10018000150304039, - 0.0, - -0.999821715401967, - -0.003733290448562814, - -0.017981847911249182, - 0.0, - 0.017517274375192557, - 0.10026327397364804, - -0.9948067254449727, - 0.0, - 0.6455098253042951, - 0.12342017333519939, - 0.17922759625229484, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "gear3", - "q": [ - -0.4348329623676183, - 0.39139715203497827, - 0.6694900968644402, - -1.711178953283276, - -0.16270597490334937, - 2.05581780107816, - 2.617633598559432 - ] - }, - { - "_id": "65bbbab25a0f2e8e04072892", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.9980187907769038, - -0.05571650543342081, - -0.02889482963615323, - 0.0, - -0.05609811498518191, - -0.9983365128919355, - -0.012568031790528969, - 0.0, - -0.028147058566118602, - 0.014164350073066843, - -0.9995034338515715, - 0.0, - 0.507308310840537, - -0.04026469732735698, - 0.3761685279429274, - 1.0 - ], - "O_T_TCP": [ - 0.9980187907769038, - -0.05571650543342081, - -0.02889482963615323, - 0.0, - -0.05609811498518191, - -0.9983365128919355, - -0.012568031790528969, - 0.0, - -0.028147058566118602, - 0.014164350073066843, - -0.9995034338515715, - 0.0, - 0.5046343402767557, - -0.038919084070415624, - 0.2812157017270281, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "shaft2", - "q": [ - -0.04717889234179201, - -0.13185073856974885, - -0.034486469687599886, - -2.0883627749325773, - 0.0077889278013025046, - 1.9273695102532702, - 0.7572235297293566 - ] - }, - { - "_id": "65bfcf0f5a0e8797bb01e281", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.9991077043349246, - -0.009049464669151653, - -0.041020107618180826, - 0.0, - -0.009843038297554691, - -0.9997578641005138, - -0.019185272149743374, - 0.0, - -0.0408373449804758, - 0.019572292532108137, - -0.9989740920664474, - 0.0, - 0.4292203229453998, - -0.033877595855194384, - 0.4236606084507305, - 1.0 - ], - "O_T_TCP": [ - 0.9991077043349246, - -0.009049464669151653, - -0.041020107618180826, - 0.0, - -0.009843038297554691, - -0.9997578641005138, - -0.019185272149743374, - 0.0, - -0.0408373449804758, - 0.019572292532108137, - -0.9989740920664474, - 0.0, - 0.4292203229453998, - -0.033877595855194384, - 0.4236606084507305, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "test", - "q": [ - -0.028742359890226728, - -0.3809770326088004, - -0.048948421164719876, - -2.2124784198559695, - -0.0017267249742586607, - 1.7897263017493257, - 0.7214576442114876 - ] - }, - { - "_id": "65ce303af045be1f60050f41", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - -0.00034130509165598697, - 0.9999088204921296, - 0.012766405683707687, - 0.0, - 0.9999006778257512, - 0.00017031581705472163, - 0.013392250644843597, - 0.0, - 0.01338911300712114, - 0.012769954401065334, - -0.9998288153066389, - 0.0, - 0.34586891787172797, - 0.28478778305517327, - 0.325633698814514, - 1.0 - ], - "O_T_TCP": [ - -0.00034130509165598697, - 0.9999088204921296, - 0.012766405683707687, - 0.0, - 0.9999006778257512, - 0.00017031581705472163, - 0.013392250644843597, - 0.0, - 0.01338911300712114, - 0.012769954401065334, - -0.9998288153066389, - 0.0, - 0.34586891787172797, - 0.28478778305517327, - 0.325633698814514, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "outward_claw", - "q": [ - -0.26322421671319607, - -0.48738894555267787, - 0.8759835133827272, - -2.392889458566437, - 0.4210987845394346, - 2.0469462890460046, - -0.42977081686632607 - ] - }, - { - "_id": "65ce3098f045be1f60050f42", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.016519502886665666, - 0.9997483636116822, - -0.014527986142256165, - 0.0, - 0.999314451509194, - -0.01603160985419312, - 0.033081133468638016, - 0.0, - 0.0328405343279425, - -0.015064800427202131, - -0.999347062382806, - 0.0, - 0.3334408867064976, - -0.33165373934959796, - 0.32422469388972297, - 1.0 - ], - "O_T_TCP": [ - 0.016519502886665666, - 0.9997483636116822, - -0.014527986142256165, - 0.0, - 0.999314451509194, - -0.01603160985419312, - 0.033081133468638016, - 0.0, - 0.0328405343279425, - -0.015064800427202131, - -0.999347062382806, - 0.0, - 0.3334408867064976, - -0.33165373934959796, - 0.32422469388972297, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "parallel_box2", - "q": [ - -0.49195081603365076, - -0.2536298767587599, - -0.28172348515657036, - -2.357436443128084, - -0.06903641326254598, - 2.1465019415994413, - -1.4987806172708702 - ] - }, - { - "_id": "65ce3113f045be1f60050f43", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.05033441300943016, - 0.998522325819256, - -0.020008963209219598, - 0.0, - 0.9980401159147424, - -0.049549287905040595, - 0.037967644043304945, - 0.0, - 0.036920821197481454, - -0.021881248318769652, - -0.9990786074849747, - 0.0, - 0.4951086360500897, - -0.1768782158810579, - 0.5007737812082008, - 1.0 - ], - "O_T_TCP": [ - 0.05033441300943016, - 0.998522325819256, - -0.020008963209219598, - 0.0, - 0.9980401159147424, - -0.049549287905040595, - 0.037967644043304945, - 0.0, - 0.036920821197481454, - -0.021881248318769652, - -0.9990786074849747, - 0.0, - 0.49824690585187564, - -0.1787381219881533, - 0.415852099571978, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "shaft1", - "q": [ - -0.6783767049908637, - -0.14173239019669984, - 0.31067318724331106, - -1.7888499834156932, - 0.03642009920908874, - 1.6961157420094577, - -1.1089101982976917 - ] - }, - { - "_id": "65ce315df045be1f60050f44", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.06323640929784359, - 0.9978521827830034, - 0.016520434483373037, - 0.0, - 0.9977655821240671, - -0.06356378921021756, - 0.020105589547900215, - 0.0, - 0.021112914320698742, - 0.015212408525423958, - -0.9996613563981266, - 0.0, - 0.3844454340627629, - -0.034519055467682386, - 0.3511666416459713, - 1.0 - ], - "O_T_TCP": [ - 0.06323640929784359, - 0.9978521827830034, - 0.016520434483373037, - 0.0, - 0.9977655821240671, - -0.06356378921021756, - 0.020105589547900215, - 0.0, - 0.021112914320698742, - 0.015212408525423958, - -0.9996613563981266, - 0.0, - 0.38624003178002225, - -0.03322600074302135, - 0.2661954263521305, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "gearbase_hole1", - "q": [ - -0.4779728950460752, - -0.5647525487445973, - 0.34118822538716465, - -2.533109467228605, - 0.2194305768719429, - 2.0038177083303617, - -0.9983835912605101 - ] - }, - { - "_id": "65ce31d5f045be1f60050f45", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.016773866766090845, - 0.998555668652971, - 0.050852344055508855, - 0.0, - 0.999651562629023, - -0.017761257817640888, - 0.019027295007331226, - 0.0, - 0.019903398084193856, - 0.05051643648500831, - -0.9985248842114842, - 0.0, - 0.6166632788352968, - -0.06722110520041134, - 0.29656367644075515, - 1.0 - ], - "O_T_TCP": [ - 0.016773866766090845, - 0.998555668652971, - 0.050852344055508855, - 0.0, - 0.999651562629023, - -0.017761257817640888, - 0.019027295007331226, - 0.0, - 0.019903398084193856, - 0.05051643648500831, - -0.9985248842114842, - 0.0, - 0.6171608637874016, - -0.06595819428828614, - 0.27160055433546804, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "shaft3", - "q": [ - -0.6255808761370809, - 0.2721908670313347, - 0.5482196540661509, - -1.844289931665387, - -0.10189347790347204, - 2.091038903571352, - -0.8129646023995416 - ] - }, - { - "_id": "65ce3207f045be1f60050f46", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - -0.15915497945225549, - 0.9860926026332786, - -0.04766359668224283, - 0.0, - 0.9850428396108329, - 0.16183700693102337, - 0.058992662427146104, - 0.0, - 0.06588723038902841, - -0.03756243183872672, - -0.9971198205762556, - 0.0, - 0.5043739432190215, - 0.08072733253032562, - 0.2875533813662191, - 1.0 - ], - "O_T_TCP": [ - -0.15915497945225549, - 0.9860926026332786, - -0.04766359668224283, - 0.0, - 0.9850428396108329, - 0.16183700693102337, - 0.058992662427146104, - 0.0, - 0.06588723038902841, - -0.03756243183872672, - -0.9971198205762556, - 0.0, - 0.5060211239787472, - 0.07978827173435744, - 0.2626253858518127, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "gearbase_hole3", - "q": [ - -0.6320993901689848, - -0.13105960240977546, - 0.7911574711811249, - -2.301617453056469, - 0.05951137099465102, - 2.2677469761391547, - -0.8261827120045797 - ] - }, - { - "_id": "65ce326bf045be1f60050f47", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.9982800059810064, - 0.0028567036015946868, - 0.058391915573158704, - 0.0, - 0.002665300689743662, - -0.9999811917035863, - 0.003355489907203343, - 0.0, - 0.05840152737118158, - -0.003194147969025999, - -0.9982880641475511, - 0.0, - 0.38329798339116805, - -0.1792311431159989, - 0.2652977941807128, - 1.0 - ], - "O_T_TCP": [ - 0.9982800059810064, - 0.0028567036015946868, - 0.058391915573158704, - 0.0, - 0.002665300689743662, - -0.9999811917035863, - 0.003355489907203343, - 0.0, - 0.05840152737118158, - -0.003194147969025999, - -0.9982880641475511, - 0.0, - 0.38738609030715077, - -0.1794547334738307, - 0.19541762969038423, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "gear1", - "q": [ - -0.743737426383209, - -0.3330958715231419, - 0.30138807209333335, - -2.598988366910377, - 0.16539736091956653, - 2.3272367282354502, - 0.21644120803806513 - ] - }, - { - "_id": "65ce3349f045be1f60050f48", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - -0.43529958087664555, - -0.8981800970094251, - 0.06138025029245873, - 0.0, - -0.9001402451487228, - 0.43304221718444524, - -0.04693318762651431, - 0.0, - 0.01557451519504121, - -0.07568228756712969, - -0.9970103438906971, - 0.0, - 0.32523152397741284, - 0.13156707585417038, - 0.31425307836568933, - 1.0 - ], - "O_T_TCP": [ - -0.43529958087664555, - -0.8981800970094251, - 0.06138025029245873, - 0.0, - -0.9001402451487228, - 0.43304221718444524, - -0.04693318762651431, - 0.0, - 0.01557451519504121, - -0.07568228756712969, - -0.9970103438906971, - 0.0, - 0.327178338376793, - 0.12210678990827917, - 0.18962678537935218, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "gear2", - "q": [ - -0.44017739813892465, - -0.8408044241896847, - 0.7812836869963553, - -2.6489709372327193, - 0.5117495865954292, - 1.9353651503056144, - 2.7550759355656957 - ] - }, - { - "_id": "65ce3392f045be1f60050f49", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - -0.19207017045850883, - -0.9745158408444792, - -0.11579582230989756, - 0.0, - -0.9619084037954584, - 0.21032850120116167, - -0.1745705908546879, - 0.0, - 0.19448071224163638, - 0.07785667044396119, - -0.9778116339220793, - 0.0, - 0.6668309353879216, - 0.1002329166290898, - 0.3839163655785476, - 1.0 - ], - "O_T_TCP": [ - -0.19207017045850883, - -0.9745158408444792, - -0.11579582230989756, - 0.0, - -0.9619084037954584, - 0.21032850120116167, - -0.1745705908546879, - 0.0, - 0.19448071224163638, - 0.07785667044396119, - -0.9778116339220793, - 0.0, - 0.6911410244181262, - 0.10996500043458494, - 0.2616899113382877, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "no_tool", - "q": [ - -0.449013091942916, - 0.2972115466353616, - 0.6671699467458255, - -1.6625072732841806, - -0.16246066116120655, - 2.102316742226921, - 2.8067302440068085 - ] - }, - { - "_id": "65ce3399f045be1f60050f4a", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.37121811244401426, - -0.911720063290249, - 0.1759101648249346, - 0.0, - -0.9285316648543275, - -0.3639617925463895, - 0.07308561986107809, - 0.0, - -0.00260909728174289, - -0.1904725312477871, - -0.9816890584352234, - 0.0, - 0.5482232455292271, - -0.09299434620256447, - 0.4961290551977721, - 1.0 - ], - "O_T_TCP": [ - 0.37121811244401426, - -0.911720063290249, - 0.1759101648249346, - 0.0, - -0.9285316648543275, - -0.3639617925463895, - 0.07308561986107809, - 0.0, - -0.00260909728174289, - -0.1904725312477871, - -0.9816890584352234, - 0.0, - 0.5478971083690092, - -0.11680341260853785, - 0.37341792289336917, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "gearbase", - "q": [ - -0.5710980864097402, - -0.030782228559778448, - 0.4631181385266316, - -1.681660839952254, - -0.1779492470121923, - 1.6707865634759267, - 1.8844083975966868 - ] - }, - { - "_id": "65cfa2c47f0205483b054d51", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.999127486937346, - 0.0020339066230885923, - 0.04148343008142645, - 0.0, - -0.0021570557684146043, - -0.9949015779177928, - 0.1007320419147068, - 0.0, - 0.041477608185149145, - -0.10073557343640634, - -0.994048264554432, - 0.0, - 0.3646566267993078, - -0.02254569907028298, - 0.40952082232522724, - 1.0 - ], - "O_T_TCP": [ - 0.9991272968724854, - 0.002032794737528359, - 0.04148806203046724, - 0.0, - -0.002158543289712084, - -0.9949017859475108, - 0.10072995537836842, - 0.0, - 0.041482108991960025, - -0.10073354124199559, - -0.9940482826816947, - 0.0, - 0.36465772773036376, - -0.02254865462649719, - 0.40952225264460873, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "test1", - "q": [ - -0.6494016965366246, - -0.753164953780274, - 0.5019667189581353, - -2.4719201877805506, - 0.2581255530648761, - 1.8439903862476348, - 0.4656479083630774 - ] - }, - { - "_id": "65cfa7905d75dfc96909caf1", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - -0.9935878908904953, - 0.08960520143219396, - -0.06880957675557518, - 0.0, - 0.09120083791165756, - 0.9956138318810523, - -0.02040225067357439, - 0.0, - 0.06668090241645759, - -0.026547431392950203, - -0.9974211202593233, - 0.0, - 0.5080013634448317, - 0.07744840851681808, - 0.310686758863888, - 1.0 - ], - "O_T_TCP": [ - -0.9935878908904953, - 0.08960520143219396, - -0.06880957675557518, - 0.0, - 0.09120083791165756, - 0.9956138318810523, - -0.02040225067357439, - 0.0, - 0.06668090241645759, - -0.026547431392950203, - -0.9974211202593233, - 0.0, - 0.5080013634448317, - 0.07744840851681808, - 0.310686758863888, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "test_object", - "q": [ - -0.6783759433123, - -0.16586847048823597, - 0.8175371052577723, - -2.261259556151273, - 0.1062502110335562, - 2.2074089546736957, - -2.190133549686696 - ] - }, - { - "_id": "65cfa8e660f0281a320a1ef1", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.9999648408880725, - 0.0002669773375134615, - 0.00714091114644764, - 0.0, - -4.124753500410915e-05, - -0.9990599232033038, - 0.04312789177385854, - 0.0, - 0.00714584989104384, - -0.0431275003184912, - -0.9990440208247148, - 0.0, - 0.5039719752053793, - 0.08282098192276433, - 0.4255527712058619, - 1.0 - ], - "O_T_TCP": [ - 0.9999648408880725, - 0.0002669773375134615, - 0.00714091114644764, - 0.0, - -4.124753500410915e-05, - -0.9990599232033038, - 0.04312789177385854, - 0.0, - 0.00714584989104384, - -0.0431275003184912, - -0.9990440208247148, - 0.0, - 0.5039719752053793, - 0.08282098192276433, - 0.4255527712058619, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "aaa", - "q": [ - -0.7009867876172065, - -0.24024575297265152, - 0.7995970885804996, - -1.9988235254230773, - 0.13235241008271123, - 1.8310430494149526, - 0.8356096653159838 - ] - }, - { - "_id": "65d8ab25365be5466b0fdc71", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.9021773910215635, - -0.038249869200695535, - 0.42964363085802043, - 0.0, - -0.03475251865213935, - -0.9992584430166064, - -0.015986663459536494, - 0.0, - 0.42994479117900936, - -0.0005084017524109786, - -0.9028550371270552, - 0.0, - 0.5071971613100603, - 0.04661989698448783, - 0.6363466027888695, - 1.0 - ], - "O_T_TCP": [ - 0.9021773910215635, - -0.038249869200695535, - 0.42964363085802043, - 0.0, - -0.03475251865213935, - -0.9992584430166064, - -0.015986663459536494, - 0.0, - 0.42994479117900936, - -0.0005084017524109786, - -0.9028550371270552, - 0.0, - 0.5071971613100603, - 0.04661989698448783, - 0.6363466027888695, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "yawgripper", - "q": [ - 0.11421029169640254, - -0.34931760987484234, - -0.004396662544784647, - -1.792835893163279, - -0.05164978933003214, - 1.8851889863806732, - 0.9384602965859719 - ] - }, - { - "_id": "65d8bc19d2850057440d7331", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.026665700373889236, - 0.9990310854222944, - -0.03473582710589628, - 0.0, - 0.9993871064511708, - -0.025869536002868367, - 0.023171652236316353, - 0.0, - 0.022251029556039845, - -0.03533310634903862, - -0.9991278513180502, - 0.0, - 0.34529017685396074, - 0.28611390088978156, - 0.32766119170850694, - 1.0 - ], - "O_T_TCP": [ - 0.026665700373889236, - 0.9990310854222944, - -0.03473582710589628, - 0.0, - 0.9993871064511708, - -0.025869536002868367, - 0.023171652236316353, - 0.0, - 0.022251029556039845, - -0.03533310634903862, - -0.9991278513180502, - 0.0, - 0.34529017685396074, - 0.28611390088978156, - 0.32766119170850694, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "outwardgripper", - "q": [ - 0.870870500266552, - -0.2957360618239955, - -0.1551851647736568, - -2.386680684173316, - -0.09917618758749594, - 2.0854945325421754, - 0.012588030038608445 - ] - }, - { - "_id": "65d8bc40d2850057440d7332", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.03396390773103101, - 0.9993034014499432, - -0.014829420872713595, - 0.0, - 0.998694979847861, - -0.033373237107500404, - 0.03840977965122197, - 0.0, - 0.037888847153174234, - -0.0161149246539099, - -0.999152012691163, - 0.0, - 0.5057945972945914, - 0.28251383833072985, - 0.32803008980848014, - 1.0 - ], - "O_T_TCP": [ - 0.03396390773103101, - 0.9993034014499432, - -0.014829420872713595, - 0.0, - 0.998694979847861, - -0.033373237107500404, - 0.03840977965122197, - 0.0, - 0.037888847153174234, - -0.0161149246539099, - -0.999152012691163, - 0.0, - 0.5057945972945914, - 0.28251383833072985, - 0.32803008980848014, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "inwardgripper", - "q": [ - 0.7132648979574978, - 0.0822906340856455, - -0.19983831917677083, - -1.9729896309966, - -0.018719173115160727, - 2.078648384739713, - -0.2285200706422465 - ] - }, - { - "_id": "65d8bc9cd2850057440d7333", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.9159565622373924, - 0.11163201311977883, - 0.38541226841175086, - 0.0, - 0.14128903439883572, - -0.9887250951913703, - -0.04940487626407901, - 0.0, - 0.3755588466496142, - 0.0997091675804246, - -0.9214193587089595, - 0.0, - 0.4479657579420528, - 0.04195357437032795, - 0.6367311153952495, - 1.0 - ], - "O_T_TCP": [ - 0.9159565622373924, - 0.11163201311977883, - 0.38541226841175086, - 0.0, - 0.14128903439883572, - -0.9887250951913703, - -0.04940487626407901, - 0.0, - 0.3755588466496142, - 0.0997091675804246, - -0.9214193587089595, - 0.0, - 0.4479657579420528, - 0.04195357437032795, - 0.6367311153952495, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "defaultgripper", - "q": [ - 0.021865537859103813, - -0.491120350155914, - 0.026315845489235032, - -1.8907418638905291, - 0.09657258354955249, - 1.7900419552325462, - 0.6985316049030074 - ] - }, - { - "_id": "65d8bcbbd2850057440d7334", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.024177210511319505, - 0.9975796811973359, - 0.06504605334162625, - 0.0, - 0.9996394690321734, - -0.024828988657792546, - 0.009230394168961284, - 0.0, - 0.010823289774918731, - 0.06480068465611978, - -0.9978395300180016, - 0.0, - 0.3348185254531909, - -0.33224274111487373, - 0.3257033634399661, - 1.0 - ], - "O_T_TCP": [ - 0.024177210511319505, - 0.9975796811973359, - 0.06504605334162625, - 0.0, - 0.9996394690321734, - -0.024828988657792546, - 0.009230394168961284, - 0.0, - 0.010823289774918731, - 0.06480068465611978, - -0.9978395300180016, - 0.0, - 0.3350891076975639, - -0.33062272399847076, - 0.30075737518951606, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "parallelgripper", - "q": [ - -0.5970329341689745, - -0.20463424487443924, - -0.19403221863983816, - -2.293547858754675, - 0.01611855673189336, - 2.05432398039057, - -1.5549974032722624 - ] - }, - { - "_id": "65d8bd19d2850057440d7335", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.01657188384160618, - 0.9990552433374398, - 0.03993419996841205, - 0.0, - 0.9998158195093524, - -0.016213384129055507, - -0.009284402788233788, - 0.0, - -0.00862832888493567, - 0.04008147660504227, - -0.9991591600810217, - 0.0, - 0.4936055805433891, - -0.33471007413492543, - 0.3264931543876324, - 1.0 - ], - "O_T_TCP": [ - 0.01657188384160618, - 0.9990552433374398, - 0.03993419996841205, - 0.0, - 0.9998158195093524, - -0.016213384129055507, - -0.009284402788233788, - 0.0, - -0.00862832888493567, - 0.04008147660504227, - -0.9991591600810217, - 0.0, - 0.4936055805433891, - -0.33471007413492543, - 0.3264931543876324, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "clampgripper", - "q": [ - -0.23730045444296113, - 0.17615896566081463, - -0.3904649787620021, - -1.854275331395351, - 0.10299784066871676, - 1.9849424978146744, - -1.4353288235127737 - ] - }, - { - "_id": "65d8c5483d14aeb8d50802f1", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.933624932569012, - 0.35628124395277827, - 0.03726804789865669, - 0.0, - 0.35693730830490317, - -0.9340340742834403, - -0.012524093529490974, - 0.0, - 0.03034811129248953, - 0.024995643923285543, - -0.9992268060484761, - 0.0, - 0.44545936807828224, - -0.1661834851828791, - 0.31646079888985057, - 1.0 - ], - "O_T_TCP": [ - 0.933624932569012, - 0.35628124395277827, - 0.03726804789865669, - 0.0, - 0.35693730830490317, - -0.9340340742834403, - -0.012524093529490974, - 0.0, - 0.03034811129248953, - 0.024995643923285543, - -0.9992268060484761, - 0.0, - 0.4492528819898434, - -0.1630590296924684, - 0.19155744813379105, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "outerring", - "q": [ - 0.11351189522575914, - -0.23726041963016775, - -0.4624005758860121, - -2.342262844293015, - -0.08520184533463585, - 2.1478742948373153, - 0.12773431512433453 - ] - }, - { - "_id": "65d8caf6d9a0e0f6b30af271", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.9955314808092588, - 0.09232745936288851, - -0.019325057995431105, - 0.0, - 0.09255090807183364, - -0.9956374718301944, - 0.011004590127132072, - 0.0, - -0.018225076926981512, - -0.012744212936571426, - -0.9997526852215167, - 0.0, - 0.4189947061869506, - -0.022521929892301625, - 0.37325351738924123, - 1.0 - ], - "O_T_TCP": [ - 0.9955314808092588, - 0.09232745936288851, - -0.019325057995431105, - 0.0, - 0.09255090807183364, - -0.9956374718301944, - 0.011004590127132072, - 0.0, - -0.018225076926981512, - -0.012744212936571426, - -0.9997526852215167, - 0.0, - 0.41671657157107794, - -0.024114956509373054, - 0.24828443173655163, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "housingringhole", - "q": [ - 0.09830670809745788, - -0.4161818052843997, - -0.1303831775015659, - -2.3745934094219336, - -0.07053159491883385, - 1.9427183595498403, - 0.6984895182107446 - ] - }, - { - "_id": "65d8db2f6acb8ae6770b96c1", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.513001060316066, - 0.8583295809161242, - 0.008999418878517074, - 0.0, - 0.8583361185675584, - -0.5130505969748391, - 0.004351942939397769, - 0.0, - 0.008352719403125902, - 0.005492080666775529, - -0.9999500333159265, - 0.0, - 0.38178532250990727, - -0.16547598632922042, - 0.31909558296374546, - 1.0 - ], - "O_T_TCP": [ - 0.513001060316066, - 0.8583295809161242, - 0.008999418878517074, - 0.0, - 0.8583361185675584, - -0.5130505969748391, - 0.004351942939397769, - 0.0, - 0.008352719403125902, - 0.005492080666775529, - -0.9999500333159265, - 0.0, - 0.382829412435298, - -0.16478947624587348, - 0.19410182879925464, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "cone", - "q": [ - 0.0869105160832405, - -0.4402206016496565, - -0.469553243825189, - -2.500408704956349, - -0.21578491499331962, - 2.098576180547614, - -0.48130604514148495 - ] - }, - { - "_id": "65d8dba16acb8ae6770b96c2", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.9969873134371252, - 0.02061782349464016, - 0.07464548948848683, - 0.0, - 0.020458013682692375, - -0.9997768639720687, - 0.002904968767964077, - 0.0, - 0.07469016554091062, - -0.0013691449225669697, - -0.9972058486659875, - 0.0, - 0.5518166349985582, - 0.14392434515325628, - 0.3121375998572833, - 1.0 - ], - "O_T_TCP": [ - 0.9969873134371252, - 0.02061782349464016, - 0.07464548948848683, - 0.0, - 0.020458013682692375, - -0.9997768639720687, - 0.002904968767964077, - 0.0, - 0.07469016554091062, - -0.0013691449225669697, - -0.9972058486659875, - 0.0, - 0.5536838891370809, - 0.1438901165301921, - 0.2872074536406336, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "outputshaftandgearstage2", - "q": [ - 0.09592684324582418, - 0.04533265894128565, - 0.1663982636708567, - -2.0732436234452254, - -0.03458040425040109, - 2.189542146382252, - 1.0457670297921173 - ] - }, - { - "_id": "65d8dbd46acb8ae6770b96c3", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.9970287495301109, - 0.06377828756600727, - 0.042973823972031255, - 0.0, - 0.06151160371655549, - -0.9967333583024899, - 0.05215057004929151, - 0.0, - 0.046160406670947514, - -0.04935317901480623, - -0.9977141276824264, - 0.0, - 0.5614982813359727, - -0.05186433520096291, - 0.24341233767382356, - 1.0 - ], - "O_T_TCP": [ - 0.9970287495301109, - 0.06377828756600727, - 0.042973823972031255, - 0.0, - 0.06151160371655549, - -0.9967333583024899, - 0.05215057004929151, - 0.0, - 0.046160406670947514, - -0.04935317901480623, - -0.9977141276824264, - 0.0, - 0.5626522915027463, - -0.05309816467633307, - 0.2184694844817629, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "ringgear", - "q": [ - 0.06709579550697092, - 0.12349651820334193, - -0.15135198544089462, - -2.158161625877613, - -0.03697298612859514, - 2.3300566612568736, - 0.6647699845240143 - ] - } -] \ No newline at end of file diff --git a/experiments/gearbox/mios_memory_backups/gearbox_text.json b/experiments/gearbox/mios_memory_backups/gearbox_text.json deleted file mode 100644 index 8f82daf4..00000000 --- a/experiments/gearbox/mios_memory_backups/gearbox_text.json +++ /dev/null @@ -1,2902 +0,0 @@ -[ - { - "_id": "643d40f9dd8c607f760e5af8", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "NullObject", - "q": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ] - }, - { - "_id": "643d40f9dd8c607f760e5af9", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "NoneObject", - "q": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ] - }, - { - "_id": "643d40f9dd8c607f760e5afa", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "EndEffector", - "q": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ] - }, - { - "_id": "6491ad548829c1097f0151a9", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "ring", - "q": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ] - }, - { - "_id": "64f8a17c8d1b98ac5f0c6b91", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "geometry": null, - "grasp_force": 1.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "TestObject1", - "q": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ] - }, - { - "_id": "6526b75ece654b001f026d61", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "gripper_connected", - "q": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ] - }, - { - "_id": "6526da7d86c098f7fe057f61", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "tool_load", - "q": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ] - }, - { - "_id": "6530168b631a09bd8e08d211", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "tool_grasp", - "q": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ] - }, - { - "_id": "65a8008494d3e83f7e035b11", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "kios_guidance", - "q": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ] - }, - { - "_id": "65b2664c677d37a9900fd7b1", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.020523632927385155, - 0.999779614758663, - 0.0004993052070315127, - 0.0, - 0.9997626802269471, - -0.020520362759945312, - -0.00585190899106819, - 0.0, - -0.0058404858401676345, - 0.000619301067554896, - -0.99998275244693, - 0.0, - 0.49407319098460234, - -0.3343870569274952, - 0.3249152265674699, - 1.0 - ], - "O_T_TCP": [ - 0.020523632927385155, - 0.999779614758663, - 0.0004993052070315127, - 0.0, - 0.9997626802269471, - -0.020520362759945312, - -0.00585190899106819, - 0.0, - -0.0058404858401676345, - 0.000619301067554896, - -0.99998275244693, - 0.0, - 0.49407319098460234, - -0.3343870569274952, - 0.3249152265674699, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "parallel_box1", - "q": [ - -0.4889897713452055, - 0.1520400121422783, - -0.11095685046074681, - -1.8813407051080338, - 0.015587054766497403, - 2.027237653652827, - -1.3705014213786446 - ] - }, - { - "_id": "65bb9aa7666d3392110788a1", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - -0.0055492434676263965, - 0.9999687095518451, - 0.003540157533654855, - 0.0, - 0.9994642785591136, - 0.005433245086445076, - 0.0319747186594265, - 0.0, - 0.03195509884729998, - 0.003715768034059008, - -0.9994823984070837, - 0.0, - 0.5073771436733974, - 0.2821340073022507, - 0.3273610300796691, - 1.0 - ], - "O_T_TCP": [ - -0.0055492434676263965, - 0.9999687095518451, - 0.003540157533654855, - 0.0, - 0.9994642785591136, - 0.005433245086445076, - 0.0319747186594265, - 0.0, - 0.03195509884729998, - 0.003715768034059008, - -0.9994823984070837, - 0.0, - 0.5073771436733974, - 0.2821340073022507, - 0.3273610300796691, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "inward_claw", - "q": [ - -0.3558367933193034, - 0.12986394129696988, - 0.8984244877664667, - -1.9720981870183556, - -0.1313234915672495, - 2.079114055554075, - -0.1914156752901866 - ] - }, - { - "_id": "65bbb462b8aab7a7d90d58c1", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.9949942224611444, - -0.05041086343049174, - 0.08617417837347448, - 0.0, - -0.04503393357509275, - -0.9969720659778388, - -0.06324074151875426, - 0.0, - 0.08910298453945457, - 0.05904454700105643, - -0.9942707878719971, - 0.0, - 0.6906053200978064, - -0.01614832324583123, - 0.2887581517923185, - 1.0 - ], - "O_T_TCP": [ - 0.9949942224611444, - -0.05041086343049174, - 0.08617417837347448, - 0.0, - -0.04503393357509275, - -0.9969720659778388, - -0.06324074151875426, - 0.0, - 0.08910298453945457, - 0.05904454700105643, - -0.9942707878719971, - 0.0, - 0.6990701036290545, - -0.01053909128073087, - 0.1943024269444788, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "test_object1", - "q": [ - -0.5258051569263356, - 0.4877585826129244, - 0.5891767306537179, - -1.6077540129764138, - -0.23719313260094485, - 2.1022328375975285, - 0.9414903357566682 - ] - }, - { - "_id": "65bbba7c5a0f2e8e04072891", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.005516821389947922, - -0.9949443596914403, - -0.10018000150304039, - 0.0, - -0.999821715401967, - -0.003733290448562814, - -0.017981847911249182, - 0.0, - 0.017517274375192557, - 0.10026327397364804, - -0.9948067254449727, - 0.0, - 0.643320166007396, - 0.11088726408849338, - 0.3035784369329164, - 1.0 - ], - "O_T_TCP": [ - 0.005516821389947922, - -0.9949443596914403, - -0.10018000150304039, - 0.0, - -0.999821715401967, - -0.003733290448562814, - -0.017981847911249182, - 0.0, - 0.017517274375192557, - 0.10026327397364804, - -0.9948067254449727, - 0.0, - 0.6455098253042951, - 0.12342017333519939, - 0.17922759625229484, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "gear3", - "q": [ - -0.4348329623676183, - 0.39139715203497827, - 0.6694900968644402, - -1.711178953283276, - -0.16270597490334937, - 2.05581780107816, - 2.617633598559432 - ] - }, - { - "_id": "65bbbab25a0f2e8e04072892", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.9980187907769038, - -0.05571650543342081, - -0.02889482963615323, - 0.0, - -0.05609811498518191, - -0.9983365128919355, - -0.012568031790528969, - 0.0, - -0.028147058566118602, - 0.014164350073066843, - -0.9995034338515715, - 0.0, - 0.507308310840537, - -0.04026469732735698, - 0.3761685279429274, - 1.0 - ], - "O_T_TCP": [ - 0.9980187907769038, - -0.05571650543342081, - -0.02889482963615323, - 0.0, - -0.05609811498518191, - -0.9983365128919355, - -0.012568031790528969, - 0.0, - -0.028147058566118602, - 0.014164350073066843, - -0.9995034338515715, - 0.0, - 0.5046343402767557, - -0.038919084070415624, - 0.2812157017270281, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "shaft2", - "q": [ - -0.04717889234179201, - -0.13185073856974885, - -0.034486469687599886, - -2.0883627749325773, - 0.0077889278013025046, - 1.9273695102532702, - 0.7572235297293566 - ] - }, - { - "_id": "65bfcf0f5a0e8797bb01e281", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.9991077043349246, - -0.009049464669151653, - -0.041020107618180826, - 0.0, - -0.009843038297554691, - -0.9997578641005138, - -0.019185272149743374, - 0.0, - -0.0408373449804758, - 0.019572292532108137, - -0.9989740920664474, - 0.0, - 0.4292203229453998, - -0.033877595855194384, - 0.4236606084507305, - 1.0 - ], - "O_T_TCP": [ - 0.9991077043349246, - -0.009049464669151653, - -0.041020107618180826, - 0.0, - -0.009843038297554691, - -0.9997578641005138, - -0.019185272149743374, - 0.0, - -0.0408373449804758, - 0.019572292532108137, - -0.9989740920664474, - 0.0, - 0.4292203229453998, - -0.033877595855194384, - 0.4236606084507305, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "test", - "q": [ - -0.028742359890226728, - -0.3809770326088004, - -0.048948421164719876, - -2.2124784198559695, - -0.0017267249742586607, - 1.7897263017493257, - 0.7214576442114876 - ] - }, - { - "_id": "65ce303af045be1f60050f41", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - -0.00034130509165598697, - 0.9999088204921296, - 0.012766405683707687, - 0.0, - 0.9999006778257512, - 0.00017031581705472163, - 0.013392250644843597, - 0.0, - 0.01338911300712114, - 0.012769954401065334, - -0.9998288153066389, - 0.0, - 0.34586891787172797, - 0.28478778305517327, - 0.325633698814514, - 1.0 - ], - "O_T_TCP": [ - -0.00034130509165598697, - 0.9999088204921296, - 0.012766405683707687, - 0.0, - 0.9999006778257512, - 0.00017031581705472163, - 0.013392250644843597, - 0.0, - 0.01338911300712114, - 0.012769954401065334, - -0.9998288153066389, - 0.0, - 0.34586891787172797, - 0.28478778305517327, - 0.325633698814514, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "outward_claw", - "q": [ - -0.26322421671319607, - -0.48738894555267787, - 0.8759835133827272, - -2.392889458566437, - 0.4210987845394346, - 2.0469462890460046, - -0.42977081686632607 - ] - }, - { - "_id": "65ce3098f045be1f60050f42", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.016519502886665666, - 0.9997483636116822, - -0.014527986142256165, - 0.0, - 0.999314451509194, - -0.01603160985419312, - 0.033081133468638016, - 0.0, - 0.0328405343279425, - -0.015064800427202131, - -0.999347062382806, - 0.0, - 0.3334408867064976, - -0.33165373934959796, - 0.32422469388972297, - 1.0 - ], - "O_T_TCP": [ - 0.016519502886665666, - 0.9997483636116822, - -0.014527986142256165, - 0.0, - 0.999314451509194, - -0.01603160985419312, - 0.033081133468638016, - 0.0, - 0.0328405343279425, - -0.015064800427202131, - -0.999347062382806, - 0.0, - 0.3334408867064976, - -0.33165373934959796, - 0.32422469388972297, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "parallel_box2", - "q": [ - -0.49195081603365076, - -0.2536298767587599, - -0.28172348515657036, - -2.357436443128084, - -0.06903641326254598, - 2.1465019415994413, - -1.4987806172708702 - ] - }, - { - "_id": "65ce3113f045be1f60050f43", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.05033441300943016, - 0.998522325819256, - -0.020008963209219598, - 0.0, - 0.9980401159147424, - -0.049549287905040595, - 0.037967644043304945, - 0.0, - 0.036920821197481454, - -0.021881248318769652, - -0.9990786074849747, - 0.0, - 0.4951086360500897, - -0.1768782158810579, - 0.5007737812082008, - 1.0 - ], - "O_T_TCP": [ - 0.05033441300943016, - 0.998522325819256, - -0.020008963209219598, - 0.0, - 0.9980401159147424, - -0.049549287905040595, - 0.037967644043304945, - 0.0, - 0.036920821197481454, - -0.021881248318769652, - -0.9990786074849747, - 0.0, - 0.49824690585187564, - -0.1787381219881533, - 0.415852099571978, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "shaft1", - "q": [ - -0.6783767049908637, - -0.14173239019669984, - 0.31067318724331106, - -1.7888499834156932, - 0.03642009920908874, - 1.6961157420094577, - -1.1089101982976917 - ] - }, - { - "_id": "65ce315df045be1f60050f44", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.06323640929784359, - 0.9978521827830034, - 0.016520434483373037, - 0.0, - 0.9977655821240671, - -0.06356378921021756, - 0.020105589547900215, - 0.0, - 0.021112914320698742, - 0.015212408525423958, - -0.9996613563981266, - 0.0, - 0.3844454340627629, - -0.034519055467682386, - 0.3511666416459713, - 1.0 - ], - "O_T_TCP": [ - 0.06323640929784359, - 0.9978521827830034, - 0.016520434483373037, - 0.0, - 0.9977655821240671, - -0.06356378921021756, - 0.020105589547900215, - 0.0, - 0.021112914320698742, - 0.015212408525423958, - -0.9996613563981266, - 0.0, - 0.38624003178002225, - -0.03322600074302135, - 0.2661954263521305, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "gearbase_hole1", - "q": [ - -0.4779728950460752, - -0.5647525487445973, - 0.34118822538716465, - -2.533109467228605, - 0.2194305768719429, - 2.0038177083303617, - -0.9983835912605101 - ] - }, - { - "_id": "65ce31d5f045be1f60050f45", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.016773866766090845, - 0.998555668652971, - 0.050852344055508855, - 0.0, - 0.999651562629023, - -0.017761257817640888, - 0.019027295007331226, - 0.0, - 0.019903398084193856, - 0.05051643648500831, - -0.9985248842114842, - 0.0, - 0.6166632788352968, - -0.06722110520041134, - 0.29656367644075515, - 1.0 - ], - "O_T_TCP": [ - 0.016773866766090845, - 0.998555668652971, - 0.050852344055508855, - 0.0, - 0.999651562629023, - -0.017761257817640888, - 0.019027295007331226, - 0.0, - 0.019903398084193856, - 0.05051643648500831, - -0.9985248842114842, - 0.0, - 0.6171608637874016, - -0.06595819428828614, - 0.27160055433546804, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "shaft3", - "q": [ - -0.6255808761370809, - 0.2721908670313347, - 0.5482196540661509, - -1.844289931665387, - -0.10189347790347204, - 2.091038903571352, - -0.8129646023995416 - ] - }, - { - "_id": "65ce3207f045be1f60050f46", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - -0.15915497945225549, - 0.9860926026332786, - -0.04766359668224283, - 0.0, - 0.9850428396108329, - 0.16183700693102337, - 0.058992662427146104, - 0.0, - 0.06588723038902841, - -0.03756243183872672, - -0.9971198205762556, - 0.0, - 0.5043739432190215, - 0.08072733253032562, - 0.2875533813662191, - 1.0 - ], - "O_T_TCP": [ - -0.15915497945225549, - 0.9860926026332786, - -0.04766359668224283, - 0.0, - 0.9850428396108329, - 0.16183700693102337, - 0.058992662427146104, - 0.0, - 0.06588723038902841, - -0.03756243183872672, - -0.9971198205762556, - 0.0, - 0.5060211239787472, - 0.07978827173435744, - 0.2626253858518127, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "gearbase_hole3", - "q": [ - -0.6320993901689848, - -0.13105960240977546, - 0.7911574711811249, - -2.301617453056469, - 0.05951137099465102, - 2.2677469761391547, - -0.8261827120045797 - ] - }, - { - "_id": "65ce326bf045be1f60050f47", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.9982800059810064, - 0.0028567036015946868, - 0.058391915573158704, - 0.0, - 0.002665300689743662, - -0.9999811917035863, - 0.003355489907203343, - 0.0, - 0.05840152737118158, - -0.003194147969025999, - -0.9982880641475511, - 0.0, - 0.38329798339116805, - -0.1792311431159989, - 0.2652977941807128, - 1.0 - ], - "O_T_TCP": [ - 0.9982800059810064, - 0.0028567036015946868, - 0.058391915573158704, - 0.0, - 0.002665300689743662, - -0.9999811917035863, - 0.003355489907203343, - 0.0, - 0.05840152737118158, - -0.003194147969025999, - -0.9982880641475511, - 0.0, - 0.38738609030715077, - -0.1794547334738307, - 0.19541762969038423, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "gear1", - "q": [ - -0.743737426383209, - -0.3330958715231419, - 0.30138807209333335, - -2.598988366910377, - 0.16539736091956653, - 2.3272367282354502, - 0.21644120803806513 - ] - }, - { - "_id": "65ce3349f045be1f60050f48", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - -0.43529958087664555, - -0.8981800970094251, - 0.06138025029245873, - 0.0, - -0.9001402451487228, - 0.43304221718444524, - -0.04693318762651431, - 0.0, - 0.01557451519504121, - -0.07568228756712969, - -0.9970103438906971, - 0.0, - 0.32523152397741284, - 0.13156707585417038, - 0.31425307836568933, - 1.0 - ], - "O_T_TCP": [ - -0.43529958087664555, - -0.8981800970094251, - 0.06138025029245873, - 0.0, - -0.9001402451487228, - 0.43304221718444524, - -0.04693318762651431, - 0.0, - 0.01557451519504121, - -0.07568228756712969, - -0.9970103438906971, - 0.0, - 0.327178338376793, - 0.12210678990827917, - 0.18962678537935218, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "gear2", - "q": [ - -0.44017739813892465, - -0.8408044241896847, - 0.7812836869963553, - -2.6489709372327193, - 0.5117495865954292, - 1.9353651503056144, - 2.7550759355656957 - ] - }, - { - "_id": "65ce3392f045be1f60050f49", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - -0.19207017045850883, - -0.9745158408444792, - -0.11579582230989756, - 0.0, - -0.9619084037954584, - 0.21032850120116167, - -0.1745705908546879, - 0.0, - 0.19448071224163638, - 0.07785667044396119, - -0.9778116339220793, - 0.0, - 0.6668309353879216, - 0.1002329166290898, - 0.3839163655785476, - 1.0 - ], - "O_T_TCP": [ - -0.19207017045850883, - -0.9745158408444792, - -0.11579582230989756, - 0.0, - -0.9619084037954584, - 0.21032850120116167, - -0.1745705908546879, - 0.0, - 0.19448071224163638, - 0.07785667044396119, - -0.9778116339220793, - 0.0, - 0.6911410244181262, - 0.10996500043458494, - 0.2616899113382877, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "no_tool", - "q": [ - -0.449013091942916, - 0.2972115466353616, - 0.6671699467458255, - -1.6625072732841806, - -0.16246066116120655, - 2.102316742226921, - 2.8067302440068085 - ] - }, - { - "_id": "65ce3399f045be1f60050f4a", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.37121811244401426, - -0.911720063290249, - 0.1759101648249346, - 0.0, - -0.9285316648543275, - -0.3639617925463895, - 0.07308561986107809, - 0.0, - -0.00260909728174289, - -0.1904725312477871, - -0.9816890584352234, - 0.0, - 0.5482232455292271, - -0.09299434620256447, - 0.4961290551977721, - 1.0 - ], - "O_T_TCP": [ - 0.37121811244401426, - -0.911720063290249, - 0.1759101648249346, - 0.0, - -0.9285316648543275, - -0.3639617925463895, - 0.07308561986107809, - 0.0, - -0.00260909728174289, - -0.1904725312477871, - -0.9816890584352234, - 0.0, - 0.5478971083690092, - -0.11680341260853785, - 0.37341792289336917, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "gearbase", - "q": [ - -0.5710980864097402, - -0.030782228559778448, - 0.4631181385266316, - -1.681660839952254, - -0.1779492470121923, - 1.6707865634759267, - 1.8844083975966868 - ] - }, - { - "_id": "65cfa2c47f0205483b054d51", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.999127486937346, - 0.0020339066230885923, - 0.04148343008142645, - 0.0, - -0.0021570557684146043, - -0.9949015779177928, - 0.1007320419147068, - 0.0, - 0.041477608185149145, - -0.10073557343640634, - -0.994048264554432, - 0.0, - 0.3646566267993078, - -0.02254569907028298, - 0.40952082232522724, - 1.0 - ], - "O_T_TCP": [ - 0.9991272968724854, - 0.002032794737528359, - 0.04148806203046724, - 0.0, - -0.002158543289712084, - -0.9949017859475108, - 0.10072995537836842, - 0.0, - 0.041482108991960025, - -0.10073354124199559, - -0.9940482826816947, - 0.0, - 0.36465772773036376, - -0.02254865462649719, - 0.40952225264460873, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "test1", - "q": [ - -0.6494016965366246, - -0.753164953780274, - 0.5019667189581353, - -2.4719201877805506, - 0.2581255530648761, - 1.8439903862476348, - 0.4656479083630774 - ] - }, - { - "_id": "65cfa7905d75dfc96909caf1", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - -0.9935878908904953, - 0.08960520143219396, - -0.06880957675557518, - 0.0, - 0.09120083791165756, - 0.9956138318810523, - -0.02040225067357439, - 0.0, - 0.06668090241645759, - -0.026547431392950203, - -0.9974211202593233, - 0.0, - 0.5080013634448317, - 0.07744840851681808, - 0.310686758863888, - 1.0 - ], - "O_T_TCP": [ - -0.9935878908904953, - 0.08960520143219396, - -0.06880957675557518, - 0.0, - 0.09120083791165756, - 0.9956138318810523, - -0.02040225067357439, - 0.0, - 0.06668090241645759, - -0.026547431392950203, - -0.9974211202593233, - 0.0, - 0.5080013634448317, - 0.07744840851681808, - 0.310686758863888, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "test_object", - "q": [ - -0.6783759433123, - -0.16586847048823597, - 0.8175371052577723, - -2.261259556151273, - 0.1062502110335562, - 2.2074089546736957, - -2.190133549686696 - ] - }, - { - "_id": "65cfa8e660f0281a320a1ef1", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.9999648408880725, - 0.0002669773375134615, - 0.00714091114644764, - 0.0, - -4.124753500410915e-05, - -0.9990599232033038, - 0.04312789177385854, - 0.0, - 0.00714584989104384, - -0.0431275003184912, - -0.9990440208247148, - 0.0, - 0.5039719752053793, - 0.08282098192276433, - 0.4255527712058619, - 1.0 - ], - "O_T_TCP": [ - 0.9999648408880725, - 0.0002669773375134615, - 0.00714091114644764, - 0.0, - -4.124753500410915e-05, - -0.9990599232033038, - 0.04312789177385854, - 0.0, - 0.00714584989104384, - -0.0431275003184912, - -0.9990440208247148, - 0.0, - 0.5039719752053793, - 0.08282098192276433, - 0.4255527712058619, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "aaa", - "q": [ - -0.7009867876172065, - -0.24024575297265152, - 0.7995970885804996, - -1.9988235254230773, - 0.13235241008271123, - 1.8310430494149526, - 0.8356096653159838 - ] - }, - { - "_id": "65d8ab25365be5466b0fdc71", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.7169152584897764, - 0.5474729385396878, - 0.4316093611270089, - 0.0, - 0.5928049051061055, - -0.8045360532186379, - 0.03584453172698144, - 0.0, - 0.3668762665534954, - 0.23016708606695613, - -0.9013461696436144, - 0.0, - 0.357246341892693, - 0.3678538874170821, - 0.6325819898273004, - 1.0 - ], - "O_T_TCP": [ - 0.7169152584897764, - 0.5474729385396878, - 0.4316093611270089, - 0.0, - 0.5928049051061055, - -0.8045360532186379, - 0.03584453172698144, - 0.0, - 0.3668762665534954, - 0.23016708606695613, - -0.9013461696436144, - 0.0, - 0.357246341892693, - 0.3678538874170821, - 0.6325819898273004, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "yawgripper", - "q": [ - 0.6201635432107406, - -0.3442022244153497, - 0.1713300790283195, - -1.7913323832101213, - -0.045338612116045424, - 1.8881573694955265, - 0.9412377265920419 - ] - } -] \ No newline at end of file diff --git a/experiments/gearbox/orders.txt b/experiments/gearbox/orders.txt deleted file mode 100644 index 03b68eef..00000000 --- a/experiments/gearbox/orders.txt +++ /dev/null @@ -1,6 +0,0 @@ -change_tool("defaultgripper", "outwardgripper") -insert("outerring", "housingringhole") -insert("cone", "outerring") -change_tool("outwardgripper", "defaultgripper") -insert("outputshaftandgearstage2", "housinginternalgear") -insert("ringgear outputshaftandgearstage2) diff --git a/experiments/gearbox/plan_parser.py b/experiments/gearbox/plan_parser.py deleted file mode 100644 index 905108e9..00000000 --- a/experiments/gearbox/plan_parser.py +++ /dev/null @@ -1,103 +0,0 @@ -""" -not BB-written -""" - -from pprint import pprint -import re - - -class PlanStep: - def __init__( - self, - start_time: float, - estimated_duration: float, - action_name: str, - action_params: list[str], - raw_action: str, - ): - self.start_time = start_time - self.estimated_duration = estimated_duration - self.estimated_end_time = start_time + estimated_duration - self.action_name = action_name - self.action_params = action_params - self.raw_action = raw_action - - def __repr__(self): - return f"PlanStep({self.action_name}, {self.action_params}, raw is {self.raw_action})" - - -def parse_plan(plan_str: str) -> list[PlanStep]: - plan_steps = [] - # Split the input string into lines - lines = plan_str.strip().split("\n") - for line in lines: - matches = re.findall(r"\((.*?)\)", line) - # pprint(matches) - action_details = matches[0].split(" ") - action_name = action_details[0] - action_params = action_details[1:] - - # Create a PlanStep instance for each line and append it to the plan_steps list - plan_step = PlanStep(0, 0, action_name, action_params, raw_action=line) - plan_steps.append(plan_step) - - return plan_steps - - -# Example usage with your provided plan -plan_str = """0.0003: (change_gripper defaulgripper outwardgripper) [d:10.0000; c:1.0000]\n10.0005: (initialize outwardgripper) [d:10.0000; c:1.0000]\n20.0007: (insert outwardgripper outerring housing) [d:23.0000; c:1.0000]\n43.0010: (insert outwardgripper cone outerring) [d:19.0000; c:1.0000]\n62.0013: (change_gripper outwardgripper defaulgripper) [d:10.0000; c:1.0000]\n72.0015: (initialize defaulgripper) [d:1.0000; c:1.0000]\n73.0017: (insert defaulgripper outputshaftandgearstage2 cone) [d:40.0000; c:1.0000]\n113.0020: (change_gripper defaulgripper parallelgripper) [d:10.0000; c:1.0000]\n123.0023: (initialize parallelgripper) [d:10.0000; c:1.0000]\n133.0025: (insert parallelgripper ringgear outputshaftandgearstage2) [d:25.0000; c:1.0000]\n158.0027: (change_gripper parallelgripper inwardgripper) [d:25.0000; c:1.0000]\n183.0030: (initialize inwardgripper) [d:10.0000; c:1.0000]\n193.0033: (insert inwardgripper gearstage1 ringgear) [d:25.0000; c:1.0000]\n218.0035: (change_gripper inwardgripper clampgripper) [d:30.0000; c:1.0000]\n248.0038: (initialize clampgripper) [d:10.0000; c:1.0000]\n258.0040: (insert clampgripper designring ringgear) [d:13.0000; c:1.0000]\n271.0042: (change_gripper clampgripper defaulgripper) [d:15.0000; c:1.0000]\n286.0045: (initialize defaulgripper) [d:1.0000; c:1.0000]\n287.0048: (insert defaulgripper driveflange gearstage1) [d:25.0000; c:1.0000]\n312.0050: (insert defaulgripper inputpinion driveflange) [d:25.0000; c:1.0000]""" - -# """0.000: (change_gripper defaulgripper outwardgripper) [10.000] -# 10.001: (insert outwardgripper outerring housing) [23.000] -# 33.002: (insert outwardgripper cone outerring) [19.000] -# 52.003: (change_gripper outwardgripper defaulgripper) [10.000] -# 62.004: (insert defaulgripper outputshaftandgearstage2 cone) [40.000] -# 102.005: (insert defaulgripper ringgear outputshaftandgearstage2) [12.000] -# 114.006: (insert defaulgripper designring ringgear) [10.000] -# 124.007: (change_gripper defaulgripper parallelgripper) [10.000] -# 134.008: (insert parallelgripper gearstage1 ringgear) [25.000] -# 159.009: (change_gripper parallelgripper outwardgripper) [20.000] -# 179.010: (insert outwardgripper driveflange ringgear) [25.000] -# 204.011: (change_gripper outwardgripper clampgripper) [25.000] -# 229.012: (insert clampgripper inputpinion driveflange) [13.000]""" - -""" -object list -defaultgripper -outwardgripper -outerring housingringhole -cone outerring -defaultgripper -outputshaftandgearstage2 housinginternalgear -ringgear outputshaftandgearstage2 #! -designring ringgear #? -parallelgripper -gearstage1 ringgear #! -driveflange ringgear #? -clampgripper -inputpinion driveflange #! - -""" - -""" -change_gripper defaulgripper outwardgripper -insert outwardgripper outerring [housing_ringhole] -insert outwardgripper1 cone1 outerring1 #? -change_gripper outwardgripper1 defaulgripper1 -insert defaulgripper1 outputshaftandgearstage21 housing_internalgear1 -insert defaulgripper1 ringgear1 outputshaftandgearstage21 #? -insert defaulgripper1 designring1 ringgear1 #? -change_gripper defaulgripper1 parallelgripper1 -insert parallelgripper1 gearstage11 ringgear1 #? -change_gripper parallelgripper1 outwardgripper1 -insert outwardgripper1 driveflange1 gearstage11 #! -change_gripper outwardgripper1 clampgripper1 -insert clampgripper1 inputpinion1 driveflange1 #? -""" - -parsed_plan = parse_plan(plan_str) - -# Display the parsed plan steps -# for step in parsed_plan: -# pprint(step) -# pprint(parsed_plan) diff --git a/experiments/gearbox/results/result.txt b/experiments/gearbox/results/result.txt deleted file mode 100644 index 7146a598..00000000 --- a/experiments/gearbox/results/result.txt +++ /dev/null @@ -1,21 +0,0 @@ -0.0003: (initialize defaultgripper) [d:1.0000; c:1.0000], 2.950111394000487 -1.0005: (change_gripper defaultgripper outwardgripper) [d:10.0000; c:1.0000], 11.625277318999906 -11.0007: (initialize outwardgripper) [d:10.0000; c:1.0000], 3.4732212919998346 -21.0010: (insert outwardgripper outerring housing) [d:23.0000; c:1.0000], 23.451959666999755 -44.0013: (insert outwardgripper cone outerring) [d:19.0000; c:1.0000], 21.970032773999264 -63.0015: (change_gripper outwardgripper defaultgripper) [d:10.0000; c:1.0000], 11.119270913000037 -73.0017: (initialize defaultgripper) [d:1.0000; c:1.0000], 3.4915204770004493 -74.0020: (insert defaultgripper outputshaftandgearstage2 cone) [d:40.0000; c:1.0000], 70.98489729800076 -114.0023: (change_gripper defaultgripper parallelgripper) [d:10.0000; c:1.0000], 12.294889592999425 -124.0025: (initialize parallelgripper) [d:10.0000; c:1.0000], 4.809501204999833 -134.0027: (insert parallelgripper ringgear outputshaftandgearstage2) [d:25.0000; c:1.0000], 26.08333666999988 -159.0030: (change_gripper parallelgripper inwardgripper) [d:25.0000; c:1.0000], 23.19184306399984 -184.0033: (initialize inwardgripper) [d:10.0000; c:1.0000], 3.3246460319996913 -194.0035: (insert inwardgripper gearstage1 ringgear) [d:25.0000; c:1.0000], 47.39597329299977 -219.0038: (change_gripper inwardgripper clampgripper) [d:30.0000; c:1.0000], 23.952048887000274 -249.0040: (initialize clampgripper) [d:10.0000; c:1.0000], 4.936134204999689 -259.0042: (insert clampgripper designring ringgear) [d:13.0000; c:1.0000], 22.89744644700022 -272.0045: (change_gripper clampgripper defaultgripper) [d:15.0000; c:1.0000], 11.735504078000304 -287.0048: (initialize defaultgripper) [d:1.0000; c:1.0000], 4.661370410000018 -288.0050: (insert defaultgripper driveflange gearstage1) [d:25.0000; c:1.0000], 26.627017857000283 -313.0052: (insert defaultgripper inputpinion driveflange) [d:25.0000; c:1.0000], 32.44564747899949 diff --git a/experiments/gearbox/results/result001.txt b/experiments/gearbox/results/result001.txt deleted file mode 100644 index 047295dc..00000000 --- a/experiments/gearbox/results/result001.txt +++ /dev/null @@ -1,21 +0,0 @@ -0.0003: (initialize defaultgripper) [d:1.0000; c:1.0000], 4.808458777999476 -1.0005: (change_gripper defaultgripper outwardgripper) [d:10.0000; c:1.0000], 11.688009651000357 -11.0007: (initialize outwardgripper) [d:10.0000; c:1.0000], 3.4573092000000543 -21.0010: (insert outwardgripper outerring housing) [d:23.0000; c:1.0000], 22.860556831000395 -44.0013: (insert outwardgripper cone outerring) [d:19.0000; c:1.0000], 22.659663784999793 -63.0015: (change_gripper outwardgripper defaultgripper) [d:10.0000; c:1.0000], 11.377012515999922 -73.0017: (initialize defaultgripper) [d:1.0000; c:1.0000], 3.3935505210001793 -74.0020: (insert defaultgripper outputshaftandgearstage2 cone) [d:40.0000; c:1.0000], 71.41381259900027 -114.0023: (change_gripper defaultgripper parallelgripper) [d:10.0000; c:1.0000], 11.916867539000123 -124.0025: (initialize parallelgripper) [d:10.0000; c:1.0000], 4.814801288999661 -134.0027: (insert parallelgripper ringgear outputshaftandgearstage2) [d:25.0000; c:1.0000], 25.5688664850004 -159.0030: (change_gripper parallelgripper inwardgripper) [d:25.0000; c:1.0000], 23.267643426000177 -184.0033: (initialize inwardgripper) [d:10.0000; c:1.0000], 3.546634418999929 -194.0035: (insert inwardgripper gearstage1 ringgear) [d:25.0000; c:1.0000], 45.24948094299998 -219.0038: (change_gripper inwardgripper clampgripper) [d:30.0000; c:1.0000], 24.02283685799921 -249.0040: (initialize clampgripper) [d:10.0000; c:1.0000], 4.694307449000007 -259.0042: (insert clampgripper designring ringgear) [d:13.0000; c:1.0000], 23.04564223700072 -272.0045: (change_gripper clampgripper defaultgripper) [d:15.0000; c:1.0000], 11.74245431899999 -287.0048: (initialize defaultgripper) [d:1.0000; c:1.0000], 4.906379768000079 -288.0050: (insert defaultgripper driveflange gearstage1) [d:25.0000; c:1.0000], 26.625840525999593 -313.0052: (insert defaultgripper inputpinion driveflange) [d:25.0000; c:1.0000], 29.195681523999156 diff --git a/experiments/gearbox/robot_commands.py b/experiments/gearbox/robot_commands.py deleted file mode 100644 index 883f21e5..00000000 --- a/experiments/gearbox/robot_commands.py +++ /dev/null @@ -1,267 +0,0 @@ -import numpy as np -import os - -from kios_robot.mios_task_factory import MiosTaskFactory -from kios_robot.robot_command import RobotCommand -from kios_robot.robot_interface import RobotInterface -from kios_robot.data_types import Toolbox -from kios_scene.scene_factory import SceneFactory - -from kios_scene.mios_ltm_manipulator import LangTermMemoryManipulator - -ri = RobotInterface() -sf = SceneFactory() - -current_dir = os.path.dirname(os.path.realpath(__file__)) -scene_file = os.path.join(current_dir, "scene.json") - -with open(scene_file, "r") as f: - scene_json = f.read() - -scene = sf.create_scene_from_json(scene_json) -ri.setup_scene(scene) - -ltm_manipulator = LangTermMemoryManipulator() - - -# def teach_object(object: str): -# ri.proprioceptor.teach_object(object) -def backup_mios_environment(backup_name: str): - ltm_manipulator.backup_mios_environment(backup_name) - - -def clear_mios_environment(): - ltm_manipulator.clear_mios_environment() - - -def show_backups(): - ltm_manipulator.show_backups() - - -# ! BUG -def restore_to_mios_environment(backup_name: str): - ltm_manipulator.restore_to_mios_environment(backup_name) - - -def teach_object_TCP(object_name: str): - ri.proprioceptor.teach_object_TCP(object_name) - ri.proprioceptor.update_scene_object_from_mios(scene=scene, object_name=object_name) - - -def get_object(object: str): - ri.proprioceptor.get_object(object) - - -def set_tool(toolbox_name: str): - if "robot_command" in locals(): - print("robot command is set") - print(f"the robot command's tasks are {robot_command.task_list}") - else: - print("robot command is not set") - - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - shared_data=None, - task_scene=scene, - robot_interface=ri, - ) - print(f"the number of tasks in the task list is {len(robot_command.task_list)}") - robot_command.add_task(ri.mios_task_factory.generate_update_tool_call(toolbox_name)) - robot_command.execute_task_list_sync() - robot_command.clear_tasks() - - -def test_screw_in(object_name: str): - # need object "test" - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - shared_data=None, - task_scene=scene, - robot_interface=ri, - ) - - O_T_OB = scene.get_object(object_name).O_T_TCP - - print(O_T_OB) - - robot_command.add_task(ri.mios_task_factory.generate_screw_in_mp(O_T_OB=O_T_OB)) - - robot_command.execute_task_list_sync() - - -def pick_test(): - - parsed_action = { - "action_name": "pick", - "args": [None, None, "test_object"], - } - - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - shared_data=None, - task_scene=scene, - robot_interface=ri, - ) - - robot_command.add_mios_task(ri.mios_task_factory.generate_gripper_home_mp()) - - pick_up_tasks = ri.mios_task_factory.generate_pick_up_skill( - parsed_action=parsed_action - ) - for task in pick_up_tasks: - robot_command.add_task(task) - - robot_command.task_list.append(ri.mios_task_factory.generate_gripper_release_mp()) - - robot_command.execute_task_list_sync() - - -def move_gripper(width: float): - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - task_scene=scene, - shared_data=None, - robot_interface=ri, - ) - - robot_command.add_task(ri.mios_task_factory.generate_gripper_move_mp(width=width)) - - robot_command.execute_task_list_sync() - - -def home_gripper(): - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - task_scene=scene, - shared_data=None, - robot_interface=ri, - ) - - robot_command.add_task(ri.mios_task_factory.generate_gripper_home_mp()) - - robot_command.execute_task_list_sync() - - -def cartesian_move(object: str): - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - task_scene=scene, - shared_data=None, - robot_interface=ri, - ) - - robot_command.add_task(ri.mios_task_factory.generate_cartesian_move_mp(object)) - - robot_command.execute_task_list_sync() - - -def insert_test(object_name: str): - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - task_scene=scene, - shared_data=None, - robot_interface=ri, - ) - - insert_action = { - "action_name": "insert", - "args": [None, None, "gear3", object_name], - } - - robot_command.add_tasks(ri.mios_task_factory.generate_insert_skill(insert_action)) - - robot_command.execute_task_list_sync() - - -def drive_test(): - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - task_scene=scene, - shared_data=None, - robot_interface=ri, - ) - - drive_action = { - "action_name": "drive", - "args": [None, None, None, None], - } - - robot_command.add_tasks(ri.mios_task_factory.generate_drive_skill(drive_action)) - - robot_command.execute_task_list_sync() - - -def change_gripper(current_tool: str, new_tool: str): - - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - task_scene=scene, - shared_data=None, - robot_interface=ri, - ) - - unload_tool_parsed_action = { - "action_name": "unload_tool", - "args": ["left_hand", current_tool], - } - - robot_command.add_tasks( - ri.mios_task_factory.generate_unload_tool_skill(unload_tool_parsed_action) - ) - - load_tool_parsed_action = { - "action_name": "load_tool", - "args": ["left_hand", new_tool], - } - - robot_command.add_tasks( - ri.mios_task_factory.generate_load_tool_skill(load_tool_parsed_action) - ) - - robot_command.execute_task_list_sync() - - -def insert_task(part1: str, part2: str): - """ - part1: the location name of the part to be inserted - part2: the location name of the part (usually a feature name) to be inserted into - """ - # * first pickup, then insert - - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - shared_data=None, - task_scene=scene, - robot_interface=ri, - ) - - pick_up_parsed_action = { - "action_name": "pick", # ! maybe this should be "pick_up" - "args": [None, None, part1], - } - - robot_command.add_tasks( - ri.mios_task_factory.generate_pick_up_skill(pick_up_parsed_action) - ) - - insert_parsed_action = { - "action_name": "insert", - "args": [None, None, part1, part2], - } - - # ! warning: the location of part1 will be refreshed after the insertion! - robot_command.add_tasks( - ri.mios_task_factory.generate_insert_skill(insert_parsed_action) - ) - - robot_command.execute_task_list_sync() diff --git a/experiments/gearbox/runtime_script.py b/experiments/gearbox/runtime_script.py deleted file mode 100644 index 2ef4bba6..00000000 --- a/experiments/gearbox/runtime_script.py +++ /dev/null @@ -1,945 +0,0 @@ -import numpy as np -import os -import json -from spatialmath import * -from spatialmath.base import trnorm -from pprint import pprint - -from kios_robot.mios_task_factory import MiosTaskFactory -from kios_robot.robot_command import RobotCommand -from kios_robot.robot_interface import RobotInterface -from kios_robot.data_types import Toolbox -from kios_scene.scene_factory import SceneFactory - -from kios_scene.mios_ltm_manipulator import LangTermMemoryManipulator -from kios_utils.bblab_utils import execution_timer - -ri = RobotInterface() -sf = SceneFactory() - -current_dir = os.path.dirname(os.path.realpath(__file__)) -scene_file = os.path.join(current_dir, "scene.json") - -with open(scene_file, "r") as f: - scene_string = f.read() - scene_json = json.loads(scene_string) - -scene = sf.create_scene_from_json(scene_json) -ri.setup_scene(scene) - -ltm_manipulator = LangTermMemoryManipulator() - - -def show_scene(): - pprint(scene) - - -# def teach_object(object: str): -# ri.proprioceptor.teach_object(object) -def backup_mios_environment(backup_name: str): - ltm_manipulator.backup_mios_environment(backup_name) - - -def clear_mios_environment(): - ltm_manipulator.clear_mios_environment() - - -def show_backups(): - ltm_manipulator.show_backups() - - -def restore_to_mios_environment(backup_name: str): - ltm_manipulator.restore_to_mios_environment(backup_name) - # ! you need to update the memory of mios. - update_mios_memory_environment() - # ! you also need to update the scene object in robot_interface. - scene = sf.create_scene_from_json(scene_json) - ri.setup_scene(scene) - - -def teach_object_TCP(object_name: str): - ri.proprioceptor.teach_object_TCP(object_name) - ri.proprioceptor.update_scene_object_from_mios(scene=scene, object_name=object_name) - - -def align_object(object_name: str, **kwargs): - ri.proprioceptor.align_object(object_name, **kwargs) - ri.proprioceptor.update_scene_object_from_mios(scene=scene, object_name=object_name) - - -def modify_object_position_rel(object_name: str, **kwargs): - ri.proprioceptor.modify_object_position(object_name, **kwargs) - ri.proprioceptor.update_scene_object_from_mios(scene=scene, object_name=object_name) - - -def update_mios_memory_environment(): - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - shared_data=None, - task_scene=scene, - robot_interface=ri, - ) - robot_command.add_task( - ri.mios_task_factory.generate_update_mios_memory_environment_call() - ) - robot_command.execute_task_list_sync() - - -def get_object(object: str): - ri.proprioceptor.get_object(object) - - -def set_tool(toolbox_name: str): - if "robot_command" in locals(): - print("robot command is set") - print(f"the robot command's tasks are {robot_command.task_list}") - else: - print("robot command is not set") - - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - shared_data=None, - task_scene=scene, - robot_interface=ri, - ) - print(f"the number of tasks in the task list is {len(robot_command.task_list)}") - robot_command.add_task(ri.mios_task_factory.generate_update_tool_call(toolbox_name)) - robot_command.execute_task_list_sync() - robot_command.clear_tasks() - - -def test_cartesian_joint_move(): - # need object "test" - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - shared_data=None, - task_scene=scene, - robot_interface=ri, - ) - - O_T_TCP = scene.get_object("test").O_T_TCP - print(O_T_TCP) - print(type(O_T_TCP)) - print(O_T_TCP.shape) - - robot_command.add_task( - ri.mios_task_factory.generate_cartesian_move_mp(O_T_TCP=O_T_TCP) - ) - - # rotate in z axis by 90 degrees - # rotation_90 = SE3.Rz(np.deg2rad(90)) - - # rotate in x axis by 90 degrees - rotation_90 = SE3.Rx(np.deg2rad(90)) - - se_O_T_TCP = SE3(trnorm(O_T_TCP)) - result = se_O_T_TCP * rotation_90 - O_T_TCP = result.A - - robot_command.add_task( - ri.mios_task_factory.generate_cartesian_move_mp(O_T_TCP=O_T_TCP) - ) - - robot_command.execute_task_list_sync() - - -def test_screw_in(object_name: str): - # need object "test" - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - shared_data=None, - task_scene=scene, - robot_interface=ri, - ) - - O_T_OB = scene.get_object(object_name).O_T_TCP - - print(O_T_OB) - - robot_command.add_task(ri.mios_task_factory.generate_screw_in_mp(O_T_OB=O_T_OB)) - - robot_command.execute_task_list_sync() - - -def tool_test(): - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - shared_data=None, - task_scene=scene, - robot_interface=ri, - ) - - robot_command.add_mios_task( - ri.mios_task_factory.generate_cartesian_move_mp("test_location") - ) - robot_command.add_mios_task( - ri.mios_task_factory.generate_update_tool_call("parallel_box1") - ) - robot_command.add_mios_task( - ri.mios_task_factory.generate_cartesian_move_mp("test_location") - ) - - robot_command.execute_task_list_sync() - - -def load_tool_test(tool_name: str): - parsed_action = { - "action_name": "load_tool", - "args": ["left_hand", tool_name], - } - - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - task_scene=scene, - shared_data=None, - robot_interface=ri, - ) - - robot_command.add_tasks( - ri.mios_task_factory.generate_load_tool_skill(parsed_action) - ) - - robot_command.execute_task_list_sync() - - -def pick_test(object_name: str): - - parsed_action = { - "action_name": "pick", - "args": [None, None, object_name], - } - - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - shared_data=None, - task_scene=scene, - robot_interface=ri, - ) - - # robot_command.add_mios_task(ri.mios_task_factory.generate_gripper_home_mp()) - - robot_command.add_tasks(ri.mios_task_factory.generate_pick_up_skill(parsed_action)) - - robot_command.execute_task_list_sync() - - -def move_gripper(width: float): - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - task_scene=scene, - shared_data=None, - robot_interface=ri, - ) - - robot_command.add_task(ri.mios_task_factory.generate_gripper_move_mp(width=width)) - - robot_command.execute_task_list_sync() - - -def home_gripper(): - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - task_scene=scene, - shared_data=None, - robot_interface=ri, - ) - - robot_command.add_task(ri.mios_task_factory.generate_gripper_home_mp()) - - robot_command.execute_task_list_sync() - - -def cartesian_move(object: str): - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - task_scene=scene, - shared_data=None, - robot_interface=ri, - ) - - robot_command.add_task(ri.mios_task_factory.generate_cartesian_move_mp(object)) - - robot_command.execute_task_list_sync() - - -def joint_move(object: str): - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - task_scene=scene, - shared_data=None, - robot_interface=ri, - ) - - robot_command.add_task(ri.mios_task_factory.generate_joint_move_mp(object)) - - robot_command.execute_task_list_sync() - - -def insert_test(object_name: str): - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - task_scene=scene, - shared_data=None, - robot_interface=ri, - ) - - insert_action = { - "action_name": "insert", - "args": [None, None, "gear3", object_name], - } - - robot_command.add_tasks(ri.mios_task_factory.generate_insert_skill(insert_action)) - - robot_command.execute_task_list_sync() - - -def drive_test(): - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - task_scene=scene, - shared_data=None, - robot_interface=ri, - ) - - drive_action = { - "action_name": "drive", - "args": [None, None, None, None], - } - - robot_command.add_tasks(ri.mios_task_factory.generate_drive_skill(drive_action)) - - robot_command.execute_task_list_sync() - - -@execution_timer -def change_gripper(current_tool: str, new_tool: str): - - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - task_scene=scene, - shared_data=None, - robot_interface=ri, - ) - - if current_tool != "defaultgripper": - unload_tool_parsed_action = { - "action_name": "unload_tool", - "args": ["left_hand", current_tool], - } - - robot_command.add_tasks( - ri.mios_task_factory.generate_unload_tool_skill(unload_tool_parsed_action) - ) - - if new_tool != "defaultgripper": - - load_tool_parsed_action = { - "action_name": "load_tool", - "args": ["left_hand", new_tool], - } - - robot_command.add_tasks( - ri.mios_task_factory.generate_load_tool_skill(load_tool_parsed_action) - ) - - result = robot_command.execute_task_list_sync() - - if result == False: - pause = input("failed to change gripper") - - -@execution_timer -def initialize_gripper(): - - robot_command = RobotCommand( - # robot_address="127.0.0.1", - robot_address="10.157.174.230", - robot_port=12000, - shared_data=None, - task_scene=scene, - robot_interface=ri, - ) - - robot_command.add_task( - ri.mios_task_factory.generate_joint_move_mp("initialposition") - ) - - robot_command.execute_task_list_sync() - - -@execution_timer -def insert_task(part1: str, part2: str): - """ - part1: the location name of the part to be inserted - part2: the location name of the part (usually a feature name) to be inserted into - """ - # * first pickup, then insert - - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - shared_data=None, - task_scene=scene, - robot_interface=ri, - ) - - pick_up_parsed_action = { - "action_name": "pick", # ! maybe this should be "pick_up" - "args": [None, None, part1], - } - - robot_command.add_tasks( - ri.mios_task_factory.generate_pick_up_skill(pick_up_parsed_action) - ) - - insert_parsed_action = { - "action_name": "insert", - "args": [None, None, part1, part2], - } - - # ! warning: the location of part1 will be refreshed after the insertion! - robot_command.add_tasks( - ri.mios_task_factory.generate_insert_skill_mod(insert_parsed_action) - ) - - robot_command.execute_task_list_sync() - - -@execution_timer -def insert_outerring(): - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - shared_data=None, - task_scene=scene, - robot_interface=ri, - ) - - pick_up_parsed_action = { - "action_name": "pick", # ! maybe this should be "pick_up" - "args": [None, None, "outerring"], - } - - robot_command.add_tasks( - ri.mios_task_factory.generate_pick_up_skill(pick_up_parsed_action) - ) - - insert_parsed_action = { - "action_name": "insert", - "args": [None, None, "outerring", "housingringhole"], - } - - param = { - "search_a": [2, 2, 2, 1, 1, 0], - "search_f": [1, 1, 1, 1.5, 1.5, 0], - "F_ext_contact": [10.0, 2.0], - "f_push": [0, 0, 3, 0, 0, 0], - "K_x": [300, 300, 0, 500, 500, 800], - "env_X": [0.01, 0.01, 0.001, 0.05, 0.05, 0.05], - } - - # ! warning: the location of part1 will be refreshed after the insertion! - robot_command.add_tasks( - ri.mios_task_factory.generate_insert_skill_mod( - insert_parsed_action, param=param - ) - ) - - robot_command.execute_task_list_sync() - - -@execution_timer -def insert_cone(): - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - shared_data=None, - task_scene=scene, - robot_interface=ri, - ) - - pick_up_parsed_action = { - "action_name": "pick", # ! maybe this should be "pick_up" - "args": [None, None, "cone"], - } - - robot_command.add_tasks( - ri.mios_task_factory.generate_pick_up_skill(pick_up_parsed_action) - ) - - insert_parsed_action = { - "action_name": "insert", - "args": [None, None, "cone", "outerring"], - } - - param = { - "search_a": [2, 2, 1, 1, 1, 0], - "search_f": [1, 1, 1, 1.5, 1.5, 0], - "F_ext_contact": [10.0, 2.0], - "f_push": [0, 0, 5, 0, 0, 0], - "K_x": [500, 500, 0, 500, 500, 800], - "env_X": [0.01, 0.01, 0.001, 0.05, 0.05, 0.05], - } - - # ! warning: the location of part1 will be refreshed after the insertion! - robot_command.add_tasks( - ri.mios_task_factory.generate_insert_skill_mod( - insert_parsed_action, param=param - ) - ) - - robot_command.execute_task_list_sync() - - -@execution_timer -def insert_outputshaftandgearstage2(): - - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - shared_data=None, - task_scene=scene, - robot_interface=ri, - ) - - pick_up_parsed_action = { - "action_name": "pick", # ! maybe this should be "pick_up" - "args": [None, None, "outputshaftandgearstage2"], - } - - robot_command.add_tasks( - ri.mios_task_factory.generate_pick_up_skill(pick_up_parsed_action) - ) - - insert_parsed_action = { - "action_name": "insert", - "args": [None, None, "outputshaftandgearstage2", "housinginternalgear"], - } - - param = { - "release_width": 0.04, - "search_a": [10, 10, 2, 3, 3, 1], - "search_f": [1, 1, 1, 1.5, 1.5, 1], - "F_ext_contact": [13.0, 2.0], - "f_push": [0, 0, 7, 0, 0, 0], - "K_x": [500, 500, 0, 500, 500, 800], - } - - # ! warning: the location of part1 will be refreshed after the insertion! - robot_command.add_tasks( - ri.mios_task_factory.generate_insert_skill_mod( - parsed_action=insert_parsed_action, - param=param, - ) - ) - - result = robot_command.execute_task_list_sync() - if result == False: - pause = input("failed to insert outputshaftandgearstage2") - - move_gripper(0.08) - - change_gripper("defaultgripper", "inwardgripper") - - robot_command.clear_tasks() - - insert_parsed_action = { - "action_name": "insert", - "args": [ - None, - None, - "outputshaftandgearstage2", - "outputshaftandgearstage2push", - ], - } - - param = { - "search_a": [1, 1, 0, 2, 2, 40], - "search_f": [1, 1, 0, 1, 1, 0.25], - "F_ext_contact": [10.0, 2.0], - "f_push": [0, 0, 20, 0, 0, 0], - "K_x": [500, 500, 0, 800, 800, 70], - "D_x": [0.7, 0.7, 1.2, 0.7, 0.7, 1.2], - } - - # ! warning: the location of part1 will be refreshed after the insertion! - robot_command.add_tasks( - ri.mios_task_factory.generate_insert_skill_mod( - parsed_action=insert_parsed_action, - param=param, - ) - ) - - robot_command.execute_task_list_sync() - - change_gripper("inwardgripper", "defaultgripper") - - -def push_outputshaftandgearstage2(): - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - shared_data=None, - task_scene=scene, - robot_interface=ri, - ) - - insert_parsed_action = { - "action_name": "insert", - "args": [ - None, - None, - "outputshaftandgearstage2", - "outputshaftandgearstage2push", - ], - } - - param = { - "width": 0.035, - "search_a": [1, 1, 0, 2, 2, 40], - "search_f": [1, 1, 0, 1, 1, 0.25], - "F_ext_contact": [10.0, 2.0], - "f_push": [0, 0, 18, 0, 0, 0], - "K_x": [500, 500, 0, 800, 800, 70], - "D_x": [0.7, 0.7, 1.2, 0.7, 0.7, 1.2], - } - - # ! warning: the location of part1 will be refreshed after the insertion! - robot_command.add_tasks( - ri.mios_task_factory.generate_insert_skill_mod( - parsed_action=insert_parsed_action, - param=param, - ) - ) - - robot_command.execute_task_list_sync() - - -@execution_timer -def insert_ringgear(): - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - shared_data=None, - task_scene=scene, - robot_interface=ri, - ) - - robot_command.add_task( - ri.mios_task_factory.generate_gripper_grasp_mp(), - ) - - robot_command.add_task( - ri.mios_task_factory.generate_move_above_mp("ringgear"), - ) - # ! - # insert_parsed_action = { - # "action_name": "insert", - # "args": [ - # None, - # None, - # "ringgear", - # "ringgear", - # ], - # } - - # param = { - # "search_a": [8, 8, 0, 0, 0, 0], - # "search_f": [1, 1, 0, 0, 0, 0], - # "F_ext_contact": [13.0, 2.0], - # "f_push": [0, 0, 8, 0, 0, 0], - # "K_x": [300, 300, 0, 600, 600, 500], - # "D_x": [0.8, 0.8, 0.7, 0.7, 0.7, 0.7], - # # "env_X": [0.01, 0.01, 0.001, 0.05, 0.05, 0.05], - # } - - # # ! warning: the location of part1 will be refreshed after the insertion! - # robot_command.add_tasks( - # ri.mios_task_factory.generate_insert_skill_standalone_mod( - # parsed_action=insert_parsed_action, - # param=param, - # ) - # ) - - robot_command.add_task( - ri.mios_task_factory.generate_reach_mp("ringgear"), - ) - - robot_command.add_task( - ri.mios_task_factory.generate_gripper_release_mp(width=0.023, speed=0.01), - ) - - robot_command.add_task( - ri.mios_task_factory.generate_move_above_mp("ringgear"), - ) - - # * skip teach new pose for ringgear - - insert_parsed_action = { - "action_name": "insert", - "args": [ - None, - None, - "ringgear", - "housingringgearhole", - ], - } - - param = { - "search_a": [8, 8, 1, 2, 2, 0], - "search_f": [1, 1, 1, 1, 1, 0], - "F_ext_contact": [13.0, 2.0], - "f_push": [0, 0, 3, 0, 0, 0], - "K_x": [500, 500, 0, 300, 300, 500], - "D_x": [0.7, 0.7, 0, 0.7, 0.7, 0.7], - "env_X": [0.01, 0.01, 0.000, 0.05, 0.05, 0.05], - } - - # ! warning: the location of part1 will be refreshed after the insertion! - robot_command.add_tasks( - ri.mios_task_factory.generate_insert_skill_standalone_mod( - parsed_action=insert_parsed_action, - param=param, - ) - ) - - robot_command.add_task( - ri.mios_task_factory.generate_gripper_grasp_mp(), - ) - - robot_command.add_task( - ri.mios_task_factory.generate_move_above_mp("housingringgearhole"), - ) - - robot_command.add_task( - ri.mios_task_factory.generate_gripper_release_mp(width=0.042), - ) - - robot_command.execute_task_list_sync() - - robot_command.clear_tasks() - - -@execution_timer -def insert_designring(): - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - shared_data=None, - task_scene=scene, - robot_interface=ri, - ) - - pick_up_parsed_action = { - "action_name": "pick", # ! maybe this should be "pick_up" - "args": [None, None, "designring"], - } - - robot_command.add_tasks( - ri.mios_task_factory.generate_pick_up_skill(pick_up_parsed_action) - ) - - insert_parsed_action = { - "action_name": "insert", - "args": [None, None, "designring", "designringhole"], - } - - param = { - "search_a": [12, 12, 1, 2, 2, 0], - "search_f": [1, 1, 1, 1.5, 1.5, 0], - "F_ext_contact": [10.0, 2.0], - "f_push": [0, 0, 10, 0, 0, 0], - "K_x": [500, 500, 0, 500, 500, 800], - } - - # ! warning: the location of part1 will be refreshed after the insertion! - robot_command.add_tasks( - ri.mios_task_factory.generate_insert_skill_mod( - insert_parsed_action, param=param - ) - ) - - robot_command.execute_task_list_sync() - - -@execution_timer -def insert_gearstage1(): - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - shared_data=None, - task_scene=scene, - robot_interface=ri, - ) - - pick_up_parsed_action = { - "action_name": "pick", # ! maybe this should be "pick_up" - "args": [None, None, "gearstage1"], - } - - robot_command.add_tasks( - ri.mios_task_factory.generate_pick_up_skill(pick_up_parsed_action) - ) - - insert_parsed_action = { - "action_name": "insert", - "args": [None, None, "gearstage1", "gearstage1hole"], - } - - param = { - "release_width": 0.03, - "search_a": [1, 1, 2, 2, 2, 40], - "search_f": [1, 1, 1, 1, 1, 0.25], - "F_ext_contact": [10.0, 2.0], - "f_push": [0, 0, 25, 0, 0, 0], - "K_x": [500, 500, 0, 800, 800, 70], - "D_x": [0.7, 0.7, 0, 0.7, 0.7, 1.2], - } - - # ! warning: the location of part1 will be refreshed after the insertion! - robot_command.add_tasks( - ri.mios_task_factory.generate_insert_skill_mod( - parsed_action=insert_parsed_action, - param=param, - ) - ) - - result = robot_command.execute_task_list_sync() - if result == False: - pause = input("failed to insert gearstage1") - - robot_command.clear_tasks() - - robot_command.add_task( - ri.mios_task_factory.generate_gripper_move_mp(width=0.03), - ) - - insert_parsed_action = { - "action_name": "insert", - "args": [ - None, - None, - "gearstage1", - "gearstage1push", - ], - } - - param = { - "search_a": [1, 1, 3, 2, 2, 40], - "search_f": [1, 1, 1, 1, 1, 0.25], - "F_ext_contact": [10.0, 2.0], - "f_push": [0, 0, 26, 0, 0, 0], - "K_x": [500, 500, 0, 800, 800, 70], - "D_x": [0.7, 0.7, 1.2, 0.7, 0.7, 1.2], - } - - # ! warning: the location of part1 will be refreshed after the insertion! - robot_command.add_tasks( - ri.mios_task_factory.generate_insert_skill_mod( - parsed_action=insert_parsed_action, - param=param, - ) - ) - - robot_command.execute_task_list_sync() - - -@execution_timer -def insert_driveflange(): - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - shared_data=None, - task_scene=scene, - robot_interface=ri, - ) - - pick_up_parsed_action = { - "action_name": "pick", # ! maybe this should be "pick_up" - "args": [None, None, "driveflange"], - } - - robot_command.add_tasks( - ri.mios_task_factory.generate_pick_up_skill(pick_up_parsed_action) - ) - - insert_parsed_action = { - "action_name": "insert", - "args": [None, None, "driveflange", "driveflangehole"], - } - - param = { - "width": 0.08, - "search_a": [8, 8, 1, 2, 2, 0], - "search_f": [1, 1, 1, 1.5, 1.5, 0], - "F_ext_contact": [10.0, 2.0], - "f_push": [0, 0, 10, 0, 0, 0], - "K_x": [500, 500, 0, 500, 500, 800], - } - - # ! warning: the location of part1 will be refreshed after the insertion! - robot_command.add_tasks( - ri.mios_task_factory.generate_insert_skill_mod( - insert_parsed_action, param=param - ) - ) - - robot_command.execute_task_list_sync() - - -@execution_timer -def insert_inputpinion(): - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - shared_data=None, - task_scene=scene, - robot_interface=ri, - ) - - pick_up_parsed_action = { - "action_name": "pick", # ! maybe this should be "pick_up" - "args": [None, None, "inputpinion"], - } - - robot_command.add_tasks( - ri.mios_task_factory.generate_pick_up_skill(pick_up_parsed_action) - ) - - insert_parsed_action = { - "action_name": "insert", - "args": [None, None, "inputpinion", "inputpinionhole"], - } - - param = { - "search_a": [3, 3, 0, 2, 2, 20], - "search_f": [1.5, 1.5, 0.1, 1.5, 1.5, 0.25], - "F_ext_contact": [10.0, 2.0], - "f_push": [0, 0, 20, 0, 0, 0], - "K_x": [300, 300, 0, 500, 500, 100], - "D_x": [0.7, 0.7, 1.2, 0.7, 0.7, 1.2], - "env_X": [0.01, 0.01, -0.002, 0.05, 0.05, 0.05], - } - - # ! warning: the location of part1 will be refreshed after the insertion! - robot_command.add_tasks( - ri.mios_task_factory.generate_insert_skill_mod( - insert_parsed_action, param=param - ) - ) - - robot_command.execute_task_list_sync() diff --git a/experiments/gearbox/scene.json b/experiments/gearbox/scene.json deleted file mode 100644 index 05a897f7..00000000 --- a/experiments/gearbox/scene.json +++ /dev/null @@ -1,169 +0,0 @@ -{ - "reference_objects": - [ - { - "object_name": "test", - "source": "mios" - }, - { - "object_name": "inwardgripper", - "source": "mios" - }, - { - "object_name": "outwardgripper", - "source": "mios" - }, - { - "object_name": "parallelgripper", - "source": "mios" - }, - { - "object_name": "clampgripper", - "source": "mios" - }, - { - "object_name": "defaultgripper", - "source": "mios" - }, - { - "object_name": "initialposition", - "source": "mios" - }, - { - "object_name": "outerring", - "source": "mios" - }, - { - "object_name": "cone", - "source": "mios" - }, - { - "object_name": "housingringhole", - "source": "mios" - }, - { - "object_name": "outputshaftandgearstage2", - "source": "mios" - }, - { - "object_name": "housinginternalgear", - "source": "mios" - }, - { - "object_name": "outputshaftandgearstage2push", - "source": "mios" - }, - { - "object_name": "ringgear", - "source": "mios" - }, - { - "object_name": "housingringgearhole", - "source": "mios" - }, - { - "object_name": "designring", - "source": "mios" - }, - { - "object_name": "designringhole", - "source": "mios" - }, - { - "object_name": "gearstage1", - "source": "mios" - }, - { - "object_name": "gearstage1hole", - "source": "mios" - }, - { - "object_name": "gearstage1push", - "source": "mios" - }, - { - "object_name": "driveflange", - "source": "mios" - }, - { - "object_name": "driveflangehole", - "source": "mios" - }, - { - "object_name": "inputpinion", - "source": "mios" - }, - { - "object_name": "inputpinionhole", - "source": "mios" - } - ], - "tools": - [ - { - "tool_name": "parallelgripper", - "EE_T_TCP": [ - [1, 0, 0, 0.0], - [0, 1, 0, 0.0], - [0, 0, 1, 0.085], - [0, 0, 0, 1] - ], - "EE_finger_width_max": 0.042, - "EE_finger_width_min": 0.01, - "load_width": 0.042, - "unload_width": 0.08 - }, - { - "tool_name": "clampgripper", - "EE_T_TCP": [ - [1, 0, 0, 0.0], - [0, 1, 0, 0.0], - [0, 0, 1, 0.096], - [0, 0, 0, 1] - ], - "EE_finger_width_max": 0.042, - "EE_finger_width_min": 0.01, - "load_width": 0.042, - "unload_width": 0.08 - }, - { - "tool_name": "inwardgripper", - "EE_T_TCP": [ - [1, 0, 0, 0.0], - [0, 1, 0, 0.0], - [0, 0, 1, 0.125], - [0, 0, 0, 1] - ], - "EE_finger_width_max": 0.042, - "EE_finger_width_min": 0.01, - "load_width": 0.042, - "unload_width": 0.08 - }, - { - "tool_name": "outwardgripper", - "EE_T_TCP": [ - [1, 0, 0, 0.0], - [0, 1, 0, 0.0], - [0, 0, 1, 0.125], - [0, 0, 0, 1] - ], - "EE_finger_width_max": 0.042, - "EE_finger_width_min": 0.01, - "load_width": 0.042, - "unload_width": 0.08 - }, - { - "tool_name": "defaultgripper", - "EE_T_TCP": [ - [1, 0, 0, 0.0], - [0, 1, 0, 0.0], - [0, 0, 1, 0.03], - [0, 0, 0, 1] - ], - "EE_finger_width_max": 0.08, - "EE_finger_width_min": 0.01, - "load_width": 0.08, - "unload_width": 0.08 - } - ] -} diff --git a/experiments/gearset/behavior_tree.json b/experiments/gearset/behavior_tree.json deleted file mode 100644 index 18441edb..00000000 --- a/experiments/gearset/behavior_tree.json +++ /dev/null @@ -1,212 +0,0 @@ -{ - "name": "insert gear3 into shaft2 selector", - "identifier": 0, - "type_name": "selector", - "children": [ - { - "summary": "check if gear3 is inserted to shaft2", - "name": "check gear3 is_inserted_to shaft2", - "identifier": 1, - "type_name": "condition", - "conditions": [ - { - "object_name": "gear3", - "property_name": "is_inserted_to", - "property_value": "shaft2", - "status": true - } - ] - }, - { - "name": "insert gear3 into shaft2 sequence", - "identifier": 2, - "type_name": "sequence", - "children": [ - { - "name": "load_tool selector", - "identifier": 3, - "type_name": "selector", - "children": [ - { - "summary": "check if inward_claw is held by left hand", - "name": "check left_hand hold inward_claw", - "identifier": 4, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "inward_claw", - "status": true - } - ] - }, - { - "name": "load_tool sequence", - "identifier": 5, - "type_name": "sequence", - "children": [ - { - "summary": "check if inward_claw is equippable", - "name": "check inward_claw is_equippable", - "identifier": 6, - "type_name": "condition", - "conditions": [ - { - "object_name": "inward_claw", - "property_name": "is_equippable", - "property_value": null, - "status": true - } - ] - }, - { - "summary": "check if left hand is free", - "name": "check left_hand is_free", - "identifier": 7, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "is_free", - "property_value": null, - "status": true - } - ] - }, - { - "summary": "equip inward_claw to left hand", - "name": "load_tool(left_hand, inward_claw)", - "identifier": 8, - "type_name": "action", - "effects": [ - { - "object_name": "left_hand", - "property_name": "is_free", - "property_value": null, - "status": false - }, - { - "object_name": "inward_claw", - "property_name": "is_equippable", - "property_value": null, - "status": false - }, - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "inward_claw", - "status": true - } - ] - } - ] - } - ] - }, - { - "name": "pick_up selector", - "identifier": 9, - "type_name": "selector", - "children": [ - { - "summary": "check if gear3 is held by inward_claw", - "name": "check inward_claw hold gear3", - "identifier": 10, - "type_name": "condition", - "conditions": [ - { - "object_name": "inward_claw", - "property_name": "hold", - "property_value": "gear3", - "status": true - } - ] - }, - { - "name": "pick_up sequence", - "identifier": 11, - "type_name": "sequence", - "children": [ - { - "summary": "check if gear3 is free", - "name": "check gear3 is_free", - "identifier": 12, - "type_name": "condition", - "conditions": [ - { - "object_name": "gear3", - "property_name": "is_free", - "property_value": null, - "status": true - } - ] - }, - { - "summary": "pick up gear3 using inward_claw", - "name": "pick_up(left_hand, inward_claw, gear3)", - "identifier": 13, - "type_name": "action", - "effects": [ - { - "object_name": "inward_claw", - "property_name": "hold", - "property_value": "gear3", - "status": false - }, - { - "object_name": "inward_claw", - "property_name": "is_free", - "property_value": null, - "status": false - } - ] - } - ] - } - ] - }, - { - "summary": "check if gear3 can be inserted to shaft2", - "name": "check gear3 can_insert_to shaft2", - "identifier": 14, - "type_name": "condition", - "conditions": [ - { - "object_name": "gear3", - "property_name": "can_insert_to", - "property_value": "shaft2", - "status": true - } - ] - }, - { - "summary": "insert gear3 into shaft2", - "name": "insert(left_hand, inward_claw, gear3, shaft2)", - "identifier": 15, - "type_name": "action", - "effects": [ - { - "object_name": "inward_claw", - "property_name": "hold", - "property_value": "gear3", - "status": false - }, - { - "object_name": "inward_claw", - "property_name": "is_free", - "property_value": null, - "status": true - }, - { - "object_name": "gear3", - "property_name": "is_inserted_to", - "property_value": "shaft2", - "status": true - } - ] - } - ] - } - ] -} \ No newline at end of file diff --git a/experiments/gearset/gearset.domain b/experiments/gearset/gearset.domain deleted file mode 100644 index e69de29b..00000000 diff --git a/experiments/gearset/gearset.problem b/experiments/gearset/gearset.problem deleted file mode 100644 index e69de29b..00000000 diff --git a/experiments/gearset/gearset.py b/experiments/gearset/gearset.py deleted file mode 100644 index 36d684c5..00000000 --- a/experiments/gearset/gearset.py +++ /dev/null @@ -1,85 +0,0 @@ -import json -import os - -from kios_bt.bt_stewardship import BehaviorTreeStewardship -from kios_scene.scene_factory import SceneFactory -from kios_bt.bt_factory import BehaviorTreeFactory -from kios_robot.robot_interface import RobotInterface -from kios_world.world_interface import WorldInterface -from kios_utils.pybt_test import ( - generate_bt_stewardship, - render_dot_tree, - tick_loop_test, - tick_1000HZ_test, - tick_frequency_test, -) - - -def core_loop(): - ####################### dirs - current_dir = os.path.dirname(os.path.abspath(__file__)) - scene_json_file_path = os.path.join(current_dir, "gearset_scene.json") - bt_json_file_path = os.path.join(current_dir, "behavior_tree.json") - world_state_json_file_path = os.path.join(current_dir, "world_state.json") - - ####################### scene - with open(scene_json_file_path, "r") as file: - json_object = json.load(file) - - gearset_scene = SceneFactory().create_scene_from_json(json_object) - - ####################### world - world_interface = WorldInterface() - with open(world_state_json_file_path, "r") as file: - json_object = json.load(file) - world_interface.load_world_from_json(json_object) - - ####################### robot - robot_interface = RobotInterface( - robot_address="127.0.0.1", - robot_port=12000, - ) - robot_interface.setup_scene(gearset_scene) - - ####################### agent - # * use the generated plan instead - - ####################### bt - bt_factory = BehaviorTreeFactory( - world_interface=world_interface, - robot_interface=robot_interface, - ) - - ####################### * old code - # behavior_tree = None - # with open(bt_json_file_path, "r") as file: - # json_object = json.load(file) - # behavior_tree = bt_factory.from_json_to_bt(json_object) - - # bt_steward = generate_bt_stewardship(behavior_tree) - # render_dot_tree(bt_steward) - # # tick_loop_test(bt_steward) - # tick_frequency_test(bt_steward) - - # * new code - behavior_tree_stewardship = BehaviorTreeStewardship( - behaviortree_factory=bt_factory, - world_interface=world_interface, - robot_interface=robot_interface, - ) - - with open(bt_json_file_path, "r") as file: - json_object = json.load(file) - behavior_tree_stewardship.load_bt_json(json_object) - - behavior_tree_stewardship.generate_behavior_tree() - behavior_tree_stewardship.setup_behavior_tree() - behavior_tree_stewardship.render_dot_tree() - sim_result = behavior_tree_stewardship.simulate_behavior_tree() - print(sim_result.to_json()) - - # ! world state has problem. gear3 is not free! - - -if __name__ == "__main__": - core_loop() diff --git a/experiments/gearset/gearset_scene.json b/experiments/gearset/gearset_scene.json deleted file mode 100644 index c8171463..00000000 --- a/experiments/gearset/gearset_scene.json +++ /dev/null @@ -1,99 +0,0 @@ -{ - "reference_objects": - [ - { - "object_name": "test_box1", - "source": "pre-defined", - "O_T_TCP": [ - [1, 0, 0, 0.0], - [0, 1, 0, 0.0], - [0, 0, 1, -0.1], - [0, 0, 0, 1] - ] - }, - { - "object_name": "TestObject1", - "source": "mios" - }, - { - "object_name": "inward_claw", - "source": "mios" - }, - { - "object_name": "gear3", - "source": "mios" - }, - { - "object_name": "shaft2", - "source": "mios" - } - ], - "tools": - [ - { - "tool_name": "parallel_box1", - "EE_T_TCP": [ - [1, 0, 0, 0.0], - [0, 1, 0, 0.0], - [0, 0, 1, 0.1], - [0, 0, 0, 1] - ], - "EE_finger_width_max": 0.042, - "EE_finger_width_min": 0.01, - "load_width": 0.042, - "unload_width": 0.08 - }, - { - "tool_name": "parallel_box2", - "EE_T_TCP": [ - [1, 0, 0, 0.0], - [0, 1, 0, 0.0], - [0, 0, 1, 0.1], - [0, 0, 0, 1] - ], - "EE_finger_width_max": 0.042, - "EE_finger_width_min": 0.01, - "load_width": 0.042, - "unload_width": 0.08 - }, - { - "tool_name": "inward_claw", - "EE_T_TCP": [ - [1, 0, 0, 0.0], - [0, 1, 0, 0.0], - [0, 0, 1, 0.1], - [0, 0, 0, 1] - ], - "EE_finger_width_max": 0.042, - "EE_finger_width_min": 0.01, - "load_width": 0.042, - "unload_width": 0.08 - }, - { - "tool_name": "outward_claw", - "EE_T_TCP": [ - [1, 0, 0, 0.0], - [0, 1, 0, 0.0], - [0, 0, 1, 0.1], - [0, 0, 0, 1] - ], - "EE_finger_width_max": 0.042, - "EE_finger_width_min": 0.01, - "load_width": 0.042, - "unload_width": 0.08 - }, - { - "tool_name": "no_tool", - "EE_T_TCP": [ - [1, 0, 0, 0.0], - [0, 1, 0, 0.0], - [0, 0, 1, 0.0], - [0, 0, 0, 1] - ], - "EE_finger_width_max": 0.08, - "EE_finger_width_min": 0.01, - "load_width": 0.08, - "unload_width": 0.08 - } - ] -} diff --git a/experiments/gearset/world_state.json b/experiments/gearset/world_state.json deleted file mode 100644 index 7866969d..00000000 --- a/experiments/gearset/world_state.json +++ /dev/null @@ -1,14 +0,0 @@ -{ - "objects": [ - {"name": "gear3", "properties": []}, - {"name": "shaft2", "properties": []}, - {"name": "inward_claw", "properties": ["is_equippable", "is_free"]}, - {"name": "left_hand", "properties": ["is_free"]} - ], - "constraints": [ - {"source": "gear3", "name": "can_insert_to", "target": "shaft2"}, - {"source": "inward_claw", "name": "can_manipulate", "target": "gear3"} - ], - "relations": [] -} - diff --git a/experiments/gearset1/baseline.py b/experiments/gearset1/baseline.py deleted file mode 100644 index b5c8c6cc..00000000 --- a/experiments/gearset1/baseline.py +++ /dev/null @@ -1,137 +0,0 @@ -import json -import os -import re -from pprint import pprint -from typing import List, Tuple, Annotated, TypedDict - -from kios_bt.bt_stewardship import BehaviorTreeStewardship -from kios_scene.scene_factory import SceneFactory -from kios_bt.bt_factory import BehaviorTreeFactory -from kios_robot.robot_interface import RobotInterface -from kios_world.world_interface import WorldInterface - -from dotenv import load_dotenv - -load_dotenv() - -from kios_utils.pybt_test import generate_bt_stewardship, render_dot_tree - - -def render_bt(bt_json: dict): - test_class = BehaviorTreeFactory() - bt = test_class.from_json_to_simple_bt(bt_json) - # bt = test_class.from_json_to_tree_root(bt_json) - bt_stewardship = generate_bt_stewardship(bt) - # bt_stewardship.setup(timeout=15) - render_dot_tree(bt_stewardship) - - -####################### dirs -current_dir = os.path.dirname(os.path.abspath(__file__)) -scene_path = os.path.join(current_dir, "scene.json") -# bt_json_file_path = os.path.join(current_dir, "behavior_tree.json") -world_state_path = os.path.join(current_dir, "world_state.json") -problem_path = os.path.join(current_dir, "gearset.problem") -domain_knowledge_path = os.path.join(current_dir, "domain_knowledge.txt") - -####################### problem -with open(problem_path, "r") as file: - problem = file.read() - -####################### scene -with open(scene_path, "r") as file: - scene_json_object = json.load(file) - -scene = SceneFactory().create_scene_from_json(scene_json_object) - -####################### world -world_interface = WorldInterface() -with open(world_state_path, "r") as file: - world_state_json = json.load(file) - world_interface.load_world_from_json(world_state_json) - -pprint(world_state_json) - - -####################### robot -robot_interface = RobotInterface( - robot_address="127.0.0.1", - robot_port=12000, -) -robot_interface.setup_scene(scene) - -####################### bt_factory -bt_factory = BehaviorTreeFactory( - world_interface=world_interface, - robot_interface=robot_interface, -) - -####################### behavior_tree_stewardship -behavior_tree_stewardship = BehaviorTreeStewardship( - behaviortree_factory=bt_factory, - world_interface=world_interface, - robot_interface=robot_interface, -) - - -def baseline_run(tree_root: dict, world_state: dict): - - sk_json = tree_root - initial_world_state = world_state - - from kios_plan.dynamic_planning import gearset_ut_dict - - # * first sim run - solultion = behavior_tree_stewardship.sk_baseline( - initial_world_state, - sk_json, - gearset_ut_dict, - ) - record = { - "target": sk_json["name"], - "initial_world_state": initial_world_state, - "result": solultion, - } - file_dir = os.path.join(current_dir, "baseline_result.jsonl") - with open(file_dir, "a") as file: - file.write(json.dumps(record) + "\n") - - -def baseline_plan(): - sk = { - # "summary": "the target is to insert the shaft1 into the gearbase_hole1", - # "name": "target: is_inserted_to(shaft1, gearbase_hole1)", - # "summary": "the target is to insert the shaft3 into the gearbase_hole3", - # "name": "target: is_inserted_to(shaft3, gearbase_hole3)", - # "summary": "the target is to insert the gear2 into the shaft2", - # "name": "target: is_inserted_to(gear2, shaft2)", - # "summary": "the target is to insert gear3 into shaft3", - # "name": "target: is_inserted_to(gear3, shaft3)", - "summary": "the target is to insert gear1 to shaft1", - "name": "target: is_inserted_to(gear1, shaft1)", - } - baseline_run(sk, world_state_json) - - -def test_result(problem_id: int): - """ - test the plan from the baseline_result.jsonl - """ - file_dir = os.path.join(current_dir, "baseline_result.jsonl") - with open(file_dir, "r") as file: - results = file.read() - data = [json.loads(line) for line in results.splitlines()] - problem = data[problem_id] - initial_world_state = problem["initial_world_state"] - sk = problem["result"] - result, node = behavior_tree_stewardship.sk_sim_run( - world_state=initial_world_state, skeleton_json=sk - ) - pprint(result.result) - pprint(result.summary) - - -if __name__ == "__main__": - pass - # baseline_plan() - test_result(15) diff --git a/experiments/gearset1/baseline_result.jsonl b/experiments/gearset1/baseline_result.jsonl deleted file mode 100644 index c4d9d3af..00000000 --- a/experiments/gearset1/baseline_result.jsonl +++ /dev/null @@ -1,17 +0,0 @@ -{"target": "target: is_inserted_to(shaft1, gearbase_hole1)", "initial_world_state": {"objects": [{"name": "clampgripper", "properties": ["is_empty"]}, {"name": "parallelgripper", "properties": ["is_empty"]}, {"name": "inwardgripper", "properties": ["is_empty"]}, {"name": "outwardgripper", "properties": []}, {"name": "defaultgripper", "properties": ["is_empty"]}, {"name": "gear1", "properties": []}, {"name": "gear2", "properties": []}, {"name": "gear3", "properties": []}, {"name": "shaft1", "properties": []}, {"name": "shaft2", "properties": []}, {"name": "shaft3", "properties": []}, {"name": "gearbase", "properties": []}, {"name": "gearbase_hole1", "properties": []}, {"name": "gearbase_hole3", "properties": []}, {"name": "left_hand", "properties": []}], "constraints": [{"source": "gear3", "name": "can_insert_to", "target": "shaft3"}, {"source": "shaft1", "name": "can_insert_to", "target": "gearbase_hole1"}, {"source": "parallelgripper", "name": "can_manipulate", "target": "gear1"}, {"source": "gear2", "name": "can_insert_to", "target": "shaft2"}, {"source": "outwardgripper", "name": "can_manipulate", "target": "gear3"}, {"source": "gear1", "name": "can_insert_to", "target": "shaft1"}, {"source": "defaultgripper", "name": "can_manipulate", "target": "shaft3"}, {"source": "outwardgripper", "name": "can_manipulate", "target": "gear2"}, {"source": "shaft3", "name": "can_insert_to", "target": "gearbase_hole3"}, {"source": "clampgripper", "name": "can_manipulate", "target": "shaft1"}], "relations": [{"source": "outwardgripper", "name": "hold", "target": "gear2"}, {"source": "left_hand", "name": "hold", "target": "outwardgripper"}]}, "result": {"summary": "selector to insert the shaft1 into the gearbase_hole1 with the clampgripper in the left_hand", "name": "selector: insert(left_hand, clampgripper, shaft1, gearbase_hole1)", "children": [{"summary": "the target is to make the shaft1 be inserted into the gearbase_hole1", "name": "target: is_inserted_to(shaft1, gearbase_hole1)"}, {"summary": "sequence to insert the shaft1 into the gearbase_hole1 with the clampgripper in the left_hand", "name": "sequence: insert(left_hand, clampgripper, shaft1, gearbase_hole1)", "children": [{"summary": "selector to change tool in left_hand from outwardgripper to clampgripper", "name": "selector: change_tool(left_hand, outwardgripper, clampgripper)", "children": [{"summary": "the target is to make left_hand hold clampgripper", "name": "target: hold(left_hand, clampgripper)"}, {"summary": "sequence to change tool in left_hand from outwardgripper to clampgripper", "name": "sequence: change_tool(left_hand, outwardgripper, clampgripper)", "children": [{"summary": "a precondition is left_hand is holding outwardgripper", "name": "precondition: hold(left_hand, outwardgripper)"}, {"summary": "selector to put down the gear2 held by the outwardgripper in the left_hand", "name": "selector: put_down(left_hand, outwardgripper, gear2)", "children": [{"summary": "the target is to make the outwardgripper empty", "name": "target: is_empty(outwardgripper)"}, {"summary": "sequence to put down the gear2 held by the outwardgripper in the left_hand", "name": "sequence: put_down(left_hand, outwardgripper, gear2)", "children": [{"summary": "a precondition is the left_hand is holding outwardgripper", "name": "precondition: hold(left_hand, outwardgripper)"}, {"summary": "a precondition is the outwardgripper is holding gear2", "name": "precondition: hold(outwardgripper, gear2)"}, {"summary": "the action to put down the gear2 held by the outwardgripper in the left_hand", "name": "action: put_down(left_hand, outwardgripper, gear2)"}]}]}, {"summary": "the action to change tool in left_hand from outwardgripper to clampgripper", "name": "action: change_tool(left_hand, outwardgripper, clampgripper)"}]}]}, {"summary": "selector to pick up the shaft1 with the clampgripper in the left_hand", "name": "selector: pick_up(left_hand, clampgripper, shaft1)", "children": [{"summary": "the target is to make the clampgripper hold the shaft1", "name": "target: hold(clampgripper, shaft1)"}, {"summary": "sequence to pick up the shaft1 with the clampgripper in the left_hand", "name": "sequence: pick_up(left_hand, clampgripper, shaft1)", "children": [{"summary": "a precondition is the clampgripper is empty", "name": "precondition: is_empty(clampgripper)"}, {"summary": "a precondition is the left_hand is holding clampgripper", "name": "precondition: hold(left_hand, clampgripper)"}, {"summary": "the action to pick up the shaft1 with the clampgripper in the left_hand", "name": "action: pick_up(left_hand, clampgripper, shaft1)"}]}]}, {"summary": "the action to insert the shaft1 into the gearbase_hole1 with the clampgripper in the left_hand", "name": "action: insert(left_hand, clampgripper, shaft1, gearbase_hole1)"}]}]}} -{"target": "target: is_inserted_to(shaft1, gearbase_hole1)", "initial_world_state": {"objects": [{"name": "clampgripper", "properties": ["is_empty"]}, {"name": "parallelgripper", "properties": ["is_empty"]}, {"name": "inwardgripper", "properties": ["is_empty"]}, {"name": "outwardgripper", "properties": ["is_empty"]}, {"name": "defaultgripper", "properties": ["is_empty"]}, {"name": "gear1", "properties": []}, {"name": "gear2", "properties": []}, {"name": "gear3", "properties": []}, {"name": "shaft1", "properties": []}, {"name": "shaft2", "properties": []}, {"name": "shaft3", "properties": []}, {"name": "gearbase", "properties": []}, {"name": "gearbase_hole1", "properties": []}, {"name": "gearbase_hole3", "properties": []}, {"name": "left_hand", "properties": []}], "constraints": [{"source": "defaultgripper", "name": "can_manipulate", "target": "shaft3"}, {"source": "gear2", "name": "can_insert_to", "target": "shaft2"}, {"source": "shaft1", "name": "can_insert_to", "target": "gearbase_hole1"}, {"source": "clampgripper", "name": "can_manipulate", "target": "shaft1"}, {"source": "parallelgripper", "name": "can_manipulate", "target": "gear1"}, {"source": "outwardgripper", "name": "can_manipulate", "target": "gear3"}, {"source": "gear1", "name": "can_insert_to", "target": "shaft1"}, {"source": "shaft3", "name": "can_insert_to", "target": "gearbase_hole3"}, {"source": "outwardgripper", "name": "can_manipulate", "target": "gear2"}, {"source": "gear3", "name": "can_insert_to", "target": "shaft3"}], "relations": [{"source": "left_hand", "name": "hold", "target": "outwardgripper"}]}, "result": {"summary": "selector to insert the shaft1 into the gearbase_hole1 with the clampgripper in the left_hand", "name": "selector: insert(left_hand, clampgripper, shaft1, gearbase_hole1)", "children": [{"summary": "the target is to make the shaft1 be inserted into the gearbase_hole1", "name": "target: is_inserted_to(shaft1, gearbase_hole1)"}, {"summary": "sequence to insert the shaft1 into the gearbase_hole1 with the clampgripper in the left_hand", "name": "sequence: insert(left_hand, clampgripper, shaft1, gearbase_hole1)", "children": [{"summary": "selector to change tool in left_hand from outwardgripper to clampgripper", "name": "selector: change_tool(left_hand, outwardgripper, clampgripper)", "children": [{"summary": "the target is to make left_hand hold clampgripper", "name": "target: hold(left_hand, clampgripper)"}, {"summary": "sequence to change tool in left_hand from outwardgripper to clampgripper", "name": "sequence: change_tool(left_hand, outwardgripper, clampgripper)", "children": [{"summary": "a precondition is left_hand is holding outwardgripper", "name": "precondition: hold(left_hand, outwardgripper)"}, {"summary": "a precondition is outwardgripper is empty", "name": "precondition: is_empty(outwardgripper)"}, {"summary": "the action to change tool in left_hand from outwardgripper to clampgripper", "name": "action: change_tool(left_hand, outwardgripper, clampgripper)"}]}]}, {"summary": "selector to pick up the shaft1 with the clampgripper in the left_hand", "name": "selector: pick_up(left_hand, clampgripper, shaft1)", "children": [{"summary": "the target is to make the clampgripper hold the shaft1", "name": "target: hold(clampgripper, shaft1)"}, {"summary": "sequence to pick up the shaft1 with the clampgripper in the left_hand", "name": "sequence: pick_up(left_hand, clampgripper, shaft1)", "children": [{"summary": "a precondition is the clampgripper is empty", "name": "precondition: is_empty(clampgripper)"}, {"summary": "a precondition is the left_hand is holding clampgripper", "name": "precondition: hold(left_hand, clampgripper)"}, {"summary": "the action to pick up the shaft1 with the clampgripper in the left_hand", "name": "action: pick_up(left_hand, clampgripper, shaft1)"}]}]}, {"summary": "the action to insert the shaft1 into the gearbase_hole1 with the clampgripper in the left_hand", "name": "action: insert(left_hand, clampgripper, shaft1, gearbase_hole1)"}]}]}} -{"target": "target: is_inserted_to(gear2, shaft2)", "initial_world_state": {"objects": [{"name": "clampgripper", "properties": ["is_empty"]}, {"name": "parallelgripper", "properties": ["is_empty"]}, {"name": "inwardgripper", "properties": ["is_empty"]}, {"name": "outwardgripper", "properties": ["is_empty"]}, {"name": "defaultgripper", "properties": ["is_empty"]}, {"name": "gear1", "properties": []}, {"name": "gear2", "properties": []}, {"name": "gear3", "properties": []}, {"name": "shaft1", "properties": []}, {"name": "shaft2", "properties": []}, {"name": "shaft3", "properties": []}, {"name": "gearbase", "properties": []}, {"name": "gearbase_hole1", "properties": []}, {"name": "gearbase_hole3", "properties": []}, {"name": "left_hand", "properties": []}], "constraints": [{"source": "shaft1", "name": "can_insert_to", "target": "gearbase_hole1"}, {"source": "clampgripper", "name": "can_manipulate", "target": "shaft1"}, {"source": "outwardgripper", "name": "can_manipulate", "target": "gear3"}, {"source": "gear3", "name": "can_insert_to", "target": "shaft3"}, {"source": "defaultgripper", "name": "can_manipulate", "target": "shaft3"}, {"source": "gear2", "name": "can_insert_to", "target": "shaft2"}, {"source": "shaft3", "name": "can_insert_to", "target": "gearbase_hole3"}, {"source": "gear1", "name": "can_insert_to", "target": "shaft1"}, {"source": "outwardgripper", "name": "can_manipulate", "target": "gear2"}, {"source": "parallelgripper", "name": "can_manipulate", "target": "gear1"}], "relations": [{"source": "left_hand", "name": "hold", "target": "outwardgripper"}]}, "result": {"summary": "selector to insert the gear2 into the shaft2 with the outwardgripper in the left_hand", "name": "selector: insert(left_hand, outwardgripper, gear2, shaft2)", "children": [{"summary": "the target is to make the gear2 be inserted into the shaft2", "name": "target: is_inserted_to(gear2, shaft2)"}, {"summary": "sequence to insert the gear2 into the shaft2 with the outwardgripper in the left_hand", "name": "sequence: insert(left_hand, outwardgripper, gear2, shaft2)", "children": [{"summary": "a precondition is the left_hand is holding outwardgripper", "name": "precondition: hold(left_hand, outwardgripper)"}, {"summary": "selector to pick up the gear2 with the outwardgripper in the left_hand", "name": "selector: pick_up(left_hand, outwardgripper, gear2)", "children": [{"summary": "the target is to make the outwardgripper hold the gear2", "name": "target: hold(outwardgripper, gear2)"}, {"summary": "sequence to pick up the gear2 with the outwardgripper in the left_hand", "name": "sequence: pick_up(left_hand, outwardgripper, gear2)", "children": [{"summary": "a precondition is the outwardgripper is empty", "name": "precondition: is_empty(outwardgripper)"}, {"summary": "a precondition is the left_hand is holding outwardgripper", "name": "precondition: hold(left_hand, outwardgripper)"}, {"summary": "the action to pick up the gear2 with the outwardgripper in the left_hand", "name": "action: pick_up(left_hand, outwardgripper, gear2)"}]}]}, {"summary": "the action to insert the gear2 into the shaft2 with the outwardgripper in the left_hand", "name": "action: insert(left_hand, outwardgripper, gear2, shaft2)"}]}]}} -{"target": "target: is_inserted_to(gear2, shaft2)", "initial_world_state": {"objects": [{"name": "clampgripper", "properties": []}, {"name": "parallelgripper", "properties": ["is_empty"]}, {"name": "inwardgripper", "properties": ["is_empty"]}, {"name": "outwardgripper", "properties": ["is_empty"]}, {"name": "defaultgripper", "properties": ["is_empty"]}, {"name": "gear1", "properties": []}, {"name": "gear2", "properties": []}, {"name": "gear3", "properties": []}, {"name": "shaft1", "properties": []}, {"name": "shaft2", "properties": []}, {"name": "shaft3", "properties": []}, {"name": "gearbase", "properties": []}, {"name": "gearbase_hole1", "properties": []}, {"name": "gearbase_hole3", "properties": []}, {"name": "left_hand", "properties": []}], "constraints": [{"source": "gear1", "name": "can_insert_to", "target": "shaft1"}, {"source": "defaultgripper", "name": "can_manipulate", "target": "shaft3"}, {"source": "outwardgripper", "name": "can_manipulate", "target": "gear2"}, {"source": "shaft1", "name": "can_insert_to", "target": "gearbase_hole1"}, {"source": "gear2", "name": "can_insert_to", "target": "shaft2"}, {"source": "gear3", "name": "can_insert_to", "target": "shaft3"}, {"source": "shaft3", "name": "can_insert_to", "target": "gearbase_hole3"}, {"source": "outwardgripper", "name": "can_manipulate", "target": "gear3"}, {"source": "parallelgripper", "name": "can_manipulate", "target": "gear1"}, {"source": "clampgripper", "name": "can_manipulate", "target": "shaft1"}], "relations": [{"source": "clampgripper", "name": "hold", "target": "shaft1"}, {"source": "left_hand", "name": "hold", "target": "clampgripper"}]}, "result": {"summary": "selector to insert the gear2 into the shaft2 with the outwardgripper in the left_hand", "name": "selector: insert(left_hand, outwardgripper, gear2, shaft2)", "children": [{"summary": "the target is to make the gear2 be inserted into the shaft2", "name": "target: is_inserted_to(gear2, shaft2)"}, {"summary": "sequence to insert the gear2 into the shaft2 with the outwardgripper in the left_hand", "name": "sequence: insert(left_hand, outwardgripper, gear2, shaft2)", "children": [{"summary": "selector to change tool in left_hand from clampgripper to outwardgripper", "name": "selector: change_tool(left_hand, clampgripper, outwardgripper)", "children": [{"summary": "the target is to make left_hand hold outwardgripper", "name": "target: hold(left_hand, outwardgripper)"}, {"summary": "sequence to change tool in left_hand from clampgripper to outwardgripper", "name": "sequence: change_tool(left_hand, clampgripper, outwardgripper)", "children": [{"summary": "a precondition is left_hand is holding clampgripper", "name": "precondition: hold(left_hand, clampgripper)"}, {"summary": "selector to put down the shaft1 held by the clampgripper in the left_hand", "name": "selector: put_down(left_hand, clampgripper, shaft1)", "children": [{"summary": "the target is to make the clampgripper empty", "name": "target: is_empty(clampgripper)"}, {"summary": "sequence to put down the shaft1 held by the clampgripper in the left_hand", "name": "sequence: put_down(left_hand, clampgripper, shaft1)", "children": [{"summary": "a precondition is the left_hand is holding clampgripper", "name": "precondition: hold(left_hand, clampgripper)"}, {"summary": "a precondition is the clampgripper is holding shaft1", "name": "precondition: hold(clampgripper, shaft1)"}, {"summary": "the action to put down the shaft1 held by the clampgripper in the left_hand", "name": "action: put_down(left_hand, clampgripper, shaft1)"}]}]}, {"summary": "the action to change tool in left_hand from clampgripper to outwardgripper", "name": "action: change_tool(left_hand, clampgripper, outwardgripper)"}]}]}, {"summary": "selector to pick up the gear2 with the outwardgripper in the left_hand", "name": "selector: pick_up(left_hand, outwardgripper, gear2)", "children": [{"summary": "the target is to make the outwardgripper hold the gear2", "name": "target: hold(outwardgripper, gear2)"}, {"summary": "sequence to pick up the gear2 with the outwardgripper in the left_hand", "name": "sequence: pick_up(left_hand, outwardgripper, gear2)", "children": [{"summary": "a precondition is the outwardgripper is empty", "name": "precondition: is_empty(outwardgripper)"}, {"summary": "a precondition is the left_hand is holding outwardgripper", "name": "precondition: hold(left_hand, outwardgripper)"}, {"summary": "the action to pick up the gear2 with the outwardgripper in the left_hand", "name": "action: pick_up(left_hand, outwardgripper, gear2)"}]}]}, {"summary": "the action to insert the gear2 into the shaft2 with the outwardgripper in the left_hand", "name": "action: insert(left_hand, outwardgripper, gear2, shaft2)"}]}]}} -{"target": "target: is_inserted_to(gear3, shaft3)", "initial_world_state": {"objects": [{"name": "clampgripper", "properties": []}, {"name": "parallelgripper", "properties": ["is_empty"]}, {"name": "inwardgripper", "properties": ["is_empty"]}, {"name": "outwardgripper", "properties": ["is_empty"]}, {"name": "defaultgripper", "properties": ["is_empty"]}, {"name": "gear1", "properties": []}, {"name": "gear2", "properties": []}, {"name": "gear3", "properties": []}, {"name": "shaft1", "properties": []}, {"name": "shaft2", "properties": []}, {"name": "shaft3", "properties": []}, {"name": "gearbase", "properties": []}, {"name": "gearbase_hole1", "properties": []}, {"name": "gearbase_hole3", "properties": []}, {"name": "left_hand", "properties": []}], "constraints": [{"source": "clampgripper", "name": "can_manipulate", "target": "shaft1"}, {"source": "gear3", "name": "can_insert_to", "target": "shaft3"}, {"source": "outwardgripper", "name": "can_manipulate", "target": "gear3"}, {"source": "shaft3", "name": "can_insert_to", "target": "gearbase_hole3"}, {"source": "gear1", "name": "can_insert_to", "target": "shaft1"}, {"source": "parallelgripper", "name": "can_manipulate", "target": "gear1"}, {"source": "defaultgripper", "name": "can_manipulate", "target": "shaft3"}, {"source": "gear2", "name": "can_insert_to", "target": "shaft2"}, {"source": "shaft1", "name": "can_insert_to", "target": "gearbase_hole1"}, {"source": "outwardgripper", "name": "can_manipulate", "target": "gear2"}], "relations": [{"source": "left_hand", "name": "hold", "target": "clampgripper"}, {"source": "clampgripper", "name": "hold", "target": "shaft1"}]}, "result": {"summary": "selector to insert the gear3 into the shaft3 with the outwardgripper in the left_hand", "name": "selector: insert(left_hand, outwardgripper, gear3, shaft3)", "children": [{"summary": "the target is to make the gear3 be inserted into the shaft3", "name": "target: is_inserted_to(gear3, shaft3)"}, {"summary": "sequence to insert the gear3 into the shaft3 with the outwardgripper in the left_hand", "name": "sequence: insert(left_hand, outwardgripper, gear3, shaft3)", "children": [{"summary": "selector to change tool in left_hand from clampgripper to outwardgripper", "name": "selector: change_tool(left_hand, clampgripper, outwardgripper)", "children": [{"summary": "the target is to make left_hand hold outwardgripper", "name": "target: hold(left_hand, outwardgripper)"}, {"summary": "sequence to change tool in left_hand from clampgripper to outwardgripper", "name": "sequence: change_tool(left_hand, clampgripper, outwardgripper)", "children": [{"summary": "a precondition is left_hand is holding clampgripper", "name": "precondition: hold(left_hand, clampgripper)"}, {"summary": "selector to put down the shaft1 held by the clampgripper in the left_hand", "name": "selector: put_down(left_hand, clampgripper, shaft1)", "children": [{"summary": "the target is to make the clampgripper empty", "name": "target: is_empty(clampgripper)"}, {"summary": "sequence to put down the shaft1 held by the clampgripper in the left_hand", "name": "sequence: put_down(left_hand, clampgripper, shaft1)", "children": [{"summary": "a precondition is the left_hand is holding clampgripper", "name": "precondition: hold(left_hand, clampgripper)"}, {"summary": "a precondition is the clampgripper is holding shaft1", "name": "precondition: hold(clampgripper, shaft1)"}, {"summary": "the action to put down the shaft1 held by the clampgripper in the left_hand", "name": "action: put_down(left_hand, clampgripper, shaft1)"}]}]}, {"summary": "the action to change tool in left_hand from clampgripper to outwardgripper", "name": "action: change_tool(left_hand, clampgripper, outwardgripper)"}]}]}, {"summary": "selector to pick up the gear3 with the outwardgripper in the left_hand", "name": "selector: pick_up(left_hand, outwardgripper, gear3)", "children": [{"summary": "the target is to make the outwardgripper hold the gear3", "name": "target: hold(outwardgripper, gear3)"}, {"summary": "sequence to pick up the gear3 with the outwardgripper in the left_hand", "name": "sequence: pick_up(left_hand, outwardgripper, gear3)", "children": [{"summary": "a precondition is the outwardgripper is empty", "name": "precondition: is_empty(outwardgripper)"}, {"summary": "a precondition is the left_hand is holding outwardgripper", "name": "precondition: hold(left_hand, outwardgripper)"}, {"summary": "the action to pick up the gear3 with the outwardgripper in the left_hand", "name": "action: pick_up(left_hand, outwardgripper, gear3)"}]}]}, {"summary": "the action to insert the gear3 into the shaft3 with the outwardgripper in the left_hand", "name": "action: insert(left_hand, outwardgripper, gear3, shaft3)"}]}]}} -{"target": "target: is_inserted_to(gear3, shaft3)", "initial_world_state": {"objects": [{"name": "clampgripper", "properties": ["is_empty"]}, {"name": "parallelgripper", "properties": ["is_empty"]}, {"name": "inwardgripper", "properties": ["is_empty"]}, {"name": "outwardgripper", "properties": ["is_empty"]}, {"name": "defaultgripper", "properties": ["is_empty"]}, {"name": "gear1", "properties": []}, {"name": "gear2", "properties": []}, {"name": "gear3", "properties": []}, {"name": "shaft1", "properties": []}, {"name": "shaft2", "properties": []}, {"name": "shaft3", "properties": []}, {"name": "gearbase", "properties": []}, {"name": "gearbase_hole1", "properties": []}, {"name": "gearbase_hole3", "properties": []}, {"name": "left_hand", "properties": []}], "constraints": [{"source": "gear2", "name": "can_insert_to", "target": "shaft2"}, {"source": "outwardgripper", "name": "can_manipulate", "target": "gear2"}, {"source": "shaft3", "name": "can_insert_to", "target": "gearbase_hole3"}, {"source": "clampgripper", "name": "can_manipulate", "target": "shaft1"}, {"source": "shaft1", "name": "can_insert_to", "target": "gearbase_hole1"}, {"source": "outwardgripper", "name": "can_manipulate", "target": "gear3"}, {"source": "gear1", "name": "can_insert_to", "target": "shaft1"}, {"source": "defaultgripper", "name": "can_manipulate", "target": "shaft3"}, {"source": "gear3", "name": "can_insert_to", "target": "shaft3"}, {"source": "parallelgripper", "name": "can_manipulate", "target": "gear1"}], "relations": [{"source": "left_hand", "name": "hold", "target": "defaultgripper"}]}, "result": {"summary": "selector to insert the gear3 into the shaft3 with the outwardgripper in the left_hand", "name": "selector: insert(left_hand, outwardgripper, gear3, shaft3)", "children": [{"summary": "the target is to make the gear3 be inserted into the shaft3", "name": "target: is_inserted_to(gear3, shaft3)"}, {"summary": "sequence to insert the gear3 into the shaft3 with the outwardgripper in the left_hand", "name": "sequence: insert(left_hand, outwardgripper, gear3, shaft3)", "children": [{"summary": "selector to change tool in left_hand from clampgripper to outwardgripper", "name": "selector: change_tool(left_hand, clampgripper, outwardgripper)", "children": [{"summary": "the target is to make left_hand hold outwardgripper", "name": "target: hold(left_hand, outwardgripper)"}, {"summary": "sequence to change tool in left_hand from clampgripper to outwardgripper", "name": "sequence: change_tool(left_hand, clampgripper, outwardgripper)", "children": [{"summary": "selector to change tool in left_hand from defaultgripper to clampgripper", "name": "selector: change_tool(left_hand, defaultgripper, clampgripper)", "children": [{"summary": "the target is to make left_hand hold clampgripper", "name": "target: hold(left_hand, clampgripper)"}, {"summary": "sequence to change tool in left_hand from defaultgripper to clampgripper", "name": "sequence: change_tool(left_hand, defaultgripper, clampgripper)", "children": [{"summary": "a precondition is left_hand is holding defaultgripper", "name": "precondition: hold(left_hand, defaultgripper)"}, {"summary": "a precondition is defaultgripper is empty", "name": "precondition: is_empty(defaultgripper)"}, {"summary": "the action to change tool in left_hand from defaultgripper to clampgripper", "name": "action: change_tool(left_hand, defaultgripper, clampgripper)"}]}]}, {"summary": "a precondition is clampgripper is empty", "name": "precondition: is_empty(clampgripper)"}, {"summary": "the action to change tool in left_hand from clampgripper to outwardgripper", "name": "action: change_tool(left_hand, clampgripper, outwardgripper)"}]}]}, {"summary": "selector to pick up the gear3 with the outwardgripper in the left_hand", "name": "selector: pick_up(left_hand, outwardgripper, gear3)", "children": [{"summary": "the target is to make the outwardgripper hold the gear3", "name": "target: hold(outwardgripper, gear3)"}, {"summary": "sequence to pick up the gear3 with the outwardgripper in the left_hand", "name": "sequence: pick_up(left_hand, outwardgripper, gear3)", "children": [{"summary": "a precondition is the outwardgripper is empty", "name": "precondition: is_empty(outwardgripper)"}, {"summary": "a precondition is the left_hand is holding outwardgripper", "name": "precondition: hold(left_hand, outwardgripper)"}, {"summary": "the action to pick up the gear3 with the outwardgripper in the left_hand", "name": "action: pick_up(left_hand, outwardgripper, gear3)"}]}]}, {"summary": "the action to insert the gear3 into the shaft3 with the outwardgripper in the left_hand", "name": "action: insert(left_hand, outwardgripper, gear3, shaft3)"}]}]}} -{"target": "target: is_inserted_to(gear3, shaft3)", "initial_world_state": {"objects": [{"name": "clampgripper", "properties": ["is_empty"]}, {"name": "parallelgripper", "properties": ["is_empty"]}, {"name": "inwardgripper", "properties": ["is_empty"]}, {"name": "outwardgripper", "properties": ["is_empty"]}, {"name": "defaultgripper", "properties": ["is_empty"]}, {"name": "gear1", "properties": []}, {"name": "gear2", "properties": []}, {"name": "gear3", "properties": []}, {"name": "shaft1", "properties": []}, {"name": "shaft2", "properties": []}, {"name": "shaft3", "properties": []}, {"name": "gearbase", "properties": []}, {"name": "gearbase_hole1", "properties": []}, {"name": "gearbase_hole3", "properties": []}, {"name": "left_hand", "properties": []}], "constraints": [{"source": "outwardgripper", "name": "can_manipulate", "target": "gear2"}, {"source": "clampgripper", "name": "can_manipulate", "target": "shaft1"}, {"source": "defaultgripper", "name": "can_manipulate", "target": "shaft3"}, {"source": "gear2", "name": "can_insert_to", "target": "shaft2"}, {"source": "shaft1", "name": "can_insert_to", "target": "gearbase_hole1"}, {"source": "gear3", "name": "can_insert_to", "target": "shaft3"}, {"source": "shaft3", "name": "can_insert_to", "target": "gearbase_hole3"}, {"source": "parallelgripper", "name": "can_manipulate", "target": "gear1"}, {"source": "outwardgripper", "name": "can_manipulate", "target": "gear3"}, {"source": "gear1", "name": "can_insert_to", "target": "shaft1"}], "relations": [{"source": "left_hand", "name": "hold", "target": "outwardgripper"}]}, "result": {"summary": "selector to insert the gear3 into the shaft3 with the outwardgripper in the left_hand", "name": "selector: insert(left_hand, outwardgripper, gear3, shaft3)", "children": [{"summary": "the target is to make the gear3 be inserted into the shaft3", "name": "target: is_inserted_to(gear3, shaft3)"}, {"summary": "sequence to insert the gear3 into the shaft3 with the outwardgripper in the left_hand", "name": "sequence: insert(left_hand, outwardgripper, gear3, shaft3)", "children": [{"summary": "a precondition is the left_hand is holding outwardgripper", "name": "precondition: hold(left_hand, outwardgripper)"}, {"summary": "selector to pick up the gear3 with the outwardgripper in the left_hand", "name": "selector: pick_up(left_hand, outwardgripper, gear3)", "children": [{"summary": "the target is to make the outwardgripper hold the gear3", "name": "target: hold(outwardgripper, gear3)"}, {"summary": "sequence to pick up the gear3 with the outwardgripper in the left_hand", "name": "sequence: pick_up(left_hand, outwardgripper, gear3)", "children": [{"summary": "a precondition is the outwardgripper is empty", "name": "precondition: is_empty(outwardgripper)"}, {"summary": "a precondition is the left_hand is holding outwardgripper", "name": "precondition: hold(left_hand, outwardgripper)"}, {"summary": "the action to pick up the gear3 with the outwardgripper in the left_hand", "name": "action: pick_up(left_hand, outwardgripper, gear3)"}]}]}, {"summary": "the action to insert the gear3 into the shaft3 with the outwardgripper in the left_hand", "name": "action: insert(left_hand, outwardgripper, gear3, shaft3)"}]}]}} -{"target": "target: is_inserted_to(gear1, shaft1)", "initial_world_state": {"objects": [{"name": "clampgripper", "properties": ["is_empty"]}, {"name": "parallelgripper", "properties": ["is_empty"]}, {"name": "inwardgripper", "properties": ["is_empty"]}, {"name": "outwardgripper", "properties": ["is_empty"]}, {"name": "defaultgripper", "properties": ["is_empty"]}, {"name": "gear1", "properties": []}, {"name": "gear2", "properties": []}, {"name": "gear3", "properties": []}, {"name": "shaft1", "properties": []}, {"name": "shaft2", "properties": []}, {"name": "shaft3", "properties": []}, {"name": "gearbase", "properties": []}, {"name": "gearbase_hole1", "properties": []}, {"name": "gearbase_hole3", "properties": []}, {"name": "left_hand", "properties": []}], "constraints": [{"source": "clampgripper", "name": "can_manipulate", "target": "shaft1"}, {"source": "outwardgripper", "name": "can_manipulate", "target": "gear2"}, {"source": "parallelgripper", "name": "can_manipulate", "target": "gear1"}, {"source": "gear1", "name": "can_insert_to", "target": "shaft1"}, {"source": "shaft1", "name": "can_insert_to", "target": "gearbase_hole1"}, {"source": "gear2", "name": "can_insert_to", "target": "shaft2"}, {"source": "outwardgripper", "name": "can_manipulate", "target": "gear3"}, {"source": "gear3", "name": "can_insert_to", "target": "shaft3"}, {"source": "shaft3", "name": "can_insert_to", "target": "gearbase_hole3"}, {"source": "defaultgripper", "name": "can_manipulate", "target": "shaft3"}], "relations": [{"source": "shaft1", "name": "is_inserted_to", "target": "gearbase_hole1"}, {"source": "left_hand", "name": "hold", "target": "outwardgripper"}]}, "result": {"summary": "selector to insert the gear1 into the shaft1 with the parallelgripper in the left_hand", "name": "selector: insert(left_hand, parallelgripper, gear1, shaft1)", "children": [{"summary": "the target is to make the gear1 be inserted into the shaft1", "name": "target: is_inserted_to(gear1, shaft1)"}, {"summary": "sequence to insert the gear1 into the shaft1 with the parallelgripper in the left_hand", "name": "sequence: insert(left_hand, parallelgripper, gear1, shaft1)", "children": [{"summary": "selector to change tool in left_hand from outwardgripper to parallelgripper", "name": "selector: change_tool(left_hand, outwardgripper, parallelgripper)", "children": [{"summary": "the target is to make left_hand hold parallelgripper", "name": "target: hold(left_hand, parallelgripper)"}, {"summary": "sequence to change tool in left_hand from outwardgripper to parallelgripper", "name": "sequence: change_tool(left_hand, outwardgripper, parallelgripper)", "children": [{"summary": "a precondition is left_hand is holding outwardgripper", "name": "precondition: hold(left_hand, outwardgripper)"}, {"summary": "a precondition is outwardgripper is empty", "name": "precondition: is_empty(outwardgripper)"}, {"summary": "the action to change tool in left_hand from outwardgripper to parallelgripper", "name": "action: change_tool(left_hand, outwardgripper, parallelgripper)"}]}]}, {"summary": "selector to pick up the gear1 with the parallelgripper in the left_hand", "name": "selector: pick_up(left_hand, parallelgripper, gear1)", "children": [{"summary": "the target is to make the parallelgripper hold the gear1", "name": "target: hold(parallelgripper, gear1)"}, {"summary": "sequence to pick up the gear1 with the parallelgripper in the left_hand", "name": "sequence: pick_up(left_hand, parallelgripper, gear1)", "children": [{"summary": "a precondition is the parallelgripper is empty", "name": "precondition: is_empty(parallelgripper)"}, {"summary": "a precondition is the left_hand is holding parallelgripper", "name": "precondition: hold(left_hand, parallelgripper)"}, {"summary": "the action to pick up the gear1 with the parallelgripper in the left_hand", "name": "action: pick_up(left_hand, parallelgripper, gear1)"}]}]}, {"summary": "the action to insert the gear1 into the shaft1 with the parallelgripper in the left_hand", "name": "action: insert(left_hand, parallelgripper, gear1, shaft1)"}]}]}} -{"target": "target: is_inserted_to(gear1, shaft1)", "initial_world_state": {"objects": [{"name": "clampgripper", "properties": ["is_empty"]}, {"name": "parallelgripper", "properties": ["is_empty"]}, {"name": "inwardgripper", "properties": ["is_empty"]}, {"name": "outwardgripper", "properties": ["is_empty"]}, {"name": "defaultgripper", "properties": ["is_empty"]}, {"name": "gear1", "properties": []}, {"name": "gear2", "properties": []}, {"name": "gear3", "properties": []}, {"name": "shaft1", "properties": []}, {"name": "shaft2", "properties": []}, {"name": "shaft3", "properties": []}, {"name": "gearbase", "properties": []}, {"name": "gearbase_hole1", "properties": []}, {"name": "gearbase_hole3", "properties": []}, {"name": "left_hand", "properties": []}], "constraints": [{"source": "parallelgripper", "name": "can_manipulate", "target": "gear1"}, {"source": "outwardgripper", "name": "can_manipulate", "target": "gear3"}, {"source": "gear2", "name": "can_insert_to", "target": "shaft2"}, {"source": "defaultgripper", "name": "can_manipulate", "target": "shaft3"}, {"source": "outwardgripper", "name": "can_manipulate", "target": "gear2"}, {"source": "gear3", "name": "can_insert_to", "target": "shaft3"}, {"source": "clampgripper", "name": "can_manipulate", "target": "shaft1"}, {"source": "shaft3", "name": "can_insert_to", "target": "gearbase_hole3"}, {"source": "shaft1", "name": "can_insert_to", "target": "gearbase_hole1"}, {"source": "gear1", "name": "can_insert_to", "target": "shaft1"}], "relations": [{"source": "shaft1", "name": "is_inserted_to", "target": "gearbase_hole1"}, {"source": "left_hand", "name": "hold", "target": "outwardgripper"}, {"source": "gear1", "name": "is_inserted_to", "target": "shaft1"}]}, "result": {"summary": "the target is to insert gear1 to shaft1", "name": "target: is_inserted_to(gear1, shaft1)"}} -{"target": "target: is_inserted_to(gear3, shaft3)", "initial_world_state": {"objects": [{"name": "clampgripper", "properties": ["is_empty"]}, {"name": "parallelgripper", "properties": ["is_empty"]}, {"name": "inwardgripper", "properties": ["is_empty"]}, {"name": "outwardgripper", "properties": ["is_empty"]}, {"name": "defaultgripper", "properties": ["is_empty"]}, {"name": "gear1", "properties": []}, {"name": "gear2", "properties": []}, {"name": "gear3", "properties": []}, {"name": "shaft1", "properties": []}, {"name": "shaft2", "properties": []}, {"name": "shaft3", "properties": []}, {"name": "gearbase", "properties": []}, {"name": "gearbase_hole1", "properties": []}, {"name": "gearbase_hole3", "properties": []}, {"name": "left_hand", "properties": []}], "constraints": [{"source": "gear2", "name": "can_insert_to", "target": "shaft2"}, {"source": "parallelgripper", "name": "can_manipulate", "target": "gear1"}, {"source": "gear3", "name": "can_insert_to", "target": "shaft3"}, {"source": "defaultgripper", "name": "can_manipulate", "target": "shaft3"}, {"source": "outwardgripper", "name": "can_manipulate", "target": "gear2"}, {"source": "gear1", "name": "can_insert_to", "target": "shaft1"}, {"source": "outwardgripper", "name": "can_manipulate", "target": "gear3"}, {"source": "shaft1", "name": "can_insert_to", "target": "gearbase_hole1"}, {"source": "clampgripper", "name": "can_manipulate", "target": "shaft1"}, {"source": "shaft3", "name": "can_insert_to", "target": "gearbase_hole3"}], "relations": [{"source": "gear1", "name": "is_inserted_to", "target": "shaft1"}, {"source": "shaft1", "name": "is_inserted_to", "target": "gearbase_hole1"}, {"source": "shaft3", "name": "is_inserted_to", "target": "gearbase_hole3"}, {"source": "left_hand", "name": "hold", "target": "defaultgripper"}]}, "result": {"summary": "selector to insert the gear3 into the shaft3 with the outwardgripper in the left_hand", "name": "selector: insert(left_hand, outwardgripper, gear3, shaft3)", "children": [{"summary": "the target is to make the gear3 be inserted into the shaft3", "name": "target: is_inserted_to(gear3, shaft3)"}, {"summary": "sequence to insert the gear3 into the shaft3 with the outwardgripper in the left_hand", "name": "sequence: insert(left_hand, outwardgripper, gear3, shaft3)", "children": [{"summary": "selector to change tool in left_hand from defaultgripper to outwardgripper", "name": "selector: change_tool(left_hand, defaultgripper, outwardgripper)", "children": [{"summary": "the target is to make left_hand hold outwardgripper", "name": "target: hold(left_hand, outwardgripper)"}, {"summary": "sequence to change tool in left_hand from defaultgripper to outwardgripper", "name": "sequence: change_tool(left_hand, defaultgripper, outwardgripper)", "children": [{"summary": "a precondition is left_hand is holding defaultgripper", "name": "precondition: hold(left_hand, defaultgripper)"}, {"summary": "a precondition is defaultgripper is empty", "name": "precondition: is_empty(defaultgripper)"}, {"summary": "the action to change tool in left_hand from defaultgripper to outwardgripper", "name": "action: change_tool(left_hand, defaultgripper, outwardgripper)"}]}]}, {"summary": "selector to pick up the gear3 with the outwardgripper in the left_hand", "name": "selector: pick_up(left_hand, outwardgripper, gear3)", "children": [{"summary": "the target is to make the outwardgripper hold the gear3", "name": "target: hold(outwardgripper, gear3)"}, {"summary": "sequence to pick up the gear3 with the outwardgripper in the left_hand", "name": "sequence: pick_up(left_hand, outwardgripper, gear3)", "children": [{"summary": "a precondition is the outwardgripper is empty", "name": "precondition: is_empty(outwardgripper)"}, {"summary": "a precondition is the left_hand is holding outwardgripper", "name": "precondition: hold(left_hand, outwardgripper)"}, {"summary": "the action to pick up the gear3 with the outwardgripper in the left_hand", "name": "action: pick_up(left_hand, outwardgripper, gear3)"}]}]}, {"summary": "the action to insert the gear3 into the shaft3 with the outwardgripper in the left_hand", "name": "action: insert(left_hand, outwardgripper, gear3, shaft3)"}]}]}} -{"target": "target: is_inserted_to(shaft1, gearbase_hole1)", "initial_world_state": {"objects": [{"name": "clampgripper", "properties": ["is_empty"]}, {"name": "parallelgripper", "properties": ["is_empty"]}, {"name": "inwardgripper", "properties": ["is_empty"]}, {"name": "outwardgripper", "properties": ["is_empty"]}, {"name": "defaultgripper", "properties": []}, {"name": "gear1", "properties": []}, {"name": "gear2", "properties": []}, {"name": "gear3", "properties": []}, {"name": "shaft1", "properties": []}, {"name": "shaft2", "properties": []}, {"name": "shaft3", "properties": []}, {"name": "gearbase", "properties": []}, {"name": "gearbase_hole1", "properties": []}, {"name": "gearbase_hole3", "properties": []}, {"name": "left_hand", "properties": []}], "constraints": [{"source": "shaft3", "name": "can_insert_to", "target": "gearbase_hole3"}, {"source": "defaultgripper", "name": "can_manipulate", "target": "shaft3"}, {"source": "gear2", "name": "can_insert_to", "target": "shaft2"}, {"source": "gear1", "name": "can_insert_to", "target": "shaft1"}, {"source": "outwardgripper", "name": "can_manipulate", "target": "gear2"}, {"source": "shaft1", "name": "can_insert_to", "target": "gearbase_hole1"}, {"source": "outwardgripper", "name": "can_manipulate", "target": "gear3"}, {"source": "clampgripper", "name": "can_manipulate", "target": "shaft1"}, {"source": "parallelgripper", "name": "can_manipulate", "target": "gear1"}, {"source": "gear3", "name": "can_insert_to", "target": "shaft3"}], "relations": [{"source": "left_hand", "name": "hold", "target": "defaultgripper"}, {"source": "defaultgripper", "name": "hold", "target": "shaft3"}]}, "result": {"summary": "selector to insert the shaft1 into the gearbase_hole1 with the clampgripper in the left_hand", "name": "selector: insert(left_hand, clampgripper, shaft1, gearbase_hole1)", "children": [{"summary": "the target is to make the shaft1 be inserted into the gearbase_hole1", "name": "target: is_inserted_to(shaft1, gearbase_hole1)"}, {"summary": "sequence to insert the shaft1 into the gearbase_hole1 with the clampgripper in the left_hand", "name": "sequence: insert(left_hand, clampgripper, shaft1, gearbase_hole1)", "children": [{"summary": "selector to change tool in left_hand from defaultgripper to clampgripper", "name": "selector: change_tool(left_hand, defaultgripper, clampgripper)", "children": [{"summary": "the target is to make left_hand hold clampgripper", "name": "target: hold(left_hand, clampgripper)"}, {"summary": "sequence to change tool in left_hand from defaultgripper to clampgripper", "name": "sequence: change_tool(left_hand, defaultgripper, clampgripper)", "children": [{"summary": "a precondition is left_hand is holding defaultgripper", "name": "precondition: hold(left_hand, defaultgripper)"}, {"summary": "selector to put down the shaft3 held by the defaultgripper in the left_hand", "name": "selector: put_down(left_hand, defaultgripper, shaft3)", "children": [{"summary": "the target is to make the defaultgripper empty", "name": "target: is_empty(defaultgripper)"}, {"summary": "sequence to put down the shaft3 held by the defaultgripper in the left_hand", "name": "sequence: put_down(left_hand, defaultgripper, shaft3)", "children": [{"summary": "a precondition is the left_hand is holding defaultgripper", "name": "precondition: hold(left_hand, defaultgripper)"}, {"summary": "a precondition is the defaultgripper is holding shaft3", "name": "precondition: hold(defaultgripper, shaft3)"}, {"summary": "the action to put down the shaft3 held by the defaultgripper in the left_hand", "name": "action: put_down(left_hand, defaultgripper, shaft3)"}]}]}, {"summary": "the action to change tool in left_hand from defaultgripper to clampgripper", "name": "action: change_tool(left_hand, defaultgripper, clampgripper)"}]}]}, {"summary": "selector to pick up the shaft1 with the clampgripper in the left_hand", "name": "selector: pick_up(left_hand, clampgripper, shaft1)", "children": [{"summary": "the target is to make the clampgripper hold the shaft1", "name": "target: hold(clampgripper, shaft1)"}, {"summary": "sequence to pick up the shaft1 with the clampgripper in the left_hand", "name": "sequence: pick_up(left_hand, clampgripper, shaft1)", "children": [{"summary": "a precondition is the clampgripper is empty", "name": "precondition: is_empty(clampgripper)"}, {"summary": "a precondition is the left_hand is holding clampgripper", "name": "precondition: hold(left_hand, clampgripper)"}, {"summary": "the action to pick up the shaft1 with the clampgripper in the left_hand", "name": "action: pick_up(left_hand, clampgripper, shaft1)"}]}]}, {"summary": "the action to insert the shaft1 into the gearbase_hole1 with the clampgripper in the left_hand", "name": "action: insert(left_hand, clampgripper, shaft1, gearbase_hole1)"}]}]}} -{"target": "target: is_inserted_to(gear2, shaft2)", "initial_world_state": {"objects": [{"name": "clampgripper", "properties": ["is_empty"]}, {"name": "parallelgripper", "properties": ["is_empty"]}, {"name": "inwardgripper", "properties": ["is_empty"]}, {"name": "outwardgripper", "properties": ["is_empty"]}, {"name": "defaultgripper", "properties": ["is_empty"]}, {"name": "gear1", "properties": []}, {"name": "gear2", "properties": []}, {"name": "gear3", "properties": []}, {"name": "shaft1", "properties": []}, {"name": "shaft2", "properties": []}, {"name": "shaft3", "properties": []}, {"name": "gearbase", "properties": []}, {"name": "gearbase_hole1", "properties": []}, {"name": "gearbase_hole3", "properties": []}, {"name": "left_hand", "properties": []}], "constraints": [{"source": "shaft1", "name": "can_insert_to", "target": "gearbase_hole1"}, {"source": "shaft3", "name": "can_insert_to", "target": "gearbase_hole3"}, {"source": "parallelgripper", "name": "can_manipulate", "target": "gear1"}, {"source": "clampgripper", "name": "can_manipulate", "target": "shaft1"}, {"source": "defaultgripper", "name": "can_manipulate", "target": "shaft3"}, {"source": "outwardgripper", "name": "can_manipulate", "target": "gear2"}, {"source": "gear3", "name": "can_insert_to", "target": "shaft3"}, {"source": "outwardgripper", "name": "can_manipulate", "target": "gear3"}, {"source": "gear1", "name": "can_insert_to", "target": "shaft1"}, {"source": "gear2", "name": "can_insert_to", "target": "shaft2"}], "relations": [{"source": "left_hand", "name": "hold", "target": "defaultgripper"}]}, "result": {"summary": "selector to insert the gear2 into the shaft2 with the outwardgripper in the left_hand", "name": "selector: insert(left_hand, outwardgripper, gear2, shaft2)", "children": [{"summary": "the target is to make the gear2 be inserted into the shaft2", "name": "target: is_inserted_to(gear2, shaft2)"}, {"summary": "sequence to insert the gear2 into the shaft2 with the outwardgripper in the left_hand", "name": "sequence: insert(left_hand, outwardgripper, gear2, shaft2)", "children": [{"summary": "selector to change tool in left_hand from defaultgripper to outwardgripper", "name": "selector: change_tool(left_hand, defaultgripper, outwardgripper)", "children": [{"summary": "the target is to make left_hand hold outwardgripper", "name": "target: hold(left_hand, outwardgripper)"}, {"summary": "sequence to change tool in left_hand from defaultgripper to outwardgripper", "name": "sequence: change_tool(left_hand, defaultgripper, outwardgripper)", "children": [{"summary": "a precondition is left_hand is holding defaultgripper", "name": "precondition: hold(left_hand, defaultgripper)"}, {"summary": "a precondition is defaultgripper is empty", "name": "precondition: is_empty(defaultgripper)"}, {"summary": "the action to change tool in left_hand from defaultgripper to outwardgripper", "name": "action: change_tool(left_hand, defaultgripper, outwardgripper)"}]}]}, {"summary": "selector to pick up the gear2 with the outwardgripper in the left_hand", "name": "selector: pick_up(left_hand, outwardgripper, gear2)", "children": [{"summary": "the target is to make the outwardgripper hold the gear2", "name": "target: hold(outwardgripper, gear2)"}, {"summary": "sequence to pick up the gear2 with the outwardgripper in the left_hand", "name": "sequence: pick_up(left_hand, outwardgripper, gear2)", "children": [{"summary": "a precondition is the outwardgripper is empty", "name": "precondition: is_empty(outwardgripper)"}, {"summary": "a precondition is the left_hand is holding outwardgripper", "name": "precondition: hold(left_hand, outwardgripper)"}, {"summary": "the action to pick up the gear2 with the outwardgripper in the left_hand", "name": "action: pick_up(left_hand, outwardgripper, gear2)"}]}]}, {"summary": "the action to insert the gear2 into the shaft2 with the outwardgripper in the left_hand", "name": "action: insert(left_hand, outwardgripper, gear2, shaft2)"}]}]}} -{"target": "target: is_inserted_to(shaft3, gearbase_hole3)", "initial_world_state": {"objects": [{"name": "clampgripper", "properties": ["is_empty"]}, {"name": "parallelgripper", "properties": ["is_empty"]}, {"name": "inwardgripper", "properties": ["is_empty"]}, {"name": "outwardgripper", "properties": ["is_empty"]}, {"name": "defaultgripper", "properties": ["is_empty"]}, {"name": "gear1", "properties": []}, {"name": "gear2", "properties": []}, {"name": "gear3", "properties": []}, {"name": "shaft1", "properties": []}, {"name": "shaft2", "properties": []}, {"name": "shaft3", "properties": []}, {"name": "gearbase", "properties": []}, {"name": "gearbase_hole1", "properties": []}, {"name": "gearbase_hole3", "properties": []}, {"name": "left_hand", "properties": []}], "constraints": [{"source": "outwardgripper", "name": "can_manipulate", "target": "gear2"}, {"source": "shaft1", "name": "can_insert_to", "target": "gearbase_hole1"}, {"source": "clampgripper", "name": "can_manipulate", "target": "shaft1"}, {"source": "defaultgripper", "name": "can_manipulate", "target": "shaft3"}, {"source": "outwardgripper", "name": "can_manipulate", "target": "gear3"}, {"source": "gear2", "name": "can_insert_to", "target": "shaft2"}, {"source": "parallelgripper", "name": "can_manipulate", "target": "gear1"}, {"source": "gear1", "name": "can_insert_to", "target": "shaft1"}, {"source": "gear3", "name": "can_insert_to", "target": "shaft3"}, {"source": "shaft3", "name": "can_insert_to", "target": "gearbase_hole3"}], "relations": [{"source": "left_hand", "name": "hold", "target": "inwardgripper"}]}, "result": {"summary": "selector to insert the shaft3 into the gearbase_hole3 with the defaultgripper in the left_hand", "name": "selector: insert(left_hand, defaultgripper, shaft3, gearbase_hole3)", "children": [{"summary": "the target is to make the shaft3 be inserted into the gearbase_hole3", "name": "target: is_inserted_to(shaft3, gearbase_hole3)"}, {"summary": "sequence to insert the shaft3 into the gearbase_hole3 with the defaultgripper in the left_hand", "name": "sequence: insert(left_hand, defaultgripper, shaft3, gearbase_hole3)", "children": [{"summary": "selector to change tool in left_hand from inwardgripper to defaultgripper", "name": "selector: change_tool(left_hand, inwardgripper, defaultgripper)", "children": [{"summary": "the target is to make left_hand hold defaultgripper", "name": "target: hold(left_hand, defaultgripper)"}, {"summary": "sequence to change tool in left_hand from inwardgripper to defaultgripper", "name": "sequence: change_tool(left_hand, inwardgripper, defaultgripper)", "children": [{"summary": "a precondition is left_hand is holding inwardgripper", "name": "precondition: hold(left_hand, inwardgripper)"}, {"summary": "a precondition is inwardgripper is empty", "name": "precondition: is_empty(inwardgripper)"}, {"summary": "the action to change tool in left_hand from inwardgripper to defaultgripper", "name": "action: change_tool(left_hand, inwardgripper, defaultgripper)"}]}]}, {"summary": "selector to pick up the shaft3 with the defaultgripper in the left_hand", "name": "selector: pick_up(left_hand, defaultgripper, shaft3)", "children": [{"summary": "the target is to make the defaultgripper hold the shaft3", "name": "target: hold(defaultgripper, shaft3)"}, {"summary": "sequence to pick up the shaft3 with the defaultgripper in the left_hand", "name": "sequence: pick_up(left_hand, defaultgripper, shaft3)", "children": [{"summary": "a precondition is the defaultgripper is empty", "name": "precondition: is_empty(defaultgripper)"}, {"summary": "a precondition is the left_hand is holding defaultgripper", "name": "precondition: hold(left_hand, defaultgripper)"}, {"summary": "the action to pick up the shaft3 with the defaultgripper in the left_hand", "name": "action: pick_up(left_hand, defaultgripper, shaft3)"}]}]}, {"summary": "the action to insert the shaft3 into the gearbase_hole3 with the defaultgripper in the left_hand", "name": "action: insert(left_hand, defaultgripper, shaft3, gearbase_hole3)"}]}]}} -{"target": "target: is_inserted_to(shaft3, gearbase_hole3)", "initial_world_state": {"objects": [{"name": "clampgripper", "properties": ["is_empty"]}, {"name": "parallelgripper", "properties": []}, {"name": "inwardgripper", "properties": ["is_empty"]}, {"name": "outwardgripper", "properties": ["is_empty"]}, {"name": "defaultgripper", "properties": ["is_empty"]}, {"name": "gear1", "properties": []}, {"name": "gear2", "properties": []}, {"name": "gear3", "properties": []}, {"name": "shaft1", "properties": []}, {"name": "shaft2", "properties": []}, {"name": "shaft3", "properties": []}, {"name": "gearbase", "properties": []}, {"name": "gearbase_hole1", "properties": []}, {"name": "gearbase_hole3", "properties": []}, {"name": "left_hand", "properties": []}], "constraints": [{"source": "shaft3", "name": "can_insert_to", "target": "gearbase_hole3"}, {"source": "gear3", "name": "can_insert_to", "target": "shaft3"}, {"source": "defaultgripper", "name": "can_manipulate", "target": "shaft3"}, {"source": "shaft1", "name": "can_insert_to", "target": "gearbase_hole1"}, {"source": "outwardgripper", "name": "can_manipulate", "target": "gear2"}, {"source": "gear1", "name": "can_insert_to", "target": "shaft1"}, {"source": "clampgripper", "name": "can_manipulate", "target": "shaft1"}, {"source": "outwardgripper", "name": "can_manipulate", "target": "gear3"}, {"source": "gear2", "name": "can_insert_to", "target": "shaft2"}, {"source": "parallelgripper", "name": "can_manipulate", "target": "gear1"}], "relations": [{"source": "left_hand", "name": "hold", "target": "parallelgripper"}, {"source": "parallelgripper", "name": "hold", "target": "gear1"}]}, "result": {"summary": "selector to insert the shaft3 into the gearbase_hole3 with the defaultgripper in the left_hand", "name": "selector: insert(left_hand, defaultgripper, shaft3, gearbase_hole3)", "children": [{"summary": "the target is to make the shaft3 be inserted into the gearbase_hole3", "name": "target: is_inserted_to(shaft3, gearbase_hole3)"}, {"summary": "sequence to insert the shaft3 into the gearbase_hole3 with the defaultgripper in the left_hand", "name": "sequence: insert(left_hand, defaultgripper, shaft3, gearbase_hole3)", "children": [{"summary": "selector to change tool in left_hand from parallelgripper to defaultgripper", "name": "selector: change_tool(left_hand, parallelgripper, defaultgripper)", "children": [{"summary": "the target is to make left_hand hold defaultgripper", "name": "target: hold(left_hand, defaultgripper)"}, {"summary": "sequence to change tool in left_hand from parallelgripper to defaultgripper", "name": "sequence: change_tool(left_hand, parallelgripper, defaultgripper)", "children": [{"summary": "a precondition is left_hand is holding parallelgripper", "name": "precondition: hold(left_hand, parallelgripper)"}, {"summary": "selector to put down the gear1 held by the parallelgripper in the left_hand", "name": "selector: put_down(left_hand, parallelgripper, gear1)", "children": [{"summary": "the target is to make the parallelgripper empty", "name": "target: is_empty(parallelgripper)"}, {"summary": "sequence to put down the gear1 held by the parallelgripper in the left_hand", "name": "sequence: put_down(left_hand, parallelgripper, gear1)", "children": [{"summary": "a precondition is the left_hand is holding parallelgripper", "name": "precondition: hold(left_hand, parallelgripper)"}, {"summary": "a precondition is the parallelgripper is holding gear1", "name": "precondition: hold(parallelgripper, gear1)"}, {"summary": "the action to put down the gear1 held by the parallelgripper in the left_hand", "name": "action: put_down(left_hand, parallelgripper, gear1)"}]}]}, {"summary": "the action to change tool in left_hand from parallelgripper to defaultgripper", "name": "action: change_tool(left_hand, parallelgripper, defaultgripper)"}]}]}, {"summary": "selector to pick up the shaft3 with the defaultgripper in the left_hand", "name": "selector: pick_up(left_hand, defaultgripper, shaft3)", "children": [{"summary": "the target is to make the defaultgripper hold the shaft3", "name": "target: hold(defaultgripper, shaft3)"}, {"summary": "sequence to pick up the shaft3 with the defaultgripper in the left_hand", "name": "sequence: pick_up(left_hand, defaultgripper, shaft3)", "children": [{"summary": "a precondition is the defaultgripper is empty", "name": "precondition: is_empty(defaultgripper)"}, {"summary": "a precondition is the left_hand is holding defaultgripper", "name": "precondition: hold(left_hand, defaultgripper)"}, {"summary": "the action to pick up the shaft3 with the defaultgripper in the left_hand", "name": "action: pick_up(left_hand, defaultgripper, shaft3)"}]}]}, {"summary": "the action to insert the shaft3 into the gearbase_hole3 with the defaultgripper in the left_hand", "name": "action: insert(left_hand, defaultgripper, shaft3, gearbase_hole3)"}]}]}} -{"target": "target: is_inserted_to(gear2, shaft2)", "initial_world_state": {"objects": [{"name": "clampgripper", "properties": ["is_empty"]}, {"name": "parallelgripper", "properties": []}, {"name": "inwardgripper", "properties": ["is_empty"]}, {"name": "outwardgripper", "properties": ["is_empty"]}, {"name": "defaultgripper", "properties": ["is_empty"]}, {"name": "gear1", "properties": []}, {"name": "gear2", "properties": []}, {"name": "gear3", "properties": []}, {"name": "shaft1", "properties": []}, {"name": "shaft2", "properties": []}, {"name": "shaft3", "properties": []}, {"name": "gearbase", "properties": []}, {"name": "gearbase_hole1", "properties": []}, {"name": "gearbase_hole3", "properties": []}, {"name": "lampbase", "properties": []}, {"name": "chair_leg1", "properties": []}, {"name": "left_hand", "properties": []}], "constraints": [{"source": "clampgripper", "name": "can_manipulate", "target": "shaft1"}, {"source": "outwardgripper", "name": "can_manipulate", "target": "gear3"}, {"source": "shaft3", "name": "can_insert_to", "target": "gearbase_hole3"}, {"source": "parallelgripper", "name": "can_manipulate", "target": "gear1"}, {"source": "gear3", "name": "can_insert_to", "target": "shaft3"}, {"source": "outwardgripper", "name": "can_manipulate", "target": "gear2"}, {"source": "shaft1", "name": "can_insert_to", "target": "gearbase_hole1"}, {"source": "defaultgripper", "name": "can_manipulate", "target": "shaft3"}, {"source": "gear1", "name": "can_insert_to", "target": "shaft1"}, {"source": "gear2", "name": "can_insert_to", "target": "shaft2"}], "relations": [{"source": "gear2", "name": "is_inserted_to", "target": "shaft2"}, {"source": "parallelgripper", "name": "hold", "target": "gear1"}, {"source": "left_hand", "name": "hold", "target": "parallelgripper"}]}, "result": {"summary": "the target is to insert the gear2 into the shaft2", "name": "target: is_inserted_to(gear2, shaft2)"}} -{"target": "target: is_inserted_to(gear3, shaft3)", "initial_world_state": {"objects": [{"name": "clampgripper", "properties": ["is_empty"]}, {"name": "parallelgripper", "properties": []}, {"name": "inwardgripper", "properties": ["is_empty"]}, {"name": "outwardgripper", "properties": ["is_empty"]}, {"name": "defaultgripper", "properties": ["is_empty"]}, {"name": "gear1", "properties": []}, {"name": "gear2", "properties": []}, {"name": "gear3", "properties": []}, {"name": "shaft1", "properties": []}, {"name": "shaft2", "properties": []}, {"name": "shaft3", "properties": []}, {"name": "gearbase", "properties": []}, {"name": "gearbase_hole1", "properties": []}, {"name": "gearbase_hole3", "properties": []}, {"name": "lampbase", "properties": []}, {"name": "chair_leg1", "properties": []}, {"name": "left_hand", "properties": []}], "constraints": [{"source": "outwardgripper", "name": "can_manipulate", "target": "gear2"}, {"source": "shaft1", "name": "can_insert_to", "target": "gearbase_hole1"}, {"source": "outwardgripper", "name": "can_manipulate", "target": "gear3"}, {"source": "parallelgripper", "name": "can_manipulate", "target": "gear1"}, {"source": "defaultgripper", "name": "can_manipulate", "target": "shaft3"}, {"source": "clampgripper", "name": "can_manipulate", "target": "shaft1"}, {"source": "gear3", "name": "can_insert_to", "target": "shaft3"}, {"source": "shaft3", "name": "can_insert_to", "target": "gearbase_hole3"}, {"source": "gear1", "name": "can_insert_to", "target": "shaft1"}, {"source": "gear2", "name": "can_insert_to", "target": "shaft2"}], "relations": [{"source": "shaft1", "name": "is_inserted_to", "target": "gearbase_hole1"}, {"source": "left_hand", "name": "hold", "target": "parallelgripper"}, {"source": "gear3", "name": "is_inserted_to", "target": "shaft3"}, {"source": "shaft3", "name": "is_inserted_to", "target": "gearbase_hole3"}, {"source": "gear2", "name": "is_inserted_to", "target": "shaft2"}]}, "result": {"summary": "the target is to insert gear3 into shaft3", "name": "target: is_inserted_to(gear3, shaft3)"}} -{"target": "target: is_inserted_to(gear1, shaft1)", "initial_world_state": {"objects": [{"name": "clampgripper", "properties": ["is_empty"]}, {"name": "parallelgripper", "properties": ["is_empty"]}, {"name": "inwardgripper", "properties": ["is_empty"]}, {"name": "outwardgripper", "properties": ["is_empty"]}, {"name": "defaultgripper", "properties": ["is_empty"]}, {"name": "gear1", "properties": []}, {"name": "gear2", "properties": []}, {"name": "gear3", "properties": []}, {"name": "shaft1", "properties": []}, {"name": "shaft2", "properties": []}, {"name": "shaft3", "properties": []}, {"name": "gearbase", "properties": []}, {"name": "gearbase_hole1", "properties": []}, {"name": "gearbase_hole3", "properties": []}, {"name": "lampbase", "properties": []}, {"name": "chair_leg1", "properties": []}, {"name": "left_hand", "properties": []}], "constraints": [{"source": "gear2", "name": "can_insert_to", "target": "shaft2"}, {"source": "outwardgripper", "name": "can_manipulate", "target": "gear3"}, {"source": "outwardgripper", "name": "can_manipulate", "target": "gear2"}, {"source": "gear3", "name": "can_insert_to", "target": "shaft3"}, {"source": "shaft3", "name": "can_insert_to", "target": "gearbase_hole3"}, {"source": "shaft1", "name": "can_insert_to", "target": "gearbase_hole1"}, {"source": "parallelgripper", "name": "can_manipulate", "target": "gear1"}, {"source": "clampgripper", "name": "can_manipulate", "target": "shaft1"}, {"source": "gear1", "name": "can_insert_to", "target": "shaft1"}, {"source": "defaultgripper", "name": "can_manipulate", "target": "shaft3"}], "relations": [{"source": "shaft3", "name": "is_inserted_to", "target": "gearbase_hole3"}, {"source": "shaft1", "name": "is_inserted_to", "target": "gearbase_hole1"}, {"source": "left_hand", "name": "hold", "target": "inwardgripper"}, {"source": "gear2", "name": "is_inserted_to", "target": "shaft2"}]}, "result": {"summary": "selector to insert the gear1 into the shaft1 with the parallelgripper in the left_hand", "name": "selector: insert(left_hand, parallelgripper, gear1, shaft1)", "children": [{"summary": "the target is to make the gear1 be inserted into the shaft1", "name": "target: is_inserted_to(gear1, shaft1)"}, {"summary": "sequence to insert the gear1 into the shaft1 with the parallelgripper in the left_hand", "name": "sequence: insert(left_hand, parallelgripper, gear1, shaft1)", "children": [{"summary": "selector to change tool in left_hand from inwardgripper to parallelgripper", "name": "selector: change_tool(left_hand, inwardgripper, parallelgripper)", "children": [{"summary": "the target is to make left_hand hold parallelgripper", "name": "target: hold(left_hand, parallelgripper)"}, {"summary": "sequence to change tool in left_hand from inwardgripper to parallelgripper", "name": "sequence: change_tool(left_hand, inwardgripper, parallelgripper)", "children": [{"summary": "a precondition is left_hand is holding inwardgripper", "name": "precondition: hold(left_hand, inwardgripper)"}, {"summary": "a precondition is inwardgripper is empty", "name": "precondition: is_empty(inwardgripper)"}, {"summary": "the action to change tool in left_hand from inwardgripper to parallelgripper", "name": "action: change_tool(left_hand, inwardgripper, parallelgripper)"}]}]}, {"summary": "selector to pick up the gear1 with the parallelgripper in the left_hand", "name": "selector: pick_up(left_hand, parallelgripper, gear1)", "children": [{"summary": "the target is to make the parallelgripper hold the gear1", "name": "target: hold(parallelgripper, gear1)"}, {"summary": "sequence to pick up the gear1 with the parallelgripper in the left_hand", "name": "sequence: pick_up(left_hand, parallelgripper, gear1)", "children": [{"summary": "a precondition is the parallelgripper is empty", "name": "precondition: is_empty(parallelgripper)"}, {"summary": "a precondition is the left_hand is holding parallelgripper", "name": "precondition: hold(left_hand, parallelgripper)"}, {"summary": "the action to pick up the gear1 with the parallelgripper in the left_hand", "name": "action: pick_up(left_hand, parallelgripper, gear1)"}]}]}, {"summary": "the action to insert the gear1 into the shaft1 with the parallelgripper in the left_hand", "name": "action: insert(left_hand, parallelgripper, gear1, shaft1)"}]}]}} diff --git a/experiments/gearset1/baseline_result_handle.py b/experiments/gearset1/baseline_result_handle.py deleted file mode 100644 index 43099b1d..00000000 --- a/experiments/gearset1/baseline_result_handle.py +++ /dev/null @@ -1,84 +0,0 @@ -import json -import os -import json -import os - -""" -methods to handle the results of the baseline method. -use this to generate the problem set from the baseline_result.jsonl file -""" - - -def remove_entities(): - file_dir = os.path.dirname(os.path.abspath(__file__)) - result_dir = os.path.join(file_dir, "baseline_result.jsonl") - - with open(result_dir, "r") as file: - results = file.read() - data = [json.loads(line) for line in results.splitlines()] - - for item in data: - ws = item["initial_world_state"] - objs: list[dict] = ws["objects"] - cstrs = ws["constraints"] - for obj in objs: - if obj["name"] in [ - "parallel_box1", - "parallel_box2", - "outward_claw", - "inward_claw", - "no_tool", - ]: - print(f"Removing {obj['name']}") - objs.remove(obj) - - for cstr in cstrs: - if cstr["source"] in [ - "parallel_box1", - "parallel_box2", - "outward_claw", - "inward_claw", - "no_tool", - ]: - cstrs.remove(cstr) - elif cstr["target"] in [ - "parallel_box1", - "parallel_box2", - "outward_claw", - "inward_claw", - "no_tool", - ]: - cstrs.remove(cstr) - else: - pass - - with open(result_dir, "w") as file: - for item in data: - file.write(json.dumps(item)) - file.write("\n") - - -def handle_baseline_results(): - file_dir = os.path.dirname(os.path.abspath(__file__)) - result_dir = os.path.join(file_dir, "baseline_result.jsonl") - - if not os.path.exists(os.path.join(file_dir, "problem_set")): - os.makedirs(os.path.join(file_dir, "problem_set")) - - with open(result_dir, "r") as file: - results = file.read() - data = [json.loads(line) for line in results.splitlines()] - - i = 0 - for item in data: - i_str = str(i).zfill(3) - entity_path = os.path.join(file_dir, "problem_set", f"problem_{i_str}.json") - with open(entity_path, "w") as file: - file.write(json.dumps(item, indent=4)) - i += 1 - - -if __name__ == "__main__": - handle_baseline_results() - # remove_entities() - pass diff --git a/experiments/gearset1/behavior_tree.json b/experiments/gearset1/behavior_tree.json deleted file mode 100644 index ba968b94..00000000 --- a/experiments/gearset1/behavior_tree.json +++ /dev/null @@ -1,232 +0,0 @@ -{ - "summary": "selector to insert shaft1 into gearbase_hole1", - "name": "selector: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)", - "identifier": 1, - "type_name": "selector", - "children": [ - { - "summary": "condition node to check if shaft1 is inserted to gearbase_hole1", - "name": "target: is_inserted_to(shaft1, gearbase_hole1)", - "identifier": 2, - "type_name": "condition", - "conditions": [ - { - "object_name": "shaft1", - "property_name": "is_inserted_to", - "property_value": "gearbase_hole1", - "status": true - } - ] - }, - { - "summary": "sequence to insert shaft1 into gearbase_hole1", - "name": "sequence: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)", - "identifier": 3, - "type_name": "sequence", - "children": [ - { - "summary": "selector to load parallel_box1", - "name": "selector: load_tool(left_hand, parallel_box1)", - "identifier": 4, - "type_name": "selector", - "children": [ - { - "summary": "condition node to check if left_hand holds parallel_box1", - "name": "target: hold(left_hand, parallel_box1)", - "identifier": 5, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "parallel_box1", - "status": true - } - ] - }, - { - "summary": "sequence to load parallel_box1", - "name": "sequence: load_tool(left_hand, parallel_box1)", - "identifier": 6, - "type_name": "sequence", - "children": [ - { - "summary": "condition node to check if parallel_box1 is equippable", - "name": "precondition: is_equippable(parallel_box1)", - "identifier": 7, - "type_name": "condition", - "conditions": [ - { - "object_name": "parallel_box1", - "property_name": "is_equippable", - "property_value": null, - "status": true - } - ] - }, - { - "summary": "condition node to check if left hand is free", - "name": "precondition: is_free(left_hand)", - "identifier": 8, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "is_free", - "property_value": null, - "status": true - } - ] - }, - { - "summary": "action node to equip parallel_box1 to left hand", - "name": "action: load_tool(left_hand, parallel_box1)", - "identifier": 9, - "type_name": "action", - "effects": [ - { - "object_name": "left_hand", - "property_name": "is_free", - "property_value": null, - "status": false - }, - { - "object_name": "parallel_box1", - "property_name": "is_equippable", - "property_value": null, - "status": false - }, - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "parallel_box1", - "status": true - } - ] - } - ] - } - ] - }, - { - "summary": "selector to use left_hand with parallel_box1 to pick up shaft1", - "name": "selector: pick_up(left_hand, parallel_box1, shaft1)", - "identifier": 10, - "type_name": "selector", - "children": [ - { - "summary": "condition node to check if parallel_box1 holds shaft1", - "name": "target: hold(parallel_box1, shaft1)", - "identifier": 11, - "type_name": "condition", - "conditions": [ - { - "object_name": "parallel_box1", - "property_name": "hold", - "property_value": "shaft1", - "status": true - } - ] - }, - { - "summary": "sequence to use left hand with parallel_box1 to pick up shaft1", - "name": "sequence: pick_up(left_hand, parallel_box1, shaft1)", - "identifier": 12, - "type_name": "sequence", - "children": [ - { - "summary": "condition node to check if parallel_box1 is free", - "name": "precondition: is_free(parallel_box1)", - "identifier": 13, - "type_name": "condition", - "conditions": [ - { - "object_name": "parallel_box1", - "property_name": "is_free", - "property_value": null, - "status": true - } - ] - }, - { - "summary": "condition node to check if left hand holds parallel_box1", - "name": "precondition: hold(left_hand, parallel_box1)", - "identifier": 14, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "parallel_box1", - "status": true - } - ] - }, - { - "summary": "action node to use left hand with parallel_box1 to pick up shaft1", - "name": "action: pick_up(left_hand, parallel_box1, shaft1)", - "identifier": 15, - "type_name": "action", - "effects": [ - { - "object_name": "parallel_box1", - "property_name": "hold", - "property_value": "shaft1", - "status": true - }, - { - "object_name": "parallel_box1", - "property_name": "is_free", - "property_value": null, - "status": false - } - ] - } - ] - } - ] - }, - { - "summary": "condition node to check if shaft1 can be inserted to gearbase_hole1", - "name": "precondition: can_insert_to(shaft1, gearbase_hole1)", - "identifier": 16, - "type_name": "condition", - "conditions": [ - { - "object_name": "shaft1", - "property_name": "can_insert_to", - "property_value": "gearbase_hole1", - "status": true - } - ] - }, - { - "summary": "action node to use left_hand with parallel_box1 to insert shaft1 to gearbase_hole1", - "name": "action: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)", - "identifier": 17, - "type_name": "action", - "effects": [ - { - "object_name": "parallel_box1", - "property_name": "hold", - "property_value": "shaft1", - "status": false - }, - { - "object_name": "parallel_box1", - "property_name": "is_free", - "property_value": null, - "status": true - }, - { - "object_name": "shaft1", - "property_name": "is_inserted_to", - "property_value": "gearbase_hole1", - "status": true - } - ] - } - ] - } - ] - } \ No newline at end of file diff --git a/experiments/gearset1/domain_knowledge.txt b/experiments/gearset1/domain_knowledge.txt deleted file mode 100644 index ba5c088b..00000000 --- a/experiments/gearset1/domain_knowledge.txt +++ /dev/null @@ -1,88 +0,0 @@ -[user] -Following is the domain knowledge you need in pddl domain format, which includes: -- the object types in the world, -- the predicates for describing the states of the world, -- the actions that the robot can apply, with their precondtions and effects defined. -""" -(define (domain robot_assembly_problem-domain) - (:requirements :strips :typing) - (:types - thing - object - tool part hand - thing - ) - (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part)) - (:action pick_up - :parameters ( ?hand - hand ?tool - tool ?part - part) - :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part)) - :effect (and (hold ?tool ?part) (not (is_free ?tool)))) - (:action put_down - :parameters ( ?hand - hand ?tool - tool ?part - part) - :precondition (and (hold ?tool ?part) (hold ?hand ?tool)) - :effect (and (not (hold ?hand ?part)) (is_free ?tool))) - (:action place - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2))) - (:action detach - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2)))) - (:action insert - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2))) - (:action pull - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2)))) - (:action screw - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2))) - (:action unscrew - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2)))) - (:action load_tool - :parameters ( ?hand - hand ?tool - tool) - :precondition (and (is_equippable ?tool) (is_free ?hand)) - :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool))) - (:action unload_tool - :parameters ( ?hand - hand ?tool - tool) - :precondition (and (hold ?hand ?tool) (is_free ?tool)) - :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool)))) -) -""" -The precidates are explained as follows: -""" -"PREDICATE_LIST" -- is_free( or ): or is free and can hold something. (default: true) -- is_equippable(): is ready to be eqquipped by a robot hand. (default: true) -- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false) -- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false) -- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false) -- can_place_to(, ): can be placed into (placement assembly constraint) (default: false) -- hold(, ): is holding (default: false) -- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false) -- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false) -- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false) -""" - -The actions are explained as follows: -""" -"ROBOT_ACTION_LIST" -- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something. -- put_down(, , ): use to put down . As preconditions, should be holding and should be holding . -- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding . -- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into . -- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding . -- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into . -- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding . -- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into . -- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free. -- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free. -""" -the items in <> indicate the type of the instances needed for the predicates and actions. - -[assistant] -Understood. I will wait for further instructions before starting to work. diff --git a/experiments/gearset1/end2end_test.py b/experiments/gearset1/end2end_test.py deleted file mode 100644 index 5cbf08d8..00000000 --- a/experiments/gearset1/end2end_test.py +++ /dev/null @@ -1,227 +0,0 @@ -import json -import os -from pprint import pprint -from typing import List, Tuple, Annotated, TypedDict - - -""" -unit tree generation -""" - -os.environ["LANGCHAIN_TRACING_V2"] = "true" -os.environ["LANGCHAIN_ENDPOINT"] = "https://api.smith.langchain.com" -os.environ["LANGCHAIN_PROJECT"] = "kios_e2e" - -from kios_bt.bt_stewardship import BehaviorTreeStewardship -from kios_scene.scene_factory import SceneFactory -from kios_bt.bt_factory import BehaviorTreeFactory -from kios_robot.robot_interface import RobotInterface -from kios_world.world_interface import WorldInterface - -from kios_agent.llm_supporter import KiosLLMSupporter -from kios_agent.data_types import KiosPromptSkeleton - -from dotenv import load_dotenv - -from langchain_openai import ChatOpenAI - -from langchain.chains.openai_functions import ( - create_structured_output_runnable, - create_openai_fn_runnable, -) -from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder -from langchain_core.output_parsers import JsonOutputParser, StrOutputParser - -from langchain.prompts.pipeline import PipelinePromptTemplate -from langchain.prompts.prompt import PromptTemplate - -from langchain_community.document_loaders import TextLoader - -from langsmith import traceable - -load_dotenv() - -from kios_utils.pybt_test import generate_bt_stewardship, render_dot_tree -from kios_utils.pddl_problem_parser import parse_problem_init, parse_problem_objects - - -def render_bt(bt_json: dict): - test_class = BehaviorTreeFactory() - bt = test_class.from_json_to_simple_bt(bt_json) - # bt = test_class.from_json_to_tree_root(bt_json) - bt_stewardship = generate_bt_stewardship(bt) - # bt_stewardship.setup(timeout=15) - render_dot_tree(bt_stewardship) - - -def get_problem(problem_id: int) -> dict: - file_dir = os.path.dirname(os.path.abspath(__file__)) - problem_dir = os.path.join( - file_dir, "problem_set", f"problem_{str(problem_id).zfill(3)}.json" - ) - with open(problem_dir, "r") as file: - return json.load(file) - - -####################### dirs -current_dir = os.path.dirname(os.path.abspath(__file__)) -scene_path = os.path.join(current_dir, "scene.json") -# bt_json_file_path = os.path.join(current_dir, "behavior_tree.json") -world_state_path = os.path.join(current_dir, "world_state.json") -problem_path = os.path.join(current_dir, "gearset.problem") -domain_knowledge_path = os.path.join(current_dir, "domain_knowledge.txt") - -####################### problem -with open(problem_path, "r") as file: - problem = file.read() - -####################### scene -with open(scene_path, "r") as file: - scene_json_object = json.load(file) - -scene = SceneFactory().create_scene_from_json(scene_json_object) - -####################### world -world_interface = WorldInterface() -with open(world_state_path, "r") as file: - world_state_json = json.load(file) - world_interface.load_world_from_json(world_state_json) - -####################### robot -robot_interface = RobotInterface( - robot_address="127.0.0.1", - robot_port=12000, -) -robot_interface.setup_scene(scene) - -####################### bt_factory -bt_factory = BehaviorTreeFactory( - world_interface=world_interface, - robot_interface=robot_interface, -) - -####################### behavior_tree_stewardship -behavior_tree_stewardship = BehaviorTreeStewardship( - behaviortree_factory=bt_factory, - world_interface=world_interface, - robot_interface=robot_interface, -) - -# * kios data prompt skeleton dir -data_dir = os.environ.get("KIOS_DATA_DIR").format(username=os.getlogin()) -print(data_dir) -prompt_sk_dir = os.path.join(data_dir, "prompt_skeletons") -prompt_dir = os.path.join(data_dir, "prompts") - -# * end 2 end bt generation ppl -system_file = os.path.join(prompt_dir, "end_to_end_v3/system.txt") -task_file = os.path.join(prompt_dir, "end_to_end_v3/task.txt") -domain_file = os.path.join(prompt_dir, "end_to_end_v3/domain.txt") -behaviortree_file = os.path.join(prompt_dir, "end_to_end_v3/behaviortree.txt") -example_file = os.path.join(prompt_dir, "end_to_end_v3/example.txt") -output_format_file = os.path.join(prompt_dir, "end_to_end_v3/output_format.txt") -state_file = os.path.join(prompt_dir, "end_to_end_v3/state.txt") -template_file = os.path.join(prompt_dir, "end_to_end_v3/template.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) -with open(state_file, "r") as f: - state_ppt = PromptTemplate.from_template(f.read()) -with open(output_format_file, "r") as f: - output_format_ppt = PromptTemplate.from_template(f.read()) -with open(behaviortree_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - behaviortree_ppt = ppt_tmp.partial(input=f.read()) -with open(example_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - example_ppt = ppt_tmp.partial(input=f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {state} - - {behaviortree} - - {output_format} - - {example} - - {template} - - {format_instructions} - """ -) - -parser = JsonOutputParser() - -format_instructions = PromptTemplate.from_template("""{input}""").partial( - input=parser.get_format_instructions() -) - -e2e_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ("behaviortree", behaviortree_ppt), - ("output_format", output_format_ppt), - ("example", example_ppt), - ("state", state_ppt), - ("format_instructions", format_instructions), - ], -) - -e2e_chain = ( - e2e_ppt_ppl - # | ChatOpenAI(model="gpt-3.5-turbo-0125", temperature=0) - # | ChatOpenAI(model="ft:gpt-3.5-turbo-0125:kifabrik-mirmi::8y1cXwVw", temperature=0) - # | ChatOpenAI( - # model="ft:gpt-3.5-turbo-0125:kifabrik-mirmi:kios-ut-gen-v2:8z2KbPsr", - # temperature=0, - # ) - | ChatOpenAI(model="gpt-4", temperature=0) - | JsonOutputParser() -) - - -@traceable(name="e2e_test_baselines") -def e2e_test(problem_id: int): - task = get_problem(problem_id) - bt = e2e_chain.invoke( - { - "target": task["target"], - "initial_state": task["initial_world_state"], - } - ) - - result, node = behavior_tree_stewardship.sk_sim_run( - task["initial_world_state"], - bt.get("behavior_tree"), - ) - - pprint(result.to_json()) - pprint(f'LLM thought flow: {bt["thought"]}') - pprint(f'the action sequence: {bt["action_sequence"]}') - render_bt(bt.get("behavior_tree")) - - -# def e2e_test_baselines(): -# for i in range(17): -# e2e_test(i) - - -if __name__ == "__main__": - pass - # e2e_test_baselines() diff --git a/experiments/gearset1/gearset.domain b/experiments/gearset1/gearset.domain deleted file mode 100644 index 39c677b0..00000000 --- a/experiments/gearset1/gearset.domain +++ /dev/null @@ -1,48 +0,0 @@ -(define (domain robot_assembly_problem-domain) - (:requirements :strips :typing) - (:types - thing - object - tool part hand - thing - ) - (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part)) - (:action pick_up - :parameters ( ?hand - hand ?tool - tool ?part - part) - :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part)) - :effect (and (hold ?tool ?part) (not (is_free ?tool)))) - (:action put_down - :parameters ( ?hand - hand ?tool - tool ?part - part) - :precondition (and (hold ?tool ?part) (hold ?hand ?tool)) - :effect (and (not (hold ?hand ?part)) (is_free ?tool))) - (:action place - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2))) - (:action detach - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2)))) - (:action insert - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2))) - (:action pull - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2)))) - (:action screw - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2))) - (:action unscrew - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2)))) - (:action load_tool - :parameters ( ?hand - hand ?tool - tool) - :precondition (and (is_equippable ?tool) (is_free ?hand)) - :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool))) - (:action unload_tool - :parameters ( ?hand - hand ?tool - tool) - :precondition (and (hold ?hand ?tool) (is_free ?tool)) - :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool)))) -) \ No newline at end of file diff --git a/experiments/gearset1/gearset.problem b/experiments/gearset1/gearset.problem deleted file mode 100644 index 7e1cecc9..00000000 --- a/experiments/gearset1/gearset.problem +++ /dev/null @@ -1,10 +0,0 @@ -(define (problem robot_assembly_problem-problem) - (:domain robot_assembly_problem-domain) - (:objects - parallel_box1 parallel_box2 inward_claw outward_claw no_tool - tool - gear1 gear2 gear3 shaft1 shaft2 shaft3 gearbase gearbase_hole1 gearbase_hole3 - part - left_hand - hand - ) - (:init (can_manipulate parallel_box2 gear1) (can_manipulate outward_claw gear2) (can_manipulate outward_claw gear3) (can_manipulate no_tool shaft3) (can_manipulate parallel_box1 shaft1) (can_insert_to shaft1 gearbase_hole1) (can_insert_to shaft3 gearbase_hole3) (can_insert_to gear3 shaft3) (can_insert_to gear2 shaft2) (can_insert_to gear1 shaft1)) - (:goal (and )) - ) \ No newline at end of file diff --git a/experiments/gearset1/gearset1_graph copy.py b/experiments/gearset1/gearset1_graph copy.py deleted file mode 100644 index 71cbe082..00000000 --- a/experiments/gearset1/gearset1_graph copy.py +++ /dev/null @@ -1,526 +0,0 @@ -import json -import os - -""" -don't use this. it is just a backup. -""" - -from kios_bt.bt_stewardship import BehaviorTreeStewardship -from kios_scene.scene_factory import SceneFactory -from kios_bt.bt_factory import BehaviorTreeFactory -from kios_robot.robot_interface import RobotInterface -from kios_world.world_interface import WorldInterface - -from kios_agent.llm_supporter import KiosLLMSupporter -from kios_agent.data_types import KiosPromptSkeleton - -from kios_agent.kios_tools import ( - BehaviorTreeExecutorTool, - BehaviorTreeSimulatorTool, - WorldStateQueryTool, -) - -from langchain_core.pydantic_v1 import BaseModel, Field -from typing import List, Tuple, Annotated, TypedDict -import operator - -from dotenv import load_dotenv - -from langchain import hub -from langchain.agents import create_openai_functions_agent, AgentExecutor -from langchain_openai import ChatOpenAI -from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder -from langchain_core.output_parsers import JsonOutputParser, StrOutputParser - -from langchain.chains.openai_functions import ( - create_structured_output_runnable, - create_openai_fn_runnable, -) -from langchain_core.utils.function_calling import convert_to_openai_function - -from langchain.agents.format_scratchpad import format_to_openai_function_messages - -from langchain_core.agents import AgentActionMessageLog, AgentFinish - -from langgraph.graph import StateGraph, END - -from langchain_core.messages import HumanMessage - - -load_dotenv() - -from langsmith import traceable - -os.environ["LANGCHAIN_TRACING_V2"] = "true" -os.environ["LANGCHAIN_ENDPOINT"] = "https://api.smith.langchain.com" -os.environ["LANGCHAIN_PROJECT"] = "kios_agent" - - -####################### dirs -current_dir = os.path.dirname(os.path.abspath(__file__)) -scene_path = os.path.join(current_dir, "scene.json") -# bt_json_file_path = os.path.join(current_dir, "behavior_tree.json") -world_state_path = os.path.join(current_dir, "world_state.json") -problem_path = os.path.join(current_dir, "gearset.problem") - - -####################### problem -with open(problem_path, "r") as file: - problem = file.read() - -####################### scene -with open(scene_path, "r") as file: - scene_json_object = json.load(file) - -scene = SceneFactory().create_scene_from_json(scene_json_object) - -####################### world -world_interface = WorldInterface() -with open(world_state_path, "r") as file: - world_state_json_object = json.load(file) - world_interface.load_world_from_json(world_state_json_object) - -####################### robot -robot_interface = RobotInterface( - robot_address="127.0.0.1", - robot_port=12000, -) -robot_interface.setup_scene(scene) - -####################### bt_factory -bt_factory = BehaviorTreeFactory( - world_interface=world_interface, - robot_interface=robot_interface, -) - -####################### behavior_tree_stewardship -behavior_tree_stewardship = BehaviorTreeStewardship( - behaviortree_factory=bt_factory, - world_interface=world_interface, - robot_interface=robot_interface, -) - -####################### * tools -# bt_executor_tool = BehaviorTreeExecutorTool(behavior_tree_stewardship) - -bt_sim_tool = BehaviorTreeSimulatorTool(metadata={"bt_stw": behavior_tree_stewardship}) - -# world_state_query_tool = WorldStateQueryTool(behavior_tree_stewardship) - -executor_tools = [bt_sim_tool] - -# * kios data prompt skeleton dir -data_dir = os.environ.get("KIOS_DATA_DIR").format(username=os.getlogin()) -print(data_dir) -prompt_sk_dir = os.path.join(data_dir, "prompt_skeletons") - -# * refiner chain -llm_spt_re_sk = KiosLLMSupporter() -with open(os.path.join(prompt_sk_dir, "skeleton_refiner.json"), "r") as f: - re_sk_pptsk_json = json.load(f) - re_sk_pptsk = KiosPromptSkeleton.from_json(re_sk_pptsk_json) - llm_spt_re_sk.initialize_from_prompt_skeleton(re_sk_pptsk) -re_sk_ppt = llm_spt_re_sk.create_prompt() -re_sk_llm = ChatOpenAI(model_name=llm_spt_re_sk.model_name, temperature=0) - -re_sk_chain = re_sk_ppt | re_sk_llm | JsonOutputParser() - - -# * skeleton generator chain -llm_spt_skeleton_generator = KiosLLMSupporter() -with open(os.path.join(prompt_sk_dir, "skeleton_generator.json"), "r") as f: - skeleton_generator_pptsk_json = json.load(f) - skeleton_generator_pptsk = KiosPromptSkeleton.from_json( - skeleton_generator_pptsk_json - ) - llm_spt_skeleton_generator.initialize_from_prompt_skeleton(skeleton_generator_pptsk) -skeleton_generator_ppt = llm_spt_skeleton_generator.create_prompt() -skeleton_generator_llm = ChatOpenAI( - model_name=llm_spt_skeleton_generator.model_name, temperature=0 -) - -skeleton_generator_chain = ( - skeleton_generator_ppt - | skeleton_generator_llm - # | StrOutputParser() - | JsonOutputParser() -) - -# ! DISCARDED behavior tree generator chain -# bt_gen_chain = skeleton_generator_chain | re_sk_chain - -############################################### * behavior tree executor agent -# ! NOW SUBSTITUTED BY THE FAKE EXECUTOR FUNCTION -# bt_exe_ppt = hub.pull("hwchase17/openai-functions-agent") -# bt_exe_ppt = ChatPromptTemplate.from_messages( -# [ -# ("system", "You are a helpful assistant"), -# ("user", "The behavior tree is: {behavior_tree}"), -# ("user", "The world state is: {world_state}"), -# MessagesPlaceholder(variable_name="agent_scratchpad"), -# ] -# ) - -# bt_exe_llm = ChatOpenAI(model="gpt-3.5-turbo", temperature=0) - - -# def executor_agent_parse(output): -# # If no function was invoked, return to user -# if "function_call" not in output.additional_kwargs: -# return AgentFinish(return_values={"output": output.content}, log=output.content) - -# # Parse out the function call -# function_call = output.additional_kwargs["function_call"] -# name = function_call["name"] -# inputs = json.loads(function_call["arguments"]) - -# # If the Response function was invoked, return to the user with the function inputs -# if name == "ExecutorResponse": -# return AgentFinish(return_values=inputs, log=str(function_call)) -# # Otherwise, return an agent action -# else: -# return AgentActionMessageLog( -# tool=name, tool_input=inputs, log="", message_log=[output] -# ) - - -# class ExecutorResponse(BaseModel): -# """the result of the behavior tree execution""" - -# hasSucceeded: bool = Field(description="If the behavior tree has succeeded or not") -# world_state: dict[str, list[dict[str, str]]] = Field( -# description="The final world state after the execution" -# ) - - -# bt_exe_llm_with_tools = bt_exe_llm.bind_functions( -# [convert_to_openai_function(t) for t in executor_tools] -# ) - - -# ! BUG -# bt_exe_agent = ( -# { -# "input": lambda x: x["input"], -# # Format agent scratchpad from intermediate steps -# "agent_scratchpad": lambda x: format_to_openai_function_messages( -# x["intermediate_steps"] -# ), -# } -# | bt_exe_ppt -# | bt_exe_llm_with_tools -# | executor_agent_parse -# ) - -# bt_exe_agent_executor = AgentExecutor( -# tools=executor_tools, agent=bt_exe_agent, verbose=True -# ) - - -# * the graph state -class PlanExecuteState(TypedDict): - plan: List[str] - behavior_tree_skeleton: dict # ! not sure if use this or not. - behavior_tree: dict - world_state: Annotated[ - List[dict], operator.add - ] # ! to add, you need to make world_state a list of dict - # = Field(default=None) - past_steps: Annotated[List[Tuple], operator.add] - response: str # ! not sure if use this or not. - # to_user: bool - user_input: str - problem: str - - -##################################################### * planner -# * output schema of the planner -class Plan(BaseModel): - """Plan to follow in future""" - - steps: List[str] = Field( - description="a list of different steps to follow, should be in sorted order" - ) - - -planner_prompt = ChatPromptTemplate.from_template( - """ - The user input should be an instruction (or several instructions) of the robot assembly task. \ - For the given user input, summarize and come up with a bullet list task plan. \ -This plan should involve individual assembly step(s) of robot assembly task.\ -Do not add any superfluous steps. \ -Make sure that each step has all the information needed - do not skip steps.\ - -You should only use assembly actions as the action in the steps. -The available assembly actions are:\ -- insert something into something\ -- place something onto something\ -- screw something into something\ - -Example:\ -user_input: "you should insert the red block into the blue hole"\ -plan: "insert the red block into the blue hole"\ - -user_input: "intert the blub into the base, then put the cover onto it."\ -plan: ["insert the blub into the base", "put the cover onto the blub"]\ - -world_state: {world_state}\ -user_input: {user_input}\ -""" -) -planner = create_structured_output_runnable( - Plan, ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), planner_prompt -) - - -##################################################### * plan_updaterner - - -class UpdaterResponse(BaseModel): - """used to response to the user for asking for more inputs.""" - - response: str - - -plan_updater_prompt = ChatPromptTemplate.from_template( - """For the given objective, come up with a simple step by step plan. \ -This plan should involve individual tasks, that if executed correctly will yield the correct answer. Do not add any superfluous steps. \ -The result of the final step should be the final answer. Make sure that each step has all the information needed - do not skip steps. - -Your objective was this: -{user_input} - -Your original plan was this: -{plan} - -You have currently done the follow steps: -{past_steps} - -The world state is: -{world_state} - -Read the world state. Check the original plan from the first step and update your plan with the steps need to be conducted next according to the world state.\ -If no more steps are needed to achieve the objective defined by user input, respond to the user for more input. Otherwise, fill out the plan.\ -Only add steps to the plan that still NEED to be done. Do not return previously done steps as part of the plan.""" -) - -plan_updater = create_openai_fn_runnable( - [Plan, UpdaterResponse], # * here two schemas are used - ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), - plan_updater_prompt, -) - - -##################################################### * graph node functions -async def user_input_step(state: PlanExecuteState): - """ - get the input from the user - """ - print(f"-----user_input_step-----") - if state["response"] is not None: - user_input = input(state["response"]) - state["response"] = None - else: - user_input = input("Your next instruction: ") - - return { - "user_input": user_input, - } - - -def user_input_should_end(state: PlanExecuteState): - """ - if the user input is empty, then end - """ - print(f"-----user_input_should_end-----") - - # print(f'user_input: {state["user_input"]}') - # print(f'world_state: {state["world_state"]}') - - if not state["user_input"] or state["user_input"] == "": - return True - else: - return False - - -async def behavior_tree_generate_step(state: PlanExecuteState): - """ - generate the behavior tree based on the instruction - """ - print(f"-----behavior_tree_generate_step-----") - - instruction = state["plan"][0] - latest_world_state = state["world_state"][-1] - - skeleton = await skeleton_generator_chain.ainvoke( - { - "problem": state["problem"], - "world_state": latest_world_state, - "instructions": instruction, - } - ) - - response = await re_sk_chain.ainvoke( - { - "input": skeleton, - } - ) - - behavior_tree = response.get("task_plan").get("behavior_tree") - - init_world_state = [response.get("initial_state")] - - return { - "behavior_tree": behavior_tree, - "world_state": init_world_state, # * this is necessary because the constraints in problem - } - - -async def behavior_tree_execute_step(state: PlanExecuteState): - """ - execute the first step of the plan, append the result to the past steps - """ - print(f"-----behavior_tree_execute_step-----") - - behavior_tree = state["behavior_tree"] - latest_world_state = state["world_state"][-1] - - # exe_input = { - # "behavior_tree": behavior_tree, - # "world_state": latest_world_state, - # } - - # agent_response = await bt_exe_agent_executor.ainvoke( - # { - # "input": exe_input, - # } - # ) - - response = behavior_tree_stewardship.fake_run( - world_state=latest_world_state, bt_json=behavior_tree - ) - - # hasSucceeded = response["hasSucceeded"] - # new_world_state = [response["world_state"]] - hasSucceeded = response.result - new_world_state = [response.world_state] - the_step = state["plan"][0] - - return { - "past_steps": (the_step, hasSucceeded), # ! why here not a list of tuple? - "world_state": new_world_state, - } - - -async def plan_step(state: PlanExecuteState): - """ - plan the steps based on user input and world state - """ - print(f"-----plan_step-----") - - plan = await planner.ainvoke( - { - "user_input": state["user_input"], - "world_state": state["world_state"], - } - ) - return {"plan": plan.steps} - - -async def plan_updater_step(state: PlanExecuteState): - """ - if return a response, then success, response the use, end. - otherwise, return the updated newplan (normally the same as the old plan with the first step popped out. - """ - print(f"-----plan_updater_step-----") - - output = await plan_updater.ainvoke(state) - if isinstance( - output, UpdaterResponse - ): # * determine if it is time to response and end - return { - "response": "Your last instruction has been finished." - } # * Don't need to update. - else: - return {"plan": output.steps} # * update the plan - - -def plan_updater_should_end(state: PlanExecuteState): - """ - end router - """ - print(f"-----plan_updater_should_end-----") - - if state["response"]: - return True - else: - return False - - -##################################################### * construct the graph - -workflow = StateGraph(PlanExecuteState) - -workflow.add_node("planner", plan_step) - -workflow.add_node("behavior_tree_generator", behavior_tree_generate_step) - -workflow.add_node("behavior_tree_executor", behavior_tree_execute_step) - -workflow.add_node("plan_updater", plan_updater_step) - -workflow.add_node("user_input_node", user_input_step) - -workflow.set_entry_point("user_input_node") - -# workflow.add_edge("user_input_node", "planner") - -workflow.add_edge( - "planner", "behavior_tree_generator" -) # ! may need to add a condition here - -workflow.add_edge("behavior_tree_generator", "behavior_tree_executor") - -workflow.add_edge("behavior_tree_executor", "plan_updater") - -workflow.add_conditional_edges( - "plan_updater", - plan_updater_should_end, - { - True: "user_input_node", - False: "behavior_tree_generator", - }, -) - -workflow.add_conditional_edges( - "user_input_node", - user_input_should_end, - { - True: END, - False: "planner", - }, -) - -app = workflow.compile() - - -config = {"recursion_limit": 50} - - -inputs = { - "world_state": [world_state_json_object], - "problem": problem, -} - - -async def core_run(): - async for event in app.astream(inputs, config=config): - for k, v in event.items(): - if k != "__end__": - print(v) - - -if __name__ == "__main__": - import asyncio - - asyncio.run(core_run()) diff --git a/experiments/gearset1/gearset1_graph.py b/experiments/gearset1/gearset1_graph.py deleted file mode 100644 index a4c47366..00000000 --- a/experiments/gearset1/gearset1_graph.py +++ /dev/null @@ -1,510 +0,0 @@ -import json -import os - -os.environ["LANGCHAIN_TRACING_V2"] = "true" -os.environ["LANGCHAIN_ENDPOINT"] = "https://api.smith.langchain.com" -os.environ["LANGCHAIN_PROJECT"] = "kios_agent" - -from kios_bt.bt_stewardship import BehaviorTreeStewardship -from kios_scene.scene_factory import SceneFactory -from kios_bt.bt_factory import BehaviorTreeFactory -from kios_robot.robot_interface import RobotInterface -from kios_world.world_interface import WorldInterface - -from kios_agent.llm_supporter import KiosLLMSupporter -from kios_agent.data_types import KiosPromptSkeleton - -from kios_agent.kios_tools import ( - BehaviorTreeExecutorTool, - BehaviorTreeSimulatorTool, - WorldStateQueryTool, -) - -from typing import List, Tuple, Annotated, TypedDict -import operator - -from dotenv import load_dotenv - -from langchain import hub - -from langchain_openai import ChatOpenAI - -from langchain.chains.openai_functions import ( - create_structured_output_runnable, - create_openai_fn_runnable, -) -from langchain_core.utils.function_calling import convert_to_openai_function -from langchain_core.agents import AgentActionMessageLog, AgentFinish -from langchain_core.messages import HumanMessage -from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder -from langchain_core.output_parsers import JsonOutputParser, StrOutputParser -from langchain_core.pydantic_v1 import BaseModel, Field - -from langchain.agents.format_scratchpad import format_to_openai_function_messages -from langchain.agents import create_openai_functions_agent, AgentExecutor - -from langchain.prompts.pipeline import PipelinePromptTemplate -from langchain.prompts.prompt import PromptTemplate - -from langgraph.graph import StateGraph, END - -from langsmith import traceable - -load_dotenv() - -from kios_utils.pybt_test import generate_bt_stewardship, render_dot_tree - - -def test_bt(bt_json: json): - test_class = BehaviorTreeFactory() - bt = test_class.from_json_to_simple_bt(bt_json) - # bt = test_class.from_json_to_tree_root(bt_json) - bt_stewardship = generate_bt_stewardship(bt) - # bt_stewardship.setup(timeout=15) - render_dot_tree(bt_stewardship) - - -####################### dirs -current_dir = os.path.dirname(os.path.abspath(__file__)) -scene_path = os.path.join(current_dir, "scene.json") -# bt_json_file_path = os.path.join(current_dir, "behavior_tree.json") -world_state_path = os.path.join(current_dir, "world_state.json") -problem_path = os.path.join(current_dir, "gearset.problem") - - -####################### problem -with open(problem_path, "r") as file: - problem = file.read() - -####################### scene -with open(scene_path, "r") as file: - scene_json_object = json.load(file) - -scene = SceneFactory().create_scene_from_json(scene_json_object) - -####################### world -world_interface = WorldInterface() -with open(world_state_path, "r") as file: - world_state_json_object = json.load(file) - world_interface.load_world_from_json(world_state_json_object) - -####################### robot -robot_interface = RobotInterface( - robot_address="127.0.0.1", - robot_port=12000, -) -robot_interface.setup_scene(scene) - -####################### bt_factory -bt_factory = BehaviorTreeFactory( - world_interface=world_interface, - robot_interface=robot_interface, -) - -####################### behavior_tree_stewardship -behavior_tree_stewardship = BehaviorTreeStewardship( - behaviortree_factory=bt_factory, - world_interface=world_interface, - robot_interface=robot_interface, -) - -####################### * tools -# bt_executor_tool = BehaviorTreeExecutorTool(behavior_tree_stewardship) - -bt_sim_tool = BehaviorTreeSimulatorTool(metadata={"bt_stw": behavior_tree_stewardship}) - -# world_state_query_tool = WorldStateQueryTool(behavior_tree_stewardship) - -executor_tools = [bt_sim_tool] - -# * kios data prompt skeleton dir -data_dir = os.environ.get("KIOS_DATA_DIR").format(username=os.getlogin()) -print(data_dir) -prompt_sk_dir = os.path.join(data_dir, "prompt_skeletons") -prompt_dir = os.path.join(data_dir, "prompts") - -# * refiner chain -llm_spt_re_sk = KiosLLMSupporter() -with open(os.path.join(prompt_sk_dir, "skeleton_refiner.json"), "r") as f: - re_sk_pptsk_json = json.load(f) - re_sk_pptsk = KiosPromptSkeleton.from_json(re_sk_pptsk_json) - llm_spt_re_sk.initialize_from_prompt_skeleton(re_sk_pptsk) -re_sk_ppt = llm_spt_re_sk.create_prompt() -re_sk_llm = ChatOpenAI(model_name=llm_spt_re_sk.model_name, temperature=0) - -re_sk_chain = re_sk_ppt | re_sk_llm | JsonOutputParser() - - -# * skeleton generator chain -llm_spt_skeleton_generator = KiosLLMSupporter() -with open(os.path.join(prompt_sk_dir, "skeleton_generator.json"), "r") as f: - skeleton_generator_pptsk_json = json.load(f) - skeleton_generator_pptsk = KiosPromptSkeleton.from_json( - skeleton_generator_pptsk_json - ) - llm_spt_skeleton_generator.initialize_from_prompt_skeleton(skeleton_generator_pptsk) -skeleton_generator_ppt = llm_spt_skeleton_generator.create_prompt() -skeleton_generator_llm = ChatOpenAI( - model_name=llm_spt_skeleton_generator.model_name, temperature=0 -) - -skeleton_generator_chain = ( - skeleton_generator_ppt - | skeleton_generator_llm - # | StrOutputParser() - | JsonOutputParser() -) - - -# * the graph state -class PlanExecuteState(TypedDict): - plan: List[str] - behavior_tree_skeleton: dict # ! not sure if use this or not. - behavior_tree: dict - world_state: Annotated[ - List[dict], operator.add - ] # ! to add, you need to make world_state a list of dict - # = Field(default=None) - past_steps: Annotated[List[Tuple], operator.add] - response: str # ! not sure if use this or not. - # to_user: bool - user_input: str - problem: str - - -##################################################### * planner -# * output schema of the planner -class Plan(BaseModel): - """Plan to follow in future""" - - steps: List[str] = Field( - description="a list of different steps to follow, should be in sorted order" - ) - - -template_file = os.path.join(prompt_dir, "planner/template.txt") -task_file = os.path.join(prompt_dir, "planner/task.txt") -system_file = os.path.join(prompt_dir, "planner/system.txt") -domain_file = os.path.join(prompt_dir, "planner/domain.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {template} - """ -) - -planner_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ], -) - -planner = create_structured_output_runnable( - Plan, ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), planner_ppt_ppl -) - - -##################################################### * plan_updaterner -class UpdaterResponse(BaseModel): - """used to response to the user for asking for more inputs.""" - - response: str - - -template_file = os.path.join(prompt_dir, "plan_updater/template.txt") -task_file = os.path.join(prompt_dir, "plan_updater/task.txt") -system_file = os.path.join(prompt_dir, "plan_updater/system.txt") -domain_file = os.path.join(prompt_dir, "plan_updater/domain.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {template} - """ -) - -plan_updater_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ], -) - -plan_updater = create_openai_fn_runnable( - [Plan, UpdaterResponse], # * here two schemas are used - ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), - plan_updater_ppt_ppl, -) - - -##################################################### * graph node functions -@traceable(name="user_input_node_step") -async def user_input_step(state: PlanExecuteState): - """ - get the input from the user - """ - print(f"-----user_input_step-----") - if state["response"] is not None: - user_input = input(state["response"]) - state["response"] = None - else: - user_input = input("Your next instruction: ") - - return { - "user_input": user_input, - } - - -@traceable(name="behavior_tree_generate_step") -async def behavior_tree_generate_step(state: PlanExecuteState): - """ - generate the behavior tree based on the instruction - """ - print(f"-----behavior_tree_generate_step-----") - - instruction = state["plan"][0] - latest_world_state = state["world_state"][-1] - - skeleton = await skeleton_generator_chain.ainvoke( - { - "problem": state["problem"], - "world_state": latest_world_state, - "instructions": instruction, - } - ) - - # sk_json = JsonOutputParser().invoke(skeleton) - sk_json = skeleton - - behavior_tree_sk = sk_json["task_plan"]["behavior_tree"] - - test_bt(behavior_tree_sk) - - pause = input("paused here") - - # TODO OBJECT CONCENTRATION - - response = await re_sk_chain.ainvoke( - { - "input": skeleton, - } - ) - - behavior_tree = response.get("task_plan").get("behavior_tree") - - init_world_state = [response.get("initial_state")] - - return { - "behavior_tree": behavior_tree, - "world_state": init_world_state, # ! this is necessary because the constraints in problem - } - - -# ! BBWARN INITIALIZE THE CONSTRAINT IN THE PROMPT OR IN THE INITIALIZATION NODE - - -@traceable(name="behavior_tree_execute_step") -async def behavior_tree_execute_step(state: PlanExecuteState): - """ - execute the first step of the plan, append the result to the past steps - """ - print(f"-----behavior_tree_execute_step-----") - - behavior_tree = state["behavior_tree"] - latest_world_state = state["world_state"][-1] - - # exe_input = { - # "behavior_tree": behavior_tree, - # "world_state": latest_world_state, - # } - - # agent_response = await bt_exe_agent_executor.ainvoke( - # { - # "input": exe_input, - # } - # ) - - response = behavior_tree_stewardship.fake_run( - world_state=latest_world_state, bt_json=behavior_tree - ) - - # hasSucceeded = response["hasSucceeded"] - # new_world_state = [response["world_state"]] - hasSucceeded = response.result - new_world_state = [response.world_state] - the_step = state["plan"][0] - - return { - "past_steps": ( - the_step, - hasSucceeded, - ), # * only one because new plan will be generated and old steps are all removed - "world_state": new_world_state, - } - - -@traceable(name="planner_step") -async def planner_step(state: PlanExecuteState): - """ - plan the steps based on user input and world state - """ - print(f"-----plan_step-----") - - plan = await planner.ainvoke( - { - "user_input": state["user_input"], - "world_state": state["world_state"], - } - ) - return {"plan": plan.steps} - - -@traceable(name="plan_updater_step") -async def plan_updater_step(state: PlanExecuteState): - """ - if return a response, then success, response the use, end. - otherwise, return the updated newplan (normally the same as the old plan with the first step popped out. - """ - print(f"-----plan_updater_step-----") - - output = await plan_updater.ainvoke(state) - if isinstance( - output, UpdaterResponse - ): # * determine if it is time to response and end - return { - "response": "Your last instruction has been finished." - } # * Don't need to update. - else: - return {"plan": output.steps} # * update the plan - - -##################################################### * construct the graph - -workflow = StateGraph(PlanExecuteState) - -workflow.add_node("planner", planner_step) - -workflow.add_node("behavior_tree_generator", behavior_tree_generate_step) - -workflow.add_node("behavior_tree_executor", behavior_tree_execute_step) - -workflow.add_node("plan_updater", plan_updater_step) - -workflow.add_node("user_input_node", user_input_step) - -workflow.set_entry_point("user_input_node") - -# workflow.add_edge("user_input_node", "planner") - -workflow.add_edge( - "planner", "behavior_tree_generator" -) # ! may need to add a condition here - -workflow.add_edge("behavior_tree_generator", "behavior_tree_executor") - -workflow.add_edge("behavior_tree_executor", "plan_updater") - - -def plan_updater_should_end(state: PlanExecuteState): - """ - end router - """ - print(f"-----plan_updater_should_end-----") - - if state["response"]: - return True - else: - return False - - -workflow.add_conditional_edges( - "plan_updater", - plan_updater_should_end, - { - True: "user_input_node", - False: "behavior_tree_generator", - }, -) - - -def user_input_should_end(state: PlanExecuteState): - """ - if the user input is empty, then end - """ - print(f"-----user_input_should_end-----") - - # print(f'user_input: {state["user_input"]}') - # print(f'world_state: {state["world_state"]}') - - if not state["user_input"] or state["user_input"] == "": - return True - else: - return False - - -workflow.add_conditional_edges( - "user_input_node", - user_input_should_end, - { - True: END, - False: "planner", - }, -) - -app = workflow.compile() - - -# config = {"recursion_limit": 50} - -inputs = { - "world_state": [world_state_json_object], - "problem": problem, -} - - -async def core_run(): - async for event in app.astream( - inputs, - # config=config, - ): - for k, v in event.items(): - if k != "__end__": - print(v) - - -if __name__ == "__main__": - import asyncio - - asyncio.run(core_run()) diff --git a/experiments/gearset1/gearset1_graph_bu_txt.py b/experiments/gearset1/gearset1_graph_bu_txt.py deleted file mode 100644 index 125e4b16..00000000 --- a/experiments/gearset1/gearset1_graph_bu_txt.py +++ /dev/null @@ -1,423 +0,0 @@ -import json -import os - -os.environ["LANGCHAIN_TRACING_V2"] = "true" -os.environ["LANGCHAIN_ENDPOINT"] = "https://api.smith.langchain.com" -os.environ["LANGCHAIN_PROJECT"] = "kios_instruction" - - -""" -... -""" - -from kios_bt.bt_stewardship import BehaviorTreeStewardship -from kios_scene.scene_factory import SceneFactory -from kios_bt.bt_factory import BehaviorTreeFactory -from kios_robot.robot_interface import RobotInterface -from kios_world.world_interface import WorldInterface - -from kios_agent.llm_supporter import KiosLLMSupporter -from kios_agent.data_types import KiosPromptSkeleton - -from typing import List, Tuple, Annotated, TypedDict -import operator - -from dotenv import load_dotenv - -from langchain_openai import ChatOpenAI - -from langchain.chains.openai_functions import ( - create_structured_output_runnable, - create_openai_fn_runnable, -) - -from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder -from langchain_core.output_parsers import JsonOutputParser, StrOutputParser -from langchain_core.pydantic_v1 import BaseModel, Field - -from langchain.agents.format_scratchpad import format_to_openai_function_messages -from langchain.agents import create_openai_functions_agent, AgentExecutor - -from langchain.prompts.pipeline import PipelinePromptTemplate -from langchain.prompts.prompt import PromptTemplate - -from langgraph.graph import StateGraph, END - -from langsmith import traceable - -load_dotenv() - - -####################### dirs -current_dir = os.path.dirname(os.path.abspath(__file__)) -scene_path = os.path.join(current_dir, "scene.json") -# bt_json_file_path = os.path.join(current_dir, "behavior_tree.json") -world_state_path = os.path.join(current_dir, "world_state.json") -problem_path = os.path.join(current_dir, "gearset.problem") - - -####################### problem -with open(problem_path, "r") as file: - problem = file.read() - -####################### scene -with open(scene_path, "r") as file: - scene_json_object = json.load(file) - -scene = SceneFactory().create_scene_from_json(scene_json_object) - -####################### world -world_interface = WorldInterface() -with open(world_state_path, "r") as file: - world_state_json_object = json.load(file) - world_interface.load_world_from_json(world_state_json_object) - -####################### robot -robot_interface = RobotInterface( - robot_address="127.0.0.1", - robot_port=12000, -) -robot_interface.setup_scene(scene) - -####################### bt_factory -bt_factory = BehaviorTreeFactory( - world_interface=world_interface, - robot_interface=robot_interface, -) - -####################### behavior_tree_stewardship -behavior_tree_stewardship = BehaviorTreeStewardship( - behaviortree_factory=bt_factory, - world_interface=world_interface, - robot_interface=robot_interface, -) - -####################### * tools - -# * kios data prompt skeleton dir -data_dir = os.environ.get("KIOS_DATA_DIR").format(username=os.getlogin()) -print(data_dir) -prompt_sk_dir = os.path.join(data_dir, "prompt_skeletons") -prompt_dir = os.path.join(data_dir, "prompts") - -# * refiner chain -llm_spt_re_sk = KiosLLMSupporter() -with open(os.path.join(prompt_sk_dir, "skeleton_refiner.json"), "r") as f: - re_sk_pptsk_json = json.load(f) - re_sk_pptsk = KiosPromptSkeleton.from_json(re_sk_pptsk_json) - llm_spt_re_sk.initialize_from_prompt_skeleton(re_sk_pptsk) -re_sk_ppt = llm_spt_re_sk.create_prompt() -re_sk_llm = ChatOpenAI(model_name=llm_spt_re_sk.model_name, temperature=0) - -re_sk_chain = re_sk_ppt | re_sk_llm | JsonOutputParser() - - -# * skeleton generator chain -llm_spt_skeleton_generator = KiosLLMSupporter() -with open(os.path.join(prompt_sk_dir, "skeleton_generator.json"), "r") as f: - skeleton_generator_pptsk_json = json.load(f) - skeleton_generator_pptsk = KiosPromptSkeleton.from_json( - skeleton_generator_pptsk_json - ) - llm_spt_skeleton_generator.initialize_from_prompt_skeleton(skeleton_generator_pptsk) -skeleton_generator_ppt = llm_spt_skeleton_generator.create_prompt() -skeleton_generator_llm = ChatOpenAI( - model_name=llm_spt_skeleton_generator.model_name, temperature=0 -) - -skeleton_generator_chain = ( - skeleton_generator_ppt - | skeleton_generator_llm - # | StrOutputParser() - | JsonOutputParser() -) - - -# * the graph state -class PlanExecuteState(TypedDict): - plan: List[str] - behavior_tree_skeleton: dict # ! not sure if use this or not. - behavior_tree: dict - world_state: Annotated[ - List[dict], operator.add - ] # ! to add, you need to make world_state a list of dict - # = Field(default=None) - past_steps: Annotated[List[Tuple], operator.add] - response: str # ! not sure if use this or not. - # to_user: bool - user_input: str - problem: str - - -##################################################### * planner -# * output schema of the planner -class Plan(BaseModel): - """Plan to follow in future""" - - steps: List[str] = Field( - description="a list of different steps to follow, should be in sorted order" - ) - - -planner_prompt_file = os.path.join(prompt_dir, "planner/template.txt") - -with open(planner_prompt_file, "w") as f: - planner_prompt = f.read() - -planner = create_structured_output_runnable( - Plan, ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), planner_prompt -) - - -##################################################### * plan_updaterner - - -class UpdaterResponse(BaseModel): - """used to response to the user for asking for more inputs.""" - - response: str - - -plan_updater_prompt = ChatPromptTemplate.from_template( - """For the given objective, come up with a simple step by step plan. \ -This plan should involve individual tasks, that if executed correctly will yield the correct answer. Do not add any superfluous steps. \ -The result of the final step should be the final answer. Make sure that each step has all the information needed - do not skip steps. - -Your objective was this: -{user_input} - -Your original plan was this: -{plan} - -You have currently done the follow steps: -{past_steps} - -The world state is: -{world_state} - -Read the world state. Check the original plan from the first step and update your plan with the steps need to be conducted next according to the world state.\ -If no more steps are needed to achieve the objective defined by user input, respond to the user for more input. Otherwise, fill out the plan.\ -Only add steps to the plan that still NEED to be done. Do not return previously done steps as part of the plan.""" -) - -plan_updater = create_openai_fn_runnable( - [Plan, UpdaterResponse], # * here two schemas are used - ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), - plan_updater_prompt, -) - - -##################################################### * graph node functions -@traceable(name="user_input_node_step") -async def user_input_step(state: PlanExecuteState): - """ - get the input from the user - """ - print(f"-----user_input_step-----") - if state["response"] is not None: - user_input = input(state["response"]) - state["response"] = None - else: - user_input = input("Your next instruction: ") - - return { - "user_input": user_input, - } - - -@traceable(name="behavior_tree_generate_step") -async def behavior_tree_generate_step(state: PlanExecuteState): - """ - generate the behavior tree based on the instruction - """ - print(f"-----behavior_tree_generate_step-----") - - instruction = state["plan"][0] - latest_world_state = state["world_state"][-1] - - skeleton = await skeleton_generator_chain.ainvoke( - { - "problem": state["problem"], - "world_state": latest_world_state, - "instructions": instruction, - } - ) - - # ! OBJECT CONCENTRATION - - response = await re_sk_chain.ainvoke( - { - "input": skeleton, - } - ) - - behavior_tree = response.get("task_plan").get("behavior_tree") - - init_world_state = [response.get("initial_state")] - - return { - "behavior_tree": behavior_tree, - "world_state": init_world_state, # ! this is necessary because the constraints in problem - } - - -# ! BBWARN INITIALIZE THE CONSTRAINT IN THE PROMPT OR IN THE INITIALIZATION NODE - - -@traceable(name="behavior_tree_execute_step") -async def behavior_tree_execute_step(state: PlanExecuteState): - """ - execute the first step of the plan, append the result to the past steps - """ - print(f"-----behavior_tree_execute_step-----") - - behavior_tree = state["behavior_tree"] - latest_world_state = state["world_state"][-1] - - response = behavior_tree_stewardship.fake_run( - world_state=latest_world_state, bt_json=behavior_tree - ) - - hasSucceeded = response.result - new_world_state = [response.world_state] - the_step = state["plan"][0] - - return { - "past_steps": (the_step, hasSucceeded), - "world_state": new_world_state, - } - - -@traceable(name="planner_step") -async def planner_step(state: PlanExecuteState): - """ - plan the steps based on user input and world state - """ - print(f"-----plan_step-----") - - plan = await planner.ainvoke( - { - "user_input": state["user_input"], - "world_state": state["world_state"], - } - ) - return {"plan": plan.steps} - - -@traceable(name="plan_updater_step") -async def plan_updater_step(state: PlanExecuteState): - """ - if return a response, then success, response the use, end. - otherwise, return the updated newplan (normally the same as the old plan with the first step popped out. - """ - print(f"-----plan_updater_step-----") - - output = await plan_updater.ainvoke(state) - if isinstance( - output, UpdaterResponse - ): # * determine if it is time to response and end - return { - "response": "Your last instruction has been finished." - } # * Don't need to update. - else: - return {"plan": output.steps} # * update the plan - - -##################################################### * construct the graph - -workflow = StateGraph(PlanExecuteState) - -workflow.add_node("planner", planner_step) - -workflow.add_node("behavior_tree_generator", behavior_tree_generate_step) - -workflow.add_node("behavior_tree_executor", behavior_tree_execute_step) - -workflow.add_node("plan_updater", plan_updater_step) - -workflow.add_node("user_input_node", user_input_step) - -workflow.set_entry_point("user_input_node") - -# workflow.add_edge("user_input_node", "planner") - -workflow.add_edge( - "planner", "behavior_tree_generator" -) # ! may need to add a condition here - -workflow.add_edge("behavior_tree_generator", "behavior_tree_executor") - -workflow.add_edge("behavior_tree_executor", "plan_updater") - - -def plan_updater_should_end(state: PlanExecuteState): - """ - end router - """ - print(f"-----plan_updater_should_end-----") - - if state["response"]: - return True - else: - return False - - -workflow.add_conditional_edges( - "plan_updater", - plan_updater_should_end, - { - True: "user_input_node", - False: "behavior_tree_generator", - }, -) - - -def user_input_should_end(state: PlanExecuteState): - """ - if the user input is empty, then end - """ - print(f"-----user_input_should_end-----") - - # print(f'user_input: {state["user_input"]}') - # print(f'world_state: {state["world_state"]}') - - if not state["user_input"] or state["user_input"] == "": - return True - else: - return False - - -workflow.add_conditional_edges( - "user_input_node", - user_input_should_end, - { - True: END, - False: "planner", - }, -) - -app = workflow.compile() - - -# config = {"recursion_limit": 50} - -inputs = { - "world_state": [world_state_json_object], - "problem": problem, -} - - -async def core_run(): - async for event in app.astream( - inputs, - # config=config, - ): - for k, v in event.items(): - if k != "__end__": - print(v) - - -if __name__ == "__main__": - import asyncio - - asyncio.run(core_run()) diff --git a/experiments/gearset1/graph.py b/experiments/gearset1/graph.py deleted file mode 100644 index 1483ab86..00000000 --- a/experiments/gearset1/graph.py +++ /dev/null @@ -1,186 +0,0 @@ -from langchain_core.pydantic_v1 import BaseModel, Field -from typing import List, Tuple, Annotated, TypedDict -import operator - -""" -this should be the implementation of one of the templates from the langgraph library -check this: -https://github.com/langchain-ai/langgraph/blob/main/examples/rewoo/rewoo.ipynb -""" - -from langchain_openai import ChatOpenAI - -import os - -os.environ["LANGCHAIN_TRACING_V2"] = "true" -os.environ["LANGCHAIN_PROJECT"] = "kios_agent" - -##################################################### * tool -from langchain_community.tools.tavily_search import TavilySearchResults -from kios_agent.kios_tools import BehaviorTreeExecutorTool, BehaviorTreeSimulatorTool - -# tools = [TavilySearchResults(max_results=3)] -tools = [BehaviorTreeSimulatorTool()] # ! need stw - -##################################################### * execution - -from langchain import hub -from langchain.agents import create_openai_functions_agent -from langchain_openai import ChatOpenAI - -prompt = hub.pull("hwchase17/openai-functions-agent") - -# Choose the LLM that will drive the agent -llm = ChatOpenAI(model="gpt-4-turbo-preview") -# Construct the OpenAI Functions agent -agent_runnable = create_openai_functions_agent(llm, tools, prompt) - -from langgraph.prebuilt import create_agent_executor - -agent_executor = create_agent_executor(agent_runnable, tools) - - -# * the graph state -class PlanExecuteState(TypedDict): - input: str - plan: List[str] - world_state: List[dict] - past_steps: Annotated[List[Tuple], operator.add] - response: str - - -##################################################### * planner -# * output schema of the planner -class Plan(BaseModel): - """Plan to follow in future""" - - steps: List[str] = Field( - description="different steps to follow, should be in sorted order" - ) - - -from langchain.chains.openai_functions import create_structured_output_runnable -from langchain_core.prompts import ChatPromptTemplate - -planner_prompt = ChatPromptTemplate.from_template( - """For the given objective, come up with a simple step by step plan. \ -This plan should involve individual tasks, that if executed correctly will yield the correct answer. Do not add any superfluous steps. \ -The result of the final step should be the final answer. Make sure that each step has all the information needed - do not skip steps. - -{objective}""" -) -planner = create_structured_output_runnable( - Plan, ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), planner_prompt -) - -##################################################### * plan_updaterner -from langchain.chains.openai_functions import create_openai_fn_runnable - - -class Response(BaseModel): - """Response to user.""" - - response: str - - -plan_updater_prompt = ChatPromptTemplate.from_template( - """For the given objective, come up with a simple step by step plan. \ -This plan should involve individual tasks, that if executed correctly will yield the correct answer. Do not add any superfluous steps. \ -The result of the final step should be the final answer. Make sure that each step has all the information needed - do not skip steps. - -Your objective was this: -{input} - -Your original plan was this: -{plan} - -You have currently done the follow steps: -{past_steps} - -Update your plan accordingly. If no more steps are needed and you can return to the user, then respond with that. Otherwise, fill out the plan. Only add steps to the plan that still NEED to be done. Do not return previously done steps as part of the plan.""" -) - -plan_updater = create_openai_fn_runnable( - [Plan, Response], # * here two schemas are used - ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), - plan_updater_prompt, -) - - -##################################################### * graph -async def execute_step(state: PlanExecuteState): - """ - execute the first step of the plan, append the result to the past steps - """ - task = state["plan"][0] - agent_response = await agent_executor.ainvoke({"input": task, "chat_history": []}) - return { - "past_steps": (task, agent_response["agent_outcome"].return_values["output"]) - } - - -async def plan_step(state: PlanExecuteState): - """ - plan the steps based on user input - """ - plan = await planner.ainvoke({"objective": state["input"]}) - return {"plan": plan.steps} - - -async def plan_updater_step(state: PlanExecuteState): - """ - if return a response, then success, response the use, end. - otherwise, return the updated newplan (normally the same as the old plan with the first step popped out. - """ - output = await plan_updater.ainvoke(state) - if isinstance(output, Response): # * determine if it is time to response and end - return {"response": output.response} - else: - return {"plan": output.steps} - - -def should_end(state: PlanExecuteState): - """ - end router - """ - if state["response"]: - return True - else: - return False - - -##################################################### * construct the graph -from langgraph.graph import StateGraph, END - -workflow = StateGraph(PlanExecuteState) - -# Add the plan node -workflow.add_node("planner", plan_step) - -# Add the execution step -workflow.add_node("agent", execute_step) - -# Add a plan_updater node -workflow.add_node("plan_updater", plan_updater_step) - -workflow.set_entry_point("planner") - -# From plan we go to agent -workflow.add_edge("planner", "agent") - -# From agent, we plan_updater -workflow.add_edge("agent", "plan_updater") - -workflow.add_conditional_edges( - "plan_updater", - # Next, we pass in the function that will determine which node is called next. - should_end, - { - # If `tools`, then we call the tool node. - True: END, - False: "agent", - }, -) - -# compiles it into a LangChain Runnable, -app = workflow.compile() diff --git a/experiments/gearset1/human_in_the_loop.py b/experiments/gearset1/human_in_the_loop.py deleted file mode 100644 index 5d1ae56e..00000000 --- a/experiments/gearset1/human_in_the_loop.py +++ /dev/null @@ -1,630 +0,0 @@ -import json -import os -from pprint import pprint -from typing import List, Tuple, Annotated, TypedDict -import operator - -os.environ["LANGCHAIN_TRACING_V2"] = "true" -os.environ["LANGCHAIN_ENDPOINT"] = "https://api.smith.langchain.com" -os.environ["LANGCHAIN_PROJECT"] = "trash" - -from kios_bt.bt_stewardship import BehaviorTreeStewardship -from kios_scene.scene_factory import SceneFactory -from kios_bt.bt_factory import BehaviorTreeFactory -from kios_robot.robot_interface import RobotInterface -from kios_world.world_interface import WorldInterface - -from kios_agent.llm_supporter import KiosLLMSupporter -from kios_agent.data_types import KiosPromptSkeleton - -from kios_agent.kios_graph import ( - plan_updater, - planner, - seq_planner_chain, - human_instruction_chain, -) - -from dotenv import load_dotenv - -from langchain import hub - -from langchain_openai import ChatOpenAI - -from langchain.chains.openai_functions import ( - create_structured_output_runnable, - create_openai_fn_runnable, -) - -from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder -from langchain_core.output_parsers import JsonOutputParser, StrOutputParser -from langchain_core.pydantic_v1 import BaseModel, Field - -from langchain.prompts.pipeline import PipelinePromptTemplate -from langchain.prompts.prompt import PromptTemplate - -from langgraph.graph import StateGraph, END - -from langsmith import traceable - -load_dotenv() - -from kios_utils.pybt_test import generate_bt_stewardship, render_dot_tree - - -def render_bt(bt_json: json): - test_class = BehaviorTreeFactory() - bt = test_class.from_json_to_simple_bt(bt_json) - # bt = test_class.from_json_to_tree_root(bt_json) - bt_stewardship = generate_bt_stewardship(bt) - # bt_stewardship.setup(timeout=15) - render_dot_tree(bt_stewardship) - - -####################### dirs -current_dir = os.path.dirname(os.path.abspath(__file__)) -scene_path = os.path.join(current_dir, "scene.json") -# bt_json_file_path = os.path.join(current_dir, "behavior_tree.json") -world_state_path = os.path.join(current_dir, "world_state.json") -domain_knowledge_path = os.path.join(current_dir, "domain_knowledge.txt") - -####################### scene -with open(scene_path, "r") as file: - scene_json_object = json.load(file) - -scene = SceneFactory().create_scene_from_json(scene_json_object) - -####################### world -world_interface = WorldInterface() -with open(world_state_path, "r") as file: - world_state_json = json.load(file) - world_interface.load_world_from_json(world_state_json) - -####################### robot -robot_interface = RobotInterface( - robot_address="127.0.0.1", - robot_port=12000, -) -robot_interface.setup_scene(scene) - -####################### bt_factory -bt_factory = BehaviorTreeFactory( - world_interface=world_interface, - robot_interface=robot_interface, -) - -####################### behavior_tree_stewardship -behavior_tree_stewardship = BehaviorTreeStewardship( - behaviortree_factory=bt_factory, - world_interface=world_interface, - robot_interface=robot_interface, -) - -# * kios data prompt skeleton dir -data_dir = os.environ.get("KIOS_DATA_DIR").format(username=os.getlogin()) -# print(data_dir) -# prompt_sk_dir = os.path.join(data_dir, "prompt_skeletons") -prompt_dir = os.path.join(data_dir, "prompts") - - -# * the graph state -class PlanExecuteState(TypedDict): - user_input: str # this is for assembly planning - plan: List[str] # this is the assembly plan - - action_sequence: List[ - str - ] # this is the action sequence for only ONE step from the assembly plan - behavior_tree: dict # this is the behavior tree for the action sequence - world_state: Annotated[ - List[dict], operator.add - ] # ! to add, you need to make world_state a list of dict - past_steps: Annotated[ - List[Tuple], operator.add - ] # this is for update the assembly plan - - user_instruction: str # this is for improve the behavior tree - last_behavior_tree: dict # record the last behavior tree for improvement - last_failed_node: dict # record the last failed node for improvement - runtime_world_state: ( - dict # record the world state after the last execution for diagnosis - ) - - BTExecutionHasSucceeded: bool # this indicates if the last execution has succeeded - - -# ########## *sequential planner for BT -# region - -# template_file = os.path.join(prompt_dir, "seq_planner/template.txt") -# task_file = os.path.join(prompt_dir, "seq_planner/task.txt") -# system_file = os.path.join(prompt_dir, "seq_planner/system.txt") -# domain_file = os.path.join(prompt_dir, "seq_planner/domain.txt") -# with open(template_file, "r") as f: -# template_ppt = PromptTemplate.from_template(f.read()) -# with open(task_file, "r") as f: -# task_ppt = PromptTemplate.from_template(f.read()) -# with open(system_file, "r") as f: -# system_ppt = PromptTemplate.from_template(f.read()) -# with open(domain_file, "r") as f: -# domain_ppt = PromptTemplate.from_template(f.read()) - -# full_template_ppt = ChatPromptTemplate.from_template( -# """{system} - -# {task} - -# {domain} - -# {template} -# """ -# ) - -# seq_planner_ppt_ppl = PipelinePromptTemplate( -# final_prompt=full_template_ppt, -# pipeline_prompts=[ -# ("template", template_ppt), -# ("task", task_ppt), -# ("system", system_ppt), -# ("domain", domain_ppt), -# ], -# ) - -# seq_planner_chain = ( -# seq_planner_ppt_ppl -# | ChatOpenAI(model="gpt-4-turbo-preview", temperature=0) -# | StrOutputParser() -# ) -# endregion - - -# # * human instruction helped behavior tree generation -# region - -# behaviortree_file = os.path.join(prompt_dir, "human_instruction/behaviortree.txt") -# template_file = os.path.join(prompt_dir, "human_instruction/template.txt") -# task_file = os.path.join(prompt_dir, "human_instruction/task.txt") -# system_file = os.path.join(prompt_dir, "human_instruction/system.txt") -# domain_file = os.path.join(prompt_dir, "human_instruction/domain.txt") -# with open(template_file, "r") as f: -# template_ppt = PromptTemplate.from_template(f.read()) -# with open(task_file, "r") as f: -# task_ppt = PromptTemplate.from_template(f.read()) -# with open(system_file, "r") as f: -# system_ppt = PromptTemplate.from_template(f.read()) -# with open(domain_file, "r") as f: -# domain_ppt = PromptTemplate.from_template(f.read()) -# with open(behaviortree_file, "r") as f: -# ppt_tmp = PromptTemplate.from_template("{input}") -# behaviortree_ppt = ppt_tmp.partial(input=f.read()) - -# # with open(state_file, "r") as f: -# # ppt_tmp = PromptTemplate.from_template("{input}") -# # state_ppt = ppt_tmp.partial(input=f.read()) - -# # with open(example_file, "r") as f: -# # ppt_tmp = PromptTemplate.from_template("{input}") -# # example_ppt = ppt_tmp.partial(input=f.read()) -# full_template_ppt = ChatPromptTemplate.from_template( -# """{system} - -# {task} - -# {domain} - -# {behaviortree} - -# {template} -# """ -# ) - -# human_instruction_ppt_ppl = PipelinePromptTemplate( -# final_prompt=full_template_ppt, -# pipeline_prompts=[ -# ("template", template_ppt), -# ("task", task_ppt), -# ("system", system_ppt), -# ("domain", domain_ppt), -# ("behaviortree", behaviortree_ppt), -# ], -# ) - -# human_instruction_chain = ( -# human_instruction_ppt_ppl -# | ChatOpenAI(model="gpt-4-turbo-preview", temperature=0) -# | JsonOutputParser() -# ) - -# endregion - - -# ##################################################### * planner -# region -# # * output schema of the planner -# class Plan(BaseModel): -# """Plan to follow in future""" - -# steps: List[str] = Field( -# description="a list of different steps to follow, should be in sorted order" -# ) - - -# template_file = os.path.join(prompt_dir, "planner/template.txt") -# task_file = os.path.join(prompt_dir, "planner/task.txt") -# system_file = os.path.join(prompt_dir, "planner/system.txt") -# domain_file = os.path.join(prompt_dir, "planner/domain.txt") -# with open(template_file, "r") as f: -# template_ppt = PromptTemplate.from_template(f.read()) -# with open(task_file, "r") as f: -# task_ppt = PromptTemplate.from_template(f.read()) -# with open(system_file, "r") as f: -# system_ppt = PromptTemplate.from_template(f.read()) -# with open(domain_file, "r") as f: -# domain_ppt = PromptTemplate.from_template(f.read()) - -# full_template_ppt = ChatPromptTemplate.from_template( -# """{system} - -# {task} - -# {domain} - -# {template} -# """ -# ) - -# planner_ppt_ppl = PipelinePromptTemplate( -# final_prompt=full_template_ppt, -# pipeline_prompts=[ -# ("template", template_ppt), -# ("task", task_ppt), -# ("system", system_ppt), -# ("domain", domain_ppt), -# ], -# ) - -# planner = create_structured_output_runnable( -# Plan, ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), planner_ppt_ppl -# ) - -# ##################################################### * plan_updaterner - -# template_file = os.path.join(prompt_dir, "plan_updater/template.txt") -# task_file = os.path.join(prompt_dir, "plan_updater/task.txt") -# system_file = os.path.join(prompt_dir, "plan_updater/system.txt") -# domain_file = os.path.join(prompt_dir, "plan_updater/domain.txt") -# with open(template_file, "r") as f: -# template_ppt = PromptTemplate.from_template(f.read()) -# with open(task_file, "r") as f: -# task_ppt = PromptTemplate.from_template(f.read()) -# with open(system_file, "r") as f: -# system_ppt = PromptTemplate.from_template(f.read()) -# with open(domain_file, "r") as f: -# domain_ppt = PromptTemplate.from_template(f.read()) - -# full_template_ppt = ChatPromptTemplate.from_template( -# """{system} - -# {task} - -# {domain} - -# {template} -# """ -# ) - -# plan_updater_ppt_ppl = PipelinePromptTemplate( -# final_prompt=full_template_ppt, -# pipeline_prompts=[ -# ("template", template_ppt), -# ("task", task_ppt), -# ("system", system_ppt), -# ("domain", domain_ppt), -# ], -# ) - -# plan_updater = create_structured_output_runnable( -# Plan, ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), plan_updater_ppt_ppl -# ) -# ! create openai fn runnable has bug -# endregion - - -##################################################### * graph node functions -@traceable(name="user_input_node_step") -async def user_input_step(state: PlanExecuteState): - """ - get the input from the user - """ - print(f"-----user_input_step-----") - - user_input = input("Please provide your instructions:\n") - - return { - "user_input": user_input, - } - - -@traceable(name="sequence_generate_step") -async def sequence_generate_step(state: PlanExecuteState): - """ - generate the sequence based on the instruction - """ - print(f"-----sequence_generate_step-----") - - plan_goal = state["plan"][0] - start_world_state = state["world_state"][-1] - - action_sequence = await seq_planner_chain.ainvoke( - { - "start_world_state": start_world_state, - "user_instruction": plan_goal, - } - ) - - return { - "action_sequence": action_sequence, - } - - -@traceable(name="behavior_tree_generate_step") -async def behavior_tree_generate_step(state: PlanExecuteState): - """ - generate the behavior tree based on the instruction - """ - print(f"-----behavior_tree_generate_step-----") - user_instruction = state["user_instruction"] - - bt_skeleton = await human_instruction_chain.ainvoke( - { - "user_instruction": state["user_instruction"], - "last_behavior_tree": state["last_behavior_tree"], - "action_sequence": state["action_sequence"], - } - ) - - # * the test is currently without sim - # * first sim run - # tree_result, skeleton_json = behavior_tree_stewardship.sk_sim_run( - # world_state=state["world_state"], skeleton_json=bt_skeleton - # ) - - # pprint(tree_result.to_json()) - - render_bt(bt_skeleton) - - return { - "last_behavior_tree": bt_skeleton, - "BTExecutionHasSucceeded": False, - } - - -@traceable(name="behavior_tree_execute_step") -async def behavior_tree_execute_step(state: PlanExecuteState): - """ - execute the first step of the plan, append the result to the past steps - """ - print(f"-----behavior_tree_execute_step-----") - - this_step = state["plan"][0] - behavior_tree_skeleton = state["last_behavior_tree"] - latest_world_state = state["world_state"][-1] - - # * first sim run - tree_result, skeleton_json = behavior_tree_stewardship.sk_sim_run( - world_state=latest_world_state, skeleton_json=behavior_tree_skeleton - ) - - pprint(tree_result.to_json()) - pause = input("paused here") - - # * check result - if tree_result.result == "success": - return { - "BTExecutionHasSucceeded": True, - "past_steps": ( - this_step, - tree_result.result, - ), # * only one because new plan will be generated and old steps are all removed - "world_state": [tree_result.world_state], - # "last_behavior_tree": None, # ! do not change the last behavior tree - "runtime_world_state": tree_result.world_state, # * this is world_state for successful execution - } - else: - return { - "BTExecutionHasSucceeded": False, - # ! do not change world state - # "last_behavior_tree": skeleton_json, # * do not change the last behavior tree - "world_state": [tree_result.world_state], - "runtime_world_state": tree_result.world_state, - } - - -@traceable(name="planner_step") -async def planner_step(state: PlanExecuteState): - """ - plan the steps based on user input and world state - """ - print(f"-----plan_step-----") - - plan = await planner.ainvoke( - { - "user_input": state["user_input"], - "world_state": state["world_state"], - } - ) - return {"plan": plan.steps} - - -@traceable(name="plan_updater_step") -async def plan_updater_step(state: PlanExecuteState): - """ - if return a response, then success, response the use, end. - otherwise, return the updated newplan (normally the same as the old plan with the first step popped out. - """ - print(f"-----plan_updater_step-----") - - # ! BOOOM! BUG! - output = await plan_updater.ainvoke( - { - "user_input": state["user_input"], - "plan": state["plan"], - "world_state": state["world_state"], - "past_steps": state["past_steps"], - } - ) - # if isinstance( - # output, UpdaterResponse - # ): # * determine if it is time to response and end - # return { - # "response": "Your last instruction has been finished." - # } # * Don't need to update. - # else: - return {"plan": output.steps} # * update the plan - - -##################################################### * construct the graph - -workflow = StateGraph(PlanExecuteState) - -workflow.add_node("planner", planner_step) -workflow.add_node("sequence_generator", sequence_generate_step) -workflow.add_node("behavior_tree_generator", behavior_tree_generate_step) -workflow.add_node("behavior_tree_executor", behavior_tree_execute_step) -workflow.add_node("plan_updater", plan_updater_step) -workflow.add_node("user_input_node", user_input_step) -workflow.set_entry_point("user_input_node") -workflow.add_edge("planner", "sequence_generator") -workflow.add_edge("sequence_generator", "behavior_tree_generator") - - -def user_instruction_should_end(state: PlanExecuteState): - """ - router for user hint. - """ - print(f"-----user_instruction_should_end-----") - user_instruction = input( - "What should I do to improve the behavior tree?\nPlease give me your hint: " - ) - state["user_instruction"] = user_instruction - if user_instruction == "": - return True # go to exectuor - else: - return False # go back to generator - - -workflow.add_conditional_edges( - "behavior_tree_generator", - user_instruction_should_end, - { - True: "behavior_tree_executor", - False: "behavior_tree_generator", - }, -) - - -def executor_should_end(state: PlanExecuteState): - """ - end router - """ - print(f"-----executor_should_end-----") - - if state["BTExecutionHasSucceeded"] == True: - # ask for user confirmation and end, or go back to the behavior tree generator if the user wants to improve - user_instruction = input( - "the behavior tree has succeeded.\nPress ENTER to confirm:" - ) - state["user_instruction"] = "" - if user_instruction == "": - state["last_behavior_tree"] = None - state["BTExecutionHasSucceeded"] = False - return True - else: - return False - else: - # ask for user hint and go back to the behavior tree generator - user_instruction = input( - "The behavior tree has failed in its execution.\nPlease give me a hint to improve it:" - ) - state["user_instruction"] = user_instruction - return False - - -workflow.add_conditional_edges( - "behavior_tree_executor", - executor_should_end, - { - True: "plan_updater", - False: "behavior_tree_generator", - }, -) - - -def plan_updater_should_end(state: PlanExecuteState): - """ - end router - """ - print(f"-----plan_updater_should_end-----") - - if state["plan"] == [] or len(state["plan"]) == 0: - print("The assembly plan has been finished.") - return True - else: - print("The assembly plan has not been finished.") - print(f'Unfinished steps: {state["plan"]}') - return False - - -workflow.add_conditional_edges( - "plan_updater", - plan_updater_should_end, - { - True: "user_input_node", - False: "sequence_generator", # ! BUG generate the sequence first - }, -) - - -def user_input_should_end(state: PlanExecuteState): - """ - if the user input is empty, then end - """ - print(f"-----user_input_should_end-----") - - if not state["user_input"] or state["user_input"] == "": - return True - else: - return False - - -workflow.add_conditional_edges( - "user_input_node", - user_input_should_end, - { - True: END, - False: "planner", - }, -) - -app = workflow.compile() - -config = {"recursion_limit": 500} - -inputs = { - "world_state": [world_state_json], -} - - -async def core_run(): - async for event in app.astream( - inputs, - config=config, - ): - for k, v in event.items(): - if k != "__end__": - print(v) - - -if __name__ == "__main__": - import asyncio - - asyncio.run(core_run()) diff --git a/experiments/gearset1/mios_memory_backups/exp.json b/experiments/gearset1/mios_memory_backups/exp.json deleted file mode 100644 index 5642fff3..00000000 --- a/experiments/gearset1/mios_memory_backups/exp.json +++ /dev/null @@ -1,2802 +0,0 @@ -[ - { - "_id": "65eb7417886cb04353024ad1", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "geometry": null, - "grasp_force": 1.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "TestObject1", - "q": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ] - }, - { - "_id": "65eb741d2c800a25d3000d41", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "NullObject", - "q": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ] - }, - { - "_id": "65eb741d2c800a25d3000d42", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "NoneObject", - "q": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ] - }, - { - "_id": "65eb741d2c800a25d3000d43", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.019190783779091135, - 0.9997381967845808, - 0.011661844153803638, - 0.0, - 0.9997923775597143, - -0.019127853922897033, - -0.005483965627780814, - 0.0, - -0.005259565119427513, - 0.011764891002485732, - -0.9999169587092991, - 0.0, - 0.29923324636565385, - -0.3208095127102042, - 0.3232705877678895, - 1.0 - ], - "O_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "EndEffector", - "q": [ - -0.26499529137214844, - -0.3644651785273301, - -0.5231116759526503, - -2.414775848723294, - -0.1991585435443978, - 2.075000562235501, - -1.426125477296326 - ] - }, - { - "_id": "65eb74332c800a25d3000d44", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - -0.9993938589627782, - -0.019959674635375253, - 0.02818284863431975, - 0.0, - -0.01984070888343409, - 0.9997834226459386, - 0.0044945494306065075, - 0.0, - -0.02826699883912122, - 0.00393273312179257, - -0.999592672235457, - 0.0, - -0.37484573960170936, - 0.3533845541917834, - 0.1748708579262803, - 1.0 - ], - "O_T_TCP": [ - -0.9993938589627782, - -0.019959674635375253, - 0.02818284863431975, - 0.0, - -0.01984070888343409, - 0.9997834226459386, - 0.0044945494306065075, - 0.0, - -0.02826699883912122, - 0.00393273312179257, - -0.999592672235457, - 0.0, - -0.37484573960170936, - 0.3533845541917834, - 0.1748708579262803, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "screwbolt", - "q": [ - 1.6550573774574837, - 0.15321099584385517, - 0.7222888105859068, - -2.3508161678625883, - -0.13811986689658284, - 2.484654771697428, - 0.10289976926765454 - ] - }, - { - "_id": "65eb74ac2c800a25d3000d45", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - -0.9754081093603869, - -0.21792603402093702, - 0.03268043437760813, - 0.0, - -0.21721197651678822, - 0.9758183257555578, - 0.024047853927084306, - 0.0, - -0.037131535089479584, - 0.016358204934740318, - -0.9991764900322725, - 0.0, - -0.49456060992516254, - 0.34523358993584274, - 0.18533646391614586, - 1.0 - ], - "O_T_TCP": [ - -0.9754081093603869, - -0.21792603402093702, - 0.03268043437760813, - 0.0, - -0.21721197651678822, - 0.9758183257555578, - 0.024047853927084306, - 0.0, - -0.037131535089479584, - 0.016358204934740318, - -0.9991764900322725, - 0.0, - -0.49456060992516254, - 0.34523358993584274, - 0.18533646391614586, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "screwhole", - "q": [ - 1.331823483964853, - 1.2933396335568335, - 1.6098604303502224, - -1.920375438578031, - -1.3170606204536117, - 1.6645034709950923, - 0.6755126488672281 - ] - }, - { - "_id": "65eb8ec1d3061a79e90e03d1", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - -1.0, - 0.0, - 0.0, - 0.0, - 0.0, - -1.0, - 0.0, - 0.521827226443675, - 0.25840698062581857, - 0.27517533664525384, - 1.0 - ], - "O_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - -1.0, - 0.0, - 0.0, - 0.0, - 0.0, - -1.0, - 0.0, - 0.521827226443675, - 0.25840698062581857, - 0.27517533664525384, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "test", - "q": [ - 0.18074043324946532, - 0.1335626147107068, - 0.2868349701521744, - -2.060772508320055, - -0.04825592930051324, - 2.2430690224108654, - 1.2386451794617466 - ] - }, - { - "_id": "65ef6331e99f2576ba086061", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - -0.9999511693327637, - 0.005795104725127399, - 0.006694969191221638, - 0.0, - 0.005676923956787325, - 0.9998204500065859, - -0.017538163650304785, - 0.0, - -0.00679553343966406, - -0.017499637341326747, - -0.9998237761816791, - 0.0, - -0.28525213823562634, - 0.5657042239474747, - 0.23866900068692073, - 1.0 - ], - "O_T_TCP": [ - -0.9999511693327637, - 0.005795104725127399, - 0.006694969191221638, - 0.0, - 0.005676923956787325, - 0.9998204500065859, - -0.017538163650304785, - 0.0, - -0.00679553343966406, - -0.017499637341326747, - -0.9998237761816791, - 0.0, - -0.28525213823562634, - 0.5657042239474747, - 0.23866900068692073, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "normalcontainer", - "q": [ - 1.48723348463888, - 0.4174561200767826, - 0.6111280596883836, - -1.8220793050632123, - -0.26309194494459087, - 2.141865217341235, - -0.1461896891569508 - ] - }, - { - "_id": "65ef6455e99f2576ba086062", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - -0.9990336274326229, - -0.043224038692152265, - 0.006651363706987428, - 0.0, - -0.04306611338611579, - 0.9988138548284101, - 0.022292155163513752, - 0.0, - -0.00760717766310412, - 0.02198458752415408, - -0.9997293677587924, - 0.0, - -0.2767815159120372, - 0.4389463906506993, - 0.3408536221713534, - 1.0 - ], - "O_T_TCP": [ - -0.9990336274326229, - -0.043224038692152265, - 0.006651363706987428, - 0.0, - -0.04306611338611579, - 0.9988138548284101, - 0.022292155163513752, - 0.0, - -0.00760717766310412, - 0.02198458752415408, - -0.9997293677587924, - 0.0, - -0.2767815159120372, - 0.4389463906506993, - 0.3408536221713534, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "normalpickplace", - "q": [ - 1.6710431030315733, - -0.11933810338041052, - 0.44976917909321673, - -2.1647275869469373, - 0.053616515815126296, - 2.079309125394282, - -0.3063374023692766 - ] - }, - { - "_id": "65f0c57f8fca00707f09dc71", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.9943836026315849, - -0.0968288341608145, - -0.042499112857760084, - 0.0, - -0.09580790301125691, - -0.9950650220230038, - 0.025440019028163983, - 0.0, - -0.04475356970019007, - -0.021225795551839743, - -0.9987725384702374, - 0.0, - 0.47926793066474926, - 0.23916614873158387, - -0.004146619879005292, - 1.0 - ], - "O_T_TCP": [ - 0.9943836026315849, - -0.0968288341608145, - -0.042499112857760084, - 0.0, - -0.09580790301125691, - -0.9950650220230038, - 0.025440019028163983, - 0.0, - -0.04475356970019007, - -0.021225795551839743, - -0.9987725384702374, - 0.0, - 0.47926793066474926, - 0.23916614873158387, - -0.004146619879005292, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "p2_sp", - "q": [ - 0.30748095076589516, - 0.6224689720722666, - 0.1322143079238894, - -2.174735181272256, - -0.1994334632820553, - 2.7361665597756706, - 1.4845305872601067 - ] - }, - { - "_id": "65f0c58a8fca00707f09dc72", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - -0.3906372923180876, - 0.9204079634018747, - 0.015245775087359574, - 0.0, - 0.9202843547242807, - 0.39086355071749906, - -0.01682670708818005, - 0.0, - -0.02144686590450544, - 0.007457452572057124, - -0.9997421759353806, - 0.0, - 0.5416091667977869, - 0.23056869998184873, - -0.005339503846320045, - 1.0 - ], - "O_T_TCP": [ - -0.3906372923180876, - 0.9204079634018747, - 0.015245775087359574, - 0.0, - 0.9202843547242807, - 0.39086355071749906, - -0.01682670708818005, - 0.0, - -0.02144686590450544, - 0.007457452572057124, - -0.9997421759353806, - 0.0, - 0.5416091667977869, - 0.23056869998184873, - -0.005339503846320045, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "p2_sp1", - "q": [ - 0.3674982954399627, - 0.7058114279278537, - 0.031059863668303392, - -2.001528142847549, - -0.011622414170942152, - 2.6900898772047506, - -0.7850909599531587 - ] - }, - { - "_id": "65f20f89ad440b37f0074621", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - -0.0398234703603432, - 0.9990306890185293, - 0.018235144880958364, - 0.0, - 0.9991950897154301, - 0.03985333597094554, - -0.0012771850955814157, - 0.0, - -0.002002717019971174, - 0.01816995511044529, - -0.9998329069678704, - 0.0, - -0.2748624745859707, - 0.44021610755685797, - 0.1766043743163428, - 1.0 - ], - "O_T_TCP": [ - -0.0398234703603432, - 0.9990306890185293, - 0.018235144880958364, - 0.0, - 0.9991950897154301, - 0.03985333597094554, - -0.0012771850955814157, - 0.0, - -0.002002717019971174, - 0.01816995511044529, - -0.9998329069678704, - 0.0, - -0.2748624745859707, - 0.44021610755685797, - 0.1766043743163428, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "shaft2", - "q": [ - 1.5506422050658024, - 0.16138674089992255, - 0.575727326601761, - -2.3337822530144487, - -0.15566768010457357, - 2.480229851319371, - 1.4168640595144946 - ] - }, - { - "_id": "65f21187983e0351740564c1", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - -0.03847458660057955, - 0.9992279041866523, - 0.006636910566905391, - 0.0, - 0.9989573170328562, - 0.0386231991192383, - -0.023943143852083248, - 0.0, - -0.024181461737390535, - 0.005708897725647523, - -0.9996912850450389, - 0.0, - -0.28040527035598145, - 0.5625029367429322, - 0.21460991049071504, - 1.0 - ], - "O_T_TCP": [ - -0.03847458660057955, - 0.9992279041866523, - 0.006636910566905391, - 0.0, - 0.9989573170328562, - 0.0386231991192383, - -0.023943143852083248, - 0.0, - -0.024181461737390535, - 0.005708897725647523, - -0.9996912850450389, - 0.0, - -0.28040527035598145, - 0.5625029367429322, - 0.21460991049071504, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "p2_n2", - "q": [ - 1.5703048682036513, - 0.39783672250131485, - 0.4982862662432487, - -1.8933198999940302, - -0.21576868739395502, - 2.251606949418013, - 1.345509883842923 - ] - }, - { - "_id": "65f21226983e0351740564c3", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - -0.7670325848858576, - -0.6410938681792032, - 0.025306378791810035, - 0.0, - -0.6395410237084117, - 0.7671364076897498, - 0.049696658761641144, - 0.0, - -0.05127465491506417, - 0.021934911546283895, - -0.9984436736335195, - 0.0, - -0.27998567269576946, - 0.5668166088777099, - 0.2224894936706944, - 1.0 - ], - "O_T_TCP": [ - -0.7670325848858576, - -0.6410938681792032, - 0.025306378791810035, - 0.0, - -0.6395410237084117, - 0.7671364076897498, - 0.049696658761641144, - 0.0, - -0.05127465491506417, - 0.021934911546283895, - -0.9984436736335195, - 0.0, - -0.27998567269576946, - 0.5668166088777099, - 0.2224894936706944, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "p2_n1", - "q": [ - 1.5286179915938458, - 0.39123391699293775, - 0.5349895313363059, - -1.8972686476502396, - -0.20874278682342615, - 2.270374859585192, - -0.8939703330265151 - ] - }, - { - "_id": "65f214fc983e0351740564c4", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - -0.030237889060397694, - 0.9941032227460016, - 0.10404421892826617, - 0.0, - 0.9990370460044222, - 0.03333771394241911, - -0.028183762010417793, - 0.0, - -0.031486771269573506, - 0.10309379653800622, - -0.9941731500852344, - 0.0, - -0.015942964370017125, - 0.4375202893228714, - 0.42062840472064134, - 1.0 - ], - "O_T_TCP": [ - -0.030237889060397694, - 0.9941032227460016, - 0.10404421892826617, - 0.0, - 0.9990370460044222, - 0.03333771394241911, - -0.028183762010417793, - 0.0, - -0.031486771269573506, - 0.10309379653800622, - -0.9941731500852344, - 0.0, - -0.015942964370017125, - 0.4375202893228714, - 0.42062840472064134, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "initialposition2", - "q": [ - 1.5506923031671005, - -0.4497734840543152, - 0.038535913673190224, - -2.3152158262018574, - 0.050607820335361695, - 1.9691166749798155, - 0.7517650229864651 - ] - }, - { - "_id": "65f374543949254a58053971", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.03173898612252293, - 0.9982326382781231, - 0.05004981056328121, - 0.0, - 0.9987842695688426, - -0.02980004258186478, - -0.03902162509636153, - 0.0, - -0.037461894541397284, - 0.05122845660945351, - -0.9979840939067026, - 0.0, - -0.2748815546694312, - 0.3324526946698264, - 0.19020321271820365, - 1.0 - ], - "O_T_TCP": [ - 0.03173898612252293, - 0.9982326382781231, - 0.05004981056328121, - 0.0, - 0.9987842695688426, - -0.02980004258186478, - -0.03902162509636153, - 0.0, - -0.037461894541397284, - 0.05122845660945351, - -0.9979840939067026, - 0.0, - -0.2748815546694312, - 0.3324526946698264, - 0.19020321271820365, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "sp_shaft2", - "q": [ - 2.3878911709999446, - -0.13898968152668084, - -0.1298367435116517, - -2.652961812604937, - -0.0403083813485689, - 2.578178659995397, - 1.5375320562521366 - ] - }, - { - "_id": "65f489ce00b781025b0db011", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.009966723549040366, - 0.9988357757215452, - 0.04699472801968561, - 0.0, - 0.999584303405681, - -0.00869749333183234, - -0.027135233478458594, - 0.0, - -0.026695419616810108, - 0.04724655148433873, - -0.9985264733317392, - 0.0, - -0.02021721107783165, - 0.5257402517302128, - 0.3253018605106694, - 1.0 - ], - "O_T_TCP": [ - 0.009966723549040366, - 0.9988357757215452, - 0.04699472801968561, - 0.0, - 0.999584303405681, - -0.00869749333183234, - -0.027135233478458594, - 0.0, - -0.026695419616810108, - 0.04724655148433873, - -0.9985264733317392, - 0.0, - -0.02021721107783165, - 0.5257402517302128, - 0.3253018605106694, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "gear1", - "q": [ - -1.2245379853384535, - 0.08946082408795829, - 2.8328730721891966, - -2.187969385201221, - -0.002710549400912387, - 2.1509781232674916, - 0.8342100091165182 - ] - }, - { - "_id": "65f489ce00b781025b0db012", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - -0.02405268971449932, - -0.999350109142552, - 0.026487249299911187, - 0.0, - -0.998520867937328, - 0.02530289761840895, - 0.0479227145204212, - 0.0, - -0.04856270912625561, - -0.025295888001814256, - -0.9984997653142029, - 0.0, - -0.29627286474447195, - 0.5402193926564312, - 0.24951815252804269, - 1.0 - ], - "O_T_TCP": [ - -0.02405268971449932, - -0.999350109142552, - 0.026487249299911187, - 0.0, - -0.998520867937328, - 0.02530289761840895, - 0.0479227145204212, - 0.0, - -0.04856270912625561, - -0.025295888001814256, - -0.9984997653142029, - 0.0, - -0.29627286474447195, - 0.5402193926564312, - 0.24951815252804269, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "inward_claw", - "q": [ - 1.1466555126372133, - 0.4911871555482781, - 1.0211879908664732, - -1.889103684814978, - -0.42553616185983023, - 2.12249806078275, - -1.560074370787497 - ] - }, - { - "_id": "65f48add00b781025b0db013", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - -0.5652402046656898, - 0.805912836269351, - 0.1760362413453204, - 0.0, - 0.8101766382360048, - 0.5825117783483679, - -0.0653803475307077, - 0.0, - -0.15523703410852738, - 0.10566688361643159, - -0.9822097397949072, - 0.0, - -0.2763305579573874, - 0.33279789980415175, - 0.3557425405921873, - 1.0 - ], - "O_T_TCP": [ - -0.5652402046656898, - 0.805912836269351, - 0.1760362413453204, - 0.0, - 0.8101766382360048, - 0.5825117783483679, - -0.0653803475307077, - 0.0, - -0.15523703410852738, - 0.10566688361643159, - -0.9822097397949072, - 0.0, - -0.2763305579573874, - 0.33279789980415175, - 0.3557425405921873, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "sp_shaft2_above", - "q": [ - 2.4060116416811943, - -0.4776864408442848, - -0.14955661941620338, - -2.5259720389105635, - -0.021758461500339422, - 2.2347033861374603, - 0.8756390861099728 - ] - }, - { - "_id": "65f5d17319ec67138e072821", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.8560476645714392, - 0.5060823187534547, - 0.10508962616794995, - 0.0, - 0.5072494013809845, - -0.8616144504353933, - 0.01730117050249211, - 0.0, - 0.09930446890095938, - 0.03849676454404434, - -0.9943121348831758, - 0.0, - 0.4977328298997675, - 0.2632478599241839, - 0.2862359205478977, - 1.0 - ], - "O_T_TCP": [ - 0.8560476645714392, - 0.5060823187534547, - 0.10508962616794995, - 0.0, - 0.5072494013809845, - -0.8616144504353933, - 0.01730117050249211, - 0.0, - 0.09930446890095938, - 0.03849676454404434, - -0.9943121348831758, - 0.0, - 0.4977328298997675, - 0.2632478599241839, - 0.2862359205478977, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "p2_sp1_above", - "q": [ - 0.529364852355879, - 0.0378608091928457, - -0.04017833637609318, - -2.1627976577105055, - -0.015072589963674544, - 2.3065233025810263, - 0.7512218856027597 - ] - }, - { - "_id": "65f623355dc5574d5609af31", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.9995210312027456, - -0.02108802959629215, - 0.022220488121427916, - 0.0, - -0.020154087917596285, - -0.9989277578944894, - -0.041447486646434044, - 0.0, - 0.023071152394727273, - 0.04098058992578088, - -0.9988935444663332, - 0.0, - 0.5558260099968472, - 0.35474503674495034, - 0.1698716055280708, - 1.0 - ], - "O_T_TCP": [ - 0.9995210312027456, - -0.02108802959629215, - 0.022220488121427916, - 0.0, - -0.020154087917596285, - -0.9989277578944894, - -0.041447486646434044, - 0.0, - 0.023071152394727273, - 0.04098058992578088, - -0.9988935444663332, - 0.0, - 0.5558260099968472, - 0.35474503674495034, - 0.1698716055280708, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "observe", - "q": [ - 0.24490834641150266, - 0.5219243896211131, - 0.35286033423301993, - -1.8072963120737229, - -0.20848363170680467, - 2.3329070408344266, - 1.4982907013541806 - ] - }, - { - "_id": "65f84a7b4e84e8ebeb016c71", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.011520231935140872, - 0.9992191997809841, - 0.037536941090327375, - 0.0, - 0.9996864032240798, - -0.010691052153897894, - -0.02221583925836677, - 0.0, - -0.02179760340272546, - 0.03778182866479363, - -0.9990482460365174, - 0.0, - -0.019621728393910066, - 0.5282196768512274, - 0.3213045334955448, - 1.0 - ], - "O_T_TCP": [ - 0.011520231935140872, - 0.9992191997809841, - 0.037536941090327375, - 0.0, - 0.9996864032240798, - -0.010691052153897894, - -0.02221583925836677, - 0.0, - -0.02179760340272546, - 0.03778182866479363, - -0.9990482460365174, - 0.0, - -0.019621728393910066, - 0.5282196768512274, - 0.3213045334955448, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "gear3", - "q": [ - 1.5803205966001672, - -0.06910919187675439, - 0.02128317169835045, - -2.1775416519636077, - 0.02634642873853542, - 2.146742638396533, - 0.8133678669977809 - ] - }, - { - "_id": "6606cd59dc7c9a8bb401e2e1", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.9961707221748939, - 0.017795973560695604, - -0.08548650482845685, - 0.0, - 0.01417788185585071, - -0.9989757578750006, - -0.04274540613163383, - 0.0, - -0.08616130093849131, - 0.041370501040424945, - -0.9954218763239286, - 0.0, - 0.3516894945719454, - 0.01240417249927435, - 0.4057604465881531, - 1.0 - ], - "O_T_TCP": [ - 0.9961707221748939, - 0.017795973560695604, - -0.08548650482845685, - 0.0, - 0.01417788185585071, - -0.9989757578750006, - -0.04274540613163383, - 0.0, - -0.08616130093849131, - 0.041370501040424945, - -0.9954218763239286, - 0.0, - 0.3516894945719454, - 0.01240417249927435, - 0.4057604465881531, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "defaultgripper", - "q": [ - -0.12472951297691537, - -0.603558807574886, - 0.11032047479027791, - -2.425834369478495, - 0.10213290844466369, - 1.7362021556695302, - 0.7115313167587582 - ] - }, - { - "_id": "6606cda7dc7c9a8bb401e2e2", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.026721811815328126, - 0.9995769685145904, - 0.010610169164358797, - 0.0, - 0.9995985336740462, - -0.02680787184703303, - 0.008053346195102389, - 0.0, - 0.008334535896975376, - 0.010390909594426423, - -0.999911278318823, - 0.0, - 0.46035259498542536, - -0.32705890800734944, - 0.3225370115547157, - 1.0 - ], - "O_T_TCP": [ - 0.026721811815328126, - 0.9995769685145904, - 0.010610169164358797, - 0.0, - 0.9995985336740462, - -0.02680787184703303, - 0.008053346195102389, - 0.0, - 0.008334535896975376, - 0.010390909594426423, - -0.999911278318823, - 0.0, - 0.461144375895638, - -0.3260717715958789, - 0.2275454401144275, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "clampgripper", - "q": [ - -0.2671814419219368, - 0.06181956611652123, - -0.3630485066560427, - -2.010536448103099, - 0.04011968715323342, - 2.068616536025648, - -1.4073721870647535 - ] - }, - { - "_id": "6606ce19dc7c9a8bb401e2e3", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.009825032844342677, - 0.9999127968835277, - 0.007655994995895023, - 0.0, - 0.9999232045030161, - -0.009777483336183012, - -0.0062235694933726075, - 0.0, - -0.0061482887888344336, - 0.0077167023950695385, - -0.9999513243398976, - 0.0, - 0.300456482288005, - -0.3222580251297459, - 0.3224839901901114, - 1.0 - ], - "O_T_TCP": [ - 0.009825032844342677, - 0.9999127968835277, - 0.007655994995895023, - 0.0, - 0.9999232045030161, - -0.009777483336183012, - -0.0062235694933726075, - 0.0, - -0.0061482887888344336, - 0.0077167023950695385, - -0.9999513243398976, - 0.0, - 0.2998723948530657, - -0.3215249384022143, - 0.22748861437782114, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "parallelgripper", - "q": [ - -0.320270861128966, - -0.345672909050657, - -0.47318206217637626, - -2.4110082609929058, - -0.17673210292173253, - 2.0841505399724483, - -1.4557532899445966 - ] - }, - { - "_id": "6606ce60dc7c9a8bb401e2e4", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.03249344521057514, - 0.9984044994399821, - -0.04597149570716935, - 0.0, - 0.9991681719615784, - -0.03133367044413276, - 0.02572765315112185, - 0.0, - 0.02424661579228814, - -0.04677013588618136, - -0.9986113638507271, - 0.0, - 0.4813456686797789, - 0.29184715246595744, - 0.326744516007914, - 1.0 - ], - "O_T_TCP": [ - 0.03249344521057514, - 0.9984044994399821, - -0.04597149570716935, - 0.0, - 0.9991681719615784, - -0.03133367044413276, - 0.02572765315112185, - 0.0, - 0.02424661579228814, - -0.04677013588618136, - -0.9986113638507271, - 0.0, - 0.4813456686797789, - 0.29184715246595744, - 0.326744516007914, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "outwardgripper", - "q": [ - 0.547243936831871, - 0.049198853157179215, - 0.013326426081909353, - -2.009669691952538, - -0.06005995021268557, - 2.0537857054869333, - -0.1648658174194606 - ] - }, - { - "_id": "6606ce9cdc7c9a8bb401e2e5", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.04946285742980003, - 0.9976997411078945, - -0.04614541395770383, - 0.0, - 0.9984707380270613, - -0.048271444059959374, - 0.026585708440250197, - 0.0, - 0.02429751646200952, - -0.04739076305620594, - -0.9985808661649435, - 0.0, - 0.3196977361216577, - 0.2970747305541146, - 0.32588162234940227, - 1.0 - ], - "O_T_TCP": [ - 0.04946285742980003, - 0.9976997411078945, - -0.04614541395770383, - 0.0, - 0.9984707380270613, - -0.048271444059959374, - 0.026585708440250197, - 0.0, - 0.02429751646200952, - -0.04739076305620594, - -0.9985808661649435, - 0.0, - 0.3196977361216577, - 0.2970747305541146, - 0.32588162234940227, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "inwardgripper", - "q": [ - 0.5466129649972945, - -0.3299939654497744, - 0.20834633539576183, - -2.419438092917727, - 0.018024931998749005, - 2.0817987441221875, - -0.001865424298784799 - ] - }, - { - "_id": "6606cedadc7c9a8bb401e2e6", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.9609061324079634, - -0.01991496375760643, - 0.27612233850326423, - 0.0, - -0.019538755255997877, - -0.999791008757942, - -0.00411372714589368, - 0.0, - 0.2761518728590894, - -0.0014422089181775396, - -0.961112929446823, - 0.0, - 0.3334074788519119, - -0.014388862181240806, - 0.5439612322873627, - 1.0 - ], - "O_T_TCP": [ - 0.9609061324079634, - -0.01991496375760643, - 0.27612233850326423, - 0.0, - -0.019538755255997877, - -0.999791008757942, - -0.00411372714589368, - 0.0, - 0.2761518728590894, - -0.0014422089181775396, - -0.961112929446823, - 0.0, - 0.3334074788519119, - -0.014388862181240806, - 0.5439612322873627, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "initialposition", - "q": [ - -0.09896254014341453, - -0.8518857829445285, - 0.028137918776326012, - -2.326811223181909, - 0.04180588018604759, - 1.7541868256727853, - 0.7262109747661475 - ] - } -] \ No newline at end of file diff --git a/experiments/gearset1/problem.py b/experiments/gearset1/problem.py deleted file mode 100644 index 2233e3f7..00000000 --- a/experiments/gearset1/problem.py +++ /dev/null @@ -1,111 +0,0 @@ -import re - -""" -script to parse the initial state from a problem file. -out-of-date. now pddl part is removed and this part should be no longer in use. -""" - - -def parse_init(problem): - init_pattern = r"\(:init\s+(.*?)\s+\)" - element_pattern = r"\((.*?)\)" - - init_match = re.search(init_pattern, problem, re.DOTALL) - if init_match: - init_content = init_match.group(1) - elements = re.findall(element_pattern, init_content) - - parsed_init = [] - for element in elements: - parts = element.split() - name = parts[0] - args = parts[1:] - parsed_init.append({"name": name, "args": args}) - - return parsed_init - else: - return [] - - -properties = [ - "is_free", - "is_equippable", -] - -constraints = [ - "can_manipulate", - "can_screw_to", - "can_insert_to", - "can_place_to", -] - -relations = [ - "hold", - "is_inserted_to", - "is_screwed_to", - "is_placed_to", -] - - -def classify_item(item_name): - if item_name in properties: - return "property" - elif item_name in constraints: - return "constraint" - elif item_name in relations: - return "relation" - else: - raise ValueError(f"Unknown item name: {item_name}") - - -# Usage example: -problem = """ -(define (problem robot_assembly_problem-problem) - (:domain robot_assembly_problem-domain) - (:objects - parallel_box1 parallel_box2 inward_claw outward_claw no_tool - tool - gear1 gear2 gear3 shaft1 shaft2 shaft3 gearbase gearbase_hole1 gearbase_hole3 - part - left_hand - hand - ) - (:init (can_manipulate parallel_box2 gear1) (can_manipulate outward_claw gear2) (can_manipulate outward_claw gear3) (can_manipulate no_tool shaft3) (can_manipulate parallel_box1 shaft1) (can_screw_to leg1 seat) (can_screw_to leg2 seat) (can_insert_to back seat) (can_screw_to nut1 seat) (can_screw_to nut2 seat) (can_screw_to blub lampbase) (can_place_to lamp blub) (can_insert_to shaft1 gearbase_hole1) (can_insert_to shaft3 gearbase_hole3) (can_insert_to gear3 shaft3) (can_insert_to gear2 shaft2) (can_insert_to gear1 shaft1)) - (:goal (and )) - )""" - -parsed_init = parse_init(problem) - -for item in parsed_init: - item_name = item.get("name") - item_type = classify_item(item_name) - - world_state = { - "objects": [], - "constraints": [], - "relations": [], - } - - if item_type == "property": - world_state["objects"].append( - {"name": item["args"][0], "properties": [item_name]} # ! list or item? - ) - - elif item_type == "constraint": - world_state["constraints"].append( - { - "source": item["args"][0], - "name": item_name, - "target": item["args"][1], - } - ) - elif item_type == "relation": - world_state["relations"].append( - { - "source": item["args"][0], - "name": item_name, - "target": item["args"][1], - } - ) - else: - pass - - -print(parsed_init) diff --git a/experiments/gearset1/problem_set/baseline_result_with_old_name.jsonl b/experiments/gearset1/problem_set/baseline_result_with_old_name.jsonl deleted file mode 100644 index 0c397d2f..00000000 --- a/experiments/gearset1/problem_set/baseline_result_with_old_name.jsonl +++ /dev/null @@ -1,17 +0,0 @@ -{"target": "target: is_inserted_to(shaft1, gearbase_hole1)", "initial_world_state": {"objects": [{"name": "clampgripper", "properties": ["is_empty"]}, {"name": "parallelgripper", "properties": ["is_empty"]}, {"name": "inwardgripper", "properties": ["is_empty"]}, {"name": "outwardgripper", "properties": []}, {"name": "defaultgripper", "properties": ["is_empty"]}, {"name": "gear1", "properties": []}, {"name": "gear2", "properties": []}, {"name": "gear3", "properties": []}, {"name": "shaft1", "properties": []}, {"name": "shaft2", "properties": []}, {"name": "shaft3", "properties": []}, {"name": "gearbase", "properties": []}, {"name": "gearbase_hole1", "properties": []}, {"name": "gearbase_hole3", "properties": []}, {"name": "left_hand", "properties": []}, {"name": "parallel_box2", "properties": []}, {"name": "outward_claw", "properties": []}, {"name": "no_tool", "properties": []}, {"name": "parallel_box1", "properties": []}], "constraints": [{"source": "gear3", "name": "can_insert_to", "target": "shaft3"}, {"source": "shaft1", "name": "can_insert_to", "target": "gearbase_hole1"}, {"source": "parallel_box2", "name": "can_manipulate", "target": "gear1"}, {"source": "parallelgripper", "name": "can_manipulate", "target": "gear1"}, {"source": "gear2", "name": "can_insert_to", "target": "shaft2"}, {"source": "outwardgripper", "name": "can_manipulate", "target": "gear3"}, {"source": "gear1", "name": "can_insert_to", "target": "shaft1"}, {"source": "defaultgripper", "name": "can_manipulate", "target": "shaft3"}, {"source": "outward_claw", "name": "can_manipulate", "target": "gear3"}, {"source": "outwardgripper", "name": "can_manipulate", "target": "gear2"}, {"source": "parallel_box1", "name": "can_manipulate", "target": "shaft1"}, {"source": "shaft3", "name": "can_insert_to", "target": "gearbase_hole3"}, {"source": "clampgripper", "name": "can_manipulate", "target": "shaft1"}, {"source": "no_tool", "name": "can_manipulate", "target": "shaft3"}, {"source": "outward_claw", "name": "can_manipulate", "target": "gear2"}], "relations": [{"source": "outwardgripper", "name": "hold", "target": "gear2"}, {"source": "left_hand", "name": "hold", "target": "outwardgripper"}]}, "result": {"summary": "selector to insert the shaft1 into the gearbase_hole1 with the clampgripper in the left_hand", "name": "selector: insert(left_hand, clampgripper, shaft1, gearbase_hole1)", "children": [{"summary": "the target is to make the shaft1 be inserted into the gearbase_hole1", "name": "target: is_inserted_to(shaft1, gearbase_hole1)"}, {"summary": "sequence to insert the shaft1 into the gearbase_hole1 with the clampgripper in the left_hand", "name": "sequence: insert(left_hand, clampgripper, shaft1, gearbase_hole1)", "children": [{"summary": "selector to change tool in left_hand from outwardgripper to clampgripper", "name": "selector: change_tool(left_hand, outwardgripper, clampgripper)", "children": [{"summary": "the target is to make left_hand hold clampgripper", "name": "target: hold(left_hand, clampgripper)"}, {"summary": "sequence to change tool in left_hand from outwardgripper to clampgripper", "name": "sequence: change_tool(left_hand, outwardgripper, clampgripper)", "children": [{"summary": "a precondition is left_hand is holding outwardgripper", "name": "precondition: hold(left_hand, outwardgripper)"}, {"summary": "selector to put down the gear2 held by the outwardgripper in the left_hand", "name": "selector: put_down(left_hand, outwardgripper, gear2)", "children": [{"summary": "the target is to make the outwardgripper empty", "name": "target: is_empty(outwardgripper)"}, {"summary": "sequence to put down the gear2 held by the outwardgripper in the left_hand", "name": "sequence: put_down(left_hand, outwardgripper, gear2)", "children": [{"summary": "a precondition is the left_hand is holding outwardgripper", "name": "precondition: hold(left_hand, outwardgripper)"}, {"summary": "a precondition is the outwardgripper is holding gear2", "name": "precondition: hold(outwardgripper, gear2)"}, {"summary": "the action to put down the gear2 held by the outwardgripper in the left_hand", "name": "action: put_down(left_hand, outwardgripper, gear2)"}]}]}, {"summary": "the action to change tool in left_hand from outwardgripper to clampgripper", "name": "action: change_tool(left_hand, outwardgripper, clampgripper)"}]}]}, {"summary": "selector to pick up the shaft1 with the clampgripper in the left_hand", "name": "selector: pick_up(left_hand, clampgripper, shaft1)", "children": [{"summary": "the target is to make the clampgripper hold the shaft1", "name": "target: hold(clampgripper, shaft1)"}, {"summary": "sequence to pick up the shaft1 with the clampgripper in the left_hand", "name": "sequence: pick_up(left_hand, clampgripper, shaft1)", "children": [{"summary": "a precondition is the clampgripper is empty", "name": "precondition: is_empty(clampgripper)"}, {"summary": "a precondition is the left_hand is holding clampgripper", "name": "precondition: hold(left_hand, clampgripper)"}, {"summary": "the action to pick up the shaft1 with the clampgripper in the left_hand", "name": "action: pick_up(left_hand, clampgripper, shaft1)"}]}]}, {"summary": "the action to insert the shaft1 into the gearbase_hole1 with the clampgripper in the left_hand", "name": "action: insert(left_hand, clampgripper, shaft1, gearbase_hole1)"}]}]}} -{"target": "target: is_inserted_to(shaft1, gearbase_hole1)", "initial_world_state": {"objects": [{"name": "clampgripper", "properties": ["is_empty"]}, {"name": "parallelgripper", "properties": ["is_empty"]}, {"name": "inwardgripper", "properties": ["is_empty"]}, {"name": "outwardgripper", "properties": ["is_empty"]}, {"name": "defaultgripper", "properties": ["is_empty"]}, {"name": "gear1", "properties": []}, {"name": "gear2", "properties": []}, {"name": "gear3", "properties": []}, {"name": "shaft1", "properties": []}, {"name": "shaft2", "properties": []}, {"name": "shaft3", "properties": []}, {"name": "gearbase", "properties": []}, {"name": "gearbase_hole1", "properties": []}, {"name": "gearbase_hole3", "properties": []}, {"name": "left_hand", "properties": []}, {"name": "parallel_box2", "properties": []}, {"name": "outward_claw", "properties": []}, {"name": "no_tool", "properties": []}, {"name": "parallel_box1", "properties": []}], "constraints": [{"source": "outward_claw", "name": "can_manipulate", "target": "gear3"}, {"source": "defaultgripper", "name": "can_manipulate", "target": "shaft3"}, {"source": "parallel_box1", "name": "can_manipulate", "target": "shaft1"}, {"source": "gear2", "name": "can_insert_to", "target": "shaft2"}, {"source": "shaft1", "name": "can_insert_to", "target": "gearbase_hole1"}, {"source": "clampgripper", "name": "can_manipulate", "target": "shaft1"}, {"source": "parallelgripper", "name": "can_manipulate", "target": "gear1"}, {"source": "outward_claw", "name": "can_manipulate", "target": "gear2"}, {"source": "no_tool", "name": "can_manipulate", "target": "shaft3"}, {"source": "parallel_box2", "name": "can_manipulate", "target": "gear1"}, {"source": "outwardgripper", "name": "can_manipulate", "target": "gear3"}, {"source": "gear1", "name": "can_insert_to", "target": "shaft1"}, {"source": "shaft3", "name": "can_insert_to", "target": "gearbase_hole3"}, {"source": "outwardgripper", "name": "can_manipulate", "target": "gear2"}, {"source": "gear3", "name": "can_insert_to", "target": "shaft3"}], "relations": [{"source": "left_hand", "name": "hold", "target": "outwardgripper"}]}, "result": {"summary": "selector to insert the shaft1 into the gearbase_hole1 with the clampgripper in the left_hand", "name": "selector: insert(left_hand, clampgripper, shaft1, gearbase_hole1)", "children": [{"summary": "the target is to make the shaft1 be inserted into the gearbase_hole1", "name": "target: is_inserted_to(shaft1, gearbase_hole1)"}, {"summary": "sequence to insert the shaft1 into the gearbase_hole1 with the clampgripper in the left_hand", "name": "sequence: insert(left_hand, clampgripper, shaft1, gearbase_hole1)", "children": [{"summary": "selector to change tool in left_hand from outwardgripper to clampgripper", "name": "selector: change_tool(left_hand, outwardgripper, clampgripper)", "children": [{"summary": "the target is to make left_hand hold clampgripper", "name": "target: hold(left_hand, clampgripper)"}, {"summary": "sequence to change tool in left_hand from outwardgripper to clampgripper", "name": "sequence: change_tool(left_hand, outwardgripper, clampgripper)", "children": [{"summary": "a precondition is left_hand is holding outwardgripper", "name": "precondition: hold(left_hand, outwardgripper)"}, {"summary": "a precondition is outwardgripper is empty", "name": "precondition: is_empty(outwardgripper)"}, {"summary": "the action to change tool in left_hand from outwardgripper to clampgripper", "name": "action: change_tool(left_hand, outwardgripper, clampgripper)"}]}]}, {"summary": "selector to pick up the shaft1 with the clampgripper in the left_hand", "name": "selector: pick_up(left_hand, clampgripper, shaft1)", "children": [{"summary": "the target is to make the clampgripper hold the shaft1", "name": "target: hold(clampgripper, shaft1)"}, {"summary": "sequence to pick up the shaft1 with the clampgripper in the left_hand", "name": "sequence: pick_up(left_hand, clampgripper, shaft1)", "children": [{"summary": "a precondition is the clampgripper is empty", "name": "precondition: is_empty(clampgripper)"}, {"summary": "a precondition is the left_hand is holding clampgripper", "name": "precondition: hold(left_hand, clampgripper)"}, {"summary": "the action to pick up the shaft1 with the clampgripper in the left_hand", "name": "action: pick_up(left_hand, clampgripper, shaft1)"}]}]}, {"summary": "the action to insert the shaft1 into the gearbase_hole1 with the clampgripper in the left_hand", "name": "action: insert(left_hand, clampgripper, shaft1, gearbase_hole1)"}]}]}} -{"target": "target: is_inserted_to(gear2, shaft2)", "initial_world_state": {"objects": [{"name": "clampgripper", "properties": ["is_empty"]}, {"name": "parallelgripper", "properties": ["is_empty"]}, {"name": "inwardgripper", "properties": ["is_empty"]}, {"name": "outwardgripper", "properties": ["is_empty"]}, {"name": "defaultgripper", "properties": ["is_empty"]}, {"name": "gear1", "properties": []}, {"name": "gear2", "properties": []}, {"name": "gear3", "properties": []}, {"name": "shaft1", "properties": []}, {"name": "shaft2", "properties": []}, {"name": "shaft3", "properties": []}, {"name": "gearbase", "properties": []}, {"name": "gearbase_hole1", "properties": []}, {"name": "gearbase_hole3", "properties": []}, {"name": "left_hand", "properties": []}, {"name": "parallel_box2", "properties": []}, {"name": "outward_claw", "properties": []}, {"name": "no_tool", "properties": []}, {"name": "parallel_box1", "properties": []}], "constraints": [{"source": "shaft1", "name": "can_insert_to", "target": "gearbase_hole1"}, {"source": "clampgripper", "name": "can_manipulate", "target": "shaft1"}, {"source": "outward_claw", "name": "can_manipulate", "target": "gear3"}, {"source": "outwardgripper", "name": "can_manipulate", "target": "gear3"}, {"source": "gear3", "name": "can_insert_to", "target": "shaft3"}, {"source": "parallel_box2", "name": "can_manipulate", "target": "gear1"}, {"source": "defaultgripper", "name": "can_manipulate", "target": "shaft3"}, {"source": "gear2", "name": "can_insert_to", "target": "shaft2"}, {"source": "shaft3", "name": "can_insert_to", "target": "gearbase_hole3"}, {"source": "gear1", "name": "can_insert_to", "target": "shaft1"}, {"source": "no_tool", "name": "can_manipulate", "target": "shaft3"}, {"source": "parallel_box1", "name": "can_manipulate", "target": "shaft1"}, {"source": "outwardgripper", "name": "can_manipulate", "target": "gear2"}, {"source": "parallelgripper", "name": "can_manipulate", "target": "gear1"}, {"source": "outward_claw", "name": "can_manipulate", "target": "gear2"}], "relations": [{"source": "left_hand", "name": "hold", "target": "outwardgripper"}]}, "result": {"summary": "selector to insert the gear2 into the shaft2 with the outwardgripper in the left_hand", "name": "selector: insert(left_hand, outwardgripper, gear2, shaft2)", "children": [{"summary": "the target is to make the gear2 be inserted into the shaft2", "name": "target: is_inserted_to(gear2, shaft2)"}, {"summary": "sequence to insert the gear2 into the shaft2 with the outwardgripper in the left_hand", "name": "sequence: insert(left_hand, outwardgripper, gear2, shaft2)", "children": [{"summary": "a precondition is the left_hand is holding outwardgripper", "name": "precondition: hold(left_hand, outwardgripper)"}, {"summary": "selector to pick up the gear2 with the outwardgripper in the left_hand", "name": "selector: pick_up(left_hand, outwardgripper, gear2)", "children": [{"summary": "the target is to make the outwardgripper hold the gear2", "name": "target: hold(outwardgripper, gear2)"}, {"summary": "sequence to pick up the gear2 with the outwardgripper in the left_hand", "name": "sequence: pick_up(left_hand, outwardgripper, gear2)", "children": [{"summary": "a precondition is the outwardgripper is empty", "name": "precondition: is_empty(outwardgripper)"}, {"summary": "a precondition is the left_hand is holding outwardgripper", "name": "precondition: hold(left_hand, outwardgripper)"}, {"summary": "the action to pick up the gear2 with the outwardgripper in the left_hand", "name": "action: pick_up(left_hand, outwardgripper, gear2)"}]}]}, {"summary": "the action to insert the gear2 into the shaft2 with the outwardgripper in the left_hand", "name": "action: insert(left_hand, outwardgripper, gear2, shaft2)"}]}]}} -{"target": "target: is_inserted_to(gear2, shaft2)", "initial_world_state": {"objects": [{"name": "clampgripper", "properties": []}, {"name": "parallelgripper", "properties": ["is_empty"]}, {"name": "inwardgripper", "properties": ["is_empty"]}, {"name": "outwardgripper", "properties": ["is_empty"]}, {"name": "defaultgripper", "properties": ["is_empty"]}, {"name": "gear1", "properties": []}, {"name": "gear2", "properties": []}, {"name": "gear3", "properties": []}, {"name": "shaft1", "properties": []}, {"name": "shaft2", "properties": []}, {"name": "shaft3", "properties": []}, {"name": "gearbase", "properties": []}, {"name": "gearbase_hole1", "properties": []}, {"name": "gearbase_hole3", "properties": []}, {"name": "left_hand", "properties": []}, {"name": "parallel_box2", "properties": []}, {"name": "outward_claw", "properties": []}, {"name": "no_tool", "properties": []}, {"name": "parallel_box1", "properties": []}], "constraints": [{"source": "gear1", "name": "can_insert_to", "target": "shaft1"}, {"source": "parallel_box2", "name": "can_manipulate", "target": "gear1"}, {"source": "defaultgripper", "name": "can_manipulate", "target": "shaft3"}, {"source": "outwardgripper", "name": "can_manipulate", "target": "gear2"}, {"source": "shaft1", "name": "can_insert_to", "target": "gearbase_hole1"}, {"source": "outward_claw", "name": "can_manipulate", "target": "gear3"}, {"source": "gear2", "name": "can_insert_to", "target": "shaft2"}, {"source": "no_tool", "name": "can_manipulate", "target": "shaft3"}, {"source": "gear3", "name": "can_insert_to", "target": "shaft3"}, {"source": "shaft3", "name": "can_insert_to", "target": "gearbase_hole3"}, {"source": "outwardgripper", "name": "can_manipulate", "target": "gear3"}, {"source": "outward_claw", "name": "can_manipulate", "target": "gear2"}, {"source": "parallelgripper", "name": "can_manipulate", "target": "gear1"}, {"source": "clampgripper", "name": "can_manipulate", "target": "shaft1"}, {"source": "parallel_box1", "name": "can_manipulate", "target": "shaft1"}], "relations": [{"source": "clampgripper", "name": "hold", "target": "shaft1"}, {"source": "left_hand", "name": "hold", "target": "clampgripper"}]}, "result": {"summary": "selector to insert the gear2 into the shaft2 with the outwardgripper in the left_hand", "name": "selector: insert(left_hand, outwardgripper, gear2, shaft2)", "children": [{"summary": "the target is to make the gear2 be inserted into the shaft2", "name": "target: is_inserted_to(gear2, shaft2)"}, {"summary": "sequence to insert the gear2 into the shaft2 with the outwardgripper in the left_hand", "name": "sequence: insert(left_hand, outwardgripper, gear2, shaft2)", "children": [{"summary": "selector to change tool in left_hand from clampgripper to outwardgripper", "name": "selector: change_tool(left_hand, clampgripper, outwardgripper)", "children": [{"summary": "the target is to make left_hand hold outwardgripper", "name": "target: hold(left_hand, outwardgripper)"}, {"summary": "sequence to change tool in left_hand from clampgripper to outwardgripper", "name": "sequence: change_tool(left_hand, clampgripper, outwardgripper)", "children": [{"summary": "a precondition is left_hand is holding clampgripper", "name": "precondition: hold(left_hand, clampgripper)"}, {"summary": "selector to put down the shaft1 held by the clampgripper in the left_hand", "name": "selector: put_down(left_hand, clampgripper, shaft1)", "children": [{"summary": "the target is to make the clampgripper empty", "name": "target: is_empty(clampgripper)"}, {"summary": "sequence to put down the shaft1 held by the clampgripper in the left_hand", "name": "sequence: put_down(left_hand, clampgripper, shaft1)", "children": [{"summary": "a precondition is the left_hand is holding clampgripper", "name": "precondition: hold(left_hand, clampgripper)"}, {"summary": "a precondition is the clampgripper is holding shaft1", "name": "precondition: hold(clampgripper, shaft1)"}, {"summary": "the action to put down the shaft1 held by the clampgripper in the left_hand", "name": "action: put_down(left_hand, clampgripper, shaft1)"}]}]}, {"summary": "the action to change tool in left_hand from clampgripper to outwardgripper", "name": "action: change_tool(left_hand, clampgripper, outwardgripper)"}]}]}, {"summary": "selector to pick up the gear2 with the outwardgripper in the left_hand", "name": "selector: pick_up(left_hand, outwardgripper, gear2)", "children": [{"summary": "the target is to make the outwardgripper hold the gear2", "name": "target: hold(outwardgripper, gear2)"}, {"summary": "sequence to pick up the gear2 with the outwardgripper in the left_hand", "name": "sequence: pick_up(left_hand, outwardgripper, gear2)", "children": [{"summary": "a precondition is the outwardgripper is empty", "name": "precondition: is_empty(outwardgripper)"}, {"summary": "a precondition is the left_hand is holding outwardgripper", "name": "precondition: hold(left_hand, outwardgripper)"}, {"summary": "the action to pick up the gear2 with the outwardgripper in the left_hand", "name": "action: pick_up(left_hand, outwardgripper, gear2)"}]}]}, {"summary": "the action to insert the gear2 into the shaft2 with the outwardgripper in the left_hand", "name": "action: insert(left_hand, outwardgripper, gear2, shaft2)"}]}]}} -{"target": "target: is_inserted_to(gear3, shaft3)", "initial_world_state": {"objects": [{"name": "clampgripper", "properties": []}, {"name": "parallelgripper", "properties": ["is_empty"]}, {"name": "inwardgripper", "properties": ["is_empty"]}, {"name": "outwardgripper", "properties": ["is_empty"]}, {"name": "defaultgripper", "properties": ["is_empty"]}, {"name": "gear1", "properties": []}, {"name": "gear2", "properties": []}, {"name": "gear3", "properties": []}, {"name": "shaft1", "properties": []}, {"name": "shaft2", "properties": []}, {"name": "shaft3", "properties": []}, {"name": "gearbase", "properties": []}, {"name": "gearbase_hole1", "properties": []}, {"name": "gearbase_hole3", "properties": []}, {"name": "left_hand", "properties": []}, {"name": "parallel_box2", "properties": []}, {"name": "outward_claw", "properties": []}, {"name": "no_tool", "properties": []}, {"name": "parallel_box1", "properties": []}], "constraints": [{"source": "clampgripper", "name": "can_manipulate", "target": "shaft1"}, {"source": "gear3", "name": "can_insert_to", "target": "shaft3"}, {"source": "outwardgripper", "name": "can_manipulate", "target": "gear3"}, {"source": "no_tool", "name": "can_manipulate", "target": "shaft3"}, {"source": "shaft3", "name": "can_insert_to", "target": "gearbase_hole3"}, {"source": "gear1", "name": "can_insert_to", "target": "shaft1"}, {"source": "parallelgripper", "name": "can_manipulate", "target": "gear1"}, {"source": "defaultgripper", "name": "can_manipulate", "target": "shaft3"}, {"source": "outward_claw", "name": "can_manipulate", "target": "gear2"}, {"source": "gear2", "name": "can_insert_to", "target": "shaft2"}, {"source": "parallel_box1", "name": "can_manipulate", "target": "shaft1"}, {"source": "parallel_box2", "name": "can_manipulate", "target": "gear1"}, {"source": "shaft1", "name": "can_insert_to", "target": "gearbase_hole1"}, {"source": "outwardgripper", "name": "can_manipulate", "target": "gear2"}, {"source": "outward_claw", "name": "can_manipulate", "target": "gear3"}], "relations": [{"source": "left_hand", "name": "hold", "target": "clampgripper"}, {"source": "clampgripper", "name": "hold", "target": "shaft1"}]}, "result": {"summary": "selector to insert the gear3 into the shaft3 with the outwardgripper in the left_hand", "name": "selector: insert(left_hand, outwardgripper, gear3, shaft3)", "children": [{"summary": "the target is to make the gear3 be inserted into the shaft3", "name": "target: is_inserted_to(gear3, shaft3)"}, {"summary": "sequence to insert the gear3 into the shaft3 with the outwardgripper in the left_hand", "name": "sequence: insert(left_hand, outwardgripper, gear3, shaft3)", "children": [{"summary": "selector to change tool in left_hand from clampgripper to outwardgripper", "name": "selector: change_tool(left_hand, clampgripper, outwardgripper)", "children": [{"summary": "the target is to make left_hand hold outwardgripper", "name": "target: hold(left_hand, outwardgripper)"}, {"summary": "sequence to change tool in left_hand from clampgripper to outwardgripper", "name": "sequence: change_tool(left_hand, clampgripper, outwardgripper)", "children": [{"summary": "a precondition is left_hand is holding clampgripper", "name": "precondition: hold(left_hand, clampgripper)"}, {"summary": "selector to put down the shaft1 held by the clampgripper in the left_hand", "name": "selector: put_down(left_hand, clampgripper, shaft1)", "children": [{"summary": "the target is to make the clampgripper empty", "name": "target: is_empty(clampgripper)"}, {"summary": "sequence to put down the shaft1 held by the clampgripper in the left_hand", "name": "sequence: put_down(left_hand, clampgripper, shaft1)", "children": [{"summary": "a precondition is the left_hand is holding clampgripper", "name": "precondition: hold(left_hand, clampgripper)"}, {"summary": "a precondition is the clampgripper is holding shaft1", "name": "precondition: hold(clampgripper, shaft1)"}, {"summary": "the action to put down the shaft1 held by the clampgripper in the left_hand", "name": "action: put_down(left_hand, clampgripper, shaft1)"}]}]}, {"summary": "the action to change tool in left_hand from clampgripper to outwardgripper", "name": "action: change_tool(left_hand, clampgripper, outwardgripper)"}]}]}, {"summary": "selector to pick up the gear3 with the outwardgripper in the left_hand", "name": "selector: pick_up(left_hand, outwardgripper, gear3)", "children": [{"summary": "the target is to make the outwardgripper hold the gear3", "name": "target: hold(outwardgripper, gear3)"}, {"summary": "sequence to pick up the gear3 with the outwardgripper in the left_hand", "name": "sequence: pick_up(left_hand, outwardgripper, gear3)", "children": [{"summary": "a precondition is the outwardgripper is empty", "name": "precondition: is_empty(outwardgripper)"}, {"summary": "a precondition is the left_hand is holding outwardgripper", "name": "precondition: hold(left_hand, outwardgripper)"}, {"summary": "the action to pick up the gear3 with the outwardgripper in the left_hand", "name": "action: pick_up(left_hand, outwardgripper, gear3)"}]}]}, {"summary": "the action to insert the gear3 into the shaft3 with the outwardgripper in the left_hand", "name": "action: insert(left_hand, outwardgripper, gear3, shaft3)"}]}]}} -{"target": "target: is_inserted_to(gear3, shaft3)", "initial_world_state": {"objects": [{"name": "clampgripper", "properties": ["is_empty"]}, {"name": "parallelgripper", "properties": ["is_empty"]}, {"name": "inwardgripper", "properties": ["is_empty"]}, {"name": "outwardgripper", "properties": ["is_empty"]}, {"name": "defaultgripper", "properties": ["is_empty"]}, {"name": "gear1", "properties": []}, {"name": "gear2", "properties": []}, {"name": "gear3", "properties": []}, {"name": "shaft1", "properties": []}, {"name": "shaft2", "properties": []}, {"name": "shaft3", "properties": []}, {"name": "gearbase", "properties": []}, {"name": "gearbase_hole1", "properties": []}, {"name": "gearbase_hole3", "properties": []}, {"name": "left_hand", "properties": []}, {"name": "parallel_box2", "properties": []}, {"name": "outward_claw", "properties": []}, {"name": "no_tool", "properties": []}, {"name": "parallel_box1", "properties": []}], "constraints": [{"source": "gear2", "name": "can_insert_to", "target": "shaft2"}, {"source": "outwardgripper", "name": "can_manipulate", "target": "gear2"}, {"source": "outward_claw", "name": "can_manipulate", "target": "gear3"}, {"source": "outward_claw", "name": "can_manipulate", "target": "gear2"}, {"source": "shaft3", "name": "can_insert_to", "target": "gearbase_hole3"}, {"source": "clampgripper", "name": "can_manipulate", "target": "shaft1"}, {"source": "shaft1", "name": "can_insert_to", "target": "gearbase_hole1"}, {"source": "outwardgripper", "name": "can_manipulate", "target": "gear3"}, {"source": "parallel_box1", "name": "can_manipulate", "target": "shaft1"}, {"source": "gear1", "name": "can_insert_to", "target": "shaft1"}, {"source": "defaultgripper", "name": "can_manipulate", "target": "shaft3"}, {"source": "no_tool", "name": "can_manipulate", "target": "shaft3"}, {"source": "parallel_box2", "name": "can_manipulate", "target": "gear1"}, {"source": "gear3", "name": "can_insert_to", "target": "shaft3"}, {"source": "parallelgripper", "name": "can_manipulate", "target": "gear1"}], "relations": [{"source": "left_hand", "name": "hold", "target": "defaultgripper"}]}, "result": {"summary": "selector to insert the gear3 into the shaft3 with the outwardgripper in the left_hand", "name": "selector: insert(left_hand, outwardgripper, gear3, shaft3)", "children": [{"summary": "the target is to make the gear3 be inserted into the shaft3", "name": "target: is_inserted_to(gear3, shaft3)"}, {"summary": "sequence to insert the gear3 into the shaft3 with the outwardgripper in the left_hand", "name": "sequence: insert(left_hand, outwardgripper, gear3, shaft3)", "children": [{"summary": "selector to change tool in left_hand from clampgripper to outwardgripper", "name": "selector: change_tool(left_hand, clampgripper, outwardgripper)", "children": [{"summary": "the target is to make left_hand hold outwardgripper", "name": "target: hold(left_hand, outwardgripper)"}, {"summary": "sequence to change tool in left_hand from clampgripper to outwardgripper", "name": "sequence: change_tool(left_hand, clampgripper, outwardgripper)", "children": [{"summary": "selector to change tool in left_hand from defaultgripper to clampgripper", "name": "selector: change_tool(left_hand, defaultgripper, clampgripper)", "children": [{"summary": "the target is to make left_hand hold clampgripper", "name": "target: hold(left_hand, clampgripper)"}, {"summary": "sequence to change tool in left_hand from defaultgripper to clampgripper", "name": "sequence: change_tool(left_hand, defaultgripper, clampgripper)", "children": [{"summary": "a precondition is left_hand is holding defaultgripper", "name": "precondition: hold(left_hand, defaultgripper)"}, {"summary": "a precondition is defaultgripper is empty", "name": "precondition: is_empty(defaultgripper)"}, {"summary": "the action to change tool in left_hand from defaultgripper to clampgripper", "name": "action: change_tool(left_hand, defaultgripper, clampgripper)"}]}]}, {"summary": "a precondition is clampgripper is empty", "name": "precondition: is_empty(clampgripper)"}, {"summary": "the action to change tool in left_hand from clampgripper to outwardgripper", "name": "action: change_tool(left_hand, clampgripper, outwardgripper)"}]}]}, {"summary": "selector to pick up the gear3 with the outwardgripper in the left_hand", "name": "selector: pick_up(left_hand, outwardgripper, gear3)", "children": [{"summary": "the target is to make the outwardgripper hold the gear3", "name": "target: hold(outwardgripper, gear3)"}, {"summary": "sequence to pick up the gear3 with the outwardgripper in the left_hand", "name": "sequence: pick_up(left_hand, outwardgripper, gear3)", "children": [{"summary": "a precondition is the outwardgripper is empty", "name": "precondition: is_empty(outwardgripper)"}, {"summary": "a precondition is the left_hand is holding outwardgripper", "name": "precondition: hold(left_hand, outwardgripper)"}, {"summary": "the action to pick up the gear3 with the outwardgripper in the left_hand", "name": "action: pick_up(left_hand, outwardgripper, gear3)"}]}]}, {"summary": "the action to insert the gear3 into the shaft3 with the outwardgripper in the left_hand", "name": "action: insert(left_hand, outwardgripper, gear3, shaft3)"}]}]}} -{"target": "target: is_inserted_to(gear3, shaft3)", "initial_world_state": {"objects": [{"name": "clampgripper", "properties": ["is_empty"]}, {"name": "parallelgripper", "properties": ["is_empty"]}, {"name": "inwardgripper", "properties": ["is_empty"]}, {"name": "outwardgripper", "properties": ["is_empty"]}, {"name": "defaultgripper", "properties": ["is_empty"]}, {"name": "gear1", "properties": []}, {"name": "gear2", "properties": []}, {"name": "gear3", "properties": []}, {"name": "shaft1", "properties": []}, {"name": "shaft2", "properties": []}, {"name": "shaft3", "properties": []}, {"name": "gearbase", "properties": []}, {"name": "gearbase_hole1", "properties": []}, {"name": "gearbase_hole3", "properties": []}, {"name": "left_hand", "properties": []}, {"name": "parallel_box2", "properties": []}, {"name": "outward_claw", "properties": []}, {"name": "no_tool", "properties": []}, {"name": "parallel_box1", "properties": []}], "constraints": [{"source": "outwardgripper", "name": "can_manipulate", "target": "gear2"}, {"source": "clampgripper", "name": "can_manipulate", "target": "shaft1"}, {"source": "no_tool", "name": "can_manipulate", "target": "shaft3"}, {"source": "parallel_box2", "name": "can_manipulate", "target": "gear1"}, {"source": "defaultgripper", "name": "can_manipulate", "target": "shaft3"}, {"source": "parallel_box1", "name": "can_manipulate", "target": "shaft1"}, {"source": "gear2", "name": "can_insert_to", "target": "shaft2"}, {"source": "outward_claw", "name": "can_manipulate", "target": "gear2"}, {"source": "outward_claw", "name": "can_manipulate", "target": "gear3"}, {"source": "shaft1", "name": "can_insert_to", "target": "gearbase_hole1"}, {"source": "gear3", "name": "can_insert_to", "target": "shaft3"}, {"source": "shaft3", "name": "can_insert_to", "target": "gearbase_hole3"}, {"source": "parallelgripper", "name": "can_manipulate", "target": "gear1"}, {"source": "outwardgripper", "name": "can_manipulate", "target": "gear3"}, {"source": "gear1", "name": "can_insert_to", "target": "shaft1"}], "relations": [{"source": "left_hand", "name": "hold", "target": "outwardgripper"}]}, "result": {"summary": "selector to insert the gear3 into the shaft3 with the outwardgripper in the left_hand", "name": "selector: insert(left_hand, outwardgripper, gear3, shaft3)", "children": [{"summary": "the target is to make the gear3 be inserted into the shaft3", "name": "target: is_inserted_to(gear3, shaft3)"}, {"summary": "sequence to insert the gear3 into the shaft3 with the outwardgripper in the left_hand", "name": "sequence: insert(left_hand, outwardgripper, gear3, shaft3)", "children": [{"summary": "a precondition is the left_hand is holding outwardgripper", "name": "precondition: hold(left_hand, outwardgripper)"}, {"summary": "selector to pick up the gear3 with the outwardgripper in the left_hand", "name": "selector: pick_up(left_hand, outwardgripper, gear3)", "children": [{"summary": "the target is to make the outwardgripper hold the gear3", "name": "target: hold(outwardgripper, gear3)"}, {"summary": "sequence to pick up the gear3 with the outwardgripper in the left_hand", "name": "sequence: pick_up(left_hand, outwardgripper, gear3)", "children": [{"summary": "a precondition is the outwardgripper is empty", "name": "precondition: is_empty(outwardgripper)"}, {"summary": "a precondition is the left_hand is holding outwardgripper", "name": "precondition: hold(left_hand, outwardgripper)"}, {"summary": "the action to pick up the gear3 with the outwardgripper in the left_hand", "name": "action: pick_up(left_hand, outwardgripper, gear3)"}]}]}, {"summary": "the action to insert the gear3 into the shaft3 with the outwardgripper in the left_hand", "name": "action: insert(left_hand, outwardgripper, gear3, shaft3)"}]}]}} -{"target": "target: is_inserted_to(gear1, shaft1)", "initial_world_state": {"objects": [{"name": "clampgripper", "properties": ["is_empty"]}, {"name": "parallelgripper", "properties": ["is_empty"]}, {"name": "inwardgripper", "properties": ["is_empty"]}, {"name": "outwardgripper", "properties": ["is_empty"]}, {"name": "defaultgripper", "properties": ["is_empty"]}, {"name": "gear1", "properties": []}, {"name": "gear2", "properties": []}, {"name": "gear3", "properties": []}, {"name": "shaft1", "properties": []}, {"name": "shaft2", "properties": []}, {"name": "shaft3", "properties": []}, {"name": "gearbase", "properties": []}, {"name": "gearbase_hole1", "properties": []}, {"name": "gearbase_hole3", "properties": []}, {"name": "left_hand", "properties": []}, {"name": "parallel_box2", "properties": []}, {"name": "outward_claw", "properties": []}, {"name": "no_tool", "properties": []}, {"name": "parallel_box1", "properties": []}], "constraints": [{"source": "clampgripper", "name": "can_manipulate", "target": "shaft1"}, {"source": "outwardgripper", "name": "can_manipulate", "target": "gear2"}, {"source": "parallelgripper", "name": "can_manipulate", "target": "gear1"}, {"source": "outward_claw", "name": "can_manipulate", "target": "gear3"}, {"source": "gear1", "name": "can_insert_to", "target": "shaft1"}, {"source": "shaft1", "name": "can_insert_to", "target": "gearbase_hole1"}, {"source": "gear2", "name": "can_insert_to", "target": "shaft2"}, {"source": "parallel_box1", "name": "can_manipulate", "target": "shaft1"}, {"source": "no_tool", "name": "can_manipulate", "target": "shaft3"}, {"source": "parallel_box2", "name": "can_manipulate", "target": "gear1"}, {"source": "outwardgripper", "name": "can_manipulate", "target": "gear3"}, {"source": "outward_claw", "name": "can_manipulate", "target": "gear2"}, {"source": "gear3", "name": "can_insert_to", "target": "shaft3"}, {"source": "shaft3", "name": "can_insert_to", "target": "gearbase_hole3"}, {"source": "defaultgripper", "name": "can_manipulate", "target": "shaft3"}], "relations": [{"source": "shaft1", "name": "is_inserted_to", "target": "gearbase_hole1"}, {"source": "left_hand", "name": "hold", "target": "outwardgripper"}]}, "result": {"summary": "selector to insert the gear1 into the shaft1 with the parallelgripper in the left_hand", "name": "selector: insert(left_hand, parallelgripper, gear1, shaft1)", "children": [{"summary": "the target is to make the gear1 be inserted into the shaft1", "name": "target: is_inserted_to(gear1, shaft1)"}, {"summary": "sequence to insert the gear1 into the shaft1 with the parallelgripper in the left_hand", "name": "sequence: insert(left_hand, parallelgripper, gear1, shaft1)", "children": [{"summary": "selector to change tool in left_hand from outwardgripper to parallelgripper", "name": "selector: change_tool(left_hand, outwardgripper, parallelgripper)", "children": [{"summary": "the target is to make left_hand hold parallelgripper", "name": "target: hold(left_hand, parallelgripper)"}, {"summary": "sequence to change tool in left_hand from outwardgripper to parallelgripper", "name": "sequence: change_tool(left_hand, outwardgripper, parallelgripper)", "children": [{"summary": "a precondition is left_hand is holding outwardgripper", "name": "precondition: hold(left_hand, outwardgripper)"}, {"summary": "a precondition is outwardgripper is empty", "name": "precondition: is_empty(outwardgripper)"}, {"summary": "the action to change tool in left_hand from outwardgripper to parallelgripper", "name": "action: change_tool(left_hand, outwardgripper, parallelgripper)"}]}]}, {"summary": "selector to pick up the gear1 with the parallelgripper in the left_hand", "name": "selector: pick_up(left_hand, parallelgripper, gear1)", "children": [{"summary": "the target is to make the parallelgripper hold the gear1", "name": "target: hold(parallelgripper, gear1)"}, {"summary": "sequence to pick up the gear1 with the parallelgripper in the left_hand", "name": "sequence: pick_up(left_hand, parallelgripper, gear1)", "children": [{"summary": "a precondition is the parallelgripper is empty", "name": "precondition: is_empty(parallelgripper)"}, {"summary": "a precondition is the left_hand is holding parallelgripper", "name": "precondition: hold(left_hand, parallelgripper)"}, {"summary": "the action to pick up the gear1 with the parallelgripper in the left_hand", "name": "action: pick_up(left_hand, parallelgripper, gear1)"}]}]}, {"summary": "the action to insert the gear1 into the shaft1 with the parallelgripper in the left_hand", "name": "action: insert(left_hand, parallelgripper, gear1, shaft1)"}]}]}} -{"target": "target: is_inserted_to(gear1, shaft1)", "initial_world_state": {"objects": [{"name": "clampgripper", "properties": ["is_empty"]}, {"name": "parallelgripper", "properties": ["is_empty"]}, {"name": "inwardgripper", "properties": ["is_empty"]}, {"name": "outwardgripper", "properties": ["is_empty"]}, {"name": "defaultgripper", "properties": ["is_empty"]}, {"name": "gear1", "properties": []}, {"name": "gear2", "properties": []}, {"name": "gear3", "properties": []}, {"name": "shaft1", "properties": []}, {"name": "shaft2", "properties": []}, {"name": "shaft3", "properties": []}, {"name": "gearbase", "properties": []}, {"name": "gearbase_hole1", "properties": []}, {"name": "gearbase_hole3", "properties": []}, {"name": "left_hand", "properties": []}, {"name": "parallel_box2", "properties": []}, {"name": "outward_claw", "properties": []}, {"name": "no_tool", "properties": []}, {"name": "parallel_box1", "properties": []}], "constraints": [{"source": "outward_claw", "name": "can_manipulate", "target": "gear3"}, {"source": "no_tool", "name": "can_manipulate", "target": "shaft3"}, {"source": "parallelgripper", "name": "can_manipulate", "target": "gear1"}, {"source": "outwardgripper", "name": "can_manipulate", "target": "gear3"}, {"source": "gear2", "name": "can_insert_to", "target": "shaft2"}, {"source": "defaultgripper", "name": "can_manipulate", "target": "shaft3"}, {"source": "outwardgripper", "name": "can_manipulate", "target": "gear2"}, {"source": "gear3", "name": "can_insert_to", "target": "shaft3"}, {"source": "parallel_box1", "name": "can_manipulate", "target": "shaft1"}, {"source": "clampgripper", "name": "can_manipulate", "target": "shaft1"}, {"source": "outward_claw", "name": "can_manipulate", "target": "gear2"}, {"source": "parallel_box2", "name": "can_manipulate", "target": "gear1"}, {"source": "shaft3", "name": "can_insert_to", "target": "gearbase_hole3"}, {"source": "shaft1", "name": "can_insert_to", "target": "gearbase_hole1"}, {"source": "gear1", "name": "can_insert_to", "target": "shaft1"}], "relations": [{"source": "shaft1", "name": "is_inserted_to", "target": "gearbase_hole1"}, {"source": "left_hand", "name": "hold", "target": "outwardgripper"}, {"source": "gear1", "name": "is_inserted_to", "target": "shaft1"}]}, "result": {"summary": "the target is to insert gear1 to shaft1", "name": "target: is_inserted_to(gear1, shaft1)"}} -{"target": "target: is_inserted_to(gear3, shaft3)", "initial_world_state": {"objects": [{"name": "clampgripper", "properties": ["is_empty"]}, {"name": "parallelgripper", "properties": ["is_empty"]}, {"name": "inwardgripper", "properties": ["is_empty"]}, {"name": "outwardgripper", "properties": ["is_empty"]}, {"name": "defaultgripper", "properties": ["is_empty"]}, {"name": "gear1", "properties": []}, {"name": "gear2", "properties": []}, {"name": "gear3", "properties": []}, {"name": "shaft1", "properties": []}, {"name": "shaft2", "properties": []}, {"name": "shaft3", "properties": []}, {"name": "gearbase", "properties": []}, {"name": "gearbase_hole1", "properties": []}, {"name": "gearbase_hole3", "properties": []}, {"name": "left_hand", "properties": []}, {"name": "parallel_box2", "properties": []}, {"name": "outward_claw", "properties": []}, {"name": "no_tool", "properties": []}, {"name": "parallel_box1", "properties": []}], "constraints": [{"source": "parallel_box1", "name": "can_manipulate", "target": "shaft1"}, {"source": "gear2", "name": "can_insert_to", "target": "shaft2"}, {"source": "parallelgripper", "name": "can_manipulate", "target": "gear1"}, {"source": "gear3", "name": "can_insert_to", "target": "shaft3"}, {"source": "defaultgripper", "name": "can_manipulate", "target": "shaft3"}, {"source": "outwardgripper", "name": "can_manipulate", "target": "gear2"}, {"source": "gear1", "name": "can_insert_to", "target": "shaft1"}, {"source": "outwardgripper", "name": "can_manipulate", "target": "gear3"}, {"source": "outward_claw", "name": "can_manipulate", "target": "gear2"}, {"source": "parallel_box2", "name": "can_manipulate", "target": "gear1"}, {"source": "shaft1", "name": "can_insert_to", "target": "gearbase_hole1"}, {"source": "outward_claw", "name": "can_manipulate", "target": "gear3"}, {"source": "no_tool", "name": "can_manipulate", "target": "shaft3"}, {"source": "clampgripper", "name": "can_manipulate", "target": "shaft1"}, {"source": "shaft3", "name": "can_insert_to", "target": "gearbase_hole3"}], "relations": [{"source": "gear1", "name": "is_inserted_to", "target": "shaft1"}, {"source": "shaft1", "name": "is_inserted_to", "target": "gearbase_hole1"}, {"source": "shaft3", "name": "is_inserted_to", "target": "gearbase_hole3"}, {"source": "left_hand", "name": "hold", "target": "defaultgripper"}]}, "result": {"summary": "selector to insert the gear3 into the shaft3 with the outwardgripper in the left_hand", "name": "selector: insert(left_hand, outwardgripper, gear3, shaft3)", "children": [{"summary": "the target is to make the gear3 be inserted into the shaft3", "name": "target: is_inserted_to(gear3, shaft3)"}, {"summary": "sequence to insert the gear3 into the shaft3 with the outwardgripper in the left_hand", "name": "sequence: insert(left_hand, outwardgripper, gear3, shaft3)", "children": [{"summary": "selector to change tool in left_hand from defaultgripper to outwardgripper", "name": "selector: change_tool(left_hand, defaultgripper, outwardgripper)", "children": [{"summary": "the target is to make left_hand hold outwardgripper", "name": "target: hold(left_hand, outwardgripper)"}, {"summary": "sequence to change tool in left_hand from defaultgripper to outwardgripper", "name": "sequence: change_tool(left_hand, defaultgripper, outwardgripper)", "children": [{"summary": "a precondition is left_hand is holding defaultgripper", "name": "precondition: hold(left_hand, defaultgripper)"}, {"summary": "a precondition is defaultgripper is empty", "name": "precondition: is_empty(defaultgripper)"}, {"summary": "the action to change tool in left_hand from defaultgripper to outwardgripper", "name": "action: change_tool(left_hand, defaultgripper, outwardgripper)"}]}]}, {"summary": "selector to pick up the gear3 with the outwardgripper in the left_hand", "name": "selector: pick_up(left_hand, outwardgripper, gear3)", "children": [{"summary": "the target is to make the outwardgripper hold the gear3", "name": "target: hold(outwardgripper, gear3)"}, {"summary": "sequence to pick up the gear3 with the outwardgripper in the left_hand", "name": "sequence: pick_up(left_hand, outwardgripper, gear3)", "children": [{"summary": "a precondition is the outwardgripper is empty", "name": "precondition: is_empty(outwardgripper)"}, {"summary": "a precondition is the left_hand is holding outwardgripper", "name": "precondition: hold(left_hand, outwardgripper)"}, {"summary": "the action to pick up the gear3 with the outwardgripper in the left_hand", "name": "action: pick_up(left_hand, outwardgripper, gear3)"}]}]}, {"summary": "the action to insert the gear3 into the shaft3 with the outwardgripper in the left_hand", "name": "action: insert(left_hand, outwardgripper, gear3, shaft3)"}]}]}} -{"target": "target: is_inserted_to(shaft1, gearbase_hole1)", "initial_world_state": {"objects": [{"name": "clampgripper", "properties": ["is_empty"]}, {"name": "parallelgripper", "properties": ["is_empty"]}, {"name": "inwardgripper", "properties": ["is_empty"]}, {"name": "outwardgripper", "properties": ["is_empty"]}, {"name": "defaultgripper", "properties": []}, {"name": "gear1", "properties": []}, {"name": "gear2", "properties": []}, {"name": "gear3", "properties": []}, {"name": "shaft1", "properties": []}, {"name": "shaft2", "properties": []}, {"name": "shaft3", "properties": []}, {"name": "gearbase", "properties": []}, {"name": "gearbase_hole1", "properties": []}, {"name": "gearbase_hole3", "properties": []}, {"name": "left_hand", "properties": []}, {"name": "parallel_box2", "properties": []}, {"name": "outward_claw", "properties": []}, {"name": "no_tool", "properties": []}, {"name": "parallel_box1", "properties": []}], "constraints": [{"source": "outward_claw", "name": "can_manipulate", "target": "gear3"}, {"source": "shaft3", "name": "can_insert_to", "target": "gearbase_hole3"}, {"source": "defaultgripper", "name": "can_manipulate", "target": "shaft3"}, {"source": "gear2", "name": "can_insert_to", "target": "shaft2"}, {"source": "gear1", "name": "can_insert_to", "target": "shaft1"}, {"source": "outward_claw", "name": "can_manipulate", "target": "gear2"}, {"source": "outwardgripper", "name": "can_manipulate", "target": "gear2"}, {"source": "shaft1", "name": "can_insert_to", "target": "gearbase_hole1"}, {"source": "no_tool", "name": "can_manipulate", "target": "shaft3"}, {"source": "outwardgripper", "name": "can_manipulate", "target": "gear3"}, {"source": "parallel_box2", "name": "can_manipulate", "target": "gear1"}, {"source": "clampgripper", "name": "can_manipulate", "target": "shaft1"}, {"source": "parallel_box1", "name": "can_manipulate", "target": "shaft1"}, {"source": "parallelgripper", "name": "can_manipulate", "target": "gear1"}, {"source": "gear3", "name": "can_insert_to", "target": "shaft3"}], "relations": [{"source": "left_hand", "name": "hold", "target": "defaultgripper"}, {"source": "defaultgripper", "name": "hold", "target": "shaft3"}]}, "result": {"summary": "selector to insert the shaft1 into the gearbase_hole1 with the clampgripper in the left_hand", "name": "selector: insert(left_hand, clampgripper, shaft1, gearbase_hole1)", "children": [{"summary": "the target is to make the shaft1 be inserted into the gearbase_hole1", "name": "target: is_inserted_to(shaft1, gearbase_hole1)"}, {"summary": "sequence to insert the shaft1 into the gearbase_hole1 with the clampgripper in the left_hand", "name": "sequence: insert(left_hand, clampgripper, shaft1, gearbase_hole1)", "children": [{"summary": "selector to change tool in left_hand from defaultgripper to clampgripper", "name": "selector: change_tool(left_hand, defaultgripper, clampgripper)", "children": [{"summary": "the target is to make left_hand hold clampgripper", "name": "target: hold(left_hand, clampgripper)"}, {"summary": "sequence to change tool in left_hand from defaultgripper to clampgripper", "name": "sequence: change_tool(left_hand, defaultgripper, clampgripper)", "children": [{"summary": "a precondition is left_hand is holding defaultgripper", "name": "precondition: hold(left_hand, defaultgripper)"}, {"summary": "selector to put down the shaft3 held by the defaultgripper in the left_hand", "name": "selector: put_down(left_hand, defaultgripper, shaft3)", "children": [{"summary": "the target is to make the defaultgripper empty", "name": "target: is_empty(defaultgripper)"}, {"summary": "sequence to put down the shaft3 held by the defaultgripper in the left_hand", "name": "sequence: put_down(left_hand, defaultgripper, shaft3)", "children": [{"summary": "a precondition is the left_hand is holding defaultgripper", "name": "precondition: hold(left_hand, defaultgripper)"}, {"summary": "a precondition is the defaultgripper is holding shaft3", "name": "precondition: hold(defaultgripper, shaft3)"}, {"summary": "the action to put down the shaft3 held by the defaultgripper in the left_hand", "name": "action: put_down(left_hand, defaultgripper, shaft3)"}]}]}, {"summary": "the action to change tool in left_hand from defaultgripper to clampgripper", "name": "action: change_tool(left_hand, defaultgripper, clampgripper)"}]}]}, {"summary": "selector to pick up the shaft1 with the clampgripper in the left_hand", "name": "selector: pick_up(left_hand, clampgripper, shaft1)", "children": [{"summary": "the target is to make the clampgripper hold the shaft1", "name": "target: hold(clampgripper, shaft1)"}, {"summary": "sequence to pick up the shaft1 with the clampgripper in the left_hand", "name": "sequence: pick_up(left_hand, clampgripper, shaft1)", "children": [{"summary": "a precondition is the clampgripper is empty", "name": "precondition: is_empty(clampgripper)"}, {"summary": "a precondition is the left_hand is holding clampgripper", "name": "precondition: hold(left_hand, clampgripper)"}, {"summary": "the action to pick up the shaft1 with the clampgripper in the left_hand", "name": "action: pick_up(left_hand, clampgripper, shaft1)"}]}]}, {"summary": "the action to insert the shaft1 into the gearbase_hole1 with the clampgripper in the left_hand", "name": "action: insert(left_hand, clampgripper, shaft1, gearbase_hole1)"}]}]}} -{"target": "target: is_inserted_to(gear2, shaft2)", "initial_world_state": {"objects": [{"name": "clampgripper", "properties": ["is_empty"]}, {"name": "parallelgripper", "properties": ["is_empty"]}, {"name": "inwardgripper", "properties": ["is_empty"]}, {"name": "outwardgripper", "properties": ["is_empty"]}, {"name": "defaultgripper", "properties": ["is_empty"]}, {"name": "gear1", "properties": []}, {"name": "gear2", "properties": []}, {"name": "gear3", "properties": []}, {"name": "shaft1", "properties": []}, {"name": "shaft2", "properties": []}, {"name": "shaft3", "properties": []}, {"name": "gearbase", "properties": []}, {"name": "gearbase_hole1", "properties": []}, {"name": "gearbase_hole3", "properties": []}, {"name": "left_hand", "properties": []}, {"name": "parallel_box2", "properties": []}, {"name": "outward_claw", "properties": []}, {"name": "no_tool", "properties": []}, {"name": "parallel_box1", "properties": []}], "constraints": [{"source": "shaft1", "name": "can_insert_to", "target": "gearbase_hole1"}, {"source": "no_tool", "name": "can_manipulate", "target": "shaft3"}, {"source": "outward_claw", "name": "can_manipulate", "target": "gear2"}, {"source": "shaft3", "name": "can_insert_to", "target": "gearbase_hole3"}, {"source": "parallelgripper", "name": "can_manipulate", "target": "gear1"}, {"source": "parallel_box2", "name": "can_manipulate", "target": "gear1"}, {"source": "clampgripper", "name": "can_manipulate", "target": "shaft1"}, {"source": "defaultgripper", "name": "can_manipulate", "target": "shaft3"}, {"source": "outwardgripper", "name": "can_manipulate", "target": "gear2"}, {"source": "outward_claw", "name": "can_manipulate", "target": "gear3"}, {"source": "gear3", "name": "can_insert_to", "target": "shaft3"}, {"source": "outwardgripper", "name": "can_manipulate", "target": "gear3"}, {"source": "parallel_box1", "name": "can_manipulate", "target": "shaft1"}, {"source": "gear1", "name": "can_insert_to", "target": "shaft1"}, {"source": "gear2", "name": "can_insert_to", "target": "shaft2"}], "relations": [{"source": "left_hand", "name": "hold", "target": "defaultgripper"}]}, "result": {"summary": "selector to insert the gear2 into the shaft2 with the outwardgripper in the left_hand", "name": "selector: insert(left_hand, outwardgripper, gear2, shaft2)", "children": [{"summary": "the target is to make the gear2 be inserted into the shaft2", "name": "target: is_inserted_to(gear2, shaft2)"}, {"summary": "sequence to insert the gear2 into the shaft2 with the outwardgripper in the left_hand", "name": "sequence: insert(left_hand, outwardgripper, gear2, shaft2)", "children": [{"summary": "selector to change tool in left_hand from defaultgripper to outwardgripper", "name": "selector: change_tool(left_hand, defaultgripper, outwardgripper)", "children": [{"summary": "the target is to make left_hand hold outwardgripper", "name": "target: hold(left_hand, outwardgripper)"}, {"summary": "sequence to change tool in left_hand from defaultgripper to outwardgripper", "name": "sequence: change_tool(left_hand, defaultgripper, outwardgripper)", "children": [{"summary": "a precondition is left_hand is holding defaultgripper", "name": "precondition: hold(left_hand, defaultgripper)"}, {"summary": "a precondition is defaultgripper is empty", "name": "precondition: is_empty(defaultgripper)"}, {"summary": "the action to change tool in left_hand from defaultgripper to outwardgripper", "name": "action: change_tool(left_hand, defaultgripper, outwardgripper)"}]}]}, {"summary": "selector to pick up the gear2 with the outwardgripper in the left_hand", "name": "selector: pick_up(left_hand, outwardgripper, gear2)", "children": [{"summary": "the target is to make the outwardgripper hold the gear2", "name": "target: hold(outwardgripper, gear2)"}, {"summary": "sequence to pick up the gear2 with the outwardgripper in the left_hand", "name": "sequence: pick_up(left_hand, outwardgripper, gear2)", "children": [{"summary": "a precondition is the outwardgripper is empty", "name": "precondition: is_empty(outwardgripper)"}, {"summary": "a precondition is the left_hand is holding outwardgripper", "name": "precondition: hold(left_hand, outwardgripper)"}, {"summary": "the action to pick up the gear2 with the outwardgripper in the left_hand", "name": "action: pick_up(left_hand, outwardgripper, gear2)"}]}]}, {"summary": "the action to insert the gear2 into the shaft2 with the outwardgripper in the left_hand", "name": "action: insert(left_hand, outwardgripper, gear2, shaft2)"}]}]}} -{"target": "target: is_inserted_to(shaft3, gearbase_hole3)", "initial_world_state": {"objects": [{"name": "clampgripper", "properties": ["is_empty"]}, {"name": "parallelgripper", "properties": ["is_empty"]}, {"name": "inwardgripper", "properties": ["is_empty"]}, {"name": "outwardgripper", "properties": ["is_empty"]}, {"name": "defaultgripper", "properties": ["is_empty"]}, {"name": "gear1", "properties": []}, {"name": "gear2", "properties": []}, {"name": "gear3", "properties": []}, {"name": "shaft1", "properties": []}, {"name": "shaft2", "properties": []}, {"name": "shaft3", "properties": []}, {"name": "gearbase", "properties": []}, {"name": "gearbase_hole1", "properties": []}, {"name": "gearbase_hole3", "properties": []}, {"name": "left_hand", "properties": []}, {"name": "parallel_box2", "properties": []}, {"name": "outward_claw", "properties": []}, {"name": "no_tool", "properties": []}, {"name": "parallel_box1", "properties": []}], "constraints": [{"source": "outwardgripper", "name": "can_manipulate", "target": "gear2"}, {"source": "shaft1", "name": "can_insert_to", "target": "gearbase_hole1"}, {"source": "outward_claw", "name": "can_manipulate", "target": "gear3"}, {"source": "no_tool", "name": "can_manipulate", "target": "shaft3"}, {"source": "clampgripper", "name": "can_manipulate", "target": "shaft1"}, {"source": "defaultgripper", "name": "can_manipulate", "target": "shaft3"}, {"source": "outwardgripper", "name": "can_manipulate", "target": "gear3"}, {"source": "parallel_box2", "name": "can_manipulate", "target": "gear1"}, {"source": "parallel_box1", "name": "can_manipulate", "target": "shaft1"}, {"source": "gear2", "name": "can_insert_to", "target": "shaft2"}, {"source": "parallelgripper", "name": "can_manipulate", "target": "gear1"}, {"source": "gear1", "name": "can_insert_to", "target": "shaft1"}, {"source": "outward_claw", "name": "can_manipulate", "target": "gear2"}, {"source": "gear3", "name": "can_insert_to", "target": "shaft3"}, {"source": "shaft3", "name": "can_insert_to", "target": "gearbase_hole3"}], "relations": [{"source": "left_hand", "name": "hold", "target": "inwardgripper"}]}, "result": {"summary": "selector to insert the shaft3 into the gearbase_hole3 with the defaultgripper in the left_hand", "name": "selector: insert(left_hand, defaultgripper, shaft3, gearbase_hole3)", "children": [{"summary": "the target is to make the shaft3 be inserted into the gearbase_hole3", "name": "target: is_inserted_to(shaft3, gearbase_hole3)"}, {"summary": "sequence to insert the shaft3 into the gearbase_hole3 with the defaultgripper in the left_hand", "name": "sequence: insert(left_hand, defaultgripper, shaft3, gearbase_hole3)", "children": [{"summary": "selector to change tool in left_hand from inwardgripper to defaultgripper", "name": "selector: change_tool(left_hand, inwardgripper, defaultgripper)", "children": [{"summary": "the target is to make left_hand hold defaultgripper", "name": "target: hold(left_hand, defaultgripper)"}, {"summary": "sequence to change tool in left_hand from inwardgripper to defaultgripper", "name": "sequence: change_tool(left_hand, inwardgripper, defaultgripper)", "children": [{"summary": "a precondition is left_hand is holding inwardgripper", "name": "precondition: hold(left_hand, inwardgripper)"}, {"summary": "a precondition is inwardgripper is empty", "name": "precondition: is_empty(inwardgripper)"}, {"summary": "the action to change tool in left_hand from inwardgripper to defaultgripper", "name": "action: change_tool(left_hand, inwardgripper, defaultgripper)"}]}]}, {"summary": "selector to pick up the shaft3 with the defaultgripper in the left_hand", "name": "selector: pick_up(left_hand, defaultgripper, shaft3)", "children": [{"summary": "the target is to make the defaultgripper hold the shaft3", "name": "target: hold(defaultgripper, shaft3)"}, {"summary": "sequence to pick up the shaft3 with the defaultgripper in the left_hand", "name": "sequence: pick_up(left_hand, defaultgripper, shaft3)", "children": [{"summary": "a precondition is the defaultgripper is empty", "name": "precondition: is_empty(defaultgripper)"}, {"summary": "a precondition is the left_hand is holding defaultgripper", "name": "precondition: hold(left_hand, defaultgripper)"}, {"summary": "the action to pick up the shaft3 with the defaultgripper in the left_hand", "name": "action: pick_up(left_hand, defaultgripper, shaft3)"}]}]}, {"summary": "the action to insert the shaft3 into the gearbase_hole3 with the defaultgripper in the left_hand", "name": "action: insert(left_hand, defaultgripper, shaft3, gearbase_hole3)"}]}]}} -{"target": "target: is_inserted_to(shaft3, gearbase_hole3)", "initial_world_state": {"objects": [{"name": "clampgripper", "properties": ["is_empty"]}, {"name": "parallelgripper", "properties": []}, {"name": "inwardgripper", "properties": ["is_empty"]}, {"name": "outwardgripper", "properties": ["is_empty"]}, {"name": "defaultgripper", "properties": ["is_empty"]}, {"name": "gear1", "properties": []}, {"name": "gear2", "properties": []}, {"name": "gear3", "properties": []}, {"name": "shaft1", "properties": []}, {"name": "shaft2", "properties": []}, {"name": "shaft3", "properties": []}, {"name": "gearbase", "properties": []}, {"name": "gearbase_hole1", "properties": []}, {"name": "gearbase_hole3", "properties": []}, {"name": "left_hand", "properties": []}, {"name": "parallel_box2", "properties": []}, {"name": "outward_claw", "properties": []}, {"name": "no_tool", "properties": []}, {"name": "parallel_box1", "properties": []}], "constraints": [{"source": "no_tool", "name": "can_manipulate", "target": "shaft3"}, {"source": "shaft3", "name": "can_insert_to", "target": "gearbase_hole3"}, {"source": "gear3", "name": "can_insert_to", "target": "shaft3"}, {"source": "parallel_box2", "name": "can_manipulate", "target": "gear1"}, {"source": "parallel_box1", "name": "can_manipulate", "target": "shaft1"}, {"source": "outward_claw", "name": "can_manipulate", "target": "gear3"}, {"source": "defaultgripper", "name": "can_manipulate", "target": "shaft3"}, {"source": "shaft1", "name": "can_insert_to", "target": "gearbase_hole1"}, {"source": "outwardgripper", "name": "can_manipulate", "target": "gear2"}, {"source": "gear1", "name": "can_insert_to", "target": "shaft1"}, {"source": "clampgripper", "name": "can_manipulate", "target": "shaft1"}, {"source": "outwardgripper", "name": "can_manipulate", "target": "gear3"}, {"source": "gear2", "name": "can_insert_to", "target": "shaft2"}, {"source": "parallelgripper", "name": "can_manipulate", "target": "gear1"}, {"source": "outward_claw", "name": "can_manipulate", "target": "gear2"}], "relations": [{"source": "left_hand", "name": "hold", "target": "parallelgripper"}, {"source": "parallelgripper", "name": "hold", "target": "gear1"}]}, "result": {"summary": "selector to insert the shaft3 into the gearbase_hole3 with the defaultgripper in the left_hand", "name": "selector: insert(left_hand, defaultgripper, shaft3, gearbase_hole3)", "children": [{"summary": "the target is to make the shaft3 be inserted into the gearbase_hole3", "name": "target: is_inserted_to(shaft3, gearbase_hole3)"}, {"summary": "sequence to insert the shaft3 into the gearbase_hole3 with the defaultgripper in the left_hand", "name": "sequence: insert(left_hand, defaultgripper, shaft3, gearbase_hole3)", "children": [{"summary": "selector to change tool in left_hand from parallelgripper to defaultgripper", "name": "selector: change_tool(left_hand, parallelgripper, defaultgripper)", "children": [{"summary": "the target is to make left_hand hold defaultgripper", "name": "target: hold(left_hand, defaultgripper)"}, {"summary": "sequence to change tool in left_hand from parallelgripper to defaultgripper", "name": "sequence: change_tool(left_hand, parallelgripper, defaultgripper)", "children": [{"summary": "a precondition is left_hand is holding parallelgripper", "name": "precondition: hold(left_hand, parallelgripper)"}, {"summary": "selector to put down the gear1 held by the parallelgripper in the left_hand", "name": "selector: put_down(left_hand, parallelgripper, gear1)", "children": [{"summary": "the target is to make the parallelgripper empty", "name": "target: is_empty(parallelgripper)"}, {"summary": "sequence to put down the gear1 held by the parallelgripper in the left_hand", "name": "sequence: put_down(left_hand, parallelgripper, gear1)", "children": [{"summary": "a precondition is the left_hand is holding parallelgripper", "name": "precondition: hold(left_hand, parallelgripper)"}, {"summary": "a precondition is the parallelgripper is holding gear1", "name": "precondition: hold(parallelgripper, gear1)"}, {"summary": "the action to put down the gear1 held by the parallelgripper in the left_hand", "name": "action: put_down(left_hand, parallelgripper, gear1)"}]}]}, {"summary": "the action to change tool in left_hand from parallelgripper to defaultgripper", "name": "action: change_tool(left_hand, parallelgripper, defaultgripper)"}]}]}, {"summary": "selector to pick up the shaft3 with the defaultgripper in the left_hand", "name": "selector: pick_up(left_hand, defaultgripper, shaft3)", "children": [{"summary": "the target is to make the defaultgripper hold the shaft3", "name": "target: hold(defaultgripper, shaft3)"}, {"summary": "sequence to pick up the shaft3 with the defaultgripper in the left_hand", "name": "sequence: pick_up(left_hand, defaultgripper, shaft3)", "children": [{"summary": "a precondition is the defaultgripper is empty", "name": "precondition: is_empty(defaultgripper)"}, {"summary": "a precondition is the left_hand is holding defaultgripper", "name": "precondition: hold(left_hand, defaultgripper)"}, {"summary": "the action to pick up the shaft3 with the defaultgripper in the left_hand", "name": "action: pick_up(left_hand, defaultgripper, shaft3)"}]}]}, {"summary": "the action to insert the shaft3 into the gearbase_hole3 with the defaultgripper in the left_hand", "name": "action: insert(left_hand, defaultgripper, shaft3, gearbase_hole3)"}]}]}} -{"target": "target: is_inserted_to(gear2, shaft2)", "initial_world_state": {"objects": [{"name": "clampgripper", "properties": ["is_empty"]}, {"name": "parallelgripper", "properties": []}, {"name": "inwardgripper", "properties": ["is_empty"]}, {"name": "outwardgripper", "properties": ["is_empty"]}, {"name": "defaultgripper", "properties": ["is_empty"]}, {"name": "gear1", "properties": []}, {"name": "gear2", "properties": []}, {"name": "gear3", "properties": []}, {"name": "shaft1", "properties": []}, {"name": "shaft2", "properties": []}, {"name": "shaft3", "properties": []}, {"name": "gearbase", "properties": []}, {"name": "gearbase_hole1", "properties": []}, {"name": "gearbase_hole3", "properties": []}, {"name": "lampbase", "properties": []}, {"name": "chair_leg1", "properties": []}, {"name": "left_hand", "properties": []}, {"name": "parallel_box2", "properties": []}, {"name": "outward_claw", "properties": []}, {"name": "no_tool", "properties": []}, {"name": "parallel_box1", "properties": []}], "constraints": [{"source": "clampgripper", "name": "can_manipulate", "target": "shaft1"}, {"source": "outwardgripper", "name": "can_manipulate", "target": "gear3"}, {"source": "outward_claw", "name": "can_manipulate", "target": "gear3"}, {"source": "shaft3", "name": "can_insert_to", "target": "gearbase_hole3"}, {"source": "parallelgripper", "name": "can_manipulate", "target": "gear1"}, {"source": "gear3", "name": "can_insert_to", "target": "shaft3"}, {"source": "outwardgripper", "name": "can_manipulate", "target": "gear2"}, {"source": "shaft1", "name": "can_insert_to", "target": "gearbase_hole1"}, {"source": "no_tool", "name": "can_manipulate", "target": "shaft3"}, {"source": "parallel_box1", "name": "can_manipulate", "target": "shaft1"}, {"source": "outward_claw", "name": "can_manipulate", "target": "gear2"}, {"source": "defaultgripper", "name": "can_manipulate", "target": "shaft3"}, {"source": "gear1", "name": "can_insert_to", "target": "shaft1"}, {"source": "gear2", "name": "can_insert_to", "target": "shaft2"}, {"source": "parallel_box2", "name": "can_manipulate", "target": "gear1"}], "relations": [{"source": "gear2", "name": "is_inserted_to", "target": "shaft2"}, {"source": "parallelgripper", "name": "hold", "target": "gear1"}, {"source": "left_hand", "name": "hold", "target": "parallelgripper"}]}, "result": {"summary": "the target is to insert the gear2 into the shaft2", "name": "target: is_inserted_to(gear2, shaft2)"}} -{"target": "target: is_inserted_to(gear3, shaft3)", "initial_world_state": {"objects": [{"name": "clampgripper", "properties": ["is_empty"]}, {"name": "parallelgripper", "properties": []}, {"name": "inwardgripper", "properties": ["is_empty"]}, {"name": "outwardgripper", "properties": ["is_empty"]}, {"name": "defaultgripper", "properties": ["is_empty"]}, {"name": "gear1", "properties": []}, {"name": "gear2", "properties": []}, {"name": "gear3", "properties": []}, {"name": "shaft1", "properties": []}, {"name": "shaft2", "properties": []}, {"name": "shaft3", "properties": []}, {"name": "gearbase", "properties": []}, {"name": "gearbase_hole1", "properties": []}, {"name": "gearbase_hole3", "properties": []}, {"name": "lampbase", "properties": []}, {"name": "chair_leg1", "properties": []}, {"name": "left_hand", "properties": []}, {"name": "parallel_box2", "properties": []}, {"name": "outward_claw", "properties": []}, {"name": "no_tool", "properties": []}, {"name": "parallel_box1", "properties": []}], "constraints": [{"source": "outwardgripper", "name": "can_manipulate", "target": "gear2"}, {"source": "shaft1", "name": "can_insert_to", "target": "gearbase_hole1"}, {"source": "parallel_box1", "name": "can_manipulate", "target": "shaft1"}, {"source": "no_tool", "name": "can_manipulate", "target": "shaft3"}, {"source": "outwardgripper", "name": "can_manipulate", "target": "gear3"}, {"source": "parallelgripper", "name": "can_manipulate", "target": "gear1"}, {"source": "defaultgripper", "name": "can_manipulate", "target": "shaft3"}, {"source": "outward_claw", "name": "can_manipulate", "target": "gear2"}, {"source": "clampgripper", "name": "can_manipulate", "target": "shaft1"}, {"source": "gear3", "name": "can_insert_to", "target": "shaft3"}, {"source": "shaft3", "name": "can_insert_to", "target": "gearbase_hole3"}, {"source": "gear1", "name": "can_insert_to", "target": "shaft1"}, {"source": "gear2", "name": "can_insert_to", "target": "shaft2"}, {"source": "parallel_box2", "name": "can_manipulate", "target": "gear1"}, {"source": "outward_claw", "name": "can_manipulate", "target": "gear3"}], "relations": [{"source": "shaft1", "name": "is_inserted_to", "target": "gearbase_hole1"}, {"source": "left_hand", "name": "hold", "target": "parallelgripper"}, {"source": "gear3", "name": "is_inserted_to", "target": "shaft3"}, {"source": "shaft3", "name": "is_inserted_to", "target": "gearbase_hole3"}, {"source": "gear2", "name": "is_inserted_to", "target": "shaft2"}]}, "result": {"summary": "the target is to insert gear3 into shaft3", "name": "target: is_inserted_to(gear3, shaft3)"}} -{"target": "target: is_inserted_to(gear1, shaft1)", "initial_world_state": {"objects": [{"name": "clampgripper", "properties": ["is_empty"]}, {"name": "parallelgripper", "properties": ["is_empty"]}, {"name": "inwardgripper", "properties": ["is_empty"]}, {"name": "outwardgripper", "properties": ["is_empty"]}, {"name": "defaultgripper", "properties": ["is_empty"]}, {"name": "gear1", "properties": []}, {"name": "gear2", "properties": []}, {"name": "gear3", "properties": []}, {"name": "shaft1", "properties": []}, {"name": "shaft2", "properties": []}, {"name": "shaft3", "properties": []}, {"name": "gearbase", "properties": []}, {"name": "gearbase_hole1", "properties": []}, {"name": "gearbase_hole3", "properties": []}, {"name": "lampbase", "properties": []}, {"name": "chair_leg1", "properties": []}, {"name": "left_hand", "properties": []}, {"name": "parallel_box2", "properties": []}, {"name": "outward_claw", "properties": []}, {"name": "no_tool", "properties": []}, {"name": "parallel_box1", "properties": []}], "constraints": [{"source": "gear2", "name": "can_insert_to", "target": "shaft2"}, {"source": "outwardgripper", "name": "can_manipulate", "target": "gear3"}, {"source": "outwardgripper", "name": "can_manipulate", "target": "gear2"}, {"source": "gear3", "name": "can_insert_to", "target": "shaft3"}, {"source": "shaft3", "name": "can_insert_to", "target": "gearbase_hole3"}, {"source": "shaft1", "name": "can_insert_to", "target": "gearbase_hole1"}, {"source": "parallelgripper", "name": "can_manipulate", "target": "gear1"}, {"source": "no_tool", "name": "can_manipulate", "target": "shaft3"}, {"source": "clampgripper", "name": "can_manipulate", "target": "shaft1"}, {"source": "parallel_box1", "name": "can_manipulate", "target": "shaft1"}, {"source": "gear1", "name": "can_insert_to", "target": "shaft1"}, {"source": "parallel_box2", "name": "can_manipulate", "target": "gear1"}, {"source": "outward_claw", "name": "can_manipulate", "target": "gear2"}, {"source": "outward_claw", "name": "can_manipulate", "target": "gear3"}, {"source": "defaultgripper", "name": "can_manipulate", "target": "shaft3"}], "relations": [{"source": "shaft3", "name": "is_inserted_to", "target": "gearbase_hole3"}, {"source": "shaft1", "name": "is_inserted_to", "target": "gearbase_hole1"}, {"source": "left_hand", "name": "hold", "target": "inwardgripper"}, {"source": "gear2", "name": "is_inserted_to", "target": "shaft2"}]}, "result": {"summary": "selector to insert the gear1 into the shaft1 with the parallelgripper in the left_hand", "name": "selector: insert(left_hand, parallelgripper, gear1, shaft1)", "children": [{"summary": "the target is to make the gear1 be inserted into the shaft1", "name": "target: is_inserted_to(gear1, shaft1)"}, {"summary": "sequence to insert the gear1 into the shaft1 with the parallelgripper in the left_hand", "name": "sequence: insert(left_hand, parallelgripper, gear1, shaft1)", "children": [{"summary": "selector to change tool in left_hand from inwardgripper to parallelgripper", "name": "selector: change_tool(left_hand, inwardgripper, parallelgripper)", "children": [{"summary": "the target is to make left_hand hold parallelgripper", "name": "target: hold(left_hand, parallelgripper)"}, {"summary": "sequence to change tool in left_hand from inwardgripper to parallelgripper", "name": "sequence: change_tool(left_hand, inwardgripper, parallelgripper)", "children": [{"summary": "a precondition is left_hand is holding inwardgripper", "name": "precondition: hold(left_hand, inwardgripper)"}, {"summary": "a precondition is inwardgripper is empty", "name": "precondition: is_empty(inwardgripper)"}, {"summary": "the action to change tool in left_hand from inwardgripper to parallelgripper", "name": "action: change_tool(left_hand, inwardgripper, parallelgripper)"}]}]}, {"summary": "selector to pick up the gear1 with the parallelgripper in the left_hand", "name": "selector: pick_up(left_hand, parallelgripper, gear1)", "children": [{"summary": "the target is to make the parallelgripper hold the gear1", "name": "target: hold(parallelgripper, gear1)"}, {"summary": "sequence to pick up the gear1 with the parallelgripper in the left_hand", "name": "sequence: pick_up(left_hand, parallelgripper, gear1)", "children": [{"summary": "a precondition is the parallelgripper is empty", "name": "precondition: is_empty(parallelgripper)"}, {"summary": "a precondition is the left_hand is holding parallelgripper", "name": "precondition: hold(left_hand, parallelgripper)"}, {"summary": "the action to pick up the gear1 with the parallelgripper in the left_hand", "name": "action: pick_up(left_hand, parallelgripper, gear1)"}]}]}, {"summary": "the action to insert the gear1 into the shaft1 with the parallelgripper in the left_hand", "name": "action: insert(left_hand, parallelgripper, gear1, shaft1)"}]}]}} diff --git a/experiments/gearset1/problem_set/problem_000.json b/experiments/gearset1/problem_set/problem_000.json deleted file mode 100644 index 78f3b68e..00000000 --- a/experiments/gearset1/problem_set/problem_000.json +++ /dev/null @@ -1,239 +0,0 @@ -{ - "target": "target: is_inserted_to(shaft1, gearbase_hole1)", - "initial_world_state": { - "objects": [ - { - "name": "clampgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "parallelgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "inwardgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "outwardgripper", - "properties": [] - }, - { - "name": "defaultgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "gear1", - "properties": [] - }, - { - "name": "gear2", - "properties": [] - }, - { - "name": "gear3", - "properties": [] - }, - { - "name": "shaft1", - "properties": [] - }, - { - "name": "shaft2", - "properties": [] - }, - { - "name": "shaft3", - "properties": [] - }, - { - "name": "gearbase", - "properties": [] - }, - { - "name": "gearbase_hole1", - "properties": [] - }, - { - "name": "gearbase_hole3", - "properties": [] - }, - { - "name": "left_hand", - "properties": [] - } - ], - "constraints": [ - { - "source": "gear3", - "name": "can_insert_to", - "target": "shaft3" - }, - { - "source": "shaft1", - "name": "can_insert_to", - "target": "gearbase_hole1" - }, - { - "source": "parallelgripper", - "name": "can_manipulate", - "target": "gear1" - }, - { - "source": "gear2", - "name": "can_insert_to", - "target": "shaft2" - }, - { - "source": "outwardgripper", - "name": "can_manipulate", - "target": "gear3" - }, - { - "source": "gear1", - "name": "can_insert_to", - "target": "shaft1" - }, - { - "source": "defaultgripper", - "name": "can_manipulate", - "target": "shaft3" - }, - { - "source": "outwardgripper", - "name": "can_manipulate", - "target": "gear2" - }, - { - "source": "shaft3", - "name": "can_insert_to", - "target": "gearbase_hole3" - }, - { - "source": "clampgripper", - "name": "can_manipulate", - "target": "shaft1" - } - ], - "relations": [ - { - "source": "outwardgripper", - "name": "hold", - "target": "gear2" - }, - { - "source": "left_hand", - "name": "hold", - "target": "outwardgripper" - } - ] - }, - "result": { - "summary": "selector to insert the shaft1 into the gearbase_hole1 with the clampgripper in the left_hand", - "name": "selector: insert(left_hand, clampgripper, shaft1, gearbase_hole1)", - "children": [ - { - "summary": "the target is to make the shaft1 be inserted into the gearbase_hole1", - "name": "target: is_inserted_to(shaft1, gearbase_hole1)" - }, - { - "summary": "sequence to insert the shaft1 into the gearbase_hole1 with the clampgripper in the left_hand", - "name": "sequence: insert(left_hand, clampgripper, shaft1, gearbase_hole1)", - "children": [ - { - "summary": "selector to change tool in left_hand from outwardgripper to clampgripper", - "name": "selector: change_tool(left_hand, outwardgripper, clampgripper)", - "children": [ - { - "summary": "the target is to make left_hand hold clampgripper", - "name": "target: hold(left_hand, clampgripper)" - }, - { - "summary": "sequence to change tool in left_hand from outwardgripper to clampgripper", - "name": "sequence: change_tool(left_hand, outwardgripper, clampgripper)", - "children": [ - { - "summary": "a precondition is left_hand is holding outwardgripper", - "name": "precondition: hold(left_hand, outwardgripper)" - }, - { - "summary": "selector to put down the gear2 held by the outwardgripper in the left_hand", - "name": "selector: put_down(left_hand, outwardgripper, gear2)", - "children": [ - { - "summary": "the target is to make the outwardgripper empty", - "name": "target: is_empty(outwardgripper)" - }, - { - "summary": "sequence to put down the gear2 held by the outwardgripper in the left_hand", - "name": "sequence: put_down(left_hand, outwardgripper, gear2)", - "children": [ - { - "summary": "a precondition is the left_hand is holding outwardgripper", - "name": "precondition: hold(left_hand, outwardgripper)" - }, - { - "summary": "a precondition is the outwardgripper is holding gear2", - "name": "precondition: hold(outwardgripper, gear2)" - }, - { - "summary": "the action to put down the gear2 held by the outwardgripper in the left_hand", - "name": "action: put_down(left_hand, outwardgripper, gear2)" - } - ] - } - ] - }, - { - "summary": "the action to change tool in left_hand from outwardgripper to clampgripper", - "name": "action: change_tool(left_hand, outwardgripper, clampgripper)" - } - ] - } - ] - }, - { - "summary": "selector to pick up the shaft1 with the clampgripper in the left_hand", - "name": "selector: pick_up(left_hand, clampgripper, shaft1)", - "children": [ - { - "summary": "the target is to make the clampgripper hold the shaft1", - "name": "target: hold(clampgripper, shaft1)" - }, - { - "summary": "sequence to pick up the shaft1 with the clampgripper in the left_hand", - "name": "sequence: pick_up(left_hand, clampgripper, shaft1)", - "children": [ - { - "summary": "a precondition is the clampgripper is empty", - "name": "precondition: is_empty(clampgripper)" - }, - { - "summary": "a precondition is the left_hand is holding clampgripper", - "name": "precondition: hold(left_hand, clampgripper)" - }, - { - "summary": "the action to pick up the shaft1 with the clampgripper in the left_hand", - "name": "action: pick_up(left_hand, clampgripper, shaft1)" - } - ] - } - ] - }, - { - "summary": "the action to insert the shaft1 into the gearbase_hole1 with the clampgripper in the left_hand", - "name": "action: insert(left_hand, clampgripper, shaft1, gearbase_hole1)" - } - ] - } - ] - } -} \ No newline at end of file diff --git a/experiments/gearset1/problem_set/problem_001.json b/experiments/gearset1/problem_set/problem_001.json deleted file mode 100644 index 84bd30de..00000000 --- a/experiments/gearset1/problem_set/problem_001.json +++ /dev/null @@ -1,212 +0,0 @@ -{ - "target": "target: is_inserted_to(shaft1, gearbase_hole1)", - "initial_world_state": { - "objects": [ - { - "name": "clampgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "parallelgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "inwardgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "outwardgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "defaultgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "gear1", - "properties": [] - }, - { - "name": "gear2", - "properties": [] - }, - { - "name": "gear3", - "properties": [] - }, - { - "name": "shaft1", - "properties": [] - }, - { - "name": "shaft2", - "properties": [] - }, - { - "name": "shaft3", - "properties": [] - }, - { - "name": "gearbase", - "properties": [] - }, - { - "name": "gearbase_hole1", - "properties": [] - }, - { - "name": "gearbase_hole3", - "properties": [] - }, - { - "name": "left_hand", - "properties": [] - } - ], - "constraints": [ - { - "source": "defaultgripper", - "name": "can_manipulate", - "target": "shaft3" - }, - { - "source": "gear2", - "name": "can_insert_to", - "target": "shaft2" - }, - { - "source": "shaft1", - "name": "can_insert_to", - "target": "gearbase_hole1" - }, - { - "source": "clampgripper", - "name": "can_manipulate", - "target": "shaft1" - }, - { - "source": "parallelgripper", - "name": "can_manipulate", - "target": "gear1" - }, - { - "source": "outwardgripper", - "name": "can_manipulate", - "target": "gear3" - }, - { - "source": "gear1", - "name": "can_insert_to", - "target": "shaft1" - }, - { - "source": "shaft3", - "name": "can_insert_to", - "target": "gearbase_hole3" - }, - { - "source": "outwardgripper", - "name": "can_manipulate", - "target": "gear2" - }, - { - "source": "gear3", - "name": "can_insert_to", - "target": "shaft3" - } - ], - "relations": [ - { - "source": "left_hand", - "name": "hold", - "target": "outwardgripper" - } - ] - }, - "result": { - "summary": "selector to insert the shaft1 into the gearbase_hole1 with the clampgripper in the left_hand", - "name": "selector: insert(left_hand, clampgripper, shaft1, gearbase_hole1)", - "children": [ - { - "summary": "the target is to make the shaft1 be inserted into the gearbase_hole1", - "name": "target: is_inserted_to(shaft1, gearbase_hole1)" - }, - { - "summary": "sequence to insert the shaft1 into the gearbase_hole1 with the clampgripper in the left_hand", - "name": "sequence: insert(left_hand, clampgripper, shaft1, gearbase_hole1)", - "children": [ - { - "summary": "selector to change tool in left_hand from outwardgripper to clampgripper", - "name": "selector: change_tool(left_hand, outwardgripper, clampgripper)", - "children": [ - { - "summary": "the target is to make left_hand hold clampgripper", - "name": "target: hold(left_hand, clampgripper)" - }, - { - "summary": "sequence to change tool in left_hand from outwardgripper to clampgripper", - "name": "sequence: change_tool(left_hand, outwardgripper, clampgripper)", - "children": [ - { - "summary": "a precondition is left_hand is holding outwardgripper", - "name": "precondition: hold(left_hand, outwardgripper)" - }, - { - "summary": "a precondition is outwardgripper is empty", - "name": "precondition: is_empty(outwardgripper)" - }, - { - "summary": "the action to change tool in left_hand from outwardgripper to clampgripper", - "name": "action: change_tool(left_hand, outwardgripper, clampgripper)" - } - ] - } - ] - }, - { - "summary": "selector to pick up the shaft1 with the clampgripper in the left_hand", - "name": "selector: pick_up(left_hand, clampgripper, shaft1)", - "children": [ - { - "summary": "the target is to make the clampgripper hold the shaft1", - "name": "target: hold(clampgripper, shaft1)" - }, - { - "summary": "sequence to pick up the shaft1 with the clampgripper in the left_hand", - "name": "sequence: pick_up(left_hand, clampgripper, shaft1)", - "children": [ - { - "summary": "a precondition is the clampgripper is empty", - "name": "precondition: is_empty(clampgripper)" - }, - { - "summary": "a precondition is the left_hand is holding clampgripper", - "name": "precondition: hold(left_hand, clampgripper)" - }, - { - "summary": "the action to pick up the shaft1 with the clampgripper in the left_hand", - "name": "action: pick_up(left_hand, clampgripper, shaft1)" - } - ] - } - ] - }, - { - "summary": "the action to insert the shaft1 into the gearbase_hole1 with the clampgripper in the left_hand", - "name": "action: insert(left_hand, clampgripper, shaft1, gearbase_hole1)" - } - ] - } - ] - } -} \ No newline at end of file diff --git a/experiments/gearset1/problem_set/problem_002.json b/experiments/gearset1/problem_set/problem_002.json deleted file mode 100644 index b3830dbb..00000000 --- a/experiments/gearset1/problem_set/problem_002.json +++ /dev/null @@ -1,188 +0,0 @@ -{ - "target": "target: is_inserted_to(gear2, shaft2)", - "initial_world_state": { - "objects": [ - { - "name": "clampgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "parallelgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "inwardgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "outwardgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "defaultgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "gear1", - "properties": [] - }, - { - "name": "gear2", - "properties": [] - }, - { - "name": "gear3", - "properties": [] - }, - { - "name": "shaft1", - "properties": [] - }, - { - "name": "shaft2", - "properties": [] - }, - { - "name": "shaft3", - "properties": [] - }, - { - "name": "gearbase", - "properties": [] - }, - { - "name": "gearbase_hole1", - "properties": [] - }, - { - "name": "gearbase_hole3", - "properties": [] - }, - { - "name": "left_hand", - "properties": [] - } - ], - "constraints": [ - { - "source": "shaft1", - "name": "can_insert_to", - "target": "gearbase_hole1" - }, - { - "source": "clampgripper", - "name": "can_manipulate", - "target": "shaft1" - }, - { - "source": "outwardgripper", - "name": "can_manipulate", - "target": "gear3" - }, - { - "source": "gear3", - "name": "can_insert_to", - "target": "shaft3" - }, - { - "source": "defaultgripper", - "name": "can_manipulate", - "target": "shaft3" - }, - { - "source": "gear2", - "name": "can_insert_to", - "target": "shaft2" - }, - { - "source": "shaft3", - "name": "can_insert_to", - "target": "gearbase_hole3" - }, - { - "source": "gear1", - "name": "can_insert_to", - "target": "shaft1" - }, - { - "source": "outwardgripper", - "name": "can_manipulate", - "target": "gear2" - }, - { - "source": "parallelgripper", - "name": "can_manipulate", - "target": "gear1" - } - ], - "relations": [ - { - "source": "left_hand", - "name": "hold", - "target": "outwardgripper" - } - ] - }, - "result": { - "summary": "selector to insert the gear2 into the shaft2 with the outwardgripper in the left_hand", - "name": "selector: insert(left_hand, outwardgripper, gear2, shaft2)", - "children": [ - { - "summary": "the target is to make the gear2 be inserted into the shaft2", - "name": "target: is_inserted_to(gear2, shaft2)" - }, - { - "summary": "sequence to insert the gear2 into the shaft2 with the outwardgripper in the left_hand", - "name": "sequence: insert(left_hand, outwardgripper, gear2, shaft2)", - "children": [ - { - "summary": "a precondition is the left_hand is holding outwardgripper", - "name": "precondition: hold(left_hand, outwardgripper)" - }, - { - "summary": "selector to pick up the gear2 with the outwardgripper in the left_hand", - "name": "selector: pick_up(left_hand, outwardgripper, gear2)", - "children": [ - { - "summary": "the target is to make the outwardgripper hold the gear2", - "name": "target: hold(outwardgripper, gear2)" - }, - { - "summary": "sequence to pick up the gear2 with the outwardgripper in the left_hand", - "name": "sequence: pick_up(left_hand, outwardgripper, gear2)", - "children": [ - { - "summary": "a precondition is the outwardgripper is empty", - "name": "precondition: is_empty(outwardgripper)" - }, - { - "summary": "a precondition is the left_hand is holding outwardgripper", - "name": "precondition: hold(left_hand, outwardgripper)" - }, - { - "summary": "the action to pick up the gear2 with the outwardgripper in the left_hand", - "name": "action: pick_up(left_hand, outwardgripper, gear2)" - } - ] - } - ] - }, - { - "summary": "the action to insert the gear2 into the shaft2 with the outwardgripper in the left_hand", - "name": "action: insert(left_hand, outwardgripper, gear2, shaft2)" - } - ] - } - ] - } -} \ No newline at end of file diff --git a/experiments/gearset1/problem_set/problem_003.json b/experiments/gearset1/problem_set/problem_003.json deleted file mode 100644 index a7d7b6ca..00000000 --- a/experiments/gearset1/problem_set/problem_003.json +++ /dev/null @@ -1,239 +0,0 @@ -{ - "target": "target: is_inserted_to(gear2, shaft2)", - "initial_world_state": { - "objects": [ - { - "name": "clampgripper", - "properties": [] - }, - { - "name": "parallelgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "inwardgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "outwardgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "defaultgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "gear1", - "properties": [] - }, - { - "name": "gear2", - "properties": [] - }, - { - "name": "gear3", - "properties": [] - }, - { - "name": "shaft1", - "properties": [] - }, - { - "name": "shaft2", - "properties": [] - }, - { - "name": "shaft3", - "properties": [] - }, - { - "name": "gearbase", - "properties": [] - }, - { - "name": "gearbase_hole1", - "properties": [] - }, - { - "name": "gearbase_hole3", - "properties": [] - }, - { - "name": "left_hand", - "properties": [] - } - ], - "constraints": [ - { - "source": "gear1", - "name": "can_insert_to", - "target": "shaft1" - }, - { - "source": "defaultgripper", - "name": "can_manipulate", - "target": "shaft3" - }, - { - "source": "outwardgripper", - "name": "can_manipulate", - "target": "gear2" - }, - { - "source": "shaft1", - "name": "can_insert_to", - "target": "gearbase_hole1" - }, - { - "source": "gear2", - "name": "can_insert_to", - "target": "shaft2" - }, - { - "source": "gear3", - "name": "can_insert_to", - "target": "shaft3" - }, - { - "source": "shaft3", - "name": "can_insert_to", - "target": "gearbase_hole3" - }, - { - "source": "outwardgripper", - "name": "can_manipulate", - "target": "gear3" - }, - { - "source": "parallelgripper", - "name": "can_manipulate", - "target": "gear1" - }, - { - "source": "clampgripper", - "name": "can_manipulate", - "target": "shaft1" - } - ], - "relations": [ - { - "source": "clampgripper", - "name": "hold", - "target": "shaft1" - }, - { - "source": "left_hand", - "name": "hold", - "target": "clampgripper" - } - ] - }, - "result": { - "summary": "selector to insert the gear2 into the shaft2 with the outwardgripper in the left_hand", - "name": "selector: insert(left_hand, outwardgripper, gear2, shaft2)", - "children": [ - { - "summary": "the target is to make the gear2 be inserted into the shaft2", - "name": "target: is_inserted_to(gear2, shaft2)" - }, - { - "summary": "sequence to insert the gear2 into the shaft2 with the outwardgripper in the left_hand", - "name": "sequence: insert(left_hand, outwardgripper, gear2, shaft2)", - "children": [ - { - "summary": "selector to change tool in left_hand from clampgripper to outwardgripper", - "name": "selector: change_tool(left_hand, clampgripper, outwardgripper)", - "children": [ - { - "summary": "the target is to make left_hand hold outwardgripper", - "name": "target: hold(left_hand, outwardgripper)" - }, - { - "summary": "sequence to change tool in left_hand from clampgripper to outwardgripper", - "name": "sequence: change_tool(left_hand, clampgripper, outwardgripper)", - "children": [ - { - "summary": "a precondition is left_hand is holding clampgripper", - "name": "precondition: hold(left_hand, clampgripper)" - }, - { - "summary": "selector to put down the shaft1 held by the clampgripper in the left_hand", - "name": "selector: put_down(left_hand, clampgripper, shaft1)", - "children": [ - { - "summary": "the target is to make the clampgripper empty", - "name": "target: is_empty(clampgripper)" - }, - { - "summary": "sequence to put down the shaft1 held by the clampgripper in the left_hand", - "name": "sequence: put_down(left_hand, clampgripper, shaft1)", - "children": [ - { - "summary": "a precondition is the left_hand is holding clampgripper", - "name": "precondition: hold(left_hand, clampgripper)" - }, - { - "summary": "a precondition is the clampgripper is holding shaft1", - "name": "precondition: hold(clampgripper, shaft1)" - }, - { - "summary": "the action to put down the shaft1 held by the clampgripper in the left_hand", - "name": "action: put_down(left_hand, clampgripper, shaft1)" - } - ] - } - ] - }, - { - "summary": "the action to change tool in left_hand from clampgripper to outwardgripper", - "name": "action: change_tool(left_hand, clampgripper, outwardgripper)" - } - ] - } - ] - }, - { - "summary": "selector to pick up the gear2 with the outwardgripper in the left_hand", - "name": "selector: pick_up(left_hand, outwardgripper, gear2)", - "children": [ - { - "summary": "the target is to make the outwardgripper hold the gear2", - "name": "target: hold(outwardgripper, gear2)" - }, - { - "summary": "sequence to pick up the gear2 with the outwardgripper in the left_hand", - "name": "sequence: pick_up(left_hand, outwardgripper, gear2)", - "children": [ - { - "summary": "a precondition is the outwardgripper is empty", - "name": "precondition: is_empty(outwardgripper)" - }, - { - "summary": "a precondition is the left_hand is holding outwardgripper", - "name": "precondition: hold(left_hand, outwardgripper)" - }, - { - "summary": "the action to pick up the gear2 with the outwardgripper in the left_hand", - "name": "action: pick_up(left_hand, outwardgripper, gear2)" - } - ] - } - ] - }, - { - "summary": "the action to insert the gear2 into the shaft2 with the outwardgripper in the left_hand", - "name": "action: insert(left_hand, outwardgripper, gear2, shaft2)" - } - ] - } - ] - } -} \ No newline at end of file diff --git a/experiments/gearset1/problem_set/problem_004.json b/experiments/gearset1/problem_set/problem_004.json deleted file mode 100644 index ae3a97c0..00000000 --- a/experiments/gearset1/problem_set/problem_004.json +++ /dev/null @@ -1,239 +0,0 @@ -{ - "target": "target: is_inserted_to(gear3, shaft3)", - "initial_world_state": { - "objects": [ - { - "name": "clampgripper", - "properties": [] - }, - { - "name": "parallelgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "inwardgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "outwardgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "defaultgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "gear1", - "properties": [] - }, - { - "name": "gear2", - "properties": [] - }, - { - "name": "gear3", - "properties": [] - }, - { - "name": "shaft1", - "properties": [] - }, - { - "name": "shaft2", - "properties": [] - }, - { - "name": "shaft3", - "properties": [] - }, - { - "name": "gearbase", - "properties": [] - }, - { - "name": "gearbase_hole1", - "properties": [] - }, - { - "name": "gearbase_hole3", - "properties": [] - }, - { - "name": "left_hand", - "properties": [] - } - ], - "constraints": [ - { - "source": "clampgripper", - "name": "can_manipulate", - "target": "shaft1" - }, - { - "source": "gear3", - "name": "can_insert_to", - "target": "shaft3" - }, - { - "source": "outwardgripper", - "name": "can_manipulate", - "target": "gear3" - }, - { - "source": "shaft3", - "name": "can_insert_to", - "target": "gearbase_hole3" - }, - { - "source": "gear1", - "name": "can_insert_to", - "target": "shaft1" - }, - { - "source": "parallelgripper", - "name": "can_manipulate", - "target": "gear1" - }, - { - "source": "defaultgripper", - "name": "can_manipulate", - "target": "shaft3" - }, - { - "source": "gear2", - "name": "can_insert_to", - "target": "shaft2" - }, - { - "source": "shaft1", - "name": "can_insert_to", - "target": "gearbase_hole1" - }, - { - "source": "outwardgripper", - "name": "can_manipulate", - "target": "gear2" - } - ], - "relations": [ - { - "source": "left_hand", - "name": "hold", - "target": "clampgripper" - }, - { - "source": "clampgripper", - "name": "hold", - "target": "shaft1" - } - ] - }, - "result": { - "summary": "selector to insert the gear3 into the shaft3 with the outwardgripper in the left_hand", - "name": "selector: insert(left_hand, outwardgripper, gear3, shaft3)", - "children": [ - { - "summary": "the target is to make the gear3 be inserted into the shaft3", - "name": "target: is_inserted_to(gear3, shaft3)" - }, - { - "summary": "sequence to insert the gear3 into the shaft3 with the outwardgripper in the left_hand", - "name": "sequence: insert(left_hand, outwardgripper, gear3, shaft3)", - "children": [ - { - "summary": "selector to change tool in left_hand from clampgripper to outwardgripper", - "name": "selector: change_tool(left_hand, clampgripper, outwardgripper)", - "children": [ - { - "summary": "the target is to make left_hand hold outwardgripper", - "name": "target: hold(left_hand, outwardgripper)" - }, - { - "summary": "sequence to change tool in left_hand from clampgripper to outwardgripper", - "name": "sequence: change_tool(left_hand, clampgripper, outwardgripper)", - "children": [ - { - "summary": "a precondition is left_hand is holding clampgripper", - "name": "precondition: hold(left_hand, clampgripper)" - }, - { - "summary": "selector to put down the shaft1 held by the clampgripper in the left_hand", - "name": "selector: put_down(left_hand, clampgripper, shaft1)", - "children": [ - { - "summary": "the target is to make the clampgripper empty", - "name": "target: is_empty(clampgripper)" - }, - { - "summary": "sequence to put down the shaft1 held by the clampgripper in the left_hand", - "name": "sequence: put_down(left_hand, clampgripper, shaft1)", - "children": [ - { - "summary": "a precondition is the left_hand is holding clampgripper", - "name": "precondition: hold(left_hand, clampgripper)" - }, - { - "summary": "a precondition is the clampgripper is holding shaft1", - "name": "precondition: hold(clampgripper, shaft1)" - }, - { - "summary": "the action to put down the shaft1 held by the clampgripper in the left_hand", - "name": "action: put_down(left_hand, clampgripper, shaft1)" - } - ] - } - ] - }, - { - "summary": "the action to change tool in left_hand from clampgripper to outwardgripper", - "name": "action: change_tool(left_hand, clampgripper, outwardgripper)" - } - ] - } - ] - }, - { - "summary": "selector to pick up the gear3 with the outwardgripper in the left_hand", - "name": "selector: pick_up(left_hand, outwardgripper, gear3)", - "children": [ - { - "summary": "the target is to make the outwardgripper hold the gear3", - "name": "target: hold(outwardgripper, gear3)" - }, - { - "summary": "sequence to pick up the gear3 with the outwardgripper in the left_hand", - "name": "sequence: pick_up(left_hand, outwardgripper, gear3)", - "children": [ - { - "summary": "a precondition is the outwardgripper is empty", - "name": "precondition: is_empty(outwardgripper)" - }, - { - "summary": "a precondition is the left_hand is holding outwardgripper", - "name": "precondition: hold(left_hand, outwardgripper)" - }, - { - "summary": "the action to pick up the gear3 with the outwardgripper in the left_hand", - "name": "action: pick_up(left_hand, outwardgripper, gear3)" - } - ] - } - ] - }, - { - "summary": "the action to insert the gear3 into the shaft3 with the outwardgripper in the left_hand", - "name": "action: insert(left_hand, outwardgripper, gear3, shaft3)" - } - ] - } - ] - } -} \ No newline at end of file diff --git a/experiments/gearset1/problem_set/problem_005.json b/experiments/gearset1/problem_set/problem_005.json deleted file mode 100644 index f06fbaec..00000000 --- a/experiments/gearset1/problem_set/problem_005.json +++ /dev/null @@ -1,236 +0,0 @@ -{ - "target": "target: is_inserted_to(gear3, shaft3)", - "initial_world_state": { - "objects": [ - { - "name": "clampgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "parallelgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "inwardgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "outwardgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "defaultgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "gear1", - "properties": [] - }, - { - "name": "gear2", - "properties": [] - }, - { - "name": "gear3", - "properties": [] - }, - { - "name": "shaft1", - "properties": [] - }, - { - "name": "shaft2", - "properties": [] - }, - { - "name": "shaft3", - "properties": [] - }, - { - "name": "gearbase", - "properties": [] - }, - { - "name": "gearbase_hole1", - "properties": [] - }, - { - "name": "gearbase_hole3", - "properties": [] - }, - { - "name": "left_hand", - "properties": [] - } - ], - "constraints": [ - { - "source": "gear2", - "name": "can_insert_to", - "target": "shaft2" - }, - { - "source": "outwardgripper", - "name": "can_manipulate", - "target": "gear2" - }, - { - "source": "shaft3", - "name": "can_insert_to", - "target": "gearbase_hole3" - }, - { - "source": "clampgripper", - "name": "can_manipulate", - "target": "shaft1" - }, - { - "source": "shaft1", - "name": "can_insert_to", - "target": "gearbase_hole1" - }, - { - "source": "outwardgripper", - "name": "can_manipulate", - "target": "gear3" - }, - { - "source": "gear1", - "name": "can_insert_to", - "target": "shaft1" - }, - { - "source": "defaultgripper", - "name": "can_manipulate", - "target": "shaft3" - }, - { - "source": "gear3", - "name": "can_insert_to", - "target": "shaft3" - }, - { - "source": "parallelgripper", - "name": "can_manipulate", - "target": "gear1" - } - ], - "relations": [ - { - "source": "left_hand", - "name": "hold", - "target": "defaultgripper" - } - ] - }, - "result": { - "summary": "selector to insert the gear3 into the shaft3 with the outwardgripper in the left_hand", - "name": "selector: insert(left_hand, outwardgripper, gear3, shaft3)", - "children": [ - { - "summary": "the target is to make the gear3 be inserted into the shaft3", - "name": "target: is_inserted_to(gear3, shaft3)" - }, - { - "summary": "sequence to insert the gear3 into the shaft3 with the outwardgripper in the left_hand", - "name": "sequence: insert(left_hand, outwardgripper, gear3, shaft3)", - "children": [ - { - "summary": "selector to change tool in left_hand from clampgripper to outwardgripper", - "name": "selector: change_tool(left_hand, clampgripper, outwardgripper)", - "children": [ - { - "summary": "the target is to make left_hand hold outwardgripper", - "name": "target: hold(left_hand, outwardgripper)" - }, - { - "summary": "sequence to change tool in left_hand from clampgripper to outwardgripper", - "name": "sequence: change_tool(left_hand, clampgripper, outwardgripper)", - "children": [ - { - "summary": "selector to change tool in left_hand from defaultgripper to clampgripper", - "name": "selector: change_tool(left_hand, defaultgripper, clampgripper)", - "children": [ - { - "summary": "the target is to make left_hand hold clampgripper", - "name": "target: hold(left_hand, clampgripper)" - }, - { - "summary": "sequence to change tool in left_hand from defaultgripper to clampgripper", - "name": "sequence: change_tool(left_hand, defaultgripper, clampgripper)", - "children": [ - { - "summary": "a precondition is left_hand is holding defaultgripper", - "name": "precondition: hold(left_hand, defaultgripper)" - }, - { - "summary": "a precondition is defaultgripper is empty", - "name": "precondition: is_empty(defaultgripper)" - }, - { - "summary": "the action to change tool in left_hand from defaultgripper to clampgripper", - "name": "action: change_tool(left_hand, defaultgripper, clampgripper)" - } - ] - } - ] - }, - { - "summary": "a precondition is clampgripper is empty", - "name": "precondition: is_empty(clampgripper)" - }, - { - "summary": "the action to change tool in left_hand from clampgripper to outwardgripper", - "name": "action: change_tool(left_hand, clampgripper, outwardgripper)" - } - ] - } - ] - }, - { - "summary": "selector to pick up the gear3 with the outwardgripper in the left_hand", - "name": "selector: pick_up(left_hand, outwardgripper, gear3)", - "children": [ - { - "summary": "the target is to make the outwardgripper hold the gear3", - "name": "target: hold(outwardgripper, gear3)" - }, - { - "summary": "sequence to pick up the gear3 with the outwardgripper in the left_hand", - "name": "sequence: pick_up(left_hand, outwardgripper, gear3)", - "children": [ - { - "summary": "a precondition is the outwardgripper is empty", - "name": "precondition: is_empty(outwardgripper)" - }, - { - "summary": "a precondition is the left_hand is holding outwardgripper", - "name": "precondition: hold(left_hand, outwardgripper)" - }, - { - "summary": "the action to pick up the gear3 with the outwardgripper in the left_hand", - "name": "action: pick_up(left_hand, outwardgripper, gear3)" - } - ] - } - ] - }, - { - "summary": "the action to insert the gear3 into the shaft3 with the outwardgripper in the left_hand", - "name": "action: insert(left_hand, outwardgripper, gear3, shaft3)" - } - ] - } - ] - } -} \ No newline at end of file diff --git a/experiments/gearset1/problem_set/problem_006.json b/experiments/gearset1/problem_set/problem_006.json deleted file mode 100644 index 2dccf6db..00000000 --- a/experiments/gearset1/problem_set/problem_006.json +++ /dev/null @@ -1,188 +0,0 @@ -{ - "target": "target: is_inserted_to(gear3, shaft3)", - "initial_world_state": { - "objects": [ - { - "name": "clampgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "parallelgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "inwardgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "outwardgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "defaultgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "gear1", - "properties": [] - }, - { - "name": "gear2", - "properties": [] - }, - { - "name": "gear3", - "properties": [] - }, - { - "name": "shaft1", - "properties": [] - }, - { - "name": "shaft2", - "properties": [] - }, - { - "name": "shaft3", - "properties": [] - }, - { - "name": "gearbase", - "properties": [] - }, - { - "name": "gearbase_hole1", - "properties": [] - }, - { - "name": "gearbase_hole3", - "properties": [] - }, - { - "name": "left_hand", - "properties": [] - } - ], - "constraints": [ - { - "source": "outwardgripper", - "name": "can_manipulate", - "target": "gear2" - }, - { - "source": "clampgripper", - "name": "can_manipulate", - "target": "shaft1" - }, - { - "source": "defaultgripper", - "name": "can_manipulate", - "target": "shaft3" - }, - { - "source": "gear2", - "name": "can_insert_to", - "target": "shaft2" - }, - { - "source": "shaft1", - "name": "can_insert_to", - "target": "gearbase_hole1" - }, - { - "source": "gear3", - "name": "can_insert_to", - "target": "shaft3" - }, - { - "source": "shaft3", - "name": "can_insert_to", - "target": "gearbase_hole3" - }, - { - "source": "parallelgripper", - "name": "can_manipulate", - "target": "gear1" - }, - { - "source": "outwardgripper", - "name": "can_manipulate", - "target": "gear3" - }, - { - "source": "gear1", - "name": "can_insert_to", - "target": "shaft1" - } - ], - "relations": [ - { - "source": "left_hand", - "name": "hold", - "target": "outwardgripper" - } - ] - }, - "result": { - "summary": "selector to insert the gear3 into the shaft3 with the outwardgripper in the left_hand", - "name": "selector: insert(left_hand, outwardgripper, gear3, shaft3)", - "children": [ - { - "summary": "the target is to make the gear3 be inserted into the shaft3", - "name": "target: is_inserted_to(gear3, shaft3)" - }, - { - "summary": "sequence to insert the gear3 into the shaft3 with the outwardgripper in the left_hand", - "name": "sequence: insert(left_hand, outwardgripper, gear3, shaft3)", - "children": [ - { - "summary": "a precondition is the left_hand is holding outwardgripper", - "name": "precondition: hold(left_hand, outwardgripper)" - }, - { - "summary": "selector to pick up the gear3 with the outwardgripper in the left_hand", - "name": "selector: pick_up(left_hand, outwardgripper, gear3)", - "children": [ - { - "summary": "the target is to make the outwardgripper hold the gear3", - "name": "target: hold(outwardgripper, gear3)" - }, - { - "summary": "sequence to pick up the gear3 with the outwardgripper in the left_hand", - "name": "sequence: pick_up(left_hand, outwardgripper, gear3)", - "children": [ - { - "summary": "a precondition is the outwardgripper is empty", - "name": "precondition: is_empty(outwardgripper)" - }, - { - "summary": "a precondition is the left_hand is holding outwardgripper", - "name": "precondition: hold(left_hand, outwardgripper)" - }, - { - "summary": "the action to pick up the gear3 with the outwardgripper in the left_hand", - "name": "action: pick_up(left_hand, outwardgripper, gear3)" - } - ] - } - ] - }, - { - "summary": "the action to insert the gear3 into the shaft3 with the outwardgripper in the left_hand", - "name": "action: insert(left_hand, outwardgripper, gear3, shaft3)" - } - ] - } - ] - } -} \ No newline at end of file diff --git a/experiments/gearset1/problem_set/problem_007.json b/experiments/gearset1/problem_set/problem_007.json deleted file mode 100644 index de8036f6..00000000 --- a/experiments/gearset1/problem_set/problem_007.json +++ /dev/null @@ -1,217 +0,0 @@ -{ - "target": "target: is_inserted_to(gear1, shaft1)", - "initial_world_state": { - "objects": [ - { - "name": "clampgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "parallelgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "inwardgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "outwardgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "defaultgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "gear1", - "properties": [] - }, - { - "name": "gear2", - "properties": [] - }, - { - "name": "gear3", - "properties": [] - }, - { - "name": "shaft1", - "properties": [] - }, - { - "name": "shaft2", - "properties": [] - }, - { - "name": "shaft3", - "properties": [] - }, - { - "name": "gearbase", - "properties": [] - }, - { - "name": "gearbase_hole1", - "properties": [] - }, - { - "name": "gearbase_hole3", - "properties": [] - }, - { - "name": "left_hand", - "properties": [] - } - ], - "constraints": [ - { - "source": "clampgripper", - "name": "can_manipulate", - "target": "shaft1" - }, - { - "source": "outwardgripper", - "name": "can_manipulate", - "target": "gear2" - }, - { - "source": "parallelgripper", - "name": "can_manipulate", - "target": "gear1" - }, - { - "source": "gear1", - "name": "can_insert_to", - "target": "shaft1" - }, - { - "source": "shaft1", - "name": "can_insert_to", - "target": "gearbase_hole1" - }, - { - "source": "gear2", - "name": "can_insert_to", - "target": "shaft2" - }, - { - "source": "outwardgripper", - "name": "can_manipulate", - "target": "gear3" - }, - { - "source": "gear3", - "name": "can_insert_to", - "target": "shaft3" - }, - { - "source": "shaft3", - "name": "can_insert_to", - "target": "gearbase_hole3" - }, - { - "source": "defaultgripper", - "name": "can_manipulate", - "target": "shaft3" - } - ], - "relations": [ - { - "source": "shaft1", - "name": "is_inserted_to", - "target": "gearbase_hole1" - }, - { - "source": "left_hand", - "name": "hold", - "target": "outwardgripper" - } - ] - }, - "result": { - "summary": "selector to insert the gear1 into the shaft1 with the parallelgripper in the left_hand", - "name": "selector: insert(left_hand, parallelgripper, gear1, shaft1)", - "children": [ - { - "summary": "the target is to make the gear1 be inserted into the shaft1", - "name": "target: is_inserted_to(gear1, shaft1)" - }, - { - "summary": "sequence to insert the gear1 into the shaft1 with the parallelgripper in the left_hand", - "name": "sequence: insert(left_hand, parallelgripper, gear1, shaft1)", - "children": [ - { - "summary": "selector to change tool in left_hand from outwardgripper to parallelgripper", - "name": "selector: change_tool(left_hand, outwardgripper, parallelgripper)", - "children": [ - { - "summary": "the target is to make left_hand hold parallelgripper", - "name": "target: hold(left_hand, parallelgripper)" - }, - { - "summary": "sequence to change tool in left_hand from outwardgripper to parallelgripper", - "name": "sequence: change_tool(left_hand, outwardgripper, parallelgripper)", - "children": [ - { - "summary": "a precondition is left_hand is holding outwardgripper", - "name": "precondition: hold(left_hand, outwardgripper)" - }, - { - "summary": "a precondition is outwardgripper is empty", - "name": "precondition: is_empty(outwardgripper)" - }, - { - "summary": "the action to change tool in left_hand from outwardgripper to parallelgripper", - "name": "action: change_tool(left_hand, outwardgripper, parallelgripper)" - } - ] - } - ] - }, - { - "summary": "selector to pick up the gear1 with the parallelgripper in the left_hand", - "name": "selector: pick_up(left_hand, parallelgripper, gear1)", - "children": [ - { - "summary": "the target is to make the parallelgripper hold the gear1", - "name": "target: hold(parallelgripper, gear1)" - }, - { - "summary": "sequence to pick up the gear1 with the parallelgripper in the left_hand", - "name": "sequence: pick_up(left_hand, parallelgripper, gear1)", - "children": [ - { - "summary": "a precondition is the parallelgripper is empty", - "name": "precondition: is_empty(parallelgripper)" - }, - { - "summary": "a precondition is the left_hand is holding parallelgripper", - "name": "precondition: hold(left_hand, parallelgripper)" - }, - { - "summary": "the action to pick up the gear1 with the parallelgripper in the left_hand", - "name": "action: pick_up(left_hand, parallelgripper, gear1)" - } - ] - } - ] - }, - { - "summary": "the action to insert the gear1 into the shaft1 with the parallelgripper in the left_hand", - "name": "action: insert(left_hand, parallelgripper, gear1, shaft1)" - } - ] - } - ] - } -} \ No newline at end of file diff --git a/experiments/gearset1/problem_set/problem_008.json b/experiments/gearset1/problem_set/problem_008.json deleted file mode 100644 index c47b230a..00000000 --- a/experiments/gearset1/problem_set/problem_008.json +++ /dev/null @@ -1,150 +0,0 @@ -{ - "target": "target: is_inserted_to(gear1, shaft1)", - "initial_world_state": { - "objects": [ - { - "name": "clampgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "parallelgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "inwardgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "outwardgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "defaultgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "gear1", - "properties": [] - }, - { - "name": "gear2", - "properties": [] - }, - { - "name": "gear3", - "properties": [] - }, - { - "name": "shaft1", - "properties": [] - }, - { - "name": "shaft2", - "properties": [] - }, - { - "name": "shaft3", - "properties": [] - }, - { - "name": "gearbase", - "properties": [] - }, - { - "name": "gearbase_hole1", - "properties": [] - }, - { - "name": "gearbase_hole3", - "properties": [] - }, - { - "name": "left_hand", - "properties": [] - } - ], - "constraints": [ - { - "source": "parallelgripper", - "name": "can_manipulate", - "target": "gear1" - }, - { - "source": "outwardgripper", - "name": "can_manipulate", - "target": "gear3" - }, - { - "source": "gear2", - "name": "can_insert_to", - "target": "shaft2" - }, - { - "source": "defaultgripper", - "name": "can_manipulate", - "target": "shaft3" - }, - { - "source": "outwardgripper", - "name": "can_manipulate", - "target": "gear2" - }, - { - "source": "gear3", - "name": "can_insert_to", - "target": "shaft3" - }, - { - "source": "clampgripper", - "name": "can_manipulate", - "target": "shaft1" - }, - { - "source": "shaft3", - "name": "can_insert_to", - "target": "gearbase_hole3" - }, - { - "source": "shaft1", - "name": "can_insert_to", - "target": "gearbase_hole1" - }, - { - "source": "gear1", - "name": "can_insert_to", - "target": "shaft1" - } - ], - "relations": [ - { - "source": "shaft1", - "name": "is_inserted_to", - "target": "gearbase_hole1" - }, - { - "source": "left_hand", - "name": "hold", - "target": "outwardgripper" - }, - { - "source": "gear1", - "name": "is_inserted_to", - "target": "shaft1" - } - ] - }, - "result": { - "summary": "the target is to insert gear1 to shaft1", - "name": "target: is_inserted_to(gear1, shaft1)" - } -} \ No newline at end of file diff --git a/experiments/gearset1/problem_set/problem_009.json b/experiments/gearset1/problem_set/problem_009.json deleted file mode 100644 index 4a8e2b82..00000000 --- a/experiments/gearset1/problem_set/problem_009.json +++ /dev/null @@ -1,227 +0,0 @@ -{ - "target": "target: is_inserted_to(gear3, shaft3)", - "initial_world_state": { - "objects": [ - { - "name": "clampgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "parallelgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "inwardgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "outwardgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "defaultgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "gear1", - "properties": [] - }, - { - "name": "gear2", - "properties": [] - }, - { - "name": "gear3", - "properties": [] - }, - { - "name": "shaft1", - "properties": [] - }, - { - "name": "shaft2", - "properties": [] - }, - { - "name": "shaft3", - "properties": [] - }, - { - "name": "gearbase", - "properties": [] - }, - { - "name": "gearbase_hole1", - "properties": [] - }, - { - "name": "gearbase_hole3", - "properties": [] - }, - { - "name": "left_hand", - "properties": [] - } - ], - "constraints": [ - { - "source": "gear2", - "name": "can_insert_to", - "target": "shaft2" - }, - { - "source": "parallelgripper", - "name": "can_manipulate", - "target": "gear1" - }, - { - "source": "gear3", - "name": "can_insert_to", - "target": "shaft3" - }, - { - "source": "defaultgripper", - "name": "can_manipulate", - "target": "shaft3" - }, - { - "source": "outwardgripper", - "name": "can_manipulate", - "target": "gear2" - }, - { - "source": "gear1", - "name": "can_insert_to", - "target": "shaft1" - }, - { - "source": "outwardgripper", - "name": "can_manipulate", - "target": "gear3" - }, - { - "source": "shaft1", - "name": "can_insert_to", - "target": "gearbase_hole1" - }, - { - "source": "clampgripper", - "name": "can_manipulate", - "target": "shaft1" - }, - { - "source": "shaft3", - "name": "can_insert_to", - "target": "gearbase_hole3" - } - ], - "relations": [ - { - "source": "gear1", - "name": "is_inserted_to", - "target": "shaft1" - }, - { - "source": "shaft1", - "name": "is_inserted_to", - "target": "gearbase_hole1" - }, - { - "source": "shaft3", - "name": "is_inserted_to", - "target": "gearbase_hole3" - }, - { - "source": "left_hand", - "name": "hold", - "target": "defaultgripper" - } - ] - }, - "result": { - "summary": "selector to insert the gear3 into the shaft3 with the outwardgripper in the left_hand", - "name": "selector: insert(left_hand, outwardgripper, gear3, shaft3)", - "children": [ - { - "summary": "the target is to make the gear3 be inserted into the shaft3", - "name": "target: is_inserted_to(gear3, shaft3)" - }, - { - "summary": "sequence to insert the gear3 into the shaft3 with the outwardgripper in the left_hand", - "name": "sequence: insert(left_hand, outwardgripper, gear3, shaft3)", - "children": [ - { - "summary": "selector to change tool in left_hand from defaultgripper to outwardgripper", - "name": "selector: change_tool(left_hand, defaultgripper, outwardgripper)", - "children": [ - { - "summary": "the target is to make left_hand hold outwardgripper", - "name": "target: hold(left_hand, outwardgripper)" - }, - { - "summary": "sequence to change tool in left_hand from defaultgripper to outwardgripper", - "name": "sequence: change_tool(left_hand, defaultgripper, outwardgripper)", - "children": [ - { - "summary": "a precondition is left_hand is holding defaultgripper", - "name": "precondition: hold(left_hand, defaultgripper)" - }, - { - "summary": "a precondition is defaultgripper is empty", - "name": "precondition: is_empty(defaultgripper)" - }, - { - "summary": "the action to change tool in left_hand from defaultgripper to outwardgripper", - "name": "action: change_tool(left_hand, defaultgripper, outwardgripper)" - } - ] - } - ] - }, - { - "summary": "selector to pick up the gear3 with the outwardgripper in the left_hand", - "name": "selector: pick_up(left_hand, outwardgripper, gear3)", - "children": [ - { - "summary": "the target is to make the outwardgripper hold the gear3", - "name": "target: hold(outwardgripper, gear3)" - }, - { - "summary": "sequence to pick up the gear3 with the outwardgripper in the left_hand", - "name": "sequence: pick_up(left_hand, outwardgripper, gear3)", - "children": [ - { - "summary": "a precondition is the outwardgripper is empty", - "name": "precondition: is_empty(outwardgripper)" - }, - { - "summary": "a precondition is the left_hand is holding outwardgripper", - "name": "precondition: hold(left_hand, outwardgripper)" - }, - { - "summary": "the action to pick up the gear3 with the outwardgripper in the left_hand", - "name": "action: pick_up(left_hand, outwardgripper, gear3)" - } - ] - } - ] - }, - { - "summary": "the action to insert the gear3 into the shaft3 with the outwardgripper in the left_hand", - "name": "action: insert(left_hand, outwardgripper, gear3, shaft3)" - } - ] - } - ] - } -} \ No newline at end of file diff --git a/experiments/gearset1/problem_set/problem_010.json b/experiments/gearset1/problem_set/problem_010.json deleted file mode 100644 index 34ae14d3..00000000 --- a/experiments/gearset1/problem_set/problem_010.json +++ /dev/null @@ -1,239 +0,0 @@ -{ - "target": "target: is_inserted_to(shaft1, gearbase_hole1)", - "initial_world_state": { - "objects": [ - { - "name": "clampgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "parallelgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "inwardgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "outwardgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "defaultgripper", - "properties": [] - }, - { - "name": "gear1", - "properties": [] - }, - { - "name": "gear2", - "properties": [] - }, - { - "name": "gear3", - "properties": [] - }, - { - "name": "shaft1", - "properties": [] - }, - { - "name": "shaft2", - "properties": [] - }, - { - "name": "shaft3", - "properties": [] - }, - { - "name": "gearbase", - "properties": [] - }, - { - "name": "gearbase_hole1", - "properties": [] - }, - { - "name": "gearbase_hole3", - "properties": [] - }, - { - "name": "left_hand", - "properties": [] - } - ], - "constraints": [ - { - "source": "shaft3", - "name": "can_insert_to", - "target": "gearbase_hole3" - }, - { - "source": "defaultgripper", - "name": "can_manipulate", - "target": "shaft3" - }, - { - "source": "gear2", - "name": "can_insert_to", - "target": "shaft2" - }, - { - "source": "gear1", - "name": "can_insert_to", - "target": "shaft1" - }, - { - "source": "outwardgripper", - "name": "can_manipulate", - "target": "gear2" - }, - { - "source": "shaft1", - "name": "can_insert_to", - "target": "gearbase_hole1" - }, - { - "source": "outwardgripper", - "name": "can_manipulate", - "target": "gear3" - }, - { - "source": "clampgripper", - "name": "can_manipulate", - "target": "shaft1" - }, - { - "source": "parallelgripper", - "name": "can_manipulate", - "target": "gear1" - }, - { - "source": "gear3", - "name": "can_insert_to", - "target": "shaft3" - } - ], - "relations": [ - { - "source": "left_hand", - "name": "hold", - "target": "defaultgripper" - }, - { - "source": "defaultgripper", - "name": "hold", - "target": "shaft3" - } - ] - }, - "result": { - "summary": "selector to insert the shaft1 into the gearbase_hole1 with the clampgripper in the left_hand", - "name": "selector: insert(left_hand, clampgripper, shaft1, gearbase_hole1)", - "children": [ - { - "summary": "the target is to make the shaft1 be inserted into the gearbase_hole1", - "name": "target: is_inserted_to(shaft1, gearbase_hole1)" - }, - { - "summary": "sequence to insert the shaft1 into the gearbase_hole1 with the clampgripper in the left_hand", - "name": "sequence: insert(left_hand, clampgripper, shaft1, gearbase_hole1)", - "children": [ - { - "summary": "selector to change tool in left_hand from defaultgripper to clampgripper", - "name": "selector: change_tool(left_hand, defaultgripper, clampgripper)", - "children": [ - { - "summary": "the target is to make left_hand hold clampgripper", - "name": "target: hold(left_hand, clampgripper)" - }, - { - "summary": "sequence to change tool in left_hand from defaultgripper to clampgripper", - "name": "sequence: change_tool(left_hand, defaultgripper, clampgripper)", - "children": [ - { - "summary": "a precondition is left_hand is holding defaultgripper", - "name": "precondition: hold(left_hand, defaultgripper)" - }, - { - "summary": "selector to put down the shaft3 held by the defaultgripper in the left_hand", - "name": "selector: put_down(left_hand, defaultgripper, shaft3)", - "children": [ - { - "summary": "the target is to make the defaultgripper empty", - "name": "target: is_empty(defaultgripper)" - }, - { - "summary": "sequence to put down the shaft3 held by the defaultgripper in the left_hand", - "name": "sequence: put_down(left_hand, defaultgripper, shaft3)", - "children": [ - { - "summary": "a precondition is the left_hand is holding defaultgripper", - "name": "precondition: hold(left_hand, defaultgripper)" - }, - { - "summary": "a precondition is the defaultgripper is holding shaft3", - "name": "precondition: hold(defaultgripper, shaft3)" - }, - { - "summary": "the action to put down the shaft3 held by the defaultgripper in the left_hand", - "name": "action: put_down(left_hand, defaultgripper, shaft3)" - } - ] - } - ] - }, - { - "summary": "the action to change tool in left_hand from defaultgripper to clampgripper", - "name": "action: change_tool(left_hand, defaultgripper, clampgripper)" - } - ] - } - ] - }, - { - "summary": "selector to pick up the shaft1 with the clampgripper in the left_hand", - "name": "selector: pick_up(left_hand, clampgripper, shaft1)", - "children": [ - { - "summary": "the target is to make the clampgripper hold the shaft1", - "name": "target: hold(clampgripper, shaft1)" - }, - { - "summary": "sequence to pick up the shaft1 with the clampgripper in the left_hand", - "name": "sequence: pick_up(left_hand, clampgripper, shaft1)", - "children": [ - { - "summary": "a precondition is the clampgripper is empty", - "name": "precondition: is_empty(clampgripper)" - }, - { - "summary": "a precondition is the left_hand is holding clampgripper", - "name": "precondition: hold(left_hand, clampgripper)" - }, - { - "summary": "the action to pick up the shaft1 with the clampgripper in the left_hand", - "name": "action: pick_up(left_hand, clampgripper, shaft1)" - } - ] - } - ] - }, - { - "summary": "the action to insert the shaft1 into the gearbase_hole1 with the clampgripper in the left_hand", - "name": "action: insert(left_hand, clampgripper, shaft1, gearbase_hole1)" - } - ] - } - ] - } -} \ No newline at end of file diff --git a/experiments/gearset1/problem_set/problem_011.json b/experiments/gearset1/problem_set/problem_011.json deleted file mode 100644 index df36acc0..00000000 --- a/experiments/gearset1/problem_set/problem_011.json +++ /dev/null @@ -1,212 +0,0 @@ -{ - "target": "target: is_inserted_to(gear2, shaft2)", - "initial_world_state": { - "objects": [ - { - "name": "clampgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "parallelgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "inwardgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "outwardgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "defaultgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "gear1", - "properties": [] - }, - { - "name": "gear2", - "properties": [] - }, - { - "name": "gear3", - "properties": [] - }, - { - "name": "shaft1", - "properties": [] - }, - { - "name": "shaft2", - "properties": [] - }, - { - "name": "shaft3", - "properties": [] - }, - { - "name": "gearbase", - "properties": [] - }, - { - "name": "gearbase_hole1", - "properties": [] - }, - { - "name": "gearbase_hole3", - "properties": [] - }, - { - "name": "left_hand", - "properties": [] - } - ], - "constraints": [ - { - "source": "shaft1", - "name": "can_insert_to", - "target": "gearbase_hole1" - }, - { - "source": "shaft3", - "name": "can_insert_to", - "target": "gearbase_hole3" - }, - { - "source": "parallelgripper", - "name": "can_manipulate", - "target": "gear1" - }, - { - "source": "clampgripper", - "name": "can_manipulate", - "target": "shaft1" - }, - { - "source": "defaultgripper", - "name": "can_manipulate", - "target": "shaft3" - }, - { - "source": "outwardgripper", - "name": "can_manipulate", - "target": "gear2" - }, - { - "source": "gear3", - "name": "can_insert_to", - "target": "shaft3" - }, - { - "source": "outwardgripper", - "name": "can_manipulate", - "target": "gear3" - }, - { - "source": "gear1", - "name": "can_insert_to", - "target": "shaft1" - }, - { - "source": "gear2", - "name": "can_insert_to", - "target": "shaft2" - } - ], - "relations": [ - { - "source": "left_hand", - "name": "hold", - "target": "defaultgripper" - } - ] - }, - "result": { - "summary": "selector to insert the gear2 into the shaft2 with the outwardgripper in the left_hand", - "name": "selector: insert(left_hand, outwardgripper, gear2, shaft2)", - "children": [ - { - "summary": "the target is to make the gear2 be inserted into the shaft2", - "name": "target: is_inserted_to(gear2, shaft2)" - }, - { - "summary": "sequence to insert the gear2 into the shaft2 with the outwardgripper in the left_hand", - "name": "sequence: insert(left_hand, outwardgripper, gear2, shaft2)", - "children": [ - { - "summary": "selector to change tool in left_hand from defaultgripper to outwardgripper", - "name": "selector: change_tool(left_hand, defaultgripper, outwardgripper)", - "children": [ - { - "summary": "the target is to make left_hand hold outwardgripper", - "name": "target: hold(left_hand, outwardgripper)" - }, - { - "summary": "sequence to change tool in left_hand from defaultgripper to outwardgripper", - "name": "sequence: change_tool(left_hand, defaultgripper, outwardgripper)", - "children": [ - { - "summary": "a precondition is left_hand is holding defaultgripper", - "name": "precondition: hold(left_hand, defaultgripper)" - }, - { - "summary": "a precondition is defaultgripper is empty", - "name": "precondition: is_empty(defaultgripper)" - }, - { - "summary": "the action to change tool in left_hand from defaultgripper to outwardgripper", - "name": "action: change_tool(left_hand, defaultgripper, outwardgripper)" - } - ] - } - ] - }, - { - "summary": "selector to pick up the gear2 with the outwardgripper in the left_hand", - "name": "selector: pick_up(left_hand, outwardgripper, gear2)", - "children": [ - { - "summary": "the target is to make the outwardgripper hold the gear2", - "name": "target: hold(outwardgripper, gear2)" - }, - { - "summary": "sequence to pick up the gear2 with the outwardgripper in the left_hand", - "name": "sequence: pick_up(left_hand, outwardgripper, gear2)", - "children": [ - { - "summary": "a precondition is the outwardgripper is empty", - "name": "precondition: is_empty(outwardgripper)" - }, - { - "summary": "a precondition is the left_hand is holding outwardgripper", - "name": "precondition: hold(left_hand, outwardgripper)" - }, - { - "summary": "the action to pick up the gear2 with the outwardgripper in the left_hand", - "name": "action: pick_up(left_hand, outwardgripper, gear2)" - } - ] - } - ] - }, - { - "summary": "the action to insert the gear2 into the shaft2 with the outwardgripper in the left_hand", - "name": "action: insert(left_hand, outwardgripper, gear2, shaft2)" - } - ] - } - ] - } -} \ No newline at end of file diff --git a/experiments/gearset1/problem_set/problem_012.json b/experiments/gearset1/problem_set/problem_012.json deleted file mode 100644 index 1dad6d6a..00000000 --- a/experiments/gearset1/problem_set/problem_012.json +++ /dev/null @@ -1,212 +0,0 @@ -{ - "target": "target: is_inserted_to(shaft3, gearbase_hole3)", - "initial_world_state": { - "objects": [ - { - "name": "clampgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "parallelgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "inwardgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "outwardgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "defaultgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "gear1", - "properties": [] - }, - { - "name": "gear2", - "properties": [] - }, - { - "name": "gear3", - "properties": [] - }, - { - "name": "shaft1", - "properties": [] - }, - { - "name": "shaft2", - "properties": [] - }, - { - "name": "shaft3", - "properties": [] - }, - { - "name": "gearbase", - "properties": [] - }, - { - "name": "gearbase_hole1", - "properties": [] - }, - { - "name": "gearbase_hole3", - "properties": [] - }, - { - "name": "left_hand", - "properties": [] - } - ], - "constraints": [ - { - "source": "outwardgripper", - "name": "can_manipulate", - "target": "gear2" - }, - { - "source": "shaft1", - "name": "can_insert_to", - "target": "gearbase_hole1" - }, - { - "source": "clampgripper", - "name": "can_manipulate", - "target": "shaft1" - }, - { - "source": "defaultgripper", - "name": "can_manipulate", - "target": "shaft3" - }, - { - "source": "outwardgripper", - "name": "can_manipulate", - "target": "gear3" - }, - { - "source": "gear2", - "name": "can_insert_to", - "target": "shaft2" - }, - { - "source": "parallelgripper", - "name": "can_manipulate", - "target": "gear1" - }, - { - "source": "gear1", - "name": "can_insert_to", - "target": "shaft1" - }, - { - "source": "gear3", - "name": "can_insert_to", - "target": "shaft3" - }, - { - "source": "shaft3", - "name": "can_insert_to", - "target": "gearbase_hole3" - } - ], - "relations": [ - { - "source": "left_hand", - "name": "hold", - "target": "inwardgripper" - } - ] - }, - "result": { - "summary": "selector to insert the shaft3 into the gearbase_hole3 with the defaultgripper in the left_hand", - "name": "selector: insert(left_hand, defaultgripper, shaft3, gearbase_hole3)", - "children": [ - { - "summary": "the target is to make the shaft3 be inserted into the gearbase_hole3", - "name": "target: is_inserted_to(shaft3, gearbase_hole3)" - }, - { - "summary": "sequence to insert the shaft3 into the gearbase_hole3 with the defaultgripper in the left_hand", - "name": "sequence: insert(left_hand, defaultgripper, shaft3, gearbase_hole3)", - "children": [ - { - "summary": "selector to change tool in left_hand from inwardgripper to defaultgripper", - "name": "selector: change_tool(left_hand, inwardgripper, defaultgripper)", - "children": [ - { - "summary": "the target is to make left_hand hold defaultgripper", - "name": "target: hold(left_hand, defaultgripper)" - }, - { - "summary": "sequence to change tool in left_hand from inwardgripper to defaultgripper", - "name": "sequence: change_tool(left_hand, inwardgripper, defaultgripper)", - "children": [ - { - "summary": "a precondition is left_hand is holding inwardgripper", - "name": "precondition: hold(left_hand, inwardgripper)" - }, - { - "summary": "a precondition is inwardgripper is empty", - "name": "precondition: is_empty(inwardgripper)" - }, - { - "summary": "the action to change tool in left_hand from inwardgripper to defaultgripper", - "name": "action: change_tool(left_hand, inwardgripper, defaultgripper)" - } - ] - } - ] - }, - { - "summary": "selector to pick up the shaft3 with the defaultgripper in the left_hand", - "name": "selector: pick_up(left_hand, defaultgripper, shaft3)", - "children": [ - { - "summary": "the target is to make the defaultgripper hold the shaft3", - "name": "target: hold(defaultgripper, shaft3)" - }, - { - "summary": "sequence to pick up the shaft3 with the defaultgripper in the left_hand", - "name": "sequence: pick_up(left_hand, defaultgripper, shaft3)", - "children": [ - { - "summary": "a precondition is the defaultgripper is empty", - "name": "precondition: is_empty(defaultgripper)" - }, - { - "summary": "a precondition is the left_hand is holding defaultgripper", - "name": "precondition: hold(left_hand, defaultgripper)" - }, - { - "summary": "the action to pick up the shaft3 with the defaultgripper in the left_hand", - "name": "action: pick_up(left_hand, defaultgripper, shaft3)" - } - ] - } - ] - }, - { - "summary": "the action to insert the shaft3 into the gearbase_hole3 with the defaultgripper in the left_hand", - "name": "action: insert(left_hand, defaultgripper, shaft3, gearbase_hole3)" - } - ] - } - ] - } -} \ No newline at end of file diff --git a/experiments/gearset1/problem_set/problem_013.json b/experiments/gearset1/problem_set/problem_013.json deleted file mode 100644 index a5df33e0..00000000 --- a/experiments/gearset1/problem_set/problem_013.json +++ /dev/null @@ -1,239 +0,0 @@ -{ - "target": "target: is_inserted_to(shaft3, gearbase_hole3)", - "initial_world_state": { - "objects": [ - { - "name": "clampgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "parallelgripper", - "properties": [] - }, - { - "name": "inwardgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "outwardgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "defaultgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "gear1", - "properties": [] - }, - { - "name": "gear2", - "properties": [] - }, - { - "name": "gear3", - "properties": [] - }, - { - "name": "shaft1", - "properties": [] - }, - { - "name": "shaft2", - "properties": [] - }, - { - "name": "shaft3", - "properties": [] - }, - { - "name": "gearbase", - "properties": [] - }, - { - "name": "gearbase_hole1", - "properties": [] - }, - { - "name": "gearbase_hole3", - "properties": [] - }, - { - "name": "left_hand", - "properties": [] - } - ], - "constraints": [ - { - "source": "shaft3", - "name": "can_insert_to", - "target": "gearbase_hole3" - }, - { - "source": "gear3", - "name": "can_insert_to", - "target": "shaft3" - }, - { - "source": "defaultgripper", - "name": "can_manipulate", - "target": "shaft3" - }, - { - "source": "shaft1", - "name": "can_insert_to", - "target": "gearbase_hole1" - }, - { - "source": "outwardgripper", - "name": "can_manipulate", - "target": "gear2" - }, - { - "source": "gear1", - "name": "can_insert_to", - "target": "shaft1" - }, - { - "source": "clampgripper", - "name": "can_manipulate", - "target": "shaft1" - }, - { - "source": "outwardgripper", - "name": "can_manipulate", - "target": "gear3" - }, - { - "source": "gear2", - "name": "can_insert_to", - "target": "shaft2" - }, - { - "source": "parallelgripper", - "name": "can_manipulate", - "target": "gear1" - } - ], - "relations": [ - { - "source": "left_hand", - "name": "hold", - "target": "parallelgripper" - }, - { - "source": "parallelgripper", - "name": "hold", - "target": "gear1" - } - ] - }, - "result": { - "summary": "selector to insert the shaft3 into the gearbase_hole3 with the defaultgripper in the left_hand", - "name": "selector: insert(left_hand, defaultgripper, shaft3, gearbase_hole3)", - "children": [ - { - "summary": "the target is to make the shaft3 be inserted into the gearbase_hole3", - "name": "target: is_inserted_to(shaft3, gearbase_hole3)" - }, - { - "summary": "sequence to insert the shaft3 into the gearbase_hole3 with the defaultgripper in the left_hand", - "name": "sequence: insert(left_hand, defaultgripper, shaft3, gearbase_hole3)", - "children": [ - { - "summary": "selector to change tool in left_hand from parallelgripper to defaultgripper", - "name": "selector: change_tool(left_hand, parallelgripper, defaultgripper)", - "children": [ - { - "summary": "the target is to make left_hand hold defaultgripper", - "name": "target: hold(left_hand, defaultgripper)" - }, - { - "summary": "sequence to change tool in left_hand from parallelgripper to defaultgripper", - "name": "sequence: change_tool(left_hand, parallelgripper, defaultgripper)", - "children": [ - { - "summary": "a precondition is left_hand is holding parallelgripper", - "name": "precondition: hold(left_hand, parallelgripper)" - }, - { - "summary": "selector to put down the gear1 held by the parallelgripper in the left_hand", - "name": "selector: put_down(left_hand, parallelgripper, gear1)", - "children": [ - { - "summary": "the target is to make the parallelgripper empty", - "name": "target: is_empty(parallelgripper)" - }, - { - "summary": "sequence to put down the gear1 held by the parallelgripper in the left_hand", - "name": "sequence: put_down(left_hand, parallelgripper, gear1)", - "children": [ - { - "summary": "a precondition is the left_hand is holding parallelgripper", - "name": "precondition: hold(left_hand, parallelgripper)" - }, - { - "summary": "a precondition is the parallelgripper is holding gear1", - "name": "precondition: hold(parallelgripper, gear1)" - }, - { - "summary": "the action to put down the gear1 held by the parallelgripper in the left_hand", - "name": "action: put_down(left_hand, parallelgripper, gear1)" - } - ] - } - ] - }, - { - "summary": "the action to change tool in left_hand from parallelgripper to defaultgripper", - "name": "action: change_tool(left_hand, parallelgripper, defaultgripper)" - } - ] - } - ] - }, - { - "summary": "selector to pick up the shaft3 with the defaultgripper in the left_hand", - "name": "selector: pick_up(left_hand, defaultgripper, shaft3)", - "children": [ - { - "summary": "the target is to make the defaultgripper hold the shaft3", - "name": "target: hold(defaultgripper, shaft3)" - }, - { - "summary": "sequence to pick up the shaft3 with the defaultgripper in the left_hand", - "name": "sequence: pick_up(left_hand, defaultgripper, shaft3)", - "children": [ - { - "summary": "a precondition is the defaultgripper is empty", - "name": "precondition: is_empty(defaultgripper)" - }, - { - "summary": "a precondition is the left_hand is holding defaultgripper", - "name": "precondition: hold(left_hand, defaultgripper)" - }, - { - "summary": "the action to pick up the shaft3 with the defaultgripper in the left_hand", - "name": "action: pick_up(left_hand, defaultgripper, shaft3)" - } - ] - } - ] - }, - { - "summary": "the action to insert the shaft3 into the gearbase_hole3 with the defaultgripper in the left_hand", - "name": "action: insert(left_hand, defaultgripper, shaft3, gearbase_hole3)" - } - ] - } - ] - } -} \ No newline at end of file diff --git a/experiments/gearset1/problem_set/problem_014.json b/experiments/gearset1/problem_set/problem_014.json deleted file mode 100644 index 345cb8b4..00000000 --- a/experiments/gearset1/problem_set/problem_014.json +++ /dev/null @@ -1,156 +0,0 @@ -{ - "target": "target: is_inserted_to(gear2, shaft2)", - "initial_world_state": { - "objects": [ - { - "name": "clampgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "parallelgripper", - "properties": [] - }, - { - "name": "inwardgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "outwardgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "defaultgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "gear1", - "properties": [] - }, - { - "name": "gear2", - "properties": [] - }, - { - "name": "gear3", - "properties": [] - }, - { - "name": "shaft1", - "properties": [] - }, - { - "name": "shaft2", - "properties": [] - }, - { - "name": "shaft3", - "properties": [] - }, - { - "name": "gearbase", - "properties": [] - }, - { - "name": "gearbase_hole1", - "properties": [] - }, - { - "name": "gearbase_hole3", - "properties": [] - }, - { - "name": "lampbase", - "properties": [] - }, - { - "name": "chair_leg1", - "properties": [] - }, - { - "name": "left_hand", - "properties": [] - } - ], - "constraints": [ - { - "source": "clampgripper", - "name": "can_manipulate", - "target": "shaft1" - }, - { - "source": "outwardgripper", - "name": "can_manipulate", - "target": "gear3" - }, - { - "source": "shaft3", - "name": "can_insert_to", - "target": "gearbase_hole3" - }, - { - "source": "parallelgripper", - "name": "can_manipulate", - "target": "gear1" - }, - { - "source": "gear3", - "name": "can_insert_to", - "target": "shaft3" - }, - { - "source": "outwardgripper", - "name": "can_manipulate", - "target": "gear2" - }, - { - "source": "shaft1", - "name": "can_insert_to", - "target": "gearbase_hole1" - }, - { - "source": "defaultgripper", - "name": "can_manipulate", - "target": "shaft3" - }, - { - "source": "gear1", - "name": "can_insert_to", - "target": "shaft1" - }, - { - "source": "gear2", - "name": "can_insert_to", - "target": "shaft2" - } - ], - "relations": [ - { - "source": "gear2", - "name": "is_inserted_to", - "target": "shaft2" - }, - { - "source": "parallelgripper", - "name": "hold", - "target": "gear1" - }, - { - "source": "left_hand", - "name": "hold", - "target": "parallelgripper" - } - ] - }, - "result": { - "summary": "the target is to insert the gear2 into the shaft2", - "name": "target: is_inserted_to(gear2, shaft2)" - } -} \ No newline at end of file diff --git a/experiments/gearset1/problem_set/problem_015.json b/experiments/gearset1/problem_set/problem_015.json deleted file mode 100644 index a4714fe9..00000000 --- a/experiments/gearset1/problem_set/problem_015.json +++ /dev/null @@ -1,166 +0,0 @@ -{ - "target": "target: is_inserted_to(gear3, shaft3)", - "initial_world_state": { - "objects": [ - { - "name": "clampgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "parallelgripper", - "properties": [] - }, - { - "name": "inwardgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "outwardgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "defaultgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "gear1", - "properties": [] - }, - { - "name": "gear2", - "properties": [] - }, - { - "name": "gear3", - "properties": [] - }, - { - "name": "shaft1", - "properties": [] - }, - { - "name": "shaft2", - "properties": [] - }, - { - "name": "shaft3", - "properties": [] - }, - { - "name": "gearbase", - "properties": [] - }, - { - "name": "gearbase_hole1", - "properties": [] - }, - { - "name": "gearbase_hole3", - "properties": [] - }, - { - "name": "lampbase", - "properties": [] - }, - { - "name": "chair_leg1", - "properties": [] - }, - { - "name": "left_hand", - "properties": [] - } - ], - "constraints": [ - { - "source": "outwardgripper", - "name": "can_manipulate", - "target": "gear2" - }, - { - "source": "shaft1", - "name": "can_insert_to", - "target": "gearbase_hole1" - }, - { - "source": "outwardgripper", - "name": "can_manipulate", - "target": "gear3" - }, - { - "source": "parallelgripper", - "name": "can_manipulate", - "target": "gear1" - }, - { - "source": "defaultgripper", - "name": "can_manipulate", - "target": "shaft3" - }, - { - "source": "clampgripper", - "name": "can_manipulate", - "target": "shaft1" - }, - { - "source": "gear3", - "name": "can_insert_to", - "target": "shaft3" - }, - { - "source": "shaft3", - "name": "can_insert_to", - "target": "gearbase_hole3" - }, - { - "source": "gear1", - "name": "can_insert_to", - "target": "shaft1" - }, - { - "source": "gear2", - "name": "can_insert_to", - "target": "shaft2" - } - ], - "relations": [ - { - "source": "shaft1", - "name": "is_inserted_to", - "target": "gearbase_hole1" - }, - { - "source": "left_hand", - "name": "hold", - "target": "parallelgripper" - }, - { - "source": "gear3", - "name": "is_inserted_to", - "target": "shaft3" - }, - { - "source": "shaft3", - "name": "is_inserted_to", - "target": "gearbase_hole3" - }, - { - "source": "gear2", - "name": "is_inserted_to", - "target": "shaft2" - } - ] - }, - "result": { - "summary": "the target is to insert gear3 into shaft3", - "name": "target: is_inserted_to(gear3, shaft3)" - } -} \ No newline at end of file diff --git a/experiments/gearset1/problem_set/problem_016.json b/experiments/gearset1/problem_set/problem_016.json deleted file mode 100644 index cd7ec465..00000000 --- a/experiments/gearset1/problem_set/problem_016.json +++ /dev/null @@ -1,235 +0,0 @@ -{ - "target": "target: is_inserted_to(gear1, shaft1)", - "initial_world_state": { - "objects": [ - { - "name": "clampgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "parallelgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "inwardgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "outwardgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "defaultgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "gear1", - "properties": [] - }, - { - "name": "gear2", - "properties": [] - }, - { - "name": "gear3", - "properties": [] - }, - { - "name": "shaft1", - "properties": [] - }, - { - "name": "shaft2", - "properties": [] - }, - { - "name": "shaft3", - "properties": [] - }, - { - "name": "gearbase", - "properties": [] - }, - { - "name": "gearbase_hole1", - "properties": [] - }, - { - "name": "gearbase_hole3", - "properties": [] - }, - { - "name": "lampbase", - "properties": [] - }, - { - "name": "chair_leg1", - "properties": [] - }, - { - "name": "left_hand", - "properties": [] - } - ], - "constraints": [ - { - "source": "gear2", - "name": "can_insert_to", - "target": "shaft2" - }, - { - "source": "outwardgripper", - "name": "can_manipulate", - "target": "gear3" - }, - { - "source": "outwardgripper", - "name": "can_manipulate", - "target": "gear2" - }, - { - "source": "gear3", - "name": "can_insert_to", - "target": "shaft3" - }, - { - "source": "shaft3", - "name": "can_insert_to", - "target": "gearbase_hole3" - }, - { - "source": "shaft1", - "name": "can_insert_to", - "target": "gearbase_hole1" - }, - { - "source": "parallelgripper", - "name": "can_manipulate", - "target": "gear1" - }, - { - "source": "clampgripper", - "name": "can_manipulate", - "target": "shaft1" - }, - { - "source": "gear1", - "name": "can_insert_to", - "target": "shaft1" - }, - { - "source": "defaultgripper", - "name": "can_manipulate", - "target": "shaft3" - } - ], - "relations": [ - { - "source": "shaft3", - "name": "is_inserted_to", - "target": "gearbase_hole3" - }, - { - "source": "shaft1", - "name": "is_inserted_to", - "target": "gearbase_hole1" - }, - { - "source": "left_hand", - "name": "hold", - "target": "inwardgripper" - }, - { - "source": "gear2", - "name": "is_inserted_to", - "target": "shaft2" - } - ] - }, - "result": { - "summary": "selector to insert the gear1 into the shaft1 with the parallelgripper in the left_hand", - "name": "selector: insert(left_hand, parallelgripper, gear1, shaft1)", - "children": [ - { - "summary": "the target is to make the gear1 be inserted into the shaft1", - "name": "target: is_inserted_to(gear1, shaft1)" - }, - { - "summary": "sequence to insert the gear1 into the shaft1 with the parallelgripper in the left_hand", - "name": "sequence: insert(left_hand, parallelgripper, gear1, shaft1)", - "children": [ - { - "summary": "selector to change tool in left_hand from inwardgripper to parallelgripper", - "name": "selector: change_tool(left_hand, inwardgripper, parallelgripper)", - "children": [ - { - "summary": "the target is to make left_hand hold parallelgripper", - "name": "target: hold(left_hand, parallelgripper)" - }, - { - "summary": "sequence to change tool in left_hand from inwardgripper to parallelgripper", - "name": "sequence: change_tool(left_hand, inwardgripper, parallelgripper)", - "children": [ - { - "summary": "a precondition is left_hand is holding inwardgripper", - "name": "precondition: hold(left_hand, inwardgripper)" - }, - { - "summary": "a precondition is inwardgripper is empty", - "name": "precondition: is_empty(inwardgripper)" - }, - { - "summary": "the action to change tool in left_hand from inwardgripper to parallelgripper", - "name": "action: change_tool(left_hand, inwardgripper, parallelgripper)" - } - ] - } - ] - }, - { - "summary": "selector to pick up the gear1 with the parallelgripper in the left_hand", - "name": "selector: pick_up(left_hand, parallelgripper, gear1)", - "children": [ - { - "summary": "the target is to make the parallelgripper hold the gear1", - "name": "target: hold(parallelgripper, gear1)" - }, - { - "summary": "sequence to pick up the gear1 with the parallelgripper in the left_hand", - "name": "sequence: pick_up(left_hand, parallelgripper, gear1)", - "children": [ - { - "summary": "a precondition is the parallelgripper is empty", - "name": "precondition: is_empty(parallelgripper)" - }, - { - "summary": "a precondition is the left_hand is holding parallelgripper", - "name": "precondition: hold(left_hand, parallelgripper)" - }, - { - "summary": "the action to pick up the gear1 with the parallelgripper in the left_hand", - "name": "action: pick_up(left_hand, parallelgripper, gear1)" - } - ] - } - ] - }, - { - "summary": "the action to insert the gear1 into the shaft1 with the parallelgripper in the left_hand", - "name": "action: insert(left_hand, parallelgripper, gear1, shaft1)" - } - ] - } - ] - } -} \ No newline at end of file diff --git a/experiments/gearset1/rec_sk_gen.py b/experiments/gearset1/rec_sk_gen.py deleted file mode 100644 index 15a852bc..00000000 --- a/experiments/gearset1/rec_sk_gen.py +++ /dev/null @@ -1,706 +0,0 @@ -import json -import os -from pprint import pprint - -""" -generate the behavior tree iteratively. -""" - -os.environ["LANGCHAIN_TRACING_V2"] = "true" -os.environ["LANGCHAIN_ENDPOINT"] = "https://api.smith.langchain.com" -os.environ["LANGCHAIN_PROJECT"] = "kios_agent" - -from kios_bt.bt_stewardship import BehaviorTreeStewardship -from kios_scene.scene_factory import SceneFactory -from kios_bt.bt_factory import BehaviorTreeFactory -from kios_robot.robot_interface import RobotInterface -from kios_world.world_interface import WorldInterface - -from kios_agent.llm_supporter import KiosLLMSupporter -from kios_agent.data_types import KiosPromptSkeleton - -from kios_agent.kios_tools import ( - BehaviorTreeExecutorTool, - BehaviorTreeSimulatorTool, - WorldStateQueryTool, -) - -from typing import List, Tuple, Annotated, TypedDict -import operator - -from dotenv import load_dotenv - -from langchain import hub - -from langchain_openai import ChatOpenAI - -from langchain.chains.openai_functions import ( - create_structured_output_runnable, - create_openai_fn_runnable, -) -from langchain_core.utils.function_calling import convert_to_openai_function -from langchain_core.agents import AgentActionMessageLog, AgentFinish -from langchain_core.messages import HumanMessage -from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder -from langchain_core.output_parsers import JsonOutputParser, StrOutputParser -from langchain_core.pydantic_v1 import BaseModel, Field - -from langchain.agents.format_scratchpad import format_to_openai_function_messages -from langchain.agents import create_openai_functions_agent, AgentExecutor - -from langchain.prompts.pipeline import PipelinePromptTemplate -from langchain.prompts.prompt import PromptTemplate - -from langchain_community.document_loaders import TextLoader -from langchain_openai import OpenAIEmbeddings - -from langchain_community.vectorstores import FAISS -from langchain.text_splitter import ( - RecursiveCharacterTextSplitter, - CharacterTextSplitter, -) - -from langgraph.graph import StateGraph, END - -from langsmith import traceable - -load_dotenv() - -from kios_utils.pybt_test import generate_bt_stewardship, render_dot_tree -from kios_utils.pddl_problem_parser import parse_problem_init, parse_problem_objects - - -def render_bt(bt_json: json): - test_class = BehaviorTreeFactory() - bt = test_class.from_json_to_simple_bt(bt_json) - # bt = test_class.from_json_to_tree_root(bt_json) - bt_stewardship = generate_bt_stewardship(bt) - # bt_stewardship.setup(timeout=15) - render_dot_tree(bt_stewardship) - - -####################### dirs -current_dir = os.path.dirname(os.path.abspath(__file__)) -scene_path = os.path.join(current_dir, "scene.json") -# bt_json_file_path = os.path.join(current_dir, "behavior_tree.json") -world_state_path = os.path.join(current_dir, "world_state.json") -problem_path = os.path.join(current_dir, "gearset.problem") -domain_knowledge_path = os.path.join(current_dir, "domain_knowledge.txt") - -####################### problem -with open(problem_path, "r") as file: - problem = file.read() - -####################### scene -with open(scene_path, "r") as file: - scene_json_object = json.load(file) - -scene = SceneFactory().create_scene_from_json(scene_json_object) - -####################### world -world_interface = WorldInterface() -with open(world_state_path, "r") as file: - world_state_json_object = json.load(file) - world_interface.load_world_from_json(world_state_json_object) - -from_problem = parse_problem_init(problem=problem) -objects = parse_problem_objects(problem=problem) -world_interface.update_world(from_problem) - -world_state_json = world_interface.get_world_to_json() -pprint(from_problem) -pprint(world_state_json) -pprint(objects) - - -####################### robot -robot_interface = RobotInterface( - robot_address="127.0.0.1", - robot_port=12000, -) -robot_interface.setup_scene(scene) - -####################### bt_factory -bt_factory = BehaviorTreeFactory( - world_interface=world_interface, - robot_interface=robot_interface, -) - -####################### behavior_tree_stewardship -behavior_tree_stewardship = BehaviorTreeStewardship( - behaviortree_factory=bt_factory, - world_interface=world_interface, - robot_interface=robot_interface, -) - -####################### * tools -# bt_executor_tool = BehaviorTreeExecutorTool(behavior_tree_stewardship) - -bt_sim_tool = BehaviorTreeSimulatorTool(metadata={"bt_stw": behavior_tree_stewardship}) - -# world_state_query_tool = WorldStateQueryTool(behavior_tree_stewardship) - -executor_tools = [bt_sim_tool] - -# * kios data prompt skeleton dir -data_dir = os.environ.get("KIOS_DATA_DIR").format(username=os.getlogin()) -print(data_dir) -prompt_sk_dir = os.path.join(data_dir, "prompt_skeletons") -prompt_dir = os.path.join(data_dir, "prompts") - - -# * the graph state -class PlanExecuteState(TypedDict): - plan: List[str] - behavior_tree: dict - world_state: Annotated[ - List[dict], operator.add - ] # ! to add, you need to make world_state a list of dict - past_steps: Annotated[List[Tuple], operator.add] - user_input: str - objects: dict[str, list[str]] - last_behavior_tree: dict - last_failed_node: dict - runtime_world_state: dict - behavior_tree_execution_summary: str - BTExecutionHasSucceeded: bool - - -########## verifier - -behaviortree_file = os.path.join(prompt_dir, "vv/behaviortree.txt") -template_file = os.path.join(prompt_dir, "vv/template.txt") -task_file = os.path.join(prompt_dir, "vv/task.txt") -system_file = os.path.join(prompt_dir, "vv/system.txt") -object_file = os.path.join(prompt_dir, "vv/object.txt") -domain_file = os.path.join(prompt_dir, "vv/domain.txt") -state_file = os.path.join(prompt_dir, "vv/state.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) -with open(object_file, "r") as f: - object_ppt = PromptTemplate.from_template(f.read()) -with open(behaviortree_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - behaviortree_ppt = ppt_tmp.partial(input=f.read()) - -with open(state_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - state_ppt = ppt_tmp.partial(input=f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {state} - - {object} - - {behaviortree} - - {template} - """ -) - -verifier_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ("object", object_ppt), - ("behaviortree", behaviortree_ppt), - ("state", state_ppt), - ], -) - -verifier_chain = ( - verifier_ppt_ppl - | ChatOpenAI(model="gpt-4-turbo-preview", temperature=0) - | StrOutputParser() -) - - -# * output schema of the recursive_sk_generator -class SolutionBehaviortree(BaseModel): - """the fixed behavior tree""" - - behavior_tree: dict = Field( - description="the behavior tree that adheres the required format." - ) - - -behaviortree_file = os.path.join(prompt_dir, "rec_sk_gen/behaviortree.txt") -template_file = os.path.join(prompt_dir, "rec_sk_gen/template.txt") -task_file = os.path.join(prompt_dir, "rec_sk_gen/task.txt") -system_file = os.path.join(prompt_dir, "rec_sk_gen/system.txt") -object_file = os.path.join(prompt_dir, "rec_sk_gen/object.txt") -domain_file = os.path.join(prompt_dir, "rec_sk_gen/domain.txt") -example_file = os.path.join(prompt_dir, "rec_sk_gen/example.txt") -state_file = os.path.join(prompt_dir, "rec_sk_gen/state.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) -with open(object_file, "r") as f: - object_ppt = PromptTemplate.from_template(f.read()) -with open(behaviortree_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - behaviortree_ppt = ppt_tmp.partial(input=f.read()) - -with open(state_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - state_ppt = ppt_tmp.partial(input=f.read()) - -with open(example_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - example_ppt = ppt_tmp.partial(input=f.read()) -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {state} - - {object} - - {behaviortree} - - {example} - - {template} - """ -) - -re_sk_gen_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ("object", object_ppt), - ("behaviortree", behaviortree_ppt), - ("example", example_ppt), - ("state", state_ppt), - ], -) - -rec_sk_gen_chain = ( - re_sk_gen_ppt_ppl - | ChatOpenAI(model="gpt-4-turbo-preview", temperature=0) - | JsonOutputParser() -) - -# recursive_sk_generator = create_structured_output_runnable( -# SolutionBehaviortree, -# ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), -# re_sk_gen_ppt_ppl, -# mode="openai-json", -# ) - - -##################################################### * planner -# * output schema of the planner -class Plan(BaseModel): - """Plan to follow in future""" - - steps: List[str] = Field( - description="a list of different steps to follow, should be in sorted order" - ) - - -template_file = os.path.join(prompt_dir, "planner/template.txt") -task_file = os.path.join(prompt_dir, "planner/task.txt") -system_file = os.path.join(prompt_dir, "planner/system.txt") -domain_file = os.path.join(prompt_dir, "planner/domain.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {template} - """ -) - -planner_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ], -) - -planner = create_structured_output_runnable( - Plan, ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), planner_ppt_ppl -) - -##################################################### * plan_updaterner - -template_file = os.path.join(prompt_dir, "plan_updater/template.txt") -task_file = os.path.join(prompt_dir, "plan_updater/task.txt") -system_file = os.path.join(prompt_dir, "plan_updater/system.txt") -domain_file = os.path.join(prompt_dir, "plan_updater/domain.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {template} - """ -) - -plan_updater_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ], -) - -plan_updater = create_structured_output_runnable( - Plan, ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), plan_updater_ppt_ppl -) -# ! create openai fn runnable has bug - - -##################################################### * graph node functions -@traceable(name="user_input_node_step") -async def user_input_step(state: PlanExecuteState): - """ - get the input from the user - """ - print(f"-----user_input_step-----") - - user_input = input("Your next instruction: ") - - return { - "user_input": user_input, - } - - -@traceable(name="behavior_tree_generate_step") -async def behavior_tree_generate_step(state: PlanExecuteState): - """ - generate the behavior tree based on the instruction - """ - print(f"-----behavior_tree_generate_step-----") - - instruction = state["plan"][0] - if state["runtime_world_state"] is None: - runtime_world_state = state["world_state"][-1] - else: - runtime_world_state = state["runtime_world_state"] - objects = state["objects"] - - # solution_bt = await recursive_sk_generator.ainvoke( - # { - # "objects": objects, - # "runtime_world_state": runtime_world_state, - # "last_behavior_tree": state["last_behavior_tree"], - # "last_failed_node": state["last_failed_node"], - # "instructions": instruction, - # } - # ) - - bt_skeleton = await rec_sk_gen_chain.ainvoke( - { - "objects": objects, - "runtime_world_state": runtime_world_state, - "last_behavior_tree": state["last_behavior_tree"], - "last_failed_node": state["last_failed_node"], - "summary": state["behavior_tree_execution_summary"], - "instructions": instruction, - } - ) - - # print(solution_bt) - # pause = input("paused here") - - # bt_skeleton = solution_bt.behavior_tree - - render_bt(bt_skeleton) - - pause = input("paused here") - - return { - "last_behavior_tree": bt_skeleton, - "BTExecutionHasSucceeded": False, - } - - -@traceable(name="behavior_tree_execute_step") -async def behavior_tree_execute_step(state: PlanExecuteState): - """ - execute the first step of the plan, append the result to the past steps - """ - print(f"-----behavior_tree_execute_step-----") - - this_step = state["plan"][0] - behavior_tree_skeleton = state["last_behavior_tree"] - latest_world_state = state["world_state"][-1] - - # * first sim run - tree_result, skeleton_json = behavior_tree_stewardship.sk_sim_run( - world_state=latest_world_state, skeleton_json=behavior_tree_skeleton - ) - - pprint(tree_result.to_json()) - pause = input("paused here") - - # * check result - if tree_result.result == "success": - - # * second fake run - tree_result, skeleton_json = behavior_tree_stewardship.sk_fake_run( - world_state=latest_world_state, skeleton_json=behavior_tree_skeleton - ) - - pprint(tree_result.to_json()) - pause = input("paused here") - - if tree_result.result == "success": - return { - "BTExecutionHasSucceeded": True, - "past_steps": ( - this_step, - tree_result.result, - ), # * only one because new plan will be generated and old steps are all removed - "world_state": [tree_result.world_state], - # * clear all - "last_behavior_tree": None, - "last_failed_node": None, - "runtime_world_state": None, - "behavior_tree_execution_summary": None, - } - else: - return { - "world_state": [tree_result.world_state], # * real world state - "last_behavior_tree": skeleton_json, - "last_failed_node": { - "summary": tree_result.final_node.get("summary"), - "name": tree_result.final_node.get("name"), - "identifier": tree_result.final_node.get("identifier"), - }, - "runtime_world_state": tree_result.world_state, - "behavior_tree_execution_summary": tree_result.summary, - } - - else: - return { - # ! do not change world state - "last_behavior_tree": skeleton_json, - "last_failed_node": { - "summary": tree_result.final_node.get("summary"), - "name": tree_result.final_node.get("name"), - "identifier": tree_result.final_node.get("identifier"), - }, - "runtime_world_state": tree_result.world_state, - "behavior_tree_execution_summary": tree_result.summary, - } - - -@traceable(name="planner_step") -async def planner_step(state: PlanExecuteState): - """ - plan the steps based on user input and world state - """ - print(f"-----plan_step-----") - - plan = await planner.ainvoke( - { - "user_input": state["user_input"], - "world_state": state["world_state"], - } - ) - return {"plan": plan.steps} - - -@traceable(name="plan_updater_step") -async def plan_updater_step(state: PlanExecuteState): - """ - if return a response, then success, response the use, end. - otherwise, return the updated newplan (normally the same as the old plan with the first step popped out. - """ - print(f"-----plan_updater_step-----") - - # ! BOOOM! BUG! - output = await plan_updater.ainvoke( - { - "user_input": state["user_input"], - "plan": state["plan"], - "world_state": state["world_state"], - "past_steps": state["past_steps"], - } - ) - # if isinstance( - # output, UpdaterResponse - # ): # * determine if it is time to response and end - # return { - # "response": "Your last instruction has been finished." - # } # * Don't need to update. - # else: - return {"plan": output.steps} # * update the plan - - -##################################################### * construct the graph - -workflow = StateGraph(PlanExecuteState) - -workflow.add_node("planner", planner_step) - -workflow.add_node("behavior_tree_generator", behavior_tree_generate_step) - -workflow.add_node("behavior_tree_executor", behavior_tree_execute_step) - -workflow.add_node("plan_updater", plan_updater_step) - -workflow.add_node("user_input_node", user_input_step) - -workflow.set_entry_point("user_input_node") - -workflow.add_edge( - "planner", "behavior_tree_generator" -) # ! may need to add a condition here - -workflow.add_edge("behavior_tree_generator", "behavior_tree_executor") - - -def executor_should_end(state: PlanExecuteState): - """ - end router - """ - print(f"-----executor_should_end-----") - - if state["BTExecutionHasSucceeded"] == True: - state["BTExecutionHasSucceeded"] = False - return True - else: - return False - - -workflow.add_conditional_edges( - "behavior_tree_executor", - executor_should_end, - { - True: "plan_updater", - False: "behavior_tree_generator", - }, -) - - -def plan_updater_should_end(state: PlanExecuteState): - """ - end router - """ - print(f"-----plan_updater_should_end-----") - - # if state["response"]: - # return True - # else: - # return False - if state["plan"] == [] or len(state["plan"]) == 0: - return True - else: - return False - - -workflow.add_conditional_edges( - "plan_updater", - plan_updater_should_end, - { - True: "user_input_node", - False: "behavior_tree_generator", - }, -) - - -def user_input_should_end(state: PlanExecuteState): - """ - if the user input is empty, then end - """ - print(f"-----user_input_should_end-----") - - # print(f'user_input: {state["user_input"]}') - # print(f'world_state: {state["world_state"]}') - - if not state["user_input"] or state["user_input"] == "": - return True - else: - return False - - -workflow.add_conditional_edges( - "user_input_node", - user_input_should_end, - { - True: END, - False: "planner", - }, -) - -app = workflow.compile() - -config = {"recursion_limit": 500} - -inputs = { - "world_state": [world_state_json], - "objects": objects, -} - - -async def core_run(): - async for event in app.astream( - inputs, - config=config, - ): - for k, v in event.items(): - if k != "__end__": - print(v) - - -if __name__ == "__main__": - import asyncio - - asyncio.run(core_run()) diff --git a/experiments/gearset1/rec_sk_seq.py b/experiments/gearset1/rec_sk_seq.py deleted file mode 100644 index e95a9fe4..00000000 --- a/experiments/gearset1/rec_sk_seq.py +++ /dev/null @@ -1,659 +0,0 @@ -import json -import os -from pprint import pprint -from typing import List, Tuple, Annotated, TypedDict -import operator - -""" -recursive generation + sequence guidance + simulation -""" - -os.environ["LANGCHAIN_TRACING_V2"] = "true" -os.environ["LANGCHAIN_ENDPOINT"] = "https://api.smith.langchain.com" -os.environ["LANGCHAIN_PROJECT"] = "kios_agent" - -from kios_bt.bt_stewardship import BehaviorTreeStewardship -from kios_scene.scene_factory import SceneFactory -from kios_bt.bt_factory import BehaviorTreeFactory -from kios_robot.robot_interface import RobotInterface -from kios_world.world_interface import WorldInterface - -from kios_agent.llm_supporter import KiosLLMSupporter -from kios_agent.data_types import KiosPromptSkeleton - -from kios_agent.kios_tools import ( - BehaviorTreeExecutorTool, - BehaviorTreeSimulatorTool, - WorldStateQueryTool, -) - -from dotenv import load_dotenv - -from langchain import hub - -from langchain_openai import ChatOpenAI - -from langchain.chains.openai_functions import ( - create_structured_output_runnable, - create_openai_fn_runnable, -) -from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder -from langchain_core.output_parsers import JsonOutputParser, StrOutputParser -from langchain_core.pydantic_v1 import BaseModel, Field - -from langchain.prompts.pipeline import PipelinePromptTemplate -from langchain.prompts.prompt import PromptTemplate - -from langgraph.graph import StateGraph, END - -from langsmith import traceable - -load_dotenv() - -from kios_utils.pybt_test import generate_bt_stewardship, render_dot_tree -from kios_utils.pddl_problem_parser import parse_problem_init, parse_problem_objects - - -def render_bt(bt_json: json): - test_class = BehaviorTreeFactory() - bt = test_class.from_json_to_simple_bt(bt_json) - # bt = test_class.from_json_to_tree_root(bt_json) - bt_stewardship = generate_bt_stewardship(bt) - # bt_stewardship.setup(timeout=15) - render_dot_tree(bt_stewardship) - - -####################### dirs -current_dir = os.path.dirname(os.path.abspath(__file__)) -scene_path = os.path.join(current_dir, "scene.json") -# bt_json_file_path = os.path.join(current_dir, "behavior_tree.json") -world_state_path = os.path.join(current_dir, "world_state.json") -problem_path = os.path.join(current_dir, "gearset.problem") -domain_knowledge_path = os.path.join(current_dir, "domain_knowledge.txt") - -####################### problem -with open(problem_path, "r") as file: - problem = file.read() - -####################### scene -with open(scene_path, "r") as file: - scene_json_object = json.load(file) - -scene = SceneFactory().create_scene_from_json(scene_json_object) - -####################### world -world_interface = WorldInterface() -with open(world_state_path, "r") as file: - world_state_json_object = json.load(file) - world_interface.load_world_from_json(world_state_json_object) - -from_problem = parse_problem_init(problem=problem) -objects = parse_problem_objects(problem=problem) -world_interface.update_world(from_problem) - -world_state_json = world_interface.get_world_to_json() -pprint(from_problem) -pprint(world_state_json) -pprint(objects) - - -####################### robot -robot_interface = RobotInterface( - robot_address="127.0.0.1", - robot_port=12000, -) -robot_interface.setup_scene(scene) - -####################### bt_factory -bt_factory = BehaviorTreeFactory( - world_interface=world_interface, - robot_interface=robot_interface, -) - -####################### behavior_tree_stewardship -behavior_tree_stewardship = BehaviorTreeStewardship( - behaviortree_factory=bt_factory, - world_interface=world_interface, - robot_interface=robot_interface, -) - -####################### * tools -# bt_executor_tool = BehaviorTreeExecutorTool(behavior_tree_stewardship) - -bt_sim_tool = BehaviorTreeSimulatorTool(metadata={"bt_stw": behavior_tree_stewardship}) - -# world_state_query_tool = WorldStateQueryTool(behavior_tree_stewardship) - -executor_tools = [bt_sim_tool] - -# * kios data prompt skeleton dir -data_dir = os.environ.get("KIOS_DATA_DIR").format(username=os.getlogin()) -print(data_dir) -prompt_sk_dir = os.path.join(data_dir, "prompt_skeletons") -prompt_dir = os.path.join(data_dir, "prompts") - - -# * the graph state -class PlanExecuteState(TypedDict): - plan: List[str] - behavior_tree: dict - world_state: Annotated[ - List[dict], operator.add - ] # ! to add, you need to make world_state a list of dict - past_steps: Annotated[List[Tuple], operator.add] - user_input: str - objects: dict[str, list[str]] - last_behavior_tree: dict - last_failed_node: dict - runtime_world_state: dict - behavior_tree_execution_summary: str - BTExecutionHasSucceeded: bool - - action_sequence: List[str] - - -########## verifier - -template_file = os.path.join(prompt_dir, "seq_planner/template.txt") -task_file = os.path.join(prompt_dir, "seq_planner/task.txt") -system_file = os.path.join(prompt_dir, "seq_planner/system.txt") -domain_file = os.path.join(prompt_dir, "seq_planner/domain.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {template} - """ -) - -seq_planner_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ], -) - -seq_planner_chain = ( - seq_planner_ppt_ppl - | ChatOpenAI(model="gpt-4-turbo-preview", temperature=0) - | StrOutputParser() -) - - -# * output schema of the recursive_sk_generator -class SolutionBehaviortree(BaseModel): - """the fixed behavior tree""" - - behavior_tree: dict = Field( - description="the behavior tree that adheres the required format." - ) - - -behaviortree_file = os.path.join(prompt_dir, "rec_sk_gen/behaviortree.txt") -template_file = os.path.join(prompt_dir, "rec_sk_gen/template.txt") -task_file = os.path.join(prompt_dir, "rec_sk_gen/task.txt") -system_file = os.path.join(prompt_dir, "rec_sk_gen/system.txt") -object_file = os.path.join(prompt_dir, "rec_sk_gen/object.txt") -domain_file = os.path.join(prompt_dir, "rec_sk_gen/domain.txt") -example_file = os.path.join(prompt_dir, "rec_sk_gen/example.txt") -state_file = os.path.join(prompt_dir, "rec_sk_gen/state.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) -with open(object_file, "r") as f: - object_ppt = PromptTemplate.from_template(f.read()) -with open(behaviortree_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - behaviortree_ppt = ppt_tmp.partial(input=f.read()) - -with open(state_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - state_ppt = ppt_tmp.partial(input=f.read()) - -with open(example_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - example_ppt = ppt_tmp.partial(input=f.read()) -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {state} - - {object} - - {behaviortree} - - {example} - - {template} - """ -) -# * recureive_sk_generator -re_sk_gen_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ("object", object_ppt), - ("behaviortree", behaviortree_ppt), - ("example", example_ppt), - ("state", state_ppt), - ], -) - -rec_sk_gen_chain = ( - re_sk_gen_ppt_ppl - | ChatOpenAI(model="gpt-4-turbo-preview", temperature=0) - | JsonOutputParser() -) - -##################################################### * planner -# * output schema of the planner -class Plan(BaseModel): - """Plan to follow in future""" - - steps: List[str] = Field( - description="a list of different steps to follow, should be in sorted order" - ) - - -template_file = os.path.join(prompt_dir, "planner/template.txt") -task_file = os.path.join(prompt_dir, "planner/task.txt") -system_file = os.path.join(prompt_dir, "planner/system.txt") -domain_file = os.path.join(prompt_dir, "planner/domain.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {template} - """ -) - -planner_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ], -) - -planner = create_structured_output_runnable( - Plan, ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), planner_ppt_ppl -) - -##################################################### * plan_updaterner - -template_file = os.path.join(prompt_dir, "plan_updater/template.txt") -task_file = os.path.join(prompt_dir, "plan_updater/task.txt") -system_file = os.path.join(prompt_dir, "plan_updater/system.txt") -domain_file = os.path.join(prompt_dir, "plan_updater/domain.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {template} - """ -) - -plan_updater_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ], -) - -plan_updater = create_structured_output_runnable( - Plan, ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), plan_updater_ppt_ppl -) -# ! create openai fn runnable has bug - - -##################################################### * graph node functions -@traceable(name="user_input_node_step") -async def user_input_step(state: PlanExecuteState): - """ - get the input from the user - """ - print(f"-----user_input_step-----") - - user_input = input("Your next instruction: ") - - return { - "user_input": user_input, - } - - -@traceable(name="sequence_generate_step") -async def sequence_generate_step(state: PlanExecuteState): - """ - generate the sequence based on the instruction - """ - print(f"-----sequence_generate_step-----") - - instruction = state["plan"][0] - start_world_state = state["world_state"][-1] - objects = state["objects"] - - action_sequence = await seq_planner_chain.ainvoke( - { - "objects": objects, - "start_world_state": start_world_state, - "user_instruction": instruction, - } - ) - - return { - "action_sequence": action_sequence, - } - - -@traceable(name="behavior_tree_generate_step") -async def behavior_tree_generate_step(state: PlanExecuteState): - """ - generate the behavior tree based on the instruction - """ - print(f"-----behavior_tree_generate_step-----") - - instruction = state["plan"][0] - if state["runtime_world_state"] is None: - runtime_world_state = state["world_state"][-1] - else: - runtime_world_state = state["runtime_world_state"] - objects = state["objects"] - - bt_skeleton = await rec_sk_gen_chain.ainvoke( - { - "objects": objects, - "runtime_world_state": runtime_world_state, - "last_behavior_tree": state["last_behavior_tree"], - "last_failed_node": state["last_failed_node"], - "summary": state["behavior_tree_execution_summary"], - "action_sequence": state["action_sequence"], - } - ) - - render_bt(bt_skeleton) - - pause = input("paused here") - - return { - "last_behavior_tree": bt_skeleton, - "BTExecutionHasSucceeded": False, - } - - -@traceable(name="behavior_tree_execute_step") -async def behavior_tree_execute_step(state: PlanExecuteState): - """ - execute the first step of the plan, append the result to the past steps - """ - print(f"-----behavior_tree_execute_step-----") - - this_step = state["plan"][0] - behavior_tree_skeleton = state["last_behavior_tree"] - latest_world_state = state["world_state"][-1] - - # * first sim run - tree_result, skeleton_json = behavior_tree_stewardship.sk_sim_run( - world_state=latest_world_state, skeleton_json=behavior_tree_skeleton - ) - - pprint(tree_result.to_json()) - pause = input("paused here") - - # * check result - if tree_result.result == "success": - - # * second fake run - tree_result, skeleton_json = behavior_tree_stewardship.sk_fake_run( - world_state=latest_world_state, skeleton_json=behavior_tree_skeleton - ) - - pprint(tree_result.to_json()) - pause = input("paused here") - - if tree_result.result == "success": - return { - "BTExecutionHasSucceeded": True, - "past_steps": ( - this_step, - tree_result.result, - ), # * only one because new plan will be generated and old steps are all removed - "world_state": [tree_result.world_state], - # * clear all - "last_behavior_tree": None, - "last_failed_node": None, - "runtime_world_state": None, - "behavior_tree_execution_summary": None, - } - else: - return { - "world_state": [tree_result.world_state], # * real world state - "last_behavior_tree": skeleton_json, - "last_failed_node": { - "summary": tree_result.final_node.get("summary"), - "name": tree_result.final_node.get("name"), - "identifier": tree_result.final_node.get("identifier"), - }, - "runtime_world_state": tree_result.world_state, - "behavior_tree_execution_summary": tree_result.summary, - } - - else: - return { - # ! do not change world state - "last_behavior_tree": skeleton_json, - "last_failed_node": { - "summary": tree_result.final_node.get("summary"), - "name": tree_result.final_node.get("name"), - "identifier": tree_result.final_node.get("identifier"), - }, - "runtime_world_state": tree_result.world_state, - "behavior_tree_execution_summary": tree_result.summary, - } - - -@traceable(name="planner_step") -async def planner_step(state: PlanExecuteState): - """ - plan the steps based on user input and world state - """ - print(f"-----plan_step-----") - - plan = await planner.ainvoke( - { - "user_input": state["user_input"], - "world_state": state["world_state"], - } - ) - return {"plan": plan.steps} - - -@traceable(name="plan_updater_step") -async def plan_updater_step(state: PlanExecuteState): - """ - if return a response, then success, response the use, end. - otherwise, return the updated newplan (normally the same as the old plan with the first step popped out. - """ - print(f"-----plan_updater_step-----") - - # ! BOOOM! BUG! - output = await plan_updater.ainvoke( - { - "user_input": state["user_input"], - "plan": state["plan"], - "world_state": state["world_state"], - "past_steps": state["past_steps"], - } - ) - # if isinstance( - # output, UpdaterResponse - # ): # * determine if it is time to response and end - # return { - # "response": "Your last instruction has been finished." - # } # * Don't need to update. - # else: - return {"plan": output.steps} # * update the plan - - -##################################################### * construct the graph - -workflow = StateGraph(PlanExecuteState) - -workflow.add_node("planner", planner_step) -workflow.add_node("sequence_generator", sequence_generate_step) -workflow.add_node("behavior_tree_generator", behavior_tree_generate_step) -workflow.add_node("behavior_tree_executor", behavior_tree_execute_step) -workflow.add_node("plan_updater", plan_updater_step) -workflow.add_node("user_input_node", user_input_step) -workflow.set_entry_point("user_input_node") -workflow.add_edge("planner", "sequence_generator") -workflow.add_edge("sequence_generator", "behavior_tree_generator") -workflow.add_edge("behavior_tree_generator", "behavior_tree_executor") - - -def executor_should_end(state: PlanExecuteState): - """ - end router - """ - print(f"-----executor_should_end-----") - - if state["BTExecutionHasSucceeded"] == True: - state["BTExecutionHasSucceeded"] = False - return True - else: - return False - - -workflow.add_conditional_edges( - "behavior_tree_executor", - executor_should_end, - { - True: "plan_updater", - False: "behavior_tree_generator", - }, -) - - -def plan_updater_should_end(state: PlanExecuteState): - """ - end router - """ - print(f"-----plan_updater_should_end-----") - - if state["plan"] == [] or len(state["plan"]) == 0: - return True - else: - return False - - -workflow.add_conditional_edges( - "plan_updater", - plan_updater_should_end, - { - True: "user_input_node", - False: "behavior_tree_generator", - }, -) - - -def user_input_should_end(state: PlanExecuteState): - """ - if the user input is empty, then end - """ - print(f"-----user_input_should_end-----") - - if not state["user_input"] or state["user_input"] == "": - return True - else: - return False - - -workflow.add_conditional_edges( - "user_input_node", - user_input_should_end, - { - True: END, - False: "planner", - }, -) - -app = workflow.compile() - -config = {"recursion_limit": 500} - -inputs = { - "world_state": [world_state_json], - "objects": objects, -} - - -async def core_run(): - async for event in app.astream( - inputs, - config=config, - ): - for k, v in event.items(): - if k != "__end__": - print(v) - - -if __name__ == "__main__": - # import asyncio - - # asyncio.run(core_run()) - pass diff --git a/experiments/gearset1/rec_sk_unit.py b/experiments/gearset1/rec_sk_unit.py deleted file mode 100644 index bcc923d6..00000000 --- a/experiments/gearset1/rec_sk_unit.py +++ /dev/null @@ -1,663 +0,0 @@ -import json -import os -from pprint import pprint -from typing import List, Tuple, Annotated, TypedDict -import operator - -os.environ["LANGCHAIN_TRACING_V2"] = "true" -os.environ["LANGCHAIN_ENDPOINT"] = "https://api.smith.langchain.com" -os.environ["LANGCHAIN_PROJECT"] = "kios_agent" - -from kios_bt.bt_stewardship import BehaviorTreeStewardship -from kios_scene.scene_factory import SceneFactory -from kios_bt.bt_factory import BehaviorTreeFactory -from kios_robot.robot_interface import RobotInterface -from kios_world.world_interface import WorldInterface - -from kios_agent.llm_supporter import KiosLLMSupporter -from kios_agent.data_types import KiosPromptSkeleton - -from kios_agent.kios_tools import ( - BehaviorTreeExecutorTool, - BehaviorTreeSimulatorTool, - WorldStateQueryTool, -) - -from dotenv import load_dotenv - -from langchain import hub - -from langchain_openai import ChatOpenAI - -from langchain.chains.openai_functions import ( - create_structured_output_runnable, - create_openai_fn_runnable, -) - -from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder -from langchain_core.output_parsers import JsonOutputParser, StrOutputParser -from langchain_core.pydantic_v1 import BaseModel, Field - -from langchain.prompts.pipeline import PipelinePromptTemplate -from langchain.prompts.prompt import PromptTemplate - -from langgraph.graph import StateGraph, END - -from langsmith import traceable - -load_dotenv() - -from kios_utils.pybt_test import generate_bt_stewardship, render_dot_tree -from kios_utils.pddl_problem_parser import parse_problem_init, parse_problem_objects - - -def render_bt(bt_json: json): - test_class = BehaviorTreeFactory() - bt = test_class.from_json_to_simple_bt(bt_json) - # bt = test_class.from_json_to_tree_root(bt_json) - bt_stewardship = generate_bt_stewardship(bt) - # bt_stewardship.setup(timeout=15) - render_dot_tree(bt_stewardship) - - -####################### dirs -current_dir = os.path.dirname(os.path.abspath(__file__)) -scene_path = os.path.join(current_dir, "scene.json") -# bt_json_file_path = os.path.join(current_dir, "behavior_tree.json") -world_state_path = os.path.join(current_dir, "world_state.json") -problem_path = os.path.join(current_dir, "gearset.problem") -domain_knowledge_path = os.path.join(current_dir, "domain_knowledge.txt") - -####################### problem -with open(problem_path, "r") as file: - problem = file.read() - -####################### scene -with open(scene_path, "r") as file: - scene_json_object = json.load(file) - -scene = SceneFactory().create_scene_from_json(scene_json_object) - -####################### world -world_interface = WorldInterface() -with open(world_state_path, "r") as file: - world_state_json_object = json.load(file) - world_interface.load_world_from_json(world_state_json_object) - -from_problem = parse_problem_init(problem=problem) -objects = parse_problem_objects(problem=problem) -world_interface.update_world(from_problem) - -world_state_json = world_interface.get_world_to_json() -pprint(from_problem) -pprint(world_state_json) -pprint(objects) - - -####################### robot -robot_interface = RobotInterface( - robot_address="127.0.0.1", - robot_port=12000, -) -robot_interface.setup_scene(scene) - -####################### bt_factory -bt_factory = BehaviorTreeFactory( - world_interface=world_interface, - robot_interface=robot_interface, -) - -####################### behavior_tree_stewardship -behavior_tree_stewardship = BehaviorTreeStewardship( - behaviortree_factory=bt_factory, - world_interface=world_interface, - robot_interface=robot_interface, -) - -####################### * tools -# bt_executor_tool = BehaviorTreeExecutorTool(behavior_tree_stewardship) - -bt_sim_tool = BehaviorTreeSimulatorTool(metadata={"bt_stw": behavior_tree_stewardship}) - -# world_state_query_tool = WorldStateQueryTool(behavior_tree_stewardship) - -executor_tools = [bt_sim_tool] - -# * kios data prompt skeleton dir -data_dir = os.environ.get("KIOS_DATA_DIR").format(username=os.getlogin()) -print(data_dir) -prompt_sk_dir = os.path.join(data_dir, "prompt_skeletons") -prompt_dir = os.path.join(data_dir, "prompts") - - -# * the graph state -class PlanExecuteState(TypedDict): - plan: List[str] - behavior_tree: dict - world_state: Annotated[ - List[dict], operator.add - ] # ! to add, you need to make world_state a list of dict - past_steps: Annotated[List[Tuple], operator.add] - user_input: str - objects: dict[str, list[str]] - last_behavior_tree: dict - last_failed_node: dict - runtime_world_state: dict - behavior_tree_execution_summary: str - BTExecutionHasSucceeded: bool - - action_sequence: List[str] - - -########## verifier - -template_file = os.path.join(prompt_dir, "seq_planner/template.txt") -task_file = os.path.join(prompt_dir, "seq_planner/task.txt") -system_file = os.path.join(prompt_dir, "seq_planner/system.txt") -domain_file = os.path.join(prompt_dir, "seq_planner/domain.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {template} - """ -) - -seq_planner_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ], -) - -seq_planner_chain = ( - seq_planner_ppt_ppl - | ChatOpenAI(model="gpt-4-turbo-preview", temperature=0) - | StrOutputParser() -) - - -# * output schema of the recursive_sk_generator -class SolutionBehaviortree(BaseModel): - """the fixed behavior tree""" - - behavior_tree: dict = Field( - description="the behavior tree that adheres the required format." - ) - - -behaviortree_file = os.path.join(prompt_dir, "rec_sk_gen/behaviortree.txt") -template_file = os.path.join(prompt_dir, "rec_sk_gen/template.txt") -task_file = os.path.join(prompt_dir, "rec_sk_gen/task.txt") -system_file = os.path.join(prompt_dir, "rec_sk_gen/system.txt") -object_file = os.path.join(prompt_dir, "rec_sk_gen/object.txt") -domain_file = os.path.join(prompt_dir, "rec_sk_gen/domain.txt") -example_file = os.path.join(prompt_dir, "rec_sk_gen/example.txt") -state_file = os.path.join(prompt_dir, "rec_sk_gen/state.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) -with open(object_file, "r") as f: - object_ppt = PromptTemplate.from_template(f.read()) -with open(behaviortree_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - behaviortree_ppt = ppt_tmp.partial(input=f.read()) - -with open(state_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - state_ppt = ppt_tmp.partial(input=f.read()) - -with open(example_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - example_ppt = ppt_tmp.partial(input=f.read()) -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {state} - - {object} - - {behaviortree} - - {example} - - {template} - """ -) - -re_sk_gen_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ("object", object_ppt), - ("behaviortree", behaviortree_ppt), - ("example", example_ppt), - ("state", state_ppt), - ], -) - -rec_sk_gen_chain = ( - re_sk_gen_ppt_ppl - | ChatOpenAI(model="gpt-4-turbo-preview", temperature=0) - | JsonOutputParser() -) - -# recursive_sk_generator = create_structured_output_runnable( -# SolutionBehaviortree, -# ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), -# re_sk_gen_ppt_ppl, -# mode="openai-json", -# ) - - -##################################################### * planner -# * output schema of the planner -class Plan(BaseModel): - """Plan to follow in future""" - - steps: List[str] = Field( - description="a list of different steps to follow, should be in sorted order" - ) - - -template_file = os.path.join(prompt_dir, "planner/template.txt") -task_file = os.path.join(prompt_dir, "planner/task.txt") -system_file = os.path.join(prompt_dir, "planner/system.txt") -domain_file = os.path.join(prompt_dir, "planner/domain.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {template} - """ -) - -planner_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ], -) - -planner = create_structured_output_runnable( - Plan, ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), planner_ppt_ppl -) - -##################################################### * plan_updaterner - -template_file = os.path.join(prompt_dir, "plan_updater/template.txt") -task_file = os.path.join(prompt_dir, "plan_updater/task.txt") -system_file = os.path.join(prompt_dir, "plan_updater/system.txt") -domain_file = os.path.join(prompt_dir, "plan_updater/domain.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {template} - """ -) - -plan_updater_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ], -) - -plan_updater = create_structured_output_runnable( - Plan, ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), plan_updater_ppt_ppl -) -# ! create openai fn runnable has bug - - -##################################################### * graph node functions -@traceable(name="user_input_node_step") -async def user_input_step(state: PlanExecuteState): - """ - get the input from the user - """ - print(f"-----user_input_step-----") - - user_input = input("Your next instruction: ") - - return { - "user_input": user_input, - } - - -@traceable(name="sequence_generate_step") -async def sequence_generate_step(state: PlanExecuteState): - """ - generate the sequence based on the instruction - """ - print(f"-----sequence_generate_step-----") - - instruction = state["plan"][0] - start_world_state = state["world_state"][-1] - objects = state["objects"] - - action_sequence = await seq_planner_chain.ainvoke( - { - "objects": objects, - "start_world_state": start_world_state, - "user_instruction": instruction, - } - ) - - return { - "action_sequence": action_sequence, - } - - -@traceable(name="behavior_tree_generate_step") -async def behavior_tree_generate_step(state: PlanExecuteState): - """ - generate the behavior tree based on the instruction - """ - print(f"-----behavior_tree_generate_step-----") - - instruction = state["plan"][0] - if state["runtime_world_state"] is None: - runtime_world_state = state["world_state"][-1] - else: - runtime_world_state = state["runtime_world_state"] - objects = state["objects"] - - bt_skeleton = await rec_sk_gen_chain.ainvoke( - { - "objects": objects, - "runtime_world_state": runtime_world_state, - "last_behavior_tree": state["last_behavior_tree"], - "last_failed_node": state["last_failed_node"], - "summary": state["behavior_tree_execution_summary"], - "action_sequence": state["action_sequence"], - } - ) - - render_bt(bt_skeleton) - - pause = input("paused here") - - return { - "last_behavior_tree": bt_skeleton, - "BTExecutionHasSucceeded": False, - } - - -@traceable(name="behavior_tree_execute_step") -async def behavior_tree_execute_step(state: PlanExecuteState): - """ - execute the first step of the plan, append the result to the past steps - """ - print(f"-----behavior_tree_execute_step-----") - - this_step = state["plan"][0] - behavior_tree_skeleton = state["last_behavior_tree"] - latest_world_state = state["world_state"][-1] - - # * first sim run - tree_result, skeleton_json = behavior_tree_stewardship.sk_sim_run( - world_state=latest_world_state, skeleton_json=behavior_tree_skeleton - ) - - pprint(tree_result.to_json()) - pause = input("paused here") - - # * check result - if tree_result.result == "success": - - # * second fake run - tree_result, skeleton_json = behavior_tree_stewardship.sk_fake_run( - world_state=latest_world_state, skeleton_json=behavior_tree_skeleton - ) - - pprint(tree_result.to_json()) - pause = input("paused here") - - if tree_result.result == "success": - return { - "BTExecutionHasSucceeded": True, - "past_steps": ( - this_step, - tree_result.result, - ), # * only one because new plan will be generated and old steps are all removed - "world_state": [tree_result.world_state], - # * clear all - "last_behavior_tree": None, - "last_failed_node": None, - "runtime_world_state": None, - "behavior_tree_execution_summary": None, - } - else: - return { - "world_state": [tree_result.world_state], # * real world state - "last_behavior_tree": skeleton_json, - "last_failed_node": { - "summary": tree_result.final_node.get("summary"), - "name": tree_result.final_node.get("name"), - "identifier": tree_result.final_node.get("identifier"), - }, - "runtime_world_state": tree_result.world_state, - "behavior_tree_execution_summary": tree_result.summary, - } - - else: - return { - # ! do not change world state - "last_behavior_tree": skeleton_json, - "last_failed_node": { - "summary": tree_result.final_node.get("summary"), - "name": tree_result.final_node.get("name"), - "identifier": tree_result.final_node.get("identifier"), - }, - "runtime_world_state": tree_result.world_state, - "behavior_tree_execution_summary": tree_result.summary, - } - - -@traceable(name="planner_step") -async def planner_step(state: PlanExecuteState): - """ - plan the steps based on user input and world state - """ - print(f"-----plan_step-----") - - plan = await planner.ainvoke( - { - "user_input": state["user_input"], - "world_state": state["world_state"], - } - ) - return {"plan": plan.steps} - - -@traceable(name="plan_updater_step") -async def plan_updater_step(state: PlanExecuteState): - """ - if return a response, then success, response the use, end. - otherwise, return the updated newplan (normally the same as the old plan with the first step popped out. - """ - print(f"-----plan_updater_step-----") - - # ! BOOOM! BUG! - output = await plan_updater.ainvoke( - { - "user_input": state["user_input"], - "plan": state["plan"], - "world_state": state["world_state"], - "past_steps": state["past_steps"], - } - ) - # if isinstance( - # output, UpdaterResponse - # ): # * determine if it is time to response and end - # return { - # "response": "Your last instruction has been finished." - # } # * Don't need to update. - # else: - return {"plan": output.steps} # * update the plan - - -##################################################### * construct the graph - -workflow = StateGraph(PlanExecuteState) - -workflow.add_node("planner", planner_step) -workflow.add_node("sequence_generator", sequence_generate_step) -workflow.add_node("behavior_tree_generator", behavior_tree_generate_step) -workflow.add_node("behavior_tree_executor", behavior_tree_execute_step) -workflow.add_node("plan_updater", plan_updater_step) -workflow.add_node("user_input_node", user_input_step) -workflow.set_entry_point("user_input_node") -workflow.add_edge("planner", "sequence_generator") -workflow.add_edge("sequence_generator", "behavior_tree_generator") -workflow.add_edge("behavior_tree_generator", "behavior_tree_executor") - - -def executor_should_end(state: PlanExecuteState): - """ - end router - """ - print(f"-----executor_should_end-----") - - if state["BTExecutionHasSucceeded"] == True: - state["BTExecutionHasSucceeded"] = False - return True - else: - return False - - -workflow.add_conditional_edges( - "behavior_tree_executor", - executor_should_end, - { - True: "plan_updater", - False: "behavior_tree_generator", - }, -) - - -def plan_updater_should_end(state: PlanExecuteState): - """ - end router - """ - print(f"-----plan_updater_should_end-----") - - if state["plan"] == [] or len(state["plan"]) == 0: - return True - else: - return False - - -workflow.add_conditional_edges( - "plan_updater", - plan_updater_should_end, - { - True: "user_input_node", - False: "behavior_tree_generator", - }, -) - - -def user_input_should_end(state: PlanExecuteState): - """ - if the user input is empty, then end - """ - print(f"-----user_input_should_end-----") - - if not state["user_input"] or state["user_input"] == "": - return True - else: - return False - - -workflow.add_conditional_edges( - "user_input_node", - user_input_should_end, - { - True: END, - False: "planner", - }, -) - -app = workflow.compile() - -config = {"recursion_limit": 500} - -inputs = { - "world_state": [world_state_json], - "objects": objects, -} - - -async def core_run(): - async for event in app.astream( - inputs, - config=config, - ): - for k, v in event.items(): - if k != "__end__": - print(v) - - -if __name__ == "__main__": - import asyncio - - asyncio.run(core_run()) diff --git a/experiments/gearset1/rec_vv.py b/experiments/gearset1/rec_vv.py deleted file mode 100644 index dfd72be2..00000000 --- a/experiments/gearset1/rec_vv.py +++ /dev/null @@ -1,737 +0,0 @@ -import json -import os -from pprint import pprint - -""" -iterative generation of behavior tree with a verifier and validator. -a good try but works not well. -""" - -os.environ["LANGCHAIN_TRACING_V2"] = "true" -os.environ["LANGCHAIN_ENDPOINT"] = "https://api.smith.langchain.com" -os.environ["LANGCHAIN_PROJECT"] = "kios_agent" - -from kios_bt.bt_stewardship import BehaviorTreeStewardship -from kios_scene.scene_factory import SceneFactory -from kios_bt.bt_factory import BehaviorTreeFactory -from kios_robot.robot_interface import RobotInterface -from kios_world.world_interface import WorldInterface - -from kios_agent.llm_supporter import KiosLLMSupporter -from kios_agent.data_types import KiosPromptSkeleton - -from kios_agent.kios_tools import ( - BehaviorTreeExecutorTool, - BehaviorTreeSimulatorTool, - WorldStateQueryTool, -) - -from typing import List, Tuple, Annotated, TypedDict -import operator - -from dotenv import load_dotenv - -from langchain import hub - -from langchain_openai import ChatOpenAI - -from langchain.chains.openai_functions import ( - create_structured_output_runnable, - create_openai_fn_runnable, -) -from langchain_core.utils.function_calling import convert_to_openai_function -from langchain_core.agents import AgentActionMessageLog, AgentFinish -from langchain_core.messages import HumanMessage -from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder -from langchain_core.output_parsers import JsonOutputParser, StrOutputParser -from langchain_core.pydantic_v1 import BaseModel, Field - -from langchain.agents.format_scratchpad import format_to_openai_function_messages -from langchain.agents import create_openai_functions_agent, AgentExecutor - -from langchain.prompts.pipeline import PipelinePromptTemplate -from langchain.prompts.prompt import PromptTemplate - -from langchain_community.document_loaders import TextLoader -from langchain_openai import OpenAIEmbeddings - -from langchain_community.vectorstores import FAISS -from langchain.text_splitter import ( - RecursiveCharacterTextSplitter, - CharacterTextSplitter, -) - -from langgraph.graph import StateGraph, END - -from langsmith import traceable - -load_dotenv() - -from kios_utils.pybt_test import generate_bt_stewardship, render_dot_tree -from kios_utils.pddl_problem_parser import parse_problem_init, parse_problem_objects - - -def render_bt(bt_json: json): - test_class = BehaviorTreeFactory() - bt = test_class.from_json_to_simple_bt(bt_json) - # bt = test_class.from_json_to_tree_root(bt_json) - bt_stewardship = generate_bt_stewardship(bt) - # bt_stewardship.setup(timeout=15) - render_dot_tree(bt_stewardship) - - -####################### dirs -current_dir = os.path.dirname(os.path.abspath(__file__)) -scene_path = os.path.join(current_dir, "scene.json") -# bt_json_file_path = os.path.join(current_dir, "behavior_tree.json") -world_state_path = os.path.join(current_dir, "world_state.json") -problem_path = os.path.join(current_dir, "gearset.problem") -domain_knowledge_path = os.path.join(current_dir, "domain_knowledge.txt") - -####################### problem -with open(problem_path, "r") as file: - problem = file.read() - -####################### scene -with open(scene_path, "r") as file: - scene_json_object = json.load(file) - -scene = SceneFactory().create_scene_from_json(scene_json_object) - -####################### world -world_interface = WorldInterface() -with open(world_state_path, "r") as file: - world_state_json_object = json.load(file) - world_interface.load_world_from_json(world_state_json_object) - -from_problem = parse_problem_init(problem=problem) -objects = parse_problem_objects(problem=problem) -world_interface.update_world(from_problem) - -world_state_json = world_interface.get_world_to_json() -pprint(from_problem) -pprint(world_state_json) -pprint(objects) - - -####################### robot -robot_interface = RobotInterface( - robot_address="127.0.0.1", - robot_port=12000, -) -robot_interface.setup_scene(scene) - -####################### bt_factory -bt_factory = BehaviorTreeFactory( - world_interface=world_interface, - robot_interface=robot_interface, -) - -####################### behavior_tree_stewardship -behavior_tree_stewardship = BehaviorTreeStewardship( - behaviortree_factory=bt_factory, - world_interface=world_interface, - robot_interface=robot_interface, -) - -####################### * tools -# bt_executor_tool = BehaviorTreeExecutorTool(behavior_tree_stewardship) - -bt_sim_tool = BehaviorTreeSimulatorTool(metadata={"bt_stw": behavior_tree_stewardship}) - -# world_state_query_tool = WorldStateQueryTool(behavior_tree_stewardship) - -executor_tools = [bt_sim_tool] - -# * kios data prompt skeleton dir -data_dir = os.environ.get("KIOS_DATA_DIR").format(username=os.getlogin()) -print(data_dir) -prompt_sk_dir = os.path.join(data_dir, "prompt_skeletons") -prompt_dir = os.path.join(data_dir, "prompts") - - -# * the graph state -class PlanExecuteState(TypedDict): - plan: List[str] - behavior_tree: dict - world_state: Annotated[ - List[dict], operator.add - ] # ! to add, you need to make world_state a list of dict - past_steps: Annotated[List[Tuple], operator.add] - user_input: str - objects: dict[str, list[str]] - last_behavior_tree: dict - last_failed_node: dict - runtime_world_state: dict - behavior_tree_execution_summary: str - BTExecutionHasSucceeded: bool - - -########## verifier - -behaviortree_file = os.path.join(prompt_dir, "vv/behaviortree.txt") -template_file = os.path.join(prompt_dir, "vv/template.txt") -task_file = os.path.join(prompt_dir, "vv/task.txt") -system_file = os.path.join(prompt_dir, "vv/system.txt") -object_file = os.path.join(prompt_dir, "vv/object.txt") -domain_file = os.path.join(prompt_dir, "vv/domain.txt") -state_file = os.path.join(prompt_dir, "vv/state.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) -with open(object_file, "r") as f: - object_ppt = PromptTemplate.from_template(f.read()) -with open(behaviortree_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - behaviortree_ppt = ppt_tmp.partial(input=f.read()) - -with open(state_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - state_ppt = ppt_tmp.partial(input=f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {state} - - {object} - - {behaviortree} - - {template} - """ -) - -verifier_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ("object", object_ppt), - ("behaviortree", behaviortree_ppt), - ("state", state_ppt), - ], -) - -verifier_chain = ( - verifier_ppt_ppl - | ChatOpenAI(model="gpt-4-turbo-preview", temperature=0) - | StrOutputParser() -) - - -# * output schema of the recursive_sk_generator -class SolutionBehaviortree(BaseModel): - """the fixed behavior tree""" - - behavior_tree: dict = Field( - description="the behavior tree that adheres the required format." - ) - - -behaviortree_file = os.path.join(prompt_dir, "rec_sk_gen/behaviortree.txt") -template_file = os.path.join(prompt_dir, "rec_sk_gen/template.txt") -task_file = os.path.join(prompt_dir, "rec_sk_gen/task.txt") -system_file = os.path.join(prompt_dir, "rec_sk_gen/system.txt") -object_file = os.path.join(prompt_dir, "rec_sk_gen/object.txt") -domain_file = os.path.join(prompt_dir, "rec_sk_gen/domain.txt") -example_file = os.path.join(prompt_dir, "rec_sk_gen/example.txt") -state_file = os.path.join(prompt_dir, "rec_sk_gen/state.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) -with open(object_file, "r") as f: - object_ppt = PromptTemplate.from_template(f.read()) -with open(behaviortree_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - behaviortree_ppt = ppt_tmp.partial(input=f.read()) - -with open(state_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - state_ppt = ppt_tmp.partial(input=f.read()) - -with open(example_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - example_ppt = ppt_tmp.partial(input=f.read()) -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {state} - - {object} - - {behaviortree} - - {example} - - {template} - """ -) - -re_sk_gen_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ("object", object_ppt), - ("behaviortree", behaviortree_ppt), - ("example", example_ppt), - ("state", state_ppt), - ], -) - -rec_sk_gen_chain = ( - re_sk_gen_ppt_ppl - | ChatOpenAI(model="gpt-4-turbo-preview", temperature=0) - | JsonOutputParser() -) - -# recursive_sk_generator = create_structured_output_runnable( -# SolutionBehaviortree, -# ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), -# re_sk_gen_ppt_ppl, -# mode="openai-json", -# ) - - -##################################################### * planner -# * output schema of the planner -class Plan(BaseModel): - """Plan to follow in future""" - - steps: List[str] = Field( - description="a list of different steps to follow, should be in sorted order" - ) - - -template_file = os.path.join(prompt_dir, "planner/template.txt") -task_file = os.path.join(prompt_dir, "planner/task.txt") -system_file = os.path.join(prompt_dir, "planner/system.txt") -domain_file = os.path.join(prompt_dir, "planner/domain.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {template} - """ -) - -planner_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ], -) - -planner = create_structured_output_runnable( - Plan, ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), planner_ppt_ppl -) - -##################################################### * plan_updaterner - -template_file = os.path.join(prompt_dir, "plan_updater/template.txt") -task_file = os.path.join(prompt_dir, "plan_updater/task.txt") -system_file = os.path.join(prompt_dir, "plan_updater/system.txt") -domain_file = os.path.join(prompt_dir, "plan_updater/domain.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {template} - """ -) - -plan_updater_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ], -) - -plan_updater = create_structured_output_runnable( - Plan, ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), plan_updater_ppt_ppl -) -# ! create openai fn runnable has bug - - -##################################################### * graph node functions -@traceable(name="user_input_node_step") -async def user_input_step(state: PlanExecuteState): - """ - get the input from the user - """ - print(f"-----user_input_step-----") - - user_input = input("Your next instruction: ") - - return { - "user_input": user_input, - } - - -@traceable(name="behavior_tree_generate_step") -async def behavior_tree_generate_step(state: PlanExecuteState): - """ - generate the behavior tree based on the instruction - """ - print(f"-----behavior_tree_generate_step-----") - - instruction = state["plan"][0] - if state["runtime_world_state"] is None: - runtime_world_state = state["world_state"][-1] - else: - runtime_world_state = state["runtime_world_state"] - objects = state["objects"] - - bt_skeleton = await rec_sk_gen_chain.ainvoke( - { - "objects": objects, - "runtime_world_state": runtime_world_state, - "last_behavior_tree": state["last_behavior_tree"], - "last_failed_node": state["last_failed_node"], - "summary": state["behavior_tree_execution_summary"], - "instructions": instruction, - } - ) - - render_bt(bt_skeleton) - - pause = input("paused here") - - return { - "last_behavior_tree": bt_skeleton, - "BTExecutionHasSucceeded": False, - } - - -@traceable(name="behavior_tree_execute_step") -async def behavior_tree_execute_step(state: PlanExecuteState): - """ - execute the first step of the plan, append the result to the past steps - """ - print(f"-----behavior_tree_execute_step-----") - - this_step = state["plan"][0] - behavior_tree_skeleton = state["last_behavior_tree"] - latest_world_state = state["world_state"][-1] - - # * first sim run - tree_result, skeleton_json = behavior_tree_stewardship.sk_sim_run( - world_state=latest_world_state, skeleton_json=behavior_tree_skeleton - ) - - pprint(tree_result.to_json()) - pause = input("paused here") - - # * check result - if tree_result.result == "success": - - # * second fake run - tree_result, skeleton_json = behavior_tree_stewardship.sk_fake_run( - world_state=latest_world_state, skeleton_json=behavior_tree_skeleton - ) - - pprint(tree_result.to_json()) - pause = input("paused here") - - if tree_result.result == "success": - return { - "BTExecutionHasSucceeded": True, - "past_steps": ( - this_step, - tree_result.result, - ), # * only one because new plan will be generated and old steps are all removed - "world_state": [tree_result.world_state], - # * clear all - "last_behavior_tree": None, - "last_failed_node": None, - "runtime_world_state": None, - "behavior_tree_execution_summary": None, - } - else: - return { - "world_state": [tree_result.world_state], # * real world state - "last_behavior_tree": skeleton_json, - "last_failed_node": { - "summary": tree_result.final_node.get("summary"), - "name": tree_result.final_node.get("name"), - "identifier": tree_result.final_node.get("identifier"), - }, - "runtime_world_state": tree_result.world_state, - "behavior_tree_execution_summary": tree_result.summary, - } - - else: - return { - # ! do not change world state - "last_behavior_tree": skeleton_json, - "last_failed_node": { - "summary": tree_result.final_node.get("summary"), - "name": tree_result.final_node.get("name"), - "identifier": tree_result.final_node.get("identifier"), - }, - "runtime_world_state": tree_result.world_state, - "behavior_tree_execution_summary": tree_result.summary, - } - - -@traceable(name="verifier_step") -async def verifier_step(state: PlanExecuteState): - """ - verify the behavior tree - """ - print(f"-----verifier_step-----") - start_world_state = state["world_state"][-1] - - # * get the objects - objects = state["objects"] - - # * get the last behavior tree - last_behavior_tree = state["last_behavior_tree"] - - # * get the last failed node - last_failed_node = state["last_failed_node"] - - # * get the runtime world state - runtime_world_state = state["runtime_world_state"] - - # * get the behavior tree execution summary - behavior_tree_execution_summary = state["behavior_tree_execution_summary"] - - # * get the verifier response - verifier_response = await verifier_chain.ainvoke( - { - "objects": objects, - "start_world_state": start_world_state, - "last_behavior_tree": last_behavior_tree, - "last_failed_node": last_failed_node, - "runtime_world_state": runtime_world_state, - "summary": behavior_tree_execution_summary, - } - ) - - # * return the verifier response - return { - "behavior_tree_execution_summary": verifier_response, - } - - -@traceable(name="planner_step") -async def planner_step(state: PlanExecuteState): - """ - plan the steps based on user input and world state - """ - print(f"-----plan_step-----") - - plan = await planner.ainvoke( - { - "user_input": state["user_input"], - "world_state": state["world_state"], - } - ) - return {"plan": plan.steps} - - -@traceable(name="plan_updater_step") -async def plan_updater_step(state: PlanExecuteState): - """ - if return a response, then success, response the use, end. - otherwise, return the updated newplan (normally the same as the old plan with the first step popped out. - """ - print(f"-----plan_updater_step-----") - - # ! BOOOM! BUG! - output = await plan_updater.ainvoke( - { - "user_input": state["user_input"], - "plan": state["plan"], - "world_state": state["world_state"], - "past_steps": state["past_steps"], - } - ) - # if isinstance( - # output, UpdaterResponse - # ): # * determine if it is time to response and end - # return { - # "response": "Your last instruction has been finished." - # } # * Don't need to update. - # else: - return {"plan": output.steps} # * update the plan - - -##################################################### * construct the graph - -workflow = StateGraph(PlanExecuteState) - -workflow.add_node("planner", planner_step) - -workflow.add_node("behavior_tree_generator", behavior_tree_generate_step) - -workflow.add_node("behavior_tree_executor", behavior_tree_execute_step) - -workflow.add_node("verifier", verifier_step) - -workflow.add_node("plan_updater", plan_updater_step) - -workflow.add_node("user_input_node", user_input_step) - -workflow.set_entry_point("user_input_node") - -workflow.add_edge( - "planner", "behavior_tree_generator" -) # ! may need to add a condition here - -workflow.add_edge("behavior_tree_generator", "behavior_tree_executor") - -workflow.add_edge("verifier", "behavior_tree_generator") - - -def executor_should_end(state: PlanExecuteState): - """ - end router - """ - print(f"-----executor_should_end-----") - - if state["BTExecutionHasSucceeded"] == True: - state["BTExecutionHasSucceeded"] = False - return True - else: - return False - - -workflow.add_conditional_edges( - "behavior_tree_executor", - executor_should_end, - { - True: "plan_updater", - False: "verifier", - }, -) - - -def plan_updater_should_end(state: PlanExecuteState): - """ - end router - """ - print(f"-----plan_updater_should_end-----") - - # if state["response"]: - # return True - # else: - # return False - if state["plan"] == [] or len(state["plan"]) == 0: - return True - else: - return False - - -workflow.add_conditional_edges( - "plan_updater", - plan_updater_should_end, - { - True: "user_input_node", - False: "behavior_tree_generator", - }, -) - - -def user_input_should_end(state: PlanExecuteState): - """ - if the user input is empty, then end - """ - print(f"-----user_input_should_end-----") - - # print(f'user_input: {state["user_input"]}') - # print(f'world_state: {state["world_state"]}') - - if not state["user_input"] or state["user_input"] == "": - return True - else: - return False - - -workflow.add_conditional_edges( - "user_input_node", - user_input_should_end, - { - True: END, - False: "planner", - }, -) - -app = workflow.compile() - -config = {"recursion_limit": 500} - -inputs = { - "world_state": [world_state_json], - "objects": objects, -} - - -async def core_run(): - async for event in app.astream( - inputs, - config=config, - ): - for k, v in event.items(): - if k != "__end__": - print(v) - - -if __name__ == "__main__": - import asyncio - - asyncio.run(core_run()) diff --git a/experiments/gearset1/recursive_generation.py b/experiments/gearset1/recursive_generation.py deleted file mode 100644 index dc8e0e93..00000000 --- a/experiments/gearset1/recursive_generation.py +++ /dev/null @@ -1,1207 +0,0 @@ -import json -import os -import re -from pprint import pprint -from typing import List, Tuple, Annotated, TypedDict -import operator -import logging -import asyncio - -""" -node expanding method script. -run the algo to generate the tree for current goal, forecast the future world state, and find the new goal to expand. -currently not a langgraph. -""" - -os.environ["LANGCHAIN_TRACING_V2"] = "true" -os.environ["LANGCHAIN_ENDPOINT"] = "https://api.smith.langchain.com" -os.environ["LANGCHAIN_PROJECT"] = "kios_agent" - -from kios_bt.bt_stewardship import BehaviorTreeStewardship -from kios_scene.scene_factory import SceneFactory -from kios_bt.bt_factory import BehaviorTreeFactory -from kios_robot.robot_interface import RobotInterface -from kios_world.world_interface import WorldInterface - -from kios_agent.llm_supporter import KiosLLMSupporter -from kios_agent.data_types import KiosPromptSkeleton - -from kios_agent.kios_tools import ( - BehaviorTreeExecutorTool, - BehaviorTreeSimulatorTool, - WorldStateQueryTool, -) - -from dotenv import load_dotenv - -from langchain import hub - -from langchain_openai import ChatOpenAI - -from langchain.chains.openai_functions import ( - create_structured_output_runnable, - create_openai_fn_runnable, -) -from langchain_core.utils.function_calling import convert_to_openai_function -from langchain_core.agents import AgentActionMessageLog, AgentFinish -from langchain_core.messages import HumanMessage -from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder -from langchain_core.output_parsers import JsonOutputParser, StrOutputParser -from langchain_core.pydantic_v1 import BaseModel, Field - -from langchain.agents.format_scratchpad import format_to_openai_function_messages -from langchain.agents import create_openai_functions_agent, AgentExecutor - -from langchain.prompts.pipeline import PipelinePromptTemplate -from langchain.prompts.prompt import PromptTemplate - -from langchain_community.document_loaders import TextLoader -from langchain_openai import OpenAIEmbeddings - -from langchain_community.vectorstores import FAISS -from langchain.text_splitter import ( - RecursiveCharacterTextSplitter, - CharacterTextSplitter, -) - -from langgraph.graph import StateGraph, END - -from langsmith import traceable - -load_dotenv() - -from kios_utils.pybt_test import generate_bt_stewardship, render_dot_tree -from kios_utils.pddl_problem_parser import parse_problem_init, parse_problem_objects - - -def render_bt(bt_json: dict): - test_class = BehaviorTreeFactory() - bt = test_class.from_json_to_simple_bt(bt_json) - # bt = test_class.from_json_to_tree_root(bt_json) - bt_stewardship = generate_bt_stewardship(bt) - # bt_stewardship.setup(timeout=15) - render_dot_tree(bt_stewardship) - - -####################### dirs -current_dir = os.path.dirname(os.path.abspath(__file__)) -scene_path = os.path.join(current_dir, "scene.json") -# bt_json_file_path = os.path.join(current_dir, "behavior_tree.json") -world_state_path = os.path.join(current_dir, "world_state.json") -problem_path = os.path.join(current_dir, "gearset.problem") -domain_knowledge_path = os.path.join(current_dir, "domain_knowledge.txt") - -####################### problem -with open(problem_path, "r") as file: - problem = file.read() - -####################### scene -with open(scene_path, "r") as file: - scene_json_object = json.load(file) - -scene = SceneFactory().create_scene_from_json(scene_json_object) - -####################### world -world_interface = WorldInterface() -with open(world_state_path, "r") as file: - world_state_json_object = json.load(file) - world_interface.load_world_from_json(world_state_json_object) - -from_problem = parse_problem_init(problem=problem) -objects = parse_problem_objects(problem=problem) -world_interface.update_world(from_problem) - -world_state_json = world_interface.get_world_to_json() -pprint(from_problem) -pprint(world_state_json) -pprint(objects) - - -####################### robot -robot_interface = RobotInterface( - robot_address="127.0.0.1", - robot_port=12000, -) -robot_interface.setup_scene(scene) - -####################### bt_factory -bt_factory = BehaviorTreeFactory( - world_interface=world_interface, - robot_interface=robot_interface, -) - -####################### behavior_tree_stewardship -behavior_tree_stewardship = BehaviorTreeStewardship( - behaviortree_factory=bt_factory, - world_interface=world_interface, - robot_interface=robot_interface, -) - -####################### * tools -# bt_executor_tool = BehaviorTreeExecutorTool(behavior_tree_stewardship) - -bt_sim_tool = BehaviorTreeSimulatorTool(metadata={"bt_stw": behavior_tree_stewardship}) - -# world_state_query_tool = WorldStateQueryTool(behavior_tree_stewardship) - -executor_tools = [bt_sim_tool] - -# * kios data prompt skeleton dir -data_dir = os.environ.get("KIOS_DATA_DIR").format(username=os.getlogin()) -print(data_dir) -prompt_sk_dir = os.path.join(data_dir, "prompt_skeletons") -prompt_dir = os.path.join(data_dir, "prompts") - - -# * the graph state -class PlanExecuteState(TypedDict): - plan: List[str] - behavior_tree: dict - world_state: Annotated[ - List[dict], operator.add - ] # ! to add, you need to make world_state a list of dict - past_steps: Annotated[List[Tuple], operator.add] - user_input: str - objects: dict[str, list[str]] - last_behavior_tree: dict - last_failed_node: dict - runtime_world_state: dict - behavior_tree_execution_summary: str - BTExecutionHasSucceeded: bool - - action_sequence: List[str] - - -########## verifier - -template_file = os.path.join(prompt_dir, "seq_planner/template.txt") -task_file = os.path.join(prompt_dir, "seq_planner/task.txt") -system_file = os.path.join(prompt_dir, "seq_planner/system.txt") -domain_file = os.path.join(prompt_dir, "seq_planner/domain.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {template} - """ -) - -seq_planner_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ], -) - -seq_planner_chain = ( - seq_planner_ppt_ppl - | ChatOpenAI(model="gpt-4-turbo-preview", temperature=0) - | StrOutputParser() -) - - -# * output schema of the recursive_sk_generator -class SolutionBehaviortree(BaseModel): - """the fixed behavior tree""" - - behavior_tree: dict = Field( - description="the behavior tree that adheres the required format." - ) - - -behaviortree_file = os.path.join(prompt_dir, "rec_sk_gen/behaviortree.txt") -template_file = os.path.join(prompt_dir, "rec_sk_gen/template.txt") -task_file = os.path.join(prompt_dir, "rec_sk_gen/task.txt") -system_file = os.path.join(prompt_dir, "rec_sk_gen/system.txt") -object_file = os.path.join(prompt_dir, "rec_sk_gen/object.txt") -domain_file = os.path.join(prompt_dir, "rec_sk_gen/domain.txt") -example_file = os.path.join(prompt_dir, "rec_sk_gen/example.txt") -state_file = os.path.join(prompt_dir, "rec_sk_gen/state.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) -with open(object_file, "r") as f: - object_ppt = PromptTemplate.from_template(f.read()) -with open(behaviortree_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - behaviortree_ppt = ppt_tmp.partial(input=f.read()) - -with open(state_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - state_ppt = ppt_tmp.partial(input=f.read()) - -with open(example_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - example_ppt = ppt_tmp.partial(input=f.read()) -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {state} - - {object} - - {behaviortree} - - {example} - - {template} - """ -) - -re_sk_gen_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ("object", object_ppt), - ("behaviortree", behaviortree_ppt), - ("example", example_ppt), - ("state", state_ppt), - ], -) - -rec_sk_gen_chain = ( - re_sk_gen_ppt_ppl - | ChatOpenAI(model="gpt-4-turbo-preview", temperature=0) - | JsonOutputParser() -) - -# recursive_sk_generator = create_structured_output_runnable( -# SolutionBehaviortree, -# ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), -# re_sk_gen_ppt_ppl, -# mode="openai-json", -# ) - - -##################################################### * planner -# * output schema of the planner -class Plan(BaseModel): - """Plan to follow in future""" - - steps: List[str] = Field( - description="a list of different steps to follow, should be in sorted order" - ) - - -template_file = os.path.join(prompt_dir, "planner/template.txt") -task_file = os.path.join(prompt_dir, "planner/task.txt") -system_file = os.path.join(prompt_dir, "planner/system.txt") -domain_file = os.path.join(prompt_dir, "planner/domain.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {template} - """ -) - -planner_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ], -) - -planner = create_structured_output_runnable( - Plan, ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), planner_ppt_ppl -) - -##################################################### * plan_updaterner - -template_file = os.path.join(prompt_dir, "plan_updater/template.txt") -task_file = os.path.join(prompt_dir, "plan_updater/task.txt") -system_file = os.path.join(prompt_dir, "plan_updater/system.txt") -domain_file = os.path.join(prompt_dir, "plan_updater/domain.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {template} - """ -) - -plan_updater_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ], -) - -plan_updater = create_structured_output_runnable( - Plan, ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), plan_updater_ppt_ppl -) -# ! create openai fn runnable has bug - - -##################################################### * graph node functions -@traceable(name="user_input_node_step") -async def user_input_step(state: PlanExecuteState): - """ - get the input from the user - """ - print(f"-----user_input_step-----") - - user_input = input("Your next instruction: ") - - return { - "user_input": user_input, - } - - -@traceable(name="sequence_generate_step") -async def sequence_generate_step(state: PlanExecuteState): - """ - generate the sequence based on the instruction - """ - print(f"-----sequence_generate_step-----") - - instruction = state["plan"][0] - start_world_state = state["world_state"][-1] - objects = state["objects"] - - action_sequence = await seq_planner_chain.ainvoke( - { - "objects": objects, - "start_world_state": start_world_state, - "user_instruction": instruction, - } - ) - - return { - "action_sequence": action_sequence, - } - - -@traceable(name="behavior_tree_generate_step") -async def behavior_tree_generate_step(state: PlanExecuteState): - """ - generate the behavior tree based on the instruction - """ - print(f"-----behavior_tree_generate_step-----") - - instruction = state["plan"][0] - if state["runtime_world_state"] is None: - runtime_world_state = state["world_state"][-1] - else: - runtime_world_state = state["runtime_world_state"] - objects = state["objects"] - - bt_skeleton = await rec_sk_gen_chain.ainvoke( - { - "objects": objects, - "runtime_world_state": runtime_world_state, - "last_behavior_tree": state["last_behavior_tree"], - "last_failed_node": state["last_failed_node"], - "summary": state["behavior_tree_execution_summary"], - "action_sequence": state["action_sequence"], - } - ) - - render_bt(bt_skeleton) - - pause = input("paused here") - - return { - "last_behavior_tree": bt_skeleton, - "BTExecutionHasSucceeded": False, - } - - -@traceable(name="behavior_tree_execute_step") -async def behavior_tree_execute_step(state: PlanExecuteState): - """ - execute the first step of the plan, append the result to the past steps - """ - print(f"-----behavior_tree_execute_step-----") - - this_step = state["plan"][0] - behavior_tree_skeleton = state["last_behavior_tree"] - latest_world_state = state["world_state"][-1] - - # * first sim run - tree_result, skeleton_json = behavior_tree_stewardship.sk_sim_run( - world_state=latest_world_state, skeleton_json=behavior_tree_skeleton - ) - - pprint(tree_result.to_json()) - pause = input("paused here") - - # * check result - if tree_result.result == "success": - - # * second fake run - tree_result, skeleton_json = behavior_tree_stewardship.sk_fake_run( - world_state=latest_world_state, skeleton_json=behavior_tree_skeleton - ) - - pprint(tree_result.to_json()) - pause = input("paused here") - - if tree_result.result == "success": - return { - "BTExecutionHasSucceeded": True, - "past_steps": ( - this_step, - tree_result.result, - ), # * only one because new plan will be generated and old steps are all removed - "world_state": [tree_result.world_state], - # * clear all - "last_behavior_tree": None, - "last_failed_node": None, - "runtime_world_state": None, - "behavior_tree_execution_summary": None, - } - else: - return { - "world_state": [tree_result.world_state], # * real world state - "last_behavior_tree": skeleton_json, - "last_failed_node": { - "summary": tree_result.final_node.get("summary"), - "name": tree_result.final_node.get("name"), - "identifier": tree_result.final_node.get("identifier"), - }, - "runtime_world_state": tree_result.world_state, - "behavior_tree_execution_summary": tree_result.summary, - } - - else: - return { - # ! do not change world state - "last_behavior_tree": skeleton_json, - "last_failed_node": { - "summary": tree_result.final_node.get("summary"), - "name": tree_result.final_node.get("name"), - "identifier": tree_result.final_node.get("identifier"), - }, - "runtime_world_state": tree_result.world_state, - "behavior_tree_execution_summary": tree_result.summary, - } - - -@traceable(name="planner_step") -async def planner_step(state: PlanExecuteState): - """ - plan the steps based on user input and world state - """ - print(f"-----plan_step-----") - - plan = await planner.ainvoke( - { - "user_input": state["user_input"], - "world_state": state["world_state"], - } - ) - return {"plan": plan.steps} - - -@traceable(name="plan_updater_step") -async def plan_updater_step(state: PlanExecuteState): - """ - if return a response, then success, response the use, end. - otherwise, return the updated newplan (normally the same as the old plan with the first step popped out. - """ - print(f"-----plan_updater_step-----") - - # ! BOOOM! BUG! - output = await plan_updater.ainvoke( - { - "user_input": state["user_input"], - "plan": state["plan"], - "world_state": state["world_state"], - "past_steps": state["past_steps"], - } - ) - # if isinstance( - # output, UpdaterResponse - # ): # * determine if it is time to response and end - # return { - # "response": "Your last instruction has been finished." - # } # * Don't need to update. - # else: - return {"plan": output.steps} # * update the plan - - -##################################################### * construct the graph - -workflow = StateGraph(PlanExecuteState) - -workflow.add_node("planner", planner_step) -workflow.add_node("sequence_generator", sequence_generate_step) -workflow.add_node("behavior_tree_generator", behavior_tree_generate_step) -workflow.add_node("behavior_tree_executor", behavior_tree_execute_step) -workflow.add_node("plan_updater", plan_updater_step) -workflow.add_node("user_input_node", user_input_step) -workflow.set_entry_point("user_input_node") -workflow.add_edge("planner", "sequence_generator") -workflow.add_edge("sequence_generator", "behavior_tree_generator") -workflow.add_edge("behavior_tree_generator", "behavior_tree_executor") - - -def executor_should_end(state: PlanExecuteState): - """ - end router - """ - print(f"-----executor_should_end-----") - - if state["BTExecutionHasSucceeded"] == True: - state["BTExecutionHasSucceeded"] = False - return True - else: - return False - - -workflow.add_conditional_edges( - "behavior_tree_executor", - executor_should_end, - { - True: "plan_updater", - False: "behavior_tree_generator", - }, -) - - -def plan_updater_should_end(state: PlanExecuteState): - """ - end router - """ - print(f"-----plan_updater_should_end-----") - - if state["plan"] == [] or len(state["plan"]) == 0: - return True - else: - return False - - -workflow.add_conditional_edges( - "plan_updater", - plan_updater_should_end, - { - True: "user_input_node", - False: "behavior_tree_generator", - }, -) - - -def user_input_should_end(state: PlanExecuteState): - """ - if the user input is empty, then end - """ - print(f"-----user_input_should_end-----") - - if not state["user_input"] or state["user_input"] == "": - return True - else: - return False - - -workflow.add_conditional_edges( - "user_input_node", - user_input_should_end, - { - True: END, - False: "planner", - }, -) - -app = workflow.compile() - -config = {"recursion_limit": 500} - -inputs = { - "world_state": [world_state_json], - "objects": objects, -} - -# * unit tree generator ppl -system_file = os.path.join(prompt_dir, "new/ut_gen/system.txt") -task_file = os.path.join(prompt_dir, "new/ut_gen/task.txt") -domain_file = os.path.join(prompt_dir, "new/ut_gen/new_domain_nl.txt") -behaviortree_file = os.path.join(prompt_dir, "new/ut_gen/new_behaviortree.txt") -template_file = os.path.join(prompt_dir, "new/ut_gen/template.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) -with open(behaviortree_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - behaviortree_ppt = ppt_tmp.partial(input=f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {behaviortree} - - {template} - - {format_instructions} - """ -) - -parser = JsonOutputParser() - -format_instructions = PromptTemplate.from_template("""{input}""").partial( - input=parser.get_format_instructions() -) - -ut_gen_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ("behaviortree", behaviortree_ppt), - ("format_instructions", format_instructions), - ], -) - -ut_gen_chain = ( - ut_gen_ppt_ppl - # | ChatOpenAI(model="gpt-3.5-turbo-0125", temperature=0) - # | ChatOpenAI(model="ft:gpt-3.5-turbo-0125:kifabrik-mirmi::8y1cXwVw", temperature=0) - | ChatOpenAI( - model="ft:gpt-3.5-turbo-0125:kifabrik-mirmi:kios-ut-gen-v2:8z2KbPsr", - temperature=0, - ) - # | ChatOpenAI(model="gpt-4", temperature=0) - | JsonOutputParser() -) - -# ! sequence planner estimation ppl -system_file = os.path.join(prompt_dir, "seq_planner_est/system.txt") -task_file = os.path.join(prompt_dir, "seq_planner_est/task.txt") -domain_file = os.path.join(prompt_dir, "seq_planner_est/domain.txt") -state_file = os.path.join(prompt_dir, "seq_planner_est/state.txt") -output_format_file = os.path.join(prompt_dir, "seq_planner_est/output_format.txt") -template_file = os.path.join(prompt_dir, "seq_planner_est/template.txt") -example_file = os.path.join(prompt_dir, "seq_planner_est/example.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) -with open(output_format_file, "r") as f: - output_format_ppt = PromptTemplate.from_template(f.read()) -with open(example_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - example_ppt = ppt_tmp.partial(input=f.read()) -with open(state_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - state_ppt = ppt_tmp.partial(input=f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {state} - - {output_format} - - {example} - - {template} - - {format_instructions} - """ -) - -parser = JsonOutputParser() - -format_instructions = PromptTemplate.from_template("""{input}""").partial( - input=parser.get_format_instructions() -) - -seq_planner_est_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ("state", state_ppt), - ("output_format", output_format_ppt), - ("format_instructions", format_instructions), - ("example", example_ppt), - ], -) - -seq_planner_est_chain = ( - seq_planner_est_ppt_ppl - # | ChatOpenAI(model="gpt-4", temperature=0) - | ChatOpenAI(model="gpt-3.5-turbo-0125", temperature=0) - | JsonOutputParser() -) - - -def seq_planner_est_test(): - return seq_planner_est_chain.invoke( - { - "start_world_state": world_state_json, - # "target": "is_inserted_to(shaft1, gearbase_hole1)", - "target": "hold(left_hand, outward_claw)", - } - ) - - -# * sequence action planner -system_file = os.path.join(prompt_dir, "new/seq_plan/system.txt") -task_file = os.path.join(prompt_dir, "new/seq_plan/task.txt") -domain_file = os.path.join(prompt_dir, "new/seq_plan/new_domain_nl.txt") -state_file = os.path.join(prompt_dir, "new/seq_plan/state.txt") -output_format_file = os.path.join(prompt_dir, "new/seq_plan/output_format.txt") -template_file = os.path.join(prompt_dir, "new/seq_plan/template.txt") -example_file = os.path.join(prompt_dir, "new/seq_plan/new_example.txt") -chain_file = os.path.join(prompt_dir, "new/seq_plan/chain.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) -with open(output_format_file, "r") as f: - output_format_ppt = PromptTemplate.from_template(f.read()) -with open(example_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - example_ppt = ppt_tmp.partial(input=f.read()) -with open(state_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - state_ppt = ppt_tmp.partial(input=f.read()) -# # * this is for gpt 3.5 -# with open(chain_file, "r") as f: -# chain_ppt = PromptTemplate.from_template(f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {state} - - {output_format} - - {example} - - {template} - - {format_instructions} - """ -) - -parser = JsonOutputParser() - -format_instructions = PromptTemplate.from_template("""{input}""").partial( - input=parser.get_format_instructions() -) - -seq_ac_pl_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ("state", state_ppt), - ("output_format", output_format_ppt), - ("format_instructions", format_instructions), - ("example", example_ppt), - # ("chain", chain_ppt), - ], -) - -seq_ac_pl_chain = ( - seq_ac_pl_ppt_ppl - | ChatOpenAI(model="gpt-4", temperature=0) - # | ChatOpenAI(model="gpt-3.5-turbo-0125", temperature=0) - | JsonOutputParser() -) - -# * state estimater -system_file = os.path.join(prompt_dir, "new/state_est/system.txt") -task_file = os.path.join(prompt_dir, "new/state_est/task.txt") -domain_file = os.path.join(prompt_dir, "new/state_est/new_domain_nl.txt") -state_file = os.path.join(prompt_dir, "new/state_est/state.txt") -output_format_file = os.path.join(prompt_dir, "new/state_est/output_format.txt") -template_file = os.path.join(prompt_dir, "new/state_est/template.txt") -example_file = os.path.join(prompt_dir, "new/state_est/new_example.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) -with open(output_format_file, "r") as f: - output_format_ppt = PromptTemplate.from_template(f.read()) -with open(example_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - example_ppt = ppt_tmp.partial(input=f.read()) -with open(state_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - state_ppt = ppt_tmp.partial(input=f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {state} - - {output_format} - - {example} - - {template} - - {format_instructions} - """ -) - -parser = JsonOutputParser() - -format_instructions = PromptTemplate.from_template("""{input}""").partial( - input=parser.get_format_instructions() -) - -state_est_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ("state", state_ppt), - ("output_format", output_format_ppt), - ("format_instructions", format_instructions), - ("example", example_ppt), - ], -) - - -state_est_ppt_ppl_chain = ( - state_est_ppt_ppl - # | ChatOpenAI(model="gpt-3.5-turbo", temperature=0) - | ChatOpenAI(model="gpt-4", temperature=0) - | JsonOutputParser() -) - - -async def core_run(): - async for event in app.astream( - inputs, - config=config, - ): - for k, v in event.items(): - if k != "__end__": - print(v) - - -def ut_gen_test(): - ut = ut_gen_chain.invoke( - { - # "action": "insert(left_hand, defaultgripper, gear1, shaft1)", - # "action": "insert(left_hand, parallel_box1, shaft1, gearbase_hole1)", - # "action": "change_tool(left_hand, parallel_box1, defaultgripper)", - "action": "insert(left_hand, clampgripper, gear2, shaft2)", - } - ) - - render_bt(ut) - - -def seq_planner_est_test(): - return seq_planner_est_chain.invoke( - { - "start_world_state": world_state_json, - "target": "is_inserted_to(shaft1, gearbase_hole1)", - # "target": "hold(left_hand, outward_claw)", - } - ) - - -def seq_action_plan_test(): - return seq_ac_pl_chain.invoke( - { - "start_world_state": world_state_json, - "target": "is_inserted_to(shaft1, gearbase_hole1)", - # "target": "hold(left_hand, outward_claw)", - } - ) - - -def state_est_test(): - return state_est_ppt_ppl_chain.invoke( - { - "start_world_state": world_state_json, - "action_plan": [ - "unload_tool(left_hand, outward_claw)", - "load_tool(left_hand, parallel_box1)", - "pick_up(left_hand, parallel_box1, shaft1)", - "insert(left_hand, parallel_box1, shaft1, gearbase_hole1)", - ], - } - ) - - -##################################### * speed imp -@traceable(name="make_plan") -def make_plan(state: dict, goal: str) -> list[str]: - print(f"----------start to make plan for the goal {goal}") - response = seq_ac_pl_chain.invoke( - { - "start_world_state": state, - "target": goal, - } - ) - print(f"finished making plan for the goal {goal}.") - pprint(f'LLM thought flow: {response["explanation"]}') - return response["task_plan"] - - -@traceable(name="estimate_state") -def estimate_state(start_world_state: dict, action_plan: list[str]) -> dict: - print("----------start to estimate the state after the action plan:") - pprint(action_plan) - response = state_est_ppt_ppl_chain.invoke( - { - "start_world_state": start_world_state, - "action_plan": action_plan, - } - ) - print(f"finished estimating the state after the action plan {action_plan}.") - return response["estimated_world_state"] - - -@traceable(name="generate_unit_subtree") -def generate_unit_subtree(action: str) -> dict: - print("----------start to generate the unit subtree for the action") - pprint(action) - response = ut_gen_chain.invoke( - { - "action": action, - } - ) - print(f"finished generating the unit subtree for the action {action}.") - return response - - -def get_node_list_from_tree(unit_subtree: dict) -> list[dict]: - children = unit_subtree["children"][1][ - "children" - ] # * the second child is a sequence - return children - - -def extract_goal(node: dict) -> str: - name = node["name"] - - -def match_type(node: dict) -> tuple[str, str]: - node_name = node["name"] - match = re.search( - r"(selector|sequence|action|precondition|condition|target):\s*(.+)", node_name - ) - if match: - node_type = match.group(1) - node_body = match.group(2) - return node_type, node_body - else: - raise ValueError(f"the node name {node_name} does not match any type.") - - -def expand_nodes( - node_list: list[dict], - start_state: dict, - overall_tree: list[dict] = None, -) -> dict: - """ - in order to monitor the tree generation, the overall tree and the node list should be the same variable when being passed in. - """ - pprint("----------check the entire tree:") - if overall_tree is not None: - render_bt(overall_tree[0]) - pprint("----------start to expand the node list:") - pprint(node_list) - pause = input("paused here! check the tree.") - - assert len(node_list) > 0 - state = start_state - - for i in range(len(node_list)): - type_name, body = match_type(node_list[i]) - # if match_type(node_list[i]) == "action": - if type_name == "action": - print(f"the node {node_list[i]['name']} is an action node. skip it.") - pause = input("paused here! check!") - # elif match_type(node_list[i]) == "precondition" or "target": - elif type_name in ["precondition", "target"]: - # goal = node_list[i]["name"] - goal = body - plan = make_plan(state, goal) - if len(plan) == 0: - logging.warning(f"No action should be performed for the goal {goal}.") - logging.warning(f'the node {node_list[i]["name"]} has been skipped.') - pause = input("paused here! check!") - else: - logging.info(f"Actions have been planed for the goal {goal}.") - pprint(f"the plan for the goal {goal} is {plan}") - pause = input("paused here! check!") - last_action = plan[-1] - unit_subtree = generate_unit_subtree(last_action) - # insert the subtree into the node_list - node_list[i] = unit_subtree - new_node_list = get_node_list_from_tree(unit_subtree) - expand_nodes( - node_list=new_node_list, - start_state=state, - overall_tree=overall_tree, - ) - state = estimate_state(state, plan) - - return node_list[0] - - -def embed_ut_nl(unit_subtree: dict) -> str: - """ - embed the unit subtree into the overall tree - """ - selector_children = unit_subtree["children"] - target = "" - # * target - for node in selector_children: - if match_type(node)[0] == "target": - target += node["summary"] - - sequence_children = selector_children[1]["children"] - preconditions = [] - action = "" - for node in sequence_children: - if match_type(node)[0] == "precondition": - preconditions.append(node["summary"]) - if match_type(node)[0] == "action": - action = node["summary"] - - embedding = f"if {' and '.join(preconditions)} then {action}, {target}" - - return embedding - - -def test_embedding_nl(): - unit_tree = ut_gen_test() - pprint(unit_tree) - nl = embed_ut_nl(unit_tree) - print(nl) - - -def test_expand_nodes(): - start_state = world_state_json - node_list = [ - { - "summary": "insert shaft1 into gearbase hole1", - "name": "target: insert shaft1 into gearbase hole1", - } - ] - # node_list = [ - # { - # "summary": "insert gear2 into shaft2", - # "name": "target: insert gear2 into shaft2", - # } - # ] - # node_list = [ - # { - # "summary": "pick up the shaft1", - # "name": "target: pick up the shaft1", - # }, - # ] - expand_nodes(node_list, start_state, node_list) - pprint(node_list) - - -if __name__ == "__main__": - pass - - # asyncio.run(core_run()) - # pprint(ut_gen_test()) - # pprint(seq_action_plan_test()) - # pprint(state_est_test()) - - test_expand_nodes() - - # test_embedding_nl() diff --git a/experiments/gearset1/runtime_script.py b/experiments/gearset1/runtime_script.py deleted file mode 100644 index c446c736..00000000 --- a/experiments/gearset1/runtime_script.py +++ /dev/null @@ -1,571 +0,0 @@ -import numpy as np -import os -import json -from spatialmath import * -from spatialmath.base import trnorm -from pprint import pprint - -from kios_robot.mios_task_factory import MiosTaskFactory -from kios_robot.robot_command import RobotCommand -from kios_robot.robot_interface import RobotInterface -from kios_robot.data_types import Toolbox -from kios_scene.scene_factory import SceneFactory - -from kios_scene.mios_ltm_manipulator import LangTermMemoryManipulator -from kios_utils.bblab_utils import execution_timer -import socket -from anran_tag.apriltag_rs import apriltag_rs -import threading - -ri = RobotInterface() -sf = SceneFactory() - -current_dir = os.path.dirname(os.path.realpath(__file__)) -scene_file = os.path.join(current_dir, "scene.json") - -with open(scene_file, "r") as f: - scene_string = f.read() - scene_json = json.loads(scene_string) - -scene = sf.create_scene_from_json(scene_json) -ri.setup_scene(scene) - -ltm_manipulator = LangTermMemoryManipulator() - - -def show_scene(): - pprint(scene) - - -# def teach_object(object: str): -# ri.proprioceptor.teach_object(object) -def backup_mios_environment(backup_name: str): - ltm_manipulator.backup_mios_environment(backup_name) - - -def clear_mios_environment(): - ltm_manipulator.clear_mios_environment() - - -def show_backups(): - ltm_manipulator.show_backups() - - -def restore_to_mios_environment(backup_name: str): - ltm_manipulator.restore_to_mios_environment(backup_name) - # ! you need to update the memory of mios. - update_mios_memory_environment() - # ! you also need to update the scene object in robot_interface. - scene = sf.create_scene_from_json(scene_json) - ri.setup_scene(scene) - - -def teach_object_TCP(object_name: str): - ri.proprioceptor.teach_object_TCP(object_name) - ri.proprioceptor.update_scene_object_from_mios(scene=scene, object_name=object_name) - - -def align_object(object_name: str, **kwargs): - ri.proprioceptor.align_object(object_name, **kwargs) - ri.proprioceptor.update_scene_object_from_mios(scene=scene, object_name=object_name) - - -def modify_object_position_rel(object_name: str, **kwargs): - ri.proprioceptor.modify_object_position(object_name, **kwargs) - ri.proprioceptor.update_scene_object_from_mios(scene=scene, object_name=object_name) - - -def update_mios_memory_environment(): - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - shared_data=None, - task_scene=scene, - robot_interface=ri, - ) - robot_command.add_task( - ri.mios_task_factory.generate_update_mios_memory_environment_call() - ) - robot_command.execute_task_list_sync() - - -def get_object(object: str): - return scene.get_object(object) - - -def set_tool(toolbox_name: str): - if "robot_command" in locals(): - print("robot command is set") - print(f"the robot command's tasks are {robot_command.task_list}") - else: - print("robot command is not set") - - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - shared_data=None, - task_scene=scene, - robot_interface=ri, - ) - print(f"the number of tasks in the task list is {len(robot_command.task_list)}") - robot_command.add_task(ri.mios_task_factory.generate_update_tool_call(toolbox_name)) - robot_command.execute_task_list_sync() - robot_command.clear_tasks() - - -def test_cartesian_joint_move(): - # need object "test" - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - shared_data=None, - task_scene=scene, - robot_interface=ri, - ) - - O_T_TCP = scene.get_object("test").O_T_TCP - print(O_T_TCP) - print(type(O_T_TCP)) - print(O_T_TCP.shape) - - robot_command.add_task( - ri.mios_task_factory.generate_cartesian_move_mp(O_T_TCP=O_T_TCP) - ) - - # rotate in z axis by 90 degrees - # rotation_90 = SE3.Rz(np.deg2rad(90)) - - # rotate in x axis by 90 degrees - rotation_90 = SE3.Rx(np.deg2rad(90)) - - se_O_T_TCP = SE3(trnorm(O_T_TCP)) - result = se_O_T_TCP * rotation_90 - O_T_TCP = result.A - - robot_command.add_task( - ri.mios_task_factory.generate_cartesian_move_mp(O_T_TCP=O_T_TCP) - ) - - robot_command.execute_task_list_sync() - - -def test_screw_in(object_name: str): - # need object "test" - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - shared_data=None, - task_scene=scene, - robot_interface=ri, - ) - - O_T_OB = scene.get_object(object_name).O_T_TCP - - print(O_T_OB) - - robot_command.add_task(ri.mios_task_factory.generate_screw_in_mp(O_T_OB=O_T_OB)) - - robot_command.execute_task_list_sync() - - -def tool_test(): - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - shared_data=None, - task_scene=scene, - robot_interface=ri, - ) - - robot_command.add_mios_task( - ri.mios_task_factory.generate_cartesian_move_mp("test_location") - ) - robot_command.add_mios_task( - ri.mios_task_factory.generate_update_tool_call("parallel_box1") - ) - robot_command.add_mios_task( - ri.mios_task_factory.generate_cartesian_move_mp("test_location") - ) - - robot_command.execute_task_list_sync() - - -def load_tool_test(tool_name: str): - parsed_action = { - "action_name": "load_tool", - "args": ["left_hand", tool_name], - } - - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - task_scene=scene, - shared_data=None, - robot_interface=ri, - ) - - robot_command.add_tasks( - ri.mios_task_factory.generate_load_tool_skill(parsed_action) - ) - - robot_command.execute_task_list_sync() - - -def pick_test(object_name: str): - - parsed_action = { - "action_name": "pick", - "args": [None, None, object_name], - } - - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - shared_data=None, - task_scene=scene, - robot_interface=ri, - ) - - robot_command.add_tasks(ri.mios_task_factory.generate_pick_up_skill(parsed_action)) - - robot_command.execute_task_list_sync() - - -def grasp(): - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - task_scene=scene, - shared_data=None, - robot_interface=ri, - ) - - robot_command.add_task(ri.mios_task_factory.generate_gripper_grasp_mp()) - - robot_command.execute_task_list_sync() - - -def move_gripper(width: float): - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - task_scene=scene, - shared_data=None, - robot_interface=ri, - ) - - robot_command.add_task(ri.mios_task_factory.generate_gripper_move_mp(width=width)) - - robot_command.execute_task_list_sync() - - -def home_gripper(): - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - task_scene=scene, - shared_data=None, - robot_interface=ri, - ) - - robot_command.add_task(ri.mios_task_factory.generate_gripper_home_mp()) - - robot_command.execute_task_list_sync() - - -def cartesian_move(object: str): - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - task_scene=scene, - shared_data=None, - robot_interface=ri, - ) - - robot_command.add_task(ri.mios_task_factory.generate_cartesian_move_mp(object)) - - robot_command.execute_task_list_sync() - - -def get_mark(): - UDP_IP = "127.0.0.1" - UDP_PORT = 8888 - sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) - - sock.bind((UDP_IP, UDP_PORT)) - while True: - data, addr = sock.recvfrom(1024) - - if data: - array = np.frombuffer(data, dtype=np.float64).reshape( - (4, 4) - ) # Adjust dtype if necessary - return array - - -def cartesian_move_T(): - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - task_scene=scene, - shared_data=None, - robot_interface=ri, - ) - robot_state = ri.proprioceptor.get_robot_state() - - EE_T_cam = np.array( - [ - [0.0, -1.0, 0.0, 0.0514976], - [1.0, 0.0, 0.0, -0.0375164], - [0.0, 0.0, 1.0, -0.0467639], - [0.0, 0.0, 0.0, 1.0], - ] - ) - - thread = threading.Thread(target=apriltag_rs) - thread.start() - - cam_T_mark = get_mark() - - O_T_obj = robot_state.O_T_TCP @ EE_T_cam @ cam_T_mark - - O_T_TCP_g = robot_state.O_T_TCP - - O_T_TCP_g[0][3] = O_T_obj[0][3] - O_T_TCP_g[1][3] = O_T_obj[1][3] - - robot_command.add_task( - ri.mios_task_factory.generate_cartesian_move_mp( - O_T_TCP=O_T_TCP_g, - ), - ) - robot_command.execute_task_list_sync() - - thread.join() - - -def joint_move(object: str): - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - task_scene=scene, - shared_data=None, - robot_interface=ri, - ) - - robot_command.add_task(ri.mios_task_factory.generate_joint_move_mp(object)) - - robot_command.execute_task_list_sync() - - -def insert_test(object_name: str): - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - task_scene=scene, - shared_data=None, - robot_interface=ri, - ) - - insert_action = { - "action_name": "insert", - "args": [None, None, "gear3", object_name], - } - - robot_command.add_tasks(ri.mios_task_factory.generate_insert_skill(insert_action)) - - robot_command.execute_task_list_sync() - - -def drive_test(): - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - task_scene=scene, - shared_data=None, - robot_interface=ri, - ) - - drive_action = { - "action_name": "drive", - "args": [None, None, None, None], - } - - robot_command.add_tasks(ri.mios_task_factory.generate_drive_skill(drive_action)) - - robot_command.execute_task_list_sync() - - -@execution_timer -def change_gripper(current_tool: str, new_tool: str): - - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - task_scene=scene, - shared_data=None, - robot_interface=ri, - ) - - if current_tool != "defaultgripper": - unload_tool_parsed_action = { - "action_name": "unload_tool", - "args": ["left_hand", current_tool], - } - - robot_command.add_tasks( - ri.mios_task_factory.generate_unload_tool_skill(unload_tool_parsed_action) - ) - - if new_tool != "defaultgripper": - - load_tool_parsed_action = { - "action_name": "load_tool", - "args": ["left_hand", new_tool], - } - - robot_command.add_tasks( - ri.mios_task_factory.generate_load_tool_skill(load_tool_parsed_action) - ) - - result = robot_command.execute_task_list_sync() - - if result == False: - pause = input("failed to change gripper") - - -@execution_timer -def initialize_gripper(): - - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - shared_data=None, - task_scene=scene, - robot_interface=ri, - ) - - robot_command.add_task( - ri.mios_task_factory.generate_joint_move_mp("initialposition") - ) - - robot_command.execute_task_list_sync() - - -@execution_timer -def insert_task(part1: str, part2: str): - """ - part1: the location name of the part to be inserted - part2: the location name of the part (usually a feature name) to be inserted into - """ - # * first pickup, then insert - - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - shared_data=None, - task_scene=scene, - robot_interface=ri, - ) - - pick_up_parsed_action = { - "action_name": "pick", # ! maybe this should be "pick_up" - "args": [None, None, part1], - } - - robot_command.add_tasks( - ri.mios_task_factory.generate_pick_up_skill(pick_up_parsed_action) - ) - - insert_parsed_action = { - "action_name": "insert", - "args": [None, None, part1, part2], - } - - # ! warning: the location of part1 will be refreshed after the insertion! - robot_command.add_tasks( - ri.mios_task_factory.generate_insert_skill_mod(insert_parsed_action) - ) - - robot_command.execute_task_list_sync() - - -def insert_test(location: str): - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - shared_data=None, - task_scene=scene, - robot_interface=ri, - ) - param = { - "search_a": [3, 3, 0, 0, 0, 0], - "search_f": [1, 1, 0, 0, 0, 0], - "search_phi": [ - 0, - 3.14159265358979323846 / 2, - 0, - 3.14159265358979323846 / 2, - 0, - 3.14159265358979323846 / 2, - ], - "F_ext_contact": [8.0, 2.0], - "f_push": [0, 0, 8, 0, 0, 0], - "K_x": [150, 150, 0, 150, 150, 0], - "env_X": [0.01, 0.01, -0.004, 0.05, 0.05, 0.05], - "D_x": [0.7, 0.7, 0, 0.7, 0.7, 1.4], - } - - robot_command.add_task( - ri.mios_task_factory.generate_insert_mp( - insertable="normalbolt", - container=location, - param=param, - ) - ) - - robot_command.execute_task_list_sync() - - -@execution_timer -def test_drive_in_mp(container: str = "normalcontainer"): - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - shared_data=None, - task_scene=scene, - robot_interface=ri, - ) - - # robot_command.add_task(ri.mios_task_factory.generate_loose_gripper_call()) - robot_command.add_task(ri.mios_task_factory.generate_drive_in_mp("bolt", container)) - - robot_command.execute_task_list_sync() - - -@execution_timer -def test_drive_out_mp(container: str = "normalcontainer"): - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - shared_data=None, - task_scene=scene, - robot_interface=ri, - ) - - # robot_command.add_task(ri.mios_task_factory.generate_loose_gripper_call()) - robot_command.add_task( - ri.mios_task_factory.generate_drive_out_mp("bolt", container) - ) - - robot_command.execute_task_list_sync() - - -if __name__ == "__main__": - pass - # apriltag_rs() - # cartesian_move_T() - # n1_sp() - -# n 44.29451922999942 -# sp 79.95160165699781 diff --git a/experiments/gearset1/scene.json b/experiments/gearset1/scene.json deleted file mode 100644 index cd1efbcd..00000000 --- a/experiments/gearset1/scene.json +++ /dev/null @@ -1,143 +0,0 @@ -{ - "reference_objects": - [ - { - "object_name": "test_box1", - "source": "pre-defined", - "O_T_TCP": [ - [1, 0, 0, 0.0], - [0, 1, 0, 0.0], - [0, 0, 1, -0.1], - [0, 0, 0, 1] - ] - }, - { - "object_name": "initialposition", - "source": "mios" - }, - { - "object_name": "inwardgripper", - "source": "mios" - }, - { - "object_name": "outwardgripper", - "source": "mios" - }, - { - "object_name": "clampgripper", - "source": "mios" - }, - { - "object_name": "parallelgripper", - "source": "mios" - }, - { - "object_name": "defaultgripper", - "source": "mios" - }, - { - "object_name": "gear1", - "source": "mios" - }, - { - "object_name": "gear2", - "source": "mios" - }, - { - "object_name": "gear3", - "source": "mios" - }, - { - "object_name": "shaft1", - "source": "mios" - }, - { - "object_name": "shaft2", - "source": "mios" - }, - { - "object_name": "shaft3", - "source": "mios" - }, - { - "object_name": "gearbase", - "source": "mios" - }, - { - "object_name": "gearbase_hole1", - "source": "mios" - }, - { - "object_name": "gearbase_hole3", - "source": "mios" - } - ], - "tools": - [ - { - "tool_name": "clampgripper", - "EE_T_TCP": [ - [1, 0, 0, 0.0], - [0, 1, 0, 0.0], - [0, 0, 1, 0.085], - [0, 0, 0, 1] - ], - "EE_finger_width_max": 0.04, - "EE_finger_width_min": 0.01, - "load_width": 0.04, - "unload_width": 0.08 - }, - { - "tool_name": "parallelgripper", - "EE_T_TCP": [ - [1, 0, 0, 0.0], - [0, 1, 0, 0.0], - [0, 0, 1, 0.07], - [0, 0, 0, 1] - ], - "EE_finger_width_max": 0.04, - "EE_finger_width_min": 0.01, - "load_width": 0.04, - "unload_width": 0.08 - }, - { - "tool_name": "inwardgripper", - "EE_T_TCP": [ - [1, 0, 0, 0.0], - [0, 1, 0, 0.0], - [0, 0, 1, 0.095], - [0, 0, 0, 1] - ], - "EE_finger_width_max": 0.04, - "EE_finger_width_min": 0.01, - "load_width": 0.04, - "unload_width": 0.08 - }, - { - "tool_name": "outwardgripper", - "EE_T_TCP": [ - [1, 0, 0, 0.0], - [0, 1, 0, 0.0], - [0, 0, 1, 0.125], - [0, 0, 0, 1] - ], - "EE_finger_width_max": 0.04, - "EE_finger_width_min": 0.01, - "load_width": 0.04, - "unload_width": 0.08 - }, - { - "tool_name": "defaultgripper", - "EE_T_TCP": [ - [1, 0, 0, 0.0], - [0, 1, 0, 0.0], - [0, 0, 1, 0.0], - [0, 0, 0, 1] - ], - "EE_finger_width_max": 0.08, - "EE_finger_width_min": 0.01, - "load_width": 0.08, - "unload_width": 0.08 - } - ] -} diff --git a/experiments/gearset1/selector_insert_gear1_into_shaft1.dot b/experiments/gearset1/selector_insert_gear1_into_shaft1.dot deleted file mode 100644 index 4d83a3cf..00000000 --- a/experiments/gearset1/selector_insert_gear1_into_shaft1.dot +++ /dev/null @@ -1,59 +0,0 @@ -digraph pastafarianism { -ordering=out; -graph [fontname="times-roman"]; -node [fontname="times-roman"]; -edge [fontname="times-roman"]; -selector [fillcolor=cyan, fontcolor=black, fontsize=9, label="selector: insert_gear1_into_shaft1", shape=octagon, style=filled]; -target [fillcolor=gray, fontcolor=black, fontsize=9, label="target: is_inserted_to(gear1, shaft1)", shape=ellipse, style=filled]; -"selector: insert_gear1_into_shaft1" -> "target: is_inserted_to(gear1, shaft1)"; -sequence [fillcolor=orange, fontcolor=black, fontsize=9, label="sequence: insert_gear1_into_shaft1", shape=box, style=filled]; -"selector: insert_gear1_into_shaft1" -> "sequence: insert_gear1_into_shaft1"; -selector [fillcolor=cyan, fontcolor=black, fontsize=9, label="selector: change_tool(left_hand, clampgripper, parallelgripper)", shape=octagon, style=filled]; -"sequence: insert_gear1_into_shaft1" -> "selector: change_tool(left_hand, clampgripper, parallelgripper)"; -target [fillcolor=gray, fontcolor=black, fontsize=9, label="target: hold(left_hand, parallelgripper)", shape=ellipse, style=filled]; -"selector: change_tool(left_hand, clampgripper, parallelgripper)" -> "target: hold(left_hand, parallelgripper)"; -sequence [fillcolor=orange, fontcolor=black, fontsize=9, label="sequence: change_tool(left_hand, clampgripper, parallelgripper)", shape=box, style=filled]; -"selector: change_tool(left_hand, clampgripper, parallelgripper)" -> "sequence: change_tool(left_hand, clampgripper, parallelgripper)"; -precondition [fillcolor=gray, fontcolor=black, fontsize=9, label="precondition: hold(left_hand, clampgripper)", shape=ellipse, style=filled]; -"sequence: change_tool(left_hand, clampgripper, parallelgripper)" -> "precondition: hold(left_hand, clampgripper)"; -precondition [fillcolor=gray, fontcolor=black, fontsize=9, label="precondition: is_empty(clampgripper)", shape=ellipse, style=filled]; -"sequence: change_tool(left_hand, clampgripper, parallelgripper)" -> "precondition: is_empty(clampgripper)"; -action [fillcolor=gray, fontcolor=black, fontsize=9, label="action: change_tool(left_hand, clampgripper, parallelgripper)", shape=ellipse, style=filled]; -"sequence: change_tool(left_hand, clampgripper, parallelgripper)" -> "action: change_tool(left_hand, clampgripper, parallelgripper)"; -selector [fillcolor=cyan, fontcolor=black, fontsize=9, label="selector: pick_up_gear1_with_parallelgripper", shape=octagon, style=filled]; -"sequence: insert_gear1_into_shaft1" -> "selector: pick_up_gear1_with_parallelgripper"; -target [fillcolor=gray, fontcolor=black, fontsize=9, label="target: hold(parallelgripper, gear1)", shape=ellipse, style=filled]; -"selector: pick_up_gear1_with_parallelgripper" -> "target: hold(parallelgripper, gear1)"; -sequence [fillcolor=orange, fontcolor=black, fontsize=9, label="sequence: pick_up_gear1_with_parallelgripper", shape=box, style=filled]; -"selector: pick_up_gear1_with_parallelgripper" -> "sequence: pick_up_gear1_with_parallelgripper"; -precondition [fillcolor=gray, fontcolor=black, fontsize=9, label="precondition: hold(left_hand, parallelgripper)", shape=ellipse, style=filled]; -"sequence: pick_up_gear1_with_parallelgripper" -> "precondition: hold(left_hand, parallelgripper)"; -precondition [fillcolor=gray, fontcolor=black, fontsize=9, label="precondition: is_empty(parallelgripper)", shape=ellipse, style=filled]; -"sequence: pick_up_gear1_with_parallelgripper" -> "precondition: is_empty(parallelgripper)"; -action [fillcolor=gray, fontcolor=black, fontsize=9, label="action: pick_up(left_hand, parallelgripper, gear1)", shape=ellipse, style=filled]; -"sequence: pick_up_gear1_with_parallelgripper" -> "action: pick_up(left_hand, parallelgripper, gear1)"; -selector [fillcolor=cyan, fontcolor=black, fontsize=9, label="selector: change_tool(left_hand, parallelgripper, clampgripper)", shape=octagon, style=filled]; -"sequence: insert_gear1_into_shaft1" -> "selector: change_tool(left_hand, parallelgripper, clampgripper)"; -target [fillcolor=gray, fontcolor=black, fontsize=9, label="target: hold(left_hand, clampgripper)", shape=ellipse, style=filled]; -"selector: change_tool(left_hand, parallelgripper, clampgripper)" -> "target: hold(left_hand, clampgripper)"; -sequence [fillcolor=orange, fontcolor=black, fontsize=9, label="sequence: change_tool(left_hand, parallelgripper, clampgripper)", shape=box, style=filled]; -"selector: change_tool(left_hand, parallelgripper, clampgripper)" -> "sequence: change_tool(left_hand, parallelgripper, clampgripper)"; -precondition [fillcolor=gray, fontcolor=black, fontsize=9, label="precondition: hold(left_hand, parallelgripper)*", shape=ellipse, style=filled]; -"sequence: change_tool(left_hand, parallelgripper, clampgripper)" -> "precondition: hold(left_hand, parallelgripper)*"; -precondition [fillcolor=gray, fontcolor=black, fontsize=9, label="precondition: hold(parallelgripper, gear1)", shape=ellipse, style=filled]; -"sequence: change_tool(left_hand, parallelgripper, clampgripper)" -> "precondition: hold(parallelgripper, gear1)"; -action [fillcolor=gray, fontcolor=black, fontsize=9, label="action: change_tool(left_hand, parallelgripper, clampgripper)", shape=ellipse, style=filled]; -"sequence: change_tool(left_hand, parallelgripper, clampgripper)" -> "action: change_tool(left_hand, parallelgripper, clampgripper)"; -selector [fillcolor=cyan, fontcolor=black, fontsize=9, label="selector: insert_gear1_shaft1", shape=octagon, style=filled]; -"sequence: insert_gear1_into_shaft1" -> "selector: insert_gear1_shaft1"; -target [fillcolor=gray, fontcolor=black, fontsize=9, label="target: is_inserted_to(gear1, shaft1)*", shape=ellipse, style=filled]; -"selector: insert_gear1_shaft1" -> "target: is_inserted_to(gear1, shaft1)*"; -sequence [fillcolor=orange, fontcolor=black, fontsize=9, label="sequence: insert_gear1_shaft1", shape=box, style=filled]; -"selector: insert_gear1_shaft1" -> "sequence: insert_gear1_shaft1"; -precondition [fillcolor=gray, fontcolor=black, fontsize=9, label="precondition: hold(left_hand, clampgripper)*", shape=ellipse, style=filled]; -"sequence: insert_gear1_shaft1" -> "precondition: hold(left_hand, clampgripper)*"; -precondition [fillcolor=gray, fontcolor=black, fontsize=9, label="precondition: hold(clampgripper, gear1)", shape=ellipse, style=filled]; -"sequence: insert_gear1_shaft1" -> "precondition: hold(clampgripper, gear1)"; -action [fillcolor=gray, fontcolor=black, fontsize=9, label="action: insert(left_hand, clampgripper, gear1, shaft1)", shape=ellipse, style=filled]; -"sequence: insert_gear1_shaft1" -> "action: insert(left_hand, clampgripper, gear1, shaft1)"; -} diff --git a/experiments/gearset1/selector_insert_gear1_into_shaft1.png b/experiments/gearset1/selector_insert_gear1_into_shaft1.png deleted file mode 100644 index a53caa57..00000000 Binary files a/experiments/gearset1/selector_insert_gear1_into_shaft1.png and /dev/null differ diff --git a/experiments/gearset1/selector_insert_gear1_into_shaft1.svg b/experiments/gearset1/selector_insert_gear1_into_shaft1.svg deleted file mode 100644 index 7eaa10e8..00000000 --- a/experiments/gearset1/selector_insert_gear1_into_shaft1.svg +++ /dev/null @@ -1,361 +0,0 @@ - - - - - - -pastafarianism - - - -selector - -selector: insert_gear1_shaft1 - - - -target - -target: is_inserted_to(gear1, shaft1)* - - - -selector: insert_gear1_into_shaft1 - -selector: insert_gear1_into_shaft1 - - - -target: is_inserted_to(gear1, shaft1) - -target: is_inserted_to(gear1, shaft1) - - - -selector: insert_gear1_into_shaft1->target: is_inserted_to(gear1, shaft1) - - - - - -sequence: insert_gear1_into_shaft1 - -sequence: insert_gear1_into_shaft1 - - - -selector: insert_gear1_into_shaft1->sequence: insert_gear1_into_shaft1 - - - - - -sequence - -sequence: insert_gear1_shaft1 - - - -selector: change_tool(left_hand, clampgripper, parallelgripper) - -selector: change_tool(left_hand, clampgripper, parallelgripper) - - - -sequence: insert_gear1_into_shaft1->selector: change_tool(left_hand, clampgripper, parallelgripper) - - - - - -selector: pick_up_gear1_with_parallelgripper - -selector: pick_up_gear1_with_parallelgripper - - - -sequence: insert_gear1_into_shaft1->selector: pick_up_gear1_with_parallelgripper - - - - - -selector: change_tool(left_hand, parallelgripper, clampgripper) - -selector: change_tool(left_hand, parallelgripper, clampgripper) - - - -sequence: insert_gear1_into_shaft1->selector: change_tool(left_hand, parallelgripper, clampgripper) - - - - - -selector: insert_gear1_shaft1 - -selector: insert_gear1_shaft1 - - - -sequence: insert_gear1_into_shaft1->selector: insert_gear1_shaft1 - - - - - -target: hold(left_hand, parallelgripper) - -target: hold(left_hand, parallelgripper) - - - -selector: change_tool(left_hand, clampgripper, parallelgripper)->target: hold(left_hand, parallelgripper) - - - - - -sequence: change_tool(left_hand, clampgripper, parallelgripper) - -sequence: change_tool(left_hand, clampgripper, parallelgripper) - - - -selector: change_tool(left_hand, clampgripper, parallelgripper)->sequence: change_tool(left_hand, clampgripper, parallelgripper) - - - - - -precondition: hold(left_hand, clampgripper) - -precondition: hold(left_hand, clampgripper) - - - -sequence: change_tool(left_hand, clampgripper, parallelgripper)->precondition: hold(left_hand, clampgripper) - - - - - -precondition: is_empty(clampgripper) - -precondition: is_empty(clampgripper) - - - -sequence: change_tool(left_hand, clampgripper, parallelgripper)->precondition: is_empty(clampgripper) - - - - - -action: change_tool(left_hand, clampgripper, parallelgripper) - -action: change_tool(left_hand, clampgripper, parallelgripper) - - - -sequence: change_tool(left_hand, clampgripper, parallelgripper)->action: change_tool(left_hand, clampgripper, parallelgripper) - - - - - -precondition - -precondition: hold(clampgripper, gear1) - - - -action - -action: insert(left_hand, clampgripper, gear1, shaft1) - - - -target: hold(parallelgripper, gear1) - -target: hold(parallelgripper, gear1) - - - -selector: pick_up_gear1_with_parallelgripper->target: hold(parallelgripper, gear1) - - - - - -sequence: pick_up_gear1_with_parallelgripper - -sequence: pick_up_gear1_with_parallelgripper - - - -selector: pick_up_gear1_with_parallelgripper->sequence: pick_up_gear1_with_parallelgripper - - - - - -precondition: hold(left_hand, parallelgripper) - -precondition: hold(left_hand, parallelgripper) - - - -sequence: pick_up_gear1_with_parallelgripper->precondition: hold(left_hand, parallelgripper) - - - - - -precondition: is_empty(parallelgripper) - -precondition: is_empty(parallelgripper) - - - -sequence: pick_up_gear1_with_parallelgripper->precondition: is_empty(parallelgripper) - - - - - -action: pick_up(left_hand, parallelgripper, gear1) - -action: pick_up(left_hand, parallelgripper, gear1) - - - -sequence: pick_up_gear1_with_parallelgripper->action: pick_up(left_hand, parallelgripper, gear1) - - - - - -target: hold(left_hand, clampgripper) - -target: hold(left_hand, clampgripper) - - - -selector: change_tool(left_hand, parallelgripper, clampgripper)->target: hold(left_hand, clampgripper) - - - - - -sequence: change_tool(left_hand, parallelgripper, clampgripper) - -sequence: change_tool(left_hand, parallelgripper, clampgripper) - - - -selector: change_tool(left_hand, parallelgripper, clampgripper)->sequence: change_tool(left_hand, parallelgripper, clampgripper) - - - - - -precondition: hold(left_hand, parallelgripper)* - -precondition: hold(left_hand, parallelgripper)* - - - -sequence: change_tool(left_hand, parallelgripper, clampgripper)->precondition: hold(left_hand, parallelgripper)* - - - - - -precondition: hold(parallelgripper, gear1) - -precondition: hold(parallelgripper, gear1) - - - -sequence: change_tool(left_hand, parallelgripper, clampgripper)->precondition: hold(parallelgripper, gear1) - - - - - -action: change_tool(left_hand, parallelgripper, clampgripper) - -action: change_tool(left_hand, parallelgripper, clampgripper) - - - -sequence: change_tool(left_hand, parallelgripper, clampgripper)->action: change_tool(left_hand, parallelgripper, clampgripper) - - - - - -target: is_inserted_to(gear1, shaft1)* - -target: is_inserted_to(gear1, shaft1)* - - - -selector: insert_gear1_shaft1->target: is_inserted_to(gear1, shaft1)* - - - - - -sequence: insert_gear1_shaft1 - -sequence: insert_gear1_shaft1 - - - -selector: insert_gear1_shaft1->sequence: insert_gear1_shaft1 - - - - - -precondition: hold(left_hand, clampgripper)* - -precondition: hold(left_hand, clampgripper)* - - - -sequence: insert_gear1_shaft1->precondition: hold(left_hand, clampgripper)* - - - - - -precondition: hold(clampgripper, gear1) - -precondition: hold(clampgripper, gear1) - - - -sequence: insert_gear1_shaft1->precondition: hold(clampgripper, gear1) - - - - - -action: insert(left_hand, clampgripper, gear1, shaft1) - -action: insert(left_hand, clampgripper, gear1, shaft1) - - - -sequence: insert_gear1_shaft1->action: insert(left_hand, clampgripper, gear1, shaft1) - - - - - diff --git a/experiments/gearset1/selector_insert_gear1_into_shaft1_corrected.dot b/experiments/gearset1/selector_insert_gear1_into_shaft1_corrected.dot deleted file mode 100644 index d79ae73b..00000000 --- a/experiments/gearset1/selector_insert_gear1_into_shaft1_corrected.dot +++ /dev/null @@ -1,47 +0,0 @@ -digraph pastafarianism { -ordering=out; -graph [fontname="times-roman"]; -node [fontname="times-roman"]; -edge [fontname="times-roman"]; -selector [fillcolor=cyan, fontcolor=black, fontsize=9, label="selector: insert_gear1_into_shaft1_corrected", shape=octagon, style=filled]; -target [fillcolor=gray, fontcolor=black, fontsize=9, label="target: is_inserted_to(gear1, shaft1)", shape=ellipse, style=filled]; -"selector: insert_gear1_into_shaft1_corrected" -> "target: is_inserted_to(gear1, shaft1)"; -sequence [fillcolor=orange, fontcolor=black, fontsize=9, label="sequence: insert_gear1_into_shaft1_optimized", shape=box, style=filled]; -"selector: insert_gear1_into_shaft1_corrected" -> "sequence: insert_gear1_into_shaft1_optimized"; -selector [fillcolor=cyan, fontcolor=black, fontsize=9, label="selector: change_tool(left_hand, clampgripper, parallelgripper)", shape=octagon, style=filled]; -"sequence: insert_gear1_into_shaft1_optimized" -> "selector: change_tool(left_hand, clampgripper, parallelgripper)"; -target [fillcolor=gray, fontcolor=black, fontsize=9, label="target: hold(left_hand, parallelgripper)", shape=ellipse, style=filled]; -"selector: change_tool(left_hand, clampgripper, parallelgripper)" -> "target: hold(left_hand, parallelgripper)"; -sequence [fillcolor=orange, fontcolor=black, fontsize=9, label="sequence: change_tool(left_hand, clampgripper, parallelgripper)", shape=box, style=filled]; -"selector: change_tool(left_hand, clampgripper, parallelgripper)" -> "sequence: change_tool(left_hand, clampgripper, parallelgripper)"; -precondition [fillcolor=gray, fontcolor=black, fontsize=9, label="precondition: hold(left_hand, clampgripper)", shape=ellipse, style=filled]; -"sequence: change_tool(left_hand, clampgripper, parallelgripper)" -> "precondition: hold(left_hand, clampgripper)"; -precondition [fillcolor=gray, fontcolor=black, fontsize=9, label="precondition: is_empty(clampgripper)", shape=ellipse, style=filled]; -"sequence: change_tool(left_hand, clampgripper, parallelgripper)" -> "precondition: is_empty(clampgripper)"; -action [fillcolor=gray, fontcolor=black, fontsize=9, label="action: change_tool(left_hand, clampgripper, parallelgripper)", shape=ellipse, style=filled]; -"sequence: change_tool(left_hand, clampgripper, parallelgripper)" -> "action: change_tool(left_hand, clampgripper, parallelgripper)"; -selector [fillcolor=cyan, fontcolor=black, fontsize=9, label="selector: pick_up_gear1_with_parallelgripper", shape=octagon, style=filled]; -"sequence: insert_gear1_into_shaft1_optimized" -> "selector: pick_up_gear1_with_parallelgripper"; -target [fillcolor=gray, fontcolor=black, fontsize=9, label="target: hold(parallelgripper, gear1)", shape=ellipse, style=filled]; -"selector: pick_up_gear1_with_parallelgripper" -> "target: hold(parallelgripper, gear1)"; -sequence [fillcolor=orange, fontcolor=black, fontsize=9, label="sequence: pick_up_gear1_with_parallelgripper", shape=box, style=filled]; -"selector: pick_up_gear1_with_parallelgripper" -> "sequence: pick_up_gear1_with_parallelgripper"; -precondition [fillcolor=gray, fontcolor=black, fontsize=9, label="precondition: hold(left_hand, parallelgripper)", shape=ellipse, style=filled]; -"sequence: pick_up_gear1_with_parallelgripper" -> "precondition: hold(left_hand, parallelgripper)"; -precondition [fillcolor=gray, fontcolor=black, fontsize=9, label="precondition: is_empty(parallelgripper)", shape=ellipse, style=filled]; -"sequence: pick_up_gear1_with_parallelgripper" -> "precondition: is_empty(parallelgripper)"; -action [fillcolor=gray, fontcolor=black, fontsize=9, label="action: pick_up(left_hand, parallelgripper, gear1)", shape=ellipse, style=filled]; -"sequence: pick_up_gear1_with_parallelgripper" -> "action: pick_up(left_hand, parallelgripper, gear1)"; -selector [fillcolor=cyan, fontcolor=black, fontsize=9, label="selector: insert_gear1_shaft1_with_parallelgripper", shape=octagon, style=filled]; -"sequence: insert_gear1_into_shaft1_optimized" -> "selector: insert_gear1_shaft1_with_parallelgripper"; -target [fillcolor=gray, fontcolor=black, fontsize=9, label="target: is_inserted_to(gear1, shaft1)*", shape=ellipse, style=filled]; -"selector: insert_gear1_shaft1_with_parallelgripper" -> "target: is_inserted_to(gear1, shaft1)*"; -sequence [fillcolor=orange, fontcolor=black, fontsize=9, label="sequence: insert_gear1_shaft1_with_parallelgripper", shape=box, style=filled]; -"selector: insert_gear1_shaft1_with_parallelgripper" -> "sequence: insert_gear1_shaft1_with_parallelgripper"; -precondition [fillcolor=gray, fontcolor=black, fontsize=9, label="precondition: hold(left_hand, parallelgripper)*", shape=ellipse, style=filled]; -"sequence: insert_gear1_shaft1_with_parallelgripper" -> "precondition: hold(left_hand, parallelgripper)*"; -precondition [fillcolor=gray, fontcolor=black, fontsize=9, label="precondition: hold(parallelgripper, gear1)", shape=ellipse, style=filled]; -"sequence: insert_gear1_shaft1_with_parallelgripper" -> "precondition: hold(parallelgripper, gear1)"; -action [fillcolor=gray, fontcolor=black, fontsize=9, label="action: insert(left_hand, parallelgripper, gear1, shaft1)", shape=ellipse, style=filled]; -"sequence: insert_gear1_shaft1_with_parallelgripper" -> "action: insert(left_hand, parallelgripper, gear1, shaft1)"; -} diff --git a/experiments/gearset1/selector_insert_gear1_into_shaft1_corrected.png b/experiments/gearset1/selector_insert_gear1_into_shaft1_corrected.png deleted file mode 100644 index c88e7fd4..00000000 Binary files a/experiments/gearset1/selector_insert_gear1_into_shaft1_corrected.png and /dev/null differ diff --git a/experiments/gearset1/selector_insert_gear1_into_shaft1_corrected.svg b/experiments/gearset1/selector_insert_gear1_into_shaft1_corrected.svg deleted file mode 100644 index 3206dfc5..00000000 --- a/experiments/gearset1/selector_insert_gear1_into_shaft1_corrected.svg +++ /dev/null @@ -1,289 +0,0 @@ - - - - - - -pastafarianism - - - -selector - -selector: insert_gear1_shaft1_with_parallelgripper - - - -target - -target: is_inserted_to(gear1, shaft1)* - - - -selector: insert_gear1_into_shaft1_corrected - -selector: insert_gear1_into_shaft1_corrected - - - -target: is_inserted_to(gear1, shaft1) - -target: is_inserted_to(gear1, shaft1) - - - -selector: insert_gear1_into_shaft1_corrected->target: is_inserted_to(gear1, shaft1) - - - - - -sequence: insert_gear1_into_shaft1_optimized - -sequence: insert_gear1_into_shaft1_optimized - - - -selector: insert_gear1_into_shaft1_corrected->sequence: insert_gear1_into_shaft1_optimized - - - - - -sequence - -sequence: insert_gear1_shaft1_with_parallelgripper - - - -selector: change_tool(left_hand, clampgripper, parallelgripper) - -selector: change_tool(left_hand, clampgripper, parallelgripper) - - - -sequence: insert_gear1_into_shaft1_optimized->selector: change_tool(left_hand, clampgripper, parallelgripper) - - - - - -selector: pick_up_gear1_with_parallelgripper - -selector: pick_up_gear1_with_parallelgripper - - - -sequence: insert_gear1_into_shaft1_optimized->selector: pick_up_gear1_with_parallelgripper - - - - - -selector: insert_gear1_shaft1_with_parallelgripper - -selector: insert_gear1_shaft1_with_parallelgripper - - - -sequence: insert_gear1_into_shaft1_optimized->selector: insert_gear1_shaft1_with_parallelgripper - - - - - -target: hold(left_hand, parallelgripper) - -target: hold(left_hand, parallelgripper) - - - -selector: change_tool(left_hand, clampgripper, parallelgripper)->target: hold(left_hand, parallelgripper) - - - - - -sequence: change_tool(left_hand, clampgripper, parallelgripper) - -sequence: change_tool(left_hand, clampgripper, parallelgripper) - - - -selector: change_tool(left_hand, clampgripper, parallelgripper)->sequence: change_tool(left_hand, clampgripper, parallelgripper) - - - - - -precondition: hold(left_hand, clampgripper) - -precondition: hold(left_hand, clampgripper) - - - -sequence: change_tool(left_hand, clampgripper, parallelgripper)->precondition: hold(left_hand, clampgripper) - - - - - -precondition: is_empty(clampgripper) - -precondition: is_empty(clampgripper) - - - -sequence: change_tool(left_hand, clampgripper, parallelgripper)->precondition: is_empty(clampgripper) - - - - - -action: change_tool(left_hand, clampgripper, parallelgripper) - -action: change_tool(left_hand, clampgripper, parallelgripper) - - - -sequence: change_tool(left_hand, clampgripper, parallelgripper)->action: change_tool(left_hand, clampgripper, parallelgripper) - - - - - -precondition - -precondition: hold(parallelgripper, gear1) - - - -action - -action: insert(left_hand, parallelgripper, gear1, shaft1) - - - -target: hold(parallelgripper, gear1) - -target: hold(parallelgripper, gear1) - - - -selector: pick_up_gear1_with_parallelgripper->target: hold(parallelgripper, gear1) - - - - - -sequence: pick_up_gear1_with_parallelgripper - -sequence: pick_up_gear1_with_parallelgripper - - - -selector: pick_up_gear1_with_parallelgripper->sequence: pick_up_gear1_with_parallelgripper - - - - - -precondition: hold(left_hand, parallelgripper) - -precondition: hold(left_hand, parallelgripper) - - - -sequence: pick_up_gear1_with_parallelgripper->precondition: hold(left_hand, parallelgripper) - - - - - -precondition: is_empty(parallelgripper) - -precondition: is_empty(parallelgripper) - - - -sequence: pick_up_gear1_with_parallelgripper->precondition: is_empty(parallelgripper) - - - - - -action: pick_up(left_hand, parallelgripper, gear1) - -action: pick_up(left_hand, parallelgripper, gear1) - - - -sequence: pick_up_gear1_with_parallelgripper->action: pick_up(left_hand, parallelgripper, gear1) - - - - - -target: is_inserted_to(gear1, shaft1)* - -target: is_inserted_to(gear1, shaft1)* - - - -selector: insert_gear1_shaft1_with_parallelgripper->target: is_inserted_to(gear1, shaft1)* - - - - - -sequence: insert_gear1_shaft1_with_parallelgripper - -sequence: insert_gear1_shaft1_with_parallelgripper - - - -selector: insert_gear1_shaft1_with_parallelgripper->sequence: insert_gear1_shaft1_with_parallelgripper - - - - - -precondition: hold(left_hand, parallelgripper)* - -precondition: hold(left_hand, parallelgripper)* - - - -sequence: insert_gear1_shaft1_with_parallelgripper->precondition: hold(left_hand, parallelgripper)* - - - - - -precondition: hold(parallelgripper, gear1) - -precondition: hold(parallelgripper, gear1) - - - -sequence: insert_gear1_shaft1_with_parallelgripper->precondition: hold(parallelgripper, gear1) - - - - - -action: insert(left_hand, parallelgripper, gear1, shaft1) - -action: insert(left_hand, parallelgripper, gear1, shaft1) - - - -sequence: insert_gear1_shaft1_with_parallelgripper->action: insert(left_hand, parallelgripper, gear1, shaft1) - - - - - diff --git a/experiments/gearset1/selector_insert_shaft1_into_gearbase_hole1.dot b/experiments/gearset1/selector_insert_shaft1_into_gearbase_hole1.dot deleted file mode 100644 index 8e4eea5c..00000000 --- a/experiments/gearset1/selector_insert_shaft1_into_gearbase_hole1.dot +++ /dev/null @@ -1,57 +0,0 @@ -digraph pastafarianism { -ordering=out; -graph [fontname="times-roman"]; -node [fontname="times-roman"]; -edge [fontname="times-roman"]; -selector [fillcolor=cyan, fontcolor=black, fontsize=9, label="selector: insert_shaft1_into_gearbase_hole1", shape=octagon, style=filled]; -target [fillcolor=gray, fontcolor=black, fontsize=9, label="target: is_inserted_to(shaft1, gearbase_hole1)", shape=ellipse, style=filled]; -"selector: insert_shaft1_into_gearbase_hole1" -> "target: is_inserted_to(shaft1, gearbase_hole1)"; -sequence [fillcolor=orange, fontcolor=black, fontsize=9, label="sequence: insert_shaft1_into_gearbase_hole1", shape=box, style=filled]; -"selector: insert_shaft1_into_gearbase_hole1" -> "sequence: insert_shaft1_into_gearbase_hole1"; -selector [fillcolor=cyan, fontcolor=black, fontsize=9, label="selector: change_tool(left_hand, outwardgripper, clampgripper)", shape=octagon, style=filled]; -"sequence: insert_shaft1_into_gearbase_hole1" -> "selector: change_tool(left_hand, outwardgripper, clampgripper)"; -target [fillcolor=gray, fontcolor=black, fontsize=9, label="target: hold(left_hand, clampgripper)", shape=ellipse, style=filled]; -"selector: change_tool(left_hand, outwardgripper, clampgripper)" -> "target: hold(left_hand, clampgripper)"; -sequence [fillcolor=orange, fontcolor=black, fontsize=9, label="sequence: change_tool(left_hand, outwardgripper, clampgripper)", shape=box, style=filled]; -"selector: change_tool(left_hand, outwardgripper, clampgripper)" -> "sequence: change_tool(left_hand, outwardgripper, clampgripper)"; -precondition [fillcolor=gray, fontcolor=black, fontsize=9, label="precondition: hold(left_hand, outwardgripper)", shape=ellipse, style=filled]; -"sequence: change_tool(left_hand, outwardgripper, clampgripper)" -> "precondition: hold(left_hand, outwardgripper)"; -selector [fillcolor=cyan, fontcolor=black, fontsize=9, label="selector: put_down_gear3_from_outwardgripper", shape=octagon, style=filled]; -"sequence: change_tool(left_hand, outwardgripper, clampgripper)" -> "selector: put_down_gear3_from_outwardgripper"; -target [fillcolor=gray, fontcolor=black, fontsize=9, label="target: is_empty(outwardgripper)", shape=ellipse, style=filled]; -"selector: put_down_gear3_from_outwardgripper" -> "target: is_empty(outwardgripper)"; -sequence [fillcolor=orange, fontcolor=black, fontsize=9, label="sequence: put_down_gear3_from_outwardgripper", shape=box, style=filled]; -"selector: put_down_gear3_from_outwardgripper" -> "sequence: put_down_gear3_from_outwardgripper"; -precondition [fillcolor=gray, fontcolor=black, fontsize=9, label="precondition: hold(left_hand, outwardgripper)*", shape=ellipse, style=filled]; -"sequence: put_down_gear3_from_outwardgripper" -> "precondition: hold(left_hand, outwardgripper)*"; -precondition [fillcolor=gray, fontcolor=black, fontsize=9, label="precondition: hold(outwardgripper, gear3)", shape=ellipse, style=filled]; -"sequence: put_down_gear3_from_outwardgripper" -> "precondition: hold(outwardgripper, gear3)"; -action [fillcolor=gray, fontcolor=black, fontsize=9, label="action: put_down(left_hand, outwardgripper, gear3)", shape=ellipse, style=filled]; -"sequence: put_down_gear3_from_outwardgripper" -> "action: put_down(left_hand, outwardgripper, gear3)"; -action [fillcolor=gray, fontcolor=black, fontsize=9, label="action: change_tool(left_hand, outwardgripper, clampgripper)", shape=ellipse, style=filled]; -"sequence: change_tool(left_hand, outwardgripper, clampgripper)" -> "action: change_tool(left_hand, outwardgripper, clampgripper)"; -selector [fillcolor=cyan, fontcolor=black, fontsize=9, label="selector: pick_up_shaft1_with_clampgripper", shape=octagon, style=filled]; -"sequence: insert_shaft1_into_gearbase_hole1" -> "selector: pick_up_shaft1_with_clampgripper"; -target [fillcolor=gray, fontcolor=black, fontsize=9, label="target: hold(clampgripper, shaft1)", shape=ellipse, style=filled]; -"selector: pick_up_shaft1_with_clampgripper" -> "target: hold(clampgripper, shaft1)"; -sequence [fillcolor=orange, fontcolor=black, fontsize=9, label="sequence: pick_up_shaft1_with_clampgripper", shape=box, style=filled]; -"selector: pick_up_shaft1_with_clampgripper" -> "sequence: pick_up_shaft1_with_clampgripper"; -precondition [fillcolor=gray, fontcolor=black, fontsize=9, label="precondition: hold(left_hand, clampgripper)", shape=ellipse, style=filled]; -"sequence: pick_up_shaft1_with_clampgripper" -> "precondition: hold(left_hand, clampgripper)"; -precondition [fillcolor=gray, fontcolor=black, fontsize=9, label="precondition: is_empty(clampgripper)", shape=ellipse, style=filled]; -"sequence: pick_up_shaft1_with_clampgripper" -> "precondition: is_empty(clampgripper)"; -action [fillcolor=gray, fontcolor=black, fontsize=9, label="action: pick_up(left_hand, clampgripper, shaft1)", shape=ellipse, style=filled]; -"sequence: pick_up_shaft1_with_clampgripper" -> "action: pick_up(left_hand, clampgripper, shaft1)"; -selector [fillcolor=cyan, fontcolor=black, fontsize=9, label="selector: insert_shaft1_gearbase_hole1", shape=octagon, style=filled]; -"sequence: insert_shaft1_into_gearbase_hole1" -> "selector: insert_shaft1_gearbase_hole1"; -target [fillcolor=gray, fontcolor=black, fontsize=9, label="target: is_inserted_to(shaft1, gearbase_hole1)*", shape=ellipse, style=filled]; -"selector: insert_shaft1_gearbase_hole1" -> "target: is_inserted_to(shaft1, gearbase_hole1)*"; -sequence [fillcolor=orange, fontcolor=black, fontsize=9, label="sequence: insert_shaft1_gearbase_hole1", shape=box, style=filled]; -"selector: insert_shaft1_gearbase_hole1" -> "sequence: insert_shaft1_gearbase_hole1"; -precondition [fillcolor=gray, fontcolor=black, fontsize=9, label="precondition: hold(left_hand, clampgripper)*", shape=ellipse, style=filled]; -"sequence: insert_shaft1_gearbase_hole1" -> "precondition: hold(left_hand, clampgripper)*"; -precondition [fillcolor=gray, fontcolor=black, fontsize=9, label="precondition: hold(clampgripper, shaft1)", shape=ellipse, style=filled]; -"sequence: insert_shaft1_gearbase_hole1" -> "precondition: hold(clampgripper, shaft1)"; -action [fillcolor=gray, fontcolor=black, fontsize=9, label="action: insert(left_hand, clampgripper, shaft1, gearbase_hole1)", shape=ellipse, style=filled]; -"sequence: insert_shaft1_gearbase_hole1" -> "action: insert(left_hand, clampgripper, shaft1, gearbase_hole1)"; -} diff --git a/experiments/gearset1/selector_insert_shaft1_into_gearbase_hole1.png b/experiments/gearset1/selector_insert_shaft1_into_gearbase_hole1.png deleted file mode 100644 index 972bbb13..00000000 Binary files a/experiments/gearset1/selector_insert_shaft1_into_gearbase_hole1.png and /dev/null differ diff --git a/experiments/gearset1/selector_insert_shaft1_into_gearbase_hole1.svg b/experiments/gearset1/selector_insert_shaft1_into_gearbase_hole1.svg deleted file mode 100644 index 3be618fe..00000000 --- a/experiments/gearset1/selector_insert_shaft1_into_gearbase_hole1.svg +++ /dev/null @@ -1,349 +0,0 @@ - - - - - - -pastafarianism - - - -selector - -selector: insert_shaft1_gearbase_hole1 - - - -target - -target: is_inserted_to(shaft1, gearbase_hole1)* - - - -selector: insert_shaft1_into_gearbase_hole1 - -selector: insert_shaft1_into_gearbase_hole1 - - - -target: is_inserted_to(shaft1, gearbase_hole1) - -target: is_inserted_to(shaft1, gearbase_hole1) - - - -selector: insert_shaft1_into_gearbase_hole1->target: is_inserted_to(shaft1, gearbase_hole1) - - - - - -sequence: insert_shaft1_into_gearbase_hole1 - -sequence: insert_shaft1_into_gearbase_hole1 - - - -selector: insert_shaft1_into_gearbase_hole1->sequence: insert_shaft1_into_gearbase_hole1 - - - - - -sequence - -sequence: insert_shaft1_gearbase_hole1 - - - -selector: change_tool(left_hand, outwardgripper, clampgripper) - -selector: change_tool(left_hand, outwardgripper, clampgripper) - - - -sequence: insert_shaft1_into_gearbase_hole1->selector: change_tool(left_hand, outwardgripper, clampgripper) - - - - - -selector: pick_up_shaft1_with_clampgripper - -selector: pick_up_shaft1_with_clampgripper - - - -sequence: insert_shaft1_into_gearbase_hole1->selector: pick_up_shaft1_with_clampgripper - - - - - -selector: insert_shaft1_gearbase_hole1 - -selector: insert_shaft1_gearbase_hole1 - - - -sequence: insert_shaft1_into_gearbase_hole1->selector: insert_shaft1_gearbase_hole1 - - - - - -target: hold(left_hand, clampgripper) - -target: hold(left_hand, clampgripper) - - - -selector: change_tool(left_hand, outwardgripper, clampgripper)->target: hold(left_hand, clampgripper) - - - - - -sequence: change_tool(left_hand, outwardgripper, clampgripper) - -sequence: change_tool(left_hand, outwardgripper, clampgripper) - - - -selector: change_tool(left_hand, outwardgripper, clampgripper)->sequence: change_tool(left_hand, outwardgripper, clampgripper) - - - - - -precondition: hold(left_hand, outwardgripper) - -precondition: hold(left_hand, outwardgripper) - - - -sequence: change_tool(left_hand, outwardgripper, clampgripper)->precondition: hold(left_hand, outwardgripper) - - - - - -selector: put_down_gear3_from_outwardgripper - -selector: put_down_gear3_from_outwardgripper - - - -sequence: change_tool(left_hand, outwardgripper, clampgripper)->selector: put_down_gear3_from_outwardgripper - - - - - -action: change_tool(left_hand, outwardgripper, clampgripper) - -action: change_tool(left_hand, outwardgripper, clampgripper) - - - -sequence: change_tool(left_hand, outwardgripper, clampgripper)->action: change_tool(left_hand, outwardgripper, clampgripper) - - - - - -precondition - -precondition: hold(clampgripper, shaft1) - - - -target: is_empty(outwardgripper) - -target: is_empty(outwardgripper) - - - -selector: put_down_gear3_from_outwardgripper->target: is_empty(outwardgripper) - - - - - -sequence: put_down_gear3_from_outwardgripper - -sequence: put_down_gear3_from_outwardgripper - - - -selector: put_down_gear3_from_outwardgripper->sequence: put_down_gear3_from_outwardgripper - - - - - -precondition: hold(left_hand, outwardgripper)* - -precondition: hold(left_hand, outwardgripper)* - - - -sequence: put_down_gear3_from_outwardgripper->precondition: hold(left_hand, outwardgripper)* - - - - - -precondition: hold(outwardgripper, gear3) - -precondition: hold(outwardgripper, gear3) - - - -sequence: put_down_gear3_from_outwardgripper->precondition: hold(outwardgripper, gear3) - - - - - -action: put_down(left_hand, outwardgripper, gear3) - -action: put_down(left_hand, outwardgripper, gear3) - - - -sequence: put_down_gear3_from_outwardgripper->action: put_down(left_hand, outwardgripper, gear3) - - - - - -action - -action: insert(left_hand, clampgripper, shaft1, gearbase_hole1) - - - -target: hold(clampgripper, shaft1) - -target: hold(clampgripper, shaft1) - - - -selector: pick_up_shaft1_with_clampgripper->target: hold(clampgripper, shaft1) - - - - - -sequence: pick_up_shaft1_with_clampgripper - -sequence: pick_up_shaft1_with_clampgripper - - - -selector: pick_up_shaft1_with_clampgripper->sequence: pick_up_shaft1_with_clampgripper - - - - - -precondition: hold(left_hand, clampgripper) - -precondition: hold(left_hand, clampgripper) - - - -sequence: pick_up_shaft1_with_clampgripper->precondition: hold(left_hand, clampgripper) - - - - - -precondition: is_empty(clampgripper) - -precondition: is_empty(clampgripper) - - - -sequence: pick_up_shaft1_with_clampgripper->precondition: is_empty(clampgripper) - - - - - -action: pick_up(left_hand, clampgripper, shaft1) - -action: pick_up(left_hand, clampgripper, shaft1) - - - -sequence: pick_up_shaft1_with_clampgripper->action: pick_up(left_hand, clampgripper, shaft1) - - - - - -target: is_inserted_to(shaft1, gearbase_hole1)* - -target: is_inserted_to(shaft1, gearbase_hole1)* - - - -selector: insert_shaft1_gearbase_hole1->target: is_inserted_to(shaft1, gearbase_hole1)* - - - - - -sequence: insert_shaft1_gearbase_hole1 - -sequence: insert_shaft1_gearbase_hole1 - - - -selector: insert_shaft1_gearbase_hole1->sequence: insert_shaft1_gearbase_hole1 - - - - - -precondition: hold(left_hand, clampgripper)* - -precondition: hold(left_hand, clampgripper)* - - - -sequence: insert_shaft1_gearbase_hole1->precondition: hold(left_hand, clampgripper)* - - - - - -precondition: hold(clampgripper, shaft1) - -precondition: hold(clampgripper, shaft1) - - - -sequence: insert_shaft1_gearbase_hole1->precondition: hold(clampgripper, shaft1) - - - - - -action: insert(left_hand, clampgripper, shaft1, gearbase_hole1) - -action: insert(left_hand, clampgripper, shaft1, gearbase_hole1) - - - -sequence: insert_shaft1_gearbase_hole1->action: insert(left_hand, clampgripper, shaft1, gearbase_hole1) - - - - - diff --git a/experiments/gearset1/selector_insert_shaft1_into_gearbase_hole1_corrected.dot b/experiments/gearset1/selector_insert_shaft1_into_gearbase_hole1_corrected.dot deleted file mode 100644 index d594b256..00000000 --- a/experiments/gearset1/selector_insert_shaft1_into_gearbase_hole1_corrected.dot +++ /dev/null @@ -1,59 +0,0 @@ -digraph pastafarianism { -ordering=out; -graph [fontname="times-roman"]; -node [fontname="times-roman"]; -edge [fontname="times-roman"]; -selector [fillcolor=cyan, fontcolor=black, fontsize=9, label="selector: insert_shaft1_into_gearbase_hole1_corrected", shape=octagon, style=filled]; -target [fillcolor=gray, fontcolor=black, fontsize=9, label="target: is_inserted_to(shaft1, gearbase_hole1)", shape=ellipse, style=filled]; -"selector: insert_shaft1_into_gearbase_hole1_corrected" -> "target: is_inserted_to(shaft1, gearbase_hole1)"; -sequence [fillcolor=orange, fontcolor=black, fontsize=9, label="sequence: insert_shaft1_into_gearbase_hole1_corrected", shape=box, style=filled]; -"selector: insert_shaft1_into_gearbase_hole1_corrected" -> "sequence: insert_shaft1_into_gearbase_hole1_corrected"; -selector [fillcolor=cyan, fontcolor=black, fontsize=9, label="selector: put_down_gear3_before_changing_tool", shape=octagon, style=filled]; -"sequence: insert_shaft1_into_gearbase_hole1_corrected" -> "selector: put_down_gear3_before_changing_tool"; -target [fillcolor=gray, fontcolor=black, fontsize=9, label="target: is_empty(outwardgripper)", shape=ellipse, style=filled]; -"selector: put_down_gear3_before_changing_tool" -> "target: is_empty(outwardgripper)"; -sequence [fillcolor=orange, fontcolor=black, fontsize=9, label="sequence: put_down_gear3", shape=box, style=filled]; -"selector: put_down_gear3_before_changing_tool" -> "sequence: put_down_gear3"; -precondition [fillcolor=gray, fontcolor=black, fontsize=9, label="precondition: hold(left_hand, outwardgripper)", shape=ellipse, style=filled]; -"sequence: put_down_gear3" -> "precondition: hold(left_hand, outwardgripper)"; -precondition [fillcolor=gray, fontcolor=black, fontsize=9, label="precondition: hold(outwardgripper, gear3)", shape=ellipse, style=filled]; -"sequence: put_down_gear3" -> "precondition: hold(outwardgripper, gear3)"; -action [fillcolor=gray, fontcolor=black, fontsize=9, label="action: put_down(left_hand, outwardgripper, gear3)", shape=ellipse, style=filled]; -"sequence: put_down_gear3" -> "action: put_down(left_hand, outwardgripper, gear3)"; -selector [fillcolor=cyan, fontcolor=black, fontsize=9, label="selector: change_tool_to_clampgripper", shape=octagon, style=filled]; -"sequence: insert_shaft1_into_gearbase_hole1_corrected" -> "selector: change_tool_to_clampgripper"; -target [fillcolor=gray, fontcolor=black, fontsize=9, label="target: hold(left_hand, clampgripper)", shape=ellipse, style=filled]; -"selector: change_tool_to_clampgripper" -> "target: hold(left_hand, clampgripper)"; -sequence [fillcolor=orange, fontcolor=black, fontsize=9, label="sequence: change_tool_to_clampgripper", shape=box, style=filled]; -"selector: change_tool_to_clampgripper" -> "sequence: change_tool_to_clampgripper"; -precondition [fillcolor=gray, fontcolor=black, fontsize=9, label="precondition: hold(left_hand, outwardgripper)*", shape=ellipse, style=filled]; -"sequence: change_tool_to_clampgripper" -> "precondition: hold(left_hand, outwardgripper)*"; -precondition [fillcolor=gray, fontcolor=black, fontsize=9, label="precondition: is_empty(outwardgripper)", shape=ellipse, style=filled]; -"sequence: change_tool_to_clampgripper" -> "precondition: is_empty(outwardgripper)"; -action [fillcolor=gray, fontcolor=black, fontsize=9, label="action: change_tool(left_hand, outwardgripper, clampgripper)", shape=ellipse, style=filled]; -"sequence: change_tool_to_clampgripper" -> "action: change_tool(left_hand, outwardgripper, clampgripper)"; -selector [fillcolor=cyan, fontcolor=black, fontsize=9, label="selector: pick_up_shaft1_with_clampgripper", shape=octagon, style=filled]; -"sequence: insert_shaft1_into_gearbase_hole1_corrected" -> "selector: pick_up_shaft1_with_clampgripper"; -target [fillcolor=gray, fontcolor=black, fontsize=9, label="target: hold(clampgripper, shaft1)", shape=ellipse, style=filled]; -"selector: pick_up_shaft1_with_clampgripper" -> "target: hold(clampgripper, shaft1)"; -sequence [fillcolor=orange, fontcolor=black, fontsize=9, label="sequence: pick_up_shaft1_with_clampgripper", shape=box, style=filled]; -"selector: pick_up_shaft1_with_clampgripper" -> "sequence: pick_up_shaft1_with_clampgripper"; -precondition [fillcolor=gray, fontcolor=black, fontsize=9, label="precondition: hold(left_hand, clampgripper)", shape=ellipse, style=filled]; -"sequence: pick_up_shaft1_with_clampgripper" -> "precondition: hold(left_hand, clampgripper)"; -precondition [fillcolor=gray, fontcolor=black, fontsize=9, label="precondition: is_empty(clampgripper)", shape=ellipse, style=filled]; -"sequence: pick_up_shaft1_with_clampgripper" -> "precondition: is_empty(clampgripper)"; -action [fillcolor=gray, fontcolor=black, fontsize=9, label="action: pick_up(left_hand, clampgripper, shaft1)", shape=ellipse, style=filled]; -"sequence: pick_up_shaft1_with_clampgripper" -> "action: pick_up(left_hand, clampgripper, shaft1)"; -selector [fillcolor=cyan, fontcolor=black, fontsize=9, label="selector: insert_shaft1_into_gearbase_hole1_action", shape=octagon, style=filled]; -"sequence: insert_shaft1_into_gearbase_hole1_corrected" -> "selector: insert_shaft1_into_gearbase_hole1_action"; -target [fillcolor=gray, fontcolor=black, fontsize=9, label="target: is_inserted_to(shaft1, gearbase_hole1)*", shape=ellipse, style=filled]; -"selector: insert_shaft1_into_gearbase_hole1_action" -> "target: is_inserted_to(shaft1, gearbase_hole1)*"; -sequence [fillcolor=orange, fontcolor=black, fontsize=9, label="sequence: insert_shaft1_into_gearbase_hole1_action", shape=box, style=filled]; -"selector: insert_shaft1_into_gearbase_hole1_action" -> "sequence: insert_shaft1_into_gearbase_hole1_action"; -precondition [fillcolor=gray, fontcolor=black, fontsize=9, label="precondition: hold(left_hand, clampgripper)*", shape=ellipse, style=filled]; -"sequence: insert_shaft1_into_gearbase_hole1_action" -> "precondition: hold(left_hand, clampgripper)*"; -precondition [fillcolor=gray, fontcolor=black, fontsize=9, label="precondition: hold(clampgripper, shaft1)", shape=ellipse, style=filled]; -"sequence: insert_shaft1_into_gearbase_hole1_action" -> "precondition: hold(clampgripper, shaft1)"; -action [fillcolor=gray, fontcolor=black, fontsize=9, label="action: insert(left_hand, clampgripper, shaft1, gearbase_hole1)", shape=ellipse, style=filled]; -"sequence: insert_shaft1_into_gearbase_hole1_action" -> "action: insert(left_hand, clampgripper, shaft1, gearbase_hole1)"; -} diff --git a/experiments/gearset1/selector_insert_shaft1_into_gearbase_hole1_corrected.png b/experiments/gearset1/selector_insert_shaft1_into_gearbase_hole1_corrected.png deleted file mode 100644 index bb6e2c9c..00000000 Binary files a/experiments/gearset1/selector_insert_shaft1_into_gearbase_hole1_corrected.png and /dev/null differ diff --git a/experiments/gearset1/selector_insert_shaft1_into_gearbase_hole1_corrected.svg b/experiments/gearset1/selector_insert_shaft1_into_gearbase_hole1_corrected.svg deleted file mode 100644 index b64f580b..00000000 --- a/experiments/gearset1/selector_insert_shaft1_into_gearbase_hole1_corrected.svg +++ /dev/null @@ -1,361 +0,0 @@ - - - - - - -pastafarianism - - - -selector - -selector: insert_shaft1_into_gearbase_hole1_action - - - -target - -target: is_inserted_to(shaft1, gearbase_hole1)* - - - -selector: insert_shaft1_into_gearbase_hole1_corrected - -selector: insert_shaft1_into_gearbase_hole1_corrected - - - -target: is_inserted_to(shaft1, gearbase_hole1) - -target: is_inserted_to(shaft1, gearbase_hole1) - - - -selector: insert_shaft1_into_gearbase_hole1_corrected->target: is_inserted_to(shaft1, gearbase_hole1) - - - - - -sequence: insert_shaft1_into_gearbase_hole1_corrected - -sequence: insert_shaft1_into_gearbase_hole1_corrected - - - -selector: insert_shaft1_into_gearbase_hole1_corrected->sequence: insert_shaft1_into_gearbase_hole1_corrected - - - - - -sequence - -sequence: insert_shaft1_into_gearbase_hole1_action - - - -selector: put_down_gear3_before_changing_tool - -selector: put_down_gear3_before_changing_tool - - - -sequence: insert_shaft1_into_gearbase_hole1_corrected->selector: put_down_gear3_before_changing_tool - - - - - -selector: change_tool_to_clampgripper - -selector: change_tool_to_clampgripper - - - -sequence: insert_shaft1_into_gearbase_hole1_corrected->selector: change_tool_to_clampgripper - - - - - -selector: pick_up_shaft1_with_clampgripper - -selector: pick_up_shaft1_with_clampgripper - - - -sequence: insert_shaft1_into_gearbase_hole1_corrected->selector: pick_up_shaft1_with_clampgripper - - - - - -selector: insert_shaft1_into_gearbase_hole1_action - -selector: insert_shaft1_into_gearbase_hole1_action - - - -sequence: insert_shaft1_into_gearbase_hole1_corrected->selector: insert_shaft1_into_gearbase_hole1_action - - - - - -target: is_empty(outwardgripper) - -target: is_empty(outwardgripper) - - - -selector: put_down_gear3_before_changing_tool->target: is_empty(outwardgripper) - - - - - -sequence: put_down_gear3 - -sequence: put_down_gear3 - - - -selector: put_down_gear3_before_changing_tool->sequence: put_down_gear3 - - - - - -precondition: hold(left_hand, outwardgripper) - -precondition: hold(left_hand, outwardgripper) - - - -sequence: put_down_gear3->precondition: hold(left_hand, outwardgripper) - - - - - -precondition: hold(outwardgripper, gear3) - -precondition: hold(outwardgripper, gear3) - - - -sequence: put_down_gear3->precondition: hold(outwardgripper, gear3) - - - - - -action: put_down(left_hand, outwardgripper, gear3) - -action: put_down(left_hand, outwardgripper, gear3) - - - -sequence: put_down_gear3->action: put_down(left_hand, outwardgripper, gear3) - - - - - -precondition - -precondition: hold(clampgripper, shaft1) - - - -action - -action: insert(left_hand, clampgripper, shaft1, gearbase_hole1) - - - -target: hold(left_hand, clampgripper) - -target: hold(left_hand, clampgripper) - - - -selector: change_tool_to_clampgripper->target: hold(left_hand, clampgripper) - - - - - -sequence: change_tool_to_clampgripper - -sequence: change_tool_to_clampgripper - - - -selector: change_tool_to_clampgripper->sequence: change_tool_to_clampgripper - - - - - -precondition: hold(left_hand, outwardgripper)* - -precondition: hold(left_hand, outwardgripper)* - - - -sequence: change_tool_to_clampgripper->precondition: hold(left_hand, outwardgripper)* - - - - - -precondition: is_empty(outwardgripper) - -precondition: is_empty(outwardgripper) - - - -sequence: change_tool_to_clampgripper->precondition: is_empty(outwardgripper) - - - - - -action: change_tool(left_hand, outwardgripper, clampgripper) - -action: change_tool(left_hand, outwardgripper, clampgripper) - - - -sequence: change_tool_to_clampgripper->action: change_tool(left_hand, outwardgripper, clampgripper) - - - - - -target: hold(clampgripper, shaft1) - -target: hold(clampgripper, shaft1) - - - -selector: pick_up_shaft1_with_clampgripper->target: hold(clampgripper, shaft1) - - - - - -sequence: pick_up_shaft1_with_clampgripper - -sequence: pick_up_shaft1_with_clampgripper - - - -selector: pick_up_shaft1_with_clampgripper->sequence: pick_up_shaft1_with_clampgripper - - - - - -precondition: hold(left_hand, clampgripper) - -precondition: hold(left_hand, clampgripper) - - - -sequence: pick_up_shaft1_with_clampgripper->precondition: hold(left_hand, clampgripper) - - - - - -precondition: is_empty(clampgripper) - -precondition: is_empty(clampgripper) - - - -sequence: pick_up_shaft1_with_clampgripper->precondition: is_empty(clampgripper) - - - - - -action: pick_up(left_hand, clampgripper, shaft1) - -action: pick_up(left_hand, clampgripper, shaft1) - - - -sequence: pick_up_shaft1_with_clampgripper->action: pick_up(left_hand, clampgripper, shaft1) - - - - - -target: is_inserted_to(shaft1, gearbase_hole1)* - -target: is_inserted_to(shaft1, gearbase_hole1)* - - - -selector: insert_shaft1_into_gearbase_hole1_action->target: is_inserted_to(shaft1, gearbase_hole1)* - - - - - -sequence: insert_shaft1_into_gearbase_hole1_action - -sequence: insert_shaft1_into_gearbase_hole1_action - - - -selector: insert_shaft1_into_gearbase_hole1_action->sequence: insert_shaft1_into_gearbase_hole1_action - - - - - -precondition: hold(left_hand, clampgripper)* - -precondition: hold(left_hand, clampgripper)* - - - -sequence: insert_shaft1_into_gearbase_hole1_action->precondition: hold(left_hand, clampgripper)* - - - - - -precondition: hold(clampgripper, shaft1) - -precondition: hold(clampgripper, shaft1) - - - -sequence: insert_shaft1_into_gearbase_hole1_action->precondition: hold(clampgripper, shaft1) - - - - - -action: insert(left_hand, clampgripper, shaft1, gearbase_hole1) - -action: insert(left_hand, clampgripper, shaft1, gearbase_hole1) - - - -sequence: insert_shaft1_into_gearbase_hole1_action->action: insert(left_hand, clampgripper, shaft1, gearbase_hole1) - - - - - diff --git a/experiments/gearset1/sk_gen_test.py b/experiments/gearset1/sk_gen_test.py deleted file mode 100644 index 72dba16e..00000000 --- a/experiments/gearset1/sk_gen_test.py +++ /dev/null @@ -1,621 +0,0 @@ -import json -import os -from pprint import pprint - -""" -test skeleton behavior tree generation. -the btw data structure should be updated. -""" - -os.environ["LANGCHAIN_TRACING_V2"] = "true" -os.environ["LANGCHAIN_ENDPOINT"] = "https://api.smith.langchain.com" -os.environ["LANGCHAIN_PROJECT"] = "default" - -from kios_bt.bt_stewardship import BehaviorTreeStewardship -from kios_scene.scene_factory import SceneFactory -from kios_bt.bt_factory import BehaviorTreeFactory -from kios_robot.robot_interface import RobotInterface -from kios_world.world_interface import WorldInterface - -from kios_agent.llm_supporter import KiosLLMSupporter -from kios_agent.data_types import KiosPromptSkeleton - -from kios_agent.kios_tools import ( - BehaviorTreeExecutorTool, - BehaviorTreeSimulatorTool, - WorldStateQueryTool, -) - -from typing import List, Tuple, Annotated, TypedDict -import operator - -from dotenv import load_dotenv - -from langchain import hub - -from langchain_openai import ChatOpenAI - -from langchain.chains.openai_functions import ( - create_structured_output_runnable, - create_openai_fn_runnable, -) -from langchain_core.utils.function_calling import convert_to_openai_function -from langchain_core.agents import AgentActionMessageLog, AgentFinish -from langchain_core.messages import HumanMessage -from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder -from langchain_core.output_parsers import JsonOutputParser, StrOutputParser -from langchain_core.pydantic_v1 import BaseModel, Field - -from langchain.prompts.pipeline import PipelinePromptTemplate -from langchain.prompts.prompt import PromptTemplate - -from langgraph.graph import StateGraph, END - -from langsmith import traceable - -load_dotenv() - -from kios_utils.pybt_test import generate_bt_stewardship, render_dot_tree -from kios_utils.pddl_problem_parser import parse_problem_init, parse_problem_objects - - -def render_bt(bt_json: json): - test_class = BehaviorTreeFactory() - bt = test_class.from_json_to_simple_bt(bt_json) - # bt = test_class.from_json_to_tree_root(bt_json) - bt_stewardship = generate_bt_stewardship(bt) - # bt_stewardship.setup(timeout=15) - render_dot_tree(bt_stewardship) - - -####################### dirs -current_dir = os.path.dirname(os.path.abspath(__file__)) -scene_path = os.path.join(current_dir, "scene.json") -# bt_json_file_path = os.path.join(current_dir, "behavior_tree.json") -world_state_path = os.path.join(current_dir, "world_state.json") -problem_path = os.path.join(current_dir, "gearset.problem") -domain_knowledge_path = os.path.join(current_dir, "domain_knowledge.txt") - -####################### problem -with open(problem_path, "r") as file: - problem = file.read() - -####################### scene -with open(scene_path, "r") as file: - scene_json_object = json.load(file) - -scene = SceneFactory().create_scene_from_json(scene_json_object) - -####################### world -world_interface = WorldInterface() -with open(world_state_path, "r") as file: - world_state_json_object = json.load(file) - world_interface.load_world_from_json(world_state_json_object) - -from_problem = parse_problem_init(problem=problem) -objects = parse_problem_objects(problem=problem) -world_interface.update_world(from_problem) - -world_state_json = world_interface.get_world_to_json() -pprint(from_problem) -pprint(world_state_json) -pprint(objects) - - -####################### robot -robot_interface = RobotInterface( - robot_address="127.0.0.1", - robot_port=12000, -) -robot_interface.setup_scene(scene) - -####################### bt_factory -bt_factory = BehaviorTreeFactory( - world_interface=world_interface, - robot_interface=robot_interface, -) - -####################### behavior_tree_stewardship -behavior_tree_stewardship = BehaviorTreeStewardship( - behaviortree_factory=bt_factory, - world_interface=world_interface, - robot_interface=robot_interface, -) - -####################### * tools -# bt_executor_tool = BehaviorTreeExecutorTool(behavior_tree_stewardship) - -bt_sim_tool = BehaviorTreeSimulatorTool(metadata={"bt_stw": behavior_tree_stewardship}) - -# world_state_query_tool = WorldStateQueryTool(behavior_tree_stewardship) - -executor_tools = [bt_sim_tool] - -# * kios data prompt skeleton dir -data_dir = os.environ.get("KIOS_DATA_DIR").format(username=os.getlogin()) -print(data_dir) -prompt_sk_dir = os.path.join(data_dir, "prompt_skeletons") -prompt_dir = os.path.join(data_dir, "prompts") - -# * refiner chain -llm_spt_re_sk = KiosLLMSupporter() -with open(os.path.join(prompt_sk_dir, "skeleton_refiner.json"), "r") as f: - re_sk_pptsk_json = json.load(f) - re_sk_pptsk = KiosPromptSkeleton.from_json(re_sk_pptsk_json) - llm_spt_re_sk.initialize_from_prompt_skeleton(re_sk_pptsk) -re_sk_ppt = llm_spt_re_sk.create_prompt() -re_sk_llm = ChatOpenAI(model_name=llm_spt_re_sk.model_name, temperature=0) - -re_sk_chain = re_sk_ppt | re_sk_llm | JsonOutputParser() - - -# * skeleton generator chain -llm_spt_skeleton_generator = KiosLLMSupporter() -with open(os.path.join(prompt_sk_dir, "skeleton_generator.json"), "r") as f: - skeleton_generator_pptsk_json = json.load(f) - skeleton_generator_pptsk = KiosPromptSkeleton.from_json( - skeleton_generator_pptsk_json - ) - llm_spt_skeleton_generator.initialize_from_prompt_skeleton(skeleton_generator_pptsk) -skeleton_generator_ppt = llm_spt_skeleton_generator.create_prompt() -skeleton_generator_llm = ChatOpenAI( - model_name=llm_spt_skeleton_generator.model_name, temperature=0 -) - -skeleton_generator_chain = ( - skeleton_generator_ppt - | skeleton_generator_llm - # | StrOutputParser() - | JsonOutputParser() -) - - -# * the graph state -class PlanExecuteState(TypedDict): - plan: List[str] - behavior_tree: dict - world_state: Annotated[ - List[dict], operator.add - ] # ! to add, you need to make world_state a list of dict - # = Field(default=None) - past_steps: Annotated[List[Tuple], operator.add] - response: str # ! not sure if use this or not. - user_input: str - problem: str - objects: dict[str, list[str]] - - -# * output schema of the planner -class SolutionSubtree(BaseModel): - """solution subtree that will be used to replace the failed condition node in the last behavior tree""" - - subtree: dict = Field( - description="a subtree that adheres the required format, which will be used to replace the failed condition node in the last behavior tree" - ) - - -behaviortree_file = os.path.join(prompt_dir, "rec_sk_gen/behaviortree.txt") -template_file = os.path.join(prompt_dir, "rec_sk_gen/template.txt") -task_file = os.path.join(prompt_dir, "rec_sk_gen/task.txt") -system_file = os.path.join(prompt_dir, "rec_sk_gen/system.txt") -object_file = os.path.join(prompt_dir, "rec_sk_gen/object.txt") -domain_file = os.path.join(prompt_dir, "rec_sk_gen/domain.txt") -example_file = os.path.join(prompt_dir, "rec_sk_gen/example.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {behaviortree} - - {domain} - - {example} - - {template} - """ -) - -planner_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ], -) - -planner = create_structured_output_runnable( - PlanExecuteState, - ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), - planner_ppt_ppl, -) - - -##################################################### * planner -# * output schema of the planner -class Plan(BaseModel): - """Plan to follow in future""" - - steps: List[str] = Field( - description="a list of different steps to follow, should be in sorted order" - ) - - -template_file = os.path.join(prompt_dir, "planner/template.txt") -task_file = os.path.join(prompt_dir, "planner/task.txt") -system_file = os.path.join(prompt_dir, "planner/system.txt") -domain_file = os.path.join(prompt_dir, "planner/domain.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {template} - """ -) - -planner_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ], -) - -planner = create_structured_output_runnable( - Plan, ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), planner_ppt_ppl -) - -##################################################### * plan_updaterner - -template_file = os.path.join(prompt_dir, "plan_updater/template.txt") -task_file = os.path.join(prompt_dir, "plan_updater/task.txt") -system_file = os.path.join(prompt_dir, "plan_updater/system.txt") -domain_file = os.path.join(prompt_dir, "plan_updater/domain.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {template} - """ -) - -plan_updater_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ], -) - -plan_updater = create_structured_output_runnable( - Plan, ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), plan_updater_ppt_ppl -) -# ! create openai fn runnable has bug - - -##################################################### * graph node functions -@traceable(name="user_input_node_step") -async def user_input_step(state: PlanExecuteState): - """ - get the input from the user - """ - print(f"-----user_input_step-----") - # if state["response"] is not None: - # user_input = input(state["response"]) - # state["response"] = None - # else: - user_input = input("Your next instruction: ") - - return { - "user_input": user_input, - } - - -@traceable(name="behavior_tree_generate_step") -async def behavior_tree_generate_step(state: PlanExecuteState): - """ - generate the behavior tree based on the instruction - """ - print(f"-----behavior_tree_generate_step-----") - - instruction = state["plan"][0] - latest_world_state = state["world_state"][-1] - objects = state["objects"] - - skeleton = await skeleton_generator_chain.ainvoke( - { - # "problem": state["problem"], - "objects": objects, - "world_state": latest_world_state, - "instructions": instruction, - } - ) - - # sk_json = JsonOutputParser().invoke(skeleton) - sk_json = skeleton - - behavior_tree_sk = sk_json["task_plan"]["behavior_tree"] - - render_bt(behavior_tree_sk) - - pause = input("paused here") - - # ! OBJECT CONCENTRATION - - # response = await re_sk_chain.ainvoke( - # { - # "input": skeleton, - # } - # ) - - behavior_tree = skeleton.get("task_plan").get("behavior_tree") - - # init_world_state = [skeleton.get("initial_state")] - - return { - "behavior_tree": behavior_tree, - # "world_state": init_world_state, # ! this is necessary because the constraints in problem - } - - -@traceable(name="behavior_tree_sim_step") -async def behavior_tree_sim_step(state: PlanExecuteState): - """ - simulate the behavior tree based on the world state - """ - print(f"-----behavior_tree_sim_step-----") - - behavior_tree = state["behavior_tree"] - latest_world_state = state["world_state"][-1] - - # exe_input = { - # "behavior_tree": behavior_tree, - # "world_state": latest_world_state, - # } - - # agent_response = await bt_sim_tool.ainvoke( - # { - # "input": exe_input, - # } - # ) - - response = behavior_tree_stewardship.sk_fake_run( - world_state=latest_world_state, bt_json=behavior_tree - ) - - # hasSucceeded = response["hasSucceeded"] - # new_world_state = [response["world_state"]] - hasSucceeded = response.result - new_world_state = [response.world_state] - the_step = state["plan"][0] - - return { - "past_steps": ( - the_step, - hasSucceeded, - ), # * only one because new plan will be generated and old steps are all removed - "world_state": new_world_state, - } - - -@traceable(name="behavior_tree_execute_step") -async def behavior_tree_execute_step(state: PlanExecuteState): - """ - execute the first step of the plan, append the result to the past steps - """ - print(f"-----behavior_tree_execute_step-----") - - behavior_tree = state["behavior_tree"] - latest_world_state = state["world_state"][-1] - - # exe_input = { - # "behavior_tree": behavior_tree, - # "world_state": latest_world_state, - # } - - # agent_response = await bt_exe_agent_executor.ainvoke( - # { - # "input": exe_input, - # } - # ) - - response = behavior_tree_stewardship.sk_fake_run( - world_state=latest_world_state, bt_json=behavior_tree - ) - - # hasSucceeded = response["hasSucceeded"] - # new_world_state = [response["world_state"]] - hasSucceeded = response.result - new_world_state = [response.world_state] - the_step = state["plan"][0] - - return { - "past_steps": ( - the_step, - hasSucceeded, - ), # * only one because new plan will be generated and old steps are all removed - "world_state": new_world_state, - } - - -@traceable(name="planner_step") -async def planner_step(state: PlanExecuteState): - """ - plan the steps based on user input and world state - """ - print(f"-----plan_step-----") - - plan = await planner.ainvoke( - { - "user_input": state["user_input"], - "world_state": state["world_state"], - } - ) - return {"plan": plan.steps} - - -@traceable(name="plan_updater_step") -async def plan_updater_step(state: PlanExecuteState): - """ - if return a response, then success, response the use, end. - otherwise, return the updated newplan (normally the same as the old plan with the first step popped out. - """ - print(f"-----plan_updater_step-----") - - # ! BOOOM! BUG! - output = await plan_updater.ainvoke( - { - "user_input": state["user_input"], - "plan": state["plan"], - "world_state": state["world_state"], - "past_steps": state["past_steps"], - } - ) - # if isinstance( - # output, UpdaterResponse - # ): # * determine if it is time to response and end - # return { - # "response": "Your last instruction has been finished." - # } # * Don't need to update. - # else: - return {"plan": output.steps} # * update the plan - - -##################################################### * construct the graph - -workflow = StateGraph(PlanExecuteState) - -workflow.add_node("planner", planner_step) - -workflow.add_node("behavior_tree_generator", behavior_tree_generate_step) - -workflow.add_node("behavior_tree_executor", behavior_tree_execute_step) - -workflow.add_node("plan_updater", plan_updater_step) - -workflow.add_node("user_input_node", user_input_step) - -workflow.set_entry_point("user_input_node") - -workflow.add_edge( - "planner", "behavior_tree_generator" -) # ! may need to add a condition here - -workflow.add_edge("behavior_tree_generator", "behavior_tree_executor") - -workflow.add_edge("behavior_tree_executor", "plan_updater") - - -def plan_updater_should_end(state: PlanExecuteState): - """ - end router - """ - print(f"-----plan_updater_should_end-----") - - # if state["response"]: - # return True - # else: - # return False - if state["plan"] == [] or len(state["plan"]) == 0: - return True - else: - return False - - -workflow.add_conditional_edges( - "plan_updater", - plan_updater_should_end, - { - True: "user_input_node", - False: "behavior_tree_generator", - }, -) - - -def user_input_should_end(state: PlanExecuteState): - """ - if the user input is empty, then end - """ - print(f"-----user_input_should_end-----") - - # print(f'user_input: {state["user_input"]}') - # print(f'world_state: {state["world_state"]}') - - if not state["user_input"] or state["user_input"] == "": - return True - else: - return False - - -workflow.add_conditional_edges( - "user_input_node", - user_input_should_end, - { - True: END, - False: "planner", - }, -) - -app = workflow.compile() - -config = {"recursion_limit": 500} - -inputs = { - "world_state": [world_state_json], - "objects": objects, -} - - -async def core_run(): - async for event in app.astream( - inputs, - config=config, - ): - for k, v in event.items(): - if k != "__end__": - print(v) - - -if __name__ == "__main__": - import asyncio - - asyncio.run(core_run()) diff --git a/experiments/gearset1/sot.py b/experiments/gearset1/sot.py deleted file mode 100644 index b827b4de..00000000 --- a/experiments/gearset1/sot.py +++ /dev/null @@ -1,1206 +0,0 @@ -import json -import os -import re -from pprint import pprint -from typing import List, Tuple, Annotated, TypedDict -import operator -import logging -import asyncio - -""" -skeleton of thought work script. -copied from the recursive_generation.py and has not been fully modified yet. -""" - -os.environ["LANGCHAIN_TRACING_V2"] = "true" -os.environ["LANGCHAIN_ENDPOINT"] = "https://api.smith.langchain.com" -os.environ["LANGCHAIN_PROJECT"] = "kios_sot" - -from kios_bt.bt_stewardship import BehaviorTreeStewardship -from kios_scene.scene_factory import SceneFactory -from kios_bt.bt_factory import BehaviorTreeFactory -from kios_robot.robot_interface import RobotInterface -from kios_world.world_interface import WorldInterface - -from kios_agent.llm_supporter import KiosLLMSupporter -from kios_agent.data_types import KiosPromptSkeleton - -from kios_agent.kios_tools import ( - BehaviorTreeExecutorTool, - BehaviorTreeSimulatorTool, - WorldStateQueryTool, -) - -from dotenv import load_dotenv - -from langchain import hub - -from langchain_openai import ChatOpenAI - -from langchain.chains.openai_functions import ( - create_structured_output_runnable, - create_openai_fn_runnable, -) -from langchain_core.utils.function_calling import convert_to_openai_function -from langchain_core.agents import AgentActionMessageLog, AgentFinish -from langchain_core.messages import HumanMessage -from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder -from langchain_core.output_parsers import JsonOutputParser, StrOutputParser -from langchain_core.pydantic_v1 import BaseModel, Field - -from langchain.agents.format_scratchpad import format_to_openai_function_messages -from langchain.agents import create_openai_functions_agent, AgentExecutor - -from langchain.prompts.pipeline import PipelinePromptTemplate -from langchain.prompts.prompt import PromptTemplate - -from langchain_community.document_loaders import TextLoader -from langchain_openai import OpenAIEmbeddings - -from langchain_community.vectorstores import FAISS -from langchain.text_splitter import ( - RecursiveCharacterTextSplitter, - CharacterTextSplitter, -) - -from langgraph.graph import StateGraph, END - -from langsmith import traceable - -load_dotenv() - -from kios_utils.pybt_test import generate_bt_stewardship, render_dot_tree -from kios_utils.pddl_problem_parser import parse_problem_init, parse_problem_objects - - -def render_bt(bt_json: dict): - test_class = BehaviorTreeFactory() - bt = test_class.from_json_to_simple_bt(bt_json) - # bt = test_class.from_json_to_tree_root(bt_json) - bt_stewardship = generate_bt_stewardship(bt) - # bt_stewardship.setup(timeout=15) - render_dot_tree(bt_stewardship) - - -####################### dirs -current_dir = os.path.dirname(os.path.abspath(__file__)) -scene_path = os.path.join(current_dir, "scene.json") -# bt_json_file_path = os.path.join(current_dir, "behavior_tree.json") -world_state_path = os.path.join(current_dir, "world_state.json") -problem_path = os.path.join(current_dir, "gearset.problem") -domain_knowledge_path = os.path.join(current_dir, "domain_knowledge.txt") - -####################### problem -with open(problem_path, "r") as file: - problem = file.read() - -####################### scene -with open(scene_path, "r") as file: - scene_json_object = json.load(file) - -scene = SceneFactory().create_scene_from_json(scene_json_object) - -####################### world -world_interface = WorldInterface() -with open(world_state_path, "r") as file: - world_state_json_object = json.load(file) - world_interface.load_world_from_json(world_state_json_object) - -from_problem = parse_problem_init(problem=problem) -objects = parse_problem_objects(problem=problem) -world_interface.update_world(from_problem) - -world_state_json = world_interface.get_world_to_json() -pprint(from_problem) -pprint(world_state_json) -pprint(objects) - - -####################### robot -robot_interface = RobotInterface( - robot_address="127.0.0.1", - robot_port=12000, -) -robot_interface.setup_scene(scene) - -####################### bt_factory -bt_factory = BehaviorTreeFactory( - world_interface=world_interface, - robot_interface=robot_interface, -) - -####################### behavior_tree_stewardship -behavior_tree_stewardship = BehaviorTreeStewardship( - behaviortree_factory=bt_factory, - world_interface=world_interface, - robot_interface=robot_interface, -) - -####################### * tools -# bt_executor_tool = BehaviorTreeExecutorTool(behavior_tree_stewardship) - -bt_sim_tool = BehaviorTreeSimulatorTool(metadata={"bt_stw": behavior_tree_stewardship}) - -# world_state_query_tool = WorldStateQueryTool(behavior_tree_stewardship) - -executor_tools = [bt_sim_tool] - -# * kios data prompt skeleton dir -data_dir = os.environ.get("KIOS_DATA_DIR").format(username=os.getlogin()) -print(data_dir) -prompt_sk_dir = os.path.join(data_dir, "prompt_skeletons") -prompt_dir = os.path.join(data_dir, "prompts") - - -# * the graph state -class PlanExecuteState(TypedDict): - plan: List[str] - behavior_tree: dict - world_state: Annotated[ - List[dict], operator.add - ] # ! to add, you need to make world_state a list of dict - past_steps: Annotated[List[Tuple], operator.add] - user_input: str - objects: dict[str, list[str]] - last_behavior_tree: dict - last_failed_node: dict - runtime_world_state: dict - behavior_tree_execution_summary: str - BTExecutionHasSucceeded: bool - - action_sequence: List[str] - - -########## verifier - -template_file = os.path.join(prompt_dir, "seq_planner/template.txt") -task_file = os.path.join(prompt_dir, "seq_planner/task.txt") -system_file = os.path.join(prompt_dir, "seq_planner/system.txt") -domain_file = os.path.join(prompt_dir, "seq_planner/domain.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {template} - """ -) - -seq_planner_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ], -) - -seq_planner_chain = ( - seq_planner_ppt_ppl - | ChatOpenAI(model="gpt-4-turbo-preview", temperature=0) - | StrOutputParser() -) - - -# * output schema of the recursive_sk_generator -class SolutionBehaviortree(BaseModel): - """the fixed behavior tree""" - - behavior_tree: dict = Field( - description="the behavior tree that adheres the required format." - ) - - -behaviortree_file = os.path.join(prompt_dir, "rec_sk_gen/behaviortree.txt") -template_file = os.path.join(prompt_dir, "rec_sk_gen/template.txt") -task_file = os.path.join(prompt_dir, "rec_sk_gen/task.txt") -system_file = os.path.join(prompt_dir, "rec_sk_gen/system.txt") -object_file = os.path.join(prompt_dir, "rec_sk_gen/object.txt") -domain_file = os.path.join(prompt_dir, "rec_sk_gen/domain.txt") -example_file = os.path.join(prompt_dir, "rec_sk_gen/example.txt") -state_file = os.path.join(prompt_dir, "rec_sk_gen/state.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) -with open(object_file, "r") as f: - object_ppt = PromptTemplate.from_template(f.read()) -with open(behaviortree_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - behaviortree_ppt = ppt_tmp.partial(input=f.read()) - -with open(state_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - state_ppt = ppt_tmp.partial(input=f.read()) - -with open(example_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - example_ppt = ppt_tmp.partial(input=f.read()) -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {state} - - {object} - - {behaviortree} - - {example} - - {template} - """ -) - -re_sk_gen_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ("object", object_ppt), - ("behaviortree", behaviortree_ppt), - ("example", example_ppt), - ("state", state_ppt), - ], -) - -rec_sk_gen_chain = ( - re_sk_gen_ppt_ppl - | ChatOpenAI(model="gpt-4-turbo-preview", temperature=0) - | JsonOutputParser() -) - -# recursive_sk_generator = create_structured_output_runnable( -# SolutionBehaviortree, -# ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), -# re_sk_gen_ppt_ppl, -# mode="openai-json", -# ) - - -##################################################### * planner -# * output schema of the planner -class Plan(BaseModel): - """Plan to follow in future""" - - steps: List[str] = Field( - description="a list of different steps to follow, should be in sorted order" - ) - - -template_file = os.path.join(prompt_dir, "planner/template.txt") -task_file = os.path.join(prompt_dir, "planner/task.txt") -system_file = os.path.join(prompt_dir, "planner/system.txt") -domain_file = os.path.join(prompt_dir, "planner/domain.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {template} - """ -) - -planner_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ], -) - -planner = create_structured_output_runnable( - Plan, ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), planner_ppt_ppl -) - -##################################################### * plan_updaterner - -template_file = os.path.join(prompt_dir, "plan_updater/template.txt") -task_file = os.path.join(prompt_dir, "plan_updater/task.txt") -system_file = os.path.join(prompt_dir, "plan_updater/system.txt") -domain_file = os.path.join(prompt_dir, "plan_updater/domain.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {template} - """ -) - -plan_updater_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ], -) - -plan_updater = create_structured_output_runnable( - Plan, ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), plan_updater_ppt_ppl -) -# ! create openai fn runnable has bug - - -##################################################### * graph node functions -@traceable(name="user_input_node_step") -async def user_input_step(state: PlanExecuteState): - """ - get the input from the user - """ - print(f"-----user_input_step-----") - - user_input = input("Your next instruction: ") - - return { - "user_input": user_input, - } - - -@traceable(name="sequence_generate_step") -async def sequence_generate_step(state: PlanExecuteState): - """ - generate the sequence based on the instruction - """ - print(f"-----sequence_generate_step-----") - - instruction = state["plan"][0] - start_world_state = state["world_state"][-1] - objects = state["objects"] - - action_sequence = await seq_planner_chain.ainvoke( - { - "objects": objects, - "start_world_state": start_world_state, - "user_instruction": instruction, - } - ) - - return { - "action_sequence": action_sequence, - } - - -@traceable(name="behavior_tree_generate_step") -async def behavior_tree_generate_step(state: PlanExecuteState): - """ - generate the behavior tree based on the instruction - """ - print(f"-----behavior_tree_generate_step-----") - - instruction = state["plan"][0] - if state["runtime_world_state"] is None: - runtime_world_state = state["world_state"][-1] - else: - runtime_world_state = state["runtime_world_state"] - objects = state["objects"] - - bt_skeleton = await rec_sk_gen_chain.ainvoke( - { - "objects": objects, - "runtime_world_state": runtime_world_state, - "last_behavior_tree": state["last_behavior_tree"], - "last_failed_node": state["last_failed_node"], - "summary": state["behavior_tree_execution_summary"], - "action_sequence": state["action_sequence"], - } - ) - - render_bt(bt_skeleton) - - pause = input("paused here") - - return { - "last_behavior_tree": bt_skeleton, - "BTExecutionHasSucceeded": False, - } - - -@traceable(name="behavior_tree_execute_step") -async def behavior_tree_execute_step(state: PlanExecuteState): - """ - execute the first step of the plan, append the result to the past steps - """ - print(f"-----behavior_tree_execute_step-----") - - this_step = state["plan"][0] - behavior_tree_skeleton = state["last_behavior_tree"] - latest_world_state = state["world_state"][-1] - - # * first sim run - tree_result, skeleton_json = behavior_tree_stewardship.sk_sim_run( - world_state=latest_world_state, skeleton_json=behavior_tree_skeleton - ) - - pprint(tree_result.to_json()) - pause = input("paused here") - - # * check result - if tree_result.result == "success": - - # * second fake run - tree_result, skeleton_json = behavior_tree_stewardship.sk_fake_run( - world_state=latest_world_state, skeleton_json=behavior_tree_skeleton - ) - - pprint(tree_result.to_json()) - pause = input("paused here") - - if tree_result.result == "success": - return { - "BTExecutionHasSucceeded": True, - "past_steps": ( - this_step, - tree_result.result, - ), # * only one because new plan will be generated and old steps are all removed - "world_state": [tree_result.world_state], - # * clear all - "last_behavior_tree": None, - "last_failed_node": None, - "runtime_world_state": None, - "behavior_tree_execution_summary": None, - } - else: - return { - "world_state": [tree_result.world_state], # * real world state - "last_behavior_tree": skeleton_json, - "last_failed_node": { - "summary": tree_result.final_node.get("summary"), - "name": tree_result.final_node.get("name"), - "identifier": tree_result.final_node.get("identifier"), - }, - "runtime_world_state": tree_result.world_state, - "behavior_tree_execution_summary": tree_result.summary, - } - - else: - return { - # ! do not change world state - "last_behavior_tree": skeleton_json, - "last_failed_node": { - "summary": tree_result.final_node.get("summary"), - "name": tree_result.final_node.get("name"), - "identifier": tree_result.final_node.get("identifier"), - }, - "runtime_world_state": tree_result.world_state, - "behavior_tree_execution_summary": tree_result.summary, - } - - -@traceable(name="planner_step") -async def planner_step(state: PlanExecuteState): - """ - plan the steps based on user input and world state - """ - print(f"-----plan_step-----") - - plan = await planner.ainvoke( - { - "user_input": state["user_input"], - "world_state": state["world_state"], - } - ) - return {"plan": plan.steps} - - -@traceable(name="plan_updater_step") -async def plan_updater_step(state: PlanExecuteState): - """ - if return a response, then success, response the use, end. - otherwise, return the updated newplan (normally the same as the old plan with the first step popped out. - """ - print(f"-----plan_updater_step-----") - - # ! BOOOM! BUG! - output = await plan_updater.ainvoke( - { - "user_input": state["user_input"], - "plan": state["plan"], - "world_state": state["world_state"], - "past_steps": state["past_steps"], - } - ) - # if isinstance( - # output, UpdaterResponse - # ): # * determine if it is time to response and end - # return { - # "response": "Your last instruction has been finished." - # } # * Don't need to update. - # else: - return {"plan": output.steps} # * update the plan - - -##################################################### * construct the graph - -workflow = StateGraph(PlanExecuteState) - -workflow.add_node("planner", planner_step) -workflow.add_node("sequence_generator", sequence_generate_step) -workflow.add_node("behavior_tree_generator", behavior_tree_generate_step) -workflow.add_node("behavior_tree_executor", behavior_tree_execute_step) -workflow.add_node("plan_updater", plan_updater_step) -workflow.add_node("user_input_node", user_input_step) -workflow.set_entry_point("user_input_node") -workflow.add_edge("planner", "sequence_generator") -workflow.add_edge("sequence_generator", "behavior_tree_generator") -workflow.add_edge("behavior_tree_generator", "behavior_tree_executor") - - -def executor_should_end(state: PlanExecuteState): - """ - end router - """ - print(f"-----executor_should_end-----") - - if state["BTExecutionHasSucceeded"] == True: - state["BTExecutionHasSucceeded"] = False - return True - else: - return False - - -workflow.add_conditional_edges( - "behavior_tree_executor", - executor_should_end, - { - True: "plan_updater", - False: "behavior_tree_generator", - }, -) - - -def plan_updater_should_end(state: PlanExecuteState): - """ - end router - """ - print(f"-----plan_updater_should_end-----") - - if state["plan"] == [] or len(state["plan"]) == 0: - return True - else: - return False - - -workflow.add_conditional_edges( - "plan_updater", - plan_updater_should_end, - { - True: "user_input_node", - False: "behavior_tree_generator", - }, -) - - -def user_input_should_end(state: PlanExecuteState): - """ - if the user input is empty, then end - """ - print(f"-----user_input_should_end-----") - - if not state["user_input"] or state["user_input"] == "": - return True - else: - return False - - -workflow.add_conditional_edges( - "user_input_node", - user_input_should_end, - { - True: END, - False: "planner", - }, -) - -app = workflow.compile() - -config = {"recursion_limit": 500} - -inputs = { - "world_state": [world_state_json], - "objects": objects, -} - -# * unit tree generator ppl -system_file = os.path.join(prompt_dir, "new/ut_gen/system.txt") -task_file = os.path.join(prompt_dir, "new/ut_gen/task.txt") -domain_file = os.path.join(prompt_dir, "new/ut_gen/new_domain_nl.txt") -behaviortree_file = os.path.join(prompt_dir, "new/ut_gen/new_behaviortree.txt") -template_file = os.path.join(prompt_dir, "new/ut_gen/template.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) -with open(behaviortree_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - behaviortree_ppt = ppt_tmp.partial(input=f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {behaviortree} - - {template} - - {format_instructions} - """ -) - -parser = JsonOutputParser() - -format_instructions = PromptTemplate.from_template("""{input}""").partial( - input=parser.get_format_instructions() -) - -ut_gen_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ("behaviortree", behaviortree_ppt), - ("format_instructions", format_instructions), - ], -) - -ut_gen_chain = ( - ut_gen_ppt_ppl - # | ChatOpenAI(model="gpt-3.5-turbo-0125", temperature=0) - # | ChatOpenAI(model="ft:gpt-3.5-turbo-0125:kifabrik-mirmi::8y1cXwVw", temperature=0) - | ChatOpenAI( - model="ft:gpt-3.5-turbo-0125:kifabrik-mirmi:kios-ut-gen-v2:8z2KbPsr", - temperature=0, - ) - # | ChatOpenAI(model="gpt-4", temperature=0) - | JsonOutputParser() -) - -# ! sequence planner estimation ppl -system_file = os.path.join(prompt_dir, "seq_planner_est/system.txt") -task_file = os.path.join(prompt_dir, "seq_planner_est/task.txt") -domain_file = os.path.join(prompt_dir, "seq_planner_est/domain.txt") -state_file = os.path.join(prompt_dir, "seq_planner_est/state.txt") -output_format_file = os.path.join(prompt_dir, "seq_planner_est/output_format.txt") -template_file = os.path.join(prompt_dir, "seq_planner_est/template.txt") -example_file = os.path.join(prompt_dir, "seq_planner_est/example.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) -with open(output_format_file, "r") as f: - output_format_ppt = PromptTemplate.from_template(f.read()) -with open(example_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - example_ppt = ppt_tmp.partial(input=f.read()) -with open(state_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - state_ppt = ppt_tmp.partial(input=f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {state} - - {output_format} - - {example} - - {template} - - {format_instructions} - """ -) - -parser = JsonOutputParser() - -format_instructions = PromptTemplate.from_template("""{input}""").partial( - input=parser.get_format_instructions() -) - -seq_planner_est_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ("state", state_ppt), - ("output_format", output_format_ppt), - ("format_instructions", format_instructions), - ("example", example_ppt), - ], -) - -seq_planner_est_chain = ( - seq_planner_est_ppt_ppl - # | ChatOpenAI(model="gpt-4", temperature=0) - | ChatOpenAI(model="gpt-3.5-turbo-0125", temperature=0) - | JsonOutputParser() -) - - -def seq_planner_est_test(): - return seq_planner_est_chain.invoke( - { - "start_world_state": world_state_json, - # "target": "is_inserted_to(shaft1, gearbase_hole1)", - "target": "hold(left_hand, outward_claw)", - } - ) - - -# * sequence action planner -system_file = os.path.join(prompt_dir, "new/seq_plan/system.txt") -task_file = os.path.join(prompt_dir, "new/seq_plan/task.txt") -domain_file = os.path.join(prompt_dir, "new/seq_plan/new_domain_nl.txt") -state_file = os.path.join(prompt_dir, "new/seq_plan/state.txt") -output_format_file = os.path.join(prompt_dir, "new/seq_plan/output_format.txt") -template_file = os.path.join(prompt_dir, "new/seq_plan/template.txt") -example_file = os.path.join(prompt_dir, "new/seq_plan/new_example.txt") -chain_file = os.path.join(prompt_dir, "new/seq_plan/chain.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) -with open(output_format_file, "r") as f: - output_format_ppt = PromptTemplate.from_template(f.read()) -with open(example_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - example_ppt = ppt_tmp.partial(input=f.read()) -with open(state_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - state_ppt = ppt_tmp.partial(input=f.read()) -# # * this is for gpt 3.5 -# with open(chain_file, "r") as f: -# chain_ppt = PromptTemplate.from_template(f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {state} - - {output_format} - - {example} - - {template} - - {format_instructions} - """ -) - -parser = JsonOutputParser() - -format_instructions = PromptTemplate.from_template("""{input}""").partial( - input=parser.get_format_instructions() -) - -seq_ac_pl_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ("state", state_ppt), - ("output_format", output_format_ppt), - ("format_instructions", format_instructions), - ("example", example_ppt), - # ("chain", chain_ppt), - ], -) - -seq_ac_pl_chain = ( - seq_ac_pl_ppt_ppl - | ChatOpenAI(model="gpt-4", temperature=0) - # | ChatOpenAI(model="gpt-3.5-turbo-0125", temperature=0) - | JsonOutputParser() -) - -# * state estimater -system_file = os.path.join(prompt_dir, "new/state_est/system.txt") -task_file = os.path.join(prompt_dir, "new/state_est/task.txt") -domain_file = os.path.join(prompt_dir, "new/state_est/new_domain_nl.txt") -state_file = os.path.join(prompt_dir, "new/state_est/state.txt") -output_format_file = os.path.join(prompt_dir, "new/state_est/output_format.txt") -template_file = os.path.join(prompt_dir, "new/state_est/template.txt") -example_file = os.path.join(prompt_dir, "new/state_est/new_example.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) -with open(output_format_file, "r") as f: - output_format_ppt = PromptTemplate.from_template(f.read()) -with open(example_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - example_ppt = ppt_tmp.partial(input=f.read()) -with open(state_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - state_ppt = ppt_tmp.partial(input=f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {state} - - {output_format} - - {example} - - {template} - - {format_instructions} - """ -) - -parser = JsonOutputParser() - -format_instructions = PromptTemplate.from_template("""{input}""").partial( - input=parser.get_format_instructions() -) - -state_est_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ("state", state_ppt), - ("output_format", output_format_ppt), - ("format_instructions", format_instructions), - ("example", example_ppt), - ], -) - - -state_est_ppt_ppl_chain = ( - state_est_ppt_ppl - # | ChatOpenAI(model="gpt-3.5-turbo", temperature=0) - | ChatOpenAI(model="gpt-4", temperature=0) - | JsonOutputParser() -) - - -async def core_run(): - async for event in app.astream( - inputs, - config=config, - ): - for k, v in event.items(): - if k != "__end__": - print(v) - - -def ut_gen_test(): - ut = ut_gen_chain.invoke( - { - # "action": "insert(left_hand, defaultgripper, gear1, shaft1)", - # "action": "insert(left_hand, parallel_box1, shaft1, gearbase_hole1)", - # "action": "change_tool(left_hand, parallel_box1, defaultgripper)", - "action": "insert(left_hand, clampgripper, gear2, shaft2)", - } - ) - - render_bt(ut) - - -def seq_planner_est_test(): - return seq_planner_est_chain.invoke( - { - "start_world_state": world_state_json, - "target": "is_inserted_to(shaft1, gearbase_hole1)", - # "target": "hold(left_hand, outward_claw)", - } - ) - - -def seq_action_plan_test(): - return seq_ac_pl_chain.invoke( - { - "start_world_state": world_state_json, - "target": "is_inserted_to(shaft1, gearbase_hole1)", - # "target": "hold(left_hand, outward_claw)", - } - ) - - -def state_est_test(): - return state_est_ppt_ppl_chain.invoke( - { - "start_world_state": world_state_json, - "action_plan": [ - "unload_tool(left_hand, outward_claw)", - "load_tool(left_hand, parallel_box1)", - "pick_up(left_hand, parallel_box1, shaft1)", - "insert(left_hand, parallel_box1, shaft1, gearbase_hole1)", - ], - } - ) - - -##################################### * speed imp -@traceable(name="make_plan") -def make_plan(state: dict, goal: str) -> list[str]: - print(f"----------start to make plan for the goal {goal}") - response = seq_ac_pl_chain.invoke( - { - "start_world_state": state, - "target": goal, - } - ) - print(f"finished making plan for the goal {goal}.") - pprint(f'LLM thought flow: {response["explanation"]}') - return response["task_plan"] - - -@traceable(name="estimate_state") -def estimate_state(start_world_state: dict, action_plan: list[str]) -> dict: - print("----------start to estimate the state after the action plan:") - pprint(action_plan) - response = state_est_ppt_ppl_chain.invoke( - { - "start_world_state": start_world_state, - "action_plan": action_plan, - } - ) - print(f"finished estimating the state after the action plan {action_plan}.") - return response["estimated_world_state"] - - -@traceable(name="generate_unit_subtree") -def generate_unit_subtree(action: str) -> dict: - print("----------start to generate the unit subtree for the action") - pprint(action) - response = ut_gen_chain.invoke( - { - "action": action, - } - ) - print(f"finished generating the unit subtree for the action {action}.") - return response - - -def get_node_list_from_tree(unit_subtree: dict) -> list[dict]: - children = unit_subtree["children"][1][ - "children" - ] # * the second child is a sequence - return children - - -def extract_goal(node: dict) -> str: - name = node["name"] - - -def match_type(node: dict) -> tuple[str, str]: - node_name = node["name"] - match = re.search( - r"(selector|sequence|action|precondition|condition|target):\s*(.+)", node_name - ) - if match: - node_type = match.group(1) - node_body = match.group(2) - return node_type, node_body - else: - raise ValueError(f"the node name {node_name} does not match any type.") - - -def expand_nodes( - node_list: list[dict], - start_state: dict, - overall_tree: list[dict] = None, -) -> dict: - """ - in order to monitor the tree generation, the overall tree and the node list should be the same variable when being passed in. - """ - pprint("----------check the entire tree:") - if overall_tree is not None: - render_bt(overall_tree[0]) - pprint("----------start to expand the node list:") - pprint(node_list) - pause = input("paused here! check the tree.") - - assert len(node_list) > 0 - state = start_state - - for i in range(len(node_list)): - type_name, body = match_type(node_list[i]) - # if match_type(node_list[i]) == "action": - if type_name == "action": - print(f"the node {node_list[i]['name']} is an action node. skip it.") - pause = input("paused here! check!") - # elif match_type(node_list[i]) == "precondition" or "target": - elif type_name in ["precondition", "target"]: - # goal = node_list[i]["name"] - goal = body - plan = make_plan(state, goal) - if len(plan) == 0: - logging.warning(f"No action should be performed for the goal {goal}.") - logging.warning(f'the node {node_list[i]["name"]} has been skipped.') - pause = input("paused here! check!") - else: - logging.info(f"Actions have been planed for the goal {goal}.") - pprint(f"the plan for the goal {goal} is {plan}") - pause = input("paused here! check!") - last_action = plan[-1] - unit_subtree = generate_unit_subtree(last_action) - # insert the subtree into the node_list - node_list[i] = unit_subtree - new_node_list = get_node_list_from_tree(unit_subtree) - expand_nodes( - node_list=new_node_list, - start_state=state, - overall_tree=overall_tree, - ) - state = estimate_state(state, plan) - - return node_list[0] - - -def embed_ut_nl(unit_subtree: dict) -> str: - """ - embed the unit subtree into the overall tree - """ - selector_children = unit_subtree["children"] - target = "" - # * target - for node in selector_children: - if match_type(node)[0] == "target": - target += node["summary"] - - sequence_children = selector_children[1]["children"] - preconditions = [] - action = "" - for node in sequence_children: - if match_type(node)[0] == "precondition": - preconditions.append(node["summary"]) - if match_type(node)[0] == "action": - action = node["summary"] - - embedding = f"if {' and '.join(preconditions)} then {action}, {target}" - - return embedding - - -def test_embedding_nl(): - unit_tree = ut_gen_test() - pprint(unit_tree) - nl = embed_ut_nl(unit_tree) - print(nl) - - -def test_expand_nodes(): - start_state = world_state_json - node_list = [ - { - "summary": "insert shaft1 into gearbase hole1", - "name": "target: insert shaft1 into gearbase hole1", - } - ] - # node_list = [ - # { - # "summary": "insert gear2 into shaft2", - # "name": "target: insert gear2 into shaft2", - # } - # ] - # node_list = [ - # { - # "summary": "pick up the shaft1", - # "name": "target: pick up the shaft1", - # }, - # ] - expand_nodes(node_list, start_state, node_list) - pprint(node_list) - - -if __name__ == "__main__": - pass - - # asyncio.run(core_run()) - # pprint(ut_gen_test()) - # pprint(seq_action_plan_test()) - # pprint(state_est_test()) - - test_expand_nodes() - - # test_embedding_nl() diff --git a/experiments/gearset1/test_graph.py b/experiments/gearset1/test_graph.py deleted file mode 100644 index faf6dff7..00000000 --- a/experiments/gearset1/test_graph.py +++ /dev/null @@ -1,78 +0,0 @@ -import os -from typing import List, Tuple, Annotated, TypedDict -import operator - -""" -a test script for the functionality of langgraph -""" - -os.environ["LANGCHAIN_TRACING_V2"] = "true" -os.environ["LANGCHAIN_ENDPOINT"] = "https://api.smith.langchain.com" -os.environ["LANGCHAIN_PROJECT"] = "kios_agent" - -from langgraph.graph import StateGraph, END - -from langsmith import traceable - - -class PlanExecuteState(TypedDict): - plan: List[str] - behavior_tree_skeleton: dict # ! not sure if use this or not. - behavior_tree: dict - world_state: Annotated[ - List[dict], operator.add - ] # ! to add, you need to make world_state a list of dict - # = Field(default=None) - past_steps: Annotated[List[Tuple], operator.add] - response: str # ! not sure if use this or not. - # to_user: bool - user_input: str - problem: str - - -@traceable(name="user_input_node_step") -async def user_input_step(state: PlanExecuteState): - user_input = input("Enter your next step: ") - return { - "user_input": user_input, - } - - -@traceable(name="user_input_conditional_edge") -def user_input_should_continue(state: PlanExecuteState): - - if state["user_input"] == "gogogo": - return True - else: - return False - - -workflow = StateGraph(PlanExecuteState) - -workflow.add_node("user_input_node", user_input_step) - -workflow.set_entry_point("user_input_node") - -workflow.add_conditional_edges( - "user_input_node", - user_input_should_continue, - { - True: "user_input_node", - False: END, - }, -) - -app = workflow.compile() - - -async def core_run(): - async for event in app.astream(input={}): - for k, v in event.items(): - if k != "__end__": - print(v) - - -if __name__ == "__main__": - import asyncio - - asyncio.run(core_run()) diff --git a/experiments/gearset1/visualize_cheat.py b/experiments/gearset1/visualize_cheat.py deleted file mode 100644 index 6225e4ae..00000000 --- a/experiments/gearset1/visualize_cheat.py +++ /dev/null @@ -1,1003 +0,0 @@ -# -""" -script to visualize the tree. -a good place to go if you don't know how to do the right thing with the btw yourself. -""" -# * this is a full tree example. -result_1 = { - "behavior_tree": { - "summary": "selector to insert gear2 into shaft2", - "name": "selector: insert(left_hand, outward_claw, gear2, shaft2)", - "identifier": 1, - "type_name": "selector", - "children": [ - { - "summary": "condition node to check if gear2 is inserted to shaft2", - "name": "target: is_inserted_to(gear2, shaft2)", - "identifier": 2, - "type_name": "condition", - "conditions": [ - { - "object_name": "gear2", - "property_name": "is_inserted_to", - "property_value": "shaft2", - "status": True, - } - ], - }, - { - "summary": "sequence to insert gear2 into shaft2", - "name": "sequence: insert(left_hand, outward_claw, gear2, shaft2)", - "identifier": 3, - "type_name": "sequence", - "children": [ - { - "summary": "selector to load outward_claw", - "name": "selector: load_tool(left_hand, outward_claw)", - "identifier": 4, - "type_name": "selector", - "children": [ - { - "summary": "condition node to check if left_hand holds outward_claw", - "name": "target: hold(left_hand, outward_claw)", - "identifier": 5, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "outward_claw", - "status": True, - } - ], - }, - { - "summary": "sequence to load outward_claw", - "name": "sequence: load_tool(left_hand, outward_claw)", - "identifier": 6, - "type_name": "sequence", - "children": [ - { - "summary": "condition node to check if outward_claw is equippable", - "name": "precondition: is_equippable(outward_claw)", - "identifier": 7, - "type_name": "condition", - "conditions": [ - { - "object_name": "outward_claw", - "property_name": "is_equippable", - "property_value": None, - "status": True, - } - ], - }, - { - "summary": "condition node to check if left hand is free", - "name": "precondition: is_free(left_hand)", - "identifier": 8, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "is_free", - "property_value": None, - "status": True, - } - ], - }, - { - "summary": "action node to equip outward_claw to left hand", - "name": "action: load_tool(left_hand, outward_claw)", - "identifier": 9, - "type_name": "action", - "effects": [ - { - "object_name": "left_hand", - "property_name": "is_free", - "property_value": None, - "status": False, - }, - { - "object_name": "outward_claw", - "property_name": "is_equippable", - "property_value": None, - "status": False, - }, - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "outward_claw", - "status": True, - }, - ], - }, - ], - }, - ], - }, - { - "summary": "selector to use left_hand with outward_claw to pick up gear2", - "name": "selector: pick_up(left_hand, outward_claw, gear2)", - "identifier": 10, - "type_name": "selector", - "children": [ - { - "summary": "condition node to check if outward_claw holds gear2", - "name": "target: hold(outward_claw, gear2)", - "identifier": 11, - "type_name": "condition", - "conditions": [ - { - "object_name": "outward_claw", - "property_name": "hold", - "property_value": "gear2", - "status": True, - } - ], - }, - { - "summary": "sequence to use left hand with outward_claw to pick up gear2", - "name": "sequence: pick_up(left_hand, outward_claw, gear2)", - "identifier": 12, - "type_name": "sequence", - "children": [ - { - "summary": "condition node to check if outward_claw is free", - "name": "precondition: is_free(outward_claw)", - "identifier": 13, - "type_name": "condition", - "conditions": [ - { - "object_name": "outward_claw", - "property_name": "is_free", - "property_value": None, - "status": True, - } - ], - }, - { - "summary": "condition node to check if left hand holds outward_claw", - "name": "precondition: hold(left_hand, outward_claw)", - "identifier": 14, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "outward_claw", - "status": True, - } - ], - }, - { - "summary": "action node to use left hand with outward_claw to pick up gear2", - "name": "action: pick_up(left_hand, outward_claw, gear2)", - "identifier": 15, - "type_name": "action", - "effects": [ - { - "object_name": "outward_claw", - "property_name": "hold", - "property_value": "gear2", - "status": True, - }, - { - "object_name": "outward_claw", - "property_name": "is_free", - "property_value": None, - "status": False, - }, - ], - }, - ], - }, - ], - }, - { - "summary": "condition node to check if gear2 can be inserted to shaft2", - "name": "precondition: can_insert_to(gear2, shaft2)", - "identifier": 16, - "type_name": "condition", - "conditions": [ - { - "object_name": "gear2", - "property_name": "can_insert_to", - "property_value": "shaft2", - "status": True, - } - ], - }, - { - "summary": "action node to use left_hand with outward_claw to insert gear2 to shaft2", - "name": "action: insert(left_hand, outward_claw, gear2, shaft2)", - "identifier": 17, - "type_name": "action", - "effects": [ - { - "object_name": "outward_claw", - "property_name": "hold", - "property_value": "gear2", - "status": False, - }, - { - "object_name": "outward_claw", - "property_name": "is_free", - "property_value": None, - "status": True, - }, - { - "object_name": "gear2", - "property_name": "is_inserted_to", - "property_value": "shaft2", - "status": True, - }, - ], - }, - ], - }, - ], - }, - "world_state": [ - { - "objects": [ - {"name": "parallel_box1", "properties": ["is_free"]}, - {"name": "shaft1", "properties": []}, - {"name": "gearbase_hole1", "properties": []}, - {"name": "left_hand", "properties": []}, - ], - "constraints": [ - { - "source": "parallel_box1", - "name": "can_manipulate", - "target": "shaft1", - }, - { - "source": "shaft1", - "name": "can_insert_to", - "target": "gearbase_hole1", - }, - ], - "relations": [ - {"source": "left_hand", "name": "hold", "target": "parallel_box1"}, - { - "source": "shaft1", - "name": "is_inserted_to", - "target": "gearbase_hole1", - }, - ], - } - ], -} -# -------------------------------------------------------- -# * this is a full tree example. -result = { - "behavior_tree": { - "summary": "selector to insert shaft1 into gearbase_hole1", - "name": "selector: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)", - "identifier": 1, - "type_name": "selector", - "children": [ - { - "summary": "condition node to check if shaft1 is inserted to gearbase_hole1", - "name": "target: is_inserted_to(shaft1, gearbase_hole1)", - "identifier": 2, - "type_name": "condition", - "conditions": [ - { - "object_name": "shaft1", - "property_name": "is_inserted_to", - "property_value": "gearbase_hole1", - "status": True, - } - ], - }, - { - "summary": "sequence to insert shaft1 into gearbase_hole1", - "name": "sequence: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)", - "identifier": 3, - "type_name": "sequence", - "children": [ - { - "summary": "selector to unload outward_claw", - "name": "selector: unload_tool(left_hand, outward_claw)", - "identifier": 4, - "type_name": "selector", - "children": [ - { - "summary": "condition node to check if left_hand is free", - "name": "target: is_free(left_hand)", - "identifier": 5, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "is_free", - "property_value": None, - "status": True, - } - ], - }, - { - "summary": "sequence to unload outward_claw", - "name": "sequence: unload_tool(left_hand, outward_claw)", - "identifier": 6, - "type_name": "sequence", - "children": [ - { - "summary": "condition node to check if left hand holds outward_claw", - "name": "precondition: hold(left_hand, outward_claw)", - "identifier": 7, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "outward_claw", - "status": True, - } - ], - }, - { - "summary": "action node to unload outward_claw from left hand", - "name": "action: unload_tool(left_hand, outward_claw)", - "identifier": 8, - "type_name": "action", - "effects": [ - { - "object_name": "left_hand", - "property_name": "is_free", - "property_value": None, - "status": True, - }, - { - "object_name": "outward_claw", - "property_name": "is_equippable", - "property_value": None, - "status": True, - }, - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "outward_claw", - "status": False, - }, - ], - }, - ], - }, - ], - }, - { - "summary": "selector to load parallel_box1", - "name": "selector: load_tool(left_hand, parallel_box1)", - "identifier": 9, - "type_name": "selector", - "children": [ - { - "summary": "condition node to check if left_hand holds parallel_box1", - "name": "target: hold(left_hand, parallel_box1)", - "identifier": 10, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "parallel_box1", - "status": True, - } - ], - }, - { - "summary": "sequence to load parallel_box1", - "name": "sequence: load_tool(left_hand, parallel_box1)", - "identifier": 11, - "type_name": "sequence", - "children": [ - { - "summary": "condition node to check if parallel_box1 is equippable", - "name": "precondition: is_equippable(parallel_box1)", - "identifier": 12, - "type_name": "condition", - "conditions": [ - { - "object_name": "parallel_box1", - "property_name": "is_equippable", - "property_value": None, - "status": True, - } - ], - }, - { - "summary": "condition node to check if left hand is free", - "name": "precondition: is_free(left_hand)", - "identifier": 13, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "is_free", - "property_value": None, - "status": True, - } - ], - }, - { - "summary": "action node to equip parallel_box1 to left hand", - "name": "action: load_tool(left_hand, parallel_box1)", - "identifier": 14, - "type_name": "action", - "effects": [ - { - "object_name": "left_hand", - "property_name": "is_free", - "property_value": None, - "status": False, - }, - { - "object_name": "parallel_box1", - "property_name": "is_equippable", - "property_value": None, - "status": False, - }, - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "parallel_box1", - "status": True, - }, - ], - }, - ], - }, - ], - }, - { - "summary": "selector to pick up shaft1 with parallel_box1", - "name": "selector: pick_up(left_hand, parallel_box1, shaft1)", - "identifier": 15, - "type_name": "selector", - "children": [ - { - "summary": "condition node to check if parallel_box1 holds shaft1", - "name": "target: hold(parallel_box1, shaft1)", - "identifier": 16, - "type_name": "condition", - "conditions": [ - { - "object_name": "parallel_box1", - "property_name": "hold", - "property_value": "shaft1", - "status": True, - } - ], - }, - { - "summary": "sequence to pick up shaft1 with parallel_box1", - "name": "sequence: pick_up(left_hand, parallel_box1, shaft1)", - "identifier": 17, - "type_name": "sequence", - "children": [ - { - "summary": "condition node to check if parallel_box1 is free", - "name": "precondition: is_free(parallel_box1)", - "identifier": 18, - "type_name": "condition", - "conditions": [ - { - "object_name": "parallel_box1", - "property_name": "is_free", - "property_value": None, - "status": True, - } - ], - }, - { - "summary": "condition node to check if left hand holds parallel_box1", - "name": "precondition: hold(left_hand, parallel_box1)", - "identifier": 19, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "parallel_box1", - "status": True, - } - ], - }, - { - "summary": "action node to pick up shaft1 with parallel_box1", - "name": "action: pick_up(left_hand, parallel_box1, shaft1)", - "identifier": 20, - "type_name": "action", - "effects": [ - { - "object_name": "parallel_box1", - "property_name": "hold", - "property_value": "shaft1", - "status": True, - }, - { - "object_name": "parallel_box1", - "property_name": "is_free", - "property_value": None, - "status": False, - }, - ], - }, - ], - }, - ], - }, - { - "summary": "condition node to check if shaft1 can be inserted to gearbase_hole1", - "name": "precondition: can_insert_to(shaft1, gearbase_hole1)", - "identifier": 21, - "type_name": "condition", - "conditions": [ - { - "object_name": "shaft1", - "property_name": "can_insert_to", - "property_value": "gearbase_hole1", - "status": True, - } - ], - }, - { - "summary": "action node to insert shaft1 into gearbase_hole1", - "name": "action: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)", - "identifier": 22, - "type_name": "action", - "effects": [ - { - "object_name": "parallel_box1", - "property_name": "hold", - "property_value": "shaft1", - "status": False, - }, - { - "object_name": "parallel_box1", - "property_name": "is_free", - "property_value": None, - "status": True, - }, - { - "object_name": "shaft1", - "property_name": "is_inserted_to", - "property_value": "gearbase_hole1", - "status": True, - }, - ], - }, - ], - }, - ], - }, - "world_state": [ - { - "objects": [ - {"name": "parallel_box1", "properties": ["is_free", "is_equippable"]}, - {"name": "parallel_box2", "properties": ["is_free", "is_equippable"]}, - {"name": "inward_claw", "properties": ["is_free", "is_equippable"]}, - {"name": "outward_claw", "properties": ["is_free"]}, - {"name": "no_tool", "properties": ["is_free", "is_equippable"]}, - {"name": "gear1", "properties": []}, - {"name": "gear2", "properties": []}, - {"name": "gear3", "properties": []}, - {"name": "shaft1", "properties": []}, - {"name": "shaft2", "properties": []}, - {"name": "shaft3", "properties": []}, - {"name": "gearbase", "properties": []}, - {"name": "gearbase_hole1", "properties": []}, - {"name": "gearbase_hole3", "properties": []}, - {"name": "left_hand", "properties": []}, - ], - "constraints": [ - { - "source": "parallel_box1", - "name": "can_manipulate", - "target": "shaft1", - }, - { - "source": "parallel_box2", - "name": "can_manipulate", - "target": "gear1", - }, - {"source": "outward_claw", "name": "can_manipulate", "target": "gear2"}, - {"source": "outward_claw", "name": "can_manipulate", "target": "gear3"}, - {"source": "no_tool", "name": "can_manipulate", "target": "shaft3"}, - { - "source": "shaft1", - "name": "can_insert_to", - "target": "gearbase_hole1", - }, - { - "source": "shaft3", - "name": "can_insert_to", - "target": "gearbase_hole3", - }, - {"source": "gear3", "name": "can_insert_to", "target": "shaft3"}, - ], - "relations": [ - {"source": "left_hand", "name": "hold", "target": "outward_claw"} - ], - } - ], -} - -import json -from kios_bt.bt_factory import BehaviorTreeFactory -from kios_utils.pybt_test import generate_bt_stewardship, render_dot_tree - -# * this is a skeleton tree example. -example1 = { - "summary": "selector to load left_hand with parallel_box1", - "name": "selector: load_tool(left_hand, parallel_box1)", - "children": [ - { - "summary": "condition node to check if left_hand holds parallel_box1", - "name": "target: hold(left_hand, parallel_box1)", - }, - { - "summary": "sequence to load left_hand with parallel_box1", - "name": "sequence: load_tool(left_hand, parallel_box1)", - "children": [ - { - "summary": "selector to make left_hand unload outward_claw", - "name": "selector: unload_tool(left_hand, outward_claw)", - "children": [ - { - "summary": "condition node to check if left_hand is free", - "name": "target: is_free(left_hand)", - }, - { - "summary": "sequence to make left_hand unload outward_claw", - "name": "sequence: unload_tool(left_hand, outward_claw)", - "children": [ - { - "summary": "condition node to check if outward_claw is free", - "name": "precondition: is_free(outward_claw)", - }, - { - "summary": "condition node to check if left hand holds outward_claw", - "name": "precondition: hold(left_hand, outward_claw)", - }, - { - "summary": "action node to make left hand unload outward_claw", - "name": "action: unload_tool(left_hand, outward_claw)", - "effects": [ - { - "summary": "left_hand will be free", - }, - { - "summary": "outward_claw will be equippable", - }, - { - "summary": "left_hand will not hold outward_claw", - }, - ], - }, - ], - }, - ], - }, - { - "summary": "condition node to check if parallel_box1 is equippable", - "name": "precondition: is_equippable(parallel_box1)", - }, - { - "summary": "action node to equip left hand with parallel_box1", - "name": "action: load_tool(left_hand, parallel_box1)", - "effects": [ - { - "summary": "left_hand will be not free", - }, - { - "summary": "parallel_box1 will be not equippable", - }, - { - "summary": "left_hand will hold parallel_box1", - }, - ], - }, - ], - }, - ], -} - - -example2 = { - "summary": "selector to insert gear1 into shaft1", - "name": "selector: insert(left_hand, parallel_box1, gear1, shaft1)", - "children": [ - { - "summary": "condition node to check if gear1 is inserted to shaft1", - "name": "target: is_inserted_to(gear1, shaft1)", - }, - { - "summary": "sequence to insert gear1 into shaft1", - "name": "sequence: insert(left_hand, parallel_box1, gear1, shaft1)", - "children": [ - { - "summary": "selector to load parallel_box1", - "name": "selector: load_tool(left_hand, parallel_box1)", - "children": [ - { - "summary": "condition node to check if left_hand holds parallel_box1", - "name": "target: hold(left_hand, parallel_box1)", - }, - { - "summary": "sequence to load parallel_box1", - "name": "sequence: load_tool(left_hand, parallel_box1)", - "children": [ - { - "summary": "condition node to check if parallel_box1 is equippable", - "name": "precondition: is_equippable(parallel_box1)", - }, - { - "summary": "selector to make left_hand unload outward_claw", - "name": "selector: unload_tool(left_hand, outward_claw)", - "children": [ - { - "summary": "condition node to check if left_hand is free", - "name": "target: is_free(left_hand)", - }, - { - "summary": "sequence to make left_hand unload outward_claw", - "name": "sequence: unload_tool(left_hand, outward_claw)", - "children": [ - { - "summary": "condition node to check if outward_claw is free", - "name": "precondition: is_free(outward_claw)", - }, - { - "summary": "condition node to check if left hand holds outward_claw", - "name": "precondition: hold(left_hand, outward_claw)", - }, - { - "summary": "action node to make left hand unload outward_claw", - "name": "action: unload_tool(left_hand, outward_claw)", - "effects": [ - { - "summary": "left_hand will be free", - }, - { - "summary": "outward_claw will be equippable", - }, - { - "summary": "left_hand will not hold outward_claw", - }, - ], - }, - ], - }, - ], - }, - { - "summary": "action node to equip parallel_box1 to left hand", - "name": "action: load_tool(left_hand, parallel_box1)", - "effects": [ - { - "summary": "left_hand will be not free", - }, - { - "summary": "parallel_box1 will be not equippable", - }, - { - "summary": "left_hand will hold parallel_box1", - }, - ], - }, - ], - }, - ], - }, - { - "summary": "selector to use left_hand with parallel_box1 to pick up gear1", - "name": "selector: pick_up(left_hand, parallel_box1, gear1)", - "children": [ - { - "summary": "condition node to check if parallel_box1 holds gear1", - "name": "target: hold(parallel_box1, gear1)", - }, - { - "summary": "sequence to use left hand with parallel_box1 to pick up gear1", - "name": "sequence: pick_up(left_hand, parallel_box1, gear1)", - "children": [ - { - "summary": "condition node to check if parallel_box1 is free", - "name": "precondition: is_free(parallel_box1)", - }, - { - "summary": "condition node to check if left hand holds parallel_box1", - "name": "precondition: hold(left_hand, parallel_box1)", - }, - { - "summary": "action node to use left hand with parallel_box1 to pick up gear1", - "name": "action: pick_up(left_hand, parallel_box1, gear1)", - "effects": [ - { - "summary": "parallel_box1 will hold gear1", - }, - { - "summary": "parallel_box1 will be not free", - }, - ], - }, - ], - }, - ], - }, - { - "summary": "condition node to check if gear1 can be inserted to shaft1", - "name": "precondition: can_insert_to(gear1, shaft1)", - }, - { - "summary": "action node to use left_hand with parallel_box1 to insert gear1 to shaft1", - "name": "action: insert(left_hand, parallel_box1, gear1, shaft1)", - "effects": [ - { - "summary": "parallel_box1 will be not holding gear1", - }, - { - "summary": "parallel_box1 will be free", - }, - { - "summary": "gear1 will be inserted to shaft1", - }, - ], - }, - ], - }, - ], -} - -result = { - "summary": "Selector to insert shaft1 into gearbase_hole1 using parallel_box1 in the left hand", - "name": "selector: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)", - "children": [ - { - "summary": "Check the target that shaft1 is inserted into gearbase_hole1", - "name": "target: is_inserted_to(shaft1, gearbase_hole1)", - }, - { - "summary": "Sequence to insert shaft1 into gearbase_hole1 using parallel_box1 in the left hand", - "name": "sequence: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)", - "children": [ - { - "summary": "Selector to load the parallel_box1 in the left hand", - "name": "selector: load_tool(left_hand, parallel_box1)", - "children": [ - { - "summary": "Check the target that the left hand is free", - "name": "target: is_free(left_hand)", - }, - { - "summary": "Sequence to load the tool in the left hand", - "name": "sequence: load_tool(left_hand, parallel_box1)", - "children": [ - { - "summary": "Check the precondition that the parallel_box1 is equippable", - "name": "precondition: is_equippable(parallel_box1)", - }, - { - "summary": "Selector to unload the outward_claw in the left hand", - "name": "selector: unload_tool(left_hand, outward_claw)", - "children": [ - { - "summary": "Check the target that the left hand is free", - "name": "target: is_free(left_hand)", - }, - { - "summary": "Sequence to unload the tool in the left hand", - "name": "sequence: unload_tool(left_hand, outward_claw)", - "children": [ - { - "summary": "Check the precondition that the left hand is holding a outward_claw", - "name": "precondition: hold(left_hand, outward_claw)", - }, - { - "summary": "Unload the outward_claw in the left hand", - "name": "action: unload_tool(left_hand, outward_claw)", - }, - ], - }, - ], - }, - { - "summary": "Load the parallel_box1 in the left hand", - "name": "action: load_tool(left_hand, parallel_box1)", - }, - ], - }, - ], - }, - { - "summary": "Selector to pick up the shaft1 with the parallel_box1 in the left hand", - "name": "selector: pick_up(left_hand, parallel_box1, shaft1)", - "children": [ - { - "summary": "Check the target that the parallel_box1 is holding the shaft1", - "name": "target: hold(parallel_box1, shaft1)", - }, - { - "summary": "Sequence to pick up the shaft1 with the parallel_box1 in the left hand", - "name": "sequence: pick_up(left_hand, parallel_box1, shaft1)", - "children": [ - { - "summary": "Check the precondition that the left hand is holding the parallel_box1", - "name": "precondition: hold(left_hand, parallel_box1)", - }, - { - "summary": "Pick up the shaft1 with the parallel_box1 in the left hand", - "name": "action: pick_up(left_hand, parallel_box1, shaft1)", - }, - ], - }, - ], - }, - { - "summary": "Insert shaft1 into gearbase_hole1 using parallel_box1", - "name": "action: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)", - }, - ], - }, - ], -} - -ut = { - "summary": "Selector to change the tool in the left hand from outwardgripper to defaultgripper", - "name": "selector: change_tool(left_hand, outwardgripper, defaultgripper)", - "children": [ - { - "summary": "the target is that the left hand is holding the default gripper", - "name": "target: hold(left_hand, defaultgripper)", - }, - { - "summary": "Sequence to change the tool in the left hand from outwardgripper to defaultgripper", - "name": "sequence: change_tool(left_hand, outwardgripper, defaultgripper)", - "children": [ - { - "summary": "A precondition is that the left hand is holding the outwardgripper", - "name": "precondition: hold(left_hand, outwardgripper)", - }, - { - "summary": "A precondition is that the outwardgripper is empty", - "name": "precondition: is_empty(outwardgripper)", - }, - { - "summary": "The action to change the tool in the left hand from outwardgripper to defaultgripper", - "name": "action: change_tool(left_hand, outwardgripper, defaultgripper)", - }, - ], - }, - ], -} - - -def render_bt(bt_json: json): - test_class = BehaviorTreeFactory() - bt = test_class.from_json_to_simple_bt(bt_json) # * use this for skeleton tree - # bt = test_class.from_json_to_tree_root(bt_json) # * use this for a full tree - bt_stewardship = generate_bt_stewardship(bt) - bt_stewardship.setup(timeout=15) - render_dot_tree(bt_stewardship) - - -def render_bt_fix_try(bt_json: json): - btf = BehaviorTreeFactory() - bt = btf.from_json_to_simple_bt(bt_json) - # bt.setup() - import py_trees - - py_trees.display.render_dot_tree( - bt, - visibility_level=py_trees.common.VisibilityLevel.ALL, - with_blackboard_variables=False, - # with_qualified_names=True, - # target_directory=os.path(__file__), - ) - - -# render_bt(result) -render_bt_fix_try(ut) diff --git a/experiments/gearset1/world_state.json b/experiments/gearset1/world_state.json deleted file mode 100644 index ad8c47df..00000000 --- a/experiments/gearset1/world_state.json +++ /dev/null @@ -1,38 +0,0 @@ -{ - "objects": [ - {"name": "clampgripper", "properties": ["is_empty"]}, - {"name": "parallelgripper", "properties": ["is_empty"]}, - {"name": "inwardgripper", "properties": ["is_empty"]}, - {"name": "outwardgripper", "properties": []}, - {"name": "defaultgripper", "properties": ["is_empty"]}, - {"name": "gear1", "properties": []}, - {"name": "gear2", "properties": []}, - {"name": "gear3", "properties": []}, - {"name": "shaft1", "properties": []}, - {"name": "shaft2", "properties": []}, - {"name": "shaft3", "properties": []}, - {"name": "gearbase", "properties": []}, - {"name": "gearbase_hole1", "properties": []}, - {"name": "gearbase_hole3", "properties": []}, - {"name": "lampbase", "properties": []}, - {"name": "chair_leg1", "properties": []}, - {"name": "left_hand", "properties": []} - ], - "constraints": [ - {"source": "clampgripper", "name": "can_manipulate", "target": "shaft1"}, - {"source": "parallelgripper", "name": "can_manipulate", "target": "gear1"}, - {"source": "outwardgripper", "name": "can_manipulate", "target": "gear2"}, - {"source": "outwardgripper", "name": "can_manipulate", "target": "gear3"}, - {"source": "defaultgripper", "name": "can_manipulate", "target": "shaft3"}, - {"source": "shaft1", "name": "can_insert_to", "target": "gearbase_hole1"}, - {"source": "shaft3", "name": "can_insert_to", "target": "gearbase_hole3"}, - {"source": "gear3", "name": "can_insert_to", "target": "shaft3"}, - {"source": "gear2", "name": "can_insert_to", "target": "shaft2"}, - {"source": "gear1", "name": "can_insert_to", "target": "shaft1"} - ], - "relations": [ - {"source": "left_hand", "name": "hold", "target": "outwardgripper"}, - {"source": "outwardgripper", "name": "hold", "target": "gear3"} - ] -} - diff --git a/experiments/gearset1/world_state_finish.json b/experiments/gearset1/world_state_finish.json deleted file mode 100644 index 889c9ae9..00000000 --- a/experiments/gearset1/world_state_finish.json +++ /dev/null @@ -1,149 +0,0 @@ -{ - "objects": [ - { - "name": "clampgripper", - "properties": ["is_empty"] - }, - { - "name": "parallelgripper", - "properties": ["is_empty"] - }, - { - "name": "inwardgripper", - "properties": ["is_empty"] - }, - { - "name": "outwardgripper", - "properties": ["is_empty"] - }, - { - "name": "defaultgripper", - "properties": ["is_empty"] - }, - { - "name": "gear1", - "properties": [] - }, - { - "name": "gear2", - "properties": [] - }, - { - "name": "gear3", - "properties": [] - }, - { - "name": "shaft1", - "properties": [] - }, - { - "name": "shaft2", - "properties": [] - }, - { - "name": "shaft3", - "properties": [] - }, - { - "name": "gearbase", - "properties": [] - }, - { - "name": "gearbase_hole1", - "properties": [] - }, - { - "name": "gearbase_hole3", - "properties": [] - }, - { - "name": "left_hand", - "properties": [] - } - ], - "constraints": [ - { - "source": "clampgripper", - "name": "can_manipulate", - "target": "shaft1" - }, - { - "source": "parallelgripper", - "name": "can_manipulate", - "target": "gear1" - }, - { - "source": "outwardgripper", - "name": "can_manipulate", - "target": "gear2" - }, - { - "source": "outwardgripper", - "name": "can_manipulate", - "target": "gear3" - }, - { - "source": "defaultgripper", - "name": "can_manipulate", - "target": "shaft3" - }, - { - "source": "shaft1", - "name": "can_insert_to", - "target": "gearbase_hole1" - }, - { - "source": "shaft3", - "name": "can_insert_to", - "target": "gearbase_hole3" - }, - { - "source": "gear3", - "name": "can_insert_to", - "target": "shaft3" - }, - { - "source": "gear2", - "name": "can_insert_to", - "target": "shaft2" - }, - { - "source": "gear1", - "name": "can_insert_to", - "target": "shaft1" - } - ], - "relations": [ - { - "source": "left_hand", - "name": "hold", - "target": "defaultgripper" - }, - { - "source": "shaft1", - "name": "is_inserted_to", - "target": "gearbase_hole1" - }, - { - "source": "shaft3", - "name": "is_inserted_to", - "target": "gearbase_hole3" - }, - { - "source": "gear2", - "name": "is_inserted_to", - "target": "shaft2" - }, - { - "source": "gear1", - "name": "is_inserted_to", - "target": "shaft1" - }, - { - "source": "gear3", - "name": "is_inserted_to", - "target": "shaft3" - } - ] -} - diff --git a/experiments/lamp/baseline.py b/experiments/lamp/baseline.py deleted file mode 100644 index 0b583c0b..00000000 --- a/experiments/lamp/baseline.py +++ /dev/null @@ -1,207 +0,0 @@ -import json -import os -import re -from pprint import pprint -from typing import List, Tuple, Annotated, TypedDict -import operator -import logging -import asyncio - -from kios_bt.bt_stewardship import BehaviorTreeStewardship -from kios_scene.scene_factory import SceneFactory -from kios_bt.bt_factory import BehaviorTreeFactory -from kios_robot.robot_interface import RobotInterface -from kios_world.world_interface import WorldInterface - -from kios_agent.llm_supporter import KiosLLMSupporter -from kios_agent.data_types import KiosPromptSkeleton - -from kios_agent.kios_tools import ( - BehaviorTreeSimulatorTool, -) - -from dotenv import load_dotenv - -load_dotenv() - -from kios_utils.pybt_test import generate_bt_stewardship, render_dot_tree -from kios_utils.pddl_problem_parser import parse_problem_init, parse_problem_objects - - -def render_bt(bt_json: dict): - test_class = BehaviorTreeFactory() - bt = test_class.from_json_to_simple_bt(bt_json) - # bt = test_class.from_json_to_tree_root(bt_json) - bt_stewardship = generate_bt_stewardship(bt) - # bt_stewardship.setup(timeout=15) - render_dot_tree(bt_stewardship) - - -####################### dirs -current_dir = os.path.dirname(os.path.abspath(__file__)) -scene_path = os.path.join(current_dir, "scene.json") -# bt_json_file_path = os.path.join(current_dir, "behavior_tree.json") -world_state_path = os.path.join(current_dir, "world_state.json") -problem_path = os.path.join(current_dir, "gearset.problem") -domain_knowledge_path = os.path.join(current_dir, "domain_knowledge.txt") - -####################### problem -# with open(problem_path, "r") as file: -# problem = file.read() - -####################### scene -with open(scene_path, "r") as file: - scene_json_object = json.load(file) - -scene = SceneFactory().create_scene_from_json(scene_json_object) - -####################### world -world_interface = WorldInterface() -with open(world_state_path, "r") as file: - world_state_json = json.load(file) - world_interface.load_world_from_json(world_state_json) - - -pprint(world_state_json) - - -####################### robot -robot_interface = RobotInterface( - robot_address="127.0.0.1", - robot_port=12000, -) -robot_interface.setup_scene(scene) - -####################### bt_factory -bt_factory = BehaviorTreeFactory( - world_interface=world_interface, - robot_interface=robot_interface, -) - -####################### behavior_tree_stewardship -behavior_tree_stewardship = BehaviorTreeStewardship( - behaviortree_factory=bt_factory, - world_interface=world_interface, - robot_interface=robot_interface, -) - - -def baseline_run(tree_root: dict, world_state: dict): - - sk_json = tree_root - initial_world_state = world_state - - # * first sim run - from kios_plan.dynamic_planning import lamp_ut_dict - - solultion = behavior_tree_stewardship.sk_baseline( - initial_world_state, sk_json, lamp_ut_dict - ) - record = { - "target": sk_json["name"], - "initial_world_state": initial_world_state, - "result": solultion, - } - file_dir = os.path.join(current_dir, "baseline_result.jsonl") - with open(file_dir, "a") as file: - file.write(json.dumps(record) + "\n") - - -def match_type(node: dict) -> tuple[str, str]: - node_name = node["name"] - match = re.search( - r"(selector|sequence|action|precondition|condition|target):\s*(.+)", node_name - ) - if match: - node_type = match.group(1) - node_body = match.group(2) - return node_type, node_body - else: - raise ValueError(f"the node name {node_name} does not match any type.") - - -def expand_nodes( - node_list: list[dict], - start_state: dict, - overall_tree: list[dict] = None, -) -> dict: - """ - in order to monitor the tree generation, the overall tree and the node list should be the same variable when being passed in. - """ - pprint("----------check the entire tree:") - if overall_tree is not None: - render_bt(overall_tree[0]) - pprint("----------start to expand the node list:") - pprint(node_list) - pause = input("paused here! check the tree.") - - assert len(node_list) > 0 - state = start_state - - for i in range(len(node_list)): - type_name, body = match_type(node_list[i]) - # if match_type(node_list[i]) == "action": - if type_name == "action": - print(f"the node {node_list[i]['name']} is an action node. skip it.") - pause = input("paused here! check!") - # elif match_type(node_list[i]) == "precondition" or "target": - elif type_name in ["precondition", "target"]: - # goal = node_list[i]["name"] - goal = body - plan = make_plan(state, goal) - if len(plan) == 0: - logging.warning(f"No action should be performed for the goal {goal}.") - logging.warning(f'the node {node_list[i]["name"]} has been skipped.') - pause = input("paused here! check!") - else: - logging.info(f"Actions have been planed for the goal {goal}.") - pprint(f"the plan for the goal {goal} is {plan}") - pause = input("paused here! check!") - last_action = plan[-1] - unit_subtree = generate_unit_subtree(last_action) - # insert the subtree into the node_list - node_list[i] = unit_subtree - new_node_list = get_node_list_from_tree(unit_subtree) - expand_nodes( - node_list=new_node_list, - start_state=state, - overall_tree=overall_tree, - ) - state = estimate_state(state, plan) - - return node_list[0] - - -def test_expand_nodes(): - start_state = world_state_json - node_list = [ - { - "summary": "screw the lampbulb into the lampbase", - "name": "target: screw the lampbulb into the lampbase", - } - ] - # node_list = [ - # { - # "summary": "place the lampbulb onto the lampbulb", - # "name": "target: place the lampshade onto the lampbulb", - # } - # ] - expand_nodes(node_list, start_state, node_list) - pprint(node_list) - - -def test_baseline(): - sk = { - "summary": "the target is to place the lampshade onto the lampbulb", - "name": "target: is_placed_to(lampshade, lampbulb)", - # "summary": "the target is to screw the lampbulb into the lampbase", - # "name": "target: is_screwed_to(lampbulb, lampbase)", - } - baseline_run(sk, world_state_json) - - -if __name__ == "__main__": - pass - - # test_expand_nodes() - test_baseline() diff --git a/experiments/lamp/baseline_result.jsonl b/experiments/lamp/baseline_result.jsonl deleted file mode 100644 index 16b6143c..00000000 --- a/experiments/lamp/baseline_result.jsonl +++ /dev/null @@ -1,4 +0,0 @@ -{"target": "target: is_screwed_to(lampbulb, lampbase)", "initial_world_state": {"objects": [{"name": "clampgripper", "properties": ["is_empty"]}, {"name": "parallelgripper", "properties": ["is_empty"]}, {"name": "inwardgripper", "properties": ["is_empty"]}, {"name": "outwardgripper", "properties": ["is_empty"]}, {"name": "defaultgripper", "properties": ["is_empty"]}, {"name": "lampbase", "properties": []}, {"name": "lampshade", "properties": []}, {"name": "lampbulb", "properties": []}, {"name": "left_hand", "properties": []}], "constraints": [{"source": "clampgripper", "name": "can_manipulate", "target": "lampbulb"}, {"source": "outwardgripper", "name": "can_manipulate", "target": "lampshade"}, {"source": "lampbulb", "name": "can_screw_to", "target": "lampbase"}, {"source": "lampshade", "name": "can_place_to", "target": "lampbulb"}], "relations": [{"source": "left_hand", "name": "hold", "target": "defaultgripper"}]}, "result": {"summary": "selector to screw the lampbulb into the lampbase with the clampgripper in the left_hand", "name": "selector: screw(left_hand, clampgripper, lampbulb, lampbase)", "children": [{"summary": "the target is to make the lampbulb be screwed into the lampbase", "name": "target: is_screwed_to(lampbulb, lampbase)"}, {"summary": "sequence to screw the lampbulb into the lampbase with the clampgripper in the left_hand", "name": "sequence: screw(left_hand, clampgripper, lampbulb, lampbase)", "children": [{"summary": "selector to change tool in left_hand from defaultgripper to clampgripper", "name": "selector: change_tool(left_hand, defaultgripper, clampgripper)", "children": [{"summary": "the target is to make left_hand hold clampgripper", "name": "target: hold(left_hand, clampgripper)"}, {"summary": "sequence to change tool in left_hand from defaultgripper to clampgripper", "name": "sequence: change_tool(left_hand, defaultgripper, clampgripper)", "children": [{"summary": "a precondition is left_hand is holding defaultgripper", "name": "precondition: hold(left_hand, defaultgripper)"}, {"summary": "a precondition is defaultgripper is empty", "name": "precondition: is_empty(defaultgripper)"}, {"summary": "the action to change tool in left_hand from defaultgripper to clampgripper", "name": "action: change_tool(left_hand, defaultgripper, clampgripper)"}]}]}, {"summary": "selector to pick up the lampbulb with the clampgripper in the left_hand", "name": "selector: pick_up(left_hand, clampgripper, lampbulb)", "children": [{"summary": "the target is to make the clampgripper hold the lampbulb", "name": "target: hold(clampgripper, lampbulb)"}, {"summary": "sequence to pick up the lampbulb with the clampgripper in the left_hand", "name": "sequence: pick_up(left_hand, clampgripper, lampbulb)", "children": [{"summary": "a precondition is the clampgripper is empty", "name": "precondition: is_empty(clampgripper)"}, {"summary": "a precondition is the left_hand is holding clampgripper", "name": "precondition: hold(left_hand, clampgripper)"}, {"summary": "the action to pick up the lampbulb with the clampgripper in the left_hand", "name": "action: pick_up(left_hand, clampgripper, lampbulb)"}]}]}, {"summary": "the action to screw the lampbulb into the lampbase with the clampgripper in the left_hand", "name": "action: screw(left_hand, clampgripper, lampbulb, lampbase)"}]}]}} -{"target": "target: is_screwed_to(lampbulb, lampbase)", "initial_world_state": {"objects": [{"name": "clampgripper", "properties": ["is_empty"]}, {"name": "parallelgripper", "properties": ["is_empty"]}, {"name": "inwardgripper", "properties": ["is_empty"]}, {"name": "outwardgripper", "properties": ["is_empty"]}, {"name": "defaultgripper", "properties": []}, {"name": "lampbase", "properties": []}, {"name": "lampshade", "properties": []}, {"name": "lampbulb", "properties": []}, {"name": "cube", "properties": []}, {"name": "designring", "properties": []}, {"name": "left_hand", "properties": []}], "constraints": [{"source": "clampgripper", "name": "can_manipulate", "target": "lampbulb"}, {"source": "outwardgripper", "name": "can_manipulate", "target": "lampshade"}, {"source": "lampbulb", "name": "can_screw_to", "target": "lampbase"}, {"source": "lampshade", "name": "can_place_to", "target": "lampbulb"}], "relations": [{"source": "left_hand", "name": "hold", "target": "defaultgripper"}, {"source": "defaultgripper", "name": "hold", "target": "cube"}]}, "result": {"summary": "selector to screw the lampbulb into the lampbase with the clampgripper in the left_hand", "name": "selector: screw(left_hand, clampgripper, lampbulb, lampbase)", "children": [{"summary": "the target is to make the lampbulb be screwed into the lampbase", "name": "target: is_screwed_to(lampbulb, lampbase)"}, {"summary": "sequence to screw the lampbulb into the lampbase with the clampgripper in the left_hand", "name": "sequence: screw(left_hand, clampgripper, lampbulb, lampbase)", "children": [{"summary": "selector to change tool in left_hand from defaultgripper to clampgripper", "name": "selector: change_tool(left_hand, defaultgripper, clampgripper)", "children": [{"summary": "the target is to make left_hand hold clampgripper", "name": "target: hold(left_hand, clampgripper)"}, {"summary": "sequence to change tool in left_hand from defaultgripper to clampgripper", "name": "sequence: change_tool(left_hand, defaultgripper, clampgripper)", "children": [{"summary": "a precondition is left_hand is holding defaultgripper", "name": "precondition: hold(left_hand, defaultgripper)"}, {"summary": "selector to put down the cube held by the defaultgripper in the left_hand", "name": "selector: put_down(left_hand, defaultgripper, cube)", "children": [{"summary": "the target is to make the defaultgripper empty", "name": "target: is_empty(defaultgripper)"}, {"summary": "sequence to put down the cube held by the defaultgripper in the left_hand", "name": "sequence: put_down(left_hand, defaultgripper, cube)", "children": [{"summary": "a precondition is the left_hand is holding defaultgripper", "name": "precondition: hold(left_hand, defaultgripper)"}, {"summary": "a precondition is the defaultgripper is holding cube", "name": "precondition: hold(defaultgripper, cube)"}, {"summary": "the action to put down the cube held by the defaultgripper in the left_hand", "name": "action: put_down(left_hand, defaultgripper, cube)"}]}]}, {"summary": "the action to change tool in left_hand from defaultgripper to clampgripper", "name": "action: change_tool(left_hand, defaultgripper, clampgripper)"}]}]}, {"summary": "selector to pick up the lampbulb with the clampgripper in the left_hand", "name": "selector: pick_up(left_hand, clampgripper, lampbulb)", "children": [{"summary": "the target is to make the clampgripper hold the lampbulb", "name": "target: hold(clampgripper, lampbulb)"}, {"summary": "sequence to pick up the lampbulb with the clampgripper in the left_hand", "name": "sequence: pick_up(left_hand, clampgripper, lampbulb)", "children": [{"summary": "a precondition is the clampgripper is empty", "name": "precondition: is_empty(clampgripper)"}, {"summary": "a precondition is the left_hand is holding clampgripper", "name": "precondition: hold(left_hand, clampgripper)"}, {"summary": "the action to pick up the lampbulb with the clampgripper in the left_hand", "name": "action: pick_up(left_hand, clampgripper, lampbulb)"}]}]}, {"summary": "the action to screw the lampbulb into the lampbase with the clampgripper in the left_hand", "name": "action: screw(left_hand, clampgripper, lampbulb, lampbase)"}]}]}} -{"target": "target: is_placed_to(lampshade, lampbulb)", "initial_world_state": {"objects": [{"name": "clampgripper", "properties": ["is_empty"]}, {"name": "parallelgripper", "properties": ["is_empty"]}, {"name": "inwardgripper", "properties": ["is_empty"]}, {"name": "outwardgripper", "properties": ["is_empty"]}, {"name": "defaultgripper", "properties": ["is_empty"]}, {"name": "lampbase", "properties": []}, {"name": "lampshade", "properties": []}, {"name": "lampbulb", "properties": []}, {"name": "cube", "properties": []}, {"name": "designring", "properties": []}, {"name": "left_hand", "properties": []}], "constraints": [{"source": "clampgripper", "name": "can_manipulate", "target": "lampbulb"}, {"source": "outwardgripper", "name": "can_manipulate", "target": "lampshade"}, {"source": "lampbulb", "name": "can_screw_to", "target": "lampbase"}, {"source": "lampshade", "name": "can_place_to", "target": "lampbulb"}], "relations": [{"source": "left_hand", "name": "hold", "target": "inwardgripper"}, {"source": "lampbulb", "name": "is_screwed_to", "target": "lampbase"}]}, "result": {"summary": "selector to place the lampshade into the lampbulb with the outwardgripper in the left_hand", "name": "selector: place(left_hand, outwardgripper, lampshade, lampbulb)", "children": [{"summary": "the target is to make the lampshade be placed into the lampbulb", "name": "target: is_placed_to(lampshade, lampbulb)"}, {"summary": "sequence to place the lampshade into the lampbulb with the outwardgripper in the left_hand", "name": "sequence: place(left_hand, outwardgripper, lampshade, lampbulb)", "children": [{"summary": "selector to change tool in left_hand from inwardgripper to outwardgripper", "name": "selector: change_tool(left_hand, inwardgripper, outwardgripper)", "children": [{"summary": "the target is to make left_hand hold outwardgripper", "name": "target: hold(left_hand, outwardgripper)"}, {"summary": "sequence to change tool in left_hand from inwardgripper to outwardgripper", "name": "sequence: change_tool(left_hand, inwardgripper, outwardgripper)", "children": [{"summary": "a precondition is left_hand is holding inwardgripper", "name": "precondition: hold(left_hand, inwardgripper)"}, {"summary": "a precondition is inwardgripper is empty", "name": "precondition: is_empty(inwardgripper)"}, {"summary": "the action to change tool in left_hand from inwardgripper to outwardgripper", "name": "action: change_tool(left_hand, inwardgripper, outwardgripper)"}]}]}, {"summary": "selector to pick up the lampshade with the outwardgripper in the left_hand", "name": "selector: pick_up(left_hand, outwardgripper, lampshade)", "children": [{"summary": "the target is to make the outwardgripper hold the lampshade", "name": "target: hold(outwardgripper, lampshade)"}, {"summary": "sequence to pick up the lampshade with the outwardgripper in the left_hand", "name": "sequence: pick_up(left_hand, outwardgripper, lampshade)", "children": [{"summary": "a precondition is the outwardgripper is empty", "name": "precondition: is_empty(outwardgripper)"}, {"summary": "a precondition is the left_hand is holding outwardgripper", "name": "precondition: hold(left_hand, outwardgripper)"}, {"summary": "the action to pick up the lampshade with the outwardgripper in the left_hand", "name": "action: pick_up(left_hand, outwardgripper, lampshade)"}]}]}, {"summary": "the action to place the lampshade into the lampbulb with the outwardgripper in the left_hand", "name": "action: place(left_hand, outwardgripper, lampshade, lampbulb)"}]}]}} -{"target": "target: is_placed_to(lampshade, lampbulb)", "initial_world_state": {"objects": [{"name": "clampgripper", "properties": ["is_empty"]}, {"name": "parallelgripper", "properties": ["is_empty"]}, {"name": "inwardgripper", "properties": ["is_empty"]}, {"name": "outwardgripper", "properties": ["is_empty"]}, {"name": "defaultgripper", "properties": ["is_empty"]}, {"name": "lampbase", "properties": []}, {"name": "lampshade", "properties": []}, {"name": "lampbulb", "properties": []}, {"name": "cube", "properties": []}, {"name": "designring", "properties": []}, {"name": "left_hand", "properties": []}], "constraints": [{"source": "clampgripper", "name": "can_manipulate", "target": "lampbulb"}, {"source": "outwardgripper", "name": "can_manipulate", "target": "lampshade"}, {"source": "lampbulb", "name": "can_screw_to", "target": "lampbase"}, {"source": "lampshade", "name": "can_place_to", "target": "lampbulb"}], "relations": [{"source": "left_hand", "name": "hold", "target": "inwardgripper"}, {"source": "lampbulb", "name": "is_screwed_to", "target": "lampbase"}, {"source": "lampshade", "name": "is_placed_to", "target": "lampbulb"}]}, "result": {"summary": "the target is to place the lampshade onto the lampbulb", "name": "target: is_placed_to(lampshade, lampbulb)"}} diff --git a/experiments/lamp/baseline_result_handle.py b/experiments/lamp/baseline_result_handle.py deleted file mode 100644 index 17d76784..00000000 --- a/experiments/lamp/baseline_result_handle.py +++ /dev/null @@ -1,22 +0,0 @@ -import json -import os -import json -import os - -file_dir = os.path.dirname(os.path.abspath(__file__)) -result_dir = os.path.join(file_dir, "baseline_result.jsonl") - -if not os.path.exists(os.path.join(file_dir, "problem_set")): - os.makedirs(os.path.join(file_dir, "problem_set")) - -with open(result_dir, "r") as file: - results = file.read() - data = [json.loads(line) for line in results.splitlines()] - -i = 0 -for item in data: - i_str = str(i).zfill(3) - entity_path = os.path.join(file_dir, "problem_set", f"problem_{i_str}.json") - with open(entity_path, "w") as file: - file.write(json.dumps(item, indent=4)) - i += 1 diff --git a/experiments/lamp/behavior_tree.json b/experiments/lamp/behavior_tree.json deleted file mode 100644 index ba968b94..00000000 --- a/experiments/lamp/behavior_tree.json +++ /dev/null @@ -1,232 +0,0 @@ -{ - "summary": "selector to insert shaft1 into gearbase_hole1", - "name": "selector: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)", - "identifier": 1, - "type_name": "selector", - "children": [ - { - "summary": "condition node to check if shaft1 is inserted to gearbase_hole1", - "name": "target: is_inserted_to(shaft1, gearbase_hole1)", - "identifier": 2, - "type_name": "condition", - "conditions": [ - { - "object_name": "shaft1", - "property_name": "is_inserted_to", - "property_value": "gearbase_hole1", - "status": true - } - ] - }, - { - "summary": "sequence to insert shaft1 into gearbase_hole1", - "name": "sequence: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)", - "identifier": 3, - "type_name": "sequence", - "children": [ - { - "summary": "selector to load parallel_box1", - "name": "selector: load_tool(left_hand, parallel_box1)", - "identifier": 4, - "type_name": "selector", - "children": [ - { - "summary": "condition node to check if left_hand holds parallel_box1", - "name": "target: hold(left_hand, parallel_box1)", - "identifier": 5, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "parallel_box1", - "status": true - } - ] - }, - { - "summary": "sequence to load parallel_box1", - "name": "sequence: load_tool(left_hand, parallel_box1)", - "identifier": 6, - "type_name": "sequence", - "children": [ - { - "summary": "condition node to check if parallel_box1 is equippable", - "name": "precondition: is_equippable(parallel_box1)", - "identifier": 7, - "type_name": "condition", - "conditions": [ - { - "object_name": "parallel_box1", - "property_name": "is_equippable", - "property_value": null, - "status": true - } - ] - }, - { - "summary": "condition node to check if left hand is free", - "name": "precondition: is_free(left_hand)", - "identifier": 8, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "is_free", - "property_value": null, - "status": true - } - ] - }, - { - "summary": "action node to equip parallel_box1 to left hand", - "name": "action: load_tool(left_hand, parallel_box1)", - "identifier": 9, - "type_name": "action", - "effects": [ - { - "object_name": "left_hand", - "property_name": "is_free", - "property_value": null, - "status": false - }, - { - "object_name": "parallel_box1", - "property_name": "is_equippable", - "property_value": null, - "status": false - }, - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "parallel_box1", - "status": true - } - ] - } - ] - } - ] - }, - { - "summary": "selector to use left_hand with parallel_box1 to pick up shaft1", - "name": "selector: pick_up(left_hand, parallel_box1, shaft1)", - "identifier": 10, - "type_name": "selector", - "children": [ - { - "summary": "condition node to check if parallel_box1 holds shaft1", - "name": "target: hold(parallel_box1, shaft1)", - "identifier": 11, - "type_name": "condition", - "conditions": [ - { - "object_name": "parallel_box1", - "property_name": "hold", - "property_value": "shaft1", - "status": true - } - ] - }, - { - "summary": "sequence to use left hand with parallel_box1 to pick up shaft1", - "name": "sequence: pick_up(left_hand, parallel_box1, shaft1)", - "identifier": 12, - "type_name": "sequence", - "children": [ - { - "summary": "condition node to check if parallel_box1 is free", - "name": "precondition: is_free(parallel_box1)", - "identifier": 13, - "type_name": "condition", - "conditions": [ - { - "object_name": "parallel_box1", - "property_name": "is_free", - "property_value": null, - "status": true - } - ] - }, - { - "summary": "condition node to check if left hand holds parallel_box1", - "name": "precondition: hold(left_hand, parallel_box1)", - "identifier": 14, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "parallel_box1", - "status": true - } - ] - }, - { - "summary": "action node to use left hand with parallel_box1 to pick up shaft1", - "name": "action: pick_up(left_hand, parallel_box1, shaft1)", - "identifier": 15, - "type_name": "action", - "effects": [ - { - "object_name": "parallel_box1", - "property_name": "hold", - "property_value": "shaft1", - "status": true - }, - { - "object_name": "parallel_box1", - "property_name": "is_free", - "property_value": null, - "status": false - } - ] - } - ] - } - ] - }, - { - "summary": "condition node to check if shaft1 can be inserted to gearbase_hole1", - "name": "precondition: can_insert_to(shaft1, gearbase_hole1)", - "identifier": 16, - "type_name": "condition", - "conditions": [ - { - "object_name": "shaft1", - "property_name": "can_insert_to", - "property_value": "gearbase_hole1", - "status": true - } - ] - }, - { - "summary": "action node to use left_hand with parallel_box1 to insert shaft1 to gearbase_hole1", - "name": "action: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)", - "identifier": 17, - "type_name": "action", - "effects": [ - { - "object_name": "parallel_box1", - "property_name": "hold", - "property_value": "shaft1", - "status": false - }, - { - "object_name": "parallel_box1", - "property_name": "is_free", - "property_value": null, - "status": true - }, - { - "object_name": "shaft1", - "property_name": "is_inserted_to", - "property_value": "gearbase_hole1", - "status": true - } - ] - } - ] - } - ] - } \ No newline at end of file diff --git a/experiments/lamp/domain_knowledge.txt b/experiments/lamp/domain_knowledge.txt deleted file mode 100644 index ba5c088b..00000000 --- a/experiments/lamp/domain_knowledge.txt +++ /dev/null @@ -1,88 +0,0 @@ -[user] -Following is the domain knowledge you need in pddl domain format, which includes: -- the object types in the world, -- the predicates for describing the states of the world, -- the actions that the robot can apply, with their precondtions and effects defined. -""" -(define (domain robot_assembly_problem-domain) - (:requirements :strips :typing) - (:types - thing - object - tool part hand - thing - ) - (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part)) - (:action pick_up - :parameters ( ?hand - hand ?tool - tool ?part - part) - :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part)) - :effect (and (hold ?tool ?part) (not (is_free ?tool)))) - (:action put_down - :parameters ( ?hand - hand ?tool - tool ?part - part) - :precondition (and (hold ?tool ?part) (hold ?hand ?tool)) - :effect (and (not (hold ?hand ?part)) (is_free ?tool))) - (:action place - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2))) - (:action detach - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2)))) - (:action insert - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2))) - (:action pull - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2)))) - (:action screw - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2))) - (:action unscrew - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2)))) - (:action load_tool - :parameters ( ?hand - hand ?tool - tool) - :precondition (and (is_equippable ?tool) (is_free ?hand)) - :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool))) - (:action unload_tool - :parameters ( ?hand - hand ?tool - tool) - :precondition (and (hold ?hand ?tool) (is_free ?tool)) - :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool)))) -) -""" -The precidates are explained as follows: -""" -"PREDICATE_LIST" -- is_free( or ): or is free and can hold something. (default: true) -- is_equippable(): is ready to be eqquipped by a robot hand. (default: true) -- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false) -- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false) -- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false) -- can_place_to(, ): can be placed into (placement assembly constraint) (default: false) -- hold(, ): is holding (default: false) -- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false) -- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false) -- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false) -""" - -The actions are explained as follows: -""" -"ROBOT_ACTION_LIST" -- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something. -- put_down(, , ): use to put down . As preconditions, should be holding and should be holding . -- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding . -- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into . -- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding . -- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into . -- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding . -- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into . -- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free. -- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free. -""" -the items in <> indicate the type of the instances needed for the predicates and actions. - -[assistant] -Understood. I will wait for further instructions before starting to work. diff --git a/experiments/lamp/gearset.domain b/experiments/lamp/gearset.domain deleted file mode 100644 index 39c677b0..00000000 --- a/experiments/lamp/gearset.domain +++ /dev/null @@ -1,48 +0,0 @@ -(define (domain robot_assembly_problem-domain) - (:requirements :strips :typing) - (:types - thing - object - tool part hand - thing - ) - (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part)) - (:action pick_up - :parameters ( ?hand - hand ?tool - tool ?part - part) - :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part)) - :effect (and (hold ?tool ?part) (not (is_free ?tool)))) - (:action put_down - :parameters ( ?hand - hand ?tool - tool ?part - part) - :precondition (and (hold ?tool ?part) (hold ?hand ?tool)) - :effect (and (not (hold ?hand ?part)) (is_free ?tool))) - (:action place - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2))) - (:action detach - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2)))) - (:action insert - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2))) - (:action pull - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2)))) - (:action screw - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2))) - (:action unscrew - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2)))) - (:action load_tool - :parameters ( ?hand - hand ?tool - tool) - :precondition (and (is_equippable ?tool) (is_free ?hand)) - :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool))) - (:action unload_tool - :parameters ( ?hand - hand ?tool - tool) - :precondition (and (hold ?hand ?tool) (is_free ?tool)) - :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool)))) -) \ No newline at end of file diff --git a/experiments/lamp/gearset1 copy.py b/experiments/lamp/gearset1 copy.py deleted file mode 100644 index cd823828..00000000 --- a/experiments/lamp/gearset1 copy.py +++ /dev/null @@ -1,89 +0,0 @@ -import json -import os - -from kios_bt.bt_stewardship import BehaviorTreeStewardship -from kios_scene.scene_factory import SceneFactory -from kios_bt.bt_factory import BehaviorTreeFactory -from kios_robot.robot_interface import RobotInterface -from kios_world.world_interface import WorldInterface -from kios_utils.pybt_test import ( - generate_bt_stewardship, - render_dot_tree, - tick_loop_test, - tick_1000HZ_test, - tick_frequency_test, -) - - -def core_loop(): - ####################### dirs - current_dir = os.path.dirname(os.path.abspath(__file__)) - scene_json_file_path = os.path.join(current_dir, "gearset_scene.json") - bt_json_file_path = os.path.join(current_dir, "behavior_tree.json") - world_state_json_file_path = os.path.join(current_dir, "world_state.json") - - ####################### scene - with open(scene_json_file_path, "r") as file: - json_object = json.load(file) - - gearset_scene = SceneFactory().create_scene_from_json(json_object) - - ####################### world - world_interface = WorldInterface() - with open(world_state_json_file_path, "r") as file: - json_object = json.load(file) - world_interface.load_world_from_json(json_object) - - ####################### robot - robot_interface = RobotInterface( - robot_address="127.0.0.1", - robot_port=12000, - ) - robot_interface.setup_scene(gearset_scene) - - ####################### agent - # * use the generated plan instead - - ####################### bt - bt_factory = BehaviorTreeFactory( - world_interface=world_interface, - robot_interface=robot_interface, - ) - - ####################### * old code - # behavior_tree = None - # with open(bt_json_file_path, "r") as file: - # json_object = json.load(file) - # behavior_tree = bt_factory.from_json_to_bt(json_object) - - # bt_steward = generate_bt_stewardship(behavior_tree) - # render_dot_tree(bt_steward) - # # tick_loop_test(bt_steward) - # tick_frequency_test(bt_steward) - - # * new code - behavior_tree_stewardship = BehaviorTreeStewardship( - behaviortree_factory=bt_factory, - world_interface=world_interface, - robot_interface=robot_interface, - ) - - with open(bt_json_file_path, "r") as file: - json_object = json.load(file) - behavior_tree_stewardship.load_bt_json(json_object) - - behavior_tree_stewardship.generate_behavior_tree() - behavior_tree_stewardship.setup_behavior_tree() - behavior_tree_stewardship.render_dot_tree() - - # ! return the tree result - behavior_tree_stewardship.tick_loop_test() - - # sim_result = behavior_tree_stewardship.simulate_behavior_tree() - # print(sim_result.to_json()) - - # ! world state has problem. gear3 is not free! - - -if __name__ == "__main__": - core_loop() diff --git a/experiments/lamp/gearset1_graph copy.py b/experiments/lamp/gearset1_graph copy.py deleted file mode 100644 index 04f9d5b5..00000000 --- a/experiments/lamp/gearset1_graph copy.py +++ /dev/null @@ -1,522 +0,0 @@ -import json -import os - -from kios_bt.bt_stewardship import BehaviorTreeStewardship -from kios_scene.scene_factory import SceneFactory -from kios_bt.bt_factory import BehaviorTreeFactory -from kios_robot.robot_interface import RobotInterface -from kios_world.world_interface import WorldInterface - -from kios_agent.llm_supporter import KiosLLMSupporter -from kios_agent.data_types import KiosPromptSkeleton - -from kios_agent.kios_tools import ( - BehaviorTreeExecutorTool, - BehaviorTreeSimulatorTool, - WorldStateQueryTool, -) - -from langchain_core.pydantic_v1 import BaseModel, Field -from typing import List, Tuple, Annotated, TypedDict -import operator - -from dotenv import load_dotenv - -from langchain import hub -from langchain.agents import create_openai_functions_agent, AgentExecutor -from langchain_openai import ChatOpenAI -from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder -from langchain_core.output_parsers import JsonOutputParser, StrOutputParser - -from langchain.chains.openai_functions import ( - create_structured_output_runnable, - create_openai_fn_runnable, -) -from langchain_core.utils.function_calling import convert_to_openai_function - -from langchain.agents.format_scratchpad import format_to_openai_function_messages - -from langchain_core.agents import AgentActionMessageLog, AgentFinish - -from langgraph.graph import StateGraph, END - -from langchain_core.messages import HumanMessage - - -load_dotenv() - -from langsmith import traceable - -os.environ["LANGCHAIN_TRACING_V2"] = "true" -os.environ["LANGCHAIN_ENDPOINT"] = "https://api.smith.langchain.com" -os.environ["LANGCHAIN_PROJECT"] = "kios_agent" - - -####################### dirs -current_dir = os.path.dirname(os.path.abspath(__file__)) -scene_path = os.path.join(current_dir, "scene.json") -# bt_json_file_path = os.path.join(current_dir, "behavior_tree.json") -world_state_path = os.path.join(current_dir, "world_state.json") -problem_path = os.path.join(current_dir, "gearset.problem") - - -####################### problem -with open(problem_path, "r") as file: - problem = file.read() - -####################### scene -with open(scene_path, "r") as file: - scene_json_object = json.load(file) - -scene = SceneFactory().create_scene_from_json(scene_json_object) - -####################### world -world_interface = WorldInterface() -with open(world_state_path, "r") as file: - world_state_json_object = json.load(file) - world_interface.load_world_from_json(world_state_json_object) - -####################### robot -robot_interface = RobotInterface( - robot_address="127.0.0.1", - robot_port=12000, -) -robot_interface.setup_scene(scene) - -####################### bt_factory -bt_factory = BehaviorTreeFactory( - world_interface=world_interface, - robot_interface=robot_interface, -) - -####################### behavior_tree_stewardship -behavior_tree_stewardship = BehaviorTreeStewardship( - behaviortree_factory=bt_factory, - world_interface=world_interface, - robot_interface=robot_interface, -) - -####################### * tools -# bt_executor_tool = BehaviorTreeExecutorTool(behavior_tree_stewardship) - -bt_sim_tool = BehaviorTreeSimulatorTool(metadata={"bt_stw": behavior_tree_stewardship}) - -# world_state_query_tool = WorldStateQueryTool(behavior_tree_stewardship) - -executor_tools = [bt_sim_tool] - -# * kios data prompt skeleton dir -data_dir = os.environ.get("KIOS_DATA_DIR").format(username=os.getlogin()) -print(data_dir) -prompt_sk_dir = os.path.join(data_dir, "prompt_skeletons") - -# * refiner chain -llm_spt_re_sk = KiosLLMSupporter() -with open(os.path.join(prompt_sk_dir, "skeleton_refiner.json"), "r") as f: - re_sk_pptsk_json = json.load(f) - re_sk_pptsk = KiosPromptSkeleton.from_json(re_sk_pptsk_json) - llm_spt_re_sk.initialize_from_prompt_skeleton(re_sk_pptsk) -re_sk_ppt = llm_spt_re_sk.create_prompt() -re_sk_llm = ChatOpenAI(model_name=llm_spt_re_sk.model_name, temperature=0) - -re_sk_chain = re_sk_ppt | re_sk_llm | JsonOutputParser() - - -# * skeleton generator chain -llm_spt_skeleton_generator = KiosLLMSupporter() -with open(os.path.join(prompt_sk_dir, "skeleton_generator.json"), "r") as f: - skeleton_generator_pptsk_json = json.load(f) - skeleton_generator_pptsk = KiosPromptSkeleton.from_json( - skeleton_generator_pptsk_json - ) - llm_spt_skeleton_generator.initialize_from_prompt_skeleton(skeleton_generator_pptsk) -skeleton_generator_ppt = llm_spt_skeleton_generator.create_prompt() -skeleton_generator_llm = ChatOpenAI( - model_name=llm_spt_skeleton_generator.model_name, temperature=0 -) - -skeleton_generator_chain = ( - skeleton_generator_ppt - | skeleton_generator_llm - # | StrOutputParser() - | JsonOutputParser() -) - -# ! DISCARDED behavior tree generator chain -# bt_gen_chain = skeleton_generator_chain | re_sk_chain - -############################################### * behavior tree executor agent -# ! NOW SUBSTITUTED BY THE FAKE EXECUTOR FUNCTION -# bt_exe_ppt = hub.pull("hwchase17/openai-functions-agent") -# bt_exe_ppt = ChatPromptTemplate.from_messages( -# [ -# ("system", "You are a helpful assistant"), -# ("user", "The behavior tree is: {behavior_tree}"), -# ("user", "The world state is: {world_state}"), -# MessagesPlaceholder(variable_name="agent_scratchpad"), -# ] -# ) - -# bt_exe_llm = ChatOpenAI(model="gpt-3.5-turbo", temperature=0) - - -# def executor_agent_parse(output): -# # If no function was invoked, return to user -# if "function_call" not in output.additional_kwargs: -# return AgentFinish(return_values={"output": output.content}, log=output.content) - -# # Parse out the function call -# function_call = output.additional_kwargs["function_call"] -# name = function_call["name"] -# inputs = json.loads(function_call["arguments"]) - -# # If the Response function was invoked, return to the user with the function inputs -# if name == "ExecutorResponse": -# return AgentFinish(return_values=inputs, log=str(function_call)) -# # Otherwise, return an agent action -# else: -# return AgentActionMessageLog( -# tool=name, tool_input=inputs, log="", message_log=[output] -# ) - - -# class ExecutorResponse(BaseModel): -# """the result of the behavior tree execution""" - -# hasSucceeded: bool = Field(description="If the behavior tree has succeeded or not") -# world_state: dict[str, list[dict[str, str]]] = Field( -# description="The final world state after the execution" -# ) - - -# bt_exe_llm_with_tools = bt_exe_llm.bind_functions( -# [convert_to_openai_function(t) for t in executor_tools] -# ) - - -# ! BUG -# bt_exe_agent = ( -# { -# "input": lambda x: x["input"], -# # Format agent scratchpad from intermediate steps -# "agent_scratchpad": lambda x: format_to_openai_function_messages( -# x["intermediate_steps"] -# ), -# } -# | bt_exe_ppt -# | bt_exe_llm_with_tools -# | executor_agent_parse -# ) - -# bt_exe_agent_executor = AgentExecutor( -# tools=executor_tools, agent=bt_exe_agent, verbose=True -# ) - - -# * the graph state -class PlanExecuteState(TypedDict): - plan: List[str] - behavior_tree_skeleton: dict # ! not sure if use this or not. - behavior_tree: dict - world_state: Annotated[ - List[dict], operator.add - ] # ! to add, you need to make world_state a list of dict - # = Field(default=None) - past_steps: Annotated[List[Tuple], operator.add] - response: str # ! not sure if use this or not. - # to_user: bool - user_input: str - problem: str - - -##################################################### * planner -# * output schema of the planner -class Plan(BaseModel): - """Plan to follow in future""" - - steps: List[str] = Field( - description="a list of different steps to follow, should be in sorted order" - ) - - -planner_prompt = ChatPromptTemplate.from_template( - """ - The user input should be an instruction (or several instructions) of the robot assembly task. \ - For the given user input, summarize and come up with a bullet list task plan. \ -This plan should involve individual assembly step(s) of robot assembly task.\ -Do not add any superfluous steps. \ -Make sure that each step has all the information needed - do not skip steps.\ - -You should only use assembly actions as the action in the steps. -The available assembly actions are:\ -- insert something into something\ -- place something onto something\ -- screw something into something\ - -Example:\ -user_input: "you should insert the red block into the blue hole"\ -plan: "insert the red block into the blue hole"\ - -user_input: "intert the blub into the base, then put the cover onto it."\ -plan: ["insert the blub into the base", "put the cover onto the blub"]\ - -world_state: {world_state}\ -user_input: {user_input}\ -""" -) -planner = create_structured_output_runnable( - Plan, ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), planner_prompt -) - - -##################################################### * plan_updaterner - - -class UpdaterResponse(BaseModel): - """used to response to the user for asking for more inputs.""" - - response: str - - -plan_updater_prompt = ChatPromptTemplate.from_template( - """For the given objective, come up with a simple step by step plan. \ -This plan should involve individual tasks, that if executed correctly will yield the correct answer. Do not add any superfluous steps. \ -The result of the final step should be the final answer. Make sure that each step has all the information needed - do not skip steps. - -Your objective was this: -{user_input} - -Your original plan was this: -{plan} - -You have currently done the follow steps: -{past_steps} - -The world state is: -{world_state} - -Read the world state. Check the original plan from the first step and update your plan with the steps need to be conducted next according to the world state.\ -If no more steps are needed to achieve the objective defined by user input, respond to the user for more input. Otherwise, fill out the plan.\ -Only add steps to the plan that still NEED to be done. Do not return previously done steps as part of the plan.""" -) - -plan_updater = create_openai_fn_runnable( - [Plan, UpdaterResponse], # * here two schemas are used - ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), - plan_updater_prompt, -) - - -##################################################### * graph node functions -async def user_input_step(state: PlanExecuteState): - """ - get the input from the user - """ - print(f"-----user_input_step-----") - if state["response"] is not None: - user_input = input(state["response"]) - state["response"] = None - else: - user_input = input("Your next instruction: ") - - return { - "user_input": user_input, - } - - -def user_input_should_end(state: PlanExecuteState): - """ - if the user input is empty, then end - """ - print(f"-----user_input_should_end-----") - - # print(f'user_input: {state["user_input"]}') - # print(f'world_state: {state["world_state"]}') - - if not state["user_input"] or state["user_input"] == "": - return True - else: - return False - - -async def behavior_tree_generate_step(state: PlanExecuteState): - """ - generate the behavior tree based on the instruction - """ - print(f"-----behavior_tree_generate_step-----") - - instruction = state["plan"][0] - latest_world_state = state["world_state"][-1] - - skeleton = await skeleton_generator_chain.ainvoke( - { - "problem": state["problem"], - "world_state": latest_world_state, - "instructions": instruction, - } - ) - - response = await re_sk_chain.ainvoke( - { - "input": skeleton, - } - ) - - behavior_tree = response.get("task_plan").get("behavior_tree") - - init_world_state = [response.get("initial_state")] - - return { - "behavior_tree": behavior_tree, - "world_state": init_world_state, # * this is necessary because the constraints in problem - } - - -async def behavior_tree_execute_step(state: PlanExecuteState): - """ - execute the first step of the plan, append the result to the past steps - """ - print(f"-----behavior_tree_execute_step-----") - - behavior_tree = state["behavior_tree"] - latest_world_state = state["world_state"][-1] - - # exe_input = { - # "behavior_tree": behavior_tree, - # "world_state": latest_world_state, - # } - - # agent_response = await bt_exe_agent_executor.ainvoke( - # { - # "input": exe_input, - # } - # ) - - response = behavior_tree_stewardship.fake_run( - world_state=latest_world_state, bt_json=behavior_tree - ) - - # hasSucceeded = response["hasSucceeded"] - # new_world_state = [response["world_state"]] - hasSucceeded = response.result - new_world_state = [response.world_state] - the_step = state["plan"][0] - - return { - "past_steps": (the_step, hasSucceeded), # ! why here not a list of tuple? - "world_state": new_world_state, - } - - -async def plan_step(state: PlanExecuteState): - """ - plan the steps based on user input and world state - """ - print(f"-----plan_step-----") - - plan = await planner.ainvoke( - { - "user_input": state["user_input"], - "world_state": state["world_state"], - } - ) - return {"plan": plan.steps} - - -async def plan_updater_step(state: PlanExecuteState): - """ - if return a response, then success, response the use, end. - otherwise, return the updated newplan (normally the same as the old plan with the first step popped out. - """ - print(f"-----plan_updater_step-----") - - output = await plan_updater.ainvoke(state) - if isinstance( - output, UpdaterResponse - ): # * determine if it is time to response and end - return { - "response": "Your last instruction has been finished." - } # * Don't need to update. - else: - return {"plan": output.steps} # * update the plan - - -def plan_updater_should_end(state: PlanExecuteState): - """ - end router - """ - print(f"-----plan_updater_should_end-----") - - if state["response"]: - return True - else: - return False - - -##################################################### * construct the graph - -workflow = StateGraph(PlanExecuteState) - -workflow.add_node("planner", plan_step) - -workflow.add_node("behavior_tree_generator", behavior_tree_generate_step) - -workflow.add_node("behavior_tree_executor", behavior_tree_execute_step) - -workflow.add_node("plan_updater", plan_updater_step) - -workflow.add_node("user_input_node", user_input_step) - -workflow.set_entry_point("user_input_node") - -# workflow.add_edge("user_input_node", "planner") - -workflow.add_edge( - "planner", "behavior_tree_generator" -) # ! may need to add a condition here - -workflow.add_edge("behavior_tree_generator", "behavior_tree_executor") - -workflow.add_edge("behavior_tree_executor", "plan_updater") - -workflow.add_conditional_edges( - "plan_updater", - plan_updater_should_end, - { - True: "user_input_node", - False: "behavior_tree_generator", - }, -) - -workflow.add_conditional_edges( - "user_input_node", - user_input_should_end, - { - True: END, - False: "planner", - }, -) - -app = workflow.compile() - - -config = {"recursion_limit": 50} - - -inputs = { - "world_state": [world_state_json_object], - "problem": problem, -} - - -async def core_run(): - async for event in app.astream(inputs, config=config): - for k, v in event.items(): - if k != "__end__": - print(v) - - -if __name__ == "__main__": - import asyncio - - asyncio.run(core_run()) diff --git a/experiments/lamp/gearset1_graph.py b/experiments/lamp/gearset1_graph.py deleted file mode 100644 index a877a619..00000000 --- a/experiments/lamp/gearset1_graph.py +++ /dev/null @@ -1,581 +0,0 @@ -import json -import os - -os.environ["LANGCHAIN_TRACING_V2"] = "true" -os.environ["LANGCHAIN_ENDPOINT"] = "https://api.smith.langchain.com" -os.environ["LANGCHAIN_PROJECT"] = "kios_agent" - -from kios_bt.bt_stewardship import BehaviorTreeStewardship -from kios_scene.scene_factory import SceneFactory -from kios_bt.bt_factory import BehaviorTreeFactory -from kios_robot.robot_interface import RobotInterface -from kios_world.world_interface import WorldInterface - -from kios_agent.llm_supporter import KiosLLMSupporter -from kios_agent.data_types import KiosPromptSkeleton - -from kios_agent.kios_tools import ( - BehaviorTreeExecutorTool, - BehaviorTreeSimulatorTool, - WorldStateQueryTool, -) - -from typing import List, Tuple, Annotated, TypedDict -import operator - -from dotenv import load_dotenv - -from langchain import hub - -from langchain_openai import ChatOpenAI - -from langchain.chains.openai_functions import ( - create_structured_output_runnable, - create_openai_fn_runnable, -) -from langchain_core.utils.function_calling import convert_to_openai_function -from langchain_core.agents import AgentActionMessageLog, AgentFinish -from langchain_core.messages import HumanMessage -from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder -from langchain_core.output_parsers import JsonOutputParser, StrOutputParser -from langchain_core.pydantic_v1 import BaseModel, Field - -from langchain.agents.format_scratchpad import format_to_openai_function_messages -from langchain.agents import create_openai_functions_agent, AgentExecutor - -from langchain.prompts.pipeline import PipelinePromptTemplate -from langchain.prompts.prompt import PromptTemplate - -from langgraph.graph import StateGraph, END - -from langsmith import traceable - -load_dotenv() - -from kios_utils.pybt_test import generate_bt_stewardship, render_dot_tree - - -def test_bt(bt_json: json): - test_class = BehaviorTreeFactory() - bt = test_class.from_json_to_simple_bt(bt_json) - # bt = test_class.from_json_to_tree_root(bt_json) - bt_stewardship = generate_bt_stewardship(bt) - # bt_stewardship.setup(timeout=15) - render_dot_tree(bt_stewardship) - - -####################### dirs -current_dir = os.path.dirname(os.path.abspath(__file__)) -scene_path = os.path.join(current_dir, "scene.json") -# bt_json_file_path = os.path.join(current_dir, "behavior_tree.json") -world_state_path = os.path.join(current_dir, "world_state.json") -problem_path = os.path.join(current_dir, "gearset.problem") - - -####################### problem -with open(problem_path, "r") as file: - problem = file.read() - -####################### scene -with open(scene_path, "r") as file: - scene_json_object = json.load(file) - -scene = SceneFactory().create_scene_from_json(scene_json_object) - -####################### world -world_interface = WorldInterface() -with open(world_state_path, "r") as file: - world_state_json_object = json.load(file) - world_interface.load_world_from_json(world_state_json_object) - -####################### robot -robot_interface = RobotInterface( - robot_address="127.0.0.1", - robot_port=12000, -) -robot_interface.setup_scene(scene) - -####################### bt_factory -bt_factory = BehaviorTreeFactory( - world_interface=world_interface, - robot_interface=robot_interface, -) - -####################### behavior_tree_stewardship -behavior_tree_stewardship = BehaviorTreeStewardship( - behaviortree_factory=bt_factory, - world_interface=world_interface, - robot_interface=robot_interface, -) - -####################### * tools -# bt_executor_tool = BehaviorTreeExecutorTool(behavior_tree_stewardship) - -bt_sim_tool = BehaviorTreeSimulatorTool(metadata={"bt_stw": behavior_tree_stewardship}) - -# world_state_query_tool = WorldStateQueryTool(behavior_tree_stewardship) - -executor_tools = [bt_sim_tool] - -# * kios data prompt skeleton dir -data_dir = os.environ.get("KIOS_DATA_DIR").format(username=os.getlogin()) -print(data_dir) -prompt_sk_dir = os.path.join(data_dir, "prompt_skeletons") -prompt_dir = os.path.join(data_dir, "prompts") - -# * refiner chain -llm_spt_re_sk = KiosLLMSupporter() -with open(os.path.join(prompt_sk_dir, "skeleton_refiner.json"), "r") as f: - re_sk_pptsk_json = json.load(f) - re_sk_pptsk = KiosPromptSkeleton.from_json(re_sk_pptsk_json) - llm_spt_re_sk.initialize_from_prompt_skeleton(re_sk_pptsk) -re_sk_ppt = llm_spt_re_sk.create_prompt() -re_sk_llm = ChatOpenAI(model_name=llm_spt_re_sk.model_name, temperature=0) - -re_sk_chain = re_sk_ppt | re_sk_llm | JsonOutputParser() - - -# * skeleton generator chain -llm_spt_skeleton_generator = KiosLLMSupporter() -with open(os.path.join(prompt_sk_dir, "skeleton_generator.json"), "r") as f: - skeleton_generator_pptsk_json = json.load(f) - skeleton_generator_pptsk = KiosPromptSkeleton.from_json( - skeleton_generator_pptsk_json - ) - llm_spt_skeleton_generator.initialize_from_prompt_skeleton(skeleton_generator_pptsk) -skeleton_generator_ppt = llm_spt_skeleton_generator.create_prompt() -skeleton_generator_llm = ChatOpenAI( - model_name=llm_spt_skeleton_generator.model_name, temperature=0 -) - -skeleton_generator_chain = ( - skeleton_generator_ppt - | skeleton_generator_llm - # | StrOutputParser() - | JsonOutputParser() -) - -# ! DISCARDED behavior tree generator chain -# bt_gen_chain = skeleton_generator_chain | re_sk_chain - -############################################### * behavior tree executor agent -# ! NOW SUBSTITUTED BY THE FAKE EXECUTOR FUNCTION -# bt_exe_ppt = hub.pull("hwchase17/openai-functions-agent") -# bt_exe_ppt = ChatPromptTemplate.from_messages( -# [ -# ("system", "You are a helpful assistant"), -# ("user", "The behavior tree is: {behavior_tree}"), -# ("user", "The world state is: {world_state}"), -# MessagesPlaceholder(variable_name="agent_scratchpad"), -# ] -# ) - -# bt_exe_llm = ChatOpenAI(model="gpt-3.5-turbo", temperature=0) - - -# def executor_agent_parse(output): -# # If no function was invoked, return to user -# if "function_call" not in output.additional_kwargs: -# return AgentFinish(return_values={"output": output.content}, log=output.content) - -# # Parse out the function call -# function_call = output.additional_kwargs["function_call"] -# name = function_call["name"] -# inputs = json.loads(function_call["arguments"]) - -# # If the Response function was invoked, return to the user with the function inputs -# if name == "ExecutorResponse": -# return AgentFinish(return_values=inputs, log=str(function_call)) -# # Otherwise, return an agent action -# else: -# return AgentActionMessageLog( -# tool=name, tool_input=inputs, log="", message_log=[output] -# ) - - -# class ExecutorResponse(BaseModel): -# """the result of the behavior tree execution""" - -# hasSucceeded: bool = Field(description="If the behavior tree has succeeded or not") -# world_state: dict[str, list[dict[str, str]]] = Field( -# description="The final world state after the execution" -# ) - - -# bt_exe_llm_with_tools = bt_exe_llm.bind_functions( -# [convert_to_openai_function(t) for t in executor_tools] -# ) - - -# ! BUG -# bt_exe_agent = ( -# { -# "input": lambda x: x["input"], -# # Format agent scratchpad from intermediate steps -# "agent_scratchpad": lambda x: format_to_openai_function_messages( -# x["intermediate_steps"] -# ), -# } -# | bt_exe_ppt -# | bt_exe_llm_with_tools -# | executor_agent_parse -# ) - -# bt_exe_agent_executor = AgentExecutor( -# tools=executor_tools, agent=bt_exe_agent, verbose=True -# ) - - -# * the graph state -class PlanExecuteState(TypedDict): - plan: List[str] - behavior_tree_skeleton: dict # ! not sure if use this or not. - behavior_tree: dict - world_state: Annotated[ - List[dict], operator.add - ] # ! to add, you need to make world_state a list of dict - # = Field(default=None) - past_steps: Annotated[List[Tuple], operator.add] - response: str # ! not sure if use this or not. - # to_user: bool - user_input: str - problem: str - - -##################################################### * planner -# * output schema of the planner -class Plan(BaseModel): - """Plan to follow in future""" - - steps: List[str] = Field( - description="a list of different steps to follow, should be in sorted order" - ) - - -template_file = os.path.join(prompt_dir, "planner/template.txt") -task_file = os.path.join(prompt_dir, "planner/task.txt") -system_file = os.path.join(prompt_dir, "planner/system.txt") -domain_file = os.path.join(prompt_dir, "planner/domain.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {template} - """ -) - -planner_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ], -) - -planner = create_structured_output_runnable( - Plan, ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), planner_ppt_ppl -) - -##################################################### * plan_updaterner - - -class UpdaterResponse(BaseModel): - """used to response to the user for asking for more inputs.""" - - response: str - - -template_file = os.path.join(prompt_dir, "plan_updater/template.txt") -task_file = os.path.join(prompt_dir, "plan_updater/task.txt") -system_file = os.path.join(prompt_dir, "plan_updater/system.txt") -domain_file = os.path.join(prompt_dir, "plan_updater/domain.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {template} - """ -) - -plan_updater_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ], -) - -plan_updater = create_openai_fn_runnable( - [Plan, UpdaterResponse], # * here two schemas are used - ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), - plan_updater_ppt_ppl, -) - - -##################################################### * graph node functions -@traceable(name="user_input_node_step") -async def user_input_step(state: PlanExecuteState): - """ - get the input from the user - """ - print(f"-----user_input_step-----") - if state["response"] is not None: - user_input = input(state["response"]) - state["response"] = None - else: - user_input = input("Your next instruction: ") - - return { - "user_input": user_input, - } - - -@traceable(name="behavior_tree_generate_step") -async def behavior_tree_generate_step(state: PlanExecuteState): - """ - generate the behavior tree based on the instruction - """ - print(f"-----behavior_tree_generate_step-----") - - instruction = state["plan"][0] - latest_world_state = state["world_state"][-1] - - skeleton = await skeleton_generator_chain.ainvoke( - { - "problem": state["problem"], - "world_state": latest_world_state, - "instructions": instruction, - } - ) - - # sk_json = JsonOutputParser().invoke(skeleton) - sk_json = skeleton - - behavior_tree_sk = sk_json["task_plan"]["behavior_tree"] - - test_bt(behavior_tree_sk) - - pause = input("paused here") - - # ! OBJECT CONCENTRATION - - response = await re_sk_chain.ainvoke( - { - "input": skeleton, - } - ) - - behavior_tree = response.get("task_plan").get("behavior_tree") - - init_world_state = [response.get("initial_state")] - - return { - "behavior_tree": behavior_tree, - "world_state": init_world_state, # ! this is necessary because the constraints in problem - } - - -# ! BBWARN INITIALIZE THE CONSTRAINT IN THE PROMPT OR IN THE INITIALIZATION NODE - - -@traceable(name="behavior_tree_execute_step") -async def behavior_tree_execute_step(state: PlanExecuteState): - """ - execute the first step of the plan, append the result to the past steps - """ - print(f"-----behavior_tree_execute_step-----") - - behavior_tree = state["behavior_tree"] - latest_world_state = state["world_state"][-1] - - # exe_input = { - # "behavior_tree": behavior_tree, - # "world_state": latest_world_state, - # } - - # agent_response = await bt_exe_agent_executor.ainvoke( - # { - # "input": exe_input, - # } - # ) - - response = behavior_tree_stewardship.fake_run( - world_state=latest_world_state, bt_json=behavior_tree - ) - - # hasSucceeded = response["hasSucceeded"] - # new_world_state = [response["world_state"]] - hasSucceeded = response.result - new_world_state = [response.world_state] - the_step = state["plan"][0] - - return { - "past_steps": ( - the_step, - hasSucceeded, - ), # * only one because new plan will be generated and old steps are all removed - "world_state": new_world_state, - } - - -@traceable(name="planner_step") -async def planner_step(state: PlanExecuteState): - """ - plan the steps based on user input and world state - """ - print(f"-----plan_step-----") - - plan = await planner.ainvoke( - { - "user_input": state["user_input"], - "world_state": state["world_state"], - } - ) - return {"plan": plan.steps} - - -@traceable(name="plan_updater_step") -async def plan_updater_step(state: PlanExecuteState): - """ - if return a response, then success, response the use, end. - otherwise, return the updated newplan (normally the same as the old plan with the first step popped out. - """ - print(f"-----plan_updater_step-----") - - output = await plan_updater.ainvoke(state) - if isinstance( - output, UpdaterResponse - ): # * determine if it is time to response and end - return { - "response": "Your last instruction has been finished." - } # * Don't need to update. - else: - return {"plan": output.steps} # * update the plan - - -##################################################### * construct the graph - -workflow = StateGraph(PlanExecuteState) - -workflow.add_node("planner", planner_step) - -workflow.add_node("behavior_tree_generator", behavior_tree_generate_step) - -workflow.add_node("behavior_tree_executor", behavior_tree_execute_step) - -workflow.add_node("plan_updater", plan_updater_step) - -workflow.add_node("user_input_node", user_input_step) - -workflow.set_entry_point("user_input_node") - -# workflow.add_edge("user_input_node", "planner") - -workflow.add_edge( - "planner", "behavior_tree_generator" -) # ! may need to add a condition here - -workflow.add_edge("behavior_tree_generator", "behavior_tree_executor") - -workflow.add_edge("behavior_tree_executor", "plan_updater") - - -def plan_updater_should_end(state: PlanExecuteState): - """ - end router - """ - print(f"-----plan_updater_should_end-----") - - if state["response"]: - return True - else: - return False - - -workflow.add_conditional_edges( - "plan_updater", - plan_updater_should_end, - { - True: "user_input_node", - False: "behavior_tree_generator", - }, -) - - -def user_input_should_end(state: PlanExecuteState): - """ - if the user input is empty, then end - """ - print(f"-----user_input_should_end-----") - - # print(f'user_input: {state["user_input"]}') - # print(f'world_state: {state["world_state"]}') - - if not state["user_input"] or state["user_input"] == "": - return True - else: - return False - - -workflow.add_conditional_edges( - "user_input_node", - user_input_should_end, - { - True: END, - False: "planner", - }, -) - -app = workflow.compile() - - -# config = {"recursion_limit": 50} - -inputs = { - "world_state": [world_state_json_object], - "problem": problem, -} - - -async def core_run(): - async for event in app.astream( - inputs, - # config=config, - ): - for k, v in event.items(): - if k != "__end__": - print(v) - - -if __name__ == "__main__": - import asyncio - - asyncio.run(core_run()) diff --git a/experiments/lamp/gearset1_graph_bu_txt.py b/experiments/lamp/gearset1_graph_bu_txt.py deleted file mode 100644 index f226fd54..00000000 --- a/experiments/lamp/gearset1_graph_bu_txt.py +++ /dev/null @@ -1,545 +0,0 @@ -import json -import os - -os.environ["LANGCHAIN_TRACING_V2"] = "true" -os.environ["LANGCHAIN_ENDPOINT"] = "https://api.smith.langchain.com" -os.environ["LANGCHAIN_PROJECT"] = "kios_agent" - -from kios_bt.bt_stewardship import BehaviorTreeStewardship -from kios_scene.scene_factory import SceneFactory -from kios_bt.bt_factory import BehaviorTreeFactory -from kios_robot.robot_interface import RobotInterface -from kios_world.world_interface import WorldInterface - -from kios_agent.llm_supporter import KiosLLMSupporter -from kios_agent.data_types import KiosPromptSkeleton - -from kios_agent.kios_tools import ( - BehaviorTreeExecutorTool, - BehaviorTreeSimulatorTool, - WorldStateQueryTool, -) - -from typing import List, Tuple, Annotated, TypedDict -import operator - -from dotenv import load_dotenv - -from langchain import hub - -from langchain_openai import ChatOpenAI - -from langchain.chains.openai_functions import ( - create_structured_output_runnable, - create_openai_fn_runnable, -) -from langchain_core.utils.function_calling import convert_to_openai_function -from langchain_core.agents import AgentActionMessageLog, AgentFinish -from langchain_core.messages import HumanMessage -from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder -from langchain_core.output_parsers import JsonOutputParser, StrOutputParser -from langchain_core.pydantic_v1 import BaseModel, Field - -from langchain.agents.format_scratchpad import format_to_openai_function_messages -from langchain.agents import create_openai_functions_agent, AgentExecutor - -from langchain.prompts.pipeline import PipelinePromptTemplate -from langchain.prompts.prompt import PromptTemplate - -from langgraph.graph import StateGraph, END - -from langsmith import traceable - -load_dotenv() - - -####################### dirs -current_dir = os.path.dirname(os.path.abspath(__file__)) -scene_path = os.path.join(current_dir, "scene.json") -# bt_json_file_path = os.path.join(current_dir, "behavior_tree.json") -world_state_path = os.path.join(current_dir, "world_state.json") -problem_path = os.path.join(current_dir, "gearset.problem") - - -####################### problem -with open(problem_path, "r") as file: - problem = file.read() - -####################### scene -with open(scene_path, "r") as file: - scene_json_object = json.load(file) - -scene = SceneFactory().create_scene_from_json(scene_json_object) - -####################### world -world_interface = WorldInterface() -with open(world_state_path, "r") as file: - world_state_json_object = json.load(file) - world_interface.load_world_from_json(world_state_json_object) - -####################### robot -robot_interface = RobotInterface( - robot_address="127.0.0.1", - robot_port=12000, -) -robot_interface.setup_scene(scene) - -####################### bt_factory -bt_factory = BehaviorTreeFactory( - world_interface=world_interface, - robot_interface=robot_interface, -) - -####################### behavior_tree_stewardship -behavior_tree_stewardship = BehaviorTreeStewardship( - behaviortree_factory=bt_factory, - world_interface=world_interface, - robot_interface=robot_interface, -) - -####################### * tools -# bt_executor_tool = BehaviorTreeExecutorTool(behavior_tree_stewardship) - -bt_sim_tool = BehaviorTreeSimulatorTool(metadata={"bt_stw": behavior_tree_stewardship}) - -# world_state_query_tool = WorldStateQueryTool(behavior_tree_stewardship) - -executor_tools = [bt_sim_tool] - -# * kios data prompt skeleton dir -data_dir = os.environ.get("KIOS_DATA_DIR").format(username=os.getlogin()) -print(data_dir) -prompt_sk_dir = os.path.join(data_dir, "prompt_skeletons") -prompt_dir = os.path.join(data_dir, "prompts") - -# * refiner chain -llm_spt_re_sk = KiosLLMSupporter() -with open(os.path.join(prompt_sk_dir, "skeleton_refiner.json"), "r") as f: - re_sk_pptsk_json = json.load(f) - re_sk_pptsk = KiosPromptSkeleton.from_json(re_sk_pptsk_json) - llm_spt_re_sk.initialize_from_prompt_skeleton(re_sk_pptsk) -re_sk_ppt = llm_spt_re_sk.create_prompt() -re_sk_llm = ChatOpenAI(model_name=llm_spt_re_sk.model_name, temperature=0) - -re_sk_chain = re_sk_ppt | re_sk_llm | JsonOutputParser() - - -# * skeleton generator chain -llm_spt_skeleton_generator = KiosLLMSupporter() -with open(os.path.join(prompt_sk_dir, "skeleton_generator.json"), "r") as f: - skeleton_generator_pptsk_json = json.load(f) - skeleton_generator_pptsk = KiosPromptSkeleton.from_json( - skeleton_generator_pptsk_json - ) - llm_spt_skeleton_generator.initialize_from_prompt_skeleton(skeleton_generator_pptsk) -skeleton_generator_ppt = llm_spt_skeleton_generator.create_prompt() -skeleton_generator_llm = ChatOpenAI( - model_name=llm_spt_skeleton_generator.model_name, temperature=0 -) - -skeleton_generator_chain = ( - skeleton_generator_ppt - | skeleton_generator_llm - # | StrOutputParser() - | JsonOutputParser() -) - -# ! DISCARDED behavior tree generator chain -# bt_gen_chain = skeleton_generator_chain | re_sk_chain - -############################################### * behavior tree executor agent -# ! NOW SUBSTITUTED BY THE FAKE EXECUTOR FUNCTION -# bt_exe_ppt = hub.pull("hwchase17/openai-functions-agent") -# bt_exe_ppt = ChatPromptTemplate.from_messages( -# [ -# ("system", "You are a helpful assistant"), -# ("user", "The behavior tree is: {behavior_tree}"), -# ("user", "The world state is: {world_state}"), -# MessagesPlaceholder(variable_name="agent_scratchpad"), -# ] -# ) - -# bt_exe_llm = ChatOpenAI(model="gpt-3.5-turbo", temperature=0) - - -# def executor_agent_parse(output): -# # If no function was invoked, return to user -# if "function_call" not in output.additional_kwargs: -# return AgentFinish(return_values={"output": output.content}, log=output.content) - -# # Parse out the function call -# function_call = output.additional_kwargs["function_call"] -# name = function_call["name"] -# inputs = json.loads(function_call["arguments"]) - -# # If the Response function was invoked, return to the user with the function inputs -# if name == "ExecutorResponse": -# return AgentFinish(return_values=inputs, log=str(function_call)) -# # Otherwise, return an agent action -# else: -# return AgentActionMessageLog( -# tool=name, tool_input=inputs, log="", message_log=[output] -# ) - - -# class ExecutorResponse(BaseModel): -# """the result of the behavior tree execution""" - -# hasSucceeded: bool = Field(description="If the behavior tree has succeeded or not") -# world_state: dict[str, list[dict[str, str]]] = Field( -# description="The final world state after the execution" -# ) - - -# bt_exe_llm_with_tools = bt_exe_llm.bind_functions( -# [convert_to_openai_function(t) for t in executor_tools] -# ) - - -# ! BUG -# bt_exe_agent = ( -# { -# "input": lambda x: x["input"], -# # Format agent scratchpad from intermediate steps -# "agent_scratchpad": lambda x: format_to_openai_function_messages( -# x["intermediate_steps"] -# ), -# } -# | bt_exe_ppt -# | bt_exe_llm_with_tools -# | executor_agent_parse -# ) - -# bt_exe_agent_executor = AgentExecutor( -# tools=executor_tools, agent=bt_exe_agent, verbose=True -# ) - - -# * the graph state -class PlanExecuteState(TypedDict): - plan: List[str] - behavior_tree_skeleton: dict # ! not sure if use this or not. - behavior_tree: dict - world_state: Annotated[ - List[dict], operator.add - ] # ! to add, you need to make world_state a list of dict - # = Field(default=None) - past_steps: Annotated[List[Tuple], operator.add] - response: str # ! not sure if use this or not. - # to_user: bool - user_input: str - problem: str - - -##################################################### * planner -# * output schema of the planner -class Plan(BaseModel): - """Plan to follow in future""" - - steps: List[str] = Field( - description="a list of different steps to follow, should be in sorted order" - ) - - -# ! now in data_dir -# planner_prompt = ChatPromptTemplate.from_template( -# """ -# The user input should be an instruction (or several instructions) of the robot assembly task. \ -# For the given user input, summarize and come up with a bullet list task plan. \ -# This plan should involve individual assembly step(s) of robot assembly task.\ -# Do not add any superfluous steps. \ -# Make sure that each step has all the information needed - do not skip steps.\ - -# You should only use assembly actions as the action in the steps. -# The available assembly actions are:\ -# - insert something into something\ -# - place something onto something\ -# - screw something into something\ - -# Example:\ -# user_input: "you should insert the red block into the blue hole"\ -# plan: "insert the red block into the blue hole"\ - -# user_input: "intert the blub into the base, then put the cover onto it."\ -# plan: ["insert the blub into the base", "put the cover onto the blub"]\ - -# world_state: {world_state}\ -# user_input: {user_input}\ -# """ -# ) - -planner_prompt_file = os.path.join(prompt_dir, "planner/template.txt") - -with open(planner_prompt_file, "w") as f: - planner_prompt = f.read() - -planner = create_structured_output_runnable( - Plan, ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), planner_prompt -) - - -##################################################### * plan_updaterner - - -class UpdaterResponse(BaseModel): - """used to response to the user for asking for more inputs.""" - - response: str - - -plan_updater_prompt = ChatPromptTemplate.from_template( - """For the given objective, come up with a simple step by step plan. \ -This plan should involve individual tasks, that if executed correctly will yield the correct answer. Do not add any superfluous steps. \ -The result of the final step should be the final answer. Make sure that each step has all the information needed - do not skip steps. - -Your objective was this: -{user_input} - -Your original plan was this: -{plan} - -You have currently done the follow steps: -{past_steps} - -The world state is: -{world_state} - -Read the world state. Check the original plan from the first step and update your plan with the steps need to be conducted next according to the world state.\ -If no more steps are needed to achieve the objective defined by user input, respond to the user for more input. Otherwise, fill out the plan.\ -Only add steps to the plan that still NEED to be done. Do not return previously done steps as part of the plan.""" -) - -plan_updater = create_openai_fn_runnable( - [Plan, UpdaterResponse], # * here two schemas are used - ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), - plan_updater_prompt, -) - - -##################################################### * graph node functions -@traceable(name="user_input_node_step") -async def user_input_step(state: PlanExecuteState): - """ - get the input from the user - """ - print(f"-----user_input_step-----") - if state["response"] is not None: - user_input = input(state["response"]) - state["response"] = None - else: - user_input = input("Your next instruction: ") - - return { - "user_input": user_input, - } - - -@traceable(name="behavior_tree_generate_step") -async def behavior_tree_generate_step(state: PlanExecuteState): - """ - generate the behavior tree based on the instruction - """ - print(f"-----behavior_tree_generate_step-----") - - instruction = state["plan"][0] - latest_world_state = state["world_state"][-1] - - skeleton = await skeleton_generator_chain.ainvoke( - { - "problem": state["problem"], - "world_state": latest_world_state, - "instructions": instruction, - } - ) - - # ! OBJECT CONCENTRATION - - response = await re_sk_chain.ainvoke( - { - "input": skeleton, - } - ) - - behavior_tree = response.get("task_plan").get("behavior_tree") - - init_world_state = [response.get("initial_state")] - - return { - "behavior_tree": behavior_tree, - "world_state": init_world_state, # ! this is necessary because the constraints in problem - } - - -# ! BBWARN INITIALIZE THE CONSTRAINT IN THE PROMPT OR IN THE INITIALIZATION NODE - - -@traceable(name="behavior_tree_execute_step") -async def behavior_tree_execute_step(state: PlanExecuteState): - """ - execute the first step of the plan, append the result to the past steps - """ - print(f"-----behavior_tree_execute_step-----") - - behavior_tree = state["behavior_tree"] - latest_world_state = state["world_state"][-1] - - # exe_input = { - # "behavior_tree": behavior_tree, - # "world_state": latest_world_state, - # } - - # agent_response = await bt_exe_agent_executor.ainvoke( - # { - # "input": exe_input, - # } - # ) - - response = behavior_tree_stewardship.fake_run( - world_state=latest_world_state, bt_json=behavior_tree - ) - - # hasSucceeded = response["hasSucceeded"] - # new_world_state = [response["world_state"]] - hasSucceeded = response.result - new_world_state = [response.world_state] - the_step = state["plan"][0] - - return { - "past_steps": (the_step, hasSucceeded), # ! why here not a list of tuple? - "world_state": new_world_state, - } - - -@traceable(name="planner_step") -async def planner_step(state: PlanExecuteState): - """ - plan the steps based on user input and world state - """ - print(f"-----plan_step-----") - - plan = await planner.ainvoke( - { - "user_input": state["user_input"], - "world_state": state["world_state"], - } - ) - return {"plan": plan.steps} - - -@traceable(name="plan_updater_step") -async def plan_updater_step(state: PlanExecuteState): - """ - if return a response, then success, response the use, end. - otherwise, return the updated newplan (normally the same as the old plan with the first step popped out. - """ - print(f"-----plan_updater_step-----") - - output = await plan_updater.ainvoke(state) - if isinstance( - output, UpdaterResponse - ): # * determine if it is time to response and end - return { - "response": "Your last instruction has been finished." - } # * Don't need to update. - else: - return {"plan": output.steps} # * update the plan - - -##################################################### * construct the graph - -workflow = StateGraph(PlanExecuteState) - -workflow.add_node("planner", planner_step) - -workflow.add_node("behavior_tree_generator", behavior_tree_generate_step) - -workflow.add_node("behavior_tree_executor", behavior_tree_execute_step) - -workflow.add_node("plan_updater", plan_updater_step) - -workflow.add_node("user_input_node", user_input_step) - -workflow.set_entry_point("user_input_node") - -# workflow.add_edge("user_input_node", "planner") - -workflow.add_edge( - "planner", "behavior_tree_generator" -) # ! may need to add a condition here - -workflow.add_edge("behavior_tree_generator", "behavior_tree_executor") - -workflow.add_edge("behavior_tree_executor", "plan_updater") - - -def plan_updater_should_end(state: PlanExecuteState): - """ - end router - """ - print(f"-----plan_updater_should_end-----") - - if state["response"]: - return True - else: - return False - - -workflow.add_conditional_edges( - "plan_updater", - plan_updater_should_end, - { - True: "user_input_node", - False: "behavior_tree_generator", - }, -) - - -def user_input_should_end(state: PlanExecuteState): - """ - if the user input is empty, then end - """ - print(f"-----user_input_should_end-----") - - # print(f'user_input: {state["user_input"]}') - # print(f'world_state: {state["world_state"]}') - - if not state["user_input"] or state["user_input"] == "": - return True - else: - return False - - -workflow.add_conditional_edges( - "user_input_node", - user_input_should_end, - { - True: END, - False: "planner", - }, -) - -app = workflow.compile() - - -# config = {"recursion_limit": 50} - -inputs = { - "world_state": [world_state_json_object], - "problem": problem, -} - - -async def core_run(): - async for event in app.astream( - inputs, - # config=config, - ): - for k, v in event.items(): - if k != "__end__": - print(v) - - -if __name__ == "__main__": - import asyncio - - asyncio.run(core_run()) diff --git a/experiments/lamp/graph.py b/experiments/lamp/graph.py deleted file mode 100644 index d0ff8d17..00000000 --- a/experiments/lamp/graph.py +++ /dev/null @@ -1,180 +0,0 @@ -from langchain_core.pydantic_v1 import BaseModel, Field -from typing import List, Tuple, Annotated, TypedDict -import operator - -from langchain_openai import ChatOpenAI - -import os - -os.environ["LANGCHAIN_TRACING_V2"] = "true" -os.environ["LANGCHAIN_PROJECT"] = "kios_agent" - -##################################################### * tool -from langchain_community.tools.tavily_search import TavilySearchResults -from kios_agent.kios_tools import BehaviorTreeExecutorTool, BehaviorTreeSimulatorTool - -# tools = [TavilySearchResults(max_results=3)] -tools = [BehaviorTreeSimulatorTool()] # ! need stw - -##################################################### * execution - -from langchain import hub -from langchain.agents import create_openai_functions_agent -from langchain_openai import ChatOpenAI - -prompt = hub.pull("hwchase17/openai-functions-agent") - -# Choose the LLM that will drive the agent -llm = ChatOpenAI(model="gpt-4-turbo-preview") -# Construct the OpenAI Functions agent -agent_runnable = create_openai_functions_agent(llm, tools, prompt) - -from langgraph.prebuilt import create_agent_executor - -agent_executor = create_agent_executor(agent_runnable, tools) - - -# * the graph state -class PlanExecuteState(TypedDict): - input: str - plan: List[str] - world_state: List[dict] - past_steps: Annotated[List[Tuple], operator.add] - response: str - - -##################################################### * planner -# * output schema of the planner -class Plan(BaseModel): - """Plan to follow in future""" - - steps: List[str] = Field( - description="different steps to follow, should be in sorted order" - ) - - -from langchain.chains.openai_functions import create_structured_output_runnable -from langchain_core.prompts import ChatPromptTemplate - -planner_prompt = ChatPromptTemplate.from_template( - """For the given objective, come up with a simple step by step plan. \ -This plan should involve individual tasks, that if executed correctly will yield the correct answer. Do not add any superfluous steps. \ -The result of the final step should be the final answer. Make sure that each step has all the information needed - do not skip steps. - -{objective}""" -) -planner = create_structured_output_runnable( - Plan, ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), planner_prompt -) - -##################################################### * plan_updaterner -from langchain.chains.openai_functions import create_openai_fn_runnable - - -class Response(BaseModel): - """Response to user.""" - - response: str - - -plan_updater_prompt = ChatPromptTemplate.from_template( - """For the given objective, come up with a simple step by step plan. \ -This plan should involve individual tasks, that if executed correctly will yield the correct answer. Do not add any superfluous steps. \ -The result of the final step should be the final answer. Make sure that each step has all the information needed - do not skip steps. - -Your objective was this: -{input} - -Your original plan was this: -{plan} - -You have currently done the follow steps: -{past_steps} - -Update your plan accordingly. If no more steps are needed and you can return to the user, then respond with that. Otherwise, fill out the plan. Only add steps to the plan that still NEED to be done. Do not return previously done steps as part of the plan.""" -) - -plan_updater = create_openai_fn_runnable( - [Plan, Response], # * here two schemas are used - ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), - plan_updater_prompt, -) - - -##################################################### * graph -async def execute_step(state: PlanExecuteState): - """ - execute the first step of the plan, append the result to the past steps - """ - task = state["plan"][0] - agent_response = await agent_executor.ainvoke({"input": task, "chat_history": []}) - return { - "past_steps": (task, agent_response["agent_outcome"].return_values["output"]) - } - - -async def plan_step(state: PlanExecuteState): - """ - plan the steps based on user input - """ - plan = await planner.ainvoke({"objective": state["input"]}) - return {"plan": plan.steps} - - -async def plan_updater_step(state: PlanExecuteState): - """ - if return a response, then success, response the use, end. - otherwise, return the updated newplan (normally the same as the old plan with the first step popped out. - """ - output = await plan_updater.ainvoke(state) - if isinstance(output, Response): # * determine if it is time to response and end - return {"response": output.response} - else: - return {"plan": output.steps} - - -def should_end(state: PlanExecuteState): - """ - end router - """ - if state["response"]: - return True - else: - return False - - -##################################################### * construct the graph -from langgraph.graph import StateGraph, END - -workflow = StateGraph(PlanExecuteState) - -# Add the plan node -workflow.add_node("planner", plan_step) - -# Add the execution step -workflow.add_node("agent", execute_step) - -# Add a plan_updater node -workflow.add_node("plan_updater", plan_updater_step) - -workflow.set_entry_point("planner") - -# From plan we go to agent -workflow.add_edge("planner", "agent") - -# From agent, we plan_updater -workflow.add_edge("agent", "plan_updater") - -workflow.add_conditional_edges( - "plan_updater", - # Next, we pass in the function that will determine which node is called next. - should_end, - { - # If `tools`, then we call the tool node. - True: END, - False: "agent", - }, -) - -# compiles it into a LangChain Runnable, -app = workflow.compile() diff --git a/experiments/lamp/problem.py b/experiments/lamp/problem.py deleted file mode 100644 index 3350099f..00000000 --- a/experiments/lamp/problem.py +++ /dev/null @@ -1,106 +0,0 @@ -import re - - -def parse_init(problem): - init_pattern = r"\(:init\s+(.*?)\s+\)" - element_pattern = r"\((.*?)\)" - - init_match = re.search(init_pattern, problem, re.DOTALL) - if init_match: - init_content = init_match.group(1) - elements = re.findall(element_pattern, init_content) - - parsed_init = [] - for element in elements: - parts = element.split() - name = parts[0] - args = parts[1:] - parsed_init.append({"name": name, "args": args}) - - return parsed_init - else: - return [] - - -properties = [ - "is_free", - "is_equippable", -] - -constraints = [ - "can_manipulate", - "can_screw_to", - "can_insert_to", - "can_place_to", -] - -relations = [ - "hold", - "is_inserted_to", - "is_screwed_to", - "is_placed_to", -] - - -def classify_item(item_name): - if item_name in properties: - return "property" - elif item_name in constraints: - return "constraint" - elif item_name in relations: - return "relation" - else: - raise ValueError(f"Unknown item name: {item_name}") - - -# Usage example: -problem = """ -(define (problem robot_assembly_problem-problem) - (:domain robot_assembly_problem-domain) - (:objects - parallel_box1 parallel_box2 inward_claw outward_claw no_tool - tool - gear1 gear2 gear3 shaft1 shaft2 shaft3 gearbase gearbase_hole1 gearbase_hole3 - part - left_hand - hand - ) - (:init (can_manipulate parallel_box2 gear1) (can_manipulate outward_claw gear2) (can_manipulate outward_claw gear3) (can_manipulate no_tool shaft3) (can_manipulate parallel_box1 shaft1) (can_screw_to leg1 seat) (can_screw_to leg2 seat) (can_insert_to back seat) (can_screw_to nut1 seat) (can_screw_to nut2 seat) (can_screw_to blub lampbase) (can_place_to lamp blub) (can_insert_to shaft1 gearbase_hole1) (can_insert_to shaft3 gearbase_hole3) (can_insert_to gear3 shaft3) (can_insert_to gear2 shaft2) (can_insert_to gear1 shaft1)) - (:goal (and )) - )""" - -parsed_init = parse_init(problem) - -for item in parsed_init: - item_name = item.get("name") - item_type = classify_item(item_name) - - world_state = { - "objects": [], - "constraints": [], - "relations": [], - } - - if item_type == "property": - world_state["objects"].append( - {"name": item["args"][0], "properties": [item_name]} # ! list or item? - ) - - elif item_type == "constraint": - world_state["constraints"].append( - { - "source": item["args"][0], - "name": item_name, - "target": item["args"][1], - } - ) - elif item_type == "relation": - world_state["relations"].append( - { - "source": item["args"][0], - "name": item_name, - "target": item["args"][1], - } - ) - else: - pass - - -print(parsed_init) diff --git a/experiments/lamp/problem_set/problem_000.json b/experiments/lamp/problem_set/problem_000.json deleted file mode 100644 index 43b78838..00000000 --- a/experiments/lamp/problem_set/problem_000.json +++ /dev/null @@ -1,158 +0,0 @@ -{ - "target": "target: is_screwed_to(lampbulb, lampbase)", - "initial_world_state": { - "objects": [ - { - "name": "clampgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "parallelgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "inwardgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "outwardgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "defaultgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "lampbase", - "properties": [] - }, - { - "name": "lampshade", - "properties": [] - }, - { - "name": "lampbulb", - "properties": [] - }, - { - "name": "left_hand", - "properties": [] - } - ], - "constraints": [ - { - "source": "clampgripper", - "name": "can_manipulate", - "target": "lampbulb" - }, - { - "source": "outwardgripper", - "name": "can_manipulate", - "target": "lampshade" - }, - { - "source": "lampbulb", - "name": "can_screw_to", - "target": "lampbase" - }, - { - "source": "lampshade", - "name": "can_place_to", - "target": "lampbulb" - } - ], - "relations": [ - { - "source": "left_hand", - "name": "hold", - "target": "defaultgripper" - } - ] - }, - "result": { - "summary": "selector to screw the lampbulb into the lampbase with the clampgripper in the left_hand", - "name": "selector: screw(left_hand, clampgripper, lampbulb, lampbase)", - "children": [ - { - "summary": "the target is to make the lampbulb be screwed into the lampbase", - "name": "target: is_screwed_to(lampbulb, lampbase)" - }, - { - "summary": "sequence to screw the lampbulb into the lampbase with the clampgripper in the left_hand", - "name": "sequence: screw(left_hand, clampgripper, lampbulb, lampbase)", - "children": [ - { - "summary": "selector to change tool in left_hand from defaultgripper to clampgripper", - "name": "selector: change_tool(left_hand, defaultgripper, clampgripper)", - "children": [ - { - "summary": "the target is to make left_hand hold clampgripper", - "name": "target: hold(left_hand, clampgripper)" - }, - { - "summary": "sequence to change tool in left_hand from defaultgripper to clampgripper", - "name": "sequence: change_tool(left_hand, defaultgripper, clampgripper)", - "children": [ - { - "summary": "a precondition is left_hand is holding defaultgripper", - "name": "precondition: hold(left_hand, defaultgripper)" - }, - { - "summary": "a precondition is defaultgripper is empty", - "name": "precondition: is_empty(defaultgripper)" - }, - { - "summary": "the action to change tool in left_hand from defaultgripper to clampgripper", - "name": "action: change_tool(left_hand, defaultgripper, clampgripper)" - } - ] - } - ] - }, - { - "summary": "selector to pick up the lampbulb with the clampgripper in the left_hand", - "name": "selector: pick_up(left_hand, clampgripper, lampbulb)", - "children": [ - { - "summary": "the target is to make the clampgripper hold the lampbulb", - "name": "target: hold(clampgripper, lampbulb)" - }, - { - "summary": "sequence to pick up the lampbulb with the clampgripper in the left_hand", - "name": "sequence: pick_up(left_hand, clampgripper, lampbulb)", - "children": [ - { - "summary": "a precondition is the clampgripper is empty", - "name": "precondition: is_empty(clampgripper)" - }, - { - "summary": "a precondition is the left_hand is holding clampgripper", - "name": "precondition: hold(left_hand, clampgripper)" - }, - { - "summary": "the action to pick up the lampbulb with the clampgripper in the left_hand", - "name": "action: pick_up(left_hand, clampgripper, lampbulb)" - } - ] - } - ] - }, - { - "summary": "the action to screw the lampbulb into the lampbase with the clampgripper in the left_hand", - "name": "action: screw(left_hand, clampgripper, lampbulb, lampbase)" - } - ] - } - ] - } -} \ No newline at end of file diff --git a/experiments/lamp/problem_set/problem_001.json b/experiments/lamp/problem_set/problem_001.json deleted file mode 100644 index d74dd904..00000000 --- a/experiments/lamp/problem_set/problem_001.json +++ /dev/null @@ -1,193 +0,0 @@ -{ - "target": "target: is_screwed_to(lampbulb, lampbase)", - "initial_world_state": { - "objects": [ - { - "name": "clampgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "parallelgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "inwardgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "outwardgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "defaultgripper", - "properties": [] - }, - { - "name": "lampbase", - "properties": [] - }, - { - "name": "lampshade", - "properties": [] - }, - { - "name": "lampbulb", - "properties": [] - }, - { - "name": "cube", - "properties": [] - }, - { - "name": "designring", - "properties": [] - }, - { - "name": "left_hand", - "properties": [] - } - ], - "constraints": [ - { - "source": "clampgripper", - "name": "can_manipulate", - "target": "lampbulb" - }, - { - "source": "outwardgripper", - "name": "can_manipulate", - "target": "lampshade" - }, - { - "source": "lampbulb", - "name": "can_screw_to", - "target": "lampbase" - }, - { - "source": "lampshade", - "name": "can_place_to", - "target": "lampbulb" - } - ], - "relations": [ - { - "source": "left_hand", - "name": "hold", - "target": "defaultgripper" - }, - { - "source": "defaultgripper", - "name": "hold", - "target": "cube" - } - ] - }, - "result": { - "summary": "selector to screw the lampbulb into the lampbase with the clampgripper in the left_hand", - "name": "selector: screw(left_hand, clampgripper, lampbulb, lampbase)", - "children": [ - { - "summary": "the target is to make the lampbulb be screwed into the lampbase", - "name": "target: is_screwed_to(lampbulb, lampbase)" - }, - { - "summary": "sequence to screw the lampbulb into the lampbase with the clampgripper in the left_hand", - "name": "sequence: screw(left_hand, clampgripper, lampbulb, lampbase)", - "children": [ - { - "summary": "selector to change tool in left_hand from defaultgripper to clampgripper", - "name": "selector: change_tool(left_hand, defaultgripper, clampgripper)", - "children": [ - { - "summary": "the target is to make left_hand hold clampgripper", - "name": "target: hold(left_hand, clampgripper)" - }, - { - "summary": "sequence to change tool in left_hand from defaultgripper to clampgripper", - "name": "sequence: change_tool(left_hand, defaultgripper, clampgripper)", - "children": [ - { - "summary": "a precondition is left_hand is holding defaultgripper", - "name": "precondition: hold(left_hand, defaultgripper)" - }, - { - "summary": "selector to put down the cube held by the defaultgripper in the left_hand", - "name": "selector: put_down(left_hand, defaultgripper, cube)", - "children": [ - { - "summary": "the target is to make the defaultgripper empty", - "name": "target: is_empty(defaultgripper)" - }, - { - "summary": "sequence to put down the cube held by the defaultgripper in the left_hand", - "name": "sequence: put_down(left_hand, defaultgripper, cube)", - "children": [ - { - "summary": "a precondition is the left_hand is holding defaultgripper", - "name": "precondition: hold(left_hand, defaultgripper)" - }, - { - "summary": "a precondition is the defaultgripper is holding cube", - "name": "precondition: hold(defaultgripper, cube)" - }, - { - "summary": "the action to put down the cube held by the defaultgripper in the left_hand", - "name": "action: put_down(left_hand, defaultgripper, cube)" - } - ] - } - ] - }, - { - "summary": "the action to change tool in left_hand from defaultgripper to clampgripper", - "name": "action: change_tool(left_hand, defaultgripper, clampgripper)" - } - ] - } - ] - }, - { - "summary": "selector to pick up the lampbulb with the clampgripper in the left_hand", - "name": "selector: pick_up(left_hand, clampgripper, lampbulb)", - "children": [ - { - "summary": "the target is to make the clampgripper hold the lampbulb", - "name": "target: hold(clampgripper, lampbulb)" - }, - { - "summary": "sequence to pick up the lampbulb with the clampgripper in the left_hand", - "name": "sequence: pick_up(left_hand, clampgripper, lampbulb)", - "children": [ - { - "summary": "a precondition is the clampgripper is empty", - "name": "precondition: is_empty(clampgripper)" - }, - { - "summary": "a precondition is the left_hand is holding clampgripper", - "name": "precondition: hold(left_hand, clampgripper)" - }, - { - "summary": "the action to pick up the lampbulb with the clampgripper in the left_hand", - "name": "action: pick_up(left_hand, clampgripper, lampbulb)" - } - ] - } - ] - }, - { - "summary": "the action to screw the lampbulb into the lampbase with the clampgripper in the left_hand", - "name": "action: screw(left_hand, clampgripper, lampbulb, lampbase)" - } - ] - } - ] - } -} \ No newline at end of file diff --git a/experiments/lamp/problem_set/problem_002.json b/experiments/lamp/problem_set/problem_002.json deleted file mode 100644 index 8b8c661d..00000000 --- a/experiments/lamp/problem_set/problem_002.json +++ /dev/null @@ -1,171 +0,0 @@ -{ - "target": "target: is_placed_to(lampshade, lampbulb)", - "initial_world_state": { - "objects": [ - { - "name": "clampgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "parallelgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "inwardgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "outwardgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "defaultgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "lampbase", - "properties": [] - }, - { - "name": "lampshade", - "properties": [] - }, - { - "name": "lampbulb", - "properties": [] - }, - { - "name": "cube", - "properties": [] - }, - { - "name": "designring", - "properties": [] - }, - { - "name": "left_hand", - "properties": [] - } - ], - "constraints": [ - { - "source": "clampgripper", - "name": "can_manipulate", - "target": "lampbulb" - }, - { - "source": "outwardgripper", - "name": "can_manipulate", - "target": "lampshade" - }, - { - "source": "lampbulb", - "name": "can_screw_to", - "target": "lampbase" - }, - { - "source": "lampshade", - "name": "can_place_to", - "target": "lampbulb" - } - ], - "relations": [ - { - "source": "left_hand", - "name": "hold", - "target": "inwardgripper" - }, - { - "source": "lampbulb", - "name": "is_screwed_to", - "target": "lampbase" - } - ] - }, - "result": { - "summary": "selector to place the lampshade into the lampbulb with the outwardgripper in the left_hand", - "name": "selector: place(left_hand, outwardgripper, lampshade, lampbulb)", - "children": [ - { - "summary": "the target is to make the lampshade be placed into the lampbulb", - "name": "target: is_placed_to(lampshade, lampbulb)" - }, - { - "summary": "sequence to place the lampshade into the lampbulb with the outwardgripper in the left_hand", - "name": "sequence: place(left_hand, outwardgripper, lampshade, lampbulb)", - "children": [ - { - "summary": "selector to change tool in left_hand from inwardgripper to outwardgripper", - "name": "selector: change_tool(left_hand, inwardgripper, outwardgripper)", - "children": [ - { - "summary": "the target is to make left_hand hold outwardgripper", - "name": "target: hold(left_hand, outwardgripper)" - }, - { - "summary": "sequence to change tool in left_hand from inwardgripper to outwardgripper", - "name": "sequence: change_tool(left_hand, inwardgripper, outwardgripper)", - "children": [ - { - "summary": "a precondition is left_hand is holding inwardgripper", - "name": "precondition: hold(left_hand, inwardgripper)" - }, - { - "summary": "a precondition is inwardgripper is empty", - "name": "precondition: is_empty(inwardgripper)" - }, - { - "summary": "the action to change tool in left_hand from inwardgripper to outwardgripper", - "name": "action: change_tool(left_hand, inwardgripper, outwardgripper)" - } - ] - } - ] - }, - { - "summary": "selector to pick up the lampshade with the outwardgripper in the left_hand", - "name": "selector: pick_up(left_hand, outwardgripper, lampshade)", - "children": [ - { - "summary": "the target is to make the outwardgripper hold the lampshade", - "name": "target: hold(outwardgripper, lampshade)" - }, - { - "summary": "sequence to pick up the lampshade with the outwardgripper in the left_hand", - "name": "sequence: pick_up(left_hand, outwardgripper, lampshade)", - "children": [ - { - "summary": "a precondition is the outwardgripper is empty", - "name": "precondition: is_empty(outwardgripper)" - }, - { - "summary": "a precondition is the left_hand is holding outwardgripper", - "name": "precondition: hold(left_hand, outwardgripper)" - }, - { - "summary": "the action to pick up the lampshade with the outwardgripper in the left_hand", - "name": "action: pick_up(left_hand, outwardgripper, lampshade)" - } - ] - } - ] - }, - { - "summary": "the action to place the lampshade into the lampbulb with the outwardgripper in the left_hand", - "name": "action: place(left_hand, outwardgripper, lampshade, lampbulb)" - } - ] - } - ] - } -} \ No newline at end of file diff --git a/experiments/lamp/problem_set/problem_003.json b/experiments/lamp/problem_set/problem_003.json deleted file mode 100644 index e1f40644..00000000 --- a/experiments/lamp/problem_set/problem_003.json +++ /dev/null @@ -1,104 +0,0 @@ -{ - "target": "target: is_placed_to(lampshade, lampbulb)", - "initial_world_state": { - "objects": [ - { - "name": "clampgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "parallelgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "inwardgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "outwardgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "defaultgripper", - "properties": [ - "is_empty" - ] - }, - { - "name": "lampbase", - "properties": [] - }, - { - "name": "lampshade", - "properties": [] - }, - { - "name": "lampbulb", - "properties": [] - }, - { - "name": "cube", - "properties": [] - }, - { - "name": "designring", - "properties": [] - }, - { - "name": "left_hand", - "properties": [] - } - ], - "constraints": [ - { - "source": "clampgripper", - "name": "can_manipulate", - "target": "lampbulb" - }, - { - "source": "outwardgripper", - "name": "can_manipulate", - "target": "lampshade" - }, - { - "source": "lampbulb", - "name": "can_screw_to", - "target": "lampbase" - }, - { - "source": "lampshade", - "name": "can_place_to", - "target": "lampbulb" - } - ], - "relations": [ - { - "source": "left_hand", - "name": "hold", - "target": "inwardgripper" - }, - { - "source": "lampbulb", - "name": "is_screwed_to", - "target": "lampbase" - }, - { - "source": "lampshade", - "name": "is_placed_to", - "target": "lampbulb" - } - ] - }, - "result": { - "summary": "the target is to place the lampshade onto the lampbulb", - "name": "target: is_placed_to(lampshade, lampbulb)" - } -} \ No newline at end of file diff --git a/experiments/lamp/rec_sk_gen.py b/experiments/lamp/rec_sk_gen.py deleted file mode 100644 index 2c4d7ac9..00000000 --- a/experiments/lamp/rec_sk_gen.py +++ /dev/null @@ -1,738 +0,0 @@ -import json -import os -from pprint import pprint - -os.environ["LANGCHAIN_TRACING_V2"] = "true" -os.environ["LANGCHAIN_ENDPOINT"] = "https://api.smith.langchain.com" -os.environ["LANGCHAIN_PROJECT"] = "kios_agent" - -from kios_bt.bt_stewardship import BehaviorTreeStewardship -from kios_scene.scene_factory import SceneFactory -from kios_bt.bt_factory import BehaviorTreeFactory -from kios_robot.robot_interface import RobotInterface -from kios_world.world_interface import WorldInterface - -from kios_agent.llm_supporter import KiosLLMSupporter -from kios_agent.data_types import KiosPromptSkeleton - -from kios_agent.kios_tools import ( - BehaviorTreeExecutorTool, - BehaviorTreeSimulatorTool, - WorldStateQueryTool, -) - -from typing import List, Tuple, Annotated, TypedDict -import operator - -from dotenv import load_dotenv - -from langchain import hub - -from langchain_openai import ChatOpenAI - -from langchain.chains.openai_functions import ( - create_structured_output_runnable, - create_openai_fn_runnable, -) -from langchain_core.utils.function_calling import convert_to_openai_function -from langchain_core.agents import AgentActionMessageLog, AgentFinish -from langchain_core.messages import HumanMessage -from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder -from langchain_core.output_parsers import JsonOutputParser, StrOutputParser -from langchain_core.pydantic_v1 import BaseModel, Field - -from langchain.agents.format_scratchpad import format_to_openai_function_messages -from langchain.agents import create_openai_functions_agent, AgentExecutor - -from langchain.prompts.pipeline import PipelinePromptTemplate -from langchain.prompts.prompt import PromptTemplate - -from langchain_community.document_loaders import TextLoader -from langchain_openai import OpenAIEmbeddings - -from langchain_community.vectorstores import FAISS -from langchain.text_splitter import ( - RecursiveCharacterTextSplitter, - CharacterTextSplitter, -) - -from langgraph.graph import StateGraph, END - -from langsmith import traceable - -load_dotenv() - -from kios_utils.pybt_test import generate_bt_stewardship, render_dot_tree -from kios_utils.pddl_problem_parser import parse_problem_init, parse_problem_objects - - -def render_bt(bt_json: json): - test_class = BehaviorTreeFactory() - bt = test_class.from_json_to_simple_bt(bt_json) - # bt = test_class.from_json_to_tree_root(bt_json) - bt_stewardship = generate_bt_stewardship(bt) - # bt_stewardship.setup(timeout=15) - render_dot_tree(bt_stewardship) - - -####################### dirs -current_dir = os.path.dirname(os.path.abspath(__file__)) -scene_path = os.path.join(current_dir, "scene.json") -# bt_json_file_path = os.path.join(current_dir, "behavior_tree.json") -world_state_path = os.path.join(current_dir, "world_state.json") -problem_path = os.path.join(current_dir, "gearset.problem") -domain_knowledge_path = os.path.join(current_dir, "domain_knowledge.txt") - -####################### problem -with open(problem_path, "r") as file: - problem = file.read() - -####################### scene -with open(scene_path, "r") as file: - scene_json_object = json.load(file) - -scene = SceneFactory().create_scene_from_json(scene_json_object) - -####################### world -world_interface = WorldInterface() -with open(world_state_path, "r") as file: - world_state_json_object = json.load(file) - world_interface.load_world_from_json(world_state_json_object) - -from_problem = parse_problem_init(problem=problem) -objects = parse_problem_objects(problem=problem) -world_interface.update_world(from_problem) - -world_state_json = world_interface.get_world_to_json() -pprint(from_problem) -pprint(world_state_json) -pprint(objects) - - -####################### robot -robot_interface = RobotInterface( - robot_address="127.0.0.1", - robot_port=12000, -) -robot_interface.setup_scene(scene) - -####################### bt_factory -bt_factory = BehaviorTreeFactory( - world_interface=world_interface, - robot_interface=robot_interface, -) - -####################### behavior_tree_stewardship -behavior_tree_stewardship = BehaviorTreeStewardship( - behaviortree_factory=bt_factory, - world_interface=world_interface, - robot_interface=robot_interface, -) - -####################### * tools -# bt_executor_tool = BehaviorTreeExecutorTool(behavior_tree_stewardship) - -bt_sim_tool = BehaviorTreeSimulatorTool(metadata={"bt_stw": behavior_tree_stewardship}) - -# world_state_query_tool = WorldStateQueryTool(behavior_tree_stewardship) - -executor_tools = [bt_sim_tool] - -# * kios data prompt skeleton dir -data_dir = os.environ.get("KIOS_DATA_DIR").format(username=os.getlogin()) -print(data_dir) -prompt_sk_dir = os.path.join(data_dir, "prompt_skeletons") -prompt_dir = os.path.join(data_dir, "prompts") - -# * retriever chain -# from langchain_experimental.text_splitter import SemanticChunker - -# # with open(domain_knowledge_path, "r") as f: -# # domain_knowledge = f.read() - -# # text_splitter = SemanticChunker(OpenAIEmbeddings()) - -# # docs = text_splitter.create_documents([domain_knowledge]) - - -# dk_loader = TextLoader(domain_knowledge_path) -# domain_knowledge_doc = dk_loader.load() - -# embeddings = OpenAIEmbeddings() - -# text_splitter = RecursiveCharacterTextSplitter( -# chunk_size=300, -# chunk_overlap=10, -# length_function=len, -# is_separator_regex=False, -# ) -# # text_splitter = CharacterTextSplitter() -# docs = text_splitter.split_documents(domain_knowledge_doc) -# vector_store = FAISS.from_documents(docs, embeddings) - -# retriever = vector_store.as_retriever() - -# docs = retriever.get_relevant_documents( -# "how to fulfill the condition 'is_free'?", -# k=3, -# threshold=0.5, -# ) - -# pprint(docs) - - -# * the graph state -class PlanExecuteState(TypedDict): - plan: List[str] - behavior_tree: dict - world_state: Annotated[ - List[dict], operator.add - ] # ! to add, you need to make world_state a list of dict - past_steps: Annotated[List[Tuple], operator.add] - user_input: str - objects: dict[str, list[str]] - last_behavior_tree: dict - last_failed_node: dict - runtime_world_state: dict - behavior_tree_execution_summary: str - BTExecutionHasSucceeded: bool - - -########## verifier - -behaviortree_file = os.path.join(prompt_dir, "vv/behaviortree.txt") -template_file = os.path.join(prompt_dir, "vv/template.txt") -task_file = os.path.join(prompt_dir, "vv/task.txt") -system_file = os.path.join(prompt_dir, "vv/system.txt") -object_file = os.path.join(prompt_dir, "vv/object.txt") -domain_file = os.path.join(prompt_dir, "vv/domain.txt") -state_file = os.path.join(prompt_dir, "vv/state.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) -with open(object_file, "r") as f: - object_ppt = PromptTemplate.from_template(f.read()) -with open(behaviortree_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - behaviortree_ppt = ppt_tmp.partial(input=f.read()) - -with open(state_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - state_ppt = ppt_tmp.partial(input=f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {state} - - {object} - - {behaviortree} - - {template} - """ -) - -verifier_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ("object", object_ppt), - ("behaviortree", behaviortree_ppt), - ("state", state_ppt), - ], -) - -verifier_chain = ( - verifier_ppt_ppl - | ChatOpenAI(model="gpt-4-turbo-preview", temperature=0) - | StrOutputParser() -) - - -# * output schema of the recursive_sk_generator -class SolutionBehaviortree(BaseModel): - """the fixed behavior tree""" - - behavior_tree: dict = Field( - description="the behavior tree that adheres the required format." - ) - - -behaviortree_file = os.path.join(prompt_dir, "rec_sk_gen/behaviortree.txt") -template_file = os.path.join(prompt_dir, "rec_sk_gen/template.txt") -task_file = os.path.join(prompt_dir, "rec_sk_gen/task.txt") -system_file = os.path.join(prompt_dir, "rec_sk_gen/system.txt") -object_file = os.path.join(prompt_dir, "rec_sk_gen/object.txt") -domain_file = os.path.join(prompt_dir, "rec_sk_gen/domain.txt") -example_file = os.path.join(prompt_dir, "rec_sk_gen/example.txt") -state_file = os.path.join(prompt_dir, "rec_sk_gen/state.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) -with open(object_file, "r") as f: - object_ppt = PromptTemplate.from_template(f.read()) -with open(behaviortree_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - behaviortree_ppt = ppt_tmp.partial(input=f.read()) - -with open(state_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - state_ppt = ppt_tmp.partial(input=f.read()) - -with open(example_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - example_ppt = ppt_tmp.partial(input=f.read()) -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {state} - - {object} - - {behaviortree} - - {example} - - {template} - """ -) - -re_sk_gen_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ("object", object_ppt), - ("behaviortree", behaviortree_ppt), - ("example", example_ppt), - ("state", state_ppt), - ], -) - -rec_sk_gen_chain = ( - re_sk_gen_ppt_ppl - | ChatOpenAI(model="gpt-4-turbo-preview", temperature=0) - | JsonOutputParser() -) - -# recursive_sk_generator = create_structured_output_runnable( -# SolutionBehaviortree, -# ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), -# re_sk_gen_ppt_ppl, -# mode="openai-json", -# ) - - -##################################################### * planner -# * output schema of the planner -class Plan(BaseModel): - """Plan to follow in future""" - - steps: List[str] = Field( - description="a list of different steps to follow, should be in sorted order" - ) - - -template_file = os.path.join(prompt_dir, "planner/template.txt") -task_file = os.path.join(prompt_dir, "planner/task.txt") -system_file = os.path.join(prompt_dir, "planner/system.txt") -domain_file = os.path.join(prompt_dir, "planner/domain.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {template} - """ -) - -planner_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ], -) - -planner = create_structured_output_runnable( - Plan, ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), planner_ppt_ppl -) - -##################################################### * plan_updaterner - -template_file = os.path.join(prompt_dir, "plan_updater/template.txt") -task_file = os.path.join(prompt_dir, "plan_updater/task.txt") -system_file = os.path.join(prompt_dir, "plan_updater/system.txt") -domain_file = os.path.join(prompt_dir, "plan_updater/domain.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {template} - """ -) - -plan_updater_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ], -) - -plan_updater = create_structured_output_runnable( - Plan, ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), plan_updater_ppt_ppl -) -# ! create openai fn runnable has bug - - -##################################################### * graph node functions -@traceable(name="user_input_node_step") -async def user_input_step(state: PlanExecuteState): - """ - get the input from the user - """ - print(f"-----user_input_step-----") - - user_input = input("Your next instruction: ") - - return { - "user_input": user_input, - } - - -@traceable(name="behavior_tree_generate_step") -async def behavior_tree_generate_step(state: PlanExecuteState): - """ - generate the behavior tree based on the instruction - """ - print(f"-----behavior_tree_generate_step-----") - - instruction = state["plan"][0] - if state["runtime_world_state"] is None: - runtime_world_state = state["world_state"][-1] - else: - runtime_world_state = state["runtime_world_state"] - objects = state["objects"] - - # solution_bt = await recursive_sk_generator.ainvoke( - # { - # "objects": objects, - # "runtime_world_state": runtime_world_state, - # "last_behavior_tree": state["last_behavior_tree"], - # "last_failed_node": state["last_failed_node"], - # "instructions": instruction, - # } - # ) - - bt_skeleton = await rec_sk_gen_chain.ainvoke( - { - "objects": objects, - "runtime_world_state": runtime_world_state, - "last_behavior_tree": state["last_behavior_tree"], - "last_failed_node": state["last_failed_node"], - "summary": state["behavior_tree_execution_summary"], - "instructions": instruction, - } - ) - - # print(solution_bt) - # pause = input("paused here") - - # bt_skeleton = solution_bt.behavior_tree - - render_bt(bt_skeleton) - - pause = input("paused here") - - return { - "last_behavior_tree": bt_skeleton, - "BTExecutionHasSucceeded": False, - } - - -@traceable(name="behavior_tree_execute_step") -async def behavior_tree_execute_step(state: PlanExecuteState): - """ - execute the first step of the plan, append the result to the past steps - """ - print(f"-----behavior_tree_execute_step-----") - - this_step = state["plan"][0] - behavior_tree_skeleton = state["last_behavior_tree"] - latest_world_state = state["world_state"][-1] - - # * first sim run - tree_result, skeleton_json = behavior_tree_stewardship.sk_sim_run( - world_state=latest_world_state, skeleton_json=behavior_tree_skeleton - ) - - pprint(tree_result.to_json()) - pause = input("paused here") - - # * check result - if tree_result.result == "success": - - # * second fake run - tree_result, skeleton_json = behavior_tree_stewardship.sk_fake_run( - world_state=latest_world_state, skeleton_json=behavior_tree_skeleton - ) - - pprint(tree_result.to_json()) - pause = input("paused here") - - if tree_result.result == "success": - return { - "BTExecutionHasSucceeded": True, - "past_steps": ( - this_step, - tree_result.result, - ), # * only one because new plan will be generated and old steps are all removed - "world_state": [tree_result.world_state], - # * clear all - "last_behavior_tree": None, - "last_failed_node": None, - "runtime_world_state": None, - "behavior_tree_execution_summary": None, - } - else: - return { - "world_state": [tree_result.world_state], # * real world state - "last_behavior_tree": skeleton_json, - "last_failed_node": { - "summary": tree_result.final_node.get("summary"), - "name": tree_result.final_node.get("name"), - "identifier": tree_result.final_node.get("identifier"), - }, - "runtime_world_state": tree_result.world_state, - "behavior_tree_execution_summary": tree_result.summary, - } - - else: - return { - # ! do not change world state - "last_behavior_tree": skeleton_json, - "last_failed_node": { - "summary": tree_result.final_node.get("summary"), - "name": tree_result.final_node.get("name"), - "identifier": tree_result.final_node.get("identifier"), - }, - "runtime_world_state": tree_result.world_state, - "behavior_tree_execution_summary": tree_result.summary, - } - - -@traceable(name="planner_step") -async def planner_step(state: PlanExecuteState): - """ - plan the steps based on user input and world state - """ - print(f"-----plan_step-----") - - plan = await planner.ainvoke( - { - "user_input": state["user_input"], - "world_state": state["world_state"], - } - ) - return {"plan": plan.steps} - - -@traceable(name="plan_updater_step") -async def plan_updater_step(state: PlanExecuteState): - """ - if return a response, then success, response the use, end. - otherwise, return the updated newplan (normally the same as the old plan with the first step popped out. - """ - print(f"-----plan_updater_step-----") - - # ! BOOOM! BUG! - output = await plan_updater.ainvoke( - { - "user_input": state["user_input"], - "plan": state["plan"], - "world_state": state["world_state"], - "past_steps": state["past_steps"], - } - ) - # if isinstance( - # output, UpdaterResponse - # ): # * determine if it is time to response and end - # return { - # "response": "Your last instruction has been finished." - # } # * Don't need to update. - # else: - return {"plan": output.steps} # * update the plan - - -##################################################### * construct the graph - -workflow = StateGraph(PlanExecuteState) - -workflow.add_node("planner", planner_step) - -workflow.add_node("behavior_tree_generator", behavior_tree_generate_step) - -workflow.add_node("behavior_tree_executor", behavior_tree_execute_step) - -workflow.add_node("plan_updater", plan_updater_step) - -workflow.add_node("user_input_node", user_input_step) - -workflow.set_entry_point("user_input_node") - -workflow.add_edge( - "planner", "behavior_tree_generator" -) # ! may need to add a condition here - -workflow.add_edge("behavior_tree_generator", "behavior_tree_executor") - - -def executor_should_end(state: PlanExecuteState): - """ - end router - """ - print(f"-----executor_should_end-----") - - if state["BTExecutionHasSucceeded"] == True: - state["BTExecutionHasSucceeded"] = False - return True - else: - return False - - -workflow.add_conditional_edges( - "behavior_tree_executor", - executor_should_end, - { - True: "plan_updater", - False: "behavior_tree_generator", - }, -) - - -def plan_updater_should_end(state: PlanExecuteState): - """ - end router - """ - print(f"-----plan_updater_should_end-----") - - # if state["response"]: - # return True - # else: - # return False - if state["plan"] == [] or len(state["plan"]) == 0: - return True - else: - return False - - -workflow.add_conditional_edges( - "plan_updater", - plan_updater_should_end, - { - True: "user_input_node", - False: "behavior_tree_generator", - }, -) - - -def user_input_should_end(state: PlanExecuteState): - """ - if the user input is empty, then end - """ - print(f"-----user_input_should_end-----") - - # print(f'user_input: {state["user_input"]}') - # print(f'world_state: {state["world_state"]}') - - if not state["user_input"] or state["user_input"] == "": - return True - else: - return False - - -workflow.add_conditional_edges( - "user_input_node", - user_input_should_end, - { - True: END, - False: "planner", - }, -) - -app = workflow.compile() - -config = {"recursion_limit": 500} - -inputs = { - "world_state": [world_state_json], - "objects": objects, -} - - -async def core_run(): - async for event in app.astream( - inputs, - config=config, - ): - for k, v in event.items(): - if k != "__end__": - print(v) - - -if __name__ == "__main__": - import asyncio - - asyncio.run(core_run()) diff --git a/experiments/lamp/rec_sk_seq.py b/experiments/lamp/rec_sk_seq.py deleted file mode 100644 index 65c5a20e..00000000 --- a/experiments/lamp/rec_sk_seq.py +++ /dev/null @@ -1,682 +0,0 @@ -import json -import os -from pprint import pprint -from typing import List, Tuple, Annotated, TypedDict -import operator - -""" -recursive generation + sequence guidance + simulation -""" - -os.environ["LANGCHAIN_TRACING_V2"] = "true" -os.environ["LANGCHAIN_ENDPOINT"] = "https://api.smith.langchain.com" -os.environ["LANGCHAIN_PROJECT"] = "kios_agent" - -from kios_bt.bt_stewardship import BehaviorTreeStewardship -from kios_scene.scene_factory import SceneFactory -from kios_bt.bt_factory import BehaviorTreeFactory -from kios_robot.robot_interface import RobotInterface -from kios_world.world_interface import WorldInterface - -from kios_agent.llm_supporter import KiosLLMSupporter -from kios_agent.data_types import KiosPromptSkeleton - -from kios_agent.kios_tools import ( - BehaviorTreeExecutorTool, - BehaviorTreeSimulatorTool, - WorldStateQueryTool, -) - -from dotenv import load_dotenv - -from langchain import hub - -from langchain_openai import ChatOpenAI - -from langchain.chains.openai_functions import ( - create_structured_output_runnable, - create_openai_fn_runnable, -) -from langchain_core.utils.function_calling import convert_to_openai_function -from langchain_core.agents import AgentActionMessageLog, AgentFinish -from langchain_core.messages import HumanMessage -from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder -from langchain_core.output_parsers import JsonOutputParser, StrOutputParser -from langchain_core.pydantic_v1 import BaseModel, Field - -from langchain.agents.format_scratchpad import format_to_openai_function_messages -from langchain.agents import create_openai_functions_agent, AgentExecutor - -from langchain.prompts.pipeline import PipelinePromptTemplate -from langchain.prompts.prompt import PromptTemplate - -from langchain_community.document_loaders import TextLoader -from langchain_openai import OpenAIEmbeddings - -from langchain_community.vectorstores import FAISS -from langchain.text_splitter import ( - RecursiveCharacterTextSplitter, - CharacterTextSplitter, -) - -from langgraph.graph import StateGraph, END - -from langsmith import traceable - -load_dotenv() - -from kios_utils.pybt_test import generate_bt_stewardship, render_dot_tree -from kios_utils.pddl_problem_parser import parse_problem_init, parse_problem_objects - - -def render_bt(bt_json: json): - test_class = BehaviorTreeFactory() - bt = test_class.from_json_to_simple_bt(bt_json) - # bt = test_class.from_json_to_tree_root(bt_json) - bt_stewardship = generate_bt_stewardship(bt) - # bt_stewardship.setup(timeout=15) - render_dot_tree(bt_stewardship) - - -####################### dirs -current_dir = os.path.dirname(os.path.abspath(__file__)) -scene_path = os.path.join(current_dir, "scene.json") -# bt_json_file_path = os.path.join(current_dir, "behavior_tree.json") -world_state_path = os.path.join(current_dir, "world_state.json") -problem_path = os.path.join(current_dir, "gearset.problem") -domain_knowledge_path = os.path.join(current_dir, "domain_knowledge.txt") - -####################### problem -with open(problem_path, "r") as file: - problem = file.read() - -####################### scene -with open(scene_path, "r") as file: - scene_json_object = json.load(file) - -scene = SceneFactory().create_scene_from_json(scene_json_object) - -####################### world -world_interface = WorldInterface() -with open(world_state_path, "r") as file: - world_state_json_object = json.load(file) - world_interface.load_world_from_json(world_state_json_object) - -from_problem = parse_problem_init(problem=problem) -objects = parse_problem_objects(problem=problem) -world_interface.update_world(from_problem) - -world_state_json = world_interface.get_world_to_json() -pprint(from_problem) -pprint(world_state_json) -pprint(objects) - - -####################### robot -robot_interface = RobotInterface( - robot_address="127.0.0.1", - robot_port=12000, -) -robot_interface.setup_scene(scene) - -####################### bt_factory -bt_factory = BehaviorTreeFactory( - world_interface=world_interface, - robot_interface=robot_interface, -) - -####################### behavior_tree_stewardship -behavior_tree_stewardship = BehaviorTreeStewardship( - behaviortree_factory=bt_factory, - world_interface=world_interface, - robot_interface=robot_interface, -) - -####################### * tools -# bt_executor_tool = BehaviorTreeExecutorTool(behavior_tree_stewardship) - -bt_sim_tool = BehaviorTreeSimulatorTool(metadata={"bt_stw": behavior_tree_stewardship}) - -# world_state_query_tool = WorldStateQueryTool(behavior_tree_stewardship) - -executor_tools = [bt_sim_tool] - -# * kios data prompt skeleton dir -data_dir = os.environ.get("KIOS_DATA_DIR").format(username=os.getlogin()) -print(data_dir) -prompt_sk_dir = os.path.join(data_dir, "prompt_skeletons") -prompt_dir = os.path.join(data_dir, "prompts") - - -# * the graph state -class PlanExecuteState(TypedDict): - plan: List[str] - behavior_tree: dict - world_state: Annotated[ - List[dict], operator.add - ] # ! to add, you need to make world_state a list of dict - past_steps: Annotated[List[Tuple], operator.add] - user_input: str - objects: dict[str, list[str]] - last_behavior_tree: dict - last_failed_node: dict - runtime_world_state: dict - behavior_tree_execution_summary: str - BTExecutionHasSucceeded: bool - - action_sequence: List[str] - - -########## verifier - -template_file = os.path.join(prompt_dir, "seq_planner/template.txt") -task_file = os.path.join(prompt_dir, "seq_planner/task.txt") -system_file = os.path.join(prompt_dir, "seq_planner/system.txt") -domain_file = os.path.join(prompt_dir, "seq_planner/domain.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {template} - """ -) - -seq_planner_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ], -) - -seq_planner_chain = ( - seq_planner_ppt_ppl - | ChatOpenAI(model="gpt-4-turbo-preview", temperature=0) - | StrOutputParser() -) - - -# * output schema of the recursive_sk_generator -class SolutionBehaviortree(BaseModel): - """the fixed behavior tree""" - - behavior_tree: dict = Field( - description="the behavior tree that adheres the required format." - ) - - -behaviortree_file = os.path.join(prompt_dir, "rec_sk_gen/behaviortree.txt") -template_file = os.path.join(prompt_dir, "rec_sk_gen/template.txt") -task_file = os.path.join(prompt_dir, "rec_sk_gen/task.txt") -system_file = os.path.join(prompt_dir, "rec_sk_gen/system.txt") -object_file = os.path.join(prompt_dir, "rec_sk_gen/object.txt") -domain_file = os.path.join(prompt_dir, "rec_sk_gen/domain.txt") -example_file = os.path.join(prompt_dir, "rec_sk_gen/example.txt") -state_file = os.path.join(prompt_dir, "rec_sk_gen/state.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) -with open(object_file, "r") as f: - object_ppt = PromptTemplate.from_template(f.read()) -with open(behaviortree_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - behaviortree_ppt = ppt_tmp.partial(input=f.read()) - -with open(state_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - state_ppt = ppt_tmp.partial(input=f.read()) - -with open(example_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - example_ppt = ppt_tmp.partial(input=f.read()) -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {state} - - {object} - - {behaviortree} - - {example} - - {template} - """ -) -# * recureive_sk_generator -re_sk_gen_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ("object", object_ppt), - ("behaviortree", behaviortree_ppt), - ("example", example_ppt), - ("state", state_ppt), - ], -) - -rec_sk_gen_chain = ( - re_sk_gen_ppt_ppl - | ChatOpenAI(model="gpt-4-turbo-preview", temperature=0) - | JsonOutputParser() -) - -# recursive_sk_generator = create_structured_output_runnable( -# SolutionBehaviortree, -# ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), -# re_sk_gen_ppt_ppl, -# mode="openai-json", -# ) - - -##################################################### * planner -# * output schema of the planner -class Plan(BaseModel): - """Plan to follow in future""" - - steps: List[str] = Field( - description="a list of different steps to follow, should be in sorted order" - ) - - -template_file = os.path.join(prompt_dir, "planner/template.txt") -task_file = os.path.join(prompt_dir, "planner/task.txt") -system_file = os.path.join(prompt_dir, "planner/system.txt") -domain_file = os.path.join(prompt_dir, "planner/domain.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {template} - """ -) - -planner_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ], -) - -planner = create_structured_output_runnable( - Plan, ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), planner_ppt_ppl -) - -##################################################### * plan_updaterner - -template_file = os.path.join(prompt_dir, "plan_updater/template.txt") -task_file = os.path.join(prompt_dir, "plan_updater/task.txt") -system_file = os.path.join(prompt_dir, "plan_updater/system.txt") -domain_file = os.path.join(prompt_dir, "plan_updater/domain.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {template} - """ -) - -plan_updater_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ], -) - -plan_updater = create_structured_output_runnable( - Plan, ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), plan_updater_ppt_ppl -) -# ! create openai fn runnable has bug - - -##################################################### * graph node functions -@traceable(name="user_input_node_step") -async def user_input_step(state: PlanExecuteState): - """ - get the input from the user - """ - print(f"-----user_input_step-----") - - user_input = input("Your next instruction: ") - - return { - "user_input": user_input, - } - - -@traceable(name="sequence_generate_step") -async def sequence_generate_step(state: PlanExecuteState): - """ - generate the sequence based on the instruction - """ - print(f"-----sequence_generate_step-----") - - instruction = state["plan"][0] - start_world_state = state["world_state"][-1] - objects = state["objects"] - - action_sequence = await seq_planner_chain.ainvoke( - { - "objects": objects, - "start_world_state": start_world_state, - "user_instruction": instruction, - } - ) - - return { - "action_sequence": action_sequence, - } - - -@traceable(name="behavior_tree_generate_step") -async def behavior_tree_generate_step(state: PlanExecuteState): - """ - generate the behavior tree based on the instruction - """ - print(f"-----behavior_tree_generate_step-----") - - instruction = state["plan"][0] - if state["runtime_world_state"] is None: - runtime_world_state = state["world_state"][-1] - else: - runtime_world_state = state["runtime_world_state"] - objects = state["objects"] - - bt_skeleton = await rec_sk_gen_chain.ainvoke( - { - "objects": objects, - "runtime_world_state": runtime_world_state, - "last_behavior_tree": state["last_behavior_tree"], - "last_failed_node": state["last_failed_node"], - "summary": state["behavior_tree_execution_summary"], - "action_sequence": state["action_sequence"], - } - ) - - render_bt(bt_skeleton) - - pause = input("paused here") - - return { - "last_behavior_tree": bt_skeleton, - "BTExecutionHasSucceeded": False, - } - - -@traceable(name="behavior_tree_execute_step") -async def behavior_tree_execute_step(state: PlanExecuteState): - """ - execute the first step of the plan, append the result to the past steps - """ - print(f"-----behavior_tree_execute_step-----") - - this_step = state["plan"][0] - behavior_tree_skeleton = state["last_behavior_tree"] - latest_world_state = state["world_state"][-1] - - # * first sim run - tree_result, skeleton_json = behavior_tree_stewardship.sk_sim_run( - world_state=latest_world_state, skeleton_json=behavior_tree_skeleton - ) - - pprint(tree_result.to_json()) - pause = input("paused here") - - # * check result - if tree_result.result == "success": - - # * second fake run - tree_result, skeleton_json = behavior_tree_stewardship.sk_fake_run( - world_state=latest_world_state, skeleton_json=behavior_tree_skeleton - ) - - pprint(tree_result.to_json()) - pause = input("paused here") - - if tree_result.result == "success": - return { - "BTExecutionHasSucceeded": True, - "past_steps": ( - this_step, - tree_result.result, - ), # * only one because new plan will be generated and old steps are all removed - "world_state": [tree_result.world_state], - # * clear all - "last_behavior_tree": None, - "last_failed_node": None, - "runtime_world_state": None, - "behavior_tree_execution_summary": None, - } - else: - return { - "world_state": [tree_result.world_state], # * real world state - "last_behavior_tree": skeleton_json, - "last_failed_node": { - "summary": tree_result.final_node.get("summary"), - "name": tree_result.final_node.get("name"), - "identifier": tree_result.final_node.get("identifier"), - }, - "runtime_world_state": tree_result.world_state, - "behavior_tree_execution_summary": tree_result.summary, - } - - else: - return { - # ! do not change world state - "last_behavior_tree": skeleton_json, - "last_failed_node": { - "summary": tree_result.final_node.get("summary"), - "name": tree_result.final_node.get("name"), - "identifier": tree_result.final_node.get("identifier"), - }, - "runtime_world_state": tree_result.world_state, - "behavior_tree_execution_summary": tree_result.summary, - } - - -@traceable(name="planner_step") -async def planner_step(state: PlanExecuteState): - """ - plan the steps based on user input and world state - """ - print(f"-----plan_step-----") - - plan = await planner.ainvoke( - { - "user_input": state["user_input"], - "world_state": state["world_state"], - } - ) - return {"plan": plan.steps} - - -@traceable(name="plan_updater_step") -async def plan_updater_step(state: PlanExecuteState): - """ - if return a response, then success, response the use, end. - otherwise, return the updated newplan (normally the same as the old plan with the first step popped out. - """ - print(f"-----plan_updater_step-----") - - # ! BOOOM! BUG! - output = await plan_updater.ainvoke( - { - "user_input": state["user_input"], - "plan": state["plan"], - "world_state": state["world_state"], - "past_steps": state["past_steps"], - } - ) - # if isinstance( - # output, UpdaterResponse - # ): # * determine if it is time to response and end - # return { - # "response": "Your last instruction has been finished." - # } # * Don't need to update. - # else: - return {"plan": output.steps} # * update the plan - - -##################################################### * construct the graph - -workflow = StateGraph(PlanExecuteState) - -workflow.add_node("planner", planner_step) -workflow.add_node("sequence_generator", sequence_generate_step) -workflow.add_node("behavior_tree_generator", behavior_tree_generate_step) -workflow.add_node("behavior_tree_executor", behavior_tree_execute_step) -workflow.add_node("plan_updater", plan_updater_step) -workflow.add_node("user_input_node", user_input_step) -workflow.set_entry_point("user_input_node") -workflow.add_edge("planner", "sequence_generator") -workflow.add_edge("sequence_generator", "behavior_tree_generator") -workflow.add_edge("behavior_tree_generator", "behavior_tree_executor") - - -def executor_should_end(state: PlanExecuteState): - """ - end router - """ - print(f"-----executor_should_end-----") - - if state["BTExecutionHasSucceeded"] == True: - state["BTExecutionHasSucceeded"] = False - return True - else: - return False - - -workflow.add_conditional_edges( - "behavior_tree_executor", - executor_should_end, - { - True: "plan_updater", - False: "behavior_tree_generator", - }, -) - - -def plan_updater_should_end(state: PlanExecuteState): - """ - end router - """ - print(f"-----plan_updater_should_end-----") - - if state["plan"] == [] or len(state["plan"]) == 0: - return True - else: - return False - - -workflow.add_conditional_edges( - "plan_updater", - plan_updater_should_end, - { - True: "user_input_node", - False: "behavior_tree_generator", - }, -) - - -def user_input_should_end(state: PlanExecuteState): - """ - if the user input is empty, then end - """ - print(f"-----user_input_should_end-----") - - if not state["user_input"] or state["user_input"] == "": - return True - else: - return False - - -workflow.add_conditional_edges( - "user_input_node", - user_input_should_end, - { - True: END, - False: "planner", - }, -) - -app = workflow.compile() - -config = {"recursion_limit": 500} - -inputs = { - "world_state": [world_state_json], - "objects": objects, -} - - -async def core_run(): - async for event in app.astream( - inputs, - config=config, - ): - for k, v in event.items(): - if k != "__end__": - print(v) - - -if __name__ == "__main__": - # import asyncio - - # asyncio.run(core_run()) - pass diff --git a/experiments/lamp/rec_sk_unit.py b/experiments/lamp/rec_sk_unit.py deleted file mode 100644 index 8808bae4..00000000 --- a/experiments/lamp/rec_sk_unit.py +++ /dev/null @@ -1,677 +0,0 @@ -import json -import os -from pprint import pprint -from typing import List, Tuple, Annotated, TypedDict -import operator - -os.environ["LANGCHAIN_TRACING_V2"] = "true" -os.environ["LANGCHAIN_ENDPOINT"] = "https://api.smith.langchain.com" -os.environ["LANGCHAIN_PROJECT"] = "kios_agent" - -from kios_bt.bt_stewardship import BehaviorTreeStewardship -from kios_scene.scene_factory import SceneFactory -from kios_bt.bt_factory import BehaviorTreeFactory -from kios_robot.robot_interface import RobotInterface -from kios_world.world_interface import WorldInterface - -from kios_agent.llm_supporter import KiosLLMSupporter -from kios_agent.data_types import KiosPromptSkeleton - -from kios_agent.kios_tools import ( - BehaviorTreeExecutorTool, - BehaviorTreeSimulatorTool, - WorldStateQueryTool, -) - -from dotenv import load_dotenv - -from langchain import hub - -from langchain_openai import ChatOpenAI - -from langchain.chains.openai_functions import ( - create_structured_output_runnable, - create_openai_fn_runnable, -) -from langchain_core.utils.function_calling import convert_to_openai_function -from langchain_core.agents import AgentActionMessageLog, AgentFinish -from langchain_core.messages import HumanMessage -from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder -from langchain_core.output_parsers import JsonOutputParser, StrOutputParser -from langchain_core.pydantic_v1 import BaseModel, Field - -from langchain.agents.format_scratchpad import format_to_openai_function_messages -from langchain.agents import create_openai_functions_agent, AgentExecutor - -from langchain.prompts.pipeline import PipelinePromptTemplate -from langchain.prompts.prompt import PromptTemplate - -from langchain_community.document_loaders import TextLoader -from langchain_openai import OpenAIEmbeddings - -from langchain_community.vectorstores import FAISS -from langchain.text_splitter import ( - RecursiveCharacterTextSplitter, - CharacterTextSplitter, -) - -from langgraph.graph import StateGraph, END - -from langsmith import traceable - -load_dotenv() - -from kios_utils.pybt_test import generate_bt_stewardship, render_dot_tree -from kios_utils.pddl_problem_parser import parse_problem_init, parse_problem_objects - - -def render_bt(bt_json: json): - test_class = BehaviorTreeFactory() - bt = test_class.from_json_to_simple_bt(bt_json) - # bt = test_class.from_json_to_tree_root(bt_json) - bt_stewardship = generate_bt_stewardship(bt) - # bt_stewardship.setup(timeout=15) - render_dot_tree(bt_stewardship) - - -####################### dirs -current_dir = os.path.dirname(os.path.abspath(__file__)) -scene_path = os.path.join(current_dir, "scene.json") -# bt_json_file_path = os.path.join(current_dir, "behavior_tree.json") -world_state_path = os.path.join(current_dir, "world_state.json") -problem_path = os.path.join(current_dir, "gearset.problem") -domain_knowledge_path = os.path.join(current_dir, "domain_knowledge.txt") - -####################### problem -with open(problem_path, "r") as file: - problem = file.read() - -####################### scene -with open(scene_path, "r") as file: - scene_json_object = json.load(file) - -scene = SceneFactory().create_scene_from_json(scene_json_object) - -####################### world -world_interface = WorldInterface() -with open(world_state_path, "r") as file: - world_state_json_object = json.load(file) - world_interface.load_world_from_json(world_state_json_object) - -from_problem = parse_problem_init(problem=problem) -objects = parse_problem_objects(problem=problem) -world_interface.update_world(from_problem) - -world_state_json = world_interface.get_world_to_json() -pprint(from_problem) -pprint(world_state_json) -pprint(objects) - - -####################### robot -robot_interface = RobotInterface( - robot_address="127.0.0.1", - robot_port=12000, -) -robot_interface.setup_scene(scene) - -####################### bt_factory -bt_factory = BehaviorTreeFactory( - world_interface=world_interface, - robot_interface=robot_interface, -) - -####################### behavior_tree_stewardship -behavior_tree_stewardship = BehaviorTreeStewardship( - behaviortree_factory=bt_factory, - world_interface=world_interface, - robot_interface=robot_interface, -) - -####################### * tools -# bt_executor_tool = BehaviorTreeExecutorTool(behavior_tree_stewardship) - -bt_sim_tool = BehaviorTreeSimulatorTool(metadata={"bt_stw": behavior_tree_stewardship}) - -# world_state_query_tool = WorldStateQueryTool(behavior_tree_stewardship) - -executor_tools = [bt_sim_tool] - -# * kios data prompt skeleton dir -data_dir = os.environ.get("KIOS_DATA_DIR").format(username=os.getlogin()) -print(data_dir) -prompt_sk_dir = os.path.join(data_dir, "prompt_skeletons") -prompt_dir = os.path.join(data_dir, "prompts") - - -# * the graph state -class PlanExecuteState(TypedDict): - plan: List[str] - behavior_tree: dict - world_state: Annotated[ - List[dict], operator.add - ] # ! to add, you need to make world_state a list of dict - past_steps: Annotated[List[Tuple], operator.add] - user_input: str - objects: dict[str, list[str]] - last_behavior_tree: dict - last_failed_node: dict - runtime_world_state: dict - behavior_tree_execution_summary: str - BTExecutionHasSucceeded: bool - - action_sequence: List[str] - - -########## verifier - -template_file = os.path.join(prompt_dir, "seq_planner/template.txt") -task_file = os.path.join(prompt_dir, "seq_planner/task.txt") -system_file = os.path.join(prompt_dir, "seq_planner/system.txt") -domain_file = os.path.join(prompt_dir, "seq_planner/domain.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {template} - """ -) - -seq_planner_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ], -) - -seq_planner_chain = ( - seq_planner_ppt_ppl - | ChatOpenAI(model="gpt-4-turbo-preview", temperature=0) - | StrOutputParser() -) - - -# * output schema of the recursive_sk_generator -class SolutionBehaviortree(BaseModel): - """the fixed behavior tree""" - - behavior_tree: dict = Field( - description="the behavior tree that adheres the required format." - ) - - -behaviortree_file = os.path.join(prompt_dir, "rec_sk_gen/behaviortree.txt") -template_file = os.path.join(prompt_dir, "rec_sk_gen/template.txt") -task_file = os.path.join(prompt_dir, "rec_sk_gen/task.txt") -system_file = os.path.join(prompt_dir, "rec_sk_gen/system.txt") -object_file = os.path.join(prompt_dir, "rec_sk_gen/object.txt") -domain_file = os.path.join(prompt_dir, "rec_sk_gen/domain.txt") -example_file = os.path.join(prompt_dir, "rec_sk_gen/example.txt") -state_file = os.path.join(prompt_dir, "rec_sk_gen/state.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) -with open(object_file, "r") as f: - object_ppt = PromptTemplate.from_template(f.read()) -with open(behaviortree_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - behaviortree_ppt = ppt_tmp.partial(input=f.read()) - -with open(state_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - state_ppt = ppt_tmp.partial(input=f.read()) - -with open(example_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - example_ppt = ppt_tmp.partial(input=f.read()) -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {state} - - {object} - - {behaviortree} - - {example} - - {template} - """ -) - -re_sk_gen_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ("object", object_ppt), - ("behaviortree", behaviortree_ppt), - ("example", example_ppt), - ("state", state_ppt), - ], -) - -rec_sk_gen_chain = ( - re_sk_gen_ppt_ppl - | ChatOpenAI(model="gpt-4-turbo-preview", temperature=0) - | JsonOutputParser() -) - -# recursive_sk_generator = create_structured_output_runnable( -# SolutionBehaviortree, -# ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), -# re_sk_gen_ppt_ppl, -# mode="openai-json", -# ) - - -##################################################### * planner -# * output schema of the planner -class Plan(BaseModel): - """Plan to follow in future""" - - steps: List[str] = Field( - description="a list of different steps to follow, should be in sorted order" - ) - - -template_file = os.path.join(prompt_dir, "planner/template.txt") -task_file = os.path.join(prompt_dir, "planner/task.txt") -system_file = os.path.join(prompt_dir, "planner/system.txt") -domain_file = os.path.join(prompt_dir, "planner/domain.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {template} - """ -) - -planner_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ], -) - -planner = create_structured_output_runnable( - Plan, ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), planner_ppt_ppl -) - -##################################################### * plan_updaterner - -template_file = os.path.join(prompt_dir, "plan_updater/template.txt") -task_file = os.path.join(prompt_dir, "plan_updater/task.txt") -system_file = os.path.join(prompt_dir, "plan_updater/system.txt") -domain_file = os.path.join(prompt_dir, "plan_updater/domain.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {template} - """ -) - -plan_updater_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ], -) - -plan_updater = create_structured_output_runnable( - Plan, ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), plan_updater_ppt_ppl -) -# ! create openai fn runnable has bug - - -##################################################### * graph node functions -@traceable(name="user_input_node_step") -async def user_input_step(state: PlanExecuteState): - """ - get the input from the user - """ - print(f"-----user_input_step-----") - - user_input = input("Your next instruction: ") - - return { - "user_input": user_input, - } - - -@traceable(name="sequence_generate_step") -async def sequence_generate_step(state: PlanExecuteState): - """ - generate the sequence based on the instruction - """ - print(f"-----sequence_generate_step-----") - - instruction = state["plan"][0] - start_world_state = state["world_state"][-1] - objects = state["objects"] - - action_sequence = await seq_planner_chain.ainvoke( - { - "objects": objects, - "start_world_state": start_world_state, - "user_instruction": instruction, - } - ) - - return { - "action_sequence": action_sequence, - } - - -@traceable(name="behavior_tree_generate_step") -async def behavior_tree_generate_step(state: PlanExecuteState): - """ - generate the behavior tree based on the instruction - """ - print(f"-----behavior_tree_generate_step-----") - - instruction = state["plan"][0] - if state["runtime_world_state"] is None: - runtime_world_state = state["world_state"][-1] - else: - runtime_world_state = state["runtime_world_state"] - objects = state["objects"] - - bt_skeleton = await rec_sk_gen_chain.ainvoke( - { - "objects": objects, - "runtime_world_state": runtime_world_state, - "last_behavior_tree": state["last_behavior_tree"], - "last_failed_node": state["last_failed_node"], - "summary": state["behavior_tree_execution_summary"], - "action_sequence": state["action_sequence"], - } - ) - - render_bt(bt_skeleton) - - pause = input("paused here") - - return { - "last_behavior_tree": bt_skeleton, - "BTExecutionHasSucceeded": False, - } - - -@traceable(name="behavior_tree_execute_step") -async def behavior_tree_execute_step(state: PlanExecuteState): - """ - execute the first step of the plan, append the result to the past steps - """ - print(f"-----behavior_tree_execute_step-----") - - this_step = state["plan"][0] - behavior_tree_skeleton = state["last_behavior_tree"] - latest_world_state = state["world_state"][-1] - - # * first sim run - tree_result, skeleton_json = behavior_tree_stewardship.sk_sim_run( - world_state=latest_world_state, skeleton_json=behavior_tree_skeleton - ) - - pprint(tree_result.to_json()) - pause = input("paused here") - - # * check result - if tree_result.result == "success": - - # * second fake run - tree_result, skeleton_json = behavior_tree_stewardship.sk_fake_run( - world_state=latest_world_state, skeleton_json=behavior_tree_skeleton - ) - - pprint(tree_result.to_json()) - pause = input("paused here") - - if tree_result.result == "success": - return { - "BTExecutionHasSucceeded": True, - "past_steps": ( - this_step, - tree_result.result, - ), # * only one because new plan will be generated and old steps are all removed - "world_state": [tree_result.world_state], - # * clear all - "last_behavior_tree": None, - "last_failed_node": None, - "runtime_world_state": None, - "behavior_tree_execution_summary": None, - } - else: - return { - "world_state": [tree_result.world_state], # * real world state - "last_behavior_tree": skeleton_json, - "last_failed_node": { - "summary": tree_result.final_node.get("summary"), - "name": tree_result.final_node.get("name"), - "identifier": tree_result.final_node.get("identifier"), - }, - "runtime_world_state": tree_result.world_state, - "behavior_tree_execution_summary": tree_result.summary, - } - - else: - return { - # ! do not change world state - "last_behavior_tree": skeleton_json, - "last_failed_node": { - "summary": tree_result.final_node.get("summary"), - "name": tree_result.final_node.get("name"), - "identifier": tree_result.final_node.get("identifier"), - }, - "runtime_world_state": tree_result.world_state, - "behavior_tree_execution_summary": tree_result.summary, - } - - -@traceable(name="planner_step") -async def planner_step(state: PlanExecuteState): - """ - plan the steps based on user input and world state - """ - print(f"-----plan_step-----") - - plan = await planner.ainvoke( - { - "user_input": state["user_input"], - "world_state": state["world_state"], - } - ) - return {"plan": plan.steps} - - -@traceable(name="plan_updater_step") -async def plan_updater_step(state: PlanExecuteState): - """ - if return a response, then success, response the use, end. - otherwise, return the updated newplan (normally the same as the old plan with the first step popped out. - """ - print(f"-----plan_updater_step-----") - - # ! BOOOM! BUG! - output = await plan_updater.ainvoke( - { - "user_input": state["user_input"], - "plan": state["plan"], - "world_state": state["world_state"], - "past_steps": state["past_steps"], - } - ) - # if isinstance( - # output, UpdaterResponse - # ): # * determine if it is time to response and end - # return { - # "response": "Your last instruction has been finished." - # } # * Don't need to update. - # else: - return {"plan": output.steps} # * update the plan - - -##################################################### * construct the graph - -workflow = StateGraph(PlanExecuteState) - -workflow.add_node("planner", planner_step) -workflow.add_node("sequence_generator", sequence_generate_step) -workflow.add_node("behavior_tree_generator", behavior_tree_generate_step) -workflow.add_node("behavior_tree_executor", behavior_tree_execute_step) -workflow.add_node("plan_updater", plan_updater_step) -workflow.add_node("user_input_node", user_input_step) -workflow.set_entry_point("user_input_node") -workflow.add_edge("planner", "sequence_generator") -workflow.add_edge("sequence_generator", "behavior_tree_generator") -workflow.add_edge("behavior_tree_generator", "behavior_tree_executor") - - -def executor_should_end(state: PlanExecuteState): - """ - end router - """ - print(f"-----executor_should_end-----") - - if state["BTExecutionHasSucceeded"] == True: - state["BTExecutionHasSucceeded"] = False - return True - else: - return False - - -workflow.add_conditional_edges( - "behavior_tree_executor", - executor_should_end, - { - True: "plan_updater", - False: "behavior_tree_generator", - }, -) - - -def plan_updater_should_end(state: PlanExecuteState): - """ - end router - """ - print(f"-----plan_updater_should_end-----") - - if state["plan"] == [] or len(state["plan"]) == 0: - return True - else: - return False - - -workflow.add_conditional_edges( - "plan_updater", - plan_updater_should_end, - { - True: "user_input_node", - False: "behavior_tree_generator", - }, -) - - -def user_input_should_end(state: PlanExecuteState): - """ - if the user input is empty, then end - """ - print(f"-----user_input_should_end-----") - - if not state["user_input"] or state["user_input"] == "": - return True - else: - return False - - -workflow.add_conditional_edges( - "user_input_node", - user_input_should_end, - { - True: END, - False: "planner", - }, -) - -app = workflow.compile() - -config = {"recursion_limit": 500} - -inputs = { - "world_state": [world_state_json], - "objects": objects, -} - - -async def core_run(): - async for event in app.astream( - inputs, - config=config, - ): - for k, v in event.items(): - if k != "__end__": - print(v) - - -if __name__ == "__main__": - import asyncio - - asyncio.run(core_run()) diff --git a/experiments/lamp/rec_vv.py b/experiments/lamp/rec_vv.py deleted file mode 100644 index ad534551..00000000 --- a/experiments/lamp/rec_vv.py +++ /dev/null @@ -1,732 +0,0 @@ -import json -import os -from pprint import pprint - -os.environ["LANGCHAIN_TRACING_V2"] = "true" -os.environ["LANGCHAIN_ENDPOINT"] = "https://api.smith.langchain.com" -os.environ["LANGCHAIN_PROJECT"] = "kios_agent" - -from kios_bt.bt_stewardship import BehaviorTreeStewardship -from kios_scene.scene_factory import SceneFactory -from kios_bt.bt_factory import BehaviorTreeFactory -from kios_robot.robot_interface import RobotInterface -from kios_world.world_interface import WorldInterface - -from kios_agent.llm_supporter import KiosLLMSupporter -from kios_agent.data_types import KiosPromptSkeleton - -from kios_agent.kios_tools import ( - BehaviorTreeExecutorTool, - BehaviorTreeSimulatorTool, - WorldStateQueryTool, -) - -from typing import List, Tuple, Annotated, TypedDict -import operator - -from dotenv import load_dotenv - -from langchain import hub - -from langchain_openai import ChatOpenAI - -from langchain.chains.openai_functions import ( - create_structured_output_runnable, - create_openai_fn_runnable, -) -from langchain_core.utils.function_calling import convert_to_openai_function -from langchain_core.agents import AgentActionMessageLog, AgentFinish -from langchain_core.messages import HumanMessage -from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder -from langchain_core.output_parsers import JsonOutputParser, StrOutputParser -from langchain_core.pydantic_v1 import BaseModel, Field - -from langchain.agents.format_scratchpad import format_to_openai_function_messages -from langchain.agents import create_openai_functions_agent, AgentExecutor - -from langchain.prompts.pipeline import PipelinePromptTemplate -from langchain.prompts.prompt import PromptTemplate - -from langchain_community.document_loaders import TextLoader -from langchain_openai import OpenAIEmbeddings - -from langchain_community.vectorstores import FAISS -from langchain.text_splitter import ( - RecursiveCharacterTextSplitter, - CharacterTextSplitter, -) - -from langgraph.graph import StateGraph, END - -from langsmith import traceable - -load_dotenv() - -from kios_utils.pybt_test import generate_bt_stewardship, render_dot_tree -from kios_utils.pddl_problem_parser import parse_problem_init, parse_problem_objects - - -def render_bt(bt_json: json): - test_class = BehaviorTreeFactory() - bt = test_class.from_json_to_simple_bt(bt_json) - # bt = test_class.from_json_to_tree_root(bt_json) - bt_stewardship = generate_bt_stewardship(bt) - # bt_stewardship.setup(timeout=15) - render_dot_tree(bt_stewardship) - - -####################### dirs -current_dir = os.path.dirname(os.path.abspath(__file__)) -scene_path = os.path.join(current_dir, "scene.json") -# bt_json_file_path = os.path.join(current_dir, "behavior_tree.json") -world_state_path = os.path.join(current_dir, "world_state.json") -problem_path = os.path.join(current_dir, "gearset.problem") -domain_knowledge_path = os.path.join(current_dir, "domain_knowledge.txt") - -####################### problem -with open(problem_path, "r") as file: - problem = file.read() - -####################### scene -with open(scene_path, "r") as file: - scene_json_object = json.load(file) - -scene = SceneFactory().create_scene_from_json(scene_json_object) - -####################### world -world_interface = WorldInterface() -with open(world_state_path, "r") as file: - world_state_json_object = json.load(file) - world_interface.load_world_from_json(world_state_json_object) - -from_problem = parse_problem_init(problem=problem) -objects = parse_problem_objects(problem=problem) -world_interface.update_world(from_problem) - -world_state_json = world_interface.get_world_to_json() -pprint(from_problem) -pprint(world_state_json) -pprint(objects) - - -####################### robot -robot_interface = RobotInterface( - robot_address="127.0.0.1", - robot_port=12000, -) -robot_interface.setup_scene(scene) - -####################### bt_factory -bt_factory = BehaviorTreeFactory( - world_interface=world_interface, - robot_interface=robot_interface, -) - -####################### behavior_tree_stewardship -behavior_tree_stewardship = BehaviorTreeStewardship( - behaviortree_factory=bt_factory, - world_interface=world_interface, - robot_interface=robot_interface, -) - -####################### * tools -# bt_executor_tool = BehaviorTreeExecutorTool(behavior_tree_stewardship) - -bt_sim_tool = BehaviorTreeSimulatorTool(metadata={"bt_stw": behavior_tree_stewardship}) - -# world_state_query_tool = WorldStateQueryTool(behavior_tree_stewardship) - -executor_tools = [bt_sim_tool] - -# * kios data prompt skeleton dir -data_dir = os.environ.get("KIOS_DATA_DIR").format(username=os.getlogin()) -print(data_dir) -prompt_sk_dir = os.path.join(data_dir, "prompt_skeletons") -prompt_dir = os.path.join(data_dir, "prompts") - - -# * the graph state -class PlanExecuteState(TypedDict): - plan: List[str] - behavior_tree: dict - world_state: Annotated[ - List[dict], operator.add - ] # ! to add, you need to make world_state a list of dict - past_steps: Annotated[List[Tuple], operator.add] - user_input: str - objects: dict[str, list[str]] - last_behavior_tree: dict - last_failed_node: dict - runtime_world_state: dict - behavior_tree_execution_summary: str - BTExecutionHasSucceeded: bool - - -########## verifier - -behaviortree_file = os.path.join(prompt_dir, "vv/behaviortree.txt") -template_file = os.path.join(prompt_dir, "vv/template.txt") -task_file = os.path.join(prompt_dir, "vv/task.txt") -system_file = os.path.join(prompt_dir, "vv/system.txt") -object_file = os.path.join(prompt_dir, "vv/object.txt") -domain_file = os.path.join(prompt_dir, "vv/domain.txt") -state_file = os.path.join(prompt_dir, "vv/state.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) -with open(object_file, "r") as f: - object_ppt = PromptTemplate.from_template(f.read()) -with open(behaviortree_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - behaviortree_ppt = ppt_tmp.partial(input=f.read()) - -with open(state_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - state_ppt = ppt_tmp.partial(input=f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {state} - - {object} - - {behaviortree} - - {template} - """ -) - -verifier_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ("object", object_ppt), - ("behaviortree", behaviortree_ppt), - ("state", state_ppt), - ], -) - -verifier_chain = ( - verifier_ppt_ppl - | ChatOpenAI(model="gpt-4-turbo-preview", temperature=0) - | StrOutputParser() -) - - -# * output schema of the recursive_sk_generator -class SolutionBehaviortree(BaseModel): - """the fixed behavior tree""" - - behavior_tree: dict = Field( - description="the behavior tree that adheres the required format." - ) - - -behaviortree_file = os.path.join(prompt_dir, "rec_sk_gen/behaviortree.txt") -template_file = os.path.join(prompt_dir, "rec_sk_gen/template.txt") -task_file = os.path.join(prompt_dir, "rec_sk_gen/task.txt") -system_file = os.path.join(prompt_dir, "rec_sk_gen/system.txt") -object_file = os.path.join(prompt_dir, "rec_sk_gen/object.txt") -domain_file = os.path.join(prompt_dir, "rec_sk_gen/domain.txt") -example_file = os.path.join(prompt_dir, "rec_sk_gen/example.txt") -state_file = os.path.join(prompt_dir, "rec_sk_gen/state.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) -with open(object_file, "r") as f: - object_ppt = PromptTemplate.from_template(f.read()) -with open(behaviortree_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - behaviortree_ppt = ppt_tmp.partial(input=f.read()) - -with open(state_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - state_ppt = ppt_tmp.partial(input=f.read()) - -with open(example_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - example_ppt = ppt_tmp.partial(input=f.read()) -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {state} - - {object} - - {behaviortree} - - {example} - - {template} - """ -) - -re_sk_gen_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ("object", object_ppt), - ("behaviortree", behaviortree_ppt), - ("example", example_ppt), - ("state", state_ppt), - ], -) - -rec_sk_gen_chain = ( - re_sk_gen_ppt_ppl - | ChatOpenAI(model="gpt-4-turbo-preview", temperature=0) - | JsonOutputParser() -) - -# recursive_sk_generator = create_structured_output_runnable( -# SolutionBehaviortree, -# ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), -# re_sk_gen_ppt_ppl, -# mode="openai-json", -# ) - - -##################################################### * planner -# * output schema of the planner -class Plan(BaseModel): - """Plan to follow in future""" - - steps: List[str] = Field( - description="a list of different steps to follow, should be in sorted order" - ) - - -template_file = os.path.join(prompt_dir, "planner/template.txt") -task_file = os.path.join(prompt_dir, "planner/task.txt") -system_file = os.path.join(prompt_dir, "planner/system.txt") -domain_file = os.path.join(prompt_dir, "planner/domain.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {template} - """ -) - -planner_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ], -) - -planner = create_structured_output_runnable( - Plan, ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), planner_ppt_ppl -) - -##################################################### * plan_updaterner - -template_file = os.path.join(prompt_dir, "plan_updater/template.txt") -task_file = os.path.join(prompt_dir, "plan_updater/task.txt") -system_file = os.path.join(prompt_dir, "plan_updater/system.txt") -domain_file = os.path.join(prompt_dir, "plan_updater/domain.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {template} - """ -) - -plan_updater_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ], -) - -plan_updater = create_structured_output_runnable( - Plan, ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), plan_updater_ppt_ppl -) -# ! create openai fn runnable has bug - - -##################################################### * graph node functions -@traceable(name="user_input_node_step") -async def user_input_step(state: PlanExecuteState): - """ - get the input from the user - """ - print(f"-----user_input_step-----") - - user_input = input("Your next instruction: ") - - return { - "user_input": user_input, - } - - -@traceable(name="behavior_tree_generate_step") -async def behavior_tree_generate_step(state: PlanExecuteState): - """ - generate the behavior tree based on the instruction - """ - print(f"-----behavior_tree_generate_step-----") - - instruction = state["plan"][0] - if state["runtime_world_state"] is None: - runtime_world_state = state["world_state"][-1] - else: - runtime_world_state = state["runtime_world_state"] - objects = state["objects"] - - bt_skeleton = await rec_sk_gen_chain.ainvoke( - { - "objects": objects, - "runtime_world_state": runtime_world_state, - "last_behavior_tree": state["last_behavior_tree"], - "last_failed_node": state["last_failed_node"], - "summary": state["behavior_tree_execution_summary"], - "instructions": instruction, - } - ) - - render_bt(bt_skeleton) - - pause = input("paused here") - - return { - "last_behavior_tree": bt_skeleton, - "BTExecutionHasSucceeded": False, - } - - -@traceable(name="behavior_tree_execute_step") -async def behavior_tree_execute_step(state: PlanExecuteState): - """ - execute the first step of the plan, append the result to the past steps - """ - print(f"-----behavior_tree_execute_step-----") - - this_step = state["plan"][0] - behavior_tree_skeleton = state["last_behavior_tree"] - latest_world_state = state["world_state"][-1] - - # * first sim run - tree_result, skeleton_json = behavior_tree_stewardship.sk_sim_run( - world_state=latest_world_state, skeleton_json=behavior_tree_skeleton - ) - - pprint(tree_result.to_json()) - pause = input("paused here") - - # * check result - if tree_result.result == "success": - - # * second fake run - tree_result, skeleton_json = behavior_tree_stewardship.sk_fake_run( - world_state=latest_world_state, skeleton_json=behavior_tree_skeleton - ) - - pprint(tree_result.to_json()) - pause = input("paused here") - - if tree_result.result == "success": - return { - "BTExecutionHasSucceeded": True, - "past_steps": ( - this_step, - tree_result.result, - ), # * only one because new plan will be generated and old steps are all removed - "world_state": [tree_result.world_state], - # * clear all - "last_behavior_tree": None, - "last_failed_node": None, - "runtime_world_state": None, - "behavior_tree_execution_summary": None, - } - else: - return { - "world_state": [tree_result.world_state], # * real world state - "last_behavior_tree": skeleton_json, - "last_failed_node": { - "summary": tree_result.final_node.get("summary"), - "name": tree_result.final_node.get("name"), - "identifier": tree_result.final_node.get("identifier"), - }, - "runtime_world_state": tree_result.world_state, - "behavior_tree_execution_summary": tree_result.summary, - } - - else: - return { - # ! do not change world state - "last_behavior_tree": skeleton_json, - "last_failed_node": { - "summary": tree_result.final_node.get("summary"), - "name": tree_result.final_node.get("name"), - "identifier": tree_result.final_node.get("identifier"), - }, - "runtime_world_state": tree_result.world_state, - "behavior_tree_execution_summary": tree_result.summary, - } - - -@traceable(name="verifier_step") -async def verifier_step(state: PlanExecuteState): - """ - verify the behavior tree - """ - print(f"-----verifier_step-----") - start_world_state = state["world_state"][-1] - - # * get the objects - objects = state["objects"] - - # * get the last behavior tree - last_behavior_tree = state["last_behavior_tree"] - - # * get the last failed node - last_failed_node = state["last_failed_node"] - - # * get the runtime world state - runtime_world_state = state["runtime_world_state"] - - # * get the behavior tree execution summary - behavior_tree_execution_summary = state["behavior_tree_execution_summary"] - - # * get the verifier response - verifier_response = await verifier_chain.ainvoke( - { - "objects": objects, - "start_world_state": start_world_state, - "last_behavior_tree": last_behavior_tree, - "last_failed_node": last_failed_node, - "runtime_world_state": runtime_world_state, - "summary": behavior_tree_execution_summary, - } - ) - - # * return the verifier response - return { - "behavior_tree_execution_summary": verifier_response, - } - - -@traceable(name="planner_step") -async def planner_step(state: PlanExecuteState): - """ - plan the steps based on user input and world state - """ - print(f"-----plan_step-----") - - plan = await planner.ainvoke( - { - "user_input": state["user_input"], - "world_state": state["world_state"], - } - ) - return {"plan": plan.steps} - - -@traceable(name="plan_updater_step") -async def plan_updater_step(state: PlanExecuteState): - """ - if return a response, then success, response the use, end. - otherwise, return the updated newplan (normally the same as the old plan with the first step popped out. - """ - print(f"-----plan_updater_step-----") - - # ! BOOOM! BUG! - output = await plan_updater.ainvoke( - { - "user_input": state["user_input"], - "plan": state["plan"], - "world_state": state["world_state"], - "past_steps": state["past_steps"], - } - ) - # if isinstance( - # output, UpdaterResponse - # ): # * determine if it is time to response and end - # return { - # "response": "Your last instruction has been finished." - # } # * Don't need to update. - # else: - return {"plan": output.steps} # * update the plan - - -##################################################### * construct the graph - -workflow = StateGraph(PlanExecuteState) - -workflow.add_node("planner", planner_step) - -workflow.add_node("behavior_tree_generator", behavior_tree_generate_step) - -workflow.add_node("behavior_tree_executor", behavior_tree_execute_step) - -workflow.add_node("verifier", verifier_step) - -workflow.add_node("plan_updater", plan_updater_step) - -workflow.add_node("user_input_node", user_input_step) - -workflow.set_entry_point("user_input_node") - -workflow.add_edge( - "planner", "behavior_tree_generator" -) # ! may need to add a condition here - -workflow.add_edge("behavior_tree_generator", "behavior_tree_executor") - -workflow.add_edge("verifier", "behavior_tree_generator") - - -def executor_should_end(state: PlanExecuteState): - """ - end router - """ - print(f"-----executor_should_end-----") - - if state["BTExecutionHasSucceeded"] == True: - state["BTExecutionHasSucceeded"] = False - return True - else: - return False - - -workflow.add_conditional_edges( - "behavior_tree_executor", - executor_should_end, - { - True: "plan_updater", - False: "verifier", - }, -) - - -def plan_updater_should_end(state: PlanExecuteState): - """ - end router - """ - print(f"-----plan_updater_should_end-----") - - # if state["response"]: - # return True - # else: - # return False - if state["plan"] == [] or len(state["plan"]) == 0: - return True - else: - return False - - -workflow.add_conditional_edges( - "plan_updater", - plan_updater_should_end, - { - True: "user_input_node", - False: "behavior_tree_generator", - }, -) - - -def user_input_should_end(state: PlanExecuteState): - """ - if the user input is empty, then end - """ - print(f"-----user_input_should_end-----") - - # print(f'user_input: {state["user_input"]}') - # print(f'world_state: {state["world_state"]}') - - if not state["user_input"] or state["user_input"] == "": - return True - else: - return False - - -workflow.add_conditional_edges( - "user_input_node", - user_input_should_end, - { - True: END, - False: "planner", - }, -) - -app = workflow.compile() - -config = {"recursion_limit": 500} - -inputs = { - "world_state": [world_state_json], - "objects": objects, -} - - -async def core_run(): - async for event in app.astream( - inputs, - config=config, - ): - for k, v in event.items(): - if k != "__end__": - print(v) - - -if __name__ == "__main__": - import asyncio - - asyncio.run(core_run()) diff --git a/experiments/lamp/scene.json b/experiments/lamp/scene.json deleted file mode 100644 index 688c50f4..00000000 --- a/experiments/lamp/scene.json +++ /dev/null @@ -1,119 +0,0 @@ -{ - "reference_objects": - [ - { - "object_name": "test_box1", - "source": "pre-defined", - "O_T_TCP": [ - [1, 0, 0, 0.0], - [0, 1, 0, 0.0], - [0, 0, 1, -0.1], - [0, 0, 0, 1] - ] - }, - { - "object_name": "TestObject1", - "source": "mios" - }, - { - "object_name": "inwardgripper", - "source": "mios" - }, - { - "object_name": "outwardgripper", - "source": "mios" - }, - { - "object_name": "clampgripper", - "source": "mios" - }, - { - "object_name": "parallelgripper", - "source": "mios" - }, - { - "object_name": "defaultgripper", - "source": "mios" - }, - { - "object_name": "lampbase", - "source": "mios" - }, - { - "object_name": "lampbulb", - "source": "mios" - }, - { - "object_name": "lampshade", - "source": "mios" - } - ], - "tools": - [ - { - "tool_name": "clampgripper", - "EE_T_TCP": [ - [1, 0, 0, 0.0], - [0, 1, 0, 0.0], - [0, 0, 1, 0.085], - [0, 0, 0, 1] - ], - "EE_finger_width_max": 0.042, - "EE_finger_width_min": 0.01, - "load_width": 0.042, - "unload_width": 0.08 - }, - { - "tool_name": "parallelgripper", - "EE_T_TCP": [ - [1, 0, 0, 0.0], - [0, 1, 0, 0.0], - [0, 0, 1, 0.07], - [0, 0, 0, 1] - ], - "EE_finger_width_max": 0.042, - "EE_finger_width_min": 0.01, - "load_width": 0.042, - "unload_width": 0.08 - }, - { - "tool_name": "inwardgripper", - "EE_T_TCP": [ - [1, 0, 0, 0.0], - [0, 1, 0, 0.0], - [0, 0, 1, 0.095], - [0, 0, 0, 1] - ], - "EE_finger_width_max": 0.042, - "EE_finger_width_min": 0.01, - "load_width": 0.042, - "unload_width": 0.08 - }, - { - "tool_name": "outwardgripper", - "EE_T_TCP": [ - [1, 0, 0, 0.0], - [0, 1, 0, 0.0], - [0, 0, 1, 0.125], - [0, 0, 0, 1] - ], - "EE_finger_width_max": 0.042, - "EE_finger_width_min": 0.01, - "load_width": 0.042, - "unload_width": 0.08 - }, - { - "tool_name": "defaultgripper", - "EE_T_TCP": [ - [1, 0, 0, 0.0], - [0, 1, 0, 0.0], - [0, 0, 1, 0.0], - [0, 0, 0, 1] - ], - "EE_finger_width_max": 0.08, - "EE_finger_width_min": 0.01, - "load_width": 0.08, - "unload_width": 0.08 - } - ] -} diff --git a/experiments/lamp/show.py b/experiments/lamp/show.py deleted file mode 100644 index ab90d92e..00000000 --- a/experiments/lamp/show.py +++ /dev/null @@ -1,1077 +0,0 @@ -result_1 = { - "behavior_tree": { - "summary": "selector to insert gear2 into shaft2", - "name": "selector: insert(left_hand, outward_claw, gear2, shaft2)", - "identifier": 1, - "type_name": "selector", - "children": [ - { - "summary": "condition node to check if gear2 is inserted to shaft2", - "name": "target: is_inserted_to(gear2, shaft2)", - "identifier": 2, - "type_name": "condition", - "conditions": [ - { - "object_name": "gear2", - "property_name": "is_inserted_to", - "property_value": "shaft2", - "status": True, - } - ], - }, - { - "summary": "sequence to insert gear2 into shaft2", - "name": "sequence: insert(left_hand, outward_claw, gear2, shaft2)", - "identifier": 3, - "type_name": "sequence", - "children": [ - { - "summary": "selector to load outward_claw", - "name": "selector: load_tool(left_hand, outward_claw)", - "identifier": 4, - "type_name": "selector", - "children": [ - { - "summary": "condition node to check if left_hand holds outward_claw", - "name": "target: hold(left_hand, outward_claw)", - "identifier": 5, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "outward_claw", - "status": True, - } - ], - }, - { - "summary": "sequence to load outward_claw", - "name": "sequence: load_tool(left_hand, outward_claw)", - "identifier": 6, - "type_name": "sequence", - "children": [ - { - "summary": "condition node to check if outward_claw is equippable", - "name": "precondition: is_equippable(outward_claw)", - "identifier": 7, - "type_name": "condition", - "conditions": [ - { - "object_name": "outward_claw", - "property_name": "is_equippable", - "property_value": None, - "status": True, - } - ], - }, - { - "summary": "condition node to check if left hand is free", - "name": "precondition: is_free(left_hand)", - "identifier": 8, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "is_free", - "property_value": None, - "status": True, - } - ], - }, - { - "summary": "action node to equip outward_claw to left hand", - "name": "action: load_tool(left_hand, outward_claw)", - "identifier": 9, - "type_name": "action", - "effects": [ - { - "object_name": "left_hand", - "property_name": "is_free", - "property_value": None, - "status": False, - }, - { - "object_name": "outward_claw", - "property_name": "is_equippable", - "property_value": None, - "status": False, - }, - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "outward_claw", - "status": True, - }, - ], - }, - ], - }, - ], - }, - { - "summary": "selector to use left_hand with outward_claw to pick up gear2", - "name": "selector: pick_up(left_hand, outward_claw, gear2)", - "identifier": 10, - "type_name": "selector", - "children": [ - { - "summary": "condition node to check if outward_claw holds gear2", - "name": "target: hold(outward_claw, gear2)", - "identifier": 11, - "type_name": "condition", - "conditions": [ - { - "object_name": "outward_claw", - "property_name": "hold", - "property_value": "gear2", - "status": True, - } - ], - }, - { - "summary": "sequence to use left hand with outward_claw to pick up gear2", - "name": "sequence: pick_up(left_hand, outward_claw, gear2)", - "identifier": 12, - "type_name": "sequence", - "children": [ - { - "summary": "condition node to check if outward_claw is free", - "name": "precondition: is_free(outward_claw)", - "identifier": 13, - "type_name": "condition", - "conditions": [ - { - "object_name": "outward_claw", - "property_name": "is_free", - "property_value": None, - "status": True, - } - ], - }, - { - "summary": "condition node to check if left hand holds outward_claw", - "name": "precondition: hold(left_hand, outward_claw)", - "identifier": 14, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "outward_claw", - "status": True, - } - ], - }, - { - "summary": "action node to use left hand with outward_claw to pick up gear2", - "name": "action: pick_up(left_hand, outward_claw, gear2)", - "identifier": 15, - "type_name": "action", - "effects": [ - { - "object_name": "outward_claw", - "property_name": "hold", - "property_value": "gear2", - "status": True, - }, - { - "object_name": "outward_claw", - "property_name": "is_free", - "property_value": None, - "status": False, - }, - ], - }, - ], - }, - ], - }, - { - "summary": "condition node to check if gear2 can be inserted to shaft2", - "name": "precondition: can_insert_to(gear2, shaft2)", - "identifier": 16, - "type_name": "condition", - "conditions": [ - { - "object_name": "gear2", - "property_name": "can_insert_to", - "property_value": "shaft2", - "status": True, - } - ], - }, - { - "summary": "action node to use left_hand with outward_claw to insert gear2 to shaft2", - "name": "action: insert(left_hand, outward_claw, gear2, shaft2)", - "identifier": 17, - "type_name": "action", - "effects": [ - { - "object_name": "outward_claw", - "property_name": "hold", - "property_value": "gear2", - "status": False, - }, - { - "object_name": "outward_claw", - "property_name": "is_free", - "property_value": None, - "status": True, - }, - { - "object_name": "gear2", - "property_name": "is_inserted_to", - "property_value": "shaft2", - "status": True, - }, - ], - }, - ], - }, - ], - }, - "world_state": [ - { - "objects": [ - {"name": "parallel_box1", "properties": ["is_free"]}, - {"name": "shaft1", "properties": []}, - {"name": "gearbase_hole1", "properties": []}, - {"name": "left_hand", "properties": []}, - ], - "constraints": [ - { - "source": "parallel_box1", - "name": "can_manipulate", - "target": "shaft1", - }, - { - "source": "shaft1", - "name": "can_insert_to", - "target": "gearbase_hole1", - }, - ], - "relations": [ - {"source": "left_hand", "name": "hold", "target": "parallel_box1"}, - { - "source": "shaft1", - "name": "is_inserted_to", - "target": "gearbase_hole1", - }, - ], - } - ], -} -# -------------------------------------------------------- - -result = { - "behavior_tree": { - "summary": "selector to insert shaft1 into gearbase_hole1", - "name": "selector: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)", - "identifier": 1, - "type_name": "selector", - "children": [ - { - "summary": "condition node to check if shaft1 is inserted to gearbase_hole1", - "name": "target: is_inserted_to(shaft1, gearbase_hole1)", - "identifier": 2, - "type_name": "condition", - "conditions": [ - { - "object_name": "shaft1", - "property_name": "is_inserted_to", - "property_value": "gearbase_hole1", - "status": True, - } - ], - }, - { - "summary": "sequence to insert shaft1 into gearbase_hole1", - "name": "sequence: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)", - "identifier": 3, - "type_name": "sequence", - "children": [ - { - "summary": "selector to unload outward_claw", - "name": "selector: unload_tool(left_hand, outward_claw)", - "identifier": 4, - "type_name": "selector", - "children": [ - { - "summary": "condition node to check if left_hand is free", - "name": "target: is_free(left_hand)", - "identifier": 5, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "is_free", - "property_value": None, - "status": True, - } - ], - }, - { - "summary": "sequence to unload outward_claw", - "name": "sequence: unload_tool(left_hand, outward_claw)", - "identifier": 6, - "type_name": "sequence", - "children": [ - { - "summary": "condition node to check if left hand holds outward_claw", - "name": "precondition: hold(left_hand, outward_claw)", - "identifier": 7, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "outward_claw", - "status": True, - } - ], - }, - { - "summary": "action node to unload outward_claw from left hand", - "name": "action: unload_tool(left_hand, outward_claw)", - "identifier": 8, - "type_name": "action", - "effects": [ - { - "object_name": "left_hand", - "property_name": "is_free", - "property_value": None, - "status": True, - }, - { - "object_name": "outward_claw", - "property_name": "is_equippable", - "property_value": None, - "status": True, - }, - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "outward_claw", - "status": False, - }, - ], - }, - ], - }, - ], - }, - { - "summary": "selector to load parallel_box1", - "name": "selector: load_tool(left_hand, parallel_box1)", - "identifier": 9, - "type_name": "selector", - "children": [ - { - "summary": "condition node to check if left_hand holds parallel_box1", - "name": "target: hold(left_hand, parallel_box1)", - "identifier": 10, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "parallel_box1", - "status": True, - } - ], - }, - { - "summary": "sequence to load parallel_box1", - "name": "sequence: load_tool(left_hand, parallel_box1)", - "identifier": 11, - "type_name": "sequence", - "children": [ - { - "summary": "condition node to check if parallel_box1 is equippable", - "name": "precondition: is_equippable(parallel_box1)", - "identifier": 12, - "type_name": "condition", - "conditions": [ - { - "object_name": "parallel_box1", - "property_name": "is_equippable", - "property_value": None, - "status": True, - } - ], - }, - { - "summary": "condition node to check if left hand is free", - "name": "precondition: is_free(left_hand)", - "identifier": 13, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "is_free", - "property_value": None, - "status": True, - } - ], - }, - { - "summary": "action node to equip parallel_box1 to left hand", - "name": "action: load_tool(left_hand, parallel_box1)", - "identifier": 14, - "type_name": "action", - "effects": [ - { - "object_name": "left_hand", - "property_name": "is_free", - "property_value": None, - "status": False, - }, - { - "object_name": "parallel_box1", - "property_name": "is_equippable", - "property_value": None, - "status": False, - }, - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "parallel_box1", - "status": True, - }, - ], - }, - ], - }, - ], - }, - { - "summary": "selector to pick up shaft1 with parallel_box1", - "name": "selector: pick_up(left_hand, parallel_box1, shaft1)", - "identifier": 15, - "type_name": "selector", - "children": [ - { - "summary": "condition node to check if parallel_box1 holds shaft1", - "name": "target: hold(parallel_box1, shaft1)", - "identifier": 16, - "type_name": "condition", - "conditions": [ - { - "object_name": "parallel_box1", - "property_name": "hold", - "property_value": "shaft1", - "status": True, - } - ], - }, - { - "summary": "sequence to pick up shaft1 with parallel_box1", - "name": "sequence: pick_up(left_hand, parallel_box1, shaft1)", - "identifier": 17, - "type_name": "sequence", - "children": [ - { - "summary": "condition node to check if parallel_box1 is free", - "name": "precondition: is_free(parallel_box1)", - "identifier": 18, - "type_name": "condition", - "conditions": [ - { - "object_name": "parallel_box1", - "property_name": "is_free", - "property_value": None, - "status": True, - } - ], - }, - { - "summary": "condition node to check if left hand holds parallel_box1", - "name": "precondition: hold(left_hand, parallel_box1)", - "identifier": 19, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "parallel_box1", - "status": True, - } - ], - }, - { - "summary": "action node to pick up shaft1 with parallel_box1", - "name": "action: pick_up(left_hand, parallel_box1, shaft1)", - "identifier": 20, - "type_name": "action", - "effects": [ - { - "object_name": "parallel_box1", - "property_name": "hold", - "property_value": "shaft1", - "status": True, - }, - { - "object_name": "parallel_box1", - "property_name": "is_free", - "property_value": None, - "status": False, - }, - ], - }, - ], - }, - ], - }, - { - "summary": "condition node to check if shaft1 can be inserted to gearbase_hole1", - "name": "precondition: can_insert_to(shaft1, gearbase_hole1)", - "identifier": 21, - "type_name": "condition", - "conditions": [ - { - "object_name": "shaft1", - "property_name": "can_insert_to", - "property_value": "gearbase_hole1", - "status": True, - } - ], - }, - { - "summary": "action node to insert shaft1 into gearbase_hole1", - "name": "action: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)", - "identifier": 22, - "type_name": "action", - "effects": [ - { - "object_name": "parallel_box1", - "property_name": "hold", - "property_value": "shaft1", - "status": False, - }, - { - "object_name": "parallel_box1", - "property_name": "is_free", - "property_value": None, - "status": True, - }, - { - "object_name": "shaft1", - "property_name": "is_inserted_to", - "property_value": "gearbase_hole1", - "status": True, - }, - ], - }, - ], - }, - ], - }, - "world_state": [ - { - "objects": [ - {"name": "parallel_box1", "properties": ["is_free", "is_equippable"]}, - {"name": "parallel_box2", "properties": ["is_free", "is_equippable"]}, - {"name": "inward_claw", "properties": ["is_free", "is_equippable"]}, - {"name": "outward_claw", "properties": ["is_free"]}, - {"name": "no_tool", "properties": ["is_free", "is_equippable"]}, - {"name": "gear1", "properties": []}, - {"name": "gear2", "properties": []}, - {"name": "gear3", "properties": []}, - {"name": "shaft1", "properties": []}, - {"name": "shaft2", "properties": []}, - {"name": "shaft3", "properties": []}, - {"name": "gearbase", "properties": []}, - {"name": "gearbase_hole1", "properties": []}, - {"name": "gearbase_hole3", "properties": []}, - {"name": "left_hand", "properties": []}, - ], - "constraints": [ - { - "source": "parallel_box1", - "name": "can_manipulate", - "target": "shaft1", - }, - { - "source": "parallel_box2", - "name": "can_manipulate", - "target": "gear1", - }, - {"source": "outward_claw", "name": "can_manipulate", "target": "gear2"}, - {"source": "outward_claw", "name": "can_manipulate", "target": "gear3"}, - {"source": "no_tool", "name": "can_manipulate", "target": "shaft3"}, - { - "source": "shaft1", - "name": "can_insert_to", - "target": "gearbase_hole1", - }, - { - "source": "shaft3", - "name": "can_insert_to", - "target": "gearbase_hole3", - }, - {"source": "gear3", "name": "can_insert_to", "target": "shaft3"}, - ], - "relations": [ - {"source": "left_hand", "name": "hold", "target": "outward_claw"} - ], - } - ], -} - -import json -from kios_bt.bt_factory import BehaviorTreeFactory -from kios_utils.pybt_test import generate_bt_stewardship, render_dot_tree - -example1 = { - "summary": "selector to load left_hand with parallel_box1", - "name": "selector: load_tool(left_hand, parallel_box1)", - "children": [ - { - "summary": "condition node to check if left_hand holds parallel_box1", - "name": "target: hold(left_hand, parallel_box1)", - }, - { - "summary": "sequence to load left_hand with parallel_box1", - "name": "sequence: load_tool(left_hand, parallel_box1)", - "children": [ - { - "summary": "selector to make left_hand unload outward_claw", - "name": "selector: unload_tool(left_hand, outward_claw)", - "children": [ - { - "summary": "condition node to check if left_hand is free", - "name": "target: is_free(left_hand)", - }, - { - "summary": "sequence to make left_hand unload outward_claw", - "name": "sequence: unload_tool(left_hand, outward_claw)", - "children": [ - { - "summary": "condition node to check if outward_claw is free", - "name": "precondition: is_free(outward_claw)", - }, - { - "summary": "condition node to check if left hand holds outward_claw", - "name": "precondition: hold(left_hand, outward_claw)", - }, - { - "summary": "action node to make left hand unload outward_claw", - "name": "action: unload_tool(left_hand, outward_claw)", - "effects": [ - { - "summary": "left_hand will be free", - }, - { - "summary": "outward_claw will be equippable", - }, - { - "summary": "left_hand will not hold outward_claw", - }, - ], - }, - ], - }, - ], - }, - { - "summary": "condition node to check if parallel_box1 is equippable", - "name": "precondition: is_equippable(parallel_box1)", - }, - { - "summary": "action node to equip left hand with parallel_box1", - "name": "action: load_tool(left_hand, parallel_box1)", - "effects": [ - { - "summary": "left_hand will be not free", - }, - { - "summary": "parallel_box1 will be not equippable", - }, - { - "summary": "left_hand will hold parallel_box1", - }, - ], - }, - ], - }, - ], -} - -result3 = { - "summary": "selector to insert shaft1 into gearbase_hole1", - "name": "selector: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)", - "children": [ - { - "summary": "condition node to check if shaft1 is inserted into gearbase_hole1", - "name": "target: is_inserted_to(shaft1, gearbase_hole1)", - }, - { - "summary": "sequence to insert shaft1 into gearbase_hole1", - "name": "sequence: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)", - "children": [ - { - "summary": "selector to unload outward_claw from left_hand", - "name": "selector: unload_tool(left_hand, outward_claw)", - "children": [ - { - "summary": "condition node to check if left_hand is free", - "name": "target: is_free(left_hand)", - }, - { - "summary": "sequence to unload outward_claw from left_hand", - "name": "sequence: unload_tool(left_hand, outward_claw)", - "children": [ - { - "summary": "condition node to check if left hand holds outward_claw", - "name": "precondition: hold(left_hand, outward_claw)", - }, - { - "summary": "action node to unload outward_claw from left hand", - "name": "action: unload_tool(left_hand, outward_claw)", - "effects": [ - { - "summary": "left_hand will be free", - "name": "effect: is_free(left_hand)", - }, - { - "summary": "outward_claw will be equippable", - "name": "effect: is_equippable(outward_claw)", - }, - { - "summary": "left_hand will not hold outward_claw", - "name": "effect: not hold(left_hand, outward_claw)", - }, - ], - }, - ], - }, - ], - }, - { - "summary": "selector to load parallel_box1 on left_hand", - "name": "selector: load_tool(left_hand, parallel_box1)", - "children": [ - { - "summary": "condition node to check if left_hand holds parallel_box1", - "name": "target: hold(left_hand, parallel_box1)", - }, - { - "summary": "sequence to load parallel_box1 on left_hand", - "name": "sequence: load_tool(left_hand, parallel_box1)", - "children": [ - { - "summary": "condition node to check if parallel_box1 is equippable", - "name": "precondition: is_equippable(parallel_box1)", - }, - { - "summary": "condition node to check if left hand is free", - "name": "precondition: is_free(left_hand)", - }, - { - "summary": "action node to load parallel_box1 on left hand", - "name": "action: load_tool(left_hand, parallel_box1)", - "effects": [ - { - "summary": "left_hand will not be free", - "name": "effect: not is_free(left_hand)", - }, - { - "summary": "parallel_box1 will not be equippable", - "name": "effect: not is_equippable(parallel_box1)", - }, - { - "summary": "left_hand will hold parallel_box1", - "name": "effect: hold(left_hand, parallel_box1)", - }, - ], - }, - ], - }, - ], - }, - { - "summary": "condition node to check if parallel_box1 can manipulate shaft1", - "name": "precondition: can_manipulate(parallel_box1, shaft1)", - }, - { - "summary": "condition node to check if shaft1 can be inserted to gearbase_hole1", - "name": "precondition: can_insert_to(shaft1, gearbase_hole1)", - }, - { - "summary": "action node to insert shaft1 into gearbase_hole1", - "name": "action: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)", - "effects": [ - { - "summary": "parallel_box1 will not hold shaft1", - "name": "effect: not hold(parallel_box1, shaft1)", - }, - { - "summary": "parallel_box1 will be free", - "name": "effect: is_free(parallel_box1)", - }, - { - "summary": "shaft1 will be inserted into gearbase_hole1", - "name": "effect: is_inserted_to(shaft1, gearbase_hole1)", - }, - ], - }, - ], - }, - ], -} - -example2 = { - "summary": "selector to insert gear1 into shaft1", - "name": "selector: insert(left_hand, parallel_box1, gear1, shaft1)", - "children": [ - { - "summary": "condition node to check if gear1 is inserted to shaft1", - "name": "target: is_inserted_to(gear1, shaft1)", - }, - { - "summary": "sequence to insert gear1 into shaft1", - "name": "sequence: insert(left_hand, parallel_box1, gear1, shaft1)", - "children": [ - { - "summary": "selector to load parallel_box1", - "name": "selector: load_tool(left_hand, parallel_box1)", - "children": [ - { - "summary": "condition node to check if left_hand holds parallel_box1", - "name": "target: hold(left_hand, parallel_box1)", - }, - { - "summary": "sequence to load parallel_box1", - "name": "sequence: load_tool(left_hand, parallel_box1)", - "children": [ - { - "summary": "condition node to check if parallel_box1 is equippable", - "name": "precondition: is_equippable(parallel_box1)", - }, - { - "summary": "selector to make left_hand unload outward_claw", - "name": "selector: unload_tool(left_hand, outward_claw)", - "children": [ - { - "summary": "condition node to check if left_hand is free", - "name": "target: is_free(left_hand)", - }, - { - "summary": "sequence to make left_hand unload outward_claw", - "name": "sequence: unload_tool(left_hand, outward_claw)", - "children": [ - { - "summary": "condition node to check if outward_claw is free", - "name": "precondition: is_free(outward_claw)", - }, - { - "summary": "condition node to check if left hand holds outward_claw", - "name": "precondition: hold(left_hand, outward_claw)", - }, - { - "summary": "action node to make left hand unload outward_claw", - "name": "action: unload_tool(left_hand, outward_claw)", - "effects": [ - { - "summary": "left_hand will be free", - }, - { - "summary": "outward_claw will be equippable", - }, - { - "summary": "left_hand will not hold outward_claw", - }, - ], - }, - ], - }, - ], - }, - { - "summary": "action node to equip parallel_box1 to left hand", - "name": "action: load_tool(left_hand, parallel_box1)", - "effects": [ - { - "summary": "left_hand will be not free", - }, - { - "summary": "parallel_box1 will be not equippable", - }, - { - "summary": "left_hand will hold parallel_box1", - }, - ], - }, - ], - }, - ], - }, - { - "summary": "selector to use left_hand with parallel_box1 to pick up gear1", - "name": "selector: pick_up(left_hand, parallel_box1, gear1)", - "children": [ - { - "summary": "condition node to check if parallel_box1 holds gear1", - "name": "target: hold(parallel_box1, gear1)", - }, - { - "summary": "sequence to use left hand with parallel_box1 to pick up gear1", - "name": "sequence: pick_up(left_hand, parallel_box1, gear1)", - "children": [ - { - "summary": "condition node to check if parallel_box1 is free", - "name": "precondition: is_free(parallel_box1)", - }, - { - "summary": "condition node to check if left hand holds parallel_box1", - "name": "precondition: hold(left_hand, parallel_box1)", - }, - { - "summary": "action node to use left hand with parallel_box1 to pick up gear1", - "name": "action: pick_up(left_hand, parallel_box1, gear1)", - "effects": [ - { - "summary": "parallel_box1 will hold gear1", - }, - { - "summary": "parallel_box1 will be not free", - }, - ], - }, - ], - }, - ], - }, - { - "summary": "condition node to check if gear1 can be inserted to shaft1", - "name": "precondition: can_insert_to(gear1, shaft1)", - }, - { - "summary": "action node to use left_hand with parallel_box1 to insert gear1 to shaft1", - "name": "action: insert(left_hand, parallel_box1, gear1, shaft1)", - "effects": [ - { - "summary": "parallel_box1 will be not holding gear1", - }, - { - "summary": "parallel_box1 will be free", - }, - { - "summary": "gear1 will be inserted to shaft1", - }, - ], - }, - ], - }, - ], -} - -result = { - "summary": "Selector to insert shaft1 into gearbase_hole1 using parallel_box1 in the left hand", - "name": "selector: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)", - "children": [ - { - "summary": "Check the target that shaft1 is inserted into gearbase_hole1", - "name": "target: is_inserted_to(shaft1, gearbase_hole1)", - }, - { - "summary": "Sequence to insert shaft1 into gearbase_hole1 using parallel_box1 in the left hand", - "name": "sequence: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)", - "children": [ - { - "summary": "Selector to load the parallel_box1 in the left hand", - "name": "selector: load_tool(left_hand, parallel_box1)", - "children": [ - { - "summary": "Check the target that the left hand is free", - "name": "target: is_free(left_hand)", - }, - { - "summary": "Sequence to load the tool in the left hand", - "name": "sequence: load_tool(left_hand, parallel_box1)", - "children": [ - { - "summary": "Check the precondition that the parallel_box1 is equippable", - "name": "precondition: is_equippable(parallel_box1)", - }, - { - "summary": "Selector to unload the outward_claw in the left hand", - "name": "selector: unload_tool(left_hand, outward_claw)", - "children": [ - { - "summary": "Check the target that the left hand is free", - "name": "target: is_free(left_hand)", - }, - { - "summary": "Sequence to unload the tool in the left hand", - "name": "sequence: unload_tool(left_hand, outward_claw)", - "children": [ - { - "summary": "Check the precondition that the left hand is holding a outward_claw", - "name": "precondition: hold(left_hand, outward_claw)", - }, - { - "summary": "Unload the outward_claw in the left hand", - "name": "action: unload_tool(left_hand, outward_claw)", - }, - ], - }, - ], - }, - { - "summary": "Load the parallel_box1 in the left hand", - "name": "action: load_tool(left_hand, parallel_box1)", - }, - ], - }, - ], - }, - { - "summary": "Selector to pick up the shaft1 with the parallel_box1 in the left hand", - "name": "selector: pick_up(left_hand, parallel_box1, shaft1)", - "children": [ - { - "summary": "Check the target that the parallel_box1 is holding the shaft1", - "name": "target: hold(parallel_box1, shaft1)", - }, - { - "summary": "Sequence to pick up the shaft1 with the parallel_box1 in the left hand", - "name": "sequence: pick_up(left_hand, parallel_box1, shaft1)", - "children": [ - { - "summary": "Check the precondition that the left hand is holding the parallel_box1", - "name": "precondition: hold(left_hand, parallel_box1)", - }, - { - "summary": "Pick up the shaft1 with the parallel_box1 in the left hand", - "name": "action: pick_up(left_hand, parallel_box1, shaft1)", - }, - ], - }, - ], - }, - { - "summary": "Insert shaft1 into gearbase_hole1 using parallel_box1", - "name": "action: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)", - }, - ], - }, - ], -} - - -def test_bt(bt_json: json): - test_class = BehaviorTreeFactory() - bt = test_class.from_json_to_simple_bt(bt_json) - # bt = test_class.from_json_to_tree_root(bt_json) - bt_stewardship = generate_bt_stewardship(bt) - # bt_stewardship.setup(timeout=15) - render_dot_tree(bt_stewardship) - - -# test_bt(result_1["behavior_tree"]) -# test_bt(example1) -# test_bt(result3) -# test_bt(example2) -test_bt(result) diff --git a/experiments/lamp/sk_gen_test.py b/experiments/lamp/sk_gen_test.py deleted file mode 100644 index ba4dfd8e..00000000 --- a/experiments/lamp/sk_gen_test.py +++ /dev/null @@ -1,662 +0,0 @@ -import json -import os -from pprint import pprint - -os.environ["LANGCHAIN_TRACING_V2"] = "true" -os.environ["LANGCHAIN_ENDPOINT"] = "https://api.smith.langchain.com" -os.environ["LANGCHAIN_PROJECT"] = "kios_agent" - -from kios_bt.bt_stewardship import BehaviorTreeStewardship -from kios_scene.scene_factory import SceneFactory -from kios_bt.bt_factory import BehaviorTreeFactory -from kios_robot.robot_interface import RobotInterface -from kios_world.world_interface import WorldInterface - -from kios_agent.llm_supporter import KiosLLMSupporter -from kios_agent.data_types import KiosPromptSkeleton - -from kios_agent.kios_tools import ( - BehaviorTreeExecutorTool, - BehaviorTreeSimulatorTool, - WorldStateQueryTool, -) - -from typing import List, Tuple, Annotated, TypedDict -import operator - -from dotenv import load_dotenv - -from langchain import hub - -from langchain_openai import ChatOpenAI - -from langchain.chains.openai_functions import ( - create_structured_output_runnable, - create_openai_fn_runnable, -) -from langchain_core.utils.function_calling import convert_to_openai_function -from langchain_core.agents import AgentActionMessageLog, AgentFinish -from langchain_core.messages import HumanMessage -from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder -from langchain_core.output_parsers import JsonOutputParser, StrOutputParser -from langchain_core.pydantic_v1 import BaseModel, Field - -from langchain.agents.format_scratchpad import format_to_openai_function_messages -from langchain.agents import create_openai_functions_agent, AgentExecutor - -from langchain.prompts.pipeline import PipelinePromptTemplate -from langchain.prompts.prompt import PromptTemplate - -from langchain_community.document_loaders import TextLoader -from langchain_openai import OpenAIEmbeddings - -from langchain_community.vectorstores import FAISS -from langchain.text_splitter import ( - RecursiveCharacterTextSplitter, - CharacterTextSplitter, -) - -from langgraph.graph import StateGraph, END - -from langsmith import traceable - -load_dotenv() - -from kios_utils.pybt_test import generate_bt_stewardship, render_dot_tree -from kios_utils.pddl_problem_parser import parse_problem_init, parse_problem_objects - - -def render_bt(bt_json: json): - test_class = BehaviorTreeFactory() - bt = test_class.from_json_to_simple_bt(bt_json) - # bt = test_class.from_json_to_tree_root(bt_json) - bt_stewardship = generate_bt_stewardship(bt) - # bt_stewardship.setup(timeout=15) - render_dot_tree(bt_stewardship) - - -####################### dirs -current_dir = os.path.dirname(os.path.abspath(__file__)) -scene_path = os.path.join(current_dir, "scene.json") -# bt_json_file_path = os.path.join(current_dir, "behavior_tree.json") -world_state_path = os.path.join(current_dir, "world_state.json") -problem_path = os.path.join(current_dir, "gearset.problem") -domain_knowledge_path = os.path.join(current_dir, "domain_knowledge.txt") - -####################### problem -with open(problem_path, "r") as file: - problem = file.read() - -####################### scene -with open(scene_path, "r") as file: - scene_json_object = json.load(file) - -scene = SceneFactory().create_scene_from_json(scene_json_object) - -####################### world -world_interface = WorldInterface() -with open(world_state_path, "r") as file: - world_state_json_object = json.load(file) - world_interface.load_world_from_json(world_state_json_object) - -from_problem = parse_problem_init(problem=problem) -objects = parse_problem_objects(problem=problem) -world_interface.update_world(from_problem) - -world_state_json = world_interface.get_world_to_json() -pprint(from_problem) -pprint(world_state_json) -pprint(objects) - - -####################### robot -robot_interface = RobotInterface( - robot_address="127.0.0.1", - robot_port=12000, -) -robot_interface.setup_scene(scene) - -####################### bt_factory -bt_factory = BehaviorTreeFactory( - world_interface=world_interface, - robot_interface=robot_interface, -) - -####################### behavior_tree_stewardship -behavior_tree_stewardship = BehaviorTreeStewardship( - behaviortree_factory=bt_factory, - world_interface=world_interface, - robot_interface=robot_interface, -) - -####################### * tools -# bt_executor_tool = BehaviorTreeExecutorTool(behavior_tree_stewardship) - -bt_sim_tool = BehaviorTreeSimulatorTool(metadata={"bt_stw": behavior_tree_stewardship}) - -# world_state_query_tool = WorldStateQueryTool(behavior_tree_stewardship) - -executor_tools = [bt_sim_tool] - -# * kios data prompt skeleton dir -data_dir = os.environ.get("KIOS_DATA_DIR").format(username=os.getlogin()) -print(data_dir) -prompt_sk_dir = os.path.join(data_dir, "prompt_skeletons") -prompt_dir = os.path.join(data_dir, "prompts") - -# * retriever chain -# from langchain_experimental.text_splitter import SemanticChunker - -# # with open(domain_knowledge_path, "r") as f: -# # domain_knowledge = f.read() - -# # text_splitter = SemanticChunker(OpenAIEmbeddings()) - -# # docs = text_splitter.create_documents([domain_knowledge]) - - -# dk_loader = TextLoader(domain_knowledge_path) -# domain_knowledge_doc = dk_loader.load() - -# embeddings = OpenAIEmbeddings() - -# text_splitter = RecursiveCharacterTextSplitter( -# chunk_size=300, -# chunk_overlap=10, -# length_function=len, -# is_separator_regex=False, -# ) -# # text_splitter = CharacterTextSplitter() -# docs = text_splitter.split_documents(domain_knowledge_doc) -# vector_store = FAISS.from_documents(docs, embeddings) - -# retriever = vector_store.as_retriever() - -# docs = retriever.get_relevant_documents( -# "how to fulfill the condition 'is_free'?", -# k=3, -# threshold=0.5, -# ) - -# pprint(docs) - -# * refiner chain -llm_spt_re_sk = KiosLLMSupporter() -with open(os.path.join(prompt_sk_dir, "skeleton_refiner.json"), "r") as f: - re_sk_pptsk_json = json.load(f) - re_sk_pptsk = KiosPromptSkeleton.from_json(re_sk_pptsk_json) - llm_spt_re_sk.initialize_from_prompt_skeleton(re_sk_pptsk) -re_sk_ppt = llm_spt_re_sk.create_prompt() -re_sk_llm = ChatOpenAI(model_name=llm_spt_re_sk.model_name, temperature=0) - -re_sk_chain = re_sk_ppt | re_sk_llm | JsonOutputParser() - - -# * skeleton generator chain -llm_spt_skeleton_generator = KiosLLMSupporter() -with open(os.path.join(prompt_sk_dir, "skeleton_generator.json"), "r") as f: - skeleton_generator_pptsk_json = json.load(f) - skeleton_generator_pptsk = KiosPromptSkeleton.from_json( - skeleton_generator_pptsk_json - ) - llm_spt_skeleton_generator.initialize_from_prompt_skeleton(skeleton_generator_pptsk) -skeleton_generator_ppt = llm_spt_skeleton_generator.create_prompt() -skeleton_generator_llm = ChatOpenAI( - model_name=llm_spt_skeleton_generator.model_name, temperature=0 -) - -skeleton_generator_chain = ( - skeleton_generator_ppt - | skeleton_generator_llm - # | StrOutputParser() - | JsonOutputParser() -) - - -# * the graph state -class PlanExecuteState(TypedDict): - plan: List[str] - behavior_tree: dict - world_state: Annotated[ - List[dict], operator.add - ] # ! to add, you need to make world_state a list of dict - # = Field(default=None) - past_steps: Annotated[List[Tuple], operator.add] - response: str # ! not sure if use this or not. - user_input: str - problem: str - objects: dict[str, list[str]] - - -# * output schema of the planner -class SolutionSubtree(BaseModel): - """solution subtree that will be used to replace the failed condition node in the last behavior tree""" - - subtree: dict = Field( - description="a subtree that adheres the required format, which will be used to replace the failed condition node in the last behavior tree" - ) - - -behaviortree_file = os.path.join(prompt_dir, "rec_sk_gen/behaviortree.txt") -template_file = os.path.join(prompt_dir, "rec_sk_gen/template.txt") -task_file = os.path.join(prompt_dir, "rec_sk_gen/task.txt") -system_file = os.path.join(prompt_dir, "rec_sk_gen/system.txt") -object_file = os.path.join(prompt_dir, "rec_sk_gen/object.txt") -domain_file = os.path.join(prompt_dir, "rec_sk_gen/domain.txt") -example_file = os.path.join(prompt_dir, "rec_sk_gen/example.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {behaviortree} - - {domain} - - {example} - - {template} - """ -) - -planner_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ], -) - -planner = create_structured_output_runnable( - Plan, ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), planner_ppt_ppl -) - - -##################################################### * planner -# * output schema of the planner -class Plan(BaseModel): - """Plan to follow in future""" - - steps: List[str] = Field( - description="a list of different steps to follow, should be in sorted order" - ) - - -template_file = os.path.join(prompt_dir, "planner/template.txt") -task_file = os.path.join(prompt_dir, "planner/task.txt") -system_file = os.path.join(prompt_dir, "planner/system.txt") -domain_file = os.path.join(prompt_dir, "planner/domain.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {template} - """ -) - -planner_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ], -) - -planner = create_structured_output_runnable( - Plan, ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), planner_ppt_ppl -) - -##################################################### * plan_updaterner - -template_file = os.path.join(prompt_dir, "plan_updater/template.txt") -task_file = os.path.join(prompt_dir, "plan_updater/task.txt") -system_file = os.path.join(prompt_dir, "plan_updater/system.txt") -domain_file = os.path.join(prompt_dir, "plan_updater/domain.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {template} - """ -) - -plan_updater_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ], -) - -plan_updater = create_structured_output_runnable( - Plan, ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), plan_updater_ppt_ppl -) -# ! create openai fn runnable has bug - - -##################################################### * graph node functions -@traceable(name="user_input_node_step") -async def user_input_step(state: PlanExecuteState): - """ - get the input from the user - """ - print(f"-----user_input_step-----") - # if state["response"] is not None: - # user_input = input(state["response"]) - # state["response"] = None - # else: - user_input = input("Your next instruction: ") - - return { - "user_input": user_input, - } - - -@traceable(name="behavior_tree_generate_step") -async def behavior_tree_generate_step(state: PlanExecuteState): - """ - generate the behavior tree based on the instruction - """ - print(f"-----behavior_tree_generate_step-----") - - instruction = state["plan"][0] - latest_world_state = state["world_state"][-1] - objects = state["objects"] - - skeleton = await skeleton_generator_chain.ainvoke( - { - # "problem": state["problem"], - "objects": objects, - "world_state": latest_world_state, - "instructions": instruction, - } - ) - - # sk_json = JsonOutputParser().invoke(skeleton) - sk_json = skeleton - - behavior_tree_sk = sk_json["task_plan"]["behavior_tree"] - - render_bt(behavior_tree_sk) - - pause = input("paused here") - - # ! OBJECT CONCENTRATION - - # response = await re_sk_chain.ainvoke( - # { - # "input": skeleton, - # } - # ) - - behavior_tree = skeleton.get("task_plan").get("behavior_tree") - - # init_world_state = [skeleton.get("initial_state")] - - return { - "behavior_tree": behavior_tree, - # "world_state": init_world_state, # ! this is necessary because the constraints in problem - } - - -@traceable(name="behavior_tree_sim_step") -async def behavior_tree_sim_step(state: PlanExecuteState): - """ - simulate the behavior tree based on the world state - """ - print(f"-----behavior_tree_sim_step-----") - - behavior_tree = state["behavior_tree"] - latest_world_state = state["world_state"][-1] - - # exe_input = { - # "behavior_tree": behavior_tree, - # "world_state": latest_world_state, - # } - - # agent_response = await bt_sim_tool.ainvoke( - # { - # "input": exe_input, - # } - # ) - - response = behavior_tree_stewardship.sk_fake_run( - world_state=latest_world_state, bt_json=behavior_tree - ) - - # hasSucceeded = response["hasSucceeded"] - # new_world_state = [response["world_state"]] - hasSucceeded = response.result - new_world_state = [response.world_state] - the_step = state["plan"][0] - - return { - "past_steps": ( - the_step, - hasSucceeded, - ), # * only one because new plan will be generated and old steps are all removed - "world_state": new_world_state, - } - - -@traceable(name="behavior_tree_execute_step") -async def behavior_tree_execute_step(state: PlanExecuteState): - """ - execute the first step of the plan, append the result to the past steps - """ - print(f"-----behavior_tree_execute_step-----") - - behavior_tree = state["behavior_tree"] - latest_world_state = state["world_state"][-1] - - # exe_input = { - # "behavior_tree": behavior_tree, - # "world_state": latest_world_state, - # } - - # agent_response = await bt_exe_agent_executor.ainvoke( - # { - # "input": exe_input, - # } - # ) - - response = behavior_tree_stewardship.sk_fake_run( - world_state=latest_world_state, bt_json=behavior_tree - ) - - # hasSucceeded = response["hasSucceeded"] - # new_world_state = [response["world_state"]] - hasSucceeded = response.result - new_world_state = [response.world_state] - the_step = state["plan"][0] - - return { - "past_steps": ( - the_step, - hasSucceeded, - ), # * only one because new plan will be generated and old steps are all removed - "world_state": new_world_state, - } - - -@traceable(name="planner_step") -async def planner_step(state: PlanExecuteState): - """ - plan the steps based on user input and world state - """ - print(f"-----plan_step-----") - - plan = await planner.ainvoke( - { - "user_input": state["user_input"], - "world_state": state["world_state"], - } - ) - return {"plan": plan.steps} - - -@traceable(name="plan_updater_step") -async def plan_updater_step(state: PlanExecuteState): - """ - if return a response, then success, response the use, end. - otherwise, return the updated newplan (normally the same as the old plan with the first step popped out. - """ - print(f"-----plan_updater_step-----") - - # ! BOOOM! BUG! - output = await plan_updater.ainvoke( - { - "user_input": state["user_input"], - "plan": state["plan"], - "world_state": state["world_state"], - "past_steps": state["past_steps"], - } - ) - # if isinstance( - # output, UpdaterResponse - # ): # * determine if it is time to response and end - # return { - # "response": "Your last instruction has been finished." - # } # * Don't need to update. - # else: - return {"plan": output.steps} # * update the plan - - -##################################################### * construct the graph - -workflow = StateGraph(PlanExecuteState) - -workflow.add_node("planner", planner_step) - -workflow.add_node("behavior_tree_generator", behavior_tree_generate_step) - -workflow.add_node("behavior_tree_executor", behavior_tree_execute_step) - -workflow.add_node("plan_updater", plan_updater_step) - -workflow.add_node("user_input_node", user_input_step) - -workflow.set_entry_point("user_input_node") - -workflow.add_edge( - "planner", "behavior_tree_generator" -) # ! may need to add a condition here - -workflow.add_edge("behavior_tree_generator", "behavior_tree_executor") - -workflow.add_edge("behavior_tree_executor", "plan_updater") - - -def plan_updater_should_end(state: PlanExecuteState): - """ - end router - """ - print(f"-----plan_updater_should_end-----") - - # if state["response"]: - # return True - # else: - # return False - if state["plan"] == [] or len(state["plan"]) == 0: - return True - else: - return False - - -workflow.add_conditional_edges( - "plan_updater", - plan_updater_should_end, - { - True: "user_input_node", - False: "behavior_tree_generator", - }, -) - - -def user_input_should_end(state: PlanExecuteState): - """ - if the user input is empty, then end - """ - print(f"-----user_input_should_end-----") - - # print(f'user_input: {state["user_input"]}') - # print(f'world_state: {state["world_state"]}') - - if not state["user_input"] or state["user_input"] == "": - return True - else: - return False - - -workflow.add_conditional_edges( - "user_input_node", - user_input_should_end, - { - True: END, - False: "planner", - }, -) - -app = workflow.compile() - -config = {"recursion_limit": 500} - -inputs = { - "world_state": [world_state_json], - "objects": objects, -} - - -async def core_run(): - async for event in app.astream( - inputs, - config=config, - ): - for k, v in event.items(): - if k != "__end__": - print(v) - - -if __name__ == "__main__": - import asyncio - - asyncio.run(core_run()) diff --git a/experiments/lamp/tasks.txt b/experiments/lamp/tasks.txt deleted file mode 100644 index e69de29b..00000000 diff --git a/experiments/lamp/ut_test.py b/experiments/lamp/ut_test.py deleted file mode 100644 index 5af47853..00000000 --- a/experiments/lamp/ut_test.py +++ /dev/null @@ -1,1208 +0,0 @@ -import json -import os -import re -from pprint import pprint -from typing import List, Tuple, Annotated, TypedDict -import operator -import logging -import asyncio - -""" -unit tree generation -""" - -os.environ["LANGCHAIN_TRACING_V2"] = "true" -os.environ["LANGCHAIN_ENDPOINT"] = "https://api.smith.langchain.com" -os.environ["LANGCHAIN_PROJECT"] = "kios_agent" - -from kios_bt.bt_stewardship import BehaviorTreeStewardship -from kios_scene.scene_factory import SceneFactory -from kios_bt.bt_factory import BehaviorTreeFactory -from kios_robot.robot_interface import RobotInterface -from kios_world.world_interface import WorldInterface - -from kios_agent.llm_supporter import KiosLLMSupporter -from kios_agent.data_types import KiosPromptSkeleton - -from kios_agent.kios_tools import ( - BehaviorTreeExecutorTool, - BehaviorTreeSimulatorTool, - WorldStateQueryTool, -) - -from dotenv import load_dotenv - -from langchain import hub - -from langchain_openai import ChatOpenAI - -from langchain.chains.openai_functions import ( - create_structured_output_runnable, - create_openai_fn_runnable, -) -from langchain_core.utils.function_calling import convert_to_openai_function -from langchain_core.agents import AgentActionMessageLog, AgentFinish -from langchain_core.messages import HumanMessage -from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder -from langchain_core.output_parsers import JsonOutputParser, StrOutputParser -from langchain_core.pydantic_v1 import BaseModel, Field - -from langchain.agents.format_scratchpad import format_to_openai_function_messages -from langchain.agents import create_openai_functions_agent, AgentExecutor - -from langchain.prompts.pipeline import PipelinePromptTemplate -from langchain.prompts.prompt import PromptTemplate - -from langchain_community.document_loaders import TextLoader -from langchain_openai import OpenAIEmbeddings - -from langchain_community.vectorstores import FAISS -from langchain.text_splitter import ( - RecursiveCharacterTextSplitter, - CharacterTextSplitter, -) - -from langgraph.graph import StateGraph, END - -from langsmith import traceable - -load_dotenv() - -from kios_utils.pybt_test import generate_bt_stewardship, render_dot_tree -from kios_utils.pddl_problem_parser import parse_problem_init, parse_problem_objects - - -def render_bt(bt_json: dict): - test_class = BehaviorTreeFactory() - bt = test_class.from_json_to_simple_bt(bt_json) - # bt = test_class.from_json_to_tree_root(bt_json) - bt_stewardship = generate_bt_stewardship(bt) - # bt_stewardship.setup(timeout=15) - render_dot_tree(bt_stewardship) - - -####################### dirs -current_dir = os.path.dirname(os.path.abspath(__file__)) -scene_path = os.path.join(current_dir, "scene.json") -# bt_json_file_path = os.path.join(current_dir, "behavior_tree.json") -world_state_path = os.path.join(current_dir, "world_state.json") -problem_path = os.path.join(current_dir, "gearset.problem") -domain_knowledge_path = os.path.join(current_dir, "domain_knowledge.txt") - -####################### problem -with open(problem_path, "r") as file: - problem = file.read() - -####################### scene -with open(scene_path, "r") as file: - scene_json_object = json.load(file) - -scene = SceneFactory().create_scene_from_json(scene_json_object) - -####################### world -world_interface = WorldInterface() -with open(world_state_path, "r") as file: - world_state_json_object = json.load(file) - pprint(world_state_json_object) - world_interface.load_world_from_json(world_state_json_object) - -# from_problem = parse_problem_init(problem=problem) -# objects = parse_problem_objects(problem=problem) -# world_interface.update_world(from_problem) - -# world_state_json = world_interface.get_world_to_json() -# pprint(from_problem) -# pprint(world_state_json) -# pprint(objects) - - -####################### robot -robot_interface = RobotInterface( - robot_address="127.0.0.1", - robot_port=12000, -) -robot_interface.setup_scene(scene) - -####################### bt_factory -bt_factory = BehaviorTreeFactory( - world_interface=world_interface, - robot_interface=robot_interface, -) - -####################### behavior_tree_stewardship -behavior_tree_stewardship = BehaviorTreeStewardship( - behaviortree_factory=bt_factory, - world_interface=world_interface, - robot_interface=robot_interface, -) - -####################### * tools -# bt_executor_tool = BehaviorTreeExecutorTool(behavior_tree_stewardship) - -bt_sim_tool = BehaviorTreeSimulatorTool(metadata={"bt_stw": behavior_tree_stewardship}) - -# world_state_query_tool = WorldStateQueryTool(behavior_tree_stewardship) - -executor_tools = [bt_sim_tool] - -# * kios data prompt skeleton dir -data_dir = os.environ.get("KIOS_DATA_DIR").format(username=os.getlogin()) -print(data_dir) -prompt_sk_dir = os.path.join(data_dir, "prompt_skeletons") -prompt_dir = os.path.join(data_dir, "prompts") - - -# * the graph state -class PlanExecuteState(TypedDict): - plan: List[str] - behavior_tree: dict - world_state: Annotated[ - List[dict], operator.add - ] # ! to add, you need to make world_state a list of dict - past_steps: Annotated[List[Tuple], operator.add] - user_input: str - objects: dict[str, list[str]] - last_behavior_tree: dict - last_failed_node: dict - runtime_world_state: dict - behavior_tree_execution_summary: str - BTExecutionHasSucceeded: bool - - action_sequence: List[str] - - -########## verifier - -template_file = os.path.join(prompt_dir, "seq_planner/template.txt") -task_file = os.path.join(prompt_dir, "seq_planner/task.txt") -system_file = os.path.join(prompt_dir, "seq_planner/system.txt") -domain_file = os.path.join(prompt_dir, "seq_planner/domain.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {template} - """ -) - -seq_planner_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ], -) - -seq_planner_chain = ( - seq_planner_ppt_ppl - | ChatOpenAI(model="gpt-4-turbo-preview", temperature=0) - | StrOutputParser() -) - - -# * output schema of the recursive_sk_generator -class SolutionBehaviortree(BaseModel): - """the fixed behavior tree""" - - behavior_tree: dict = Field( - description="the behavior tree that adheres the required format." - ) - - -behaviortree_file = os.path.join(prompt_dir, "rec_sk_gen/behaviortree.txt") -template_file = os.path.join(prompt_dir, "rec_sk_gen/template.txt") -task_file = os.path.join(prompt_dir, "rec_sk_gen/task.txt") -system_file = os.path.join(prompt_dir, "rec_sk_gen/system.txt") -object_file = os.path.join(prompt_dir, "rec_sk_gen/object.txt") -domain_file = os.path.join(prompt_dir, "rec_sk_gen/domain.txt") -example_file = os.path.join(prompt_dir, "rec_sk_gen/example.txt") -state_file = os.path.join(prompt_dir, "rec_sk_gen/state.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) -with open(object_file, "r") as f: - object_ppt = PromptTemplate.from_template(f.read()) -with open(behaviortree_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - behaviortree_ppt = ppt_tmp.partial(input=f.read()) - -with open(state_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - state_ppt = ppt_tmp.partial(input=f.read()) - -with open(example_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - example_ppt = ppt_tmp.partial(input=f.read()) -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {state} - - {object} - - {behaviortree} - - {example} - - {template} - """ -) - -re_sk_gen_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ("object", object_ppt), - ("behaviortree", behaviortree_ppt), - ("example", example_ppt), - ("state", state_ppt), - ], -) - -rec_sk_gen_chain = ( - re_sk_gen_ppt_ppl - | ChatOpenAI(model="gpt-4-turbo-preview", temperature=0) - | JsonOutputParser() -) - -# recursive_sk_generator = create_structured_output_runnable( -# SolutionBehaviortree, -# ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), -# re_sk_gen_ppt_ppl, -# mode="openai-json", -# ) - - -##################################################### * planner -# * output schema of the planner -class Plan(BaseModel): - """Plan to follow in future""" - - steps: List[str] = Field( - description="a list of different steps to follow, should be in sorted order" - ) - - -template_file = os.path.join(prompt_dir, "planner/template.txt") -task_file = os.path.join(prompt_dir, "planner/task.txt") -system_file = os.path.join(prompt_dir, "planner/system.txt") -domain_file = os.path.join(prompt_dir, "planner/domain.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {template} - """ -) - -planner_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ], -) - -planner = create_structured_output_runnable( - Plan, ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), planner_ppt_ppl -) - -##################################################### * plan_updaterner - -template_file = os.path.join(prompt_dir, "plan_updater/template.txt") -task_file = os.path.join(prompt_dir, "plan_updater/task.txt") -system_file = os.path.join(prompt_dir, "plan_updater/system.txt") -domain_file = os.path.join(prompt_dir, "plan_updater/domain.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {template} - """ -) - -plan_updater_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ], -) - -plan_updater = create_structured_output_runnable( - Plan, ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), plan_updater_ppt_ppl -) -# ! create openai fn runnable has bug - - -##################################################### * graph node functions -@traceable(name="user_input_node_step") -async def user_input_step(state: PlanExecuteState): - """ - get the input from the user - """ - print(f"-----user_input_step-----") - - user_input = input("Your next instruction: ") - - return { - "user_input": user_input, - } - - -@traceable(name="sequence_generate_step") -async def sequence_generate_step(state: PlanExecuteState): - """ - generate the sequence based on the instruction - """ - print(f"-----sequence_generate_step-----") - - instruction = state["plan"][0] - start_world_state = state["world_state"][-1] - objects = state["objects"] - - action_sequence = await seq_planner_chain.ainvoke( - { - "objects": objects, - "start_world_state": start_world_state, - "user_instruction": instruction, - } - ) - - return { - "action_sequence": action_sequence, - } - - -@traceable(name="behavior_tree_generate_step") -async def behavior_tree_generate_step(state: PlanExecuteState): - """ - generate the behavior tree based on the instruction - """ - print(f"-----behavior_tree_generate_step-----") - - instruction = state["plan"][0] - if state["runtime_world_state"] is None: - runtime_world_state = state["world_state"][-1] - else: - runtime_world_state = state["runtime_world_state"] - objects = state["objects"] - - bt_skeleton = await rec_sk_gen_chain.ainvoke( - { - "objects": objects, - "runtime_world_state": runtime_world_state, - "last_behavior_tree": state["last_behavior_tree"], - "last_failed_node": state["last_failed_node"], - "summary": state["behavior_tree_execution_summary"], - "action_sequence": state["action_sequence"], - } - ) - - render_bt(bt_skeleton) - - pause = input("paused here") - - return { - "last_behavior_tree": bt_skeleton, - "BTExecutionHasSucceeded": False, - } - - -@traceable(name="behavior_tree_execute_step") -async def behavior_tree_execute_step(state: PlanExecuteState): - """ - execute the first step of the plan, append the result to the past steps - """ - print(f"-----behavior_tree_execute_step-----") - - this_step = state["plan"][0] - behavior_tree_skeleton = state["last_behavior_tree"] - latest_world_state = state["world_state"][-1] - - # * first sim run - tree_result, skeleton_json = behavior_tree_stewardship.sk_sim_run( - world_state=latest_world_state, skeleton_json=behavior_tree_skeleton - ) - - pprint(tree_result.to_json()) - pause = input("paused here") - - # * check result - if tree_result.result == "success": - - # * second fake run - tree_result, skeleton_json = behavior_tree_stewardship.sk_fake_run( - world_state=latest_world_state, skeleton_json=behavior_tree_skeleton - ) - - pprint(tree_result.to_json()) - pause = input("paused here") - - if tree_result.result == "success": - return { - "BTExecutionHasSucceeded": True, - "past_steps": ( - this_step, - tree_result.result, - ), # * only one because new plan will be generated and old steps are all removed - "world_state": [tree_result.world_state], - # * clear all - "last_behavior_tree": None, - "last_failed_node": None, - "runtime_world_state": None, - "behavior_tree_execution_summary": None, - } - else: - return { - "world_state": [tree_result.world_state], # * real world state - "last_behavior_tree": skeleton_json, - "last_failed_node": { - "summary": tree_result.final_node.get("summary"), - "name": tree_result.final_node.get("name"), - "identifier": tree_result.final_node.get("identifier"), - }, - "runtime_world_state": tree_result.world_state, - "behavior_tree_execution_summary": tree_result.summary, - } - - else: - return { - # ! do not change world state - "last_behavior_tree": skeleton_json, - "last_failed_node": { - "summary": tree_result.final_node.get("summary"), - "name": tree_result.final_node.get("name"), - "identifier": tree_result.final_node.get("identifier"), - }, - "runtime_world_state": tree_result.world_state, - "behavior_tree_execution_summary": tree_result.summary, - } - - -@traceable(name="planner_step") -async def planner_step(state: PlanExecuteState): - """ - plan the steps based on user input and world state - """ - print(f"-----plan_step-----") - - plan = await planner.ainvoke( - { - "user_input": state["user_input"], - "world_state": state["world_state"], - } - ) - return {"plan": plan.steps} - - -@traceable(name="plan_updater_step") -async def plan_updater_step(state: PlanExecuteState): - """ - if return a response, then success, response the use, end. - otherwise, return the updated newplan (normally the same as the old plan with the first step popped out. - """ - print(f"-----plan_updater_step-----") - - # ! BOOOM! BUG! - output = await plan_updater.ainvoke( - { - "user_input": state["user_input"], - "plan": state["plan"], - "world_state": state["world_state"], - "past_steps": state["past_steps"], - } - ) - # if isinstance( - # output, UpdaterResponse - # ): # * determine if it is time to response and end - # return { - # "response": "Your last instruction has been finished." - # } # * Don't need to update. - # else: - return {"plan": output.steps} # * update the plan - - -##################################################### * construct the graph - -workflow = StateGraph(PlanExecuteState) - -workflow.add_node("planner", planner_step) -workflow.add_node("sequence_generator", sequence_generate_step) -workflow.add_node("behavior_tree_generator", behavior_tree_generate_step) -workflow.add_node("behavior_tree_executor", behavior_tree_execute_step) -workflow.add_node("plan_updater", plan_updater_step) -workflow.add_node("user_input_node", user_input_step) -workflow.set_entry_point("user_input_node") -workflow.add_edge("planner", "sequence_generator") -workflow.add_edge("sequence_generator", "behavior_tree_generator") -workflow.add_edge("behavior_tree_generator", "behavior_tree_executor") - - -def executor_should_end(state: PlanExecuteState): - """ - end router - """ - print(f"-----executor_should_end-----") - - if state["BTExecutionHasSucceeded"] == True: - state["BTExecutionHasSucceeded"] = False - return True - else: - return False - - -workflow.add_conditional_edges( - "behavior_tree_executor", - executor_should_end, - { - True: "plan_updater", - False: "behavior_tree_generator", - }, -) - - -def plan_updater_should_end(state: PlanExecuteState): - """ - end router - """ - print(f"-----plan_updater_should_end-----") - - if state["plan"] == [] or len(state["plan"]) == 0: - return True - else: - return False - - -workflow.add_conditional_edges( - "plan_updater", - plan_updater_should_end, - { - True: "user_input_node", - False: "behavior_tree_generator", - }, -) - - -def user_input_should_end(state: PlanExecuteState): - """ - if the user input is empty, then end - """ - print(f"-----user_input_should_end-----") - - if not state["user_input"] or state["user_input"] == "": - return True - else: - return False - - -workflow.add_conditional_edges( - "user_input_node", - user_input_should_end, - { - True: END, - False: "planner", - }, -) - -app = workflow.compile() - -config = {"recursion_limit": 500} - -inputs = { - "world_state": [world_state_json_object], - # "objects": objects, # ! warning! -} - -# * unit tree generator ppl -system_file = os.path.join(prompt_dir, "new/ut_gen/system.txt") -task_file = os.path.join(prompt_dir, "new/ut_gen/task.txt") -domain_file = os.path.join(prompt_dir, "new/ut_gen/new_domain_nl.txt") -behaviortree_file = os.path.join(prompt_dir, "new/ut_gen/new_behaviortree.txt") -template_file = os.path.join(prompt_dir, "new/ut_gen/template.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) -with open(behaviortree_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - behaviortree_ppt = ppt_tmp.partial(input=f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {behaviortree} - - {template} - - {format_instructions} - """ -) - -parser = JsonOutputParser() - -format_instructions = PromptTemplate.from_template("""{input}""").partial( - input=parser.get_format_instructions() -) - -ut_gen_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ("behaviortree", behaviortree_ppt), - ("format_instructions", format_instructions), - ], -) - -ut_gen_chain = ( - ut_gen_ppt_ppl - # | ChatOpenAI(model="gpt-3.5-turbo-0125", temperature=0) - # | ChatOpenAI(model="ft:gpt-3.5-turbo-0125:kifabrik-mirmi::8y1cXwVw", temperature=0) - | ChatOpenAI( - model="ft:gpt-3.5-turbo-0125:kifabrik-mirmi:kios-ut-gen-v2:8z2KbPsr", - temperature=0, - ) - # | ChatOpenAI(model="gpt-4", temperature=0) - | JsonOutputParser() -) - -# ! sequence planner estimation ppl -system_file = os.path.join(prompt_dir, "seq_planner_est/system.txt") -task_file = os.path.join(prompt_dir, "seq_planner_est/task.txt") -domain_file = os.path.join(prompt_dir, "seq_planner_est/domain.txt") -state_file = os.path.join(prompt_dir, "seq_planner_est/state.txt") -output_format_file = os.path.join(prompt_dir, "seq_planner_est/output_format.txt") -template_file = os.path.join(prompt_dir, "seq_planner_est/template.txt") -example_file = os.path.join(prompt_dir, "seq_planner_est/example.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) -with open(output_format_file, "r") as f: - output_format_ppt = PromptTemplate.from_template(f.read()) -with open(example_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - example_ppt = ppt_tmp.partial(input=f.read()) -with open(state_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - state_ppt = ppt_tmp.partial(input=f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {state} - - {output_format} - - {example} - - {template} - - {format_instructions} - """ -) - -parser = JsonOutputParser() - -format_instructions = PromptTemplate.from_template("""{input}""").partial( - input=parser.get_format_instructions() -) - -seq_planner_est_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ("state", state_ppt), - ("output_format", output_format_ppt), - ("format_instructions", format_instructions), - ("example", example_ppt), - ], -) - -seq_planner_est_chain = ( - seq_planner_est_ppt_ppl - # | ChatOpenAI(model="gpt-4", temperature=0) - | ChatOpenAI(model="gpt-3.5-turbo-0125", temperature=0) - | JsonOutputParser() -) - - -async def core_run(): - async for event in app.astream( - inputs, - config=config, - ): - for k, v in event.items(): - if k != "__end__": - print(v) - - -def seq_planner_est_test(): - return seq_planner_est_chain.invoke( - { - "start_world_state": world_state_json_object, - # "target": "is_inserted_to(shaft1, gearbase_hole1)", - "target": "hold(left_hand, outward_claw)", - } - ) - - -# * sequence action planner -system_file = os.path.join(prompt_dir, "new/seq_plan/system.txt") -task_file = os.path.join(prompt_dir, "new/seq_plan/task.txt") -domain_file = os.path.join(prompt_dir, "new/seq_plan/new_domain_nl.txt") -state_file = os.path.join(prompt_dir, "new/seq_plan/state.txt") -output_format_file = os.path.join(prompt_dir, "new/seq_plan/output_format.txt") -template_file = os.path.join(prompt_dir, "new/seq_plan/template.txt") -example_file = os.path.join(prompt_dir, "new/seq_plan/new_example.txt") -chain_file = os.path.join(prompt_dir, "new/seq_plan/chain.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) -with open(output_format_file, "r") as f: - output_format_ppt = PromptTemplate.from_template(f.read()) -with open(example_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - example_ppt = ppt_tmp.partial(input=f.read()) -with open(state_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - state_ppt = ppt_tmp.partial(input=f.read()) -# # * this is for gpt 3.5 -# with open(chain_file, "r") as f: -# chain_ppt = PromptTemplate.from_template(f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {state} - - {output_format} - - {example} - - {template} - - {format_instructions} - """ -) - -parser = JsonOutputParser() - -format_instructions = PromptTemplate.from_template("""{input}""").partial( - input=parser.get_format_instructions() -) - -seq_ac_pl_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ("state", state_ppt), - ("output_format", output_format_ppt), - ("format_instructions", format_instructions), - ("example", example_ppt), - # ("chain", chain_ppt), - ], -) - -seq_ac_pl_chain = ( - seq_ac_pl_ppt_ppl - | ChatOpenAI(model="gpt-4", temperature=0) - # | ChatOpenAI(model="gpt-3.5-turbo-0125", temperature=0) - | JsonOutputParser() -) - -# * state estimater -system_file = os.path.join(prompt_dir, "new/state_est/system.txt") -task_file = os.path.join(prompt_dir, "new/state_est/task.txt") -domain_file = os.path.join(prompt_dir, "new/state_est/new_domain_nl.txt") -state_file = os.path.join(prompt_dir, "new/state_est/state.txt") -output_format_file = os.path.join(prompt_dir, "new/state_est/output_format.txt") -template_file = os.path.join(prompt_dir, "new/state_est/template.txt") -example_file = os.path.join(prompt_dir, "new/state_est/new_example.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) -with open(output_format_file, "r") as f: - output_format_ppt = PromptTemplate.from_template(f.read()) -with open(example_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - example_ppt = ppt_tmp.partial(input=f.read()) -with open(state_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - state_ppt = ppt_tmp.partial(input=f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {state} - - {output_format} - - {example} - - {template} - - {format_instructions} - """ -) - -parser = JsonOutputParser() - -format_instructions = PromptTemplate.from_template("""{input}""").partial( - input=parser.get_format_instructions() -) - -state_est_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ("state", state_ppt), - ("output_format", output_format_ppt), - ("format_instructions", format_instructions), - ("example", example_ppt), - ], -) - - -state_est_ppt_ppl_chain = ( - state_est_ppt_ppl - # | ChatOpenAI(model="gpt-3.5-turbo", temperature=0) - | ChatOpenAI(model="gpt-4", temperature=0) - | JsonOutputParser() -) - - -async def core_run(): - raise NotImplementedError - async for event in app.astream( - inputs, - config=config, - ): - for k, v in event.items(): - if k != "__end__": - print(v) - - -def ut_gen_test(): - ut = ut_gen_chain.invoke( - { - # "action": "insert(left_hand, defaultgripper, gear1, shaft1)", - # "action": "insert(left_hand, parallel_box1, shaft1, gearbase_hole1)", - # "action": "change_tool(left_hand, parallel_box1, defaultgripper)", - "action": "insert(left_hand, clampgripper, gear2, shaft2)", - } - ) - - render_bt(ut) - - -def seq_planner_est_test(): - return seq_planner_est_chain.invoke( - { - "start_world_state": world_state_json_object, - "target": "is_inserted_to(shaft1, gearbase_hole1)", - # "target": "hold(left_hand, outward_claw)", - } - ) - - -def seq_action_plan_test(): - return seq_ac_pl_chain.invoke( - { - "start_world_state": world_state_json_object, - "target": "is_inserted_to(shaft1, gearbase_hole1)", - # "target": "hold(left_hand, outward_claw)", - } - ) - - -def state_est_test(): - raise NotImplementedError - return state_est_ppt_ppl_chain.invoke( - { - "start_world_state": world_state_json_object, - "action_plan": [ - "pick_up(left_hand, parallel_box1, shaft1)", - "insert(left_hand, parallel_box1, shaft1, gearbase_hole1)", - ], - } - ) - - -##################################### * speed imp -@traceable(name="make_plan") -def make_plan(state: dict, goal: str) -> list[str]: - print(f"----------start to make plan for the goal {goal}") - response = seq_ac_pl_chain.invoke( - { - "start_world_state": state, - "target": goal, - } - ) - print(f"finished making plan for the goal {goal}.") - pprint(f'LLM thought flow: {response["explanation"]}') - return response["task_plan"] - - -@traceable(name="estimate_state") -def estimate_state(start_world_state: dict, action_plan: list[str]) -> dict: - print("----------start to estimate the state after the action plan:") - pprint(action_plan) - response = state_est_ppt_ppl_chain.invoke( - { - "start_world_state": start_world_state, - "action_plan": action_plan, - } - ) - print(f"finished estimating the state after the action plan {action_plan}.") - return response["estimated_world_state"] - - -@traceable(name="generate_unit_subtree") -def generate_unit_subtree(action: str) -> dict: - print("----------start to generate the unit subtree for the action") - pprint(action) - response = ut_gen_chain.invoke( - { - "action": action, - } - ) - print(f"finished generating the unit subtree for the action {action}.") - return response - - -def get_node_list_from_tree(unit_subtree: dict) -> list[dict]: - children = unit_subtree["children"][1][ - "children" - ] # * the second child is a sequence - return children - - -def extract_goal(node: dict) -> str: - name = node["name"] - - -def match_type(node: dict) -> tuple[str, str]: - node_name = node["name"] - match = re.search( - r"(selector|sequence|action|precondition|condition|target):\s*(.+)", node_name - ) - if match: - node_type = match.group(1) - node_body = match.group(2) - return node_type, node_body - else: - raise ValueError(f"the node name {node_name} does not match any type.") - - -def expand_nodes( - node_list: list[dict], - start_state: dict, - overall_tree: list[dict] = None, -) -> dict: - """ - in order to monitor the tree generation, the overall tree and the node list should be the same variable when being passed in. - """ - pprint("----------check the entire tree:") - if overall_tree is not None: - render_bt(overall_tree[0]) - pprint("----------start to expand the node list:") - pprint(node_list) - pause = input("paused here! check the tree.") - - assert len(node_list) > 0 - state = start_state - - for i in range(len(node_list)): - type_name, body = match_type(node_list[i]) - # if match_type(node_list[i]) == "action": - if type_name == "action": - print(f"the node {node_list[i]['name']} is an action node. skip it.") - pause = input("paused here! check!") - # elif match_type(node_list[i]) == "precondition" or "target": - elif type_name in ["precondition", "target"]: - # goal = node_list[i]["name"] - goal = body - plan = make_plan(state, goal) - if len(plan) == 0: - logging.warning(f"No action should be performed for the goal {goal}.") - logging.warning(f'the node {node_list[i]["name"]} has been skipped.') - pause = input("paused here! check!") - else: - logging.info(f"Actions have been planed for the goal {goal}.") - pprint(f"the plan for the goal {goal} is {plan}") - pause = input("paused here! check!") - last_action = plan[-1] - unit_subtree = generate_unit_subtree(last_action) - # insert the subtree into the node_list - node_list[i] = unit_subtree - new_node_list = get_node_list_from_tree(unit_subtree) - expand_nodes( - node_list=new_node_list, - start_state=state, - overall_tree=overall_tree, - ) - state = estimate_state(state, plan) - - return node_list[0] - - -def embed_ut_nl(unit_subtree: dict) -> str: - """ - embed the unit subtree into the overall tree - """ - selector_children = unit_subtree["children"] - target = "" - # * target - for node in selector_children: - if match_type(node)[0] == "target": - target += node["summary"] - - sequence_children = selector_children[1]["children"] - preconditions = [] - action = "" - for node in sequence_children: - if match_type(node)[0] == "precondition": - preconditions.append(node["summary"]) - if match_type(node)[0] == "action": - action = node["summary"] - - embedding = f"if {' and '.join(preconditions)} then {action}, {target}" - - return embedding - - -def test_embedding_nl(): - unit_tree = ut_gen_test() - pprint(unit_tree) - nl = embed_ut_nl(unit_tree) - print(nl) - - -def test_expand_nodes(): - start_state = world_state_json_object - node_list = [ - # { - # "summary": "screw lampbulb into lampbase", - # "name": "target: screw the bulb into the base of the lamp", - # } - { - "summary": "place the lamp shard onto the bulb of the lamp", - "name": "target: place the lamp shard onto the bulb of the lamp", - } - ] - expand_nodes(node_list, start_state, node_list) - pprint(node_list) - - -if __name__ == "__main__": - pass - - # asyncio.run(core_run()) - # pprint(ut_gen_test()) - # pprint(seq_action_plan_test()) - # pprint(state_est_test()) - - test_expand_nodes() - - # test_embedding_nl() diff --git a/experiments/lamp/world_state copy.json b/experiments/lamp/world_state copy.json deleted file mode 100644 index 5796816c..00000000 --- a/experiments/lamp/world_state copy.json +++ /dev/null @@ -1,50 +0,0 @@ -{ - "objects": [ - { - "name": "clampgripper", - "properties": ["is_empty"] - }, - { - "name": "parallelgripper", - "properties": ["is_empty"] - }, - { - "name": "inwardgripper", - "properties": ["is_empty"] - }, - { - "name": "outwardgripper", - "properties": ["is_empty"] - }, - { - "name": "defaultgripper", - "properties": ["is_empty"] - }, - { - "name": "lampbase", - "properties": [] - }, - { - "name": "lampshade", - "properties": [] - }, - { - "name": "lampbulb", - "properties": [] - }, - { - "name": "left_hand", - "properties": [] - } - ], - "constraints": [ - {"source": "clampgripper", "name": "can_manipulate", "target": "lampbulb"}, - {"source": "outwardgripper", "name": "can_manipulate", "target": "lampshade"}, - {"source": "lampbulb", "name": "can_screw_to", "target": "lampbase"}, - {"source": "lampshade", "name": "can_place_to", "target": "lampbulb"} - ], - "relations": [ - {"source": "left_hand", "name": "hold", "target": "defaultgripper"} - ] -} - diff --git a/experiments/lamp/world_state.json b/experiments/lamp/world_state.json deleted file mode 100644 index cb605451..00000000 --- a/experiments/lamp/world_state.json +++ /dev/null @@ -1,60 +0,0 @@ -{ - "objects": [ - { - "name": "clampgripper", - "properties": ["is_empty"] - }, - { - "name": "parallelgripper", - "properties": ["is_empty"] - }, - { - "name": "inwardgripper", - "properties": ["is_empty"] - }, - { - "name": "outwardgripper", - "properties": ["is_empty"] - }, - { - "name": "defaultgripper", - "properties": ["is_empty"] - }, - { - "name": "lampbase", - "properties": [] - }, - { - "name": "lampshade", - "properties": [] - }, - { - "name": "lampbulb", - "properties": [] - }, - { - "name": "cube", - "properties": [] - }, - { - "name": "designring", - "properties": [] - }, - { - "name": "left_hand", - "properties": [] - } - ], - "constraints": [ - {"source": "clampgripper", "name": "can_manipulate", "target": "lampbulb"}, - {"source": "outwardgripper", "name": "can_manipulate", "target": "lampshade"}, - {"source": "lampbulb", "name": "can_screw_to", "target": "lampbase"}, - {"source": "lampshade", "name": "can_place_to", "target": "lampbulb"} - ], - "relations": [ - {"source": "left_hand", "name": "hold", "target": "inwardgripper"}, - {"source": "lampbulb", "name": "is_screwed_to", "target": "lampbase"}, - {"source": "lampshade", "name": "is_placed_to", "target": "lampbulb"} - ] -} - diff --git a/experiments/workstation/experiment.py b/experiments/workstation/experiment.py deleted file mode 100644 index bdf926cd..00000000 --- a/experiments/workstation/experiment.py +++ /dev/null @@ -1,117 +0,0 @@ -import socket -from pprint import pprint -from runtime_script import * -from plan_parser import parse_plan, PlanStep - - -HOST = "10.157.175.108" # The server's hostname or IP address -PORT = 65431 # The port used by the server - -connected = False -# while not connected: -# try: -# with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s: -# s.connect((HOST, PORT)) -# data = s.recv(4096) -# connected = True -# if connected: -# print("Connected to the server") -# s.close() - -# except socket.error as e: -# print(f"Connection failed: {e}") -# Add any necessary delay before retrying the connection - - -# print(f"Received {data!r}") - - -order_dict: dict[str, object] = { - "outerring": insert_outerring, - "cone": insert_cone, - "outputshaftandgearstage2": insert_outputshaftandgearstage2, - "ringgear": insert_ringgear, - "designring": insert_designring, - "gearstage1": insert_gearstage1, - "driveflange": insert_driveflange, - "inputpinion": insert_inputpinion, -} - -# workflow -# 1. receive plan - -# 2. parse plan -# task_plan = parse_plan(data.decode()) -task_plan = parse_plan( - """ -0.0000: (initialize defaultgripper) [d:10.0000; c:1.0000] -0.0003: (change_gripper defaultgripper outwardgripper) [d:10.0000; c:1.0000] -10.0005: (initialize outwardgripper) [d:10.0000; c:1.0000] -20.0007: (insert outwardgripper outerring housing) [d:23.0000; c:1.0000] -43.0010: (insert outwardgripper cone outerring) [d:19.0000; c:1.0000] -62.0013: (change_gripper outwardgripper defaultgripper) [d:10.0000; c:1.0000] -72.0015: (initialize defaultgripper) [d:1.0000; c:1.0000] -73.0017: (insert defaultgripper outputshaftandgearstage2 cone) [d:40.0000; c:1.0000] -113.0020: (change_gripper defaultgripper parallelgripper) [d:10.0000; c:1.0000] -123.0023: (initialize parallelgripper) [d:10.0000; c:1.0000] -133.0025: (insert parallelgripper ringgear outputshaftandgearstage2) [d:25.0000; c:1.0000] -158.0027: (change_gripper parallelgripper inwardgripper) [d:25.0000; c:1.0000] -183.0030: (initialize inwardgripper) [d:10.0000; c:1.0000] -193.0033: (insert inwardgripper gearstage1 ringgear) [d:25.0000; c:1.0000] -218.0035: (change_gripper inwardgripper clampgripper) [d:30.0000; c:1.0000] -248.0038: (initialize clampgripper) [d:10.0000; c:1.0000] -258.0040: (insert clampgripper designring ringgear) [d:13.0000; c:1.0000] -271.0042: (change_gripper clampgripper defaultgripper) [d:15.0000; c:1.0000] -286.0045: (initialize defaultgripper) [d:1.0000; c:1.0000] -287.0048: (insert defaultgripper driveflange gearstage1) [d:25.0000; c:1.0000] -312.0050: (insert defaultgripper inputpinion driveflange) [d:25.0000; c:1.0000] -""" -) -# 3. execute plan -set_tool("defaultgripper") - - -def result_montage(plan_step: PlanStep, execution_time: float, last_result: str) -> str: - print(type(last_result)) - last_result += f"{plan_step.raw_action}, {execution_time}\n" - return last_result - - -result = "" -for plan_step in task_plan: - pprint(f"Executing {plan_step}") - action_name = plan_step.action_name - action_params = plan_step.action_params - execution_time = 0 - if action_name == "change_gripper": - execution_time = change_gripper(action_params[0], action_params[1]) - elif action_name == "initialize": - execution_time = initialize_gripper() - elif action_name == "insert": - part_name = action_params[1] - execution_time = order_dict[part_name]() - else: - raise ValueError(f"Unknown action name: {action_name}") - - result = result_montage(plan_step, execution_time, result) - -pprint(result) -file_path = os.path.join(os.path.dirname(__file__), "result.txt") -with open(file_path, "w") as f: - f.write(result) - f.close() - -# 4. send execution time back - - -def test_montage(): - result = "" - for plan_step in task_plan: - result = result_montage(plan_step, 1, result) - - pprint(result) - - -if __name__ == "__main__": - # test_montage() - pass diff --git a/experiments/workstation/mios_memory_backups/p1.json b/experiments/workstation/mios_memory_backups/p1.json deleted file mode 100644 index 4898d17d..00000000 --- a/experiments/workstation/mios_memory_backups/p1.json +++ /dev/null @@ -1,1502 +0,0 @@ -[ - { - "_id": "65ea15e1153280ec90059137", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "geometry": null, - "grasp_force": 1.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "TestObject1", - "q": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ] - }, - { - "_id": "65ea16c0866d42f47d065a81", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "NullObject", - "q": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ] - }, - { - "_id": "65ea16c0866d42f47d065a82", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "NoneObject", - "q": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ] - }, - { - "_id": "65ea16c0866d42f47d065a83", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.042287769943322374, - 0.9980517903062266, - -0.04566306254474026, - 0.0, - 0.9990804305161161, - -0.04198911477450946, - 0.007480274152296298, - 0.0, - 0.0055484562605105535, - -0.045938280752606364, - -0.9989288708384696, - 0.0, - -0.26888258842678553, - 0.5138899787320308, - 0.24173276815231357, - 1.0 - ], - "O_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "EndEffector", - "q": [ - 1.6365671767472172, - 0.23061633128327919, - 0.43165040675425126, - -2.016175535998035, - -0.09652444181839626, - 2.1807552888520347, - 1.3740795837962296 - ] - }, - { - "_id": "65ea1a22774d13c06f0ab061", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "test1", - "q": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ] - }, - { - "_id": "65eb2c96f34d2aed38043a31", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "test2", - "q": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ] - }, - { - "_id": "65f0c1b866ba81ce730084d1", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.9942893648763174, - -0.08568876646518844, - -0.06345739591830227, - 0.0, - -0.08572927432493825, - -0.9963066161587593, - 0.002089263796302965, - 0.0, - -0.06340327054456749, - 0.0033628884755651555, - -0.9979823226216751, - 0.0, - 0.5455366490412883, - 0.2997315091727984, - 0.005848237661269337, - 1.0 - ], - "O_T_TCP": [ - 0.9942893648763174, - -0.08568876646518844, - -0.06345739591830227, - 0.0, - -0.08572927432493825, - -0.9963066161587593, - 0.002089263796302965, - 0.0, - -0.06340327054456749, - 0.0033628884755651555, - -0.9979823226216751, - 0.0, - 0.5455366490412883, - 0.2997315091727984, - 0.005848237661269337, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "p1_sp1", - "q": [ - 0.3462297654564948, - 0.763475231704608, - 0.15242745088954474, - -1.8409007817485883, - -0.13086174158455272, - 2.5389470826374514, - 1.4410418963132232 - ] - }, - { - "_id": "65f0c2b366ba81ce730084d2", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.9989181631966062, - -0.040443764137969046, - -0.02252891651808989, - 0.0, - -0.04040494464074937, - -0.9991713832943138, - 0.0021758114610663888, - 0.0, - -0.02259868178169498, - -0.001263202283979461, - -0.9997438191365424, - 0.0, - 0.6723192102331684, - 0.1647639776697898, - 0.022932490731639197, - 1.0 - ], - "O_T_TCP": [ - 0.9989181631966062, - -0.040443764137969046, - -0.02252891651808989, - 0.0, - -0.04040494464074937, - -0.9991713832943138, - 0.0021758114610663888, - 0.0, - -0.02259868178169498, - -0.001263202283979461, - -0.9997438191365424, - 0.0, - 0.6723192102331684, - 0.1647639776697898, - 0.022932490731639197, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "p1_sp2", - "q": [ - 0.17995285498081146, - 0.8774959976561396, - 0.06959877055272953, - -1.5531564836000213, - -0.07408472516139349, - 2.4057519419723086, - 1.1064279931609828 - ] - }, - { - "_id": "65f206f8cc3ba507550a6ff1", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.02315269158307143, - 0.965175082430463, - 0.26054128277303057, - 0.0, - 0.9993563585151953, - -0.015293582118205694, - -0.03215154650169135, - 0.0, - -0.02704778279539906, - 0.26112300986211173, - -0.9649265211229322, - 0.0, - 0.009127540953774844, - 0.3975794951895181, - 0.5661061123661979, - 1.0 - ], - "O_T_TCP": [ - 0.02315269158307143, - 0.965175082430463, - 0.26054128277303057, - 0.0, - 0.9993563585151953, - -0.015293582118205694, - -0.03215154650169135, - 0.0, - -0.02704778279539906, - 0.26112300986211173, - -0.9649265211229322, - 0.0, - 0.009127540953774844, - 0.3975794951895181, - 0.5661061123661979, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "initialposition1", - "q": [ - 1.6026304399290074, - -0.6303371550189459, - -0.04813256275288606, - -2.130002201632451, - 0.001543009983380258, - 1.76419766599399, - 0.7940453165595731 - ] - }, - { - "_id": "65f20753cc3ba507550a6ff2", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.021982037329596582, - 0.9977446921449092, - -0.06326978925766114, - 0.0, - 0.9997436921287516, - -0.022138906592690365, - -0.0017792596382354192, - 0.0, - -0.0031760319629688225, - -0.06321567805857012, - -0.9979948350911267, - 0.0, - -0.27773647180912625, - 0.36657797217967986, - 0.27310663202830676, - 1.0 - ], - "O_T_TCP": [ - 0.021982037329596582, - 0.9977446921449092, - -0.06326978925766114, - 0.0, - 0.9997436921287516, - -0.022138906592690365, - -0.0017792596382354192, - 0.0, - -0.0031760319629688225, - -0.06321567805857012, - -0.9979948350911267, - 0.0, - -0.27773647180912625, - 0.36657797217967986, - 0.27310663202830676, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "shaft1", - "q": [ - 1.6408883520379005, - -0.19673739984072505, - 0.5540051769339669, - -2.3971655053925094, - 0.17244883296322921, - 2.17195843217108, - 1.3185539446100591 - ] - }, - { - "_id": "65f20765cc3ba507550a6ff3", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.03293439988601239, - 0.9987900577924511, - -0.0362559327601579, - 0.0, - 0.9994208324819104, - -0.03264493724284296, - 0.00854719689631032, - 0.0, - 0.0073534242085959226, - -0.036517134370048415, - -0.999305972187602, - 0.0, - -0.3958818737254737, - 0.48449203683366127, - 0.2626215337425854, - 1.0 - ], - "O_T_TCP": [ - 0.03293439988601239, - 0.9987900577924511, - -0.0362559327601579, - 0.0, - 0.9994208324819104, - -0.03264493724284296, - 0.00854719689631032, - 0.0, - 0.0073534242085959226, - -0.036517134370048415, - -0.999305972187602, - 0.0, - -0.3958818737254737, - 0.48449203683366127, - 0.2626215337425854, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "p1_n1", - "q": [ - 1.7225042117540545, - 0.3657234275382862, - 0.5893249507535949, - -1.8132862764520385, - -0.20844032673041035, - 2.079098628333355, - 1.6369054819966353 - ] - }, - { - "_id": "65f208923d5d3aa33909fa01", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.9999794063468496, - 0.003907442838487518, - -0.002581796636912931, - 0.0, - 0.003914417146569122, - -0.9999790620267289, - 0.0027018045366418294, - 0.0, - -0.0025712349367766453, - -0.0027119073382606647, - -0.9999930171303639, - 0.0, - 0.6831131784135639, - 0.15091057610990297, - 0.03872875431078723, - 1.0 - ], - "O_T_TCP": [ - 0.9999794063468496, - 0.003907442838487518, - -0.002581796636912931, - 0.0, - 0.003914417146569122, - -0.9999790620267289, - 0.0027018045366418294, - 0.0, - -0.0025712349367766453, - -0.0027119073382606647, - -0.9999930171303639, - 0.0, - 0.6831131784135639, - 0.15091057610990297, - 0.03872875431078723, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "gear3", - "q": [ - 0.6594461028586938, - 0.959457393611405, - -0.5421085347795604, - -1.518537465363218, - 0.6018891127984255, - 2.3034558436616237, - 0.6758813112820262 - ] - }, - { - "_id": "65f35fda57dfe07413073051", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.9999650294866161, - 0.00028945537872346935, - 0.0071135729856895156, - 0.0, - 0.00027974340259437455, - -0.9999894009905329, - 0.0013662179299346368, - 0.0, - 0.007114030015113163, - -0.0013642064426819475, - -0.9999737644147099, - 0.0, - 0.6083333407636259, - 0.33128662413723653, - 0.23664705308657602, - 1.0 - ], - "O_T_TCP": [ - 0.9999650294866161, - 0.00028945537872346935, - 0.0071135729856895156, - 0.0, - 0.00027974340259437455, - -0.9999894009905329, - 0.0013662179299346368, - 0.0, - 0.007114030015113163, - -0.0013642064426819475, - -0.9999737644147099, - 0.0, - 0.6083333407636259, - 0.33128662413723653, - 0.23664705308657602, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "observe", - "q": [ - 0.3459440073869611, - 0.5302255583652264, - 0.187803218214187, - -1.5665877921623095, - -0.11445701398452121, - 2.091846121125751, - 1.3501843255584438 - ] - }, - { - "_id": "65f3636b57dfe07413073052", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.0341290648824769, - 0.9989770754074333, - -0.02933865439769084, - 0.0, - 0.9992787200320924, - -0.03458174649380318, - -0.015062848391641067, - 0.0, - -0.016062331392090512, - -0.028803966649530662, - -0.9994560185498427, - 0.0, - -0.27153903748409997, - 0.6016748175843831, - 0.24058478569993352, - 1.0 - ], - "O_T_TCP": [ - 0.0341290648824769, - 0.9989770754074333, - -0.02933865439769084, - 0.0, - 0.9992787200320924, - -0.03458174649380318, - -0.015062848391641067, - 0.0, - -0.016062331392090512, - -0.028803966649530662, - -0.9994560185498427, - 0.0, - -0.27153903748409997, - 0.6016748175843831, - 0.24058478569993352, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "sp_shaft1", - "q": [ - 1.6408446938644339, - 0.46180202664409314, - 0.4038521069080699, - -1.695381354750248, - -0.17296325870354967, - 2.0984616136285994, - 1.3461262951300184 - ] - }, - { - "_id": "65f3637957dfe07413073053", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.04414296473111611, - 0.998550232651831, - -0.030488988781691134, - 0.0, - 0.9989616227813466, - -0.043802808383940066, - 0.011736144552070059, - 0.0, - 0.010383826460317257, - -0.030975994309240324, - -0.9994661894854653, - 0.0, - -0.26837836825969313, - 0.5104141379096356, - 0.2418546257938379, - 1.0 - ], - "O_T_TCP": [ - 0.04414296473111611, - 0.998550232651831, - -0.030488988781691134, - 0.0, - 0.9989616227813466, - -0.043802808383940066, - 0.011736144552070059, - 0.0, - 0.010383826460317257, - -0.030975994309240324, - -0.9994661894854653, - 0.0, - -0.26837836825969313, - 0.5104141379096356, - 0.2418546257938379, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "sp_shaft2", - "q": [ - 1.6444225503142595, - 0.21419751004882376, - 0.42614262542097237, - -2.03883945578859, - -0.1020472758615212, - 2.1995776044125415, - 1.3834399376704072 - ] - } -] \ No newline at end of file diff --git a/experiments/workstation/mios_memory_backups/p2.json b/experiments/workstation/mios_memory_backups/p2.json deleted file mode 100644 index eb26eb63..00000000 --- a/experiments/workstation/mios_memory_backups/p2.json +++ /dev/null @@ -1,2102 +0,0 @@ -[ - { - "_id": "65eb7417886cb04353024ad1", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "geometry": null, - "grasp_force": 1.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "TestObject1", - "q": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ] - }, - { - "_id": "65eb741d2c800a25d3000d41", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "NullObject", - "q": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ] - }, - { - "_id": "65eb741d2c800a25d3000d42", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "NoneObject", - "q": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ] - }, - { - "_id": "65eb741d2c800a25d3000d43", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.9183359205924548, - 0.3957289486829569, - -0.0062034687956667766, - 0.0, - 0.3957578510017352, - -0.9183338156449309, - 0.004412857885622489, - 0.0, - -0.003950635619572146, - -0.006507682681223793, - -0.9999710208522654, - 0.0, - 0.5417627689942376, - 0.23399058036255296, - -0.015003143594720176, - 1.0 - ], - "O_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "EndEffector", - "q": [ - 0.5118141559725405, - 0.7337109254316344, - -0.0929440812059468, - -1.991821720303295, - 0.14003232403347937, - 2.713309294303267, - 0.6932960660630652 - ] - }, - { - "_id": "65eb74332c800a25d3000d44", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - -0.9993938589627782, - -0.019959674635375253, - 0.02818284863431975, - 0.0, - -0.01984070888343409, - 0.9997834226459386, - 0.0044945494306065075, - 0.0, - -0.02826699883912122, - 0.00393273312179257, - -0.999592672235457, - 0.0, - -0.37484573960170936, - 0.3533845541917834, - 0.1748708579262803, - 1.0 - ], - "O_T_TCP": [ - -0.9993938589627782, - -0.019959674635375253, - 0.02818284863431975, - 0.0, - -0.01984070888343409, - 0.9997834226459386, - 0.0044945494306065075, - 0.0, - -0.02826699883912122, - 0.00393273312179257, - -0.999592672235457, - 0.0, - -0.37484573960170936, - 0.3533845541917834, - 0.1748708579262803, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "screwbolt", - "q": [ - 1.6550573774574837, - 0.15321099584385517, - 0.7222888105859068, - -2.3508161678625883, - -0.13811986689658284, - 2.484654771697428, - 0.10289976926765454 - ] - }, - { - "_id": "65eb74ac2c800a25d3000d45", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - -0.9754081093603869, - -0.21792603402093702, - 0.03268043437760813, - 0.0, - -0.21721197651678822, - 0.9758183257555578, - 0.024047853927084306, - 0.0, - -0.037131535089479584, - 0.016358204934740318, - -0.9991764900322725, - 0.0, - -0.49456060992516254, - 0.34523358993584274, - 0.18533646391614586, - 1.0 - ], - "O_T_TCP": [ - -0.9754081093603869, - -0.21792603402093702, - 0.03268043437760813, - 0.0, - -0.21721197651678822, - 0.9758183257555578, - 0.024047853927084306, - 0.0, - -0.037131535089479584, - 0.016358204934740318, - -0.9991764900322725, - 0.0, - -0.49456060992516254, - 0.34523358993584274, - 0.18533646391614586, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "screwhole", - "q": [ - 1.331823483964853, - 1.2933396335568335, - 1.6098604303502224, - -1.920375438578031, - -1.3170606204536117, - 1.6645034709950923, - 0.6755126488672281 - ] - }, - { - "_id": "65eb8ec1d3061a79e90e03d1", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - -1.0, - 0.0, - 0.0, - 0.0, - 0.0, - -1.0, - 0.0, - 0.521827226443675, - 0.25840698062581857, - 0.27517533664525384, - 1.0 - ], - "O_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - -1.0, - 0.0, - 0.0, - 0.0, - 0.0, - -1.0, - 0.0, - 0.521827226443675, - 0.25840698062581857, - 0.27517533664525384, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "test", - "q": [ - 0.18074043324946532, - 0.1335626147107068, - 0.2868349701521744, - -2.060772508320055, - -0.04825592930051324, - 2.2430690224108654, - 1.2386451794617466 - ] - }, - { - "_id": "65ef6331e99f2576ba086061", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - -0.9999511693327637, - 0.005795104725127399, - 0.006694969191221638, - 0.0, - 0.005676923956787325, - 0.9998204500065859, - -0.017538163650304785, - 0.0, - -0.00679553343966406, - -0.017499637341326747, - -0.9998237761816791, - 0.0, - -0.28525213823562634, - 0.5657042239474747, - 0.23866900068692073, - 1.0 - ], - "O_T_TCP": [ - -0.9999511693327637, - 0.005795104725127399, - 0.006694969191221638, - 0.0, - 0.005676923956787325, - 0.9998204500065859, - -0.017538163650304785, - 0.0, - -0.00679553343966406, - -0.017499637341326747, - -0.9998237761816791, - 0.0, - -0.28525213823562634, - 0.5657042239474747, - 0.23866900068692073, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "normalcontainer", - "q": [ - 1.48723348463888, - 0.4174561200767826, - 0.6111280596883836, - -1.8220793050632123, - -0.26309194494459087, - 2.141865217341235, - -0.1461896891569508 - ] - }, - { - "_id": "65ef6455e99f2576ba086062", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - -0.9990336274326229, - -0.043224038692152265, - 0.006651363706987428, - 0.0, - -0.04306611338611579, - 0.9988138548284101, - 0.022292155163513752, - 0.0, - -0.00760717766310412, - 0.02198458752415408, - -0.9997293677587924, - 0.0, - -0.2767815159120372, - 0.4389463906506993, - 0.3408536221713534, - 1.0 - ], - "O_T_TCP": [ - -0.9990336274326229, - -0.043224038692152265, - 0.006651363706987428, - 0.0, - -0.04306611338611579, - 0.9988138548284101, - 0.022292155163513752, - 0.0, - -0.00760717766310412, - 0.02198458752415408, - -0.9997293677587924, - 0.0, - -0.2767815159120372, - 0.4389463906506993, - 0.3408536221713534, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "normalpickplace", - "q": [ - 1.6710431030315733, - -0.11933810338041052, - 0.44976917909321673, - -2.1647275869469373, - 0.053616515815126296, - 2.079309125394282, - -0.3063374023692766 - ] - }, - { - "_id": "65f0c57f8fca00707f09dc71", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.9943836026315849, - -0.0968288341608145, - -0.042499112857760084, - 0.0, - -0.09580790301125691, - -0.9950650220230038, - 0.025440019028163983, - 0.0, - -0.04475356970019007, - -0.021225795551839743, - -0.9987725384702374, - 0.0, - 0.47926793066474926, - 0.23916614873158387, - -0.004146619879005292, - 1.0 - ], - "O_T_TCP": [ - 0.9943836026315849, - -0.0968288341608145, - -0.042499112857760084, - 0.0, - -0.09580790301125691, - -0.9950650220230038, - 0.025440019028163983, - 0.0, - -0.04475356970019007, - -0.021225795551839743, - -0.9987725384702374, - 0.0, - 0.47926793066474926, - 0.23916614873158387, - -0.004146619879005292, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "p2_sp", - "q": [ - 0.30748095076589516, - 0.6224689720722666, - 0.1322143079238894, - -2.174735181272256, - -0.1994334632820553, - 2.7361665597756706, - 1.4845305872601067 - ] - }, - { - "_id": "65f0c58a8fca00707f09dc72", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - -0.3906372923180876, - 0.9204079634018747, - 0.015245775087359574, - 0.0, - 0.9202843547242807, - 0.39086355071749906, - -0.01682670708818005, - 0.0, - -0.02144686590450544, - 0.007457452572057124, - -0.9997421759353806, - 0.0, - 0.5416091667977869, - 0.23056869998184873, - -0.005339503846320045, - 1.0 - ], - "O_T_TCP": [ - -0.3906372923180876, - 0.9204079634018747, - 0.015245775087359574, - 0.0, - 0.9202843547242807, - 0.39086355071749906, - -0.01682670708818005, - 0.0, - -0.02144686590450544, - 0.007457452572057124, - -0.9997421759353806, - 0.0, - 0.5416091667977869, - 0.23056869998184873, - -0.005339503846320045, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "p2_sp1", - "q": [ - 0.3674982954399627, - 0.7058114279278537, - 0.031059863668303392, - -2.001528142847549, - -0.011622414170942152, - 2.6900898772047506, - -0.7850909599531587 - ] - }, - { - "_id": "65f20f89ad440b37f0074621", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - -0.0398234703603432, - 0.9990306890185293, - 0.018235144880958364, - 0.0, - 0.9991950897154301, - 0.03985333597094554, - -0.0012771850955814157, - 0.0, - -0.002002717019971174, - 0.01816995511044529, - -0.9998329069678704, - 0.0, - -0.2748624745859707, - 0.44021610755685797, - 0.1766043743163428, - 1.0 - ], - "O_T_TCP": [ - -0.0398234703603432, - 0.9990306890185293, - 0.018235144880958364, - 0.0, - 0.9991950897154301, - 0.03985333597094554, - -0.0012771850955814157, - 0.0, - -0.002002717019971174, - 0.01816995511044529, - -0.9998329069678704, - 0.0, - -0.2748624745859707, - 0.44021610755685797, - 0.1766043743163428, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "shaft2", - "q": [ - 1.5506422050658024, - 0.16138674089992255, - 0.575727326601761, - -2.3337822530144487, - -0.15566768010457357, - 2.480229851319371, - 1.4168640595144946 - ] - }, - { - "_id": "65f21187983e0351740564c1", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - -0.03847458660057955, - 0.9992279041866523, - 0.006636910566905391, - 0.0, - 0.9989573170328562, - 0.0386231991192383, - -0.023943143852083248, - 0.0, - -0.024181461737390535, - 0.005708897725647523, - -0.9996912850450389, - 0.0, - -0.28040527035598145, - 0.5625029367429322, - 0.21460991049071504, - 1.0 - ], - "O_T_TCP": [ - -0.03847458660057955, - 0.9992279041866523, - 0.006636910566905391, - 0.0, - 0.9989573170328562, - 0.0386231991192383, - -0.023943143852083248, - 0.0, - -0.024181461737390535, - 0.005708897725647523, - -0.9996912850450389, - 0.0, - -0.28040527035598145, - 0.5625029367429322, - 0.21460991049071504, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "p2_n2", - "q": [ - 1.5703048682036513, - 0.39783672250131485, - 0.4982862662432487, - -1.8933198999940302, - -0.21576868739395502, - 2.251606949418013, - 1.345509883842923 - ] - }, - { - "_id": "65f21226983e0351740564c3", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - -0.7670325848858576, - -0.6410938681792032, - 0.025306378791810035, - 0.0, - -0.6395410237084117, - 0.7671364076897498, - 0.049696658761641144, - 0.0, - -0.05127465491506417, - 0.021934911546283895, - -0.9984436736335195, - 0.0, - -0.27998567269576946, - 0.5668166088777099, - 0.2224894936706944, - 1.0 - ], - "O_T_TCP": [ - -0.7670325848858576, - -0.6410938681792032, - 0.025306378791810035, - 0.0, - -0.6395410237084117, - 0.7671364076897498, - 0.049696658761641144, - 0.0, - -0.05127465491506417, - 0.021934911546283895, - -0.9984436736335195, - 0.0, - -0.27998567269576946, - 0.5668166088777099, - 0.2224894936706944, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "p2_n1", - "q": [ - 1.5286179915938458, - 0.39123391699293775, - 0.5349895313363059, - -1.8972686476502396, - -0.20874278682342615, - 2.270374859585192, - -0.8939703330265151 - ] - }, - { - "_id": "65f214fc983e0351740564c4", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - -0.030237889060397694, - 0.9941032227460016, - 0.10404421892826617, - 0.0, - 0.9990370460044222, - 0.03333771394241911, - -0.028183762010417793, - 0.0, - -0.031486771269573506, - 0.10309379653800622, - -0.9941731500852344, - 0.0, - -0.015942964370017125, - 0.4375202893228714, - 0.42062840472064134, - 1.0 - ], - "O_T_TCP": [ - -0.030237889060397694, - 0.9941032227460016, - 0.10404421892826617, - 0.0, - 0.9990370460044222, - 0.03333771394241911, - -0.028183762010417793, - 0.0, - -0.031486771269573506, - 0.10309379653800622, - -0.9941731500852344, - 0.0, - -0.015942964370017125, - 0.4375202893228714, - 0.42062840472064134, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "initialposition2", - "q": [ - 1.5506923031671005, - -0.4497734840543152, - 0.038535913673190224, - -2.3152158262018574, - 0.050607820335361695, - 1.9691166749798155, - 0.7517650229864651 - ] - }, - { - "_id": "65f374543949254a58053971", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.03173898612252293, - 0.9982326382781231, - 0.05004981056328121, - 0.0, - 0.9987842695688426, - -0.02980004258186478, - -0.03902162509636153, - 0.0, - -0.037461894541397284, - 0.05122845660945351, - -0.9979840939067026, - 0.0, - -0.2748815546694312, - 0.3324526946698264, - 0.19020321271820365, - 1.0 - ], - "O_T_TCP": [ - 0.03173898612252293, - 0.9982326382781231, - 0.05004981056328121, - 0.0, - 0.9987842695688426, - -0.02980004258186478, - -0.03902162509636153, - 0.0, - -0.037461894541397284, - 0.05122845660945351, - -0.9979840939067026, - 0.0, - -0.2748815546694312, - 0.3324526946698264, - 0.19020321271820365, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "sp_shaft2", - "q": [ - 2.3878911709999446, - -0.13898968152668084, - -0.1298367435116517, - -2.652961812604937, - -0.0403083813485689, - 2.578178659995397, - 1.5375320562521366 - ] - }, - { - "_id": "65f489ce00b781025b0db011", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.009966723549040366, - 0.9988357757215452, - 0.04699472801968561, - 0.0, - 0.999584303405681, - -0.00869749333183234, - -0.027135233478458594, - 0.0, - -0.026695419616810108, - 0.04724655148433873, - -0.9985264733317392, - 0.0, - -0.02021721107783165, - 0.5257402517302128, - 0.3253018605106694, - 1.0 - ], - "O_T_TCP": [ - 0.009966723549040366, - 0.9988357757215452, - 0.04699472801968561, - 0.0, - 0.999584303405681, - -0.00869749333183234, - -0.027135233478458594, - 0.0, - -0.026695419616810108, - 0.04724655148433873, - -0.9985264733317392, - 0.0, - -0.02021721107783165, - 0.5257402517302128, - 0.3253018605106694, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "gear1", - "q": [ - -1.2245379853384535, - 0.08946082408795829, - 2.8328730721891966, - -2.187969385201221, - -0.002710549400912387, - 2.1509781232674916, - 0.8342100091165182 - ] - }, - { - "_id": "65f489ce00b781025b0db012", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - -0.02405268971449932, - -0.999350109142552, - 0.026487249299911187, - 0.0, - -0.998520867937328, - 0.02530289761840895, - 0.0479227145204212, - 0.0, - -0.04856270912625561, - -0.025295888001814256, - -0.9984997653142029, - 0.0, - -0.29627286474447195, - 0.5402193926564312, - 0.24951815252804269, - 1.0 - ], - "O_T_TCP": [ - -0.02405268971449932, - -0.999350109142552, - 0.026487249299911187, - 0.0, - -0.998520867937328, - 0.02530289761840895, - 0.0479227145204212, - 0.0, - -0.04856270912625561, - -0.025295888001814256, - -0.9984997653142029, - 0.0, - -0.29627286474447195, - 0.5402193926564312, - 0.24951815252804269, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "inward_claw", - "q": [ - 1.1466555126372133, - 0.4911871555482781, - 1.0211879908664732, - -1.889103684814978, - -0.42553616185983023, - 2.12249806078275, - -1.560074370787497 - ] - }, - { - "_id": "65f48add00b781025b0db013", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - -0.5652402046656898, - 0.805912836269351, - 0.1760362413453204, - 0.0, - 0.8101766382360048, - 0.5825117783483679, - -0.0653803475307077, - 0.0, - -0.15523703410852738, - 0.10566688361643159, - -0.9822097397949072, - 0.0, - -0.2763305579573874, - 0.33279789980415175, - 0.3557425405921873, - 1.0 - ], - "O_T_TCP": [ - -0.5652402046656898, - 0.805912836269351, - 0.1760362413453204, - 0.0, - 0.8101766382360048, - 0.5825117783483679, - -0.0653803475307077, - 0.0, - -0.15523703410852738, - 0.10566688361643159, - -0.9822097397949072, - 0.0, - -0.2763305579573874, - 0.33279789980415175, - 0.3557425405921873, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "sp_shaft2_above", - "q": [ - 2.4060116416811943, - -0.4776864408442848, - -0.14955661941620338, - -2.5259720389105635, - -0.021758461500339422, - 2.2347033861374603, - 0.8756390861099728 - ] - }, - { - "_id": "65f5d17319ec67138e072821", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.8560476645714392, - 0.5060823187534547, - 0.10508962616794995, - 0.0, - 0.5072494013809845, - -0.8616144504353933, - 0.01730117050249211, - 0.0, - 0.09930446890095938, - 0.03849676454404434, - -0.9943121348831758, - 0.0, - 0.4977328298997675, - 0.2632478599241839, - 0.2862359205478977, - 1.0 - ], - "O_T_TCP": [ - 0.8560476645714392, - 0.5060823187534547, - 0.10508962616794995, - 0.0, - 0.5072494013809845, - -0.8616144504353933, - 0.01730117050249211, - 0.0, - 0.09930446890095938, - 0.03849676454404434, - -0.9943121348831758, - 0.0, - 0.4977328298997675, - 0.2632478599241839, - 0.2862359205478977, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "p2_sp1_above", - "q": [ - 0.529364852355879, - 0.0378608091928457, - -0.04017833637609318, - -2.1627976577105055, - -0.015072589963674544, - 2.3065233025810263, - 0.7512218856027597 - ] - }, - { - "_id": "65f623355dc5574d5609af31", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.9995210312027456, - -0.02108802959629215, - 0.022220488121427916, - 0.0, - -0.020154087917596285, - -0.9989277578944894, - -0.041447486646434044, - 0.0, - 0.023071152394727273, - 0.04098058992578088, - -0.9988935444663332, - 0.0, - 0.5558260099968472, - 0.35474503674495034, - 0.1698716055280708, - 1.0 - ], - "O_T_TCP": [ - 0.9995210312027456, - -0.02108802959629215, - 0.022220488121427916, - 0.0, - -0.020154087917596285, - -0.9989277578944894, - -0.041447486646434044, - 0.0, - 0.023071152394727273, - 0.04098058992578088, - -0.9988935444663332, - 0.0, - 0.5558260099968472, - 0.35474503674495034, - 0.1698716055280708, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "observe", - "q": [ - 0.24490834641150266, - 0.5219243896211131, - 0.35286033423301993, - -1.8072963120737229, - -0.20848363170680467, - 2.3329070408344266, - 1.4982907013541806 - ] - } -] \ No newline at end of file diff --git a/experiments/workstation/mios_memory_backups/p3.json b/experiments/workstation/mios_memory_backups/p3.json deleted file mode 100644 index 7e92e792..00000000 --- a/experiments/workstation/mios_memory_backups/p3.json +++ /dev/null @@ -1,1502 +0,0 @@ -[ - { - "_id": "65eb7417886cb04353024ad1", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "geometry": null, - "grasp_force": 1.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "TestObject1", - "q": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ] - }, - { - "_id": "65eb741d2c800a25d3000d41", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "NullObject", - "q": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ] - }, - { - "_id": "65eb741d2c800a25d3000d42", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "NoneObject", - "q": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ] - }, - { - "_id": "65eb741d2c800a25d3000d43", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.014200600442557191, - 0.999206966928041, - -0.03693950582701284, - 0.0, - 0.9997967988450969, - -0.013686440453118644, - 0.014134683159816981, - 0.0, - 0.013618165733630751, - -0.037133435598406074, - -0.9992175206243685, - 0.0, - -0.3061818124267212, - 0.1671510130147556, - 0.5738349259906845, - 1.0 - ], - "O_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "EndEffector", - "q": [ - 1.5189826725265736, - -0.8016634476811754, - 0.6073344519849408, - -1.9399075706548876, - 0.4520823260290046, - 1.2279291600391575, - 1.3417976757538272 - ] - }, - { - "_id": "65eb74332c800a25d3000d44", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - -0.9993938589627782, - -0.019959674635375253, - 0.02818284863431975, - 0.0, - -0.01984070888343409, - 0.9997834226459386, - 0.0044945494306065075, - 0.0, - -0.02826699883912122, - 0.00393273312179257, - -0.999592672235457, - 0.0, - -0.37484573960170936, - 0.3533845541917834, - 0.1748708579262803, - 1.0 - ], - "O_T_TCP": [ - -0.9993938589627782, - -0.019959674635375253, - 0.02818284863431975, - 0.0, - -0.01984070888343409, - 0.9997834226459386, - 0.0044945494306065075, - 0.0, - -0.02826699883912122, - 0.00393273312179257, - -0.999592672235457, - 0.0, - -0.37484573960170936, - 0.3533845541917834, - 0.1748708579262803, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "screwbolt", - "q": [ - 1.6550573774574837, - 0.15321099584385517, - 0.7222888105859068, - -2.3508161678625883, - -0.13811986689658284, - 2.484654771697428, - 0.10289976926765454 - ] - }, - { - "_id": "65eb74ac2c800a25d3000d45", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - -0.9754081093603869, - -0.21792603402093702, - 0.03268043437760813, - 0.0, - -0.21721197651678822, - 0.9758183257555578, - 0.024047853927084306, - 0.0, - -0.037131535089479584, - 0.016358204934740318, - -0.9991764900322725, - 0.0, - -0.49456060992516254, - 0.34523358993584274, - 0.18533646391614586, - 1.0 - ], - "O_T_TCP": [ - -0.9754081093603869, - -0.21792603402093702, - 0.03268043437760813, - 0.0, - -0.21721197651678822, - 0.9758183257555578, - 0.024047853927084306, - 0.0, - -0.037131535089479584, - 0.016358204934740318, - -0.9991764900322725, - 0.0, - -0.49456060992516254, - 0.34523358993584274, - 0.18533646391614586, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "screwhole", - "q": [ - 1.331823483964853, - 1.2933396335568335, - 1.6098604303502224, - -1.920375438578031, - -1.3170606204536117, - 1.6645034709950923, - 0.6755126488672281 - ] - }, - { - "_id": "65eb8ec1d3061a79e90e03d1", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - -1.0, - 0.0, - 0.0, - 0.0, - 0.0, - -1.0, - 0.0, - 0.521827226443675, - 0.25840698062581857, - 0.27517533664525384, - 1.0 - ], - "O_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - -1.0, - 0.0, - 0.0, - 0.0, - 0.0, - -1.0, - 0.0, - 0.521827226443675, - 0.25840698062581857, - 0.27517533664525384, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "test", - "q": [ - 0.18074043324946532, - 0.1335626147107068, - 0.2868349701521744, - -2.060772508320055, - -0.04825592930051324, - 2.2430690224108654, - 1.2386451794617466 - ] - }, - { - "_id": "65ef6331e99f2576ba086061", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - -0.9999511693327637, - 0.005795104725127399, - 0.006694969191221638, - 0.0, - 0.005676923956787325, - 0.9998204500065859, - -0.017538163650304785, - 0.0, - -0.00679553343966406, - -0.017499637341326747, - -0.9998237761816791, - 0.0, - -0.28525213823562634, - 0.5657042239474747, - 0.23866900068692073, - 1.0 - ], - "O_T_TCP": [ - -0.9999511693327637, - 0.005795104725127399, - 0.006694969191221638, - 0.0, - 0.005676923956787325, - 0.9998204500065859, - -0.017538163650304785, - 0.0, - -0.00679553343966406, - -0.017499637341326747, - -0.9998237761816791, - 0.0, - -0.28525213823562634, - 0.5657042239474747, - 0.23866900068692073, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "normalcontainer", - "q": [ - 1.48723348463888, - 0.4174561200767826, - 0.6111280596883836, - -1.8220793050632123, - -0.26309194494459087, - 2.141865217341235, - -0.1461896891569508 - ] - }, - { - "_id": "65ef6455e99f2576ba086062", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - -0.9990336274326229, - -0.043224038692152265, - 0.006651363706987428, - 0.0, - -0.04306611338611579, - 0.9988138548284101, - 0.022292155163513752, - 0.0, - -0.00760717766310412, - 0.02198458752415408, - -0.9997293677587924, - 0.0, - -0.2767815159120372, - 0.4389463906506993, - 0.3408536221713534, - 1.0 - ], - "O_T_TCP": [ - -0.9990336274326229, - -0.043224038692152265, - 0.006651363706987428, - 0.0, - -0.04306611338611579, - 0.9988138548284101, - 0.022292155163513752, - 0.0, - -0.00760717766310412, - 0.02198458752415408, - -0.9997293677587924, - 0.0, - -0.2767815159120372, - 0.4389463906506993, - 0.3408536221713534, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "normalpickplace", - "q": [ - 1.6710431030315733, - -0.11933810338041052, - 0.44976917909321673, - -2.1647275869469373, - 0.053616515815126296, - 2.079309125394282, - -0.3063374023692766 - ] - }, - { - "_id": "65f0c57f8fca00707f09dc71", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.9943836026315849, - -0.0968288341608145, - -0.042499112857760084, - 0.0, - -0.09580790301125691, - -0.9950650220230038, - 0.025440019028163983, - 0.0, - -0.04475356970019007, - -0.021225795551839743, - -0.9987725384702374, - 0.0, - 0.47926793066474926, - 0.23916614873158387, - -0.004146619879005292, - 1.0 - ], - "O_T_TCP": [ - 0.9943836026315849, - -0.0968288341608145, - -0.042499112857760084, - 0.0, - -0.09580790301125691, - -0.9950650220230038, - 0.025440019028163983, - 0.0, - -0.04475356970019007, - -0.021225795551839743, - -0.9987725384702374, - 0.0, - 0.47926793066474926, - 0.23916614873158387, - -0.004146619879005292, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "p2_sp", - "q": [ - 0.30748095076589516, - 0.6224689720722666, - 0.1322143079238894, - -2.174735181272256, - -0.1994334632820553, - 2.7361665597756706, - 1.4845305872601067 - ] - }, - { - "_id": "65f0c58a8fca00707f09dc72", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - -1.0, - 0.0, - 0.0, - 0.0, - 0.0, - -1.0, - 0.0, - 0.47926574591224724, - 0.2391667486795837, - 0.000851668509789333, - 1.0 - ], - "O_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - -1.0, - 0.0, - 0.0, - 0.0, - 0.0, - -1.0, - 0.0, - 0.47926574591224724, - 0.2391667486795837, - 0.000851668509789333, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "p2_sp1", - "q": [ - 0.3074809791130353, - 0.6224715037264296, - 0.1322163470370205, - -2.1747368392604187, - -0.1994324504037289, - 2.7361665597756706, - 1.4845236736801344 - ] - }, - { - "_id": "65f0c90302781731fa077fc1", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - -1.0, - 0.0, - 0.0, - 0.0, - 0.0, - -1.0, - 0.0, - 0.46427543988270176, - 0.09451115271991717, - 0.27356154422388645, - 1.0 - ], - "O_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - -1.0, - 0.0, - 0.0, - 0.0, - 0.0, - -1.0, - 0.0, - 0.46427543988270176, - 0.09451115271991717, - 0.27356154422388645, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "p3_sp1", - "q": [ - -0.15396518107882165, - -0.13305540426572163, - 0.35980236073607763, - -2.3169006355352564, - 0.009758118150134882, - 2.1126170528729755, - 0.5451880992129445 - ] - }, - { - "_id": "65f1f1ed2f5edeacb80ca502", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - -0.022428185668289153, - 0.7976012549648388, - 0.602751990015256, - 0.0, - 0.99870465727592, - -0.009541038811531106, - 0.04978677550306445, - 0.0, - 0.04546175003192464, - 0.6030994581888228, - -0.7963694323719259, - 0.0, - 0.021030588638465065, - 0.17098193114482207, - 0.815525967232651, - 1.0 - ], - "O_T_TCP": [ - -0.022428185668289153, - 0.7976012549648388, - 0.602751990015256, - 0.0, - 0.99870465727592, - -0.009541038811531106, - 0.04978677550306445, - 0.0, - 0.04546175003192464, - 0.6030994581888228, - -0.7963694323719259, - 0.0, - 0.021030588638465065, - 0.17098193114482207, - 0.815525967232651, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "initialposition3", - "q": [ - 1.5399258230464499, - -0.9537460691134678, - -0.017030666166464505, - -1.6540864067955072, - -0.032728623227940665, - 1.3499860814824516, - 0.7149023481880881 - ] - }, - { - "_id": "65f1f2e5cdb7d512fd0105f1", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.015122373673635682, - 0.9988427531215684, - -0.04544463942214867, - 0.0, - 0.999570559622232, - -0.013978767339836716, - 0.025377890010103748, - 0.0, - 0.024713737300809172, - -0.04580977957502074, - -0.9986444288553933, - 0.0, - -0.303945816124299, - 0.16206723765320832, - 0.42934613654633463, - 1.0 - ], - "O_T_TCP": [ - 0.015122373673635682, - 0.9988427531215684, - -0.04544463942214867, - 0.0, - 0.999570559622232, - -0.013978767339836716, - 0.025377890010103748, - 0.0, - 0.024713737300809172, - -0.04580977957502074, - -0.9986444288553933, - 0.0, - -0.303945816124299, - 0.16206723765320832, - 0.42934613654633463, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "gear2", - "q": [ - 1.2555071329677783, - -1.2271554659625938, - 0.8020651020845783, - -2.330712927422328, - 0.7589113699891958, - 1.2754252565701802, - 1.0481310031845252 - ] - }, - { - "_id": "65f1f339cdb7d512fd0105f2", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.048599880218834504, - 0.9987264876198011, - -0.01281411216349281, - 0.0, - 0.9986431913583882, - -0.048821364754247164, - -0.01757832751641594, - 0.0, - -0.018181893800433897, - -0.011942651186672752, - -0.9997633679128594, - 0.0, - -0.2760860562053573, - 0.4046761398389411, - 0.46807466166218, - 1.0 - ], - "O_T_TCP": [ - 0.048599880218834504, - 0.9987264876198011, - -0.01281411216349281, - 0.0, - 0.9986431913583882, - -0.048821364754247164, - -0.01757832751641594, - 0.0, - -0.018181893800433897, - -0.011942651186672752, - -0.9997633679128594, - 0.0, - -0.2760860562053573, - 0.4046761398389411, - 0.46807466166218, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "p3_n1", - "q": [ - 1.4354606213506897, - -0.28817165779230886, - 0.6296703716913858, - -1.95615939217283, - 0.1918240264256796, - 1.715982329527537, - 1.2800507355570263 - ] - } -] \ No newline at end of file diff --git a/experiments/workstation/mios_memory_backups/p4.json b/experiments/workstation/mios_memory_backups/p4.json deleted file mode 100644 index eaa90382..00000000 --- a/experiments/workstation/mios_memory_backups/p4.json +++ /dev/null @@ -1,1802 +0,0 @@ -[ - { - "_id": "65eb7417886cb04353024ad1", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "geometry": null, - "grasp_force": 1.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "TestObject1", - "q": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ] - }, - { - "_id": "65eb741d2c800a25d3000d41", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "NullObject", - "q": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ] - }, - { - "_id": "65eb741d2c800a25d3000d42", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "NoneObject", - "q": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ] - }, - { - "_id": "65eb741d2c800a25d3000d43", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - -0.9971671749247119, - -0.07408511682738264, - 0.012237958103812894, - 0.0, - -0.07422214305607265, - 0.997169749144124, - -0.011149518967940383, - 0.0, - -0.011377527249762645, - -0.012026493355390673, - -0.999862948274039, - 0.0, - -0.48432929787402185, - 0.2810334897138025, - 0.37223305950433444, - 1.0 - ], - "O_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "EndEffector", - "q": [ - 2.498910892391414, - 0.004927555259877824, - 0.1120868133818894, - -1.9471612110897965, - 0.016810300753069708, - 1.9557363281326794, - 0.17428947087924881 - ] - }, - { - "_id": "65eb74332c800a25d3000d44", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - -0.9993938589627782, - -0.019959674635375253, - 0.02818284863431975, - 0.0, - -0.01984070888343409, - 0.9997834226459386, - 0.0044945494306065075, - 0.0, - -0.02826699883912122, - 0.00393273312179257, - -0.999592672235457, - 0.0, - -0.37484573960170936, - 0.3533845541917834, - 0.1748708579262803, - 1.0 - ], - "O_T_TCP": [ - -0.9993938589627782, - -0.019959674635375253, - 0.02818284863431975, - 0.0, - -0.01984070888343409, - 0.9997834226459386, - 0.0044945494306065075, - 0.0, - -0.02826699883912122, - 0.00393273312179257, - -0.999592672235457, - 0.0, - -0.37484573960170936, - 0.3533845541917834, - 0.1748708579262803, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "screwbolt", - "q": [ - 1.6550573774574837, - 0.15321099584385517, - 0.7222888105859068, - -2.3508161678625883, - -0.13811986689658284, - 2.484654771697428, - 0.10289976926765454 - ] - }, - { - "_id": "65eb74ac2c800a25d3000d45", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - -0.9754081093603869, - -0.21792603402093702, - 0.03268043437760813, - 0.0, - -0.21721197651678822, - 0.9758183257555578, - 0.024047853927084306, - 0.0, - -0.037131535089479584, - 0.016358204934740318, - -0.9991764900322725, - 0.0, - -0.49456060992516254, - 0.34523358993584274, - 0.18533646391614586, - 1.0 - ], - "O_T_TCP": [ - -0.9754081093603869, - -0.21792603402093702, - 0.03268043437760813, - 0.0, - -0.21721197651678822, - 0.9758183257555578, - 0.024047853927084306, - 0.0, - -0.037131535089479584, - 0.016358204934740318, - -0.9991764900322725, - 0.0, - -0.49456060992516254, - 0.34523358993584274, - 0.18533646391614586, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "screwhole", - "q": [ - 1.331823483964853, - 1.2933396335568335, - 1.6098604303502224, - -1.920375438578031, - -1.3170606204536117, - 1.6645034709950923, - 0.6755126488672281 - ] - }, - { - "_id": "65eb8ec1d3061a79e90e03d1", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - -1.0, - 0.0, - 0.0, - 0.0, - 0.0, - -1.0, - 0.0, - 0.521827226443675, - 0.25840698062581857, - 0.27517533664525384, - 1.0 - ], - "O_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - -1.0, - 0.0, - 0.0, - 0.0, - 0.0, - -1.0, - 0.0, - 0.521827226443675, - 0.25840698062581857, - 0.27517533664525384, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "test", - "q": [ - 0.18074043324946532, - 0.1335626147107068, - 0.2868349701521744, - -2.060772508320055, - -0.04825592930051324, - 2.2430690224108654, - 1.2386451794617466 - ] - }, - { - "_id": "65ef6331e99f2576ba086061", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - -0.9999511693327637, - 0.005795104725127399, - 0.006694969191221638, - 0.0, - 0.005676923956787325, - 0.9998204500065859, - -0.017538163650304785, - 0.0, - -0.00679553343966406, - -0.017499637341326747, - -0.9998237761816791, - 0.0, - -0.28525213823562634, - 0.5657042239474747, - 0.23866900068692073, - 1.0 - ], - "O_T_TCP": [ - -0.9999511693327637, - 0.005795104725127399, - 0.006694969191221638, - 0.0, - 0.005676923956787325, - 0.9998204500065859, - -0.017538163650304785, - 0.0, - -0.00679553343966406, - -0.017499637341326747, - -0.9998237761816791, - 0.0, - -0.28525213823562634, - 0.5657042239474747, - 0.23866900068692073, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "normalcontainer", - "q": [ - 1.48723348463888, - 0.4174561200767826, - 0.6111280596883836, - -1.8220793050632123, - -0.26309194494459087, - 2.141865217341235, - -0.1461896891569508 - ] - }, - { - "_id": "65ef6455e99f2576ba086062", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - -0.9990336274326229, - -0.043224038692152265, - 0.006651363706987428, - 0.0, - -0.04306611338611579, - 0.9988138548284101, - 0.022292155163513752, - 0.0, - -0.00760717766310412, - 0.02198458752415408, - -0.9997293677587924, - 0.0, - -0.2767815159120372, - 0.4389463906506993, - 0.3408536221713534, - 1.0 - ], - "O_T_TCP": [ - -0.9990336274326229, - -0.043224038692152265, - 0.006651363706987428, - 0.0, - -0.04306611338611579, - 0.9988138548284101, - 0.022292155163513752, - 0.0, - -0.00760717766310412, - 0.02198458752415408, - -0.9997293677587924, - 0.0, - -0.2767815159120372, - 0.4389463906506993, - 0.3408536221713534, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "normalpickplace", - "q": [ - 1.6710431030315733, - -0.11933810338041052, - 0.44976917909321673, - -2.1647275869469373, - 0.053616515815126296, - 2.079309125394282, - -0.3063374023692766 - ] - }, - { - "_id": "65f0c57f8fca00707f09dc71", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.9943836026315849, - -0.0968288341608145, - -0.042499112857760084, - 0.0, - -0.09580790301125691, - -0.9950650220230038, - 0.025440019028163983, - 0.0, - -0.04475356970019007, - -0.021225795551839743, - -0.9987725384702374, - 0.0, - 0.47926793066474926, - 0.23916614873158387, - -0.004146619879005292, - 1.0 - ], - "O_T_TCP": [ - 0.9943836026315849, - -0.0968288341608145, - -0.042499112857760084, - 0.0, - -0.09580790301125691, - -0.9950650220230038, - 0.025440019028163983, - 0.0, - -0.04475356970019007, - -0.021225795551839743, - -0.9987725384702374, - 0.0, - 0.47926793066474926, - 0.23916614873158387, - -0.004146619879005292, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "p2_sp", - "q": [ - 0.30748095076589516, - 0.6224689720722666, - 0.1322143079238894, - -2.174735181272256, - -0.1994334632820553, - 2.7361665597756706, - 1.4845305872601067 - ] - }, - { - "_id": "65f0c58a8fca00707f09dc72", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - -1.0, - 0.0, - 0.0, - 0.0, - 0.0, - -1.0, - 0.0, - 0.47926574591224724, - 0.2391667486795837, - 0.000851668509789333, - 1.0 - ], - "O_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - -1.0, - 0.0, - 0.0, - 0.0, - 0.0, - -1.0, - 0.0, - 0.47926574591224724, - 0.2391667486795837, - 0.000851668509789333, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "p2_sp1", - "q": [ - 0.3074809791130353, - 0.6224715037264296, - 0.1322163470370205, - -2.1747368392604187, - -0.1994324504037289, - 2.7361665597756706, - 1.4845236736801344 - ] - }, - { - "_id": "65f0c90302781731fa077fc1", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - -1.0, - 0.0, - 0.0, - 0.0, - 0.0, - -1.0, - 0.0, - 0.46427543988270176, - 0.09451115271991717, - 0.27356154422388645, - 1.0 - ], - "O_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - -1.0, - 0.0, - 0.0, - 0.0, - 0.0, - -1.0, - 0.0, - 0.46427543988270176, - 0.09451115271991717, - 0.27356154422388645, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "p3_sp1", - "q": [ - -0.15396518107882165, - -0.13305540426572163, - 0.35980236073607763, - -2.3169006355352564, - 0.009758118150134882, - 2.1126170528729755, - 0.5451880992129445 - ] - }, - { - "_id": "65f0d3e8fb84cf2a120e3631", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - -1.0, - 0.0, - 0.0, - 0.0, - 0.0, - -1.0, - 0.0, - 0.6880759848080192, - 0.23697792849771698, - 0.24691212085827882, - 1.0 - ], - "O_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - -1.0, - 0.0, - 0.0, - 0.0, - 0.0, - -1.0, - 0.0, - 0.6880759848080192, - 0.23697792849771698, - 0.24691212085827882, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "p4_sp1", - "q": [ - 0.28226058822451977, - 0.6121795218116358, - 0.07728357154549213, - -1.405678696857797, - -0.09612049579315013, - 2.0572903097410777, - 0.772790184580692 - ] - }, - { - "_id": "65f0d464fb84cf2a120e3632", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - -1.0, - 0.0, - 0.0, - 0.0, - 0.0, - -1.0, - 0.0, - 0.5753072320945516, - 0.35772376513316784, - 0.22929512401381655, - 1.0 - ], - "O_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - -1.0, - 0.0, - 0.0, - 0.0, - 0.0, - -1.0, - 0.0, - 0.5753072320945516, - 0.35772376513316784, - 0.22929512401381655, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "p4_sp2", - "q": [ - 0.5236941784766681, - 0.46791346897149877, - 0.052360733283235594, - -1.6832814310630315, - -0.0989316663477156, - 2.1822466918627415, - 1.325044317010376 - ] - }, - { - "_id": "65f32b512aae7f1d4308a7b1", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.08009874125275696, - 0.9678025193124069, - 0.23858588005892276, - 0.0, - 0.9965866221100188, - -0.07307428233946373, - -0.03815757844941213, - 0.0, - -0.01949488392949292, - 0.24083250706616996, - -0.970370884270957, - 0.0, - 0.011456568303596941, - 0.3093701710869116, - 0.6560632707334411, - 1.0 - ], - "O_T_TCP": [ - 0.08009874125275696, - 0.9678025193124069, - 0.23858588005892276, - 0.0, - 0.9965866221100188, - -0.07307428233946373, - -0.03815757844941213, - 0.0, - -0.01949488392949292, - 0.24083250706616996, - -0.970370884270957, - 0.0, - 0.011456568303596941, - 0.3093701710869116, - 0.6560632707334411, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "initialposition4", - "q": [ - 1.5685654488138667, - -0.744127185242312, - -0.02750540666985012, - -1.9386031585457497, - 0.006881550318673541, - 1.4373830195524373, - 0.8402223529073997 - ] - }, - { - "_id": "65f32c042aae7f1d4308a7b2", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - -0.9973517269792552, - -0.07182856724856343, - 0.010532640681439017, - 0.0, - -0.0718743913004388, - 0.9973958653326211, - -0.0040381423688853335, - 0.0, - -0.01021535496323699, - -0.0047845675209116844, - -0.9999363751942486, - 0.0, - -0.4845312667796605, - 0.2787442395474375, - 0.3655360680522394, - 1.0 - ], - "O_T_TCP": [ - -0.9973517269792552, - -0.07182856724856343, - 0.010532640681439017, - 0.0, - -0.0718743913004388, - 0.9973958653326211, - -0.0040381423688853335, - 0.0, - -0.01021535496323699, - -0.0047845675209116844, - -0.9999363751942486, - 0.0, - -0.4845312667796605, - 0.2787442395474375, - 0.3655360680522394, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "gear1", - "q": [ - 1.7656999614621554, - 0.00611851440141337, - 0.8526876161809552, - -1.9661423878291835, - 0.005051759417305576, - 1.9766216791470843, - 0.18827315614486706 - ] - }, - { - "_id": "65f32d513d9b116ec2020011", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - -0.45572411600412527, - 0.8900954860745709, - -0.0051286122041071976, - 0.0, - 0.890062684780662, - 0.4557517643247159, - 0.007713195003142733, - 0.0, - 0.009203031302940302, - -0.0010497175835281363, - -0.999957100233721, - 0.0, - -0.3067364005816951, - 0.21929096272579776, - 0.41811326749729216, - 1.0 - ], - "O_T_TCP": [ - -0.45572411600412527, - 0.8900954860745709, - -0.0051286122041071976, - 0.0, - 0.890062684780662, - 0.4557517643247159, - 0.007713195003142733, - 0.0, - 0.009203031302940302, - -0.0010497175835281363, - -0.999957100233721, - 0.0, - -0.3067364005816951, - 0.21929096272579776, - 0.41811326749729216, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "p4_n1", - "q": [ - 1.765150419290651, - -0.7128181948715463, - 0.5863144904939752, - -2.373621218649856, - 0.36856206523047547, - 1.7312944560050965, - 0.9080589480610101 - ] - }, - { - "_id": "65f32e523d9b116ec2020012", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - -0.5795607653930256, - 0.8144478085288553, - 0.027655619696610772, - 0.0, - 0.812663200497495, - 0.5801471497752977, - -0.05466766929342204, - 0.0, - -0.06056945854543529, - -0.009208709139745983, - -0.9981215058135419, - 0.0, - -0.3018067869677364, - 0.21347095602128882, - 0.5044969485843687, - 1.0 - ], - "O_T_TCP": [ - -0.5795607653930256, - 0.8144478085288553, - 0.027655619696610772, - 0.0, - 0.812663200497495, - 0.5801471497752977, - -0.05466766929342204, - 0.0, - -0.06056945854543529, - -0.009208709139745983, - -0.9981215058135419, - 0.0, - -0.3018067869677364, - 0.21347095602128882, - 0.5044969485843687, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "p4_n1_above", - "q": [ - 2.367753699133986, - -0.6141025635759121, - 0.09455957488009804, - -2.20885624538338, - 0.10054808651036064, - 1.6372959554981268, - 1.0369381578692933 - ] - } -] \ No newline at end of file diff --git a/experiments/workstation/plan_parser.py b/experiments/workstation/plan_parser.py deleted file mode 100644 index 905108e9..00000000 --- a/experiments/workstation/plan_parser.py +++ /dev/null @@ -1,103 +0,0 @@ -""" -not BB-written -""" - -from pprint import pprint -import re - - -class PlanStep: - def __init__( - self, - start_time: float, - estimated_duration: float, - action_name: str, - action_params: list[str], - raw_action: str, - ): - self.start_time = start_time - self.estimated_duration = estimated_duration - self.estimated_end_time = start_time + estimated_duration - self.action_name = action_name - self.action_params = action_params - self.raw_action = raw_action - - def __repr__(self): - return f"PlanStep({self.action_name}, {self.action_params}, raw is {self.raw_action})" - - -def parse_plan(plan_str: str) -> list[PlanStep]: - plan_steps = [] - # Split the input string into lines - lines = plan_str.strip().split("\n") - for line in lines: - matches = re.findall(r"\((.*?)\)", line) - # pprint(matches) - action_details = matches[0].split(" ") - action_name = action_details[0] - action_params = action_details[1:] - - # Create a PlanStep instance for each line and append it to the plan_steps list - plan_step = PlanStep(0, 0, action_name, action_params, raw_action=line) - plan_steps.append(plan_step) - - return plan_steps - - -# Example usage with your provided plan -plan_str = """0.0003: (change_gripper defaulgripper outwardgripper) [d:10.0000; c:1.0000]\n10.0005: (initialize outwardgripper) [d:10.0000; c:1.0000]\n20.0007: (insert outwardgripper outerring housing) [d:23.0000; c:1.0000]\n43.0010: (insert outwardgripper cone outerring) [d:19.0000; c:1.0000]\n62.0013: (change_gripper outwardgripper defaulgripper) [d:10.0000; c:1.0000]\n72.0015: (initialize defaulgripper) [d:1.0000; c:1.0000]\n73.0017: (insert defaulgripper outputshaftandgearstage2 cone) [d:40.0000; c:1.0000]\n113.0020: (change_gripper defaulgripper parallelgripper) [d:10.0000; c:1.0000]\n123.0023: (initialize parallelgripper) [d:10.0000; c:1.0000]\n133.0025: (insert parallelgripper ringgear outputshaftandgearstage2) [d:25.0000; c:1.0000]\n158.0027: (change_gripper parallelgripper inwardgripper) [d:25.0000; c:1.0000]\n183.0030: (initialize inwardgripper) [d:10.0000; c:1.0000]\n193.0033: (insert inwardgripper gearstage1 ringgear) [d:25.0000; c:1.0000]\n218.0035: (change_gripper inwardgripper clampgripper) [d:30.0000; c:1.0000]\n248.0038: (initialize clampgripper) [d:10.0000; c:1.0000]\n258.0040: (insert clampgripper designring ringgear) [d:13.0000; c:1.0000]\n271.0042: (change_gripper clampgripper defaulgripper) [d:15.0000; c:1.0000]\n286.0045: (initialize defaulgripper) [d:1.0000; c:1.0000]\n287.0048: (insert defaulgripper driveflange gearstage1) [d:25.0000; c:1.0000]\n312.0050: (insert defaulgripper inputpinion driveflange) [d:25.0000; c:1.0000]""" - -# """0.000: (change_gripper defaulgripper outwardgripper) [10.000] -# 10.001: (insert outwardgripper outerring housing) [23.000] -# 33.002: (insert outwardgripper cone outerring) [19.000] -# 52.003: (change_gripper outwardgripper defaulgripper) [10.000] -# 62.004: (insert defaulgripper outputshaftandgearstage2 cone) [40.000] -# 102.005: (insert defaulgripper ringgear outputshaftandgearstage2) [12.000] -# 114.006: (insert defaulgripper designring ringgear) [10.000] -# 124.007: (change_gripper defaulgripper parallelgripper) [10.000] -# 134.008: (insert parallelgripper gearstage1 ringgear) [25.000] -# 159.009: (change_gripper parallelgripper outwardgripper) [20.000] -# 179.010: (insert outwardgripper driveflange ringgear) [25.000] -# 204.011: (change_gripper outwardgripper clampgripper) [25.000] -# 229.012: (insert clampgripper inputpinion driveflange) [13.000]""" - -""" -object list -defaultgripper -outwardgripper -outerring housingringhole -cone outerring -defaultgripper -outputshaftandgearstage2 housinginternalgear -ringgear outputshaftandgearstage2 #! -designring ringgear #? -parallelgripper -gearstage1 ringgear #! -driveflange ringgear #? -clampgripper -inputpinion driveflange #! - -""" - -""" -change_gripper defaulgripper outwardgripper -insert outwardgripper outerring [housing_ringhole] -insert outwardgripper1 cone1 outerring1 #? -change_gripper outwardgripper1 defaulgripper1 -insert defaulgripper1 outputshaftandgearstage21 housing_internalgear1 -insert defaulgripper1 ringgear1 outputshaftandgearstage21 #? -insert defaulgripper1 designring1 ringgear1 #? -change_gripper defaulgripper1 parallelgripper1 -insert parallelgripper1 gearstage11 ringgear1 #? -change_gripper parallelgripper1 outwardgripper1 -insert outwardgripper1 driveflange1 gearstage11 #! -change_gripper outwardgripper1 clampgripper1 -insert clampgripper1 inputpinion1 driveflange1 #? -""" - -parsed_plan = parse_plan(plan_str) - -# Display the parsed plan steps -# for step in parsed_plan: -# pprint(step) -# pprint(parsed_plan) diff --git a/experiments/workstation/robot_commands.py b/experiments/workstation/robot_commands.py deleted file mode 100644 index 883f21e5..00000000 --- a/experiments/workstation/robot_commands.py +++ /dev/null @@ -1,267 +0,0 @@ -import numpy as np -import os - -from kios_robot.mios_task_factory import MiosTaskFactory -from kios_robot.robot_command import RobotCommand -from kios_robot.robot_interface import RobotInterface -from kios_robot.data_types import Toolbox -from kios_scene.scene_factory import SceneFactory - -from kios_scene.mios_ltm_manipulator import LangTermMemoryManipulator - -ri = RobotInterface() -sf = SceneFactory() - -current_dir = os.path.dirname(os.path.realpath(__file__)) -scene_file = os.path.join(current_dir, "scene.json") - -with open(scene_file, "r") as f: - scene_json = f.read() - -scene = sf.create_scene_from_json(scene_json) -ri.setup_scene(scene) - -ltm_manipulator = LangTermMemoryManipulator() - - -# def teach_object(object: str): -# ri.proprioceptor.teach_object(object) -def backup_mios_environment(backup_name: str): - ltm_manipulator.backup_mios_environment(backup_name) - - -def clear_mios_environment(): - ltm_manipulator.clear_mios_environment() - - -def show_backups(): - ltm_manipulator.show_backups() - - -# ! BUG -def restore_to_mios_environment(backup_name: str): - ltm_manipulator.restore_to_mios_environment(backup_name) - - -def teach_object_TCP(object_name: str): - ri.proprioceptor.teach_object_TCP(object_name) - ri.proprioceptor.update_scene_object_from_mios(scene=scene, object_name=object_name) - - -def get_object(object: str): - ri.proprioceptor.get_object(object) - - -def set_tool(toolbox_name: str): - if "robot_command" in locals(): - print("robot command is set") - print(f"the robot command's tasks are {robot_command.task_list}") - else: - print("robot command is not set") - - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - shared_data=None, - task_scene=scene, - robot_interface=ri, - ) - print(f"the number of tasks in the task list is {len(robot_command.task_list)}") - robot_command.add_task(ri.mios_task_factory.generate_update_tool_call(toolbox_name)) - robot_command.execute_task_list_sync() - robot_command.clear_tasks() - - -def test_screw_in(object_name: str): - # need object "test" - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - shared_data=None, - task_scene=scene, - robot_interface=ri, - ) - - O_T_OB = scene.get_object(object_name).O_T_TCP - - print(O_T_OB) - - robot_command.add_task(ri.mios_task_factory.generate_screw_in_mp(O_T_OB=O_T_OB)) - - robot_command.execute_task_list_sync() - - -def pick_test(): - - parsed_action = { - "action_name": "pick", - "args": [None, None, "test_object"], - } - - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - shared_data=None, - task_scene=scene, - robot_interface=ri, - ) - - robot_command.add_mios_task(ri.mios_task_factory.generate_gripper_home_mp()) - - pick_up_tasks = ri.mios_task_factory.generate_pick_up_skill( - parsed_action=parsed_action - ) - for task in pick_up_tasks: - robot_command.add_task(task) - - robot_command.task_list.append(ri.mios_task_factory.generate_gripper_release_mp()) - - robot_command.execute_task_list_sync() - - -def move_gripper(width: float): - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - task_scene=scene, - shared_data=None, - robot_interface=ri, - ) - - robot_command.add_task(ri.mios_task_factory.generate_gripper_move_mp(width=width)) - - robot_command.execute_task_list_sync() - - -def home_gripper(): - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - task_scene=scene, - shared_data=None, - robot_interface=ri, - ) - - robot_command.add_task(ri.mios_task_factory.generate_gripper_home_mp()) - - robot_command.execute_task_list_sync() - - -def cartesian_move(object: str): - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - task_scene=scene, - shared_data=None, - robot_interface=ri, - ) - - robot_command.add_task(ri.mios_task_factory.generate_cartesian_move_mp(object)) - - robot_command.execute_task_list_sync() - - -def insert_test(object_name: str): - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - task_scene=scene, - shared_data=None, - robot_interface=ri, - ) - - insert_action = { - "action_name": "insert", - "args": [None, None, "gear3", object_name], - } - - robot_command.add_tasks(ri.mios_task_factory.generate_insert_skill(insert_action)) - - robot_command.execute_task_list_sync() - - -def drive_test(): - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - task_scene=scene, - shared_data=None, - robot_interface=ri, - ) - - drive_action = { - "action_name": "drive", - "args": [None, None, None, None], - } - - robot_command.add_tasks(ri.mios_task_factory.generate_drive_skill(drive_action)) - - robot_command.execute_task_list_sync() - - -def change_gripper(current_tool: str, new_tool: str): - - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - task_scene=scene, - shared_data=None, - robot_interface=ri, - ) - - unload_tool_parsed_action = { - "action_name": "unload_tool", - "args": ["left_hand", current_tool], - } - - robot_command.add_tasks( - ri.mios_task_factory.generate_unload_tool_skill(unload_tool_parsed_action) - ) - - load_tool_parsed_action = { - "action_name": "load_tool", - "args": ["left_hand", new_tool], - } - - robot_command.add_tasks( - ri.mios_task_factory.generate_load_tool_skill(load_tool_parsed_action) - ) - - robot_command.execute_task_list_sync() - - -def insert_task(part1: str, part2: str): - """ - part1: the location name of the part to be inserted - part2: the location name of the part (usually a feature name) to be inserted into - """ - # * first pickup, then insert - - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - shared_data=None, - task_scene=scene, - robot_interface=ri, - ) - - pick_up_parsed_action = { - "action_name": "pick", # ! maybe this should be "pick_up" - "args": [None, None, part1], - } - - robot_command.add_tasks( - ri.mios_task_factory.generate_pick_up_skill(pick_up_parsed_action) - ) - - insert_parsed_action = { - "action_name": "insert", - "args": [None, None, part1, part2], - } - - # ! warning: the location of part1 will be refreshed after the insertion! - robot_command.add_tasks( - ri.mios_task_factory.generate_insert_skill(insert_parsed_action) - ) - - robot_command.execute_task_list_sync() diff --git a/experiments/workstation/runtime_script.py b/experiments/workstation/runtime_script.py deleted file mode 100644 index 2416eca2..00000000 --- a/experiments/workstation/runtime_script.py +++ /dev/null @@ -1,1335 +0,0 @@ -import numpy as np -import os -import json -from spatialmath import * -from spatialmath.base import trnorm -from pprint import pprint - -from kios_robot.mios_task_factory import MiosTaskFactory -from kios_robot.robot_command import RobotCommand -from kios_robot.robot_interface import RobotInterface -from kios_robot.data_types import Toolbox -from kios_scene.scene_factory import SceneFactory - -from kios_scene.mios_ltm_manipulator import LangTermMemoryManipulator -from kios_utils.bblab_utils import execution_timer -import socket -from anran_tag.apriltag_rs import apriltag_rs -import threading - -ri = RobotInterface() -sf = SceneFactory() - -current_dir = os.path.dirname(os.path.realpath(__file__)) -scene_file = os.path.join(current_dir, "scene.json") - -with open(scene_file, "r") as f: - scene_string = f.read() - scene_json = json.loads(scene_string) - -scene = sf.create_scene_from_json(scene_json) -ri.setup_scene(scene) - -ltm_manipulator = LangTermMemoryManipulator() - - -def show_scene(): - pprint(scene) - - -# def teach_object(object: str): -# ri.proprioceptor.teach_object(object) -def backup_mios_environment(backup_name: str): - ltm_manipulator.backup_mios_environment(backup_name) - - -def clear_mios_environment(): - ltm_manipulator.clear_mios_environment() - - -def show_backups(): - ltm_manipulator.show_backups() - - -def restore_to_mios_environment(backup_name: str): - ltm_manipulator.restore_to_mios_environment(backup_name) - # ! you need to update the memory of mios. - update_mios_memory_environment() - # ! you also need to update the scene object in robot_interface. - scene = sf.create_scene_from_json(scene_json) - ri.setup_scene(scene) - - -def teach_object_TCP(object_name: str): - ri.proprioceptor.teach_object_TCP(object_name) - ri.proprioceptor.update_scene_object_from_mios(scene=scene, object_name=object_name) - - -def align_object(object_name: str, **kwargs): - ri.proprioceptor.align_object(object_name, **kwargs) - ri.proprioceptor.update_scene_object_from_mios(scene=scene, object_name=object_name) - - -def modify_object_position_rel(object_name: str, **kwargs): - ri.proprioceptor.modify_object_position(object_name, **kwargs) - ri.proprioceptor.update_scene_object_from_mios(scene=scene, object_name=object_name) - - -def update_mios_memory_environment(): - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - shared_data=None, - task_scene=scene, - robot_interface=ri, - ) - robot_command.add_task( - ri.mios_task_factory.generate_update_mios_memory_environment_call() - ) - robot_command.execute_task_list_sync() - - -def get_object(object: str): - return scene.get_object(object) - - -def set_tool(toolbox_name: str): - if "robot_command" in locals(): - print("robot command is set") - print(f"the robot command's tasks are {robot_command.task_list}") - else: - print("robot command is not set") - - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - shared_data=None, - task_scene=scene, - robot_interface=ri, - ) - print(f"the number of tasks in the task list is {len(robot_command.task_list)}") - robot_command.add_task(ri.mios_task_factory.generate_update_tool_call(toolbox_name)) - robot_command.execute_task_list_sync() - robot_command.clear_tasks() - - -def test_cartesian_joint_move(): - # need object "test" - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - shared_data=None, - task_scene=scene, - robot_interface=ri, - ) - - O_T_TCP = scene.get_object("test").O_T_TCP - print(O_T_TCP) - print(type(O_T_TCP)) - print(O_T_TCP.shape) - - robot_command.add_task( - ri.mios_task_factory.generate_cartesian_move_mp(O_T_TCP=O_T_TCP) - ) - - # rotate in z axis by 90 degrees - # rotation_90 = SE3.Rz(np.deg2rad(90)) - - # rotate in x axis by 90 degrees - rotation_90 = SE3.Rx(np.deg2rad(90)) - - se_O_T_TCP = SE3(trnorm(O_T_TCP)) - result = se_O_T_TCP * rotation_90 - O_T_TCP = result.A - - robot_command.add_task( - ri.mios_task_factory.generate_cartesian_move_mp(O_T_TCP=O_T_TCP) - ) - - robot_command.execute_task_list_sync() - - -def test_screw_in(object_name: str): - # need object "test" - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - shared_data=None, - task_scene=scene, - robot_interface=ri, - ) - - O_T_OB = scene.get_object(object_name).O_T_TCP - - print(O_T_OB) - - robot_command.add_task(ri.mios_task_factory.generate_screw_in_mp(O_T_OB=O_T_OB)) - - robot_command.execute_task_list_sync() - - -def tool_test(): - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - shared_data=None, - task_scene=scene, - robot_interface=ri, - ) - - robot_command.add_mios_task( - ri.mios_task_factory.generate_cartesian_move_mp("test_location") - ) - robot_command.add_mios_task( - ri.mios_task_factory.generate_update_tool_call("parallel_box1") - ) - robot_command.add_mios_task( - ri.mios_task_factory.generate_cartesian_move_mp("test_location") - ) - - robot_command.execute_task_list_sync() - - -def load_tool_test(tool_name: str): - parsed_action = { - "action_name": "load_tool", - "args": ["left_hand", tool_name], - } - - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - task_scene=scene, - shared_data=None, - robot_interface=ri, - ) - - robot_command.add_tasks( - ri.mios_task_factory.generate_load_tool_skill(parsed_action) - ) - - robot_command.execute_task_list_sync() - - -def pick_test(object_name: str): - - parsed_action = { - "action_name": "pick", - "args": [None, None, object_name], - } - - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - shared_data=None, - task_scene=scene, - robot_interface=ri, - ) - - robot_command.add_tasks(ri.mios_task_factory.generate_pick_up_skill(parsed_action)) - - robot_command.execute_task_list_sync() - - -def grasp(): - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - task_scene=scene, - shared_data=None, - robot_interface=ri, - ) - - robot_command.add_task(ri.mios_task_factory.generate_gripper_grasp_mp()) - - robot_command.execute_task_list_sync() - - -def move_gripper(width: float): - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - task_scene=scene, - shared_data=None, - robot_interface=ri, - ) - - robot_command.add_task(ri.mios_task_factory.generate_gripper_move_mp(width=width)) - - robot_command.execute_task_list_sync() - - -def home_gripper(): - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - task_scene=scene, - shared_data=None, - robot_interface=ri, - ) - - robot_command.add_task(ri.mios_task_factory.generate_gripper_home_mp()) - - robot_command.execute_task_list_sync() - - -def cartesian_move(object: str): - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - task_scene=scene, - shared_data=None, - robot_interface=ri, - ) - - robot_command.add_task(ri.mios_task_factory.generate_cartesian_move_mp(object)) - - robot_command.execute_task_list_sync() - - -def get_mark(): - UDP_IP = "127.0.0.1" - UDP_PORT = 8888 - sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) - - sock.bind((UDP_IP, UDP_PORT)) - while True: - data, addr = sock.recvfrom(1024) - - if data: - array = np.frombuffer(data, dtype=np.float64).reshape( - (4, 4) - ) # Adjust dtype if necessary - return array - - -def cartesian_move_T(): - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - task_scene=scene, - shared_data=None, - robot_interface=ri, - ) - robot_state = ri.proprioceptor.get_robot_state() - - EE_T_cam = np.array( - [ - [0.0, -1.0, 0.0, 0.0514976], - [1.0, 0.0, 0.0, -0.0375164], - [0.0, 0.0, 1.0, -0.0467639], - [0.0, 0.0, 0.0, 1.0], - ] - ) - - thread = threading.Thread(target=apriltag_rs) - thread.start() - - cam_T_mark = get_mark() - - O_T_obj = robot_state.O_T_TCP @ EE_T_cam @ cam_T_mark - - O_T_TCP_g = robot_state.O_T_TCP - - O_T_TCP_g[0][3] = O_T_obj[0][3] - O_T_TCP_g[1][3] = O_T_obj[1][3] - - robot_command.add_task( - ri.mios_task_factory.generate_cartesian_move_mp( - O_T_TCP=O_T_TCP_g, - ), - ) - robot_command.execute_task_list_sync() - - thread.join() - - -def joint_move(object: str): - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - task_scene=scene, - shared_data=None, - robot_interface=ri, - ) - - robot_command.add_task(ri.mios_task_factory.generate_joint_move_mp(object)) - - robot_command.execute_task_list_sync() - - -def insert_test(object_name: str): - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - task_scene=scene, - shared_data=None, - robot_interface=ri, - ) - - insert_action = { - "action_name": "insert", - "args": [None, None, "gear3", object_name], - } - - robot_command.add_tasks(ri.mios_task_factory.generate_insert_skill(insert_action)) - - robot_command.execute_task_list_sync() - - -def drive_test(): - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - task_scene=scene, - shared_data=None, - robot_interface=ri, - ) - - drive_action = { - "action_name": "drive", - "args": [None, None, None, None], - } - - robot_command.add_tasks(ri.mios_task_factory.generate_drive_skill(drive_action)) - - robot_command.execute_task_list_sync() - - -@execution_timer -def change_gripper(current_tool: str, new_tool: str): - - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - task_scene=scene, - shared_data=None, - robot_interface=ri, - ) - - if current_tool != "defaultgripper": - unload_tool_parsed_action = { - "action_name": "unload_tool", - "args": ["left_hand", current_tool], - } - - robot_command.add_tasks( - ri.mios_task_factory.generate_unload_tool_skill(unload_tool_parsed_action) - ) - - if new_tool != "defaultgripper": - - load_tool_parsed_action = { - "action_name": "load_tool", - "args": ["left_hand", new_tool], - } - - robot_command.add_tasks( - ri.mios_task_factory.generate_load_tool_skill(load_tool_parsed_action) - ) - - result = robot_command.execute_task_list_sync() - - if result == False: - pause = input("failed to change gripper") - - -@execution_timer -def initialize_gripper(): - - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - shared_data=None, - task_scene=scene, - robot_interface=ri, - ) - - robot_command.add_task( - ri.mios_task_factory.generate_joint_move_mp("initialposition") - ) - - robot_command.execute_task_list_sync() - - -@execution_timer -def insert_task(part1: str, part2: str): - """ - part1: the location name of the part to be inserted - part2: the location name of the part (usually a feature name) to be inserted into - """ - # * first pickup, then insert - - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - shared_data=None, - task_scene=scene, - robot_interface=ri, - ) - - pick_up_parsed_action = { - "action_name": "pick", # ! maybe this should be "pick_up" - "args": [None, None, part1], - } - - robot_command.add_tasks( - ri.mios_task_factory.generate_pick_up_skill(pick_up_parsed_action) - ) - - insert_parsed_action = { - "action_name": "insert", - "args": [None, None, part1, part2], - } - - # ! warning: the location of part1 will be refreshed after the insertion! - robot_command.add_tasks( - ri.mios_task_factory.generate_insert_skill_mod(insert_parsed_action) - ) - - robot_command.execute_task_list_sync() - - -def insert_test(location: str): - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - shared_data=None, - task_scene=scene, - robot_interface=ri, - ) - param = { - "search_a": [3, 3, 0, 0, 0, 0], - "search_f": [1, 1, 0, 0, 0, 0], - "search_phi": [ - 0, - 3.14159265358979323846 / 2, - 0, - 3.14159265358979323846 / 2, - 0, - 3.14159265358979323846 / 2, - ], - "F_ext_contact": [8.0, 2.0], - "f_push": [0, 0, 8, 0, 0, 0], - "K_x": [150, 150, 0, 150, 150, 0], - "env_X": [0.01, 0.01, -0.004, 0.05, 0.05, 0.05], - "D_x": [0.7, 0.7, 0, 0.7, 0.7, 1.4], - } - - robot_command.add_task( - ri.mios_task_factory.generate_insert_mp( - insertable="normalbolt", - container=location, - param=param, - ) - ) - - robot_command.execute_task_list_sync() - - -@execution_timer -def test_drive_in_mp(container: str = "normalcontainer"): - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - shared_data=None, - task_scene=scene, - robot_interface=ri, - ) - - # robot_command.add_task(ri.mios_task_factory.generate_loose_gripper_call()) - robot_command.add_task(ri.mios_task_factory.generate_drive_in_mp("bolt", container)) - - robot_command.execute_task_list_sync() - - -@execution_timer -def test_drive_out_mp(container: str = "normalcontainer"): - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - shared_data=None, - task_scene=scene, - robot_interface=ri, - ) - - # robot_command.add_task(ri.mios_task_factory.generate_loose_gripper_call()) - robot_command.add_task( - ri.mios_task_factory.generate_drive_out_mp("bolt", container) - ) - - robot_command.execute_task_list_sync() - - -# def insert_test(object_name: str): -# robot_command = RobotCommand( -# robot_address="127.0.0.1", -# robot_port=12000, -# task_scene=scene, -# shared_data=None, -# robot_interface=ri, -# ) - -# insert_action = { -# "action_name": "insert", -# "args": [None, None, "gear3", object_name], -# } - -# robot_command.add_tasks(ri.mios_task_factory.generate_insert_skill(insert_action)) - -# robot_command.execute_task_list_sync() - - -def n3_task(): - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - task_scene=scene, - shared_data=None, - robot_interface=ri, - ) - - task_list = [] - - task_list.append(ri.mios_task_factory.generate_joint_move_mp("initialposition3")) - - task_list.append(ri.mios_task_factory.generate_move_above_mp("gear2")) - - task_list.append(ri.mios_task_factory.generate_reach_mp("gear2")) - - task_list.append(ri.mios_task_factory.generate_gripper_grasp_mp()) - - task_list.append(ri.mios_task_factory.generate_move_above_mp("gear2")) - - p1 = { - "search_a": [5, 5, 0, 0, 0, 0], - "search_f": [2, 2, 0, 0, 0, 0], - "search_phi": [ - 0, - 3.14159265358979323846 / 2, - 0, - 3.14159265358979323846 / 2, - 0, - 3.14159265358979323846 / 2, - ], - "F_ext_contact": [10.0, 2.0], - "f_push": [0, 0, 8, 0, 0, 0], - "K_x": [100, 100, 0, 800, 800, 800], - "env_X": [0.01, 0.01, -0.004, 0.05, 0.05, 0.05], - # "D_x": [0.7, 0.7, 0, 0.7, 0.7, 1.4], - } - - task_list.append(ri.mios_task_factory.generate_insert_mp("gear2", "p3_n1", p1)) - - task_list.append(ri.mios_task_factory.generate_gripper_move_mp(0.08)) - - task_list.append(ri.mios_task_factory.generate_move_above_mp("p3_n1")) - - task_list.append(ri.mios_task_factory.generate_reach_mp("p3_n1")) - - task_list.append(ri.mios_task_factory.generate_gripper_grasp_mp()) - - task_list.append(ri.mios_task_factory.generate_move_above_mp("p3_n1")) - - task_list.append(ri.mios_task_factory.generate_move_above_mp("gear2")) - - task_list.append(ri.mios_task_factory.generate_reach_mp("gear2")) - - task_list.append(ri.mios_task_factory.generate_gripper_move_mp(0.08)) - - task_list.append(ri.mios_task_factory.generate_move_above_mp("gear2")) - - task_list.append(ri.mios_task_factory.generate_joint_move_mp("initialposition3")) - - robot_command.add_tasks(task_list) - - robot_command.execute_task_list_sync() - - -@execution_timer -def n4_task(): - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - task_scene=scene, - shared_data=None, - robot_interface=ri, - ) - - task_list = [] - - task_list.append(ri.mios_task_factory.generate_joint_move_mp("initialposition4")) - - task_list.append(ri.mios_task_factory.generate_move_above_mp("gear1")) - - task_list.append(ri.mios_task_factory.generate_reach_mp("gear1")) - - task_list.append(ri.mios_task_factory.generate_gripper_grasp_mp()) - - task_list.append(ri.mios_task_factory.generate_move_above_mp("gear1")) - - task_list.append(ri.mios_task_factory.generate_joint_move_mp("p4_n1_above")) - - p1 = { - "search_a": [5, 5, 0, 1, 1, 8], - "search_f": [2, 2, 0, 1, 1, 1], - "search_phi": [ - 0, - 3.14159265358979323846 / 2, - 0, - 3.14159265358979323846 / 2, - 0, - 3.14159265358979323846 / 2, - ], - "F_ext_contact": [10.0, 2.0], - "f_push": [0, 0, 10, 0, 0, 0], - "K_x": [100, 100, 0, 800, 800, 150], - "env_X": [0.01, 0.01, -0.008, 0.05, 0.05, 0.05], - # "D_x": [0.7, 0.7, 0, 0.7, 0.7, 1.4], - } - - task_list.append(ri.mios_task_factory.generate_insert_mp("gear1", "p4_n1", p1)) - - task_list.append(ri.mios_task_factory.generate_gripper_move_mp(0.08)) - - task_list.append(ri.mios_task_factory.generate_move_above_mp("p4_n1")) - - task_list.append(ri.mios_task_factory.generate_reach_mp("p4_n1")) - - task_list.append(ri.mios_task_factory.generate_gripper_grasp_mp()) - - task_list.append(ri.mios_task_factory.generate_move_above_mp("p4_n1")) - - task_list.append(ri.mios_task_factory.generate_move_above_mp("gear1")) - - task_list.append(ri.mios_task_factory.generate_reach_mp("gear1")) - - task_list.append(ri.mios_task_factory.generate_gripper_move_mp(0.08)) - - task_list.append(ri.mios_task_factory.generate_move_above_mp("gear1")) - - task_list.append(ri.mios_task_factory.generate_joint_move_mp("initialposition4")) - - robot_command.add_tasks(task_list) - - robot_command.execute_task_list_sync() - - -@execution_timer -def n1_task(): - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - task_scene=scene, - shared_data=None, - robot_interface=ri, - ) - - task_list = [] - - task_list.append(ri.mios_task_factory.generate_joint_move_mp("initialposition1")) - - task_list.append(ri.mios_task_factory.generate_move_above_mp("shaft1")) - - task_list.append(ri.mios_task_factory.generate_reach_mp("shaft1")) - - task_list.append(ri.mios_task_factory.generate_gripper_grasp_mp()) - - task_list.append(ri.mios_task_factory.generate_move_above_mp("shaft1")) - - p1 = { - "search_a": [5, 5, 0, 0, 0, 0], - "search_f": [2, 2, 0, 0, 0, 0], - "search_phi": [ - 0, - 3.14159265358979323846 / 2, - 0, - 3.14159265358979323846 / 2, - 0, - 3.14159265358979323846 / 2, - ], - "F_ext_contact": [10.0, 2.0], - "f_push": [0, 0, 8, 0, 0, 0], - "K_x": [100, 100, 0, 800, 800, 800], - "env_X": [0.01, 0.01, -0.004, 0.05, 0.05, 0.05], - # "D_x": [0.7, 0.7, 0, 0.7, 0.7, 1.4], - } - - task_list.append(ri.mios_task_factory.generate_insert_mp("shaft1", "p1_n1", p1)) - - task_list.append(ri.mios_task_factory.generate_gripper_move_mp(0.08)) - - task_list.append(ri.mios_task_factory.generate_move_above_mp("p1_n1")) - - task_list.append(ri.mios_task_factory.generate_reach_mp("p1_n1")) - - task_list.append(ri.mios_task_factory.generate_gripper_grasp_mp()) - - task_list.append(ri.mios_task_factory.generate_move_above_mp("p1_n1")) - - task_list.append(ri.mios_task_factory.generate_move_above_mp("shaft1")) - - task_list.append(ri.mios_task_factory.generate_reach_mp("shaft1")) - - # task_list.append(ri.mios_task_factory.generate_insert_mp("", "shaft1")) - - task_list.append(ri.mios_task_factory.generate_gripper_move_mp(0.08)) - - task_list.append(ri.mios_task_factory.generate_move_above_mp("shaft1")) - - task_list.append(ri.mios_task_factory.generate_joint_move_mp("initialposition1")) - - robot_command.add_tasks(task_list) - - robot_command.execute_task_list_sync() - - -@execution_timer -def n2_task(): - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - task_scene=scene, - shared_data=None, - robot_interface=ri, - ) - - task_list = [] - - dx_d = [0.5, 5] - - ddx_d = [0.5, 2] - - task_list.append(ri.mios_task_factory.generate_joint_move_mp("initialposition2")) - - task_list.append(ri.mios_task_factory.generate_move_above_mp("shaft2")) - - task_list.append(ri.mios_task_factory.generate_reach_mp("shaft2")) - - task_list.append(ri.mios_task_factory.generate_gripper_grasp_mp()) - - task_list.append(ri.mios_task_factory.generate_move_above_mp("shaft2")) - - task_list.append(ri.mios_task_factory.generate_move_above_mp("p2_n1")) - - task_list.append( - ri.mios_task_factory.generate_cartesian_move_mp( - "p2_n1", - dx_d=dx_d, - ddx_d=ddx_d, - ) - ) - - for i in range(2): - - # *1 - task_list.append( - ri.mios_task_factory.generate_cartesian_move_mp( - "p2_n2", - dx_d=dx_d, - ddx_d=ddx_d, - ) - ) - - task_list.append(ri.mios_task_factory.generate_gripper_move_mp(0.05)) - - # *2 - task_list.append( - ri.mios_task_factory.generate_cartesian_move_mp( - "p2_n1", - dx_d=dx_d, - ddx_d=ddx_d, - ) - ) - - task_list.append(ri.mios_task_factory.generate_gripper_grasp_mp()) - - task_list.append( - ri.mios_task_factory.generate_cartesian_move_mp( - "p2_n2", - dx_d=dx_d, - ddx_d=ddx_d, - ) - ) - - task_list.append(ri.mios_task_factory.generate_gripper_move_mp(0.05)) - - # *mid - - task_list.append(ri.mios_task_factory.generate_move_above_mp("p2_n2")) - - # *1 - for i in range(3): - task_list.append( - ri.mios_task_factory.generate_cartesian_move_mp( - "p2_n2", - dx_d=dx_d, - ddx_d=ddx_d, - ) - ) - task_list.append(ri.mios_task_factory.generate_gripper_grasp_mp()) - - task_list.append( - ri.mios_task_factory.generate_cartesian_move_mp( - "p2_n1", - dx_d=dx_d, - ddx_d=ddx_d, - ) - ) - - task_list.append(ri.mios_task_factory.generate_gripper_move_mp(0.05)) - - task_list.append(ri.mios_task_factory.generate_gripper_grasp_mp()) - - # * end - - task_list.append(ri.mios_task_factory.generate_move_above_mp("p2_n1")) - - task_list.append(ri.mios_task_factory.generate_move_above_mp("shaft2")) - - task_list.append(ri.mios_task_factory.generate_reach_mp("shaft2")) - - task_list.append(ri.mios_task_factory.generate_gripper_move_mp(0.05)) - - task_list.append(ri.mios_task_factory.generate_move_above_mp("shaft2")) - - task_list.append(ri.mios_task_factory.generate_joint_move_mp("initialposition2")) - - robot_command.add_tasks(task_list) - - robot_command.execute_task_list_sync() - - -# @execution_timer -# def sp2(): -# robot_command = RobotCommand( -# robot_address="127.0.0.1", -# robot_port=12000, -# task_scene=scene, -# shared_data=None, -# robot_interface=ri, -# ) - -# task_list = [] - -# dx_d = [0.5, 5] - -# ddx_d = [0.5, 2] - -# task_list.append(ri.mios_task_factory.generate_joint_move_mp("initialposition2")) - -# task_list.append(ri.mios_task_factory.generate_joint_move_mp("sp_shaft2_above")) - -# task_list.append(ri.mios_task_factory.generate_move_above_mp("sp_shaft2")) - -# task_list.append(ri.mios_task_factory.generate_reach_mp("sp_shaft2")) - -# task_list.append(ri.mios_task_factory.generate_gripper_grasp_mp()) - -# task_list.append(ri.mios_task_factory.generate_move_above_mp("sp_shaft2")) - -# task_list.append(ri.mios_task_factory.generate_joint_move_mp("initialposition2")) - -# task_list.append(ri.mios_task_factory.generate_joint_move_mp("p2_sp1_above")) - -# task_list.append(ri.mios_task_factory.generate_move_above_mp("p2_sp1")) - -# p1 = { -# "search_a": [16, 12, 0, 0, 0, 0], -# "search_f": [1, 0.5, 0, 0, 0, 0], -# "search_phi": [ -# 0, -# 3.14159265358979323846 / 2, -# 0, -# 3.14159265358979323846 / 2, -# 0, -# 3.14159265358979323846 / 2, -# ], -# "F_ext_contact": [10.0, 2.0], -# "f_push": [0, 0, 12, 0, 0, 0], -# "K_x": [700, 700, 0, 800, 800, 800], -# "env_X": [0.04, 0.04, -0.004, 0.05, 0.05, 0.05], -# # "D_x": [0.7, 0.7, 0, 0.7, 0.7, 1.4], -# } - -# task_list.append( -# ri.mios_task_factory.generate_insert_mp( -# "sp_shaft2", -# "p2_sp1", -# param=p1, -# ) -# ) - -# robot_command.add_tasks(task_list) - -# robot_command.execute_task_list_sync() - -# robot_command.clear_tasks() - -# task_list = [] - -# O_T_EE_1 = ri.proprioceptor.get_robot_state().O_T_EE - -# O_T_EE_1[2][3] -= 0.02 - -# # * ROTATE IN Z AXIS BY 90 DEGREES - -# # * WITH NP -# rotation_90 = np.array([[0, -1, 0, 0], [1, 0, 0, 0], [0, 0, 1, 0], [0, 0, 0, 1]]) - -# # after rotation -# O_T_EE_2 = O_T_EE_1 @ rotation_90 - -# for i in range(4): - -# # *1 -# task_list.append( -# ri.mios_task_factory.generate_cartesian_move_mp( -# O_T_TCP=O_T_EE_2, -# dx_d=dx_d, -# ddx_d=ddx_d, -# ) -# ) - -# task_list.append(ri.mios_task_factory.generate_gripper_move_mp(0.05)) - -# # *2 -# task_list.append( -# ri.mios_task_factory.generate_cartesian_move_mp( -# O_T_TCP=O_T_EE_1, -# dx_d=dx_d, -# ddx_d=ddx_d, -# ) -# ) - -# task_list.append(ri.mios_task_factory.generate_gripper_grasp_mp()) - -# task_list.append( -# ri.mios_task_factory.generate_cartesian_move_mp( -# O_T_TCP=O_T_EE_2, -# dx_d=dx_d, -# ddx_d=ddx_d, -# ) -# ) - -# task_list.append(ri.mios_task_factory.generate_gripper_move_mp(0.08)) - -# # *mid - -# task_list.append(ri.mios_task_factory.generate_move_above_mp("p2_sp1")) - -# task_list.append(ri.mios_task_factory.generate_joint_move_mp("initialposition2")) - -# robot_command.add_tasks(task_list) - -# robot_command.execute_task_list_sync() - - -# * sp 77.00060167299671 -# * n 134.96007828999427 -def n2_loop(): - while True: - n2_task() - - -@execution_timer -def sp2(): - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - task_scene=scene, - shared_data=None, - robot_interface=ri, - ) - - task_list = [] - - dx_d = [0.5, 5] - - ddx_d = [0.5, 2] - - task_list.append(ri.mios_task_factory.generate_joint_move_mp("initialposition2")) - - task_list.append(ri.mios_task_factory.generate_joint_move_mp("observe")) - - robot_command.add_tasks(task_list) - - robot_command.execute_task_list_sync() - - robot_command.clear_tasks() - - task_list = [] - - # observe - - robot_state = ri.proprioceptor.get_robot_state() - - EE_T_cam = np.array( - [ - [0.0, -1.0, 0.0, 0.0514976], - [1.0, 0.0, 0.0, -0.0375164], - [0.0, 0.0, 1.0, -0.0467639], - [0.0, 0.0, 0.0, 1.0], - ] - ) - thread = threading.Thread(target=apriltag_rs) - thread.start() - - cam_T_mark = get_mark() - - thread.join() - - O_T_obj = robot_state.O_T_TCP @ EE_T_cam @ cam_T_mark - - p2_sp1 = get_object("p2_sp1") - - p2_sp1.O_T_TCP[0][3] = O_T_obj[0][3] - 0.155 - - p2_sp1.O_T_TCP[1][3] = O_T_obj[1][3] - 0.093 - - task_list.append(ri.mios_task_factory.generate_joint_move_mp("initialposition2")) - - task_list.append(ri.mios_task_factory.generate_joint_move_mp("sp_shaft2_above")) - - task_list.append(ri.mios_task_factory.generate_move_above_mp("sp_shaft2")) - - task_list.append(ri.mios_task_factory.generate_reach_mp("sp_shaft2")) - - task_list.append(ri.mios_task_factory.generate_gripper_grasp_mp()) - - task_list.append(ri.mios_task_factory.generate_move_above_mp("sp_shaft2")) - - task_list.append(ri.mios_task_factory.generate_joint_move_mp("initialposition2")) - - task_list.append(ri.mios_task_factory.generate_joint_move_mp("p2_sp1_above")) - - task_list.append(ri.mios_task_factory.generate_move_above_mp("p2_sp1")) - - p1 = { - "search_a": [8, 8, 0, 0, 0, 0], - "search_f": [1, 0.5, 0, 0, 0, 0], - "search_phi": [ - 0, - 3.14159265358979323846 / 2, - 0, - 3.14159265358979323846 / 2, - 0, - 3.14159265358979323846 / 2, - ], - "F_ext_contact": [10.0, 2.0], - "f_push": [0, 0, 12, 0, 0, 0], - "K_x": [700, 700, 0, 800, 800, 800], - "env_X": [0.04, 0.04, -0.004, 0.05, 0.05, 0.05], - # "D_x": [0.7, 0.7, 0, 0.7, 0.7, 1.4], - } - - task_list.append( - ri.mios_task_factory.generate_insert_mp( - "sp_shaft2", - "p2_sp1", - param=p1, - ) - ) - - robot_command.add_tasks(task_list) - - robot_command.execute_task_list_sync() - - robot_command.clear_tasks() - - task_list = [] - - O_T_EE_1 = ri.proprioceptor.get_robot_state().O_T_EE - - O_T_EE_1[2][3] -= 0.02 - - # * ROTATE IN Z AXIS BY 90 DEGREES - - # * WITH NP - rotation_90 = np.array([[0, -1, 0, 0], [1, 0, 0, 0], [0, 0, 1, 0], [0, 0, 0, 1]]) - - # after rotation - O_T_EE_2 = O_T_EE_1 @ rotation_90 - - for i in range(4): - - # *1 - task_list.append( - ri.mios_task_factory.generate_cartesian_move_mp( - O_T_TCP=O_T_EE_2, - dx_d=dx_d, - ddx_d=ddx_d, - ) - ) - - task_list.append(ri.mios_task_factory.generate_gripper_move_mp(0.05)) - - # *2 - task_list.append( - ri.mios_task_factory.generate_cartesian_move_mp( - O_T_TCP=O_T_EE_1, - dx_d=dx_d, - ddx_d=ddx_d, - ) - ) - - task_list.append(ri.mios_task_factory.generate_gripper_grasp_mp()) - - task_list.append( - ri.mios_task_factory.generate_cartesian_move_mp( - O_T_TCP=O_T_EE_2, - dx_d=dx_d, - ddx_d=ddx_d, - ) - ) - - task_list.append(ri.mios_task_factory.generate_gripper_move_mp(0.08)) - - # *mid - - task_list.append(ri.mios_task_factory.generate_move_above_mp("p2_sp1")) - - task_list.append(ri.mios_task_factory.generate_joint_move_mp("initialposition2")) - - robot_command.add_tasks(task_list) - - robot_command.execute_task_list_sync() - - -# * sp 77.00060167299671 -# * n 134.96007828999427 -def n2_loop(): - while True: - n2_task() - - -@execution_timer -def n1_sp(): - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - task_scene=scene, - shared_data=None, - robot_interface=ri, - ) - - task_list = [] - - task_list.append(ri.mios_task_factory.generate_joint_move_mp("initialposition1")) - - task_list.append(ri.mios_task_factory.generate_move_above_mp("sp_shaft1")) - - task_list.append(ri.mios_task_factory.generate_reach_mp("sp_shaft1")) - - task_list.append(ri.mios_task_factory.generate_gripper_grasp_mp()) - - task_list.append(ri.mios_task_factory.generate_move_above_mp("sp_shaft1")) - - task_list.append(ri.mios_task_factory.generate_joint_move_mp("initialposition1")) - - task_list.append(ri.mios_task_factory.generate_joint_move_mp("observe")) - - robot_command.add_tasks(task_list) - - robot_command.execute_task_list_sync() - - # * clear all tasks - - robot_command.clear_tasks() - - task_list = [] - - # * p1_sp1 - - robot_state = ri.proprioceptor.get_robot_state() - - EE_T_cam = np.array( - [ - [0.0, -1.0, 0.0, 0.0514976], - [1.0, 0.0, 0.0, -0.0375164], - [0.0, 0.0, 1.0, -0.0467639], - [0.0, 0.0, 0.0, 1.0], - ] - ) - thread = threading.Thread(target=apriltag_rs) - thread.start() - - cam_T_mark = get_mark() - - thread.join() - - O_T_obj = robot_state.O_T_TCP @ EE_T_cam @ cam_T_mark - - p1_sp1 = get_object("p1_sp1") - - p1_sp2 = get_object("p1_sp2") - - p1_sp1.O_T_TCP[0][3] = O_T_obj[0][3] - 0.145 - - p1_sp1.O_T_TCP[1][3] = O_T_obj[1][3] + 0.045 - - p1_sp2.O_T_TCP[0][3] = O_T_obj[0][3] - 0.04 - - p1_sp2.O_T_TCP[1][3] = O_T_obj[1][3] - 0.085 - - p1 = { - "search_a": [8, 8, 0, 0, 0, 0], - "search_f": [1, 1, 0, 0, 0, 0], - "search_phi": [ - 0, - 3.14159265358979323846 / 2, - 0, - 3.14159265358979323846 / 2, - 0, - 3.14159265358979323846 / 2, - ], - "F_ext_contact": [10.0, 2.0], - "f_push": [0, 0, 8, 0, 0, 0], - "K_x": [500, 500, 0, 800, 800, 800], - "env_X": [0.04, 0.04, -0.004, 0.05, 0.05, 0.05], - # "D_x": [0.7, 0.7, 0, 0.7, 0.7, 1.4], - } - - task_list.append(ri.mios_task_factory.generate_insert_mp("sp_shaft1", "p1_sp1", p1)) - - task_list.append(ri.mios_task_factory.generate_gripper_move_mp(0.08)) - - task_list.append(ri.mios_task_factory.generate_move_above_mp("p1_sp1")) - - task_list.append(ri.mios_task_factory.generate_joint_move_mp("initialposition1")) - - # * p1_sp2 - - task_list.append(ri.mios_task_factory.generate_move_above_mp("sp_shaft2")) - - task_list.append(ri.mios_task_factory.generate_reach_mp("sp_shaft2")) - - task_list.append(ri.mios_task_factory.generate_gripper_grasp_mp()) - - task_list.append(ri.mios_task_factory.generate_move_above_mp("sp_shaft2")) - - task_list.append(ri.mios_task_factory.generate_joint_move_mp("initialposition1")) - - task_list.append(ri.mios_task_factory.generate_joint_move_mp("observe")) - - task_list.append(ri.mios_task_factory.generate_insert_mp("sp_shaft2", "p1_sp2", p1)) - - task_list.append(ri.mios_task_factory.generate_gripper_move_mp(0.08)) - - task_list.append(ri.mios_task_factory.generate_move_above_mp("p1_sp2")) - - task_list.append(ri.mios_task_factory.generate_joint_move_mp("initialposition1")) - - robot_command.add_tasks(task_list) - - robot_command.execute_task_list_sync() - - -if __name__ == "__main__": - pass - # apriltag_rs() - # cartesian_move_T() - sp2() - -# n 44.29451922999942 -# sp 79.95160165699781 diff --git a/experiments/workstation/scene.json b/experiments/workstation/scene.json deleted file mode 100644 index fbc3707e..00000000 --- a/experiments/workstation/scene.json +++ /dev/null @@ -1,137 +0,0 @@ -{ - "reference_objects": - [ - { - "object_name": "initialposition", - "source": "mios" - }, - { - "object_name": "normalpickplace", - "source": "mios" - }, - { - "object_name": "normalcontainer", - "source": "mios" - }, - { - "object_name": "p1_sp1", - "source": "mios" - }, - { - "object_name": "p1_sp2", - "source": "mios" - }, - { - "object_name": "p2_sp1", - "source": "mios" - }, - { - "object_name": "p2_sp1_above", - "source": "mios" - }, - { - "object_name": "p3_sp1", - "source": "mios" - }, - { - "object_name": "p4_sp1", - "source": "mios" - }, - { - "object_name": "p4_sp2", - "source": "mios" - }, - { - "object_name": "initialposition1", - "source": "mios" - }, - { - "object_name": "initialposition2", - "source":"mios" - }, - { - "object_name": "initialposition3", - "source": "mios" - }, - { - "object_name": "initialposition4", - "source": "mios" - }, - { - "object_name": "p1_n1", - "source":"mios" - }, - { - "object_name": "p2_n1", - "source":"mios" - }, - { - "object_name": "p2_n2", - "source":"mios" - }, - { - "object_name": "p3_n1", - "source": "mios" - }, - { - "object_name": "p4_n1_above", - "source": "mios" - }, - { - "object_name": "p4_n1", - "source": "mios" - }, - { - "object_name": "shaft1", - "source": "mios" - }, - { - "object_name": "gear2", - "source":"mios" - }, - { - "object_name": "shaft2", - "source":"mios" - }, - { - "object_name": "sp_shaft2", - "source":"mios" - }, - { - "object_name": "sp_shaft2_above", - "source":"mios" - }, - { - "object_name": "gear1", - "source":"mios" - }, - { - "object_name": "observe", - "source":"mios" - }, - { - "object_name": "sp_shaft2", - "source":"mios" - }, - { - "object_name": "sp_shaft1", - "source":"mios" - } - ], - "tools": - [ - { - "tool_name": "defaultgripper", - "EE_T_TCP": [ - [1, 0, 0, 0.0], - [0, 1, 0, 0.0], - [0, 0, 1, 0.03], - [0, 0, 0, 1] - ], - "EE_finger_width_max": 0.08, - "EE_finger_width_min": 0.01, - "load_width": 0.08, - "unload_width": 0.08 - } - ] -} diff --git a/experiments/workstation/station_demo.py b/experiments/workstation/station_demo.py deleted file mode 100644 index 1bd77362..00000000 --- a/experiments/workstation/station_demo.py +++ /dev/null @@ -1,45 +0,0 @@ -import queue -import threading -import socket -from runtime_script import n2_task as nx -from runtime_script import sp2 as spx - -# Create a queue to contain tasks -task_queue = queue.Queue() - -this_ip = "10.157.174.230" - - -# Function to execute a task -def execute_task(task_function): - task_function() - - -# Function to listen for UDP messages -def listen_udp(): - # Create a UDP socket - udp_socket = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) - udp_socket.bind((this_ip, 12321)) - - while True: - # Receive UDP message - data, addr = udp_socket.recvfrom(1024) - message = data.decode("utf-8") - - # Add a special task to the queue - task_queue.put(spx) - - -# Start a thread to listen for UDP messages -udp_thread = threading.Thread(target=listen_udp) -udp_thread.start() - -while True: - - if task_queue.empty(): - task_queue.put(nx) - # Pop a task from the queue - task = task_queue.get() - - # Execute the task - execute_task(task) diff --git a/experiments/workstation/vision.py b/experiments/workstation/vision.py deleted file mode 100644 index afdb8775..00000000 --- a/experiments/workstation/vision.py +++ /dev/null @@ -1,3 +0,0 @@ -from anran_tag.apriltag_rs import apriltag_rs - -apriltag_rs() diff --git a/kios_bt_planning/CHECK_LIST b/kios_bt_planning/CHECK_LIST deleted file mode 100644 index 48baf89c..00000000 --- a/kios_bt_planning/CHECK_LIST +++ /dev/null @@ -1,4 +0,0 @@ -- req list -- smith key -- api key -- python version diff --git a/kios_bt_planning/__init__.py b/kios_bt_planning/__init__.py deleted file mode 100644 index e69de29b..00000000 diff --git a/kios_bt_planning/backups/BT_planning/.coveragerc b/kios_bt_planning/backups/BT_planning/.coveragerc deleted file mode 100644 index 4c09af2f..00000000 --- a/kios_bt_planning/backups/BT_planning/.coveragerc +++ /dev/null @@ -1,18 +0,0 @@ -# .coveragerc to control coverage.py -[run] -branch = True -omit = - *\__init__.py - *\site-packages\* - *eloquent* - *lib* - *\tests\* - *\py_trees-release* -[report] -# Regexes for lines to exclude from consideration -exclude_lines = - # Have to re-enable the standard pragma - pragma: no cover - -ignore_errors = True -show_missing = True diff --git a/kios_bt_planning/backups/BT_planning/.gitignore b/kios_bt_planning/backups/BT_planning/.gitignore deleted file mode 100644 index b6e47617..00000000 --- a/kios_bt_planning/backups/BT_planning/.gitignore +++ /dev/null @@ -1,129 +0,0 @@ -# Byte-compiled / optimized / DLL files -__pycache__/ -*.py[cod] -*$py.class - -# C extensions -*.so - -# Distribution / packaging -.Python -build/ -develop-eggs/ -dist/ -downloads/ -eggs/ -.eggs/ -lib/ -lib64/ -parts/ -sdist/ -var/ -wheels/ -pip-wheel-metadata/ -share/python-wheels/ -*.egg-info/ -.installed.cfg -*.egg -MANIFEST - -# PyInstaller -# Usually these files are written by a python script from a template -# before PyInstaller builds the exe, so as to inject date/other infos into it. -*.manifest -*.spec - -# Installer logs -pip-log.txt -pip-delete-this-directory.txt - -# Unit test / coverage reports -htmlcov/ -.tox/ -.nox/ -.coverage -.coverage.* -.cache -nosetests.xml -coverage.xml -*.cover -*.py,cover -.hypothesis/ -.pytest_cache/ - -# Translations -*.mo -*.pot - -# Django stuff: -*.log -local_settings.py -db.sqlite3 -db.sqlite3-journal - -# Flask stuff: -instance/ -.webassets-cache - -# Scrapy stuff: -.scrapy - -# Sphinx documentation -docs/_build/ - -# PyBuilder -target/ - -# Jupyter Notebook -.ipynb_checkpoints - -# IPython -profile_default/ -ipython_config.py - -# pyenv -.python-version - -# pipenv -# According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control. -# However, in case of collaboration, if having platform-specific dependencies or dependencies -# having no cross-platform support, pipenv may install dependencies that don't work, or not -# install all needed dependencies. -#Pipfile.lock - -# PEP 582; used by e.g. github.com/David-OConnor/pyflow -__pypackages__/ - -# Celery stuff -celerybeat-schedule -celerybeat.pid - -# SageMath parsed files -*.sage.py - -# Environments -.env -.venv -env/ -venv/ -ENV/ -env.bak/ -venv.bak/ - -# Spyder project settings -.spyderproject -.spyproject - -# Rope project settings -.ropeproject - -# mkdocs documentation -/site - -# mypy -.mypy_cache/ -.dmypy.json -dmypy.json - -# Pyre type checker -.pyre/ diff --git a/kios_bt_planning/backups/BT_planning/.pylintrc b/kios_bt_planning/backups/BT_planning/.pylintrc deleted file mode 100644 index d195c61d..00000000 --- a/kios_bt_planning/backups/BT_planning/.pylintrc +++ /dev/null @@ -1,594 +0,0 @@ -[MASTER] - -# A comma-separated list of package or module names from where C extensions may -# be loaded. Extensions are loading into the active Python interpreter and may -# run arbitrary code. -extension-pkg-whitelist= - -# Add files or directories to the blacklist. They should be base names, not -# paths. -ignore=CVS - -# Add files or directories matching the regex patterns to the blacklist. The -# regex matches against base names, not paths. -ignore-patterns= - -# Python code to execute, usually for sys.path manipulation such as -# pygtk.require(). -#init-hook= - -# Use multiple processes to speed up Pylint. Specifying 0 will auto-detect the -# number of processors available to use. -jobs=1 - -# Control the amount of potential inferred values when inferring a single -# object. This can help the performance when dealing with large functions or -# complex, nested conditions. -limit-inference-results=100 - -# List of plugins (as comma separated values of python module names) to load, -# usually to register additional checkers. -load-plugins= - -# Pickle collected data for later comparisons. -persistent=yes - -# Specify a configuration file. -#rcfile= - -# When enabled, pylint would attempt to guess common misconfiguration and emit -# user-friendly hints instead of false-positive error messages. -suggestion-mode=yes - -# Allow loading of arbitrary C extensions. Extensions are imported into the -# active Python interpreter and may run arbitrary code. -unsafe-load-any-extension=no - - -[MESSAGES CONTROL] - -# Only show warnings with the listed confidence levels. Leave empty to show -# all. Valid levels: HIGH, INFERENCE, INFERENCE_FAILURE, UNDEFINED. -confidence= - -# Disable the message, report, category or checker with the given id(s). You -# can either give multiple identifiers separated by comma (,) or put this -# option multiple times (only on the command line, not in the configuration -# file where it should appear only once). You can also use "--disable=all" to -# disable everything first and then reenable specific checks. For example, if -# you want to run only the similarities checker, you can use "--disable=all -# --enable=similarities". If you want to run only the classes checker, but have -# no Warning level messages displayed, use "--disable=all --enable=classes -# --disable=W". -disable=print-statement, - parameter-unpacking, - unpacking-in-except, - old-raise-syntax, - backtick, - long-suffix, - old-ne-operator, - old-octal-literal, - import-star-module-level, - non-ascii-bytes-literal, - raw-checker-failed, - bad-inline-option, - locally-disabled, - file-ignored, - suppressed-message, - useless-suppression, - deprecated-pragma, - use-symbolic-message-instead, - apply-builtin, - basestring-builtin, - buffer-builtin, - cmp-builtin, - coerce-builtin, - execfile-builtin, - file-builtin, - long-builtin, - raw_input-builtin, - reduce-builtin, - standarderror-builtin, - unicode-builtin, - xrange-builtin, - coerce-method, - delslice-method, - getslice-method, - setslice-method, - no-absolute-import, - old-division, - dict-iter-method, - dict-view-method, - next-method-called, - metaclass-assignment, - indexing-exception, - raising-string, - reload-builtin, - oct-method, - hex-method, - nonzero-method, - cmp-method, - input-builtin, - round-builtin, - intern-builtin, - unichr-builtin, - map-builtin-not-iterating, - zip-builtin-not-iterating, - range-builtin-not-iterating, - filter-builtin-not-iterating, - using-cmp-argument, - eq-without-hash, - div-method, - idiv-method, - rdiv-method, - exception-message-attribute, - invalid-str-codec, - sys-max-int, - bad-python3-import, - deprecated-string-function, - deprecated-str-translate-call, - deprecated-itertools-function, - deprecated-types-field, - next-method-defined, - dict-items-not-iterating, - dict-keys-not-iterating, - dict-values-not-iterating, - deprecated-operator-function, - deprecated-urllib-function, - xreadlines-attribute, - deprecated-sys-function, - exception-escape, - comprehension-escape, - consider-using-enumerate - -# Enable the message, report, category or checker with the given id(s). You can -# either give multiple identifier separated by comma (,) or put this option -# multiple time (only on the command line, not in the configuration file where -# it should appear only once). See also the "--disable" option for examples. -enable=c-extension-no-member - - -[REPORTS] - -# Python expression which should return a score less than or equal to 10. You -# have access to the variables 'error', 'warning', 'refactor', and 'convention' -# which contain the number of messages in each category, as well as 'statement' -# which is the total number of statements analyzed. This score is used by the -# global evaluation report (RP0004). -evaluation=10.0 - ((float(5 * error + warning + refactor + convention) / statement) * 10) - -# Template used to display messages. This is a python new-style format string -# used to format the message information. See doc for all details. -#msg-template= - -# Set the output format. Available formats are text, parseable, colorized, json -# and msvs (visual studio). You can also give a reporter class, e.g. -# mypackage.mymodule.MyReporterClass. -output-format=text - -# Tells whether to display a full report or only the messages. -reports=no - -# Activate the evaluation score. -score=yes - - -[REFACTORING] - -# Maximum number of nested blocks for function / method body -max-nested-blocks=6 - -# Complete name of functions that never returns. When checking for -# inconsistent-return-statements if a never returning function is called then -# it will be considered as an explicit return statement and no message will be -# printed. -never-returning-functions=sys.exit - - -[BASIC] - -# Naming style matching correct argument names. -argument-naming-style=snake_case - -# Regular expression matching correct argument names. Overrides argument- -# naming-style. -#argument-rgx= - -# Naming style matching correct attribute names. -attr-naming-style=snake_case - -# Regular expression matching correct attribute names. Overrides attr-naming- -# style. -#attr-rgx= - -# Bad variable names which should always be refused, separated by a comma. -bad-names=foo, - bar, - baz, - toto, - tutu, - tata - -# Naming style matching correct class attribute names. -class-attribute-naming-style=any - -# Regular expression matching correct class attribute names. Overrides class- -# attribute-naming-style. -#class-attribute-rgx= - -# Naming style matching correct class names. -class-naming-style=PascalCase - -# Regular expression matching correct class names. Overrides class-naming- -# style. -#class-rgx= - -# Naming style matching correct constant names. -const-naming-style=UPPER_CASE - -# Regular expression matching correct constant names. Overrides const-naming- -# style. -#const-rgx= - -# Minimum line length for functions/classes that require docstrings, shorter -# ones are exempt. -docstring-min-length=-1 - -# Naming style matching correct function names. -function-naming-style=snake_case - -# Regular expression matching correct function names. Overrides function- -# naming-style. -#function-rgx= - -# Good variable names which should always be accepted, separated by a comma. -good-names=d, - e, - i, - j, - k, - l, - m, - n, - f, - x, - y, - z, - p, - ex, - bt, - vc, - Run, - _ - -# Include a hint for the correct naming format with invalid-name. -include-naming-hint=no - -# Naming style matching correct inline iteration names. -inlinevar-naming-style=any - -# Regular expression matching correct inline iteration names. Overrides -# inlinevar-naming-style. -#inlinevar-rgx= - -# Naming style matching correct method names. -method-naming-style=snake_case - -# Regular expression matching correct method names. Overrides method-naming- -# style. -#method-rgx= - -# Naming style matching correct module names. -module-naming-style=snake_case - -# Regular expression matching correct module names. Overrides module-naming- -# style. -#module-rgx= - -# Colon-delimited sets of names that determine each other's naming style when -# the name regexes allow several styles. -name-group= - -# Regular expression which should only match function or class names that do -# not require a docstring. -no-docstring-rgx=^_ - -# List of decorators that produce properties, such as abc.abstractproperty. Add -# to this list to register other decorators that produce valid properties. -# These decorators are taken in consideration only for invalid-name. -property-classes=abc.abstractproperty - -# Naming style matching correct variable names. -variable-naming-style=snake_case - -# Regular expression matching correct variable names. Overrides variable- -# naming-style. -#variable-rgx= - - -[FORMAT] - -# Expected format of line ending, e.g. empty (any line ending), LF or CRLF. -expected-line-ending-format= - -# Regexp for a line that is allowed to be longer than the limit. -ignore-long-lines=^\s*(# )??$ - -# Number of spaces of indent required inside a hanging or continued line. -indent-after-paren=4 - -# String used as indentation unit. This is usually " " (4 spaces) or "\t" (1 -# tab). -indent-string=' ' - -# Maximum number of characters on a single line. -max-line-length=120 - -# Maximum number of lines in a module. -max-module-lines=1000 - -# List of optional constructs for which whitespace checking is disabled. `dict- -# separator` is used to allow tabulation in dicts, etc.: {1 : 1,\n222: 2}. -# `trailing-comma` allows a space between comma and closing bracket: (a, ). -# `empty-line` allows space-only lines. -no-space-check=trailing-comma, - dict-separator - -# Allow the body of a class to be on the same line as the declaration if body -# contains single statement. -single-line-class-stmt=no - -# Allow the body of an if to be on the same line as the test if there is no -# else. -single-line-if-stmt=no - - -[LOGGING] - -# Format style used to check logging format string. `old` means using % -# formatting, `new` is for `{}` formatting,and `fstr` is for f-strings. -logging-format-style=old - -# Logging modules to check that the string format arguments are in logging -# function parameter format. -logging-modules=logging - - -[MISCELLANEOUS] - -# List of note tags to take in consideration, separated by a comma. -notes=FIXME, - XXX, - TODO - - -[SIMILARITIES] - -# Ignore comments when computing similarities. -ignore-comments=yes - -# Ignore docstrings when computing similarities. -ignore-docstrings=yes - -# Ignore imports when computing similarities. -ignore-imports=no - -# Minimum lines number of a similarity. -min-similarity-lines=100 - - -[SPELLING] - -# Limits count of emitted suggestions for spelling mistakes. -max-spelling-suggestions=4 - -# Spelling dictionary name. Available dictionaries: none. To make it work, -# install the python-enchant package. -spelling-dict= - -# List of comma separated words that should not be checked. -spelling-ignore-words= - -# A path to a file that contains the private dictionary; one word per line. -spelling-private-dict-file= - -# Tells whether to store unknown words to the private dictionary (see the -# --spelling-private-dict-file option) instead of raising a message. -spelling-store-unknown-words=no - - -[STRING] - -# This flag controls whether the implicit-str-concat-in-sequence should -# generate a warning on implicit string concatenation in sequences defined over -# several lines. -check-str-concat-over-line-jumps=no - - -[TYPECHECK] - -# List of decorators that produce context managers, such as -# contextlib.contextmanager. Add to this list to register other decorators that -# produce valid context managers. -contextmanager-decorators=contextlib.contextmanager - -# List of members which are set dynamically and missed by pylint inference -# system, and so shouldn't trigger E1101 when accessed. Python regular -# expressions are accepted. -generated-members= - -# Tells whether missing members accessed in mixin class should be ignored. A -# mixin class is detected if its name ends with "mixin" (case insensitive). -ignore-mixin-members=yes - -# Tells whether to warn about missing members when the owner of the attribute -# is inferred to be None. -ignore-none=yes - -# This flag controls whether pylint should warn about no-member and similar -# checks whenever an opaque object is returned when inferring. The inference -# can return multiple potential results while evaluating a Python object, but -# some branches might not be evaluated, which results in partial inference. In -# that case, it might be useful to still emit no-member and other checks for -# the rest of the inferred objects. -ignore-on-opaque-inference=yes - -# List of class names for which member attributes should not be checked (useful -# for classes with dynamically set attributes). This supports the use of -# qualified names. -ignored-classes=optparse.Values,thread._local,_thread._local - -# List of module names for which member attributes should not be checked -# (useful for modules/projects where namespaces are manipulated during runtime -# and thus existing member attributes cannot be deduced by static analysis). It -# supports qualified module names, as well as Unix pattern matching. -ignored-modules= - -# Show a hint with possible names when a member name was not found. The aspect -# of finding the hint is based on edit distance. -missing-member-hint=yes - -# The minimum edit distance a name should have in order to be considered a -# similar match for a missing member name. -missing-member-hint-distance=1 - -# The total number of similar names that should be taken in consideration when -# showing a hint for a missing member. -missing-member-max-choices=1 - -# List of decorators that change the signature of a decorated function. -signature-mutators= - - -[VARIABLES] - -# List of additional names supposed to be defined in builtins. Remember that -# you should avoid defining new builtins when possible. -additional-builtins= - -# Tells whether unused global variables should be treated as a violation. -allow-global-unused-variables=yes - -# List of strings which can identify a callback function by name. A callback -# name must start or end with one of those strings. -callbacks=cb_, - _cb - -# A regular expression matching the name of dummy variables (i.e. expected to -# not be used). -dummy-variables-rgx=_+$|(_[a-zA-Z0-9_]*[a-zA-Z0-9]+?$)|dummy|^ignored_|^unused_ - -# Argument names that match this expression will be ignored. Default to name -# with leading underscore. -ignored-argument-names=_.*|^ignored_|^unused_ - -# Tells whether we should check for unused import in __init__ files. -init-import=no - -# List of qualified module names which can have objects that can redefine -# builtins. -redefining-builtins-modules=six.moves,past.builtins,future.builtins,builtins,io - - -[CLASSES] - -# List of method names used to declare (i.e. assign) instance attributes. -defining-attr-methods=__init__, - __new__, - setUp, - __post_init__ - -# List of member names, which should be excluded from the protected access -# warning. -exclude-protected=_asdict, - _fields, - _replace, - _source, - _make - -# List of valid names for the first argument in a class method. -valid-classmethod-first-arg=cls - -# List of valid names for the first argument in a metaclass class method. -valid-metaclass-classmethod-first-arg=cls - - -[DESIGN] - -# Maximum number of arguments for function / method. -max-args=5 - -# Maximum number of attributes for a class (see R0902). -max-attributes=12 - -# Maximum number of boolean expressions in an if statement (see R0916). -max-bool-expr=5 - -# Maximum number of branch for function / method body. -max-branches=12 - -# Maximum number of locals for function / method body. -max-locals=16 - -# Maximum number of parents for a class (see R0901). -max-parents=7 - -# Maximum number of public methods for a class (see R0904). -max-public-methods=20 - -# Maximum number of return / yield for function / method body. -max-returns=6 - -# Maximum number of statements in function / method body. -max-statements=50 - -# Minimum number of public methods for a class (see R0903). -min-public-methods=2 - - -[IMPORTS] - -# List of modules that can be imported at any level, not just the top level -# one. -allow-any-import-level= - -# Allow wildcard imports from modules that define __all__. -allow-wildcard-with-all=no - -# Analyse import fallback blocks. This can be used to support both Python 2 and -# 3 compatible code, which means that the block might have code that exists -# only in one or another interpreter, leading to false positives when analysed. -analyse-fallback-blocks=no - -# Deprecated modules which should not be used, separated by a comma. -deprecated-modules=optparse,tkinter.tix - -# Create a graph of external dependencies in the given file (report RP0402 must -# not be disabled). -ext-import-graph= - -# Create a graph of every (i.e. internal and external) dependencies in the -# given file (report RP0402 must not be disabled). -import-graph= - -# Create a graph of internal dependencies in the given file (report RP0402 must -# not be disabled). -int-import-graph= - -# Force import order to recognize a module as part of the standard -# compatibility libraries. -known-standard-library= - -# Force import order to recognize a module as part of a third party library. -known-third-party=enchant - -# Couples of modules and preferred modules, separated by a comma. -preferred-modules= - - -[EXCEPTIONS] - -# Exceptions that will emit a warning when being caught. Defaults to -# "BaseException, Exception". -overgeneral-exceptions=BaseException, - Exception diff --git a/kios_bt_planning/backups/BT_planning/BT_SETTINGS.yaml b/kios_bt_planning/backups/BT_planning/BT_SETTINGS.yaml deleted file mode 100644 index d932b5d0..00000000 --- a/kios_bt_planning/backups/BT_planning/BT_SETTINGS.yaml +++ /dev/null @@ -1,9 +0,0 @@ -fallback_nodes: - - 'f(' -sequence_nodes: - - 's(' -action_nodes: - - 'mv2' - - 'mv3' -up_node: - - ')' \ No newline at end of file diff --git a/kios_bt_planning/backups/BT_planning/COLCON_IGNORE b/kios_bt_planning/backups/BT_planning/COLCON_IGNORE deleted file mode 100644 index e69de29b..00000000 diff --git a/kios_bt_planning/backups/BT_planning/LICENSE b/kios_bt_planning/backups/BT_planning/LICENSE deleted file mode 100644 index 7e5d97d5..00000000 --- a/kios_bt_planning/backups/BT_planning/LICENSE +++ /dev/null @@ -1,201 +0,0 @@ - Apache License - Version 2.0, January 2004 - http://www.apache.org/licenses/ - - TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION - - 1. Definitions. - - "License" shall mean the terms and conditions for use, reproduction, - and distribution as defined by Sections 1 through 9 of this document. - - "Licensor" shall mean the copyright owner or entity authorized by - the copyright owner that is granting the License. - - "Legal Entity" shall mean the union of the acting entity and all - other entities that control, are controlled by, or are under common - control with that entity. For the purposes of this definition, - "control" means (i) the power, direct or indirect, to cause the - direction or management of such entity, whether by contract or - otherwise, or (ii) ownership of fifty percent (50%) or more of the - outstanding shares, or (iii) beneficial ownership of such entity. - - "You" (or "Your") shall mean an individual or Legal Entity - exercising permissions granted by this License. - - "Source" form shall mean the preferred form for making modifications, - including but not limited to software source code, documentation - source, and configuration files. - - "Object" form shall mean any form resulting from mechanical - transformation or translation of a Source form, including but - not limited to compiled object code, generated documentation, - and conversions to other media types. - - "Work" shall mean the work of authorship, whether in Source or - Object form, made available under the License, as indicated by a - copyright notice that is included in or attached to the work - (an example is provided in the Appendix below). - - "Derivative Works" shall mean any work, whether in Source or Object - form, that is based on (or derived from) the Work and for which the - editorial revisions, annotations, elaborations, or other modifications - represent, as a whole, an original work of authorship. For the purposes - of this License, Derivative Works shall not include works that remain - separable from, or merely link (or bind by name) to the interfaces of, - the Work and Derivative Works thereof. - - "Contribution" shall mean any work of authorship, including - the original version of the Work and any modifications or additions - to that Work or Derivative Works thereof, that is intentionally - submitted to Licensor for inclusion in the Work by the copyright owner - or by an individual or Legal Entity authorized to submit on behalf of - the copyright owner. For the purposes of this definition, "submitted" - means any form of electronic, verbal, or written communication sent - to the Licensor or its representatives, including but not limited to - communication on electronic mailing lists, source code control systems, - and issue tracking systems that are managed by, or on behalf of, the - Licensor for the purpose of discussing and improving the Work, but - excluding communication that is conspicuously marked or otherwise - designated in writing by the copyright owner as "Not a Contribution." - - "Contributor" shall mean Licensor and any individual or Legal Entity - on behalf of whom a Contribution has been received by Licensor and - subsequently incorporated within the Work. - - 2. Grant of Copyright License. Subject to the terms and conditions of - this License, each Contributor hereby grants to You a perpetual, - worldwide, non-exclusive, no-charge, royalty-free, irrevocable - copyright license to reproduce, prepare Derivative Works of, - publicly display, publicly perform, sublicense, and distribute the - Work and such Derivative Works in Source or Object form. - - 3. Grant of Patent License. Subject to the terms and conditions of - this License, each Contributor hereby grants to You a perpetual, - worldwide, non-exclusive, no-charge, royalty-free, irrevocable - (except as stated in this section) patent license to make, have made, - use, offer to sell, sell, import, and otherwise transfer the Work, - where such license applies only to those patent claims licensable - by such Contributor that are necessarily infringed by their - Contribution(s) alone or by combination of their Contribution(s) - with the Work to which such Contribution(s) was submitted. If You - institute patent litigation against any entity (including a - cross-claim or counterclaim in a lawsuit) alleging that the Work - or a Contribution incorporated within the Work constitutes direct - or contributory patent infringement, then any patent licenses - granted to You under this License for that Work shall terminate - as of the date such litigation is filed. - - 4. Redistribution. You may reproduce and distribute copies of the - Work or Derivative Works thereof in any medium, with or without - modifications, and in Source or Object form, provided that You - meet the following conditions: - - (a) You must give any other recipients of the Work or - Derivative Works a copy of this License; and - - (b) You must cause any modified files to carry prominent notices - stating that You changed the files; and - - (c) You must retain, in the Source form of any Derivative Works - that You distribute, all copyright, patent, trademark, and - attribution notices from the Source form of the Work, - excluding those notices that do not pertain to any part of - the Derivative Works; and - - (d) If the Work includes a "NOTICE" text file as part of its - distribution, then any Derivative Works that You distribute must - include a readable copy of the attribution notices contained - within such NOTICE file, excluding those notices that do not - pertain to any part of the Derivative Works, in at least one - of the following places: within a NOTICE text file distributed - as part of the Derivative Works; within the Source form or - documentation, if provided along with the Derivative Works; or, - within a display generated by the Derivative Works, if and - wherever such third-party notices normally appear. The contents - of the NOTICE file are for informational purposes only and - do not modify the License. You may add Your own attribution - notices within Derivative Works that You distribute, alongside - or as an addendum to the NOTICE text from the Work, provided - that such additional attribution notices cannot be construed - as modifying the License. - - You may add Your own copyright statement to Your modifications and - may provide additional or different license terms and conditions - for use, reproduction, or distribution of Your modifications, or - for any such Derivative Works as a whole, provided Your use, - reproduction, and distribution of the Work otherwise complies with - the conditions stated in this License. - - 5. Submission of Contributions. Unless You explicitly state otherwise, - any Contribution intentionally submitted for inclusion in the Work - by You to the Licensor shall be under the terms and conditions of - this License, without any additional terms or conditions. - Notwithstanding the above, nothing herein shall supersede or modify - the terms of any separate license agreement you may have executed - with Licensor regarding such Contributions. - - 6. Trademarks. This License does not grant permission to use the trade - names, trademarks, service marks, or product names of the Licensor, - except as required for reasonable and customary use in describing the - origin of the Work and reproducing the content of the NOTICE file. - - 7. Disclaimer of Warranty. Unless required by applicable law or - agreed to in writing, Licensor provides the Work (and each - Contributor provides its Contributions) on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or - implied, including, without limitation, any warranties or conditions - of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A - PARTICULAR PURPOSE. You are solely responsible for determining the - appropriateness of using or redistributing the Work and assume any - risks associated with Your exercise of permissions under this License. - - 8. Limitation of Liability. In no event and under no legal theory, - whether in tort (including negligence), contract, or otherwise, - unless required by applicable law (such as deliberate and grossly - negligent acts) or agreed to in writing, shall any Contributor be - liable to You for damages, including any direct, indirect, special, - incidental, or consequential damages of any character arising as a - result of this License or out of the use or inability to use the - Work (including but not limited to damages for loss of goodwill, - work stoppage, computer failure or malfunction, or any and all - other commercial damages or losses), even if such Contributor - has been advised of the possibility of such damages. - - 9. Accepting Warranty or Additional Liability. While redistributing - the Work or Derivative Works thereof, You may choose to offer, - and charge a fee for, acceptance of support, warranty, indemnity, - or other liability obligations and/or rights consistent with this - License. However, in accepting such obligations, You may act only - on Your own behalf and on Your sole responsibility, not on behalf - of any other Contributor, and only if You agree to indemnify, - defend, and hold each Contributor harmless for any liability - incurred by, or claims asserted against, such Contributor by reason - of your accepting any such warranty or additional liability. - - END OF TERMS AND CONDITIONS - - APPENDIX: How to apply the Apache License to your work. - - To apply the Apache License to your work, attach the following - boilerplate notice, with the fields enclosed by brackets "[]" - replaced with your own identifying information. (Don't include - the brackets!) The text should be enclosed in the appropriate - comment syntax for the file format. We also recommend that a - file or class name and description of purpose be included on the - same "printed page" as the copyright notice for easier - identification within third-party archives. - - Copyright [2021] [Jonathan Styrud] - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. diff --git a/kios_bt_planning/backups/BT_planning/README.md b/kios_bt_planning/backups/BT_planning/README.md deleted file mode 100644 index 98779eb6..00000000 --- a/kios_bt_planning/backups/BT_planning/README.md +++ /dev/null @@ -1,24 +0,0 @@ -# Combining planning and learning of behavior trees for robotic assembly - -This repository contains an implementation of a Genetic Programming (GP) algorithm that evolves Behavior Trees (BTs) to solve different assembly tasks. The tasks consist in picking LEGO DUPLO bricks and placing them in different configurations. The BTs are executed in the AGX Dynamics environment, but they can also be simulated in a high-level state machine without involving physics. The GP is bootstrapped by a BT created by a planner and partially solving the task (depending on the initial configurations). - -## Set-up -Put BT_SETTINGS.yaml file(s) with whatever name(s) the scripts use in your ros2 workspace root folder in order for the scripts to find them properly with relative paths. - -Build with -"colcon build --packages-select behavior_tree_learning --symlink-install" - -## Code coverage -To run code coverage, in consol run -"pytest --cov" - -## How to run the simulation -To run duplo simulation start an agx environment console and run -python agx_.py -And then in a py trees console run -ros2 run behavior_tree_learning \ - \ - \ - \ - \ - ' - - def __init__(self, tree): - if isinstance(tree, pt.trees.BehaviourTree): - self.tree = tree.root - else: - self.tree = tree - self.temp_dir = tempfile.mkdtemp() - self.html_document = os.path.join(self.temp_dir, 'bt_debug.html') - self.svg_document = os.path.join(self.temp_dir, 'tree.svg') - - dot_graph(self.tree, True).write_svg(self.svg_document, encoding='utf8') - with open(self.html_document, 'w') as f: - f.write(self.DISPLAY_HTML) - - self.__thread = threading.Thread(target=self.__open_browser) - self.__thread.start() - - def __open_browser(self): - if not webbrowser.get(f'"{self.CHROME_PATH}" %s').open('file://' + self.html_document): - webbrowser.open('file://' + self.html_document) - - def __del__(self): - while os.path.isdir(self.temp_dir): - try: - f = open(self.svg_document, encoding='utf8') - f.close() - shutil.rmtree(self.temp_dir) - except IOError: - pass - - def tick(self) -> pt.common.Status: - """Tick the tree once and display its status.""" - self.tree.tick_once() - self.update_graph() - return self.tree.status - - def tick_interactive(self): - """ - Block until the tree returns SUCCESS. - - It Lets the user tick the tree by pressing enter in the console. - """ - while self.tree.status != pt.common.Status.SUCCESS: - input('Press ENTER to tick tree') - self.tick() - - def requires_manual_closing(self) -> bool: - """Return if this object requires that the browser is closed manually.""" - return self.__thread.is_alive() - - def update_graph(self): - """Update the visualized graph.""" - dot_graph(self.tree, True).write_svg(self.svg_document, encoding='utf8') diff --git a/kios_bt_planning/backups/dynamic_bt/bt_learning/learning_from_demo/demonstration.py b/kios_bt_planning/backups/dynamic_bt/bt_learning/learning_from_demo/demonstration.py deleted file mode 100644 index 124b7d3a..00000000 --- a/kios_bt_planning/backups/dynamic_bt/bt_learning/learning_from_demo/demonstration.py +++ /dev/null @@ -1,325 +0,0 @@ -""" -Abstract class definig important aspects of the demonstration. - -- the action description -- the equivalent actions handling actions pre- and post- conditions. -""" - -# Copyright (c) 2022, ABB -# All rights reserved. -# -# Redistribution and use in source and binary forms, with -# or without modification, are permitted provided that -# the following conditions are met: -# -# * Redistributions of source code must retain the -# above copyright notice, this list of conditions -# and the following disclaimer. -# * Redistributions in binary form must reproduce the -# above copyright notice, this list of conditions -# and the following disclaimer in the documentation -# and/or other materials provided with the -# distribution. -# * Neither the name of ABB nor the names of its -# contributors may be used to endorse or promote -# products derived from this software without -# specific prior written permission. -# -# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" -# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE -# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE -# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE -# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL -# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR -# SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER -# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, -# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF -# THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - -from abc import ABC, abstractmethod -from dataclasses import dataclass -import pickle -from typing import Any, List - -import numpy as np - - -class Demonstration(ABC): - """ - The Demonstration class represents demonstrations on an abstract level. - - A subclass has to be made that can read demonstrations. - """ - - def __init__(self, default_action_type: 'Action'): - """ - Initialize a demonstration object. - - Args: - ---- - default_action_type: class of actions that do not have a specific registered type. - - """ - if not issubclass(default_action_type, Action): - raise ValueError('default_action_type has to be a subclass of Action') - self.__default_action_type = default_action_type - - # Dictionary of action types with a custom action class - self.__registered_classes = {} - - def register_action(self, action_type: Any, action_class: Any): - """Register actions of type "action_type" to be instances of action_class.""" - self.__registered_classes[action_type] = action_class - - def make_action(self, action_type: str, *args, **kwargs) -> 'Action': - """ - Create and return an instance of the class corresponding to action type. - - The constructor takes *args and **kwargs. - """ - action_class = self.__registered_classes.get(action_type, self.__default_action_type) - return action_class(*args, **kwargs) - - @abstractmethod - def demonstrations(self): - """Return a list of demos where each demo is in turn a list of actions.""" - pass - - def n_demonstrations(self) -> int: - """Return the number of demonstrations.""" - return len(self.demonstrations()) - - def all_actions(self) -> List[str]: - """Return a list of all actions without reference to the demonstration they belong to.""" - actions = [] - for demo in self.demonstrations(): - actions += demo - - return actions - - -class Action(ABC): - """ - Represent a single action as demonstrated. - - The Action class does not represent a collection of equivalent actions. - """ - - def __init__(self, action_type: str, default_frame: List[str]): - """ - Instantiate a new Action. - - This constructor has to be called byt the subclass constructor. - - Args: - ---- - - action_type: action type. - - default_frame: is a list of default frames (strings, one for each target). - - Important attributes to override: - -------------------------------- - - _parameters: list of action-specific parameters. Defaults to []. - - _exclude_frames: list of frames to exclude when inferring similar actions. - Defaults to []. - - _n_targets: the number of targets. Defaults to len(default_frame). - - _has_heuristic: whether this action can determine reference frames with a heuristic. - Defaults to False. - - Other attributes: - ---------------- - - type: string with the action type. - - frame: list of frames (strings). One for each target. - - """ - self.type = action_type - self.frame = default_frame - if not hasattr(self, 'parameters'): - self.parameters = [] - if not hasattr(self, 'exclude_frames'): - self.exclude_frames = [] - if not hasattr(self, 'n_targets'): - self.n_targets = len(default_frame) - if not hasattr(self, 'has_heuristic'): - self.has_heuristic = False - - @abstractmethod - def target_position(self, frame: str, i: int) -> np.ndarray: - """Return the target position in the given frame.""" - pass - - @abstractmethod - def target_orientation(self, frame: str, i: int) -> np.ndarray: - """Return the target orientation in the given frame.""" - pass - - def heuristic(self) -> None: - """ - Override this in a subclass to infer properties about the action using heuristics. - - This function is typically where paramters are set. - If has_heuristic is True, it is assumed that this function infers reference frame - and all actions of this type are assumed equivalent if the parameters and frame are equal. - Even though has_heuristic is False this function can be overriden to set parameters. - """ - pass - - def get_excluded_frames(self) -> List[str]: - """Override this in a subclass to personalize the frames to exclude during clustering.""" - pass - - def __eq__(self, other) -> bool: - """Return whether this Action is equal to another (gvein as input parameter).""" - try: - np.testing.assert_equal(self.type, other.type) - np.testing.assert_equal(self.parameters, other.parameters) - np.testing.assert_equal(self.frame, other.frame) - for target, frame in enumerate(self.frame): - np.testing.assert_equal( - self.target_position(frame, target), - other.target_position(frame, target) - ) - np.testing.assert_equal( - self.target_orientation(frame, target), - other.target_orientation(frame, target) - ) - return True - except AssertionError: - return False - - def __neq__(self, other) -> bool: - return not self.__eq__(other) - - -class EquivalentAction(ABC): - """ - Represent an equivalence class of actions as determined by the clustering. - - Attributes - ---------- - - targets: [n_targets x 3] array with mean targets. - It is possible to have multiple targets (NOT USED AT THE MOMENT) - - target_orientations: [n_targets] quaternion array of mean target orientations. - - max_distance: [n_targets] array with the maximal euclidean distance to targets observed. - - frames: list of reference frames for each target. - - actions: list of Action that are considered equivalent. - - additional_preconditions: list of Condition that represent additional preconditions. - - """ - - # List of used action names to give each action a unique name - __names = {} - - def __init__(self, actions: List[Action]): - """ - Construct a new instance of EquivalentAction. - - Args: - ---- - - actions: list of Action containing equivalent actions. - - """ - self.actions = actions - self.additional_preconditions = [] - - if actions[0].type in self.__names: - EquivalentAction.__names[actions[0].type] += 1 - else: - EquivalentAction.__names[actions[0].type] = 0 - self.name = actions[0].type + str(EquivalentAction.__names[actions[0].type]) - - # Don't do any more calculations if there are no targets - if actions[0].n_targets == 0: - self.targets = None - self.target_orientations = None - self.max_distance = None - self.frames = None - return - - self.frames = actions[0].frame - - # Combine all targets into a n_actions x n_targets x 3 (or 2) array of target positions - # and a n_actions x n_targets array of target orientations - all_targets = np.zeros((len(actions), actions[0].n_targets, 3)) - all_orientations = np.zeros((len(actions), actions[0].n_targets), dtype=np.ndarray) - for action_idx, action in enumerate(actions): - for target in range(actions[0].n_targets): - all_targets[action_idx, target, :] =\ - action.target_position(self.frames[target], target) - all_orientations[action_idx, target] =\ - action.target_orientation(self.frames[target], target) - - # Compute mean position - # Shape: n_targets x 3 - self.targets = np.mean(all_targets, axis=0) - self.targets = np.round(self.targets, 3) - # Shape: n_actions x n_targets - distances = np.linalg.norm(all_targets - self.targets, axis=2) - # max distance of a demonstrated action to the mean - # TODO: (use to compute action tolerance) --> NOT USED (this was called spread!) - # Shape: n_targets - self.max_distance = np.max(distances, axis=0) - self.max_distance = np.round(self.max_distance, 3) - - # Compute mean orientation - # TODO: now orientation is not used to determine the target - self.target_orientations = all_orientations[0] - - @abstractmethod - def preconditions(self) -> List[str]: - """Return a list of strings corresponding to pre-conditions.""" - pass - - def preconditions_with_additional(self) -> List[str]: - """ - Return a list of strings with this action's pre-conditions. - - It includes those specified in additional_preconditions - which are present in the beginning of the returned list. - """ - return self.preconditions() + self.additional_preconditions - - @abstractmethod - def postconditions(self) -> List[str]: - """Return a list of strings corresponding to post-conditions.""" - pass - - @abstractmethod - def action_string(self) -> str: - """Return a string representing this action for use with the planner.""" - pass - - def save_actions(self, directory: str): - """Save the list of demonstrated actions as a pickle file in the given directory.""" - with open(directory + '/' + self.name + '.pkl', 'wb') as f: - pickle.dump( - ActionInfo( - actions=self.actions, - additional_preconditions=self.additional_preconditions, - equivalent_action=self), - f - ) - - def can_form_constraint(self) -> bool: - """Return True if this action can be a part of a constraint.""" - return True - - def __repr__(self) -> str: - return self.action_string() - - -@dataclass -class ActionInfo: - """ - Information about an EquivalentAction. - - Attributes - ---------- - - actions: is the list of lfd.demonstration.Action that makes up the EquivalentAction. - - additional_preconditions: is a list of additional preconditions. - - equivalent_action: is the EquivalentAction. - - """ - - actions: List[Action] - additional_preconditions: list - equivalent_action: EquivalentAction diff --git a/kios_bt_planning/backups/dynamic_bt/bt_learning/learning_from_demo/goal_identification.py b/kios_bt_planning/backups/dynamic_bt/bt_learning/learning_from_demo/goal_identification.py deleted file mode 100644 index c8ac2d03..00000000 --- a/kios_bt_planning/backups/dynamic_bt/bt_learning/learning_from_demo/goal_identification.py +++ /dev/null @@ -1,93 +0,0 @@ -"""Module containing methods that allow to identify task goals.""" - -# Copyright (c) 2022, ABB -# All rights reserved. -# -# Redistribution and use in source and binary forms, with -# or without modification, are permitted provided that -# the following conditions are met: -# -# * Redistributions of source code must retain the -# above copyright notice, this list of conditions -# and the following disclaimer. -# * Redistributions in binary form must reproduce the -# above copyright notice, this list of conditions -# and the following disclaimer in the documentation -# and/or other materials provided with the -# distribution. -# * Neither the name of ABB nor the names of its -# contributors may be used to endorse or promote -# products derived from this software without -# specific prior written permission. -# -# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" -# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE -# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE -# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE -# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL -# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR -# SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER -# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, -# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF -# THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - -from typing import Any, List - -from behaviors.common_behaviors import RSequence -from bt_learning.planner.constraints_identification import contains_conflicting -from bt_learning.learning_from_demo.demonstration import Demonstration -from py_trees.trees import BehaviourTree - - -def goal_conditions_for_demo( - demo: Demonstration, - behaviors: Any -) -> List[str]: - """ - Infer the goal conditions of a single demonstration. - - Args - ---- - demo: the demonstration to infer the goal of. - behavior: check the behavior to remove conflicting conditions. - - Returns - ------- - goals: list of the goals inferred in the demonstration. - - """ - goals = [] - for i in range(len(demo)-1, -1, -1): - for condition in demo[i].postconditions(): - if condition not in goals and not contains_conflicting(behaviors, goals, condition): - goals.append(condition) - - goals.reverse() - return goals - - -def goal_tree( - goals: List[str], - behaviors: Any, - world_interface: Any -) -> BehaviourTree: - """ - Construct a Behavior Tree strarting from the goals. - - Args - ---- - goals: list of all goals inferred from the demonstration. - behaviors: behavior in the demontration, as defined in robot_behaviors package. - world_interface: interface to the robot. - - Returns - ------- - tree: a Behavior Tree of goal conditions. - - """ - tree = RSequence() - for goal in goals: - node, _ = behaviors.get_node(goal, world_interface, None) - tree.add_child(node) - - return tree diff --git a/kios_bt_planning/backups/dynamic_bt/bt_learning/learning_from_demo/learning.py b/kios_bt_planning/backups/dynamic_bt/bt_learning/learning_from_demo/learning.py deleted file mode 100644 index a3761a73..00000000 --- a/kios_bt_planning/backups/dynamic_bt/bt_learning/learning_from_demo/learning.py +++ /dev/null @@ -1,214 +0,0 @@ -"""Methods that allow to learn and build a BT from demonstration.""" - -# Copyright (c) 2022, ABB -# All rights reserved. -# -# Redistribution and use in source and binary forms, with -# or without modification, are permitted provided that -# the following conditions are met: -# -# * Redistributions of source code must retain the -# above copyright notice, this list of conditions -# and the following disclaimer. -# * Redistributions in binary form must reproduce the -# above copyright notice, this list of conditions -# and the following disclaimer in the documentation -# and/or other materials provided with the -# distribution. -# * Neither the name of ABB nor the names of its -# contributors may be used to endorse or promote -# products derived from this software without -# specific prior written permission. -# -# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" -# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE -# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE -# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE -# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL -# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR -# SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER -# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, -# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF -# THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - -import logging -import os -from typing import Any, List - -from bt_learning.planner.constraints_identification import infer_preconditions -from bt_learning.learning_from_demo.demonstration import Demonstration -import bt_learning.learning_from_demo.lfd_behaviors as lfd_bt -from bt_learning.learning_from_demo.goal_identification import goal_conditions_for_demo, goal_tree -import bt_learning.planner.lfd_planner as planner -import py_trees as pt -from py_trees.trees import BehaviourTree -from simulation.py_trees_interface import PyTree, PyTreeParameters -import yaml - - -logger = logging.getLogger('learning') - - -def prepare_bt_settings( - directory_path: str, - demonstrations: List[Demonstration], - behaviors: lfd_bt.Behaviors -): - """ - Create the directory and populate it with settings and pickle files for the planner. - - Args: - ---- - directory_path: path to the directory to create. - demonstrations: is a list of demonstrations. - behaviors: is a Behaviors object. - - """ - if not os.path.exists(directory_path): - os.mkdir(directory_path) - - actions = behaviors.get_actions(demonstrations) - conditions = behaviors.get_conditions(demonstrations) - - for demo in demonstrations: - for action in demo: - if action.action_string() not in actions: - action.save_actions(directory_path) - actions.append(action.action_string()) - - for condition in action.preconditions_with_additional() + action.postconditions(): - - if condition not in conditions: - conditions.append(condition) - - settings = { - 'fallback_nodes': ['f('], - 'sequence_nodes': ['s('], - 'condition_nodes': conditions, - 'action_nodes': actions, - 'up_node': [')'] - } - - with open(directory_path + '/BT_SETTINGS.yaml', 'w') as f: - yaml.dump(settings, f) - - -def __plan( - behaviors: lfd_bt.Behaviors, - world_interface: Any, - goals: List[str], - iterations: int -) -> BehaviourTree: - """ - Plan a behavior tree. - - Args - ---- - behaviors: type of behaviors in the demonstration. - world_interface: is an object that is passed to each behavior during execution. - goals: is a list of goal conditions (strings) or None. - If it is None, the function tries to guess the goal conditions. - iterations: if iterations > 1, world_interface must define a reset() function. - - Returns - ------- - tree: the Behavior Tree. - - """ - tree = goal_tree(goals, behaviors, world_interface) - - for i in range(iterations): - world_interface.reset() - tree = planner.plan(world_interface, behaviors, tree) - - return tree - - -def learn_tree( - directory_path: str, - demonstrations: List[Demonstration], - behaviors: lfd_bt.Behaviors, - world_interface: Any, - iterations: int = 20, - callback: Any = None -) -> PyTree: - """ - Learns a behavior tree from demonstrations. - - Args: - ---- - directory_path: is the path to a directory where settings and resulting tree are saved. - demonstrations: is a list of demonstrations. - behaviors: type of behaviors in the demonstration. - world_interface: is an object passed to each behavior during execution for planning. - iterations: is the number of times that the planner is run for each subtask - If iterations > 1, world_interface must define a reset() function. - callback: optional callback function counting the planning steps. - - If callback is given, it is called with two arguments after each planning step: - - planning step number. - - max planning step number. - - """ - # Find goal of each demonstration - goals = list(map( - lambda demo: (demo, goal_conditions_for_demo(demo, behaviors)), demonstrations)) - - # Group demonstration by goal condition (subtask) - subtasks = [] - subtask_goals = [] - while len(goals) > 0: - demo, goal_conditions = goals.pop() - subtask = [demo] - subtask_goals.append(goal_conditions) - for i in range(len(goals)-1, -1, -1): - if set(goal_conditions) == set(goals[i][1]): - other_demo, _ = goals.pop(i) - subtask.append(other_demo) - - subtasks.append(subtask) - - total_iterations = len(subtask_goals) - iteration = 0 - # Find constraints for each subtask and plan - subtask_trees = [] - for subtask, goals in zip(subtasks, subtask_goals): - infer_preconditions(subtask, behaviors) - - prepare_bt_settings(directory_path, demonstrations, behaviors) - - tree = __plan(behaviors, world_interface, goals, iterations) - subtask_trees.append(tree) - - iteration += 1 - - if callback is not None: - callback(iteration, total_iterations-1) - - # Reset inferred preconditions - for demo in subtask: - for action in demo: - action.additional_preconditions.clear() - - # Build a tree of all subtasks - # Memory fallback so that if one solution doesn't work, another is attempted. - - root = pt.composites.Selector('Fallback', memory=True) - for subtask in subtask_trees: - root.add_child(subtask) - - # Resolve conflicts resulting from pasting the trees together - planner.handle_priority(root, behaviors) - - if len(root.children) == 1: - # Only one child so root is unnecessary. Replace with child. - root = root.children[0] - root.parent = None - - parameters = PyTreeParameters() - parameters.behavior_lists = behaviors.get_behavior_list() - parameters.behaviors = behaviors - - tree = PyTree('', parameters, world_interface=world_interface, root=root) - - return tree diff --git a/kios_bt_planning/backups/dynamic_bt/bt_learning/learning_from_demo/learning_postprocessing.py b/kios_bt_planning/backups/dynamic_bt/bt_learning/learning_from_demo/learning_postprocessing.py deleted file mode 100644 index 2b70cb6c..00000000 --- a/kios_bt_planning/backups/dynamic_bt/bt_learning/learning_from_demo/learning_postprocessing.py +++ /dev/null @@ -1,228 +0,0 @@ -"""Methods that allow to modify a learnt tree.""" - -# Copyright (c) 2022, ABB -# All rights reserved. -# -# Redistribution and use in source and binary forms, with -# or without modification, are permitted provided that -# the following conditions are met: -# -# * Redistributions of source code must retain the -# above copyright notice, this list of conditions -# and the following disclaimer. -# * Redistributions in binary form must reproduce the -# above copyright notice, this list of conditions -# and the following disclaimer in the documentation -# and/or other materials provided with the -# distribution. -# * Neither the name of ABB nor the names of its -# contributors may be used to endorse or promote -# products derived from this software without -# specific prior written permission. -# -# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" -# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE -# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE -# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE -# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL -# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR -# SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER -# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, -# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF -# THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - -from copy import copy, deepcopy -import logging -import math -import os -from typing import Any, List, Tuple - -from behaviors.behavior_lists import BehaviorLists -from bt_learning.learning_from_demo.demonstration import Demonstration -import numpy.random as prob -import random -import yaml - -# Notation [(Xmin, Xmax), (Ymin, Ymax), (Zmin, Zmax)] -WORKSPACE = [(0.1, 0.5), (-0.5, 0.5), (0.002, 0.4)] - - -logger = logging.getLogger('learning_postprocessing') - - -def update_bt_settings( - file_path: str, - demonstrations: Demonstration, - behaviors: BehaviorLists -) -> None: - """Update the BT settings with the added behaviors.""" - file_name = file_path + '/BT_SETTINGS.yaml' - with open(file_name) as f: - data = yaml.safe_load(f) - - data_actions = data['action_nodes'] - data_conditions = data['condition_nodes'] - - actions = behaviors.get_actions(demonstrations) - conditions = behaviors.get_conditions(demonstrations) - - for action in actions: - if action not in data_actions: - data_actions.append(action) - for condition in conditions: - if condition not in data_conditions: - data_conditions.append(condition) - - settings = { - 'fallback_nodes': ['f('], - 'sequence_nodes': ['s('], - 'condition_nodes': data_conditions, - 'action_nodes': data_actions, - 'up_node': [')'] - } - - with open(file_name, 'w') as f: - yaml.dump(settings, f) - - -def add_synthetic_demos( - file_path: str, - action_description: dict = {'grasp': [], 'place': [], 'move': []}, - target_demo: int = 1, - samples: int = 2, - dialogue_menu: Any = None, - workspace: List[Tuple[float, float]] = WORKSPACE -) -> None: - """Load an existing demonstration to generate 2 synthetic demonstrations.""" - demo_to_replicate = os.path.join(file_path, 'demo%d' % (target_demo)) - - synthetic_data = [] - # this frame is demonstration specific: reset only at the beginning of generation method - pick_target_frame = None - - # ------ Load frames to exclude ------ - file_name = os.path.join(file_path, 'exclude_frames.yaml') - with open(file_name) as f: - excluded_frames = yaml.safe_load(f) - - # ------ Load demo ------ - # Count how many data files are there --> number of actions performed in a single demo - n_files = len([name for name in os.listdir(demo_to_replicate)]) - for i in range(n_files): - file_name = os.path.join(demo_to_replicate, 'data_%d.yaml' % (i + 1)) - with open(file_name) as f: - data = yaml.safe_load(f) - - # ----------- Inspect data ------------ - # Compute the distance of the recorded positions to find the most meaningful ref. frame - target_ref_frame = None - move_frames = [] - min_distance = float('inf') - for ref_frame in data['vec_pos']: - # Skip the frame that is excluded from the distance computation - if ref_frame in excluded_frames[data['type']]: - continue - position = data['vec_pos'][ref_frame] - # distance to the end effector of the robot - distance3 = math.sqrt(sum(x*x for x in position)) - distance2 = math.sqrt(sum(x*x for x in position[0:2])) - if distance3 < min_distance: - # In case of pick we consider the shortest distance - if data['type'] in action_description['grasp']: - min_distance = distance3 - target_ref_frame = deepcopy(ref_frame) - pick_target_frame = deepcopy(ref_frame) - # In case of place we disregard the ref_frame of the object held - elif data['type'] in action_description['place'] and\ - ref_frame != pick_target_frame: - min_distance = distance3 - target_ref_frame = deepcopy(ref_frame) - if distance2 < 1.5 and data['type'] in action_description['move']: - move_frames.append(ref_frame) - - # disambiguate move action - if len(move_frames) == 1: - target_ref_frame = deepcopy(move_frames[0]) - if target_ref_frame is None: - # this means that we inspected a move action - if dialogue_menu is not None: - target_ref_frame = disambiguate_navigation( - move_frames, deepcopy(dialogue_menu), i+1) - else: - target_ref_frame = random.choice(move_frames) - - # ------ Generate synthetic data ------ - for j in range(samples): - new_data = copy(data) - for ref_frame in data['vec_pos']: - nav = True if data['type'] in action_description['move'] else False - # Sample uniformly in a given interval. - sample = sampling_heuristic( - new_data, ref_frame, target_ref_frame, pick_target_frame, workspace, nav) - new_data['vec_pos'][ref_frame] = sample - # DO NOT CHANGE ORIENTATION - - # print(f'New Data:\n {new_data}') - synthetic_data.append(deepcopy(new_data)) - - # ------ Save demo ------ - folder_nr = target_demo - for i in range(samples): - folder_exists = True - while folder_exists: - folder_nr += 1 - new_demo_folder = os.path.join(file_path, 'demo%d' % (folder_nr)) - folder_exists = os.path.isdir(new_demo_folder) - - os.mkdir(new_demo_folder) - - for j in range(n_files): - new_demo_file = os.path.join(new_demo_folder, 'data_%d.yaml' % (j + 1)) - # print(f'Dumping:\n {synthetic_data[i*samples + j]}') - with open(new_demo_file, 'w') as f: - yaml.dump(synthetic_data[i + j*samples], f, default_flow_style=None) - - -def disambiguate_navigation(move_frames: List[str], dialogue_menu: Any, idx: int = 0) -> str: - """Disambiguate the navigation action using human input.""" - # check properties of the menu - dialogue_menu.set_title( - f'Move action {idx} ambiguous! Select the desired target reference frame.') - dialogue_menu.set_options(move_frames) - # run menu - key_ = dialogue_menu.initialize_layout() - window = dialogue_menu.get_window() - event, values = window.read() - disambiguated_frame = values[key_] - window.close() - - return disambiguated_frame - - -def sampling_heuristic( - data: dict, - input_frame: str, - action_target_frame: str, - pick_object: str, - workspace: List[Tuple[float, float]], - navigation: bool = False -) -> List[float]: - """Heuristic for determining the variation to add to the data in case of manipulation.""" - # The default frame is always the first of the list - sample = [None, None, None] - data_position = data['vec_pos'][input_frame] - low_ = 0.01 if navigation else 0.001 - high_ = 0.05 if navigation else 0.01 - - if input_frame == action_target_frame or input_frame == pick_object: - sample[0] = prob.uniform(low=low_, high=high_)*prob.choice((-1, +1)) + data_position[0] - sample[1] = prob.uniform(low=low_, high=high_)*prob.choice((-1, +1)) + data_position[1] - else: - sample[0] = prob.uniform(low=workspace[0][0], high=workspace[0][1]) - sample[1] = prob.uniform(low=workspace[1][0], high=workspace[1][1]) - - # the height is not very important so we do not variate much - sample[2] = prob.uniform(low=0.0, high=0.005) + data_position[2] - sample = [float(round(num, 3)) for num in sample] - - return sample diff --git a/kios_bt_planning/backups/dynamic_bt/bt_learning/learning_from_demo/lfd_behaviors.py b/kios_bt_planning/backups/dynamic_bt/bt_learning/learning_from_demo/lfd_behaviors.py deleted file mode 100644 index 6bb17467..00000000 --- a/kios_bt_planning/backups/dynamic_bt/bt_learning/learning_from_demo/lfd_behaviors.py +++ /dev/null @@ -1,161 +0,0 @@ -"""Extension of the Behaviors class with elements required for the LfD framework.""" - -# Copyright (c) 2022, ABB -# All rights reserved. -# -# Redistribution and use in source and binary forms, with -# or without modification, are permitted provided that -# the following conditions are met: -# -# * Redistributions of source code must retain the -# above copyright notice, this list of conditions -# and the following disclaimer. -# * Redistributions in binary form must reproduce the -# above copyright notice, this list of conditions -# and the following disclaimer in the documentation -# and/or other materials provided with the -# distribution. -# * Neither the name of ABB nor the names of its -# contributors may be used to endorse or promote -# products derived from this software without -# specific prior written permission. -# -# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" -# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE -# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE -# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE -# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL -# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR -# SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER -# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, -# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF -# THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - -from abc import ABC, abstractmethod -import logging -from typing import Any, List, Tuple - -from behaviors.behavior_lists import BehaviorLists, ParameterizedNode -import behaviors.common_behaviors as cb -from bt_learning.learning_from_demo.demonstration import Demonstration -import yaml - - -logger = logging.getLogger("lfd_behaviors") - - -class Behaviors(ABC): - """Represent classes that parse behavior strings and return behavior tree representations.""" - - def __init__(self, directory_path: str): - """Directory_path is the path to the directory where planner settings are saved.""" - self.directory_path = directory_path - - def get_behavior_list(self) -> BehaviorLists: - """Return a Behavior List from parsing a yaml file.""" - # initialize the dictionary an populate it while parsing the yaml - condition_nodes = [] - action_nodes = [] - - file = self.directory_path + "/BT_SETTINGS.yaml" - if file is not None: - try: - with open(file) as f: - bt_settings = yaml.load(f, Loader=yaml.FullLoader) - # print('Loading BT settings.') - except FileNotFoundError: - print("BT settings not ready.") - print("File " + file + " not found.") - return None - try: - node_name = bt_settings["condition_nodes"] - for node in node_name: - condition_nodes.append(node) - except KeyError: - pass - try: - node_name = bt_settings["action_nodes"] - for node in node_name: - action_nodes.append(node) - except KeyError: - pass - - behavior_list = BehaviorLists( - condition_nodes=condition_nodes, action_nodes=action_nodes - ) - - return behavior_list - - @abstractmethod - def get_node( - self, - node: str or ParameterizedNode, - world_interface: Any, - condition_parameters: Any, - ) -> Tuple[Any, bool]: - """ - Get a node from the behavior tree from its string representation. - - Args - ---- - node: ParametrizedNode or string representation of the node. - world_itnterface: interface to the robot. - condition_parameters: list of the condition for the node. - - Returns - ------- - node: the node of the Behavior Tree. - has_children: if the node has children then it is a control node. - - """ - has_children = False - node, has_children = cb.get_node(node) - - return node, has_children - - def get_condition_parameters(self, condition: str) -> List[Any]: - """Return a list of parameters associated with condition.""" - pass - - @abstractmethod - def get_actions(self, demonstrations: List[Demonstration]) -> List[Any]: - """ - Return a list of available actions not necessarily included in the demonstration. - - Args: - ---- - demonstrations: is a list of lists of EquivalentAction. - - """ - pass - - @abstractmethod - def get_conditions(self, demonstrations: List[Demonstration]) -> List[Any]: - """ - Return a list of available conditions not necessarily included in the demonstration. - - Args: - ---- - demonstrations: is a list of lists of EquivalentAction. - - """ - pass - - @abstractmethod - def compatible(self, condition1: str, condition2: str) -> bool: - """ - Determine if two conditions are compatible. - - It must satisfy compatible(condition1, condition2) == compatible(condition2, condition1). - - Args - ---- - condition1: string representing the first condition. - condition2: string representing the second condition. - - Returns - ------- - True if condition1 and condition2 are compatible, i.e. can be True at the same time. - - """ - pass diff --git a/kios_bt_planning/backups/dynamic_bt/bt_learning/learning_from_demo/plot_clusters.py b/kios_bt_planning/backups/dynamic_bt/bt_learning/learning_from_demo/plot_clusters.py deleted file mode 100644 index 281ffcf3..00000000 --- a/kios_bt_planning/backups/dynamic_bt/bt_learning/learning_from_demo/plot_clusters.py +++ /dev/null @@ -1,119 +0,0 @@ -"""This module contains a function to plot the clusters generated for frame inference.""" - -# Copyright (c) 2022, ABB -# All rights reserved. -# -# Redistribution and use in source and binary forms, with -# or without modification, are permitted provided that -# the following conditions are met: -# -# * Redistributions of source code must retain the -# above copyright notice, this list of conditions -# and the following disclaimer. -# * Redistributions in binary form must reproduce the -# above copyright notice, this list of conditions -# and the following disclaimer in the documentation -# and/or other materials provided with the -# distribution. -# * Neither the name of ABB nor the names of its -# contributors may be used to endorse or promote -# products derived from this software without -# specific prior written permission. -# -# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" -# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE -# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE -# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE -# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL -# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR -# SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER -# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, -# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF -# THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - -import numpy as np -import matplotlib.pyplot as plt -from matplotlib.colors import hsv_to_rgb -import math -import sys -from typing import Any - -import bt_learning.learning_from_demo.clustering as clustering -# Import this to allow 3D plot -from mpl_toolkits.mplot3d import axes3d # noqa: F401 - -import logging -logging.getLogger('matplotlib').setLevel(logging.WARNING) - - -def plot_clusters(demo_dir: str, demonstration: Any, show: bool = False) -> None: - """Plot the clusters of the learning framework.""" - try: - partition = clustering.initial_partition(demonstration.all_actions()) - except AttributeError: - print('The given object has no method "all_actions()".') - sys.exit(1) - - try: - n_frames = len(demonstration.frames) - except AttributeError: - print('The given object has no attribute "frames".') - sys.exit(1) - n_rows = math.floor(math.sqrt(n_frames)) - for i, group in enumerate(partition): - fig = plt.figure(i) - fig.suptitle(f'Action group {i}: {group[0].type}({group[0].parameters})') - - try: - equivalent = clustering.cluster( - group, demonstration.frames, 0, len(demonstration.demonstrations())) - except AttributeError: - print('The given object has no method "demonstrations()".') - sys.exit(1) - - for j, frame in enumerate(demonstration.frames): - ax = fig.add_subplot(n_rows, math.ceil(n_frames/n_rows), j+1, projection='3d') - ax.set_title(frame) - for action in group: - # Find action - equivalent_grouping = next(x for x in equivalent if action in x) - idx = equivalent.index(equivalent_grouping) - color = hsv_to_rgb( - [idx/len(equivalent), 1, idx/len(equivalent)+0.1]).reshape((1, 3)) - - if frame == action.frame[0]: - marker = 'x' - else: - marker = 'o' - - position = action.target_position(frame) - scaled_position = [x*100 if x < 1 else x for x in position] - unit = [' [cm]' if x < 1 else ' [m]' for x in position] - # format the position values - formatted_val = [] - for val in scaled_position: - formatted_val.append(float(np.format_float_positional( - val, precision=3, unique=False, fractional=False, trim='k'))) - # plot - ax.scatter(scaled_position[0], scaled_position[1], - scaled_position[2], c=color, marker=marker) - # Uncomment if it is desired to display axis labels. Not recommended. - ax.set_xlabel('x' + unit[0], fontsize=9, labelpad=-18, color='r') - ax.set_ylabel('y' + unit[0], fontsize=9, labelpad=-18, color='r') - ax.set_zlabel('z' + unit[0], fontsize=9, labelpad=-18, color='r') - # Uncomment if it is desired to remove the values of the samples - # ax.set_xticklabels([]) - # ax.set_yticklabels([]) - # ax.set_zticklabels([]) - - # set the spacing between subplots - plt.subplots_adjust(left=0.1, bottom=0.1, right=0.9, top=0.9, wspace=0.6, hspace=0.4) - - format = 'svg' - name = demo_dir + '/clusters_' + str(i) + '.' + format - plt.savefig(name, format=format) - - if show: - plt.show() - - plt.close() diff --git a/kios_bt_planning/backups/dynamic_bt/bt_learning/learning_from_demo/render_simplified.py b/kios_bt_planning/backups/dynamic_bt/bt_learning/learning_from_demo/render_simplified.py deleted file mode 100644 index 1abb1a95..00000000 --- a/kios_bt_planning/backups/dynamic_bt/bt_learning/learning_from_demo/render_simplified.py +++ /dev/null @@ -1,105 +0,0 @@ -"""Methods to render a shrinked version of the BT.""" - -# Copyright (c) 2022, ABB -# All rights reserved. -# -# Redistribution and use in source and binary forms, with -# or without modification, are permitted provided that -# the following conditions are met: -# -# * Redistributions of source code must retain the -# above copyright notice, this list of conditions -# and the following disclaimer. -# * Redistributions in binary form must reproduce the -# above copyright notice, this list of conditions -# and the following disclaimer in the documentation -# and/or other materials provided with the -# distribution. -# * Neither the name of ABB nor the names of its -# contributors may be used to endorse or promote -# products derived from this software without -# specific prior written permission. -# -# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" -# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE -# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE -# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE -# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL -# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR -# SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER -# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, -# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF -# THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - -from copy import copy -from typing import Any - -from behaviors.common_behaviors import RandomSelector, RSequence -import bt_learning.learning_from_demo.render_tree as rt -import py_trees as pt - - -def write_simplified_svg(tree: Any, file: str): - simplified_figure(tree, file, lambda t, f: rt.dot_graph(t).write_svg(f, encoding='utf8')) - - -def write_simplified_tikz(tree: Any, file: str): - simplified_figure(tree, file, rt.write_tikz_tree) - - -def write_simplified_py_trees(tree: Any, file: str): - simplified_figure(tree, file, lambda t, f: rt.py_trees_dot(t).write_svg(f)) - - -def simplified_figure( - tree: Any, - file: str, - generation_method: Any -): - """ - Write an image of a simplified BT where only place nodes are present. - - Args: - ---- - tree: the Behavior Tree to save as image. - file: path with the name of the image. - generation_method: function that creates the image of the tree and saves it in file. - - """ - if isinstance(tree, pt.trees.BehaviourTree): - tree = tree.root - - __collapse_nodes(tree) - - if len(tree.children) == 1: - # If only one child replace with child - tree = tree.children[0] - - generation_method(tree, file) - - -def __collapse_nodes(tree: Any): - if isinstance(tree, RSequence): - children = copy(tree.children) - for child in children: - __collapse_nodes(child) - if len(tree.children) == 1 and tree.parent is not None: - tree.parent.replace_child(tree, tree.children[0]) - elif isinstance(tree, pt.composites.Selector) or isinstance(tree, RandomSelector): - name = tree.children[0].name - if not (name.startswith('object_at') or name.startswith('object_roughly_at')) and\ - tree.parent is not None: - tree.parent.remove_child(tree) - else: - for child in tree.children: - __collapse_nodes(child) - elif not (tree.name.startswith('place') or - tree.name.startswith('object_at') or - tree.name.startswith('object_roughly_at')): - tree.parent.remove_child(tree) - - -class LeafNode(pt.behaviour.Behaviour): - - def __init__(self, name: str): - super().__init__(name) diff --git a/kios_bt_planning/backups/dynamic_bt/bt_learning/learning_from_demo/render_tree.py b/kios_bt_planning/backups/dynamic_bt/bt_learning/learning_from_demo/render_tree.py deleted file mode 100644 index 7689f113..00000000 --- a/kios_bt_planning/backups/dynamic_bt/bt_learning/learning_from_demo/render_tree.py +++ /dev/null @@ -1,279 +0,0 @@ -"""Methods to render the BT.""" - -# Copyright (c) 2022, ABB -# All rights reserved. -# -# Redistribution and use in source and binary forms, with -# or without modification, are permitted provided that -# the following conditions are met: -# -# * Redistributions of source code must retain the -# above copyright notice, this list of conditions -# and the following disclaimer. -# * Redistributions in binary form must reproduce the -# above copyright notice, this list of conditions -# and the following disclaimer in the documentation -# and/or other materials provided with the -# distribution. -# * Neither the name of ABB nor the names of its -# contributors may be used to endorse or promote -# products derived from this software without -# specific prior written permission. -# -# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" -# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE -# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE -# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE -# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL -# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR -# SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER -# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, -# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF -# THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - -import re -from typing import Any - -from behaviors.common_behaviors import RandomSelector, RSequence -from behaviors.common_behaviors import ActionBehavior -import py_trees as pt -import pydot - - -_NUMBER_REGEX = r'-?\d+(?:.\d+)?(?:e-?\d+)?' - - -class __TreeState: - - regex = re.compile(f'\\({_NUMBER_REGEX}, {_NUMBER_REGEX}, {_NUMBER_REGEX}\\)') - - def __init__(self, short_names: bool): - self.__number = 0 - self.__position_number = 1 - self.__positions = {} - self.__short_names = short_names - - def get_number(self): - self.__number += 1 - return self.__number - - def get_positions(self): - return self.__positions - - def get_name(self, name: str, latex: bool = False): - if not self.__short_names: - return name - - position = self.regex.search(name) - if position is None: - return name - position = position[0] - - if position in self.__positions: - pos_name = self.__positions[position] - else: - if latex: - pos_name = '$p_{%d}$' % self.__position_number - else: - pos_name = 'p%d' % self.__position_number - - self.__positions[position] = pos_name - self.__position_number += 1 - - return self.regex.sub(pos_name, name) - - -def write_tikz_tree( - tree: Any, - file: str, - short_names: bool = True -): - """Write latex code to file to draw tree using tikz.""" - if isinstance(tree, pt.trees.BehaviourTree): - root = tree.root - else: - root = tree - - state = __TreeState(short_names) - - with open(file, 'w') as f: - print('% Please compile your document with LuaLaTex and add', file=f) - print('% the following to your document preamble:', file=f) - print(r"""% \usepackage{tikz} - % \usetikzlibrary{arrows,decorations.pathmorphing,backgrounds,fit,positioning, - calc,shapes,graphs,graphdrawing} - % \usegdlibrary{trees} - % \tikzset{control/.style={draw, inner sep=1pt, regular polygon, - regular polygon sides=4, minimum width=1cm, fill=blue!15}} - % \tikzset{action/.style={draw, rectangle, inner sep=0.2cm, fill=green!40}} - % \tikzset{condition/.style={draw, minimum width=1cm, ellipse, inner sep=2pt, - fill=yellow!80}} - % \tikzset{>=stealth} - % \tikzset{level distance=1.5cm} - % \tikzset{sibling distance=2.5cm} - % \tikzset{grow=down}""", file=f) - print(r'\begin{tikzpicture}', file=f) - print('\t\\graph[tree layout] {', file=f) - __generate_tikz(root, f, 2, state) - print('\n\t};', file=f) - print(r'\end{tikzpicture}', file=f) - - return state.get_positions() - - -def dot_graph( - tree: Any, - include_status: bool = False, - short_names: bool = True -) -> pydot.Dot: - """Create and return a pydot graph from tree.""" - if isinstance(tree, pt.trees.BehaviourTree): - root = tree.root - else: - root = tree - - state = __TreeState(short_names) - - graph = pydot.Dot('BT', graph_type='digraph') - __generate_dot_tree(root, graph, state, include_status) - return graph - - -def py_trees_dot( - tree: Any, - short_names: bool = True, -) -> Any: - """Return a pydot graph in the style of py_trees.""" - if isinstance(tree, pt.trees.BehaviourTree): - root = tree.root - else: - root = tree - - state = __TreeState(short_names) - - display_tree = __display_names(root, state) - return pt.display.dot_tree(display_tree) - - -def __generate_tikz( - root: Any, - fp: str, - depth: int, - state: Any -): - if hasattr(root, 'get_display_name'): - name = root.get_display_name() - else: - name = root.name - name = state.get_name(name, True) - - number = state.get_number() - - if isinstance(root, RSequence) or isinstance(root, pt.composites.Sequence): - fp.write('\t'*depth + f'n{number}[as=$\\rightarrow$, control] -> ' + '{\n') - elif isinstance(root, RandomSelector): - fp.write('\t'*depth + f'n{number}[as=??, control] -> ' + '{\n') - elif isinstance(root, pt.composites.Selector): - fp.write('\t'*depth + f'n{number}[as=?, control] -> ' + '{\n') - elif isinstance(root, ActionBehavior): - fp.write('\t'*depth + f'n{number}[as={{{name}}}, action]') - elif isinstance(root, pt.behaviour.Behaviour): - fp.write('\t'*depth + f'n{number}[as={{{name}}}, condition]') - - if isinstance(root, pt.composites.Composite): - for i, child in enumerate(root.children): - __generate_tikz(child, fp, depth+1, state) - if i < len(root.children)-1: - fp.write(',') - fp.write('\n') - fp.write('\t'*depth + '}') - - return state.get_positions() - - -def __generate_dot_tree( - root: Any, - graph: pydot.Dot, - state: Any, - include_status: bool -): - if hasattr(root, 'get_display_name'): - name = root.get_display_name() - else: - name = root.name - name = state.get_name(name) - - number = state.get_number() - node = f'n{number}' - node_instance = None - - if isinstance(root, RSequence) or isinstance(root, pt.composites.Sequence): - node_instance = pydot.Node( - node, label='→', shape='square', margin='0.1', width='0', height='0') - elif isinstance(root, RandomSelector): - node_instance = pydot.Node( - node, label='??', shape='square', margin='0.1', width='0', height='0') - elif isinstance(root, pt.composites.Selector): - node_instance = pydot.Node( - node, label='?', shape='square', margin='0.1', width='0', height='0') - elif isinstance(root, ActionBehavior): - node_instance = pydot.Node( - node, label=name, shape='box') - elif isinstance(root, pt.behaviour.Behaviour): - node_instance = pydot.Node( - node, label=name, shape='ellipse') - - if node_instance is not None: - if include_status: - if root.status == pt.common.Status.SUCCESS: - node_instance.set('color', 'lightgreen') - node_instance.set('penwidth', '2') - elif root.status == pt.common.Status.FAILURE: - node_instance.set('color', 'red') - node_instance.set('penwidth', '2') - elif root.status == pt.common.Status.RUNNING: - node_instance.set('color', 'orange') - node_instance.set('penwidth', '2') - graph.add_node(node_instance) - - if isinstance(root, pt.composites.Composite): - for i, child in enumerate(root.children): - child_node = __generate_dot_tree(child, graph, state, include_status) - edge = pydot.Edge(node, child_node) - if include_status: - color = graph.get_node(child_node)[0].get('color') - if color is not None: - edge.set('color', color) - edge.set('penwidth', '2') - graph.add_edge(edge) - - return node - - -def __display_names( - root: Any, - state: Any -): - """Create a new tree with empty behaviors and display names.""" - if hasattr(root, 'get_display_name'): - name = root.get_display_name() - else: - name = root.name - name = state.get_name(name) - - if isinstance(root, pt.composites.Composite): - if root.name == 'Sequence': - # control = RSequence() - control = pt.composites.Sequence('Sequence', memory=False) - elif root.name == 'Fallback': - control = pt.composites.Selector('Fallback') - else: - raise ValueError('Unknown control flow node ' + root.name) - - for child in root.children: - copy = __display_names(child, state) - control.add_child(copy) - - return control - else: - return pt.behaviour.Behaviour(name=name) diff --git a/kios_bt_planning/backups/dynamic_bt/bt_learning/metalearning/__init__.py b/kios_bt_planning/backups/dynamic_bt/bt_learning/metalearning/__init__.py deleted file mode 100644 index e69de29b..00000000 diff --git a/kios_bt_planning/backups/dynamic_bt/bt_learning/metalearning/hardcode_metadata.py b/kios_bt_planning/backups/dynamic_bt/bt_learning/metalearning/hardcode_metadata.py deleted file mode 100644 index 23367059..00000000 --- a/kios_bt_planning/backups/dynamic_bt/bt_learning/metalearning/hardcode_metadata.py +++ /dev/null @@ -1,168 +0,0 @@ -"""Extract metadata from a Behavior Tree built with backchaining.""" - -# Copyright (c) 2022, ABB -# All rights reserved. -# -# Redistribution and use in source and binary forms, with -# or without modification, are permitted provided that -# the following conditions are met: -# -# * Redistributions of source code must retain the -# above copyright notice, this list of conditions -# and the following disclaimer. -# * Redistributions in binary form must reproduce the -# above copyright notice, this list of conditions -# and the following disclaimer in the documentation -# and/or other materials provided with the -# distribution. -# * Neither the name of ABB nor the names of its -# contributors may be used to endorse or promote -# products derived from this software without -# specific prior written permission. -# -# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" -# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE -# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE -# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE -# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL -# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR -# SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER -# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, -# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF -# THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - -from datetime import datetime -import os -import re -from typing import List - -from bt_library.module_path import get_btlibrary_path -import yaml -from .yaml_custom_dumper import add_custom_dumper - - -def get_timestamp() -> str: - """Return a string indicating current time.""" - dateTimeObj = datetime.now() - current = str(dateTimeObj.year) + str(dateTimeObj.month) + \ - str(dateTimeObj.day) + '_' + str(dateTimeObj.hour) + str(dateTimeObj.minute) - - return current - - -def get_indexed_name(file_name: str, file_path: str) -> str: - """Return a string indicating how many instances of the file are there.""" - indexed_name = file_name - count = len([ - name for name in os.listdir(file_path) - if os.path.isfile(os.path.join(file_path, name)) and name.startswith(file_name) - ]) - - indexed_name = indexed_name + '_' + str(count + 1) + '.yaml' - return indexed_name - - -def find_task_trees(bt: str) -> List[str]: - """ - Find the subtrees of a Behavior Tree solving a task. - - Note - ---- - The BT needs to be deisgned according to the backchaining principle for this to work. - According to this principle, a subtask is represented with a Fallback as a root. - If multiple tasks are solved by the BT, then the BT is a squence of those tasks. - - """ - # If the root is sequence, then remove it and also the 'up' node closing it. - if bt[0] == 's(': - bt = bt[1:-1] - - open_count = 0 - lv = 0 - subtrees = [] - subtree = [] - for node in bt: - if open_count == 0: - # Skip node until next fallback - if node == 'f(': - open_count += 1 - lv += 1 - subtree.append(node) - else: - continue - else: - if node == 's(' or node == 'f(': - lv += 1 - elif node == ')': - lv -= 1 - - subtree.append(node) - - # Reset - if lv == 0: - open_count = 0 - if len(subtree) > 0: - subtrees.append(subtree) - subtree = [] - - return subtrees - - -def build_metadata(bt: List[str]) -> dict: - """Build the metadata from the goals of the Beahvior Tree""" - subtrees = find_task_trees(bt) - goals = [tree[1] for tree in subtrees] - - metadata = { - 'label': '', - 'description': '', - 'target_object': '', - 'position_predicate': '', - 'referring_object': '', - 'hri': False, - 'tree': bt - } - - for goal in goals: - if goal == 'clear': - metadata['hri'] = True - continue - elif goal.startswith('object_at'): - match_str = f'^object_at (.+) (.+) (.+)$' - match = re.match(match_str, goal) - action = 'place' - - elif goal.startswith('object_roughly_at'): - match_str = f'^object_roughly_at (.+) (.+) (.+)$' - match = re.match(match_str, goal) - action = 'drop' - - metadata['target_object'] = str(match[1]) - metadata['position_predicate'] = str(match[2]) - metadata['referring_object'] = str(match[3]) - metadata['label'] = action + '_' + str(match[2]) - metadata['description'] = action + ' ' + \ - str(match[1]) + ' ' + str(match[2]) + ' ' + str(match[3]) - - return metadata - - -def add_bt_to_library(behavior_tree: List[str], library_dir: str = None) -> str: - """Add a Yaml file to the input dir with a BT and its metadata.""" - metadata = build_metadata(behavior_tree) - - add_custom_dumper() - - if library_dir is None: - library_dir = get_btlibrary_path() - - name_with_stamp = metadata['label'] + '_' + get_timestamp() - indexed_name = get_indexed_name(name_with_stamp, library_dir) - file_path = os.path.join(library_dir, indexed_name) - - if os.path.isfile(file_path): - os.remove(file_path) - with open(file_path, 'w') as f: - yaml.dump(metadata, f, sort_keys=False) - - return file_path diff --git a/kios_bt_planning/backups/dynamic_bt/bt_learning/metalearning/smart_load.py b/kios_bt_planning/backups/dynamic_bt/bt_learning/metalearning/smart_load.py deleted file mode 100644 index 3057ae06..00000000 --- a/kios_bt_planning/backups/dynamic_bt/bt_learning/metalearning/smart_load.py +++ /dev/null @@ -1,99 +0,0 @@ -"""Load a previously learnt BT with custom parameters.""" - -# Copyright (c) 2022, ABB -# All rights reserved. -# -# Redistribution and use in source and binary forms, with -# or without modification, are permitted provided that -# the following conditions are met: -# -# * Redistributions of source code must retain the -# above copyright notice, this list of conditions -# and the following disclaimer. -# * Redistributions in binary form must reproduce the -# above copyright notice, this list of conditions -# and the following disclaimer in the documentation -# and/or other materials provided with the -# distribution. -# * Neither the name of ABB nor the names of its -# contributors may be used to endorse or promote -# products derived from this software without -# specific prior written permission. -# -# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" -# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE -# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE -# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE -# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL -# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR -# SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER -# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, -# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF -# THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - -from copy import deepcopy -import re -from typing import List - - -# TODO: handle case in which also the Load Battery behavior is considered - - -def handle_hri(bt: List[str], use_hri: bool) -> List[str]: - """Check if disambiguation subtree is in the BT and then add/remove it.""" - hri_subtree = ['f(', 'clear', 'disambiguate', ')'] - - # If HRI subtree is there but not required, remove it - if set(hri_subtree).issubset(set(bt)) and not use_hri: - hri_root_idx = bt.index('clear') - 1 - handled_bt = bt[0:hri_root_idx] + bt[hri_root_idx + len(hri_subtree):] - # If HRI subtree is not there but required, add it - elif not set(hri_subtree).issubset(set(bt)) and use_hri: - if bt[0] == 's(': - handled_bt = bt[0] + hri_subtree + bt[1:] - else: - handled_bt = ['s('] + hri_subtree + bt + [')'] - # HRI subtree not there and not required, or HRI already there and required - else: - handled_bt = bt - - return handled_bt - - -def load_bt( - bt: List[str], - metadata: dict, - target_obj: str = None, - predicate: str = None, - referring_obj: str = None, - use_hri: bool = False -) -> List[str]: - """Load a Behavior Tree from the BT library, overwriting parameters if necessary.""" - modified_bt = deepcopy(bt) - new_target = target_obj if target_obj is not None else metadata['target_object'] - new_predicate = predicate if predicate is not None else metadata['position_predicate'] - new_reference = referring_obj if referring_obj is not None else metadata['referring_object'] - - for i, node in enumerate(bt): - # Actions - if node.startswith('pick'): - match = re.match('(.+) (.+)$', node) - modified_bt[i] = str(match[1]) + ' ' + new_target - elif node.startswith('place') or node.startswith('drop'): - match = re.match('(.+) (.+) (.+) (.+)$', node) - modified_bt[i] = str(match[1]) + ' ' + new_target + ' ' + \ - new_predicate + ' ' + new_reference - # Conditions - elif node.startswith('in_gripper'): - match = re.match('(.+) (.+)$', node) - modified_bt[i] = str(match[1]) + ' ' + new_target - elif node.startswith('object_'): - match = re.match('(.+) (.+) (.+) (.+)$', node) - modified_bt[i] = str(match[1]) + ' ' + new_target + ' ' + \ - new_predicate + ' ' + new_reference - else: - continue - - modified_bt = handle_hri(modified_bt, use_hri) - - return modified_bt diff --git a/kios_bt_planning/backups/dynamic_bt/bt_learning/metalearning/yaml_custom_dumper.py b/kios_bt_planning/backups/dynamic_bt/bt_learning/metalearning/yaml_custom_dumper.py deleted file mode 100644 index b7689673..00000000 --- a/kios_bt_planning/backups/dynamic_bt/bt_learning/metalearning/yaml_custom_dumper.py +++ /dev/null @@ -1,84 +0,0 @@ -"""Custom YAML dumper for dictionaries.""" - -# Copyright (c) 2022, ABB -# All rights reserved. -# -# Redistribution and use in source and binary forms, with -# or without modification, are permitted provided that -# the following conditions are met: -# -# * Redistributions of source code must retain the -# above copyright notice, this list of conditions -# and the following disclaimer. -# * Redistributions in binary form must reproduce the -# above copyright notice, this list of conditions -# and the following disclaimer in the documentation -# and/or other materials provided with the -# distribution. -# * Neither the name of ABB nor the names of its -# contributors may be used to endorse or promote -# products derived from this software without -# specific prior written permission. -# -# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" -# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE -# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE -# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE -# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL -# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR -# SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER -# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, -# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF -# THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - -import yaml - - -def represent_bool_custom(dumper: yaml.Dumper, data: bool): - """ - Represent bool data. - - Equivalent to the original function, just with capitalized initials. - """ - if data: - value = 'True' - else: - value = 'False' - return yaml.representer.BaseRepresenter.represent_scalar( - dumper, 'tag:yaml.org,2002:bool', value) - - -def represent_dict_custom(dumper: yaml.Dumper, dictionary: dict): - """ - Represent dictionary data. - - - Keys are treated as in the original representer. - - Bool values are treated according to the modified bool representer. - - String values are single quoted. - - Other types of values are treated by default. - """ - value = [] - node = yaml.MappingNode('tag:yaml.org,2002:map', value) - - for item_key in dictionary: - item_value = dictionary[item_key] - node_key = yaml.representer.BaseRepresenter.represent_data(dumper, item_key) - if type(item_value) == str: - node_value = yaml.representer.BaseRepresenter.represent_scalar( - dumper, 'tag:yaml.org,2002:str', item_value, style="'") - elif type(item_value) == bool: - node_value = represent_bool_custom(dumper, item_value) - elif type(item_value) == list: - node_value = yaml.representer.BaseRepresenter.represent_sequence( - dumper, 'tag:yaml.org,2002:seq', item_value) - else: - node_value = yaml.representer.BaseRepresenter.represent_data(dumper, item_value) - - value.append((node_key, node_value)) - - return node - - -def add_custom_dumper(): - """Add the custom dumper to the Dumper class.""" - yaml.add_representer(dict, represent_dict_custom) diff --git a/kios_bt_planning/backups/dynamic_bt/bt_learning/planner/__init__.py b/kios_bt_planning/backups/dynamic_bt/bt_learning/planner/__init__.py deleted file mode 100644 index e69de29b..00000000 diff --git a/kios_bt_planning/backups/dynamic_bt/bt_learning/planner/constraints_identification.py b/kios_bt_planning/backups/dynamic_bt/bt_learning/planner/constraints_identification.py deleted file mode 100644 index f0c715b6..00000000 --- a/kios_bt_planning/backups/dynamic_bt/bt_learning/planner/constraints_identification.py +++ /dev/null @@ -1,337 +0,0 @@ -"""The constraints module contains functions for inferring task constraints. - -- for LFD. - -""" - -# Copyright (c) 2022, ABB -# All rights reserved. -# -# Redistribution and use in source and binary forms, with -# or without modification, are permitted provided that -# the following conditions are met: -# -# * Redistributions of source code must retain the -# above copyright notice, this list of conditions -# and the following disclaimer. -# * Redistributions in binary form must reproduce the -# above copyright notice, this list of conditions -# and the following disclaimer in the documentation -# and/or other materials provided with the -# distribution. -# * Neither the name of ABB nor the names of its -# contributors may be used to endorse or promote -# products derived from this software without -# specific prior written permission. -# -# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" -# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE -# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE -# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE -# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL -# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR -# SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER -# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, -# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF -# THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - -import logging -from typing import Any, Dict, List, Type - - -logger = logging.getLogger("constraints_identification") - - -class Constraint: - """ - for LFD. - A single ordering constraint of two actions. - """ - - def __init__(self, action_a, action_b, distance: float = 1): - """ - Create a constraint action_a < action_b. - - Both actions must be instances of EquivalentAction. - """ - if action_a is action_b: - raise ValueError("Both actions are the same object.") - - self.first = action_a - self.second = action_b - self.distance = distance - - def reverse_constraint(self) -> Type["Constraint"]: - """ - Reverse the logic order of the constraint. - - If this constraint is A < B, it returns a new Constraint representing B < A. - """ - return Constraint(self.second, self.first) - - def __eq__(self, other: Type["Constraint"]) -> bool: - """Return True if the constraint is equal to other.""" - return self.first == other.first and self.second == other.second - - def __str__(self) -> str: - """Print the constraints to string.""" - return f"{self.first} < {self.second}" - - -class ConstraintSet: - """ - For LFD. - A set of constraints.""" - - def __init__(self): - """Construct a new empty set of constraints.""" - self.__active = [] - self.__conflicting = [] - - def active_constraints(self) -> List["Constraint"]: - """Return a list of active constraints that do not conflict with each other.""" - return self.__active - - def conflicting_constraints(self) -> List["Constraint"]: - """Return a list of conflicting constraints.""" - return self.__conflicting - - def add(self, constraint: Type["Constraint"]): - """ - Add a new constraint. - - If it conflicts with an active constraint, the active constraint will be removed - and added to conflicting constraints. The new constraint is ignored. - - Args: - ---- - constraint: the new constraint to add. - - """ - if constraint in self.__conflicting: - return - - if constraint.reverse_constraint() in self.__active: - # The constraint conflicts with an active constraint - self.__active.remove(constraint.reverse_constraint()) - self.__conflicting.append(constraint) - self.__conflicting.append(constraint.reverse_constraint()) - elif constraint in self.__active: - # If the constraint has been encountered previously, we update to the new constraint - # if it has less distance - constraint_idx = self.__active.index(constraint) - if constraint.distance < self.__active[constraint_idx].distance: - del self.__active[constraint_idx] - self.__active.append(constraint) - else: - self.__active.append(constraint) - - -def infer_order_constraints(demonstrations: List) -> Type["ConstraintSet"]: - """ - for LFD. - Infer the order of the constraints in the demonstration. - - Args - ---- - demonstrations: list of demos where each demon is a list of EquivalentAction. - - Returns - ------- - A ConstraintSet of order constraints. - - """ - constraints = ConstraintSet() - - for demonstration in demonstrations: - for action_a_idx, action_a in enumerate(demonstration): - if action_a_idx >= len(demonstration) - 1: - # We have reached the end of demonstration. There is no next action. - continue - - for action_b_idx in range(action_a_idx + 1, len(demonstration)): - if not demonstration[action_b_idx].can_form_constraint(): - continue - if action_a is not demonstration[action_b_idx]: - # Do not add constraints between an action and itself. - constraints.add( - Constraint( - action_a, - demonstration[action_b_idx], - action_b_idx - action_a_idx, - ) - ) - - msg = "Detected constraints:" - for c in constraints.active_constraints(): - msg += "\n\t" + str(c) - logger.info(msg) - - return constraints - - -def infer_preconditions(demonstrations: List, behaviors: Type[Any]): - """ - Add additional inferred preconditions to the actions in demonstrations. - - Args: - ---- - demonstrations: list of demos where each demo is a list of EquivalentAction. - behaviors: instance of Behaviors. - - """ - order_constraints = infer_order_constraints(demonstrations) - - # added_conditions is a dictionary with constraints of the added conditions. - # The first key is an EquivalentAction instance and the second key is a condition string. - added_conditions = {} - - # Convert order constraints to preconditions - for constraint in order_constraints.active_constraints(): - logger.debug("Handling %s", constraint) - - for condition in constraint.first.postconditions(): - # Does condition conflict with a postcondition? - conflicting = get_conflicting( - behaviors, constraint.second.postconditions(), condition - ) - if conflicting is not None: - logger.debug( - '"%s" conflicting with the postcondition %s', condition, conflicting - ) - continue - - # Does the new condition conflict with an existing precondition? - if contains_conflicting( - behaviors, constraint.second.preconditions(), condition - ): - logger.debug( - "%s conflicts with existing precondition %s", - condition, - get_conflicting( - behaviors, constraint.second.preconditions(), condition - ), - ) - continue - - # Does the new condition conflict with a previously added precondition? - if contains_conflicting( - behaviors, constraint.second.additional_preconditions, condition - ): - # Keep the constraint with lowest distance - conflicting_conditions = [ - c - for c in constraint.second.additional_preconditions - if not behaviors.compatible(c, condition) - ] - # conflicting_distance = lambda c: added_conditions[constraint.second][c].distance - if all( - map( - lambda c: constraint.distance - < get_conflicting_distance( # pylint: disable=cell-var-from-loop - added_conditions, constraint, c - ), # pylint: disable=cell-var-from-loop - conflicting_conditions, - ) - ): - # New constraint has shorter distance than all conflicting constraints - for conflicting_condition in conflicting_conditions: - logger.debug( - 'Replacing "%s" with "%s" for "%s"', - conflicting_condition, - condition, - constraint.second.action_string(), - ) - constraint.second.additional_preconditions.remove( - conflicting_condition - ) - constraint.second.additional_preconditions.append(condition) - added_conditions[constraint.second][condition] = constraint - continue - - # No conflicts detected. Ok to add the new precondition if it doesn't already exist - if condition not in constraint.second.preconditions_with_additional(): - logger.info( - 'Added "%s" as a precondition of "%s"', - condition, - constraint.second.action_string(), - ) - constraint.second.additional_preconditions.append(condition) - if constraint.second not in added_conditions: - added_conditions[constraint.second] = {} - added_conditions[constraint.second][condition] = constraint - - elif ( - condition not in constraint.second.preconditions() - and constraint.distance - < added_conditions[constraint.second][condition].distance - ): - # We have encountered the same condition with lower constraint distance. - # Update the number. - added_conditions[constraint.second][condition] = constraint - - -def get_conflicting_distance( - added_conditions: Dict, constraint: Type["Constraint"], conflicting_condition: Any -) -> float: - """ - for LFD. - Get the distance of two conflicting conditions. - - Args - ---- - added_conditions: dictionary with the conditions. - constraint: constraint with conflicting conditions. - conflicting_condition: the condition that is causing conflict. - - Returns - ------- - The conflicting distance. - - """ - return added_conditions[constraint.second][conflicting_condition].distance - - -def get_conflicting( - behaviors: Type[Any], condition_list: List[str], condition: str -) -> str or None: - """ - Get the conflicting conditions. - - Args - ---- - behaviors: the behavior to consider the conditions of. - condition_list: the list of condition to check for compatibility. - condition: baseline for comparison. - - Returns - ------- - The first condition in condition_list that the input condition conflicts with. - None is returned if there are no conflicts. - - """ - for c in condition_list: - if not behaviors.compatible(condition, c): - return c - - return None - - -def contains_conflicting( - behaviors: Type[Any], condition_list: List[str], condition: str -) -> bool: - """ - Check if the condition list contains conflicting conditions. - - Args - ---- - behaviors: the behavior to consider the conditions of. - condition_list: the list of condition to check for compatibility. - condition: baseline for comparison. - - Returns - ------- - True if condition_list contains at least one element that conflicts with input condition. - - """ - return get_conflicting(behaviors, condition_list, condition) is not None diff --git a/kios_bt_planning/backups/dynamic_bt/bt_learning/planner/lfd_planner.py b/kios_bt_planning/backups/dynamic_bt/bt_learning/planner/lfd_planner.py deleted file mode 100644 index c44a201a..00000000 --- a/kios_bt_planning/backups/dynamic_bt/bt_learning/planner/lfd_planner.py +++ /dev/null @@ -1,304 +0,0 @@ -""" -Simple task planner inspired 'Towards Blended Reactive Planning and Acting using Behavior Trees'. - -Generates a BT to solve task given a set of goals and behaviors with pre- and post-conditions. -Since the conditions are not always static, it runs the tree while evaluating the conditions. - -LFD: learning from demo? -""" - -# Copyright (c) 2022, ABB -# All rights reserved. -# -# Redistribution and use in source and binary forms, with -# or without modification, are permitted provided that -# the following conditions are met: -# -# * Redistributions of source code must retain the -# above copyright notice, this list of conditions -# and the following disclaimer. -# * Redistributions in binary form must reproduce the -# above copyright notice, this list of conditions -# and the following disclaimer in the documentation -# and/or other materials provided with the -# distribution. -# * Neither the name of ABB nor the names of its -# contributors may be used to endorse or promote -# products derived from this software without -# specific prior written permission. -# -# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" -# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE -# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE -# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE -# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL -# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR -# SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER -# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, -# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF -# THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - -import logging -from typing import Any, List, Tuple - -from behaviors.common_behaviors import RSequence, ActionBehavior -from bt_learning.planner.constraints_identification import contains_conflicting -import py_trees as pt -from py_trees.composites import Selector, Sequence - -logger = logging.getLogger("lfd-planner") - - -def handle_precondition( - precondition: List[str], behaviors: Any, world_interface: Any -) -> List[pt.trees.BehaviourTree]: - """Handle pre-condition by exploiting the backchaining method.""" - # print("Condition in: ", precondition) - condition_parameters = behaviors.get_condition_parameters(precondition) - trees = [] - best_cost = float("inf") # the less the better - - action_list = behaviors.get_behavior_list().action_nodes - - for action in action_list: - action_node, _ = behaviors.get_node( - action, world_interface, condition_parameters - ) - - # * find the action node that can satisfy the precondition and has the lowest cost - if precondition in action_node.get_postconditions(): - # Only keep the actions with lowest cost - if action_node.cost() > best_cost: - continue - elif action_node.cost() < best_cost: - trees.clear() # unefficient loop - best_cost = action_node.cost() - - action_preconditions = action_node.get_preconditions() - - # * load the guards on the left of the action node in the sequence - if action_preconditions != []: - bt = RSequence("Sequence") - # bt = Sequence('Sequence', memory=False) - - for action_precondition in action_preconditions: - child, _ = behaviors.get_node( - action_precondition, - world_interface, - behaviors.get_condition_parameters(action_precondition), - ) - bt.add_child(child) - - bt.add_child(action_node) - else: - bt = action_node - - # * load the sequence as result - trees.append(bt) - # print("ERROR, no matching action found to ensure precondition") - return trees - - -def extend_leaf_node( - leaf_node: pt.behaviour.Behaviour, behaviors: Any, world_interface: Any -) -> None: - """ - Extend the failing node. - - If leaf node fails, it should be replaced with a selector that checks leaf node - and a subtree that fixes the condition whenever it's not met. - - BB: the failing node will become the terminating condition of the expanded subtree. - The inserted subtree will be a sequencce of guards as preconditions and the action node - to fix the terminating condition. - - """ - # the root fallback node - bt = Selector(name="Fallback") - # Make the replacing subtree still failing. The parent node might get confused - # if one of its children suddenly changes status to INVALID. - bt.stop(pt.common.Status.FAILURE) - # * replace the leaf node with the new selector - leaf_node.parent.replace_child(leaf_node, bt) - # * add the leafnode as terminating condition - bt.add_child(leaf_node) - print("What is failing?", leaf_node.name) - - # * find a sequence of guards and action node to fix the condition - extended = handle_precondition( - leaf_node.name, behaviors, world_interface - ) # * load the sequence into the selector - for tree in extended: - bt.add_child(tree) - - -def expand_tree( - node: pt.composites.Composite or pt.behaviour.Behaviour, - behaviors: Any, # ? what is behaviors? - world_interface: Any, - depth: int = 0, -) -> None: - """Expand the part of the tree that fails. - - BB : should only expand the failed condition nodes according to the def of expanding BT. - Action node should be given a chance for a rollout. - - """ - # print("TREE COMING IN :", node) - - # If the recursion is very deep there is some problem and we should abort - # ??? - if depth >= 100: - logger.warning("Maximum condition expansion depth reached") - return - # * if the node is a control flow node, recursively expand its children - if node.name == "Fallback": - # print("Fallback node fails\n") - for index, child in enumerate(node.children): - if index >= 1: # Normally there will only be two children - expand_tree(child, behaviors, world_interface, depth + 1) - - elif node.name == "Sequence" or node.name == "RandomSelector": - # print("Sequence node fails\n") - for i in range(len(node.children)): - if node.children[i].status == pt.common.Status.FAILURE: - # print("Child that fails: ", node.children[i].name) - expand_tree(node.children[i], behaviors, world_interface, depth + 1) - - # * ...until here. Finally the failed atomic node is found and is fixed by extend_leaf_node - elif ( - isinstance(node, pt.behaviour.Behaviour) - and node.status == pt.common.Status.FAILURE - ): - extend_leaf_node(node, behaviors, world_interface) - # else: - # print("Tree", node.name) - - -def get_tree_effects(tree: pt.trees.BehaviourTree) -> List[str]: - """ - Return all effects of a tree as action strings. - - I.e. traverses the tree and returns the first child of each fallback node, - and the postconditions of the last child of each sequence node. - - BB: according to the def, in a sequence only the last node can be an action node (so it is the only one - that has effects). As for the fallback node, the first child is the termination condition, which is the - effect of the fallback node. - - The effect of the sequence is the objective effect the action node expresses to the world. - The effect of the fallback node is the reason the action node is chosen. - """ - if isinstance(tree, ActionBehavior): - effects = tree.get_postconditions() - elif len(tree.children) > 0: - conditions = [] - for child in tree.children: - conditions += get_tree_effects(child) - effects = conditions - else: - effects = [] - - return effects - - -def conflicting_subtrees( - trees: List[pt.trees.BehaviourTree], behaviors: Any -) -> Tuple[int, int]: - """ - Return the indices of the conflicting subtrees in the list trees. - - It returns (-1, -1) if there are no conflicts. - It is assumed that each element of trees is a fallback node or a condition node. - """ - for i in range(len(trees) - 2, -1, -1): - if isinstance(trees[i], Selector) and not ( - isinstance(trees[i], RSequence) or isinstance(trees[i], Sequence) - ): - # The condition is the first child of a fallback node - high_priority_condition = trees[i].children[0].name - else: - high_priority_condition = trees[i].name - - for j in range(i + 1, len(trees)): - effects = get_tree_effects(trees[j]) - if contains_conflicting(behaviors, effects, high_priority_condition): - return i, j - - return -1, -1 - - -def handle_priority(tree: pt.trees.BehaviourTree, behaviors: Any) -> bool: - """Detect subtrees that have conflicting effects and reorders them. - - args: - tree: the tree to be reordered - behaviors: the behavior that may result in conflicting effects - - """ - if isinstance(tree, RSequence) or isinstance(tree, Sequence): - subtrees = tree.children - # The last child of all sequence nodes is an action node except - # for the topmost sequence node that contains all goal conditions. - # Don't include the last child unless it is the topmost node - i, j = conflicting_subtrees(subtrees, behaviors) - - n_tries = 0 - - # a loop that runs until there are no more conflicts - while i != -1: - logger.info("Detected conflicting subtrees") - logger.debug( - "Conflicting subtrees:\n%s-----------------\n%s", - pt.display.unicode_tree(subtrees[i]), - pt.display.unicode_tree(subtrees[j]), - ) - # Place child j before child i - tree_to_move = subtrees[j] - tree.remove_child(tree_to_move) - tree.insert_child(tree_to_move, i) - - i, j = conflicting_subtrees(subtrees, behaviors) - - n_tries += 1 - if n_tries > len(subtrees): - logger.warning( - "Could not find a configuration of subtrees without conflicting effects" - ) - return False - - return True - # for other types of nodes (fallback nodes) just recurse - for child in tree.children: - if not handle_priority(child, behaviors): - return False - return True - - -def plan( - world_interface: Any, behaviors: Any, tree: pt.trees.BehaviourTree -) -> pt.trees.BehaviourTree: # ? what is world_interface? - """ - Generate a BT to solve a task given: initial tree and behaviors with pre- and post-conditions. - - Since the conditions are not always static, it runs the tree while evaluating the conditions. - """ - # print(pt.display.unicode_tree(root=tree, show_status=True)) - for _ in range(100): - if not handle_priority(tree, behaviors): - # * print("Could not handle priority") - break - # * Run the tree once - tree.tick_once() - # print("Tick: ", i) - # print(pt.display.unicode_tree(root=tree, show_status=True)) - # * for the tick results: SUCCESS, FAILURE, RUNNING - # * should be running if an action node is ticked (may succeed or fail in rollouts) - # * should be Failure or Success if a condition node is ticked (immediate feedback) - if tree.status is pt.common.Status.FAILURE: - expand_tree(tree, behaviors, world_interface) # * main logic - # print(pt.display.unicode_tree(root=tree, show_status=True)) - elif tree.status is pt.common.Status.SUCCESS: # * terminal condition - break - - return tree diff --git a/kios_bt_planning/backups/dynamic_bt/py_trees/__init__.py b/kios_bt_planning/backups/dynamic_bt/py_trees/__init__.py deleted file mode 100644 index 7d3c0304..00000000 --- a/kios_bt_planning/backups/dynamic_bt/py_trees/__init__.py +++ /dev/null @@ -1,20 +0,0 @@ -from . import behaviour # noqa -from . import behaviours # noqa -from . import blackboard # noqa -from . import common # noqa -from . import composites # noqa -from . import console # noqa -from . import decorators # noqa -from . import display # noqa -from . import idioms # noqa -from . import pt_logging # noqa -from . import meta # noqa -from . import syntax_highlighting # noqa -from . import tests # noqa -from . import timers # noqa -from . import trees # noqa -from . import utilities # noqa -from . import visitors # noqa - -name = 'py_trees' -__version__ = '0.1.0' \ No newline at end of file diff --git a/kios_bt_planning/backups/dynamic_bt/py_trees/behaviour.py b/kios_bt_planning/backups/dynamic_bt/py_trees/behaviour.py deleted file mode 100644 index 320ab3ff..00000000 --- a/kios_bt_planning/backups/dynamic_bt/py_trees/behaviour.py +++ /dev/null @@ -1,384 +0,0 @@ -#!/usr/bin/env python -# -# License: BSD -# https://raw.githubusercontent.com/splintered-reality/py_trees/devel/LICENSE -# -############################################################################## -# Documentation -############################################################################## - -""" -The core behaviour template. All behaviours, standalone and composite, inherit -from this class. -""" - -############################################################################## -# Imports -############################################################################## - -import re -import typing -import uuid - -from py_trees import blackboard -from py_trees import common -from py_trees import pt_logging - -############################################################################## -# Behaviour BluePrint -############################################################################## - - -class Behaviour(object): - """ - Defines the basic properties and methods required of a node in a - behaviour tree. When implementing your own behaviour, - subclass this class. - - Args: - name: the behaviour name, defaults to auto-generating from the class name - - Raises: - TypeError: if the provided name is not a string - - Attributes: - ~py_trees.behaviours.Behaviour.id (:class:`uuid.UUID`): automagically generated unique identifier for the behaviour - ~py_trees.behaviours.Behaviour.name (:obj:`str`): the behaviour name - ~py_trees.behaviours.Behaviour.blackboards (typing.List[py_trees.blackboard.Client]): collection of attached blackboard clients - ~py_trees.behaviours.Behaviour.status (:class:`~py_trees.common.Status`): the behaviour status (:data:`~py_trees.common.Status.INVALID`, :data:`~py_trees.common.Status.RUNNING`, :data:`~py_trees.common.Status.FAILURE`, :data:`~py_trees.common.Status.SUCCESS`) - ~py_trees.behaviours.Behaviour.parent (:class:`~py_trees.behaviour.Behaviour`): a :class:`~py_trees.composites.Composite` instance if nested in a tree, otherwise None - ~py_trees.behaviours.Behaviour.children ([:class:`~py_trees.behaviour.Behaviour`]): empty for regular behaviours, populated for composites - ~py_trees.behaviours.Behaviour.logger (:class:`logging.Logger`): a simple logging mechanism - ~py_trees.behaviours.Behaviour.feedback_message(:obj:`str`): improve debugging with a simple message - ~py_trees.behaviours.Behaviour.blackbox_level (:class:`~py_trees.common.BlackBoxLevel`): a helper variable for dot graphs and runtime gui's to collapse/explode entire subtrees dependent upon the blackbox level. - - .. seealso:: - * :ref:`Skeleton Behaviour Template ` - * :ref:`The Lifecycle Demo ` - * :ref:`The Action Behaviour Demo ` - - """ - def __init__( - self, - name: typing.Union[str, common.Name]=common.Name.AUTO_GENERATED - ): - if not name or name == common.Name.AUTO_GENERATED: - name = self.__class__.__name__ - if not isinstance(name, str): - raise TypeError("a behaviour name should be a string, but you passed in {}".format(type(name))) - self.id = uuid.uuid4() # used to uniquely identify this node (helps with removing children from a tree) - self.name: str = name - self.blackboards: typing.List[blackboard.Client] = [] - self.qualified_name = "{}/{}".format(self.__class__.__qualname__, self.name) # convenience - self.status = common.Status.INVALID - self.iterator = self.tick() - self.parent: typing.Optional[Behaviour] = None # will get set if a behaviour is added to a composite - self.children: typing.List[Behaviour] = [] # only set by composite behaviours - self.logger = pt_logging.Logger(name) - self.feedback_message = "" # useful for debugging, or human readable updates, but not necessary to implement - self.blackbox_level = common.BlackBoxLevel.NOT_A_BLACKBOX - - ############################################ - # User Customisable Callbacks - ############################################ - - def setup(self, **kwargs): - """ - .. note:: User Customisable Callback - - Subclasses may override this method for any one-off delayed construction & - validation that is necessary prior to ticking the tree. Such construction is best - done here rather than in __init__ so that trees can be instantiated on the fly for - easy rendering to dot graphs without imposing runtime requirements (e.g. establishing - a middleware connection to a sensor or a driver to a serial port). - - Equally as important, executing methods which validate the configuration of - behaviours will increase confidence that your tree will successfully tick - without logical software errors before actually ticking. This is useful both - before a tree's first tick and immediately after any modifications to a tree - has been made between ticks. - - .. tip:: - - Faults are notified to the user of the behaviour via exceptions. - Choice of exception to use is left to the user. - - .. warning:: - - The kwargs argument is for distributing objects at runtime to behaviours - before ticking. For example, a simulator instance with which behaviours can - interact with the simulator's python api, a ros2 node for setting up - communications. Use sparingly, as this is not proof against keyword conflicts - amongst disparate libraries of behaviours. - - Args: - **kwargs (:obj:`dict`): distribute arguments to this - behaviour and in turn, all of it's children - - Raises: - Exception: if this behaviour has a fault in construction or configuration - - .. seealso:: :meth:`py_trees.behaviour.Behaviour.shutdown` - """ - pass - - def initialise(self): - """ - .. note:: User Customisable Callback - - Subclasses may override this method to perform any necessary initialising/clearing/resetting - of variables when when preparing to enter this behaviour if it was not previously - :data:`~py_trees.common.Status.RUNNING`. i.e. Expect this to trigger more than once! - """ - pass - - def terminate(self, new_status): - """ - .. note:: User Customisable Callback - - Subclasses may override this method to clean up. It will be triggered when a behaviour either - finishes execution (switching from :data:`~py_trees.common.Status.RUNNING` - to :data:`~py_trees.common.Status.FAILURE` || :data:`~py_trees.common.Status.SUCCESS`) - or it got interrupted by a higher priority branch (switching to - :data:`~py_trees.common.Status.INVALID`). Remember that the :meth:`~py_trees.behaviour.Behaviour.initialise` method - will handle resetting of variables before re-entry, so this method is about - disabling resources until this behaviour's next tick. This could be a indeterminably - long time. e.g. - - * cancel an external action that got started - * shut down any tempoarary communication handles - - Args: - new_status (:class:`~py_trees.common.Status`): the behaviour is transitioning to this new status - - .. warning:: Do not set `self.status = new_status` here, that is automatically handled - by the :meth:`~py_trees.behaviour.Behaviour.stop` method. Use the argument purely for introspection purposes (e.g. - comparing the current state in `self.status` with the state it will transition to in - `new_status`. - """ - pass - - def update(self): - """ - .. note:: User Customisable Callback - - Returns: - :class:`~py_trees.common.Status`: the behaviour's new status :class:`~py_trees.common.Status` - - Subclasses may override this method to perform any logic required to - arrive at a decision on the behaviour's new status. It is the primary worker function called on - by the :meth:`~py_trees.behaviour.Behaviour.tick` mechanism. - - .. tip:: This method should be almost instantaneous and non-blocking - - """ - return common.Status.INVALID - - def shutdown(self): - """ - .. note:: User Customisable Callback - - Subclasses may override this method for any custom destruction of infrastructure - usually brought into being in :meth:`~py_trees.behaviour.Behaviour.setup`. - - Raises: - Exception: of whatever flavour the child raises when errors occur on destruction - - .. seealso:: :meth:`py_trees.behaviour.Behaviour.setup` - """ - pass - - ############################################ - # Private Methods - use inside a behaviour - ############################################ - - def attach_blackboard_client( - self, - name: str=None, - namespace: str=None - ) -> blackboard.Client: - """ - Create and attach a blackboard to this behaviour. - - Args: - name: human-readable (not necessarily unique) name for the client - namespace: sandbox the client to variables behind this namespace - - Returns: - a handle to the attached blackboard client - """ - if name is None: - count = len(self.blackboards) - name = self.name if (count == 0) else self.name + "-{}".format(count) - new_blackboard = blackboard.Client( - name=name, - namespace=namespace - ) - self.blackboards.append(new_blackboard) - return new_blackboard - - ############################################ - # Public - lifecycle API - ############################################ - - def setup_with_descendants(self): - """ - Iterates over this child, it's children (it's children's children, ...) - calling the user defined :meth:`~py_trees.behaviour.Behaviour.setup` - on each in turn. - """ - for child in self.children: - for node in child.iterate(): - node.setup() - self.setup() - - def tick_once(self): - """ - A direct means of calling tick on this object without - using the generator mechanism. - """ - # no logger necessary here...it directly relays to tick - for unused in self.tick(): - pass - - def tick(self): - """ - This function is a generator that can be used by an iterator on - an entire behaviour tree. It handles the logic for deciding when to - call the user's :meth:`~py_trees.behaviour.Behaviour.initialise` and :meth:`~py_trees.behaviour.Behaviour.terminate` methods as well as making the - actual call to the user's :meth:`~py_trees.behaviour.Behaviour.update` method that determines the - behaviour's new status once the tick has finished. Once done, it will - then yield itself (generator mechanism) so that it can be used as part of - an iterator for the entire tree. - - .. code-block:: python - - for node in my_behaviour.tick(): - print("Do something") - - .. note:: - - This is a generator function, you must use this with *yield*. If you need a direct call, - prefer :meth:`~py_trees.behaviour.Behaviour.tick_once` instead. - - Yields: - :class:`~py_trees.behaviour.Behaviour`: a reference to itself - - .. warning:: Override this method only in exceptional circumstances, prefer overriding :meth:`~py_trees.behaviour.Behaviour.update` instead. - - """ - if self.logger.level < 1: - self.logger.debug("%s.tick()" % (self.__class__.__name__)) - if self.status != common.Status.RUNNING: - self.initialise() - # don't set self.status yet, terminate() may need to check what the current state is first - new_status = self.update() - if new_status not in list(common.Status): - self.logger.error("A behaviour returned an invalid status, setting to INVALID [%s][%s]" % (new_status, self.name)) - new_status = common.Status.INVALID - if new_status != common.Status.RUNNING: - self.stop(new_status) - self.status = new_status - yield self - - def iterate(self, direct_descendants=False): - """ - Generator that provides iteration over this behaviour and all its children. - To traverse the entire tree: - - .. code-block:: python - - for node in my_behaviour.iterate(): - print("Name: {0}".format(node.name)) - - Args: - direct_descendants (:obj:`bool`): only yield children one step away from this behaviour. - - Yields: - :class:`~py_trees.behaviour.Behaviour`: one of it's children - """ - for child in self.children: - if not direct_descendants: - for node in child.iterate(): - yield node - else: - yield child - yield self - - def visit(self, visitor): - """ - This is functionality that enables external introspection into the behaviour. It gets used - by the tree manager classes to collect information as ticking traverses a tree. - - Args: - visitor (:obj:`object`): the visiting class, must have a run(:class:`~py_trees.behaviour.Behaviour`) method. - """ - visitor.run(self) - - def stop(self, new_status=common.Status.INVALID): - """ - Args: - new_status (:class:`~py_trees.common.Status`): the behaviour is transitioning to this new status - - This calls the user defined :meth:`~py_trees.behaviour.Behaviour.terminate` method and also resets the - generator. It will finally set the new status once the user's :meth:`~py_trees.behaviour.Behaviour.terminate` - function has been called. - - .. warning:: Override this method only in exceptional circumstances, prefer overriding :meth:`~py_trees.behaviour.Behaviour.terminate` instead. - """ - if self.logger.level < 1: - self.logger.debug("%s.stop(%s)" % (self.__class__.__name__, "%s->%s" % (self.status, new_status) if self.status != new_status else "%s" % new_status)) - self.terminate(new_status) - self.status = new_status - self.iterator = self.tick() - - ############################################ - # Public - introspection API - ############################################ - def has_parent_with_name(self, name): - """ - Searches through this behaviour's parents, and their parents, looking for - a behaviour with the same name as that specified. - - Args: - name (:obj:`str`): name of the parent to match, can be a regular expression - - Returns: - bool: whether a parent was found or not - """ - pattern = re.compile(name) - b = self - while b.parent is not None: - if pattern.match(b.parent.name) is not None: - return True - b = b.parent - return False - - def has_parent_with_instance_type(self, instance_type): - """ - Moves up through this behaviour's parents looking for - a behaviour with the same instance type as that specified. - - Args: - instance_type (:obj:`str`): instance type of the parent to match - - Returns: - bool: whether a parent was found or not - """ - b = self - while b.parent is not None: - if isinstance(b.parent, instance_type): - return True - b = b.parent - return False - - def tip(self): - """ - Get the *tip* of this behaviour's subtree (if it has one) after it's last - tick. This corresponds to the the deepest node that was running before the - subtree traversal reversed direction and headed back to this node. - - Returns: - :class:`~py_trees.behaviour.Behaviour` or :obj:`None`: child behaviour, itself or :obj:`None` if its status is :data:`~py_trees.common.Status.INVALID` - """ - return self if self.status != common.Status.INVALID else None diff --git a/kios_bt_planning/backups/dynamic_bt/py_trees/behaviours.py b/kios_bt_planning/backups/dynamic_bt/py_trees/behaviours.py deleted file mode 100644 index 13c58c78..00000000 --- a/kios_bt_planning/backups/dynamic_bt/py_trees/behaviours.py +++ /dev/null @@ -1,738 +0,0 @@ -#!/usr/bin/env python -# -# License: BSD -# https://raw.githubusercontent.com/splintered-reality/py_trees/devel/LICENSE -# -############################################################################## -# Documentation -############################################################################## -""" -A library of fundamental behaviours for use. -""" - -############################################################################## -# Imports -############################################################################## - -import copy -import functools -import operator -import typing - -from py_trees import behaviour -from py_trees import blackboard -from py_trees import common -from py_trees import meta - -############################################################################## -# Function Behaviours -############################################################################## - - -def success(self): - if self.logger.level < 1: - self.logger.debug("%s.update()" % self.__class__.__name__) - self.feedback_message = "success" - return common.Status.SUCCESS - - -def failure(self): - if self.logger.level < 1: - self.logger.debug("%s.update()" % self.__class__.__name__) - self.feedback_message = "failure" - return common.Status.FAILURE - - -def running(self): - if self.logger.level < 1: - self.logger.debug("%s.update()" % self.__class__.__name__) - self.feedback_message = "running" - return common.Status.RUNNING - - -def dummy(self): - if self.logger.level < 1: - self.logger.debug("%s.update()" % self.__class__.__name__) - self.feedback_message = "crash test dummy" - return common.Status.RUNNING - - -Success = meta.create_behaviour_from_function(success) -""" -Do nothing but tick over with :data:`~py_trees.common.Status.SUCCESS`. -""" - -Failure = meta.create_behaviour_from_function(failure) -""" -Do nothing but tick over with :data:`~py_trees.common.Status.FAILURE`. -""" - -Running = meta.create_behaviour_from_function(running) -""" -Do nothing but tick over with :data:`~py_trees.common.Status.RUNNING`. -""" - -Dummy = meta.create_behaviour_from_function(dummy) -""" -Crash test dummy used for anything dangerous. -""" - -############################################################################## -# Standalone Behaviours -############################################################################## - - -class Periodic(behaviour.Behaviour): - """ - Simply periodically rotates it's status over the - :data:`~py_trees.common.Status.RUNNING`, :data:`~py_trees.common.Status.SUCCESS`, - :data:`~py_trees.common.Status.FAILURE` states. - That is, :data:`~py_trees.common.Status.RUNNING` for N ticks, - :data:`~py_trees.common.Status.SUCCESS` for N ticks, - :data:`~py_trees.common.Status.FAILURE` for N ticks... - - Args: - name (:obj:`str`): name of the behaviour - n (:obj:`int`): period value (in ticks) - - .. note:: It does not reset the count when initialising. - """ - def __init__(self, name, n): - super(Periodic, self).__init__(name) - self.count = 0 - self.period = n - self.response = common.Status.RUNNING - - def update(self): - self.count += 1 - if self.count > self.period: - if self.response == common.Status.FAILURE: - if self.logger.level < 1: - self.feedback_message = "flip to running" - self.response = common.Status.RUNNING - elif self.response == common.Status.RUNNING: - if self.logger.level < 1: - self.feedback_message = "flip to success" - self.response = common.Status.SUCCESS - else: - if self.logger.level < 1: - self.feedback_message = "flip to failure" - self.response = common.Status.FAILURE - self.count = 0 - else: - if self.logger.level < 1: - self.feedback_message = "constant" - return self.response - - -class StatusSequence(behaviour.Behaviour): - """ - Cycle through the specified sequence of states. - - Args: - name: name of the behaviour - sequence: list of status values to cycle through - eventually: status to use eventually, None to re-cycle the sequence - """ - def __init__( - self, - name: str, - sequence: typing.List[common.Status], - eventually: typing.Optional[common.Status] - ): - super(StatusSequence, self).__init__(name) - self.sequence = sequence - self.eventually = eventually - self.current_sequence = copy.copy(sequence) - - def update(self): - if self.current_sequence: - status = self.current_sequence.pop(0) - elif self.eventually is not None: - status = self.eventually - else: - self.current_sequence = copy.copy(self.sequence) - status = self.current_sequence.pop(0) - return status - - -class SuccessEveryN(behaviour.Behaviour): - """ - This behaviour updates it's status with :data:`~py_trees.common.Status.SUCCESS` - once every N ticks, :data:`~py_trees.common.Status.FAILURE` otherwise. - - Args: - name (:obj:`str`): name of the behaviour - n (:obj:`int`): trigger success on every n'th tick - - .. tip:: - Use with decorators to change the status value as desired, e.g. - :meth:`py_trees.decorators.FailureIsRunning` - """ - def __init__(self, name, n): - super(SuccessEveryN, self).__init__(name) - self.count = 0 - self.every_n = n - - def update(self): - self.count += 1 - if self.logger.level < 1: - self.logger.debug("%s.update()][%s]" % (self.__class__.__name__, self.count)) - if self.count % self.every_n == 0: - if self.logger.level < 1: - self.feedback_message = "now" - return common.Status.SUCCESS - else: - if self.logger.level < 1: - self.feedback_message = "not yet" - return common.Status.FAILURE - - -class TickCounter(behaviour.Behaviour): - """ - A useful utility behaviour for demos and tests. Simply - ticks with :data:`~py_trees.common.Status.RUNNING` for - the specified number of ticks before returning the - requested completion status (:data:`~py_trees.common.Status.SUCCESS` - or :data:`~py_trees.common.Status.FAILURE`). - - This behaviour will reset the tick counter when initialising. - - Args: - name: name of the behaviour - duration: number of ticks to run - completion_status: status to switch to once the counter has expired - """ - def __init__( - self, - duration: int, - name=common.Name.AUTO_GENERATED, - completion_status: common.Status=common.Status.SUCCESS - ): - super().__init__(name=name) - self.completion_status = completion_status - self.duration = duration - self.counter = 0 - - def initialise(self): - """ - Reset the tick counter. - """ - self.counter = 0 - - def update(self): - """ - Increment the tick counter and return the appropriate status for this behaviour - based on the tick count. - - Returns - :data:`~py_trees.common.Status.RUNNING` while not expired, the given completion status otherwise - """ - self.counter += 1 - if self.counter <= self.duration: - return common.Status.RUNNING - else: - return self.completion_status - - -class Count(behaviour.Behaviour): - """ - A counting behaviour that updates its status at each tick depending on - the value of the counter. The status will move through the states in order - - :data:`~py_trees.common.Status.FAILURE`, :data:`~py_trees.common.Status.RUNNING`, - :data:`~py_trees.common.Status.SUCCESS`. - - This behaviour is useful for simple testing and demo scenarios. - - Args: - name (:obj:`str`): name of the behaviour - fail_until (:obj:`int`): set status to :data:`~py_trees.common.Status.FAILURE` until the counter reaches this value - running_until (:obj:`int`): set status to :data:`~py_trees.common.Status.RUNNING` until the counter reaches this value - success_until (:obj:`int`): set status to :data:`~py_trees.common.Status.SUCCESS` until the counter reaches this value - reset (:obj:`bool`): whenever invalidated (usually by a sequence reinitialising, or higher priority interrupting) - - Attributes: - count (:obj:`int`): a simple counter which increments every tick - """ - def __init__(self, name="Count", fail_until=3, running_until=5, success_until=6, reset=True): - super(Count, self).__init__(name) - self.count = 0 - self.fail_until = fail_until - self.running_until = running_until - self.success_until = success_until - self.number_count_resets = 0 - self.number_updated = 0 - self.reset = reset - - def terminate(self, new_status): - if self.logger.level < 1: - self.logger.debug("%s.terminate(%s->%s)" % (self.__class__.__name__, self.status, new_status)) - # reset only if udpate got us into an invalid state - if new_status == common.Status.INVALID and self.reset: - self.count = 0 - self.number_count_resets += 1 - self.feedback_message = "" - - def update(self): - self.number_updated += 1 - self.count += 1 - if self.count <= self.fail_until: - if self.logger.level < 1: - self.logger.debug("%s.update()[%s: failure]" % (self.__class__.__name__, self.count)) - self.feedback_message = "failing" - return common.Status.FAILURE - elif self.count <= self.running_until: - if self.logger.level < 1: - self.logger.debug("%s.update()[%s: running]" % (self.__class__.__name__, self.count)) - self.feedback_message = "running" - return common.Status.RUNNING - elif self.count <= self.success_until: - if self.logger.level < 1: - self.logger.debug("%s.update()[%s: success]" % (self.__class__.__name__, self.count)) - self.feedback_message = "success" - return common.Status.SUCCESS - else: - if self.logger.level < 1: - self.logger.debug("%s.update()[%s: failure]" % (self.__class__.__name__, self.count)) - self.feedback_message = "failing forever more" - return common.Status.FAILURE - - def __repr__(self): - """ - Simple string representation of the object. - - Returns: - :obj:`str`: string representation - """ - s = "%s\n" % self.name - s += " Status : %s\n" % self.status - s += " Count : %s\n" % self.count - s += " Resets : %s\n" % self.number_count_resets - s += " Updates: %s\n" % self.number_updated - return s - -############################################################################## -# Blackboard Behaviours -############################################################################## - - -class BlackboardToStatus(behaviour.Behaviour): - """ - This behaviour reverse engineers the :class:`~py_trees.decorators.StatusToBlackboard` - decorator. Used in conjuction with that decorator, this behaviour can be used to - reflect the status of a decision elsewhere in the tree. - - .. note:: - - A word of caution. The consequences of a behaviour's status should be discernable - upon inspection of the tree graph. If using StatusToBlackboard - and BlackboardToStatus to reflect a behaviour's status across a tree, - this is no longer true. The graph of the tree communicates the local consequences, - but not the reflected consequences at the point BlackboardToStatus is used. A - recommendation, use this class only where other options are infeasible or impractical. - - Args: - variable_name: name of the variable look for, may be nested, e.g. battery.percentage - name: name of the behaviour - - Raises: - KeyError: if the variable doesn't exist - TypeError: if the variable isn't of type :py:data:`~py_trees.common.Status` - """ - def __init__( - self, - variable_name: str, - name: typing.Union[str, common.Name]=common.Name.AUTO_GENERATED - ): - super().__init__(name=name) - name_components = variable_name.split('.') - self.key = name_components[0] - self.key_attributes = '.'.join(name_components[1:]) # empty string if no other parts - self.variable_name = variable_name - self.blackboard = self.attach_blackboard_client() - self.blackboard.register_key(key=self.key, access=common.Access.READ) - - def update(self) -> common.Status: - """ - Check for existence. - - Returns: - :data:`~py_trees.common.Status.SUCCESS` if key found, :data:`~py_trees.common.Status.FAILURE` otherwise. - """ - if self.logger.level < 1: - self.logger.debug("%s.update()" % self.__class__.__name__) - # raises a KeyError if the variable doesn't exist - status = self.blackboard.get(self.variable_name) - if type(status) != common.Status: - raise TypeError(f"{self.variable_name} is not of type py_trees.common.Status") - if self.logger.level < 1: - self.feedback_message = f"{self.variable_name}: {status}" - return status - - -class CheckBlackboardVariableExists(behaviour.Behaviour): - """ - Check the blackboard to verify if a specific variable (key-value pair) - exists. This is non-blocking, so will always tick with - status :data:`~py_trees.common.Status.FAILURE` - :data:`~py_trees.common.Status.SUCCESS`. - - .. seealso:: - - :class:`~py_trees.behaviours.WaitForBlackboardVariable` for - the blocking counterpart to this behaviour. - - Args: - variable_name: name of the variable look for, may be nested, e.g. battery.percentage - name: name of the behaviour - """ - def __init__( - self, - variable_name: str, - name: typing.Union[str, common.Name]=common.Name.AUTO_GENERATED - ): - super().__init__(name=name) - self.variable_name = variable_name - name_components = variable_name.split('.') - self.key = name_components[0] - self.key_attributes = '.'.join(name_components[1:]) # empty string if no other parts - self.blackboard = self.attach_blackboard_client() - self.blackboard.register_key(key=self.key, access=common.Access.READ) - - def update(self) -> common.Status: - """ - Check for existence. - - Returns: - :data:`~py_trees.common.Status.SUCCESS` if key found, :data:`~py_trees.common.Status.FAILURE` otherwise. - """ - if self.logger.level < 1: - self.logger.debug("%s.update()" % self.__class__.__name__) - try: - unused_value = self.blackboard.get(self.variable_name) - if self.logger.level < 1: - self.feedback_message = "variable '{}' found".format(self.variable_name) - return common.Status.SUCCESS - except KeyError: - if self.logger.level < 1: - self.feedback_message = "variable '{}' not found".format(self.variable_name) - return common.Status.FAILURE - - -class WaitForBlackboardVariable(CheckBlackboardVariableExists): - """ - Wait for the blackboard variable to become available on the blackboard. - This is blocking, so it will tick with - status :data:`~py_trees.common.Status.SUCCESS` if the variable is found, - and :data:`~py_trees.common.Status.RUNNING` otherwise. - - .. seealso:: - - :class:`~py_trees.behaviours.CheckBlackboardVariableExists` for - the non-blocking counterpart to this behaviour. - - Args: - variable_name: name of the variable to wait for, may be nested, e.g. battery.percentage - name: name of the behaviour - """ - def __init__( - self, - variable_name: str, - name: typing.Union[str, common.Name]=common.Name.AUTO_GENERATED - ): - super().__init__(name=name, variable_name=variable_name) - - def update(self) -> common.Status: - """ - Check for existence, wait otherwise. - - Returns: - :data:`~py_trees.common.Status.SUCCESS` if key found, :data:`~py_trees.common.Status.RUNNING` otherwise. - """ - if self.logger.level < 1: - self.logger.debug("%s.update()" % self.__class__.__name__) - new_status = super().update() - # CheckBlackboardExists only returns SUCCESS || FAILURE - if new_status == common.Status.SUCCESS: - if self.logger.level < 1: - self.feedback_message = "'{}' found".format(self.key) - return common.Status.SUCCESS - else: # new_status == common.Status.FAILURE - if self.logger.level < 1: - self.feedback_message = "waiting for key '{}'...".format(self.key) - return common.Status.RUNNING - - -class UnsetBlackboardVariable(behaviour.Behaviour): - """ - Unset the specified variable (key-value pair) from the blackboard. - - This always returns - :data:`~py_trees.common.Status.SUCCESS` regardless of whether - the variable was already present or not. - - Args: - key: unset this key-value pair - name: name of the behaviour - """ - def __init__(self, - key: str, - name: typing.Union[str, common.Name]=common.Name.AUTO_GENERATED, - ): - super().__init__(name=name) - self.key = key - self.blackboard = self.attach_blackboard_client() - self.blackboard.register_key(key=self.key, access=common.Access.WRITE) - - def update(self) -> common.Status: - """ - Unset and always return success. - - Returns: - :data:`~py_trees.common.Status.SUCCESS` - """ - if self.logger.level < 1: - if self.blackboard.unset(self.key): - self.feedback_message = "'{}' found and removed".format(self.key) - else: - self.feedback_message = "'{}' not found, nothing to remove" - return common.Status.SUCCESS - - -class SetBlackboardVariable(behaviour.Behaviour): - """ - Set the specified variable on the blackboard. - - Args: - variable_name: name of the variable to set, may be nested, e.g. battery.percentage - variable_value: value of the variable to set - overwrite: when False, do not set the variable if it already exists - name: name of the behaviour - """ - def __init__( - self, - variable_name: str, - variable_value: typing.Union[typing.Any, typing.Callable[[], typing.Any]], - overwrite: bool = True, - name: typing.Union[str, common.Name]=common.Name.AUTO_GENERATED, - ): - super().__init__(name=name) - self.variable_name = variable_name - name_components = variable_name.split('.') - self.key = name_components[0] - self.key_attributes = '.'.join(name_components[1:]) # empty string if no other parts - self.blackboard = self.attach_blackboard_client() - self.blackboard.register_key(key=self.key, access=common.Access.WRITE) - self.variable_value_generator = variable_value if callable(variable_value) else lambda: variable_value - self.overwrite = overwrite - - def update(self) -> common.Status: - """ - Always return success. - - Returns: - :data:`~py_trees.common.Status.FAILURE` if no overwrite requested and the variable exists, :data:`~py_trees.common.Status.SUCCESS` otherwise - """ - if self.blackboard.set( - self.variable_name, - self.variable_value_generator(), - overwrite=self.overwrite - ): - return common.Status.SUCCESS - else: - return common.Status.FAILURE - - -class CheckBlackboardVariableValue(behaviour.Behaviour): - """ - Inspect a blackboard variable and if it exists, check that it - meets the specified criteria (given by operation type and expected value). - This is non-blocking, so it will always tick with - :data:`~py_trees.common.Status.SUCCESS` or - :data:`~py_trees.common.Status.FAILURE`. - - Args: - check: a comparison expression to check against - name: name of the behaviour - - .. note:: - If the variable does not yet exist on the blackboard, the behaviour will - return with status :data:`~py_trees.common.Status.FAILURE`. - - .. tip:: - The python `operator module`_ includes many useful comparison operations. - """ - def __init__( - self, - check: common.ComparisonExpression, - name: typing.Union[str, common.Name]=common.Name.AUTO_GENERATED - ): - super().__init__(name=name) - self.check = check - name_components = self.check.variable.split('.') - self.key = name_components[0] - self.key_attributes = '.'.join(name_components[1:]) # empty string if no other parts - self.blackboard = self.attach_blackboard_client() - self.blackboard.register_key(key=self.key, access=common.Access.READ) - - def update(self): - """ - Check for existence, or the appropriate match on the expected value. - - Returns: - :class:`~py_trees.common.Status`: :data:`~py_trees.common.Status.FAILURE` if not matched, :data:`~py_trees.common.Status.SUCCESS` otherwise. - """ - if self.logger.level < 1: - self.logger.debug("%s.update()" % self.__class__.__name__) - try: - value = self.blackboard.get(self.key) - if self.key_attributes: - try: - value = operator.attrgetter(self.key_attributes)(value) - except AttributeError: - if self.logger.level < 1: - self.feedback_message = 'blackboard key-value pair exists, but the value does not have the requested nested attributes [{}]'.format(self.variable_name) - return common.Status.FAILURE - except KeyError: - if self.logger.level < 1: - self.feedback_message = "key '{}' does not yet exist on the blackboard".format(self.check.variable) - return common.Status.FAILURE - - success = self.check.operator(value, self.check.value) - - if success: - if self.logger.level < 1: - self.feedback_message = "'%s' comparison succeeded [v: %s][e: %s]" % (self.check.variable, value, self.check.value) - return common.Status.SUCCESS - else: - if self.logger.level < 1: - self.feedback_message = "'%s' comparison failed [v: %s][e: %s]" % (self.check.variable, value, self.check.value) - return common.Status.FAILURE - - -class WaitForBlackboardVariableValue(CheckBlackboardVariableValue): - """ - Inspect a blackboard variable and if it exists, check that it - meets the specified criteria (given by operation type and expected value). - This is blocking, so it will always tick with - :data:`~py_trees.common.Status.SUCCESS` or - :data:`~py_trees.common.Status.RUNNING`. - - .. seealso:: - - :class:`~py_trees.behaviours.CheckBlackboardVariableValue` for - the non-blocking counterpart to this behaviour. - - .. note:: - If the variable does not yet exist on the blackboard, the behaviour will - return with status :data:`~py_trees.common.Status.RUNNING`. - - Args: - check: a comparison expression to check against - name: name of the behaviour - """ - def __init__( - self, - check: common.ComparisonExpression, - name: typing.Union[str, common.Name]=common.Name.AUTO_GENERATED - ): - super().__init__( - check=check, - name=name - ) - - def update(self): - """ - Check for existence, or the appropriate match on the expected value. - - Returns: - :class:`~py_trees.common.Status`: :data:`~py_trees.common.Status.FAILURE` if not matched, :data:`~py_trees.common.Status.SUCCESS` otherwise. - """ - new_status = super().update() - if new_status == common.Status.FAILURE: - return common.Status.RUNNING - else: - return new_status - - -class CheckBlackboardVariableValues(behaviour.Behaviour): - """ - Apply a logical operation across a set of blackboard variable checks. - This is non-blocking, so will always tick with status - :data:`~py_trees.common.Status.FAILURE` or - :data:`~py_trees.common.Status.SUCCESS`. - - Args: - checks: a list of comparison checks to apply to blackboard variables - logical_operator: a logical check to apply across the results of the blackboard variable checks - name: name of the behaviour - namespace: optionally store results of the checks (boolean) under this namespace - - .. tip:: - The python `operator module`_ includes many useful logical operators, e.g. operator.xor. - - Raises: - ValueError if less than two variable checks are specified (insufficient for logical operations) - """ - def __init__( - self, - checks: typing.List[common.ComparisonExpression], - operator: typing.Callable[[bool, bool], bool], - name: typing.Union[str, common.Name]=common.Name.AUTO_GENERATED, - namespace: typing.Optional[str]=None, - ): - super().__init__(name=name) - self.checks = checks - self.operator = operator - self.blackboard = self.attach_blackboard_client() - if len(checks) < 2: - raise ValueError("Must be at least two variables to operate on [only {} provided]".format(len(checks))) - for check in self.checks: - self.blackboard.register_key( - key=blackboard.Blackboard.key(check.variable), - access=common.Access.READ - ) - self.blackboard_results = None - if namespace is not None: - self.blackboard_results = self.attach_blackboard_client(namespace=namespace) - for counter in range(1, len(self.checks) + 1): - self.blackboard_results.register_key( - key=str(counter), - access=common.Access.WRITE - ) - - def update(self) -> common.Status: - """ - Applies comparison checks on each variable and a logical check across the - complete set of variables. - - Returns: - :data:`~py_trees.common.Status.FAILURE` if key retrieval or logical checks failed, :data:`~py_trees.common.Status.SUCCESS` otherwise. - """ - if self.logger.level < 1: - self.logger.debug("%s.update()" % self.__class__.__name__) - results = [] - for check in self.checks: - try: - value = self.blackboard.get(check.variable) - except KeyError: - if self.logger.level < 1: - self.feedback_message = "variable '{}' does not yet exist on the blackboard".format(check.variable) - return common.Status.FAILURE - results.append(check.operator(value, check.value)) - if self.blackboard_results is not None: - for counter in range(1, len(results) + 1): - self.blackboard_results.set(str(counter), results[counter - 1]) - logical_result = functools.reduce(self.operator, results) - if logical_result: - if self.logger.level < 1: - self.feedback_message = "[{}]".format( - "|".join(["T" if result else "F" for result in results]) - ) - return common.Status.SUCCESS - else: - if self.logger.level < 1: - self.feedback_message = "[{}]".format( - "|".join(["T" if result else "F" for result in results]) - ) - return common.Status.FAILURE diff --git a/kios_bt_planning/backups/dynamic_bt/py_trees/blackboard.py b/kios_bt_planning/backups/dynamic_bt/py_trees/blackboard.py deleted file mode 100644 index 0a80ece8..00000000 --- a/kios_bt_planning/backups/dynamic_bt/py_trees/blackboard.py +++ /dev/null @@ -1,1351 +0,0 @@ -#!/usr/bin/env python3 -# -*- coding: utf-8 -*- -# -# License: BSD -# https://raw.githubusercontent.com/splintered-reality/py_trees/devel/LICENSE -# -############################################################################## -# Documentation -############################################################################## - -""" -Blackboards are not a necessary component of behaviour tree implementations, -but are nonetheless, a fairly common mechanism for sharing data between -behaviours in the tree. See, for example, the `design notes`_ -for blackboards in Unreal Engine. - -.. image:: images/blackboard.jpg - :width: 300px - :align: center - -Implementations vary widely depending on the needs of -the framework using them. The simplest implementations take the -form of a key-value store with global access, while more -rigorous implementations scope access or form a secondary -graph overlaying the tree connecting data ports between behaviours. - -The *'Zen of PyTrees'* is to enable rapid development, yet be rich -enough so that *all* of the magic is exposed for debugging purposes. -The first implementation of a blackboard was merely a global key-value -store with an api that lent itself to ease of use, but did not -expose the data sharing between behaviours which meant any tooling -used to introspect or visualise the tree, only told half the story. - -The current implementation adopts a strategy similar to that of a -filesystem. Each client (subsequently behaviour) registers itself -for read/write access to keys on the blackboard. This is less to -do with permissions and more to do with tracking users of keys -on the blackboard - extremely helpful with debugging. - -The alternative approach of layering a secondary data graph -with parameter and input-output ports on each behaviour was -discarded as being too heavy for the zen requirements of py_trees. -This is in part due to the wiring costs, but also due to -complexity arising from a tree's partial graph execution -(a feature which makes trees different from most computational -graph frameworks) and not to regress on py_trees' capability to -dynamically insert and prune subtrees on the fly. - -A high-level list of existing / planned features: - -* [+] Centralised key-value store -* [+] Client connections with namespaced read/write access to the store -* [+] Integration with behaviours for key-behaviour associations (debugging) -* [+] Activity stream that logs read/write operations by clients -* [+] Exclusive locks for writing -* [+] Framework for key remappings - -.. include:: weblinks.rst -""" - -############################################################################## -# Imports -############################################################################## - -import enum -import itertools -import operator -import re -import typing -import uuid - -from py_trees import common -from py_trees import console -from py_trees import utilities - -############################################################################## -# Classes -############################################################################## - - -class KeyMetaData(object): - """ - Stores the aggregated metadata for a key on the blackboard. - """ - def __init__(self): - self.read = set() - self.write = set() - self.exclusive = set() - - -class ActivityType(enum.Enum): - """An enumerator representing the operation on a blackboard variable""" - - READ = "READ" - """Read from the blackboard""" - INITIALISED = "INITIALISED" - """Initialised a key-value pair on the blackboard""" - WRITE = "WRITE" - """Wrote to the blackboard.""" - ACCESSED = "ACCESSED" - """Key accessed, either for reading, or modification of the value's internal attributes (e.g. foo.bar).""" - ACCESS_DENIED = "ACCESS_DENIED" - """Client did not have access to read/write a key.""" - NO_KEY = "NO_KEY" - """Tried to access a key that does not yet exist on the blackboard.""" - NO_OVERWRITE = "NO_OVERWRITE" - """Tried to write but variable already exists and a no-overwrite request was respected.""" - UNSET = "UNSET" - """Key was removed from the blackboard""" - - -class ActivityItem(object): - """ - Recorded data pertaining to activity on the blackboard. - - Args: - key: name of the variable on the blackboard - client_name: convenient name of the client performing the operation - client_id: unique id of the client performing the operation - activity_type: type of activity - previous_value: of the given key (None if this field is not relevant) - current_value: current value for the given key (None if this field is not relevant) - """ - def __init__( - self, - key, - client_name: str, - client_id: uuid.UUID, - activity_type: str, - previous_value: typing.Any=None, - current_value: typing.Any=None): - # TODO validity checks for values passed/not passed on the - # respective activity types. Note: consider using an enum - # for 'no value here' since None is a perfectly acceptable - # value for a key - self.key = key - self.client_name = client_name - self.client_id = client_id - self.activity_type = activity_type - self.previous_value = previous_value - self.current_value = current_value - - -class ActivityStream(object): - """ - Storage container with convenience methods for manipulating the stored - activity stream. - - Attributes: - data (typing.List[ActivityItem]: list of activity items, earliest first - maximum_size (int): pop items if this size is exceeded - """ - - def __init__(self, maximum_size: int=500): - """ - Initialise the stream with a maximum storage limit. - - Args: - maximum_size: pop items from the stream if this size is exceeded - """ - self.data: typing.List[ActivityItem] = [] - self.maximum_size = maximum_size - - def push(self, activity_item: ActivityItem): - """ - Push the next activity item to the stream. - - Args: - activity_item: new item to append to the stream - """ - if len(self.data) > self.maximum_size: - self.data.pop() - self.data.append(activity_item) - - def clear(self): - """ - Delete all activities from the stream. - """ - self.data = [] - - -class Blackboard(object): - """ - Centralised key-value store for sharing data between behaviours. - This class is a coat-hanger for the centralised data store, metadata - for it's administration and static methods for interacting with it. - - This api is intended for authors of debugging and introspection - tools on the blackboard. Users should make use of the :class:`Client`. - - Attributes: - Blackboard.clients (typing.Dict[uuid.UUID, str]): client uuid-name registry - Blackboard.storage (typing.Dict[str, typing.Any]): key-value data store - Blackboard.metadata (typing.Dict[str, KeyMetaData]): key associated metadata - Blackboard.activity_stream (ActivityStream): logged activity - Blackboard.separator (char): namespace separator character - """ - storage: typing.Dict[str, typing.Any] = {} # key-value storage - metadata: typing.Dict[str, KeyMetaData] = {} # key-metadata information - clients: typing.Dict[uuid.UUID, str] = {} # client id-name pairs - activity_stream: typing.Optional[ActivityStream] = None - separator: str = "/" - - @staticmethod - def keys() -> typing.Set[str]: - """ - Get the set of blackboard keys. - - Returns: - the complete set of keys registered by clients - """ - # return registered keys, those on the blackboard are not - # necessarily written to yet - return set(Blackboard.metadata.keys()) - - @staticmethod - def get(variable_name: str) -> typing.Any: - """ - Extract the value associated with the given a variable name, - can be nested, e.g. battery.percentage. This differs from the - client get method in that it doesn't pass through the client access - checks. To be used for utility tooling (e.g. display methods) and not by - users directly. - - Args: - variable_name: of the variable to get, can be nested, e.g. battery.percentage - - Raises: - KeyError: if the variable or it's nested attributes do not yet exist on the blackboard - - Return: - The stored value for the given variable - """ - variable_name = Blackboard.absolute_name(Blackboard.separator, variable_name) - name_components = variable_name.split('.') - key = name_components[0] - key_attributes = '.'.join(name_components[1:]) - # can raise KeyError - value = Blackboard.storage[key] - if key_attributes: - try: - value = operator.attrgetter(key_attributes)(value) - except AttributeError: - raise KeyError("Key exists, but does not have the specified nested attributes [{}]".format(variable_name)) - return value - - @staticmethod - def set(variable_name: str, value: typing.Any): - """ - Set the value associated with the given a variable name, - can be nested, e.g. battery.percentage. This differs from the - client get method in that it doesn't pass through the client access - checks. To be used for utility tooling (e.g. display methods) and not by - users directly. - - Args: - variable_name: of the variable to set, can be nested, e.g. battery.percentage - - Raises: - AttributeError: if it is attempting to set a nested attribute tha does not exist. - """ - variable_name = Blackboard.absolute_name(Blackboard.separator, variable_name) - name_components = variable_name.split('.') - key = name_components[0] - key_attributes = '.'.join(name_components[1:]) - if not key_attributes: - Blackboard.storage[key] = value - else: - setattr(Blackboard.storage[key], key_attributes, value) - Blackboard.metadata.setdefault(key, KeyMetaData()) - - @staticmethod - def unset(key: str): - """ - For when you need to completely remove a blackboard variable (key-value pair), - this provides a convenient helper method. - - Args: - key: name of the variable to remove - - Returns: - True if the variable was removed, False if it was already absent - """ - try: - key = Blackboard.absolute_name(Blackboard.separator, key) - del Blackboard.storage[key] - return True - except KeyError: - return False - - @staticmethod - def exists(name: str) -> bool: - """ - Check if the specified variable exists on the blackboard. - - Args: - name: name of the variable, can be nested, e.g. battery.percentage - - Raises: - AttributeError: if the client does not have read access to the variable - """ - try: - name = Blackboard.absolute_name(Blackboard.separator, name) - unused_value = Blackboard.get(name) - return True - except KeyError: - return False - - @staticmethod - def keys_filtered_by_regex(regex: str) -> typing.Set[str]: - """ - Get the set of blackboard keys filtered by regex. - - Args: - regex: a python regex string - - Returns: - subset of keys that have been registered and match the pattern - """ - pattern = re.compile(regex) - return {key for key in Blackboard.metadata.keys() if pattern.search(key) is not None} - - @staticmethod - def keys_filtered_by_clients( - client_ids: typing.Union[typing.Set[uuid.UUID], typing.List[uuid.UUID]] - ) -> typing.Set[str]: - """ - Get the set of blackboard keys filtered by client unique identifiers. - - Args: - client_ids: set of client uuid's. - - Returns: - subset of keys that have been registered by the specified clients - """ - # forgive users if they sent a list instead of a set - if isinstance(client_ids, list): - client_ids = set(client_ids) - keys = set() - for key in Blackboard.metadata.keys(): - # for sets, | is union, & is intersection - key_clients = ( - set(Blackboard.metadata[key].read) | - set(Blackboard.metadata[key].write) | - set(Blackboard.metadata[key].exclusive) - ) - if key_clients & client_ids: - keys.add(key) - return keys - - @staticmethod - def enable_activity_stream(maximum_size: int=500): - """ - Enable logging of activities on the blackboard. - - Args: - maximum_size: pop items from the stream if this size is exceeded - - Raises: - RuntimeError if the activity stream is already enabled - """ - if Blackboard.activity_stream is None: - Blackboard.activity_stream = ActivityStream(maximum_size) - else: - RuntimeError("activity stream is already enabled for this blackboard") - - @staticmethod - def disable_activity_stream(): - """ - Disable logging of activities on the blackboard - """ - Blackboard.activity_stream = None - - @staticmethod - def clear(): - """ - Completely clear all key, value and client information from the blackboard. - Also deletes the activity stream. - """ - Blackboard.storage.clear() - Blackboard.metadata.clear() - Blackboard.clients.clear() - Blackboard.activity_stream = None - - @staticmethod - def absolute_name(namespace: str, key: str) -> str: - """ - Generate the fully qualified key name from namespace and name arguments. - - **Examples** - - .. code-block:: python - - '/' + 'foo' = '/foo' - '/' + '/foo' = '/foo' - '/foo' + 'bar' = '/foo/bar' - '/foo/' + 'bar' = '/foo/bar' - '/foo' + '/foo/bar' = '/foo/bar' - '/foo' + '/bar' = '/bar' - '/foo' + 'foo/bar' = '/foo/foo/bar' - - Args: - namespace: namespace the key should be embedded in - key: key name (relative or absolute) - - Returns: - the absolute name - - .. warning:: - - To expedite the method call (it's used with high frequency - in blackboard key lookups), no checks are made to ensure - the namespace argument leads with a "/". Nor does it check - that a name in absolute form is actually embedded in the - specified namespace, it just returns the given (absolute) - name directly. - """ - # it's already absolute - if key.startswith(Blackboard.separator): - return key - # remove leading and trailing separators - namespace = namespace if namespace.endswith(Blackboard.separator) else namespace + Blackboard.separator - key = key.strip(Blackboard.separator) - return "{}{}".format(namespace, key) - - @staticmethod - def relative_name(namespace: str, key: str) -> str: - """ - **Examples** - - .. code-block:: python - - '/' + 'foo' = '/foo' - '/' + '/foo' = '/foo' - '/foo' + 'bar' = '/foo/bar' - '/foo/' + 'bar' = '/foo/bar' - '/foo' + '/bar' => KeyError('/bar' is not in 'foo') - '/foo' + 'foo/bar' = '/foo/foo/bar' - - Args: - namespace: namespace the key should be embedded in - key: key name (relative or absolute) - - Returns: - the absolute name - - Raises: - KeyError if the key is not in the specified namespace - - .. warning:: - - To expedite the method call (it's used with high frequency - in blackboard key lookups), no checks are made to ensure - the namespace argument leads with a "/" - """ - # it's already relative - if not key.startswith(Blackboard.separator): - return key - # remove leading and trailing separators - namespace = namespace if namespace.endswith(Blackboard.separator) else namespace + Blackboard.separator - if key.startswith(namespace): - return key.lstrip(namespace) - else: - raise KeyError("key '{}' is not in namespace '{}'".format( - key, namespace) - ) - - @staticmethod - def key(variable_name: str) -> str: - """ - Extract the key for an arbitrary blackboard variable, keeping in mind that blackboard variable - names can be pointing to a nested attribute within a key. - - Example: '/foo/bar.woohoo -> /foo/bar'. - - Args: - variable_name: blackboard variable name - can be nested, e.g. battery.percentage - - Returns: - name of the underlying key - """ - name_components = variable_name.split('.') - key = name_components[0] - return key - - @staticmethod - def key_with_attributes(variable_name: str) -> typing.Tuple[str, str]: - """ - Extract the key for an arbitrary blackboard variable, keeping in mind that blackboard variable - names can be pointing to a nested attribute within a key, - - Example: '/foo/bar.woohoo -> (/foo/bar'. 'woohoo') - - Args: - variable_name: blackboard variable name - can be nested, e.g. battery.percentage - - Returns: - a tuple consisting of the key and it's attributes (in string form) - """ - name_components = variable_name.split('.') - key = name_components[0] - key_attributes = '.'.join(name_components[1:]) - return (key, key_attributes) - - -class Client(object): - """ - Client to the key-value store for sharing data between behaviours. - - **Examples** - - Blackboard clients will accept a user-friendly name or create one - for you if none is provided. Regardless of what name is chosen, clients - are always uniquely identified via a uuid generated on construction. - - .. code-block:: python - - provided = py_trees.blackboard.Client(name="Provided") - print(provided) - generated = py_trees.blackboard.Client() - print(generated) - - .. figure:: images/blackboard_client_instantiation.png - :align: center - - Client Instantiation - - Register read/write access for keys on the blackboard. Note, registration is - not initialisation. - - .. code-block:: python - - blackboard = py_trees.blackboard.Client(name="Client") - blackboard.register_key(key="foo", access=py_trees.common.Access.WRITE) - blackboard.register_key(key="bar", access=py_trees.common.Access.READ) - blackboard.foo = "foo" - print(blackboard) - - .. figure:: images/blackboard_read_write.png - :align: center - - Variable Read/Write Registration - - Keys and clients can make use of namespaces, designed by the '/' char. Most - methods permit a flexible expression of either relative or absolute names. - - .. code-block:: python - - blackboard = py_trees.blackboard.Client(name="Global") - parameters = py_trees.blackboard.Client(name="Parameters", namespace="parameters") - - blackboard.register_key(key="foo", access=py_trees.common.Access.WRITE) - blackboard.register_key(key="/bar", access=py_trees.common.Access.WRITE) - blackboard.register_key(key="/parameters/default_speed", access=py_trees.common.Access.WRITE) - parameters.register_key(key="aggressive_speed", access=py_trees.common.Access.WRITE) - - blackboard.foo = "foo" - blackboard.bar = "bar" - blackboard.parameters.default_speed = 20.0 - parameters.aggressive_speed = 60.0 - - miss_daisy = blackboard.parameters.default_speed - van_diesel = parameters.aggressive_speed - - print(blackboard) - print(parameters) - - .. figure:: images/blackboard_namespaces.png - :align: center - - Namespaces and Namespaced Clients - - - Disconnected instances will discover the centralised - key-value store. - - .. code-block:: python - - def check_foo(): - blackboard = py_trees.blackboard.Client(name="Reader") - blackboard.register_key(key="foo", access=py_trees.common.Access.READ) - print("Foo: {}".format(blackboard.foo)) - - - blackboard = py_trees.blackboard.Client(name="Writer") - blackboard.register_key(key="foo", access=py_trees.common.Access.WRITE) - blackboard.foo = "bar" - check_foo() - - To respect an already initialised key on the blackboard: - - .. code-block:: python - - blackboard = Client(name="Writer") - blackboard.register_key(key="foo", access=py_trees.common.Access.READ) - result = blackboard.set("foo", "bar", overwrite=False) - - Store complex objects on the blackboard: - - .. code-block:: python - - class Nested(object): - def __init__(self): - self.foo = None - self.bar = None - - def __str__(self): - return str(self.__dict__) - - - writer = py_trees.blackboard.Client(name="Writer") - writer.register_key(key="nested", access=py_trees.common.Access.WRITE) - reader = py_trees.blackboard.Client(name="Reader") - reader.register_key(key="nested", access=py_trees.common.Access.READ) - - writer.nested = Nested() - writer.nested.foo = "I am foo" - writer.nested.bar = "I am bar" - - foo = reader.nested.foo - print(writer) - print(reader) - - .. figure:: images/blackboard_nested.png - :align: center - - Log and display the activity stream: - - .. code-block:: python - - py_trees.blackboard.Blackboard.enable_activity_stream(maximum_size=100) - reader = py_trees.blackboard.Client(name="Reader") - reader.register_key(key="foo", access=py_trees.common.Access.READ) - writer = py_trees.blackboard.Client(name="Writer") - writer.register_key(key="foo", access=py_trees.common.Access.WRITE) - writer.foo = "bar" - writer.foo = "foobar" - unused_result = reader.foo - print(py_trees.display.unicode_blackboard_activity_stream()) - py_trees.blackboard.Blackboard.activity_stream.clear() - - .. figure:: images/blackboard_activity_stream.png - :align: center - - Display the blackboard on the console, or part thereof: - - .. code-block:: python - - writer = py_trees.blackboard.Client(name="Writer") - for key in {"foo", "bar", "dude", "dudette"}: - writer.register_key(key=key, access=py_trees.common.Access.WRITE) - - reader = py_trees.blackboard.Client(name="Reader") - for key in {"foo", "bar"}: - reader.register_key(key="key", access=py_trees.common.Access.READ) - - writer.foo = "foo" - writer.bar = "bar" - writer.dude = "bob" - - # all key-value pairs - print(py_trees.display.unicode_blackboard()) - # various filtered views - print(py_trees.display.unicode_blackboard(key_filter={"foo"})) - print(py_trees.display.unicode_blackboard(regex_filter="dud*")) - print(py_trees.display.unicode_blackboard(client_filter={reader.unique_identifier})) - # list the clients associated with each key - print(py_trees.display.unicode_blackboard(display_only_key_metadata=True)) - - .. figure:: images/blackboard_display.png - :align: center - - Behaviours are not automagically connected to the blackboard but you may - manually attach one or more clients so that associations between behaviours - and variables can be tracked - this is very useful for introspection and - debugging. - - Creating a custom behaviour with blackboard variables: - - .. code-block:: python - - class Foo(py_trees.behaviour.Behaviour): - - def __init__(self, name): - super().__init__(name=name) - self.blackboard = self.attach_blackboard_client(name="Foo Global") - self.parameters = self.attach_blackboard_client(name="Foo Params", namespace="foo_parameters_") - self.state = self.attach_blackboard_client(name="Foo State", namespace="foo_state_") - - # create a key 'foo_parameters_init' on the blackboard - self.parameters.register_key("init", access=py_trees.common.Access.READ) - # create a key 'foo_state_number_of_noodles' on the blackboard - self.state.register_key("number_of_noodles", access=py_trees.common.Access.WRITE) - - def initialise(self): - self.state.number_of_noodles = self.parameters.init - - def update(self): - self.state.number_of_noodles += 1 - self.feedback_message = self.state.number_of_noodles - if self.state.number_of_noodles > 5: - return py_trees.common.Status.SUCCESS - else: - return py_trees.common.Status.RUNNING - - - # could equivalently do directly via the Blackboard static methods if - # not interested in tracking / visualising the application configuration - configuration = py_trees.blackboard.Client(name="App Config") - configuration.register_key("foo_parameters_init", access=py_trees.common.Access.WRITE) - configuration.foo_parameters_init = 3 - - foo = Foo(name="The Foo") - for i in range(1, 8): - foo.tick_once() - print("Number of Noodles: {}".format(foo.feedback_message)) - - Rendering a dot graph for a behaviour tree, complete with blackboard variables: - - .. code-block:: python - - # in code - py_trees.display.render_dot_tree(py_trees.demos.blackboard.create_root()) - # command line tools - py-trees-render --with-blackboard-variables py_trees.demos.blackboard.create_root - - .. graphviz:: dot/demo-blackboard.dot - :align: center - :caption: Tree with Blackboard Variables - - And to demonstrate that it doesn't become a tangled nightmare at scale, an example of - a more complex tree: - - .. graphviz:: dot/blackboard-with-variables.dot - :align: center - :caption: A more complex tree - - Debug deeper with judicious application of the tree, blackboard and activity stream - display methods around the tree tick (refer to - :class:`py_trees.visitors.DisplaySnapshotVisitor` for examplar code): - - .. figure:: images/blackboard_trees.png - :align: center - - Tree level debugging - - .. seealso:: - - * :ref:`py-trees-demo-blackboard ` - * :ref:`py-trees-demo-namespaces ` - * :ref:`py-trees-demo-remappings ` - * :class:`py_trees.visitors.DisplaySnapshotVisitor` - * :class:`py_trees.behaviours.SetBlackboardVariable` - * :class:`py_trees.behaviours.UnsetBlackboardVariable` - * :class:`py_trees.behaviours.CheckBlackboardVariableExists` - * :class:`py_trees.behaviours.WaitForBlackboardVariable` - * :class:`py_trees.behaviours.CheckBlackboardVariableValue` - * :class:`py_trees.behaviours.WaitForBlackboardVariableValue` - - Attributes: - name (str): client's convenient, but not necessarily unique identifier - namespace (str): apply this as a prefix to any key/variable name operations - unique_identifier (uuid.UUID): client's unique identifier - read (typing.Set[str]): set of absolute key names with read access - write (typing.Set[str]): set of absolute key names with write access - exclusive (typing.Set[str]): set of absolute key names with exclusive write access - required (typing.Set[str]): set of absolute key names required to have data present - remappings (typing.Dict[str, str]: client key names with blackboard remappings - namespaces (typing.Set[str]: a cached list of namespaces this client accesses - """ - def __init__( - self, *, - name: str=None, - namespace: str=None): - """ - Args: - name: client's convenient identifier (stringifies the uuid if None) - namespace: prefix to apply to key/variable name operations - read: list of keys for which this client has read access - write: list of keys for which this client has write access - exclusive: list of keys for which this client has exclusive write access - - Raises: - TypeError: if the provided name is not of type str - ValueError: if the unique identifier has already been registered - """ - - # unique identifier - super().__setattr__("unique_identifier", uuid.uuid4()) - if super().__getattribute__("unique_identifier") in Blackboard.clients.keys(): - raise ValueError("this unique identifier has already been registered") - - # name - if name is None or not name: - name = utilities.truncate( - original=str(super().__getattribute__("unique_identifier")).replace('-', '_'), - length=7 - ) - super().__setattr__("name", name) - else: - if not isinstance(name, str): - raise TypeError("provided name is not of type str [{}]".format(type(name))) - super().__setattr__("name", name) - - # namespaces - namespace = "" if namespace is None else namespace - if not namespace.startswith(Blackboard.separator): - namespace = Blackboard.separator + namespace - super().__setattr__("namespace", namespace) - super().__setattr__("namespaces", set()) - - super().__setattr__("read", set()) - super().__setattr__("write", set()) - super().__setattr__("exclusive", set()) - super().__setattr__("required", set()) - super().__setattr__("remappings", {}) - Blackboard.clients[ - super().__getattribute__("unique_identifier") - ] = self.name - - def id(self) -> uuid.UUID: - """ - The unique identifier for this client. - - Returns: - The uuid.UUID object - """ - return super().__getattribute__("unique_identifier") - - def __setattr__(self, name: str, value: typing.Any): - """ - Convenience attribute style referencing with checking against - permissions. - - Raises: - AttributeError: if the client does not have write access to the variable - """ - # print("__setattr__ [{}][{}]".format(name, value)) - name = Blackboard.absolute_name(super().__getattribute__("namespace"), name) - if ( - (name not in super().__getattribute__("write")) and - (name not in super().__getattribute__("exclusive")) - ): - if Blackboard.activity_stream is not None: - Blackboard.activity_stream.push( - self._generate_activity_item(name, ActivityType.ACCESS_DENIED) - ) - raise AttributeError("client '{}' does not have write access to '{}'".format(self.name, name)) - remapped_name = super().__getattribute__("remappings")[name] - if Blackboard.activity_stream is not None: - if remapped_name in Blackboard.storage.keys(): - Blackboard.activity_stream.push( - self._generate_activity_item( - key=remapped_name, - activity_type=ActivityType.WRITE, - previous_value=Blackboard.storage[remapped_name], - current_value=value - ) - ) - else: - Blackboard.activity_stream.push( - self._generate_activity_item( - key=remapped_name, - activity_type=ActivityType.INITIALISED, - current_value=value - ) - ) - Blackboard.storage[remapped_name] = value - - def __getattr__(self, name: str): - """ - Convenience attribute style referencing with checking against - permissions. - - Raises: - AttributeError: if the client does not have read access to the variable - KeyError: if the variable does not yet exist on the blackboard - """ - # print("__getattr__ [{}]".format(name)) - name = Blackboard.absolute_name(super().__getattribute__("namespace"), name) - read_key = False - write_key = False - if name in super().__getattribute__("read"): - read_key = True - elif name in super().__getattribute__("write"): - write_key = True - elif name in super().__getattribute__("exclusive"): - write_key = True - else: - if name in super().__getattribute__("namespaces"): - return IntermediateVariableFetcher(blackboard=self, namespace=name) - if Blackboard.activity_stream is not None: - Blackboard.activity_stream.push( - self._generate_activity_item(name, ActivityType.ACCESS_DENIED) - ) - raise AttributeError("client '{}' does not have read/write access to '{}'".format(self.name, name)) - remapped_name = super().__getattribute__("remappings")[name] - try: - if write_key: - if Blackboard.activity_stream is not None: - if utilities.is_primitive(Blackboard.storage[remapped_name]): - activity_type = ActivityType.READ - else: # could be a nested class object being accessed to write an attribute - activity_type = ActivityType.ACCESSED - Blackboard.activity_stream.push( - self._generate_activity_item( - key=remapped_name, - activity_type=activity_type, - current_value=Blackboard.storage[remapped_name], - ) - ) - return Blackboard.storage[remapped_name] - if read_key: - if Blackboard.activity_stream is not None: - Blackboard.activity_stream.push( - self._generate_activity_item( - key=remapped_name, - activity_type=ActivityType.READ, - current_value=Blackboard.storage[remapped_name], - ) - ) - return Blackboard.storage[remapped_name] - except KeyError as e: - if Blackboard.activity_stream is not None: - Blackboard.activity_stream.push( - self._generate_activity_item(remapped_name, ActivityType.NO_KEY) - ) - raise KeyError("client '{}' tried to access '{}' but it does not yet exist on the blackboard".format(self.name, remapped_name)) from e - - def set(self, name: str, value: typing.Any, overwrite: bool=True) -> bool: - """ - Set, conditionally depending on whether the variable already exists or otherwise. - - This is most useful when initialising variables and multiple elements - seek to do so. A good policy to adopt for your applications in these situations is - a first come, first served policy. Ensure global configuration has the first - opportunity followed by higher priority behaviours in the tree and so forth. - Lower priority behaviours would use this to respect the pre-configured - setting and at most, just validate that it is acceptable to the functionality - of it's own behaviour. - - Args: - name: name of the variable to set - value: value of the variable to set - overwrite: do not set if the variable already exists on the blackboard - - Returns: - success or failure (overwrite is False and variable already set) - - Raises: - AttributeError: if the client does not have write access to the variable - KeyError: if the variable does not yet exist on the blackboard - """ - name = Blackboard.absolute_name(super().__getattribute__("namespace"), name) - name_components = name.split('.') - key = name_components[0] - key_attributes = '.'.join(name_components[1:]) - if ( - (key not in super().__getattribute__("write")) and - (key not in super().__getattribute__("exclusive")) - ): - if Blackboard.activity_stream is not None: - Blackboard.activity_stream.push( - self._generate_activity_item(key, ActivityType.ACCESS_DENIED) - ) - raise AttributeError("client '{}' does not have write access to '{}'".format(self.name, name)) - remapped_key = super().__getattribute__("remappings")[key] - if not overwrite: - if remapped_key in Blackboard.storage: - if Blackboard.activity_stream is not None: - Blackboard.activity_stream.push( - self._generate_activity_item( - key=remapped_key, - activity_type=ActivityType.NO_OVERWRITE, - current_value=Blackboard.storage[remapped_key]) - ) - return False - if not key_attributes: - setattr(self, key, value) - return True - else: - blackboard_object = getattr(self, key) - try: - setattr(blackboard_object, key_attributes, value) - return True - except AttributeError: # when the object doesn't have the attributes - return False - - def exists(self, name: str) -> bool: - """ - Check if the specified variable exists on the blackboard. - - Args: - name: name of the variable to get, can be nested, e.g. battery.percentage - - Raises: - AttributeError: if the client does not have read access to the variable - """ - try: - unused_value = self.get(name) - return True - except KeyError: - return False - - def absolute_name(self, key: str) -> str: - """ - Generate the fully qualified key name for this key. - - .. code-block:: python - - blackboard = Client(name="FooBar", namespace="foo") - blackboard.register_key(key="bar", access=py_trees.common.Access.READ) - print("{}".format(blackboard.absolute_name("bar"))) # "/foo/bar" - - Args: - key: name of the key - - Returns: - the absolute name - - Raises: - KeyError: if the key is not registered with this client - """ - if not self.is_registered(key=key): - raise KeyError("key '{}' is not in namespace '{}'".format( - key, super().__getattribute__("namespace")) - ) - return Blackboard.absolute_name( - super().__getattribute__("namespace"), - key - ) - - def get(self, name: str) -> typing.Any: - """ - Method based accessor to the blackboard variables (as opposed to simply using - '.'). - - Args: - name: name of the variable to get, can be nested, e.g. battery.percentage - - Raises: - AttributeError: if the client does not have read access to the variable - KeyError: if the variable or it's nested attributes do not yet exist on the blackboard - """ - # key attributes is an empty string if not a nested variable name - name_components = name.split('.') - key = name_components[0] - key_attributes = '.'.join(name_components[1:]) - value = getattr(self, key) # will run through client access checks in __getattr__ - if key_attributes: - try: - value = operator.attrgetter(key_attributes)(value) - except AttributeError: - raise KeyError("Key exists, but does not have the specified nested attributes [{}]".format(name)) - return value - - def unset(self, key: str): - """ - For when you need to completely remove a blackboard variable (key-value pair), - this provides a convenient helper method. - - Args: - key: name of the variable to remove - - Returns: - True if the variable was removed, False if it was already absent - """ - key = Blackboard.absolute_name(super().__getattribute__("namespace"), key) - remapped_key = super().__getattribute__("remappings")[key] - if Blackboard.activity_stream is not None: - Blackboard.activity_stream.push( - self._generate_activity_item(remapped_key, ActivityType.UNSET) - ) - # Three means of handling a non-existent key - 1) raising a KeyError, 2) catching - # the KeyError and passing, 3) catch the KeyError and return True/False. - # Option 1) is inconvenient - requires a redundant try/catch 99% of cases - # Option 2) hides information - bad - # Option 3) no extra code necessary and information is there if desired - try: - del Blackboard.storage[remapped_key] - return True - except KeyError: - return False - - def _generate_activity_item(self, key, activity_type, previous_value=None, current_value=None): - return ActivityItem( - key=key, - client_name=super().__getattribute__("name"), - client_id=super().__getattribute__("unique_identifier"), - # use strings here, so displaying the streams is agnostic of the enum - activity_type=activity_type.value, - previous_value=previous_value, - current_value=current_value - ) - - def _update_namespaces(self, added_key=None): - """ - Update the namespace cache. - - Args: - added_key: hint on the most recent operation to enable an smart check/rebuild - """ - if added_key is not None: - namespace = added_key.rsplit("/", 1)[0] - while namespace: - super().__getattribute__("namespaces").add(namespace) - namespace = namespace.rsplit("/", 1)[0] - else: - # completely rebuild - super().__getattribute__("namespaces").clear() - for key in itertools.chain( - super().__getattribute__("read"), - super().__getattribute__("write"), - super().__getattribute__("exclusive") - ): - namespace = key.rsplit("/", 1)[0] - while namespace: - super().__getattribute__("namespaces").add(namespace) - namespace = namespace.rsplit("/", 1)[0] - - def __str__(self): - indent = " " - s = console.green + "Blackboard Client" + console.reset + "\n" - s += console.white + indent + "Client Data" + console.reset + "\n" - keys = ["name", "namespace", "unique_identifier", "read", "write", "exclusive"] - s += self._stringify_key_value_pairs(keys, self.__dict__, 2 * indent) - keys = {k for k, v in self.remappings.items() if k != v} - if keys: - s += console.white + indent + "Remappings" + console.reset + "\n" - s += self._stringify_key_value_pairs( - keys=keys, - key_value_dict=self.remappings, - indent=2 * indent, - separator=console.right_arrow - ) - s += console.white + indent + "Variables" + console.reset + "\n" - keys = self.remappings.values() - s += self._stringify_key_value_pairs(keys, Blackboard.storage, 2 * indent) - return s - - def _stringify_key_value_pairs(self, keys, key_value_dict, indent, separator=":"): - s = "" - max_length = 0 - for key in keys: - max_length = len(key) if len(key) > max_length else max_length - for key in keys: - try: - value = key_value_dict[key] - lines = ('{0}'.format(value)).split('\n') - if len(lines) > 1: - s += console.cyan + indent + '{0: <{1}}'.format(key, max_length + 1) + console.reset + separator + "\n" - for line in lines: - s += console.yellow + indent + " {0}\n".format(line) + console.reset - else: - s += console.cyan + indent + '{0: <{1}}'.format(key, max_length + 1) + console.reset + separator + " " + console.yellow + '{0}\n'.format(value) + console.reset - except KeyError: - s += console.cyan + indent + '{0: <{1}}'.format(key, max_length + 1) + console.reset + separator + " " + console.yellow + "-\n" + console.reset - s += console.reset - return s - - def unregister(self, clear: bool=True): - """ - Unregister this blackboard client and if requested, clear key-value pairs if this - client is the last user of those variables. - - Args: - clear: remove key-values pairs from the blackboard - """ - self.unregister_all_keys(clear) - del Blackboard.clients[super().__getattribute__("unique_identifier")] - - def unregister_all_keys(self, clear: bool=True): - """ - Unregister all keys currently registered by this blackboard client and if requested, - clear key-value pairs if this client is the last user of those variables. - - Args: - clear: remove key-values pairs from the blackboard - """ - for key in itertools.chain(set(self.read), set(self.write), set(self.exclusive)): - self.unregister_key(key=key, clear=clear, update_namespace_cache=False) - self._update_namespaces() - - def verify_required_keys_exist(self): - """ - En-masse check of existence on the blackboard for all keys that were hitherto - registered as 'required'. - - Raises: KeyError if any of the required keys do not exist on the blackboard - """ - absent = set() - for key in super().__getattribute__("required"): - if not self.exists(key): - absent.add(key) - if absent: - raise KeyError("keys required, but not yet on the blackboard [{}]".format(absent)) - - def is_registered( - self, - key: str, - access: typing.Union[None, common.Access]=None - ) -> bool: - """ - Check to see if the specified key is registered. - - Args: - key: in either relative or absolute form - access: access property, if None, just checks for registration, regardless of property - - Returns: - if registered, True otherwise False - """ - absolute_name = Blackboard.absolute_name( - super().__getattribute__("namespace"), - key - ) - if access == common.Access.READ: - return absolute_name in self.read - elif access == common.Access.WRITE: - return absolute_name in self.write - elif access == common.Access.EXCLUSIVE_WRITE: - return absolute_name in self.exclusive - else: - return absolute_name in self.read | self.write | self.exclusive - - def register_key( - self, - key: str, - access: common.Access, - required: bool=False, - remap_to: str=None, - ): - """ - Register a key on the blackboard to associate with this client. - - Args: - key: key to register - access: access level (read, write, exclusive write) - required: if true, check key exists when calling - :meth:`~verify_required_keys_exist` - remap_to: remap the key to this location on the blackboard - - Note the remap simply changes the storage location. From the perspective of - the client, access via the specified 'key' remains the same. - - Raises: - AttributeError if exclusive write access is requested, but write access has already been given to another client - TypeError if the access argument is of incorrect type - """ - key = Blackboard.absolute_name(super().__getattribute__("namespace"), key) - super().__getattribute__("remappings")[key] = key if remap_to is None else remap_to - remapped_key = super().__getattribute__("remappings")[key] - if access == common.Access.READ: - super().__getattribute__("read").add(key) - Blackboard.metadata.setdefault(remapped_key, KeyMetaData()) - Blackboard.metadata[remapped_key].read.add(super().__getattribute__("unique_identifier")) - elif access == common.Access.WRITE: - conflicts = set() - try: - for unique_identifier in Blackboard.metadata[remapped_key].exclusive: - conflicts.add(Blackboard.clients[unique_identifier]) - if conflicts: - raise AttributeError("'{}' requested write on key '{}', but this key already associated with an exclusive writer[{}]".format( - super().__getattribute__("name"), - remapped_key, - conflicts) - ) - except KeyError: - pass # no readers or writers on the key yet - super().__getattribute__("write").add(key) - Blackboard.metadata.setdefault(remapped_key, KeyMetaData()) - Blackboard.metadata[remapped_key].write.add(super().__getattribute__("unique_identifier")) - elif access == common.Access.EXCLUSIVE_WRITE: - try: - key_metadata = Blackboard.metadata[remapped_key] - conflicts = set() - for unique_identifier in (key_metadata.write | key_metadata.exclusive): - conflicts.add(Blackboard.clients[unique_identifier]) - if conflicts: - raise AttributeError("'{}' requested exclusive write on key '{}', but this key is already associated [{}]".format( - super().__getattribute__("name"), - remapped_key, - conflicts) - ) - except KeyError: - pass # no readers or writers on the key yet - super().__getattribute__("exclusive").add(key) - Blackboard.metadata.setdefault(remapped_key, KeyMetaData()) - Blackboard.metadata[remapped_key].exclusive.add(super().__getattribute__("unique_identifier")) - else: - raise TypeError("access argument is of incorrect type [{}]".format(type(access))) - if required: - super().__getattribute__("required").add(key) - self._update_namespaces(added_key=key) - - def unregister_key( - self, - key: str, - clear: bool=True, - update_namespace_cache: bool=True): - """ - Unegister a key associated with this client. - - Args: - key: key to unregister - clear: remove key-values pairs from the blackboard - update_namespace_cache: disable if you are batching - - A method that batches calls to this method is :meth:`unregister_all_keys()`. - - Raises: - KeyError if the key has not been previously registered - """ - key = Blackboard.absolute_name(super().__getattribute__("namespace"), key) - remapped_key = super().__getattribute__("remappings")[key] - super().__getattribute__("read").discard(key) # doesn't throw exceptions if it not present - super().__getattribute__("write").discard(key) - super().__getattribute__("exclusive").discard(key) - Blackboard.metadata[remapped_key].read.discard(super().__getattribute__("unique_identifier")) - Blackboard.metadata[remapped_key].write.discard(super().__getattribute__("unique_identifier")) - Blackboard.metadata[remapped_key].exclusive.discard(super().__getattribute__("unique_identifier")) - if ( - (not Blackboard.metadata[remapped_key].read) and - (not Blackboard.metadata[remapped_key].write) and - (not Blackboard.metadata[remapped_key].exclusive) - ): - del Blackboard.metadata[remapped_key] - if clear: - try: - del Blackboard.storage[remapped_key] - except KeyError: - pass # perfectly legitimate for a registered key to not exist on the blackboard - del super().__getattribute__("remappings")[key] - if update_namespace_cache: - self._update_namespaces() - - -class IntermediateVariableFetcher(object): - def __init__(self, blackboard, namespace): - super().__setattr__("blackboard", blackboard) - super().__setattr__("namespace", namespace) - - def __getattr__(self, name: str): - # print("Fetcher:__getattr__ [{}]".format(name)) - name = Blackboard.absolute_name(self.namespace, name) - return self.blackboard.get(name) - - def __setattr__(self, name: str, value: typing.Any): - # print("Fetcher:__setattr__ [{}][{}]".format(name, value)) - name = Blackboard.absolute_name(self.namespace, name) - return self.blackboard.set(name, value) diff --git a/kios_bt_planning/backups/dynamic_bt/py_trees/common.py b/kios_bt_planning/backups/dynamic_bt/py_trees/common.py deleted file mode 100644 index 645e57e3..00000000 --- a/kios_bt_planning/backups/dynamic_bt/py_trees/common.py +++ /dev/null @@ -1,251 +0,0 @@ -#!/usr/bin/env python -# -# License: BSD -# https://raw.githubusercontent.com/splintered-reality/py_trees/devel/LICENSE -# -############################################################################## -# Documentation -############################################################################## - -""" -Common definitions, methods and variables used by the py_trees library. -""" - -############################################################################## -# Imports -############################################################################## - -import enum -import math -import typing - -############################################################################## -# General -############################################################################## - - -class Name(enum.Enum): - """ - Naming conventions. - """ - AUTO_GENERATED = "AUTO_GENERATED" - """:py:data:`~py_trees.common.Name.AUTO_GENERATED` leaves it to the behaviour to generate a useful, informative name.""" - - -class Status(enum.Enum): - """An enumerator representing the status of a behaviour """ - - SUCCESS = "SUCCESS" - """Behaviour check has passed, or execution of its action has finished with a successful result.""" - FAILURE = "FAILURE" - """Behaviour check has failed, or execution of its action finished with a failed result.""" - RUNNING = "RUNNING" - """Behaviour is in the middle of executing some action, result still pending.""" - INVALID = "INVALID" - """Behaviour is uninitialised and inactive, i.e. this is the status before first entry, and after a higher priority switch has occurred.""" - - -class Duration(enum.Enum): - """ - Naming conventions. - """ - INFINITE = math.inf - """:py:data:`~py_trees.common.Duration.INFINITE` oft used for perpetually blocking operations.""" - UNTIL_THE_BATTLE_OF_ALFREDO = math.inf - """:py:data:`~py_trees.common.Duration.UNTIL_THE_BATTLE_OF_ALFREDO` is an alias for :py:data:`~py_trees.common.Duration.INFINITE`.""" - - -class Access(enum.Enum): - """ - Use to distinguish types of access to, e.g. variables on a blackboard. - """ - - READ = "READ" - """Read access.""" - WRITE = "WRITE" - """Write access, implicitly also grants read access.""" - EXCLUSIVE_WRITE = "EXCLUSIVE_WRITE" - """Exclusive lock for writing, i.e. no other writer permitted.""" - - -############################################################################## -# Policies -############################################################################## - -class ParallelPolicy(object): - """ - Configurable policies for :py:class:`~py_trees.composites.Parallel` behaviours. - """ - class Base(object): - """ - Base class for parallel policies. Should never be used directly. - """ - def __init__(self, synchronise=False): - """ - Default policy configuration. - - Args: - synchronise (:obj:`bool`): stop ticking of children with status :py:data:`~py_trees.common.Status.SUCCESS` until the policy criteria is met - """ - self.synchronise = synchronise - - class SuccessOnAll(Base): - """ - Return :py:data:`~py_trees.common.Status.SUCCESS` only when each and every child returns - :py:data:`~py_trees.common.Status.SUCCESS`. If synchronisation is requested, any children that - tick with :data:`~py_trees.common.Status.SUCCESS` will be skipped on subsequent ticks until - the policy criteria is met, or one of the children returns status :data:`~py_trees.common.Status.FAILURE`. - """ - def __init__(self, synchronise=True): - """ - Policy configuration. - - Args: - synchronise (:obj:`bool`): stop ticking of children with status :py:data:`~py_trees.common.Status.SUCCESS` until the policy criteria is met - """ - super().__init__(synchronise=synchronise) - - class SuccessOnOne(Base): - """ - Return :py:data:`~py_trees.common.Status.SUCCESS` so long as at least one child has :py:data:`~py_trees.common.Status.SUCCESS` - and the remainder are :py:data:`~py_trees.common.Status.RUNNING` - """ - def __init__(self): - """ - No configuration necessary for this policy. - """ - super().__init__(synchronise=False) - - class SuccessOnSelected(Base): - """ - Return :py:data:`~py_trees.common.Status.SUCCESS` so long as each child in a specified list returns - :py:data:`~py_trees.common.Status.SUCCESS`. If synchronisation is requested, any children that - tick with :data:`~py_trees.common.Status.SUCCESS` will be skipped on subsequent ticks until - the policy criteria is met, or one of the children returns status :data:`~py_trees.common.Status.FAILURE`. - """ - def __init__(self, children, synchronise=True): - """ - Policy configuraiton. - - Args: - children ([:class:`~py_trees.behaviour.Behaviour`]): list of children to succeed on - synchronise (:obj:`bool`): stop ticking of children with status :py:data:`~py_trees.common.Status.SUCCESS` until the policy criteria is met - """ - super().__init__(synchronise=synchronise) - self.children = children - - -class OneShotPolicy(enum.Enum): - """Policy rules for :py:class:`~py_trees.decorators.OneShot` (decorator) or :py:meth:`~py_trees.idioms.oneshot (idiom) oneshots.""" - - ON_COMPLETION = [Status.SUCCESS, Status.FAILURE] - """Return :py:data:`~py_trees.common.Status.SUCCESS` after the specified child/subtree reaches completion (:py:data:`~py_trees.common.Status.SUCCESS` || :py:data:`~py_trees.common.Status.FAILURE`).""" - ON_SUCCESSFUL_COMPLETION = [Status.SUCCESS] - """Permits the oneshot to keep trying until it's first success.""" - - -class ClearingPolicy(enum.IntEnum): - """ - Policy rules for behaviours to dictate when data should be cleared/reset. - """ - ON_INITIALISE = 1 - """Clear when entering the :py:meth:`~py_trees.behaviour.Behaviour.initialise` method.""" - ON_SUCCESS = 2 - """Clear when returning :py:data:`~py_trees.common.Status.SUCCESS`.""" - NEVER = 3 - """Never clear the data""" - -############################################################################## -# Blackboards -############################################################################## - - -class ComparisonExpression(object): - """ - Store the parameters for a univariate comparison operation - (i.e. between a variable and a value). - - Args: - variable: name of the variable to compare - value: value to compare against - operator: a callable comparison operator - - .. tip:: - The python `operator module`_ includes many useful comparison operations, e.g. operator.ne - """ - def __init__( - self, - variable: str, - value: typing.Any, - operator: typing.Callable[[typing.Any, typing.Any], bool] - ): - self.variable = variable - self.value = value - self.operator = operator - - -############################################################################## -# BlackBoxes -############################################################################## - -class BlackBoxLevel(enum.IntEnum): - """ - Whether a behaviour is a blackbox entity that may be considered collapsible - (i.e. everything in its subtree will not be visualised) by - visualisation tools. - - Blackbox levels are increasingly persistent in visualisations. - - Visualisations by default, should always collapse blackboxes that represent - `DETAIL`. - """ - DETAIL = 1 - """A blackbox that encapsulates detailed activity.""" - COMPONENT = 2 - """A blackbox that encapsulates a subgroup of functionalities as a single group.""" - BIG_PICTURE = 3 - """A blackbox that represents a big picture part of the entire tree view.""" - NOT_A_BLACKBOX = 4 - """Not a blackbox, do not ever collapse.""" - - -class VisibilityLevel(enum.IntEnum): - """ - Closely associated with the :py:class:`~py_trees.common.BlackBoxLevel` for a - behaviour. This sets the visibility level to be used for visualisations. - - Visibility levels correspond to reducing levels of visibility in a visualisation. - """ - ALL = 0 - """Do not collapse any behaviour.""" - DETAIL = BlackBoxLevel.DETAIL - """Collapse blackboxes marked with :py:data:`~py_trees.common.BlackBoxLevel.DETAIL` or lower.""" - COMPONENT = BlackBoxLevel.COMPONENT - """Collapse blackboxes marked with :py:data:`~py_trees.common.BlackBoxLevel.COMPONENT` or lower.""" - BIG_PICTURE = BlackBoxLevel.BIG_PICTURE - """Collapse any blackbox that isn't marked with :py:data:`~py_trees.common.BlackBoxLevel.BIG_PICTURE`.""" - - -visibility_level_strings = ["all", "detail", "component", "big_picture"] -"""Convenient string representations to use for command line input (amongst other things).""" - - -def string_to_visibility_level(level): - """ - Will convert a string to a visibility level. Note that it will quietly return ALL if - the string is not matched to any visibility level string identifier. - - Args: - level (str): visibility level as a string - - Returns: - :class:`~py_trees.common.VisibilityLevel`: visibility level enum - """ - if level == "detail": - return VisibilityLevel.DETAIL - elif level == "component": - return VisibilityLevel.COMPONENT - elif level == "big_picture": - return VisibilityLevel.BIG_PICTURE - else: - return VisibilityLevel.ALL diff --git a/kios_bt_planning/backups/dynamic_bt/py_trees/composites.py b/kios_bt_planning/backups/dynamic_bt/py_trees/composites.py deleted file mode 100644 index 106f2eb9..00000000 --- a/kios_bt_planning/backups/dynamic_bt/py_trees/composites.py +++ /dev/null @@ -1,680 +0,0 @@ -#!/usr/bin/env python -# -# License: BSD -# https://raw.githubusercontent.com/splintered-reality/py_trees/devel/LICENSE -# -############################################################################## -# Documentation -############################################################################## - -""" -Composites are responsible for directing the path traced through -the tree on a given tick (execution). They are the **factories** -(Sequences and Parallels) and **decision makers** (Selectors) of a behaviour -tree. - -.. graphviz:: dot/composites.dot - :align: center - :caption: PyTree Composites - -Composite behaviours typically manage children and apply some logic to the way -they execute and return a result, but generally don't do anything themselves. -Perform the checks or actions you need to do in the non-composite behaviours. - -Most any desired functionality can be authored with a combination of these -three composites. In fact, it is precisely this feature that makes behaviour -trees attractive - it breaks down complex decision making logic to just three -primitive elements. It is possible and often desirable to extend this set with -custom composites of your own, but think carefully before you do - in almost -every case, a combination of the existing composites will serve and as a -result, you will merely compound the complexity inherent in your tree logic. -This this makes it confoundingly difficult to design, introspect and debug. As -an example, design sessions often revolve around a sketched graph on a -whiteboard. When these graphs are composed of just five elements (Selectors, -Sequences, Parallels, Decorators and Behaviours), it is very easy to understand -the logic at a glance. Double the number of fundamental elements and you may as -well be back at the terminal parsing code. - -.. tip:: You should never need to subclass or create new composites. - -The basic operational modes of the three composites in this library are as follows: - -* :class:`~py_trees.composites.Selector`: select a child to execute based on cascading priorities -* :class:`~py_trees.composites.Sequence`: execute children sequentially -* :class:`~py_trees.composites.Parallel`: execute children concurrently - -This library does provide some flexibility in *how* each composite is implemented without -breaking the fundamental nature of each (as described above). Selectors and Sequences can -be configured with or without memory (resumes or resets if children are RUNNING) and -the results of a parallel can be configured to wait upon all children completing, succeed -on one, all or a subset thereof. - -.. tip:: Follow the links in each composite's documentation to the relevant demo programs. - -""" - -############################################################################## -# Imports -############################################################################## - -import itertools -import typing - -from py_trees import behaviour -from py_trees import common - -############################################################################## -# Composites -############################################################################## - - -class Composite(behaviour.Behaviour): - """ - The parent class to all composite behaviours, i.e. those that - have children. - - Args: - name (:obj:`str`): the composite behaviour name - children ([:class:`~py_trees.behaviour.Behaviour`]): list of children to add - """ - def __init__(self, - name: typing.Union[str, common.Name]=common.Name.AUTO_GENERATED, - children: typing.List[behaviour.Behaviour]=None - ): - super(Composite, self).__init__(name) - if children is not None: - for child in children: - self.add_child(child) - else: - self.children = [] - self.current_child = None - - ############################################ - # Worker Overrides - ############################################ - - def stop(self, new_status=common.Status.INVALID): - """ - There is generally two use cases that must be supported here. - - 1) Whenever the composite has gone to a recognised state (i.e. :data:`~py_trees.common.Status.FAILURE` or SUCCESS), - or 2) when a higher level parent calls on it to truly stop (INVALID). - - In only the latter case will children need to be forcibly stopped as well. In the first case, they will - have stopped themselves appropriately already. - - Args: - new_status (:class:`~py_trees.common.Status`): behaviour will transition to this new status - """ - if self.logger.level < 1: - self.logger.debug("%s.stop()[%s]" % (self.__class__.__name__, "%s->%s" % (self.status, new_status) if self.status != new_status else "%s" % new_status)) - # priority interrupted - if new_status == common.Status.INVALID: - self.current_child = None - for child in self.children: - child.stop(new_status) - # This part just replicates the Behaviour.stop function. We replicate it here so that - # the Behaviour logging doesn't duplicate the composite logging here, just a bit cleaner this way. - self.terminate(new_status) - self.status = new_status - self.iterator = self.tick() - - def tip(self): - """ - Recursive function to extract the last running node of the tree. - - Returns: - :class::`~py_trees.behaviour.Behaviour`: the tip function of the current child of this composite or None - """ - if self.current_child is not None: - return self.current_child.tip() - else: - return super().tip() - - ############################################ - # Children - ############################################ - - def add_child(self, child): - """ - Adds a child. - - Args: - child (:class:`~py_trees.behaviour.Behaviour`): child to add - - Raises: - TypeError: if the child is not an instance of :class:`~py_trees.behaviour.Behaviour` - RuntimeError: if the child already has a parent - - Returns: - uuid.UUID: unique id of the child - """ - if not isinstance(child, behaviour.Behaviour): - raise TypeError("children must be behaviours, but you passed in {}".format(type(child))) - self.children.append(child) - if child.parent is not None: - raise RuntimeError("behaviour '{}' already has parent '{}'".format(child.name, child.parent.name)) - child.parent = self - return child.id - - def add_children(self, children): - """ - Append a list of children to the current list. - - Args: - children ([:class:`~py_trees.behaviour.Behaviour`]): list of children to add - """ - for child in children: - self.add_child(child) - return self - - def remove_child(self, child): - """ - Remove the child behaviour from this composite. - - Args: - child (:class:`~py_trees.behaviour.Behaviour`): child to delete - - Returns: - :obj:`int`: index of the child that was removed - - .. todo:: Error handling for when child is not in this list - """ - if self.current_child is not None and (self.current_child.id == child.id): - self.current_child = None - if child.status == common.Status.RUNNING: - child.stop(common.Status.INVALID) - child_index = self.children.index(child) - self.children.remove(child) - child.parent = None - return child_index - - def remove_all_children(self): - """ - Remove all children. Makes sure to stop each child if necessary. - """ - self.current_child = None - for child in self.children: - if child.status == common.Status.RUNNING: - child.stop(common.Status.INVALID) - child.parent = None - # makes sure to delete it for this class and all references to it - # http://stackoverflow.com/questions/850795/clearing-python-lists - del self.children[:] - - def replace_child(self, child, replacement): - """ - Replace the child behaviour with another. - - Args: - child (:class:`~py_trees.behaviour.Behaviour`): child to delete - replacement (:class:`~py_trees.behaviour.Behaviour`): child to insert - """ - if self.logger.level < 1: - self.logger.debug("%s.replace_child()[%s->%s]" % (self.__class__.__name__, child.name, replacement.name)) - child_index = self.children.index(child) - self.remove_child(child) - self.insert_child(replacement, child_index) - child.parent = None - - def remove_child_by_id(self, child_id): - """ - Remove the child with the specified id. - - Args: - child_id (uuid.UUID): unique id of the child - - Raises: - IndexError: if the child was not found - """ - child = next((c for c in self.children if c.id == child_id), None) - if child is not None: - self.remove_child(child) - else: - raise IndexError('child was not found with the specified id [%s]' % child_id) - - def prepend_child(self, child): - """ - Prepend the child before all other children. - - Args: - child (:class:`~py_trees.behaviour.Behaviour`): child to insert - - Returns: - uuid.UUID: unique id of the child - """ - self.children.insert(0, child) - child.parent = self - return child.id - - def insert_child(self, child, index): - """ - Insert child at the specified index. This simply directly calls - the python list's :obj:`insert` method using the child and index arguments. - - Args: - child (:class:`~py_trees.behaviour.Behaviour`): child to insert - index (:obj:`int`): index to insert it at - - Returns: - uuid.UUID: unique id of the child - """ - self.children.insert(index, child) - child.parent = self - return child.id - -############################################################################## -# Selector -############################################################################## - - -class Selector(Composite): - """ - Selectors are the decision makers. - - .. graphviz:: dot/selector.dot - - A selector executes each of its child behaviours in turn until one of them - succeeds (at which point it itself returns :data:`~py_trees.common.Status.RUNNING` or :data:`~py_trees.common.Status.SUCCESS`, - or it runs out of children at which point it itself returns :data:`~py_trees.common.Status.FAILURE`. - We usually refer to selecting children as a means of *choosing between priorities*. - Each child and its subtree represent a decreasingly lower priority path. - - .. note:: - - Switching from a low -> high priority branch causes a `stop(INVALID)` signal to be sent to the previously - executing low priority branch. This signal will percolate down that child's own subtree. Behaviours - should make sure that they catch this and *destruct* appropriately. - - .. seealso:: The :ref:`py-trees-demo-selector-program` program demos higher priority switching under a selector. - - Args: - name (:obj:`str`): the composite behaviour name - memory (:obj:`bool`): if :data:`~py_trees.common.Status.RUNNING` on the previous tick, resume with the :data:`~py_trees.common.Status.RUNNING` child - children ([:class:`~py_trees.behaviour.Behaviour`]): list of children to add - """ - - def __init__(self, name="Selector", memory=False, children=None): - super(Selector, self).__init__(name, children) - self.memory = memory - - def tick(self): - """ - Run the tick behaviour for this selector. Note that the status - of the tick is always determined by its children, not - by the user customised update function. - - Yields: - :class:`~py_trees.behaviour.Behaviour`: a reference to itself or one of its children - """ - if self.logger.level < 1: - self.logger.debug("%s.tick()" % self.__class__.__name__) - # initialise - if self.status != common.Status.RUNNING: - # selector specific initialisation - leave initialise() free for users to - # re-implement without having to make calls to super() - if self.logger.level < 1: - self.logger.debug("%s.tick() [!RUNNING->reset current_child]" % self.__class__.__name__) - self.current_child = self.children[0] if self.children else None - - # reset the children - don't need to worry since they will be handled - # a) prior to a remembered starting point, or - # b) invalidated by a higher level priority - - # user specific initialisation - self.initialise() - - # customised work - self.update() - - # nothing to do - if not self.children: - self.current_child = None - self.stop(common.Status.FAILURE) - yield self - return - - # starting point - if self.memory: - index = self.children.index(self.current_child) - # clear out preceding status' - not actually necessary but helps - # visualise the case of memory vs no memory - for child in itertools.islice(self.children, None, index): - child.stop(common.Status.INVALID) - else: - index = 0 - - # actual work - previous = self.current_child - for child in itertools.islice(self.children, index, None): - for node in child.tick(): - yield node - if node is child: - if node.status == common.Status.RUNNING or node.status == common.Status.SUCCESS: - self.current_child = child - self.status = node.status - if previous is None or previous != self.current_child: - # we interrupted, invalidate everything at a lower priority - passed = False - for child in self.children: - if passed: - if child.status != common.Status.INVALID: - child.stop(common.Status.INVALID) - passed = True if child == self.current_child else passed - yield self - return - # all children failed, set failure ourselves and current child to the last bugger who failed us - self.status = common.Status.FAILURE - try: - self.current_child = self.children[-1] - except IndexError: - self.current_child = None - yield self - - def stop(self, new_status=common.Status.INVALID): - """ - Stopping a selector requires setting the current child to none. Note that it - is important to implement this here instead of terminate, so users are free - to subclass this easily with their own terminate and not have to remember - that they need to call this function manually. - - Args: - new_status (:class:`~py_trees.common.Status`): the composite is transitioning to this new status - """ - # retain information about the last running child if the new status is - # SUCCESS or FAILURE - if new_status == common.Status.INVALID: - self.current_child = None - Composite.stop(self, new_status) - -############################################################################## -# Sequence -############################################################################## - - -class Sequence(Composite): - """ - Sequences are the factory lines of Behaviour Trees - - .. graphviz:: dot/sequence.dot - - A sequence will progressively tick over each of its children so long as - each child returns :data:`~py_trees.common.Status.SUCCESS`. If any child returns - :data:`~py_trees.common.Status.FAILURE` or :data:`~py_trees.common.Status.RUNNING` the sequence will halt and the parent will adopt - the result of this child. If it reaches the last child, it returns with - that result regardless. - - .. note:: - - The sequence halts once it sees a child is RUNNING and then returns - the result. *It does not get stuck in the running behaviour*. - - .. seealso:: The :ref:`py-trees-demo-sequence-program` program demos a simple sequence in action. - - Args: - name: the composite behaviour name - memory: if :data:`~py_trees.common.Status.RUNNING` on the previous tick, resume with the :data:`~py_trees.common.Status.RUNNING` child - children: list of children to add - - """ - def __init__( - self, - name: str="Sequence", - memory: bool=True, - children: typing.List[behaviour.Behaviour]=None - ): - super(Sequence, self).__init__(name, children) - self.memory = memory - - def tick(self): - """ - Tick over the children. - - Yields: - :class:`~py_trees.behaviour.Behaviour`: a reference to itself or one of its children - """ - if self.logger.level < 1: - self.logger.debug("%s.tick()" % self.__class__.__name__) - - # initialise - index = 0 - if self.status != common.Status.RUNNING or not self.memory: - self.current_child = self.children[0] if self.children else None - for child in self.children: - if child.status != common.Status.INVALID: - child.stop(common.Status.INVALID) - # user specific initialisation - self.initialise() - else: # self.memory is True and status is RUNNING - index = self.children.index(self.current_child) - - # customised work - self.update() - - # nothing to do - if not self.children: - self.current_child = None - self.stop(common.Status.SUCCESS) - yield self - return - - # actual work - for child in itertools.islice(self.children, index, None): - for node in child.tick(): - yield node - if node is child and node.status != common.Status.SUCCESS: - self.status = node.status - yield self - return - try: - # advance if there is 'next' sibling - self.current_child = self.children[index + 1] - index += 1 - except IndexError: - pass - - self.stop(common.Status.SUCCESS) - yield self - - -############################################################################## -# Parallel -############################################################################## - - -class Parallel(Composite): - """ - Parallels enable a kind of concurrency - - .. graphviz:: dot/parallel.dot - - Ticks every child every time the parallel is run (a poor man's form of parallelism). - - * Parallels will return :data:`~py_trees.common.Status.FAILURE` if any - child returns :py:data:`~py_trees.common.Status.FAILURE` - * Parallels with policy :class:`~py_trees.common.ParallelPolicy.SuccessOnAll` - only returns :py:data:`~py_trees.common.Status.SUCCESS` if **all** children - return :py:data:`~py_trees.common.Status.SUCCESS` - * Parallels with policy :class:`~py_trees.common.ParallelPolicy.SuccessOnOne` - return :py:data:`~py_trees.common.Status.SUCCESS` if **at least one** child - returns :py:data:`~py_trees.common.Status.SUCCESS` and others are - :py:data:`~py_trees.common.Status.RUNNING` - * Parallels with policy :class:`~py_trees.common.ParallelPolicy.SuccessOnSelected` - only returns :py:data:`~py_trees.common.Status.SUCCESS` if a **specified subset** - of children return :py:data:`~py_trees.common.Status.SUCCESS` - - Policies :class:`~py_trees.common.ParallelPolicy.SuccessOnAll` and - :class:`~py_trees.common.ParallelPolicy.SuccessOnSelected` may be configured to be - *synchronised* in which case children that tick with - :data:`~py_trees.common.Status.SUCCESS` will be skipped on subsequent ticks until - the policy criteria is met, or one of the children returns - status :data:`~py_trees.common.Status.FAILURE`. - - Parallels with policy :class:`~py_trees.common.ParallelPolicy.SuccessOnSelected` will - check in both the :meth:`~py_trees.behaviour.Behaviour.setup` and - :meth:`~py_trees.behaviour.Behaviour.tick` methods to to verify the - selected set of children is actually a subset of the children of this parallel. - - .. seealso:: - * :ref:`Context Switching Demo ` - """ - def __init__(self, - name: typing.Union[str, common.Name]=common.Name.AUTO_GENERATED, - policy: common.ParallelPolicy.Base=common.ParallelPolicy.SuccessOnAll(), - children: typing.List[behaviour.Behaviour]=None - ): - """ - Args: - name (:obj:`str`): the composite behaviour name - policy (:class:`~py_trees.common.ParallelPolicy`): policy to use for deciding success or otherwise - children ([:class:`~py_trees.behaviour.Behaviour`]): list of children to add - """ - super(Parallel, self).__init__(name, children) - self.policy = policy - - def setup(self, **kwargs): - """ - Detect before ticking whether the policy configuration is invalid. - - Args: - **kwargs (:obj:`dict`): distribute arguments to this - behaviour and in turn, all of it's children - - Raises: - RuntimeError: if the parallel's policy configuration is invalid - Exception: be ready to catch if any of the children raise an exception - """ - if self.logger.level < 1: - self.logger.debug("%s.setup()" % (self.__class__.__name__)) - self.validate_policy_configuration() - - def tick(self): - """ - Tick over the children. - - Yields: - :class:`~py_trees.behaviour.Behaviour`: a reference to itself or one of its children - - Raises: - RuntimeError: if the policy configuration was invalid - """ - if self.logger.level < 1: - self.logger.debug("%s.tick()" % self.__class__.__name__) - self.validate_policy_configuration() - - # reset - if self.status != common.Status.RUNNING: - if self.logger.level < 1: - self.logger.debug("%s.tick(): re-initialising" % self.__class__.__name__) - for child in self.children: - # reset the children, this ensures old SUCCESS/FAILURE status flags - # don't break the synchronisation logic below - if child.status != common.Status.INVALID: - child.stop(common.Status.INVALID) - self.current_child = None - # subclass (user) handling - self.initialise() - - # nothing to do - if not self.children: - self.current_child = None - self.stop(common.Status.SUCCESS) - yield self - return - - # process them all first - for child in self.children: - if self.policy.synchronise and child.status == common.Status.SUCCESS: - continue - for node in child.tick(): - yield node - - # determine new status - new_status = common.Status.RUNNING - self.current_child = self.children[-1] - try: - failed_child = next(child for child in self.children if child.status == common.Status.FAILURE) - self.current_child = failed_child - new_status = common.Status.FAILURE - except StopIteration: - if type(self.policy) is common.ParallelPolicy.SuccessOnAll: - if all([c.status == common.Status.SUCCESS for c in self.children]): - new_status = common.Status.SUCCESS - self.current_child = self.children[-1] - elif type(self.policy) is common.ParallelPolicy.SuccessOnOne: - successful = [child for child in self.children if child.status == common.Status.SUCCESS] - if successful: - new_status = common.Status.SUCCESS - self.current_child = successful[-1] - elif type(self.policy) is common.ParallelPolicy.SuccessOnSelected: - if all([c.status == common.Status.SUCCESS for c in self.policy.children]): - new_status = common.Status.SUCCESS - self.current_child = self.policy.children[-1] - else: - raise RuntimeError("this parallel has been configured with an unrecognised policy [{}]".format(type(self.policy))) - # this parallel may have children that are still running - # so if the parallel itself has reached a final status, then - # these running children need to be terminated so they don't dangle - if new_status != common.Status.RUNNING: - self.stop(new_status) - self.status = new_status - yield self - - def stop(self, new_status: common.Status=common.Status.INVALID): - """ - For interrupts or any of the termination conditions, ensure that any - running children are stopped. - - Args: - new_status : the composite is transitioning to this new status - """ - # clean up dangling (running) children - for child in self.children: - if child.status == common.Status.RUNNING: - # interrupt it exactly as if it was interrupted by a higher priority - child.stop(common.Status.INVALID) - # only nec. thing here is to make sure the status gets set to INVALID if - # it was a higher priority interrupt (new_status == INVALID) - Composite.stop(self, new_status) - - def validate_policy_configuration(self): - """ - Policy configuration can be invalid if: - - * Policy is SuccessOnSelected and no behaviours have been specified - * Policy is SuccessOnSelected and behaviours that are not children exist - - Raises: - RuntimeError: if policy configuration was invalid - """ - if type(self.policy) is common.ParallelPolicy.SuccessOnSelected: - if not self.policy.children: - error_message = ("policy SuccessOnSelected requires a non-empty " - "selection of children [{}]".format(self.name)) - self.logger.error(error_message) - raise RuntimeError(error_message) - missing_children_names = [child.name for child in self.policy.children if child not in self.children] - - if missing_children_names: - error_message = ("policy SuccessOnSelected has selected behaviours that are " - "not children of this parallel {}[{}]""".format(missing_children_names, self.name)) - self.logger.error(error_message) - raise RuntimeError(error_message) - - -############################################################################## -# Shrinked Tree -############################################################################## - - -class ShrinkedTree(Composite): - """ - This is not a control flow node, but just a placeholder representing a Behavior Tree. - - It can be used for visualization only when one wants to shrink an entire BT to be represented - as subtree of another larger BT. Because of that, this is just a white box with a name. - - """ - - def __init__(self, name="ShrinkedTree"): - super(ShrinkedTree, self).__init__(name) diff --git a/kios_bt_planning/backups/dynamic_bt/py_trees/console.py b/kios_bt_planning/backups/dynamic_bt/py_trees/console.py deleted file mode 100644 index 15fc0e6f..00000000 --- a/kios_bt_planning/backups/dynamic_bt/py_trees/console.py +++ /dev/null @@ -1,340 +0,0 @@ -# -# License: BSD -# https://raw.githubusercontent.com/splintered-reality/py_trees/devel/LICENSE -# - -############################################################################## -# Description -############################################################################## - -""" -Simple colour definitions and syntax highlighting for the console. - ----- - -**Colour Definitions** - -The current list of colour definitions include: - - * ``Regular``: black, red, green, yellow, blue, magenta, cyan, white, - * ``Bold``: bold, bold_black, bold_red, bold_green, bold_yellow, bold_blue, bold_magenta, bold_cyan, bold_white - -These colour definitions can be used in the following way: - -.. code-block:: python - - import py_trees.console as console - print(console.cyan + " Name" + console.reset + ": " + console.yellow + "Dude" + console.reset) - -""" - -############################################################################## -# Imports -############################################################################## - -import os -import sys - - -############################################################################## -# Special Characters -############################################################################## - - -def has_unicode(encoding: str=sys.stdout.encoding) -> bool: - """ - Define whether the specified encoding has unicode symbols. Usually used to check - if the stdout is capable or otherwise (e.g. Jenkins CI can often be configured - with unicode disabled). - - Args: - encoding (:obj:`str`, optional): the encoding to check against. - - Returns: - :obj:`bool`: true if capable, false otherwise - """ - try: - u'\u26A1'.encode(encoding) - except TypeError: - # if sys.stdout.encoding is not available, it is None - # this will occur if you run nosetests3 without -s - return False - except UnicodeError: - return False - return True - - -def define_symbol_or_fallback(original: str, fallback: str, encoding: str=sys.stdout.encoding): - """ - Return the correct encoding according to the specified encoding. Used to - make sure we get an appropriate symbol, even if the shell is merely ascii as - is often the case on, e.g. Jenkins CI. - - Args: - original (:obj:`str`): the unicode string (usually just a character) - fallback (:obj:`str`): the fallback ascii string - encoding (:obj:`str`, optional): the encoding to check against. - - Returns: - :obj:`str`: either original or fallback depending on whether exceptions were thrown. - """ - try: - original.encode(encoding) - except UnicodeError: - return fallback - return original - - -circle = u'\u26ac' -lightning_bolt = u'\u26A1' -double_vertical_line = u'\u2016' -check_mark = u'\u2713' -multiplication_x = u'\u2715' -left_arrow = u'\u2190' # u'\u2190' -right_arrow = u'\u2192' -left_right_arrow = u'\u2194' -forbidden_circle = u'\u29B8' -circled_m = u'\u24c2' - -############################################################################## -# Keypress -############################################################################## - - -def read_single_keypress(): - """Waits for a single keypress on stdin. - - This is a silly function to call if you need to do it a lot because it has - to store stdin's current setup, setup stdin for reading single keystrokes - then read the single keystroke then revert stdin back after reading the - keystroke. - - Returns: - :obj:`int`: the character of the key that was pressed - - Raises: - KeyboardInterrupt: if CTRL-C was pressed (keycode 0x03) - """ - def read_single_keypress_unix(): - """For Unix case, where fcntl, termios is available.""" - import fcntl - import termios - fd = sys.stdin.fileno() - # save old state - flags_save = fcntl.fcntl(fd, fcntl.F_GETFL) - attrs_save = termios.tcgetattr(fd) - # make raw - the way to do this comes from the termios(3) man page. - attrs = list(attrs_save) # copy the stored version to update - # iflag - attrs[0] &= ~(termios.IGNBRK | termios.BRKINT | termios.PARMRK | - termios.ISTRIP | termios.INLCR | termios. IGNCR | - termios.ICRNL | termios.IXON) - # oflag - attrs[1] &= ~termios.OPOST - # cflag - attrs[2] &= ~(termios.CSIZE | termios. PARENB) - attrs[2] |= termios.CS8 - # lflag - attrs[3] &= ~(termios.ECHONL | termios.ECHO | termios.ICANON | - termios.ISIG | termios.IEXTEN) - termios.tcsetattr(fd, termios.TCSANOW, attrs) - # turn off non-blocking - fcntl.fcntl(fd, fcntl.F_SETFL, flags_save & ~os.O_NONBLOCK) - # read a single keystroke - ret = sys.stdin.read(1) # returns a single character - if ord(ret) == 3: # CTRL-C - termios.tcsetattr(fd, termios.TCSAFLUSH, attrs_save) - fcntl.fcntl(fd, fcntl.F_SETFL, flags_save) - raise KeyboardInterrupt("Ctrl-c") - # restore old state - termios.tcsetattr(fd, termios.TCSAFLUSH, attrs_save) - fcntl.fcntl(fd, fcntl.F_SETFL, flags_save) - return ret - - def read_single_keypress_windows(): - """Windows case, can't use fcntl and termios. - Not same implementation as for Unix, requires a newline to continue. - """ - import msvcrt # noqa - # read a single keystroke - ret = sys.stdin.read(1) - if ord(ret) == 3: # CTRL-C - raise KeyboardInterrupt("Ctrl-c") - return ret - try: - return read_single_keypress_unix() - except ImportError as e_unix: - try: - return read_single_keypress_windows() - except ImportError as e_windows: - raise ImportError("Neither unix nor windows implementations supported [{}][{}]".format(str(e_unix), str(e_windows))) - -############################################################################## -# Methods -############################################################################## - - -def console_has_colours(): - """ - Detects if the console (stdout) has colourising capability. - """ - if os.environ.get("PY_TREES_DISABLE_COLORS"): - return False - # From django.core.management.color.supports_color - # https://github.com/django/django/blob/master/django/core/management/color.py - plat = sys.platform - supported_platform = plat != 'Pocket PC' and (plat != 'win32' or - 'ANSICON' in os.environ) - # isatty is not always implemented, #6223. - is_a_tty = hasattr(sys.stdout, 'isatty') and sys.stdout.isatty() - if not supported_platform or not is_a_tty: - return False - return True - - -has_colours = console_has_colours() -""" Whether the loading program has access to colours or not.""" - - -if has_colours: - # reset = "\x1b[0;0m" - reset = "\x1b[0m" - bold = "\x1b[%sm" % '1' - dim = "\x1b[%sm" % '2' - underlined = "\x1b[%sm" % '4' - blink = "\x1b[%sm" % '5' - black, red, green, yellow, blue, magenta, cyan, white = ["\x1b[%sm" % str(i) for i in range(30, 38)] - bold_black, bold_red, bold_green, bold_yellow, bold_blue, bold_magenta, bold_cyan, bold_white = ["\x1b[%sm" % ('1;' + str(i)) for i in range(30, 38)] -else: - reset = "" - bold = "" - dim = "" - underlined = "" - blink = "" - black, red, green, yellow, blue, magenta, cyan, white = ["" for i in range(30, 38)] - bold_black, bold_red, bold_green, bold_yellow, bold_blue, bold_magenta, bold_cyan, bold_white = ["" for i in range(30, 38)] - -colours = [bold, dim, underlined, blink, - black, red, green, yellow, blue, magenta, cyan, white, - bold_black, bold_red, bold_green, bold_yellow, bold_blue, bold_magenta, bold_cyan, bold_white - ] -"""List of all available colours.""" - - -def pretty_print(msg, colour=white): - if has_colours: - seq = colour + msg + reset - sys.stdout.write(seq) - else: - sys.stdout.write(msg) - - -def pretty_println(msg, colour=white): - if has_colours: - seq = colour + msg + reset - sys.stdout.write(seq) - sys.stdout.write("\n") - else: - sys.stdout.write(msg) - - -############################################################################## -# Console -############################################################################## - - -def banner(msg): - print(green + "\n" + 80 * "*" + reset) - print(green + "* " + bold_white + msg.center(80) + reset) - print(green + 80 * "*" + "\n" + reset) - - -def debug(msg): - print(green + msg + reset) - - -def warning(msg): - print(yellow + msg + reset) - - -def info(msg): - print(msg) - - -def error(msg): - print(red + msg + reset) - - -def logdebug(message): - ''' - Prefixes ``[DEBUG]`` and colours the message green. - - Args: - message (:obj:`str`): message to log. - ''' - print(green + "[DEBUG] " + message + reset) - - -def loginfo(message): - ''' - Prefixes ``[ INFO]`` to the message. - - Args: - message (:obj:`str`): message to log. - ''' - print("[ INFO] " + message) - - -def logwarn(message): - ''' - Prefixes ``[ WARN]`` and colours the message yellow. - - Args: - message (:obj:`str`): message to log. - ''' - print(yellow + "[ WARN] " + message + reset) - - -def logerror(message): - ''' - Prefixes ``[ERROR]`` and colours the message red. - - Args: - message (:obj:`str`): message to log. - ''' - print(red + "[ERROR] " + message + reset) - - -def logfatal(message): - ''' - Prefixes ``[FATAL]`` and colours the message bold red. - - Args: - message (:obj:`str`): message to log. - ''' - print(bold_red + "[FATAL] " + message + reset) - - -############################################################################## -# Main -############################################################################## - -if __name__ == '__main__': - for colour in colours: - pretty_print("dude\n", colour) - logdebug("loginfo message") - logwarn("logwarn message") - logerror("logerror message") - logfatal("logfatal message") - pretty_print("red\n", red) - print("some normal text") - print(cyan + " Name" + reset + ": " + yellow + "Dude" + reset) - print("special characters are\n") - print("lightning_bolt: {}".format(lightning_bolt)) - print("double_vertical_line: {}".format(double_vertical_line)) - print("check_mark: {}".format(check_mark)) - print("multiplication_x: {}".format(multiplication_x)) - print("left_arrow: {}".format(left_arrow)) - print("right_arrow: {}".format(right_arrow)) - print("circled_m: {}".format(circled_m)) - # print("has unicode: {}".format(has_unicode())) diff --git a/kios_bt_planning/backups/dynamic_bt/py_trees/decorators.py b/kios_bt_planning/backups/dynamic_bt/py_trees/decorators.py deleted file mode 100644 index aaa30987..00000000 --- a/kios_bt_planning/backups/dynamic_bt/py_trees/decorators.py +++ /dev/null @@ -1,728 +0,0 @@ -#!/usr/bin/env python -# -# License: BSD -# https://raw.githubusercontent.com/splintered-reality/py_trees/devel/LICENSE -# -############################################################################## -# Documentation -############################################################################## - -""" -Decorators are behaviours that manage a single child and provide common -modifications to their underlying child behaviour (e.g. inverting the result). -That is, they provide a means for behaviours to wear different 'hats' and -this combinatorially expands the capabilities of your behaviour library. - -.. image:: images/many-hats.png - :width: 40px - :align: center - -An example: - -.. graphviz:: dot/decorators.dot - :align: center - -.. literalinclude:: examples/decorators.py - :language: python - :linenos: - - -**Decorators (Hats)** - -Decorators with very specific functionality: - -* :class:`py_trees.decorators.Condition` -* :class:`py_trees.decorators.EternalGuard` -* :class:`py_trees.decorators.Inverter` -* :class:`py_trees.decorators.OneShot` -* :class:`py_trees.decorators.StatusToBlackboard` -* :class:`py_trees.decorators.Timeout` - -And the X is Y family: - -* :class:`py_trees.decorators.FailureIsRunning` -* :class:`py_trees.decorators.FailureIsSuccess` -* :class:`py_trees.decorators.RunningIsFailure` -* :class:`py_trees.decorators.RunningIsSuccess` -* :class:`py_trees.decorators.SuccessIsFailure` -* :class:`py_trees.decorators.SuccessIsRunning` - -**Decorators for Blocking Behaviours** - -It is worth making a note of the effect of decorators on -behaviours that return :data:`~py_trees.common.Status.RUNNING` for -some time before finally returning :data:`~py_trees.common.Status.SUCCESS` -or :data:`~py_trees.common.Status.FAILURE` (blocking behaviours) since -the results are often at first, surprising. - -A decorator, such as :func:`py_trees.decorators.RunningIsSuccess` on -a blocking behaviour will immediately terminate the underlying child and -re-intialise on it's next tick. This is necessary to ensure the underlying -child isn't left in a dangling state (i.e. -:data:`~py_trees.common.Status.RUNNING`), but is often not what is being -sought. - -The typical use case being attempted is to convert the blocking -behaviour into a non-blocking behaviour. If the underlying child has no -state being modified in either the :meth:`~py_trees.behaviour.Behaviour.initialise` -or :meth:`~py_trees.behaviour.Behaviour.terminate` methods (e.g. machinery is -entirely launched at init or setup time), then conversion to a non-blocking -representative of the original succeeds. Otherwise, another approach is -needed. Usually this entails writing a non-blocking counterpart, or -combination of behaviours to affect the non-blocking characteristics. -""" - -############################################################################## -# Imports -############################################################################## - -import functools -import inspect -import time -import typing - -from typing import Callable, List, Set, Union # noqa - -from py_trees import behaviour -from py_trees import blackboard -from py_trees import common - -############################################################################## -# Classes -############################################################################## - - -class Decorator(behaviour.Behaviour): - """ - A decorator is responsible for handling the lifecycle of a single - child beneath - - Args: - child: the child to be decorated - name: the decorator name - - Raises: - TypeError: if the child is not an instance of :class:`~py_trees.behaviour.Behaviour` - """ - - def __init__( - self, - child: behaviour.Behaviour = None, - name=common.Name.AUTO_GENERATED - ): - # Initialise - super().__init__(name=name) - if child is not None: - self.add_child(child) - else: - self.children = [] - - def tick(self): - """ - A decorator's tick is exactly the same as a normal proceedings for - a Behaviour's tick except that it also ticks the decorated child node. - - Yields: - :class:`~py_trees.behaviour.Behaviour`: a reference to itself or one of its children - """ - if self.logger.level < 1: - self.logger.debug("%s.tick()" % self.__class__.__name__) - # initialise just like other behaviours/composites - if self.status != common.Status.RUNNING: - self.initialise() - # interrupt proceedings and process the child node - # (including any children it may have as well) - for node in self.decorated.tick(): - yield node - # resume normal proceedings for a Behaviour's tick - new_status = self.update() - if new_status not in list(common.Status): - self.logger.error( - "A behaviour returned an invalid status, setting to INVALID [%s][%s]" % (new_status, self.name)) - new_status = common.Status.INVALID - if new_status != common.Status.RUNNING: - self.stop(new_status) - self.status = new_status - yield self - - def stop(self, new_status): - """ - As with other composites, it checks if the child is running - and stops it if that is the case. - - Args: - new_status (:class:`~py_trees.common.Status`): the behaviour is transitioning to this new status - """ - if self.logger.level < 1: - self.logger.debug("%s.stop(%s)" % (self.__class__.__name__, new_status)) - self.terminate(new_status) - # priority interrupt handling - if new_status == common.Status.INVALID: - self.decorated.stop(new_status) - # if the decorator returns SUCCESS/FAILURE and should stop the child - if self.decorated.status == common.Status.RUNNING: - self.decorated.stop(common.Status.INVALID) - self.status = new_status - - def tip(self): - """ - Get the *tip* of this behaviour's subtree (if it has one) after it's last - tick. This corresponds to the the deepest node that was running before the - subtree traversal reversed direction and headed back to this node. - - Returns: - :class:`~py_trees.behaviour.Behaviour` or :obj:`None`: child behaviour, itself or :obj:`None` if its status is :data:`~py_trees.common.Status.INVALID` - """ - if self.decorated.status != common.Status.INVALID: - return self.decorated.tip() - else: - return super().tip() - - def add_child(self, child): - """ - Adds a child. - - Args: - child (:class:`~py_trees.behaviour.Behaviour`): child to add - - Raises: - TypeError: if the child is not an instance of :class:`~py_trees.behaviour.Behaviour` - RuntimeError: if the child already has a parent - - Returns: - uuid.UUID: unique id of the child - """ - if not isinstance(child, behaviour.Behaviour): - raise TypeError( - "children must be behaviours, but you passed in {}".format(type(child))) - self.children.append(child) - if child.parent is not None: - raise RuntimeError("behaviour '{}' already has parent '{}'".format( - child.name, child.parent.name)) - # Give a convenient alias - self.decorated = self.children[0] - self.decorated.parent = self - return child.id - -############################################################################## -# Decorators -############################################################################## - - -class StatusToBlackboard(Decorator): - """ - Reflect the status of the decorator's child to the blackboard. - - Args: - child: the child behaviour or subtree - variable_name: name of the blackboard variable, may be nested, e.g. foo.status - name: the decorator name - """ - - def __init__( - self, - *, - child: behaviour.Behaviour, - variable_name: str, - name: Union[str, common.Name] = common.Name.AUTO_GENERATED, - ): - super().__init__(child=child, name=name) - self.variable_name = variable_name - name_components = variable_name.split('.') - self.key = name_components[0] - self.key_attributes = '.'.join(name_components[1:]) # empty string if no other parts - self.blackboard = self.attach_blackboard_client(self.name) - self.blackboard.register_key(key=self.key, access=common.Access.WRITE) - - def update(self): - """ - Reflect the decorated child's status to the blackboard and return - - Returns: the decorated child's status - """ - self.blackboard.set( - name=self.variable_name, - value=self.decorated.status, - overwrite=True - ) - return self.decorated.status - - -ConditionType = Union[ - Callable[[], bool], - Callable[[], common.Status], - Callable[[blackboard.Blackboard], bool], - Callable[[blackboard.Blackboard], common.Status] -] - - -class EternalGuard(Decorator): - """ - A decorator that continually guards the execution of a subtree. - If at any time the guard's condition check fails, then the child - behaviour/subtree is invalidated. - - .. note:: This decorator's behaviour is stronger than the - :term:`guard` typical of a conditional check at the beginning of a - sequence of tasks as it continues to check on every tick whilst the - task (or sequence of tasks) runs. - - Args: - child: the child behaviour or subtree - condition: a functional check that determines execution or not of the subtree - blackboard_keys: provide read access for the conditional function to these keys - name: the decorator name - - Examples: - - Simple conditional function returning True/False: - - .. code-block:: python - - def check(): - return True - - foo = py_trees.behaviours.Foo() - eternal_guard = py_trees.decorators.EternalGuard( - name="Eternal Guard", - condition=check, - child=foo - ) - - Simple conditional function returning SUCCESS/FAILURE: - - .. code-block:: python - - def check(): - return py_trees.common.Status.SUCCESS - - foo = py_trees.behaviours.Foo() - eternal_guard = py_trees.decorators.EternalGuard( - name="Eternal Guard", - condition=check, - child=foo - ) - - Conditional function that makes checks against data on the blackboard (the - blackboard client with pre-configured access is provided by the EternalGuard - instance): - - .. code-block:: python - - def check(blackboard): - return blackboard.velocity > 3.0 - - foo = py_trees.behaviours.Foo() - eternal_guard = py_trees.decorators.EternalGuard( - name="Eternal Guard", - condition=check, - blackboard_keys={"velocity"}, - child=foo - ) - - .. seealso:: :meth:`py_trees.idioms.eternal_guard` - """ - - def __init__( - self, - *, - child: behaviour.Behaviour, - # Condition is one of 4 callable types illustrated in the docstring, partials complicate - # it as well. When typing_extensions are available (very recent) more generally, can use - # Protocols to handle it. Probably also a sign that it's not a very clean api though... - condition: typing.Any, - blackboard_keys: Union[List[str], Set[str]] = [], - name: Union[str, common.Name] = common.Name.AUTO_GENERATED, - ): - super().__init__(name=name, child=child) - self.blackboard = self.attach_blackboard_client(self.name) - for key in blackboard_keys: - self.blackboard.register_key(key=key, access=common.Access.READ) - condition_signature = inspect.signature(condition) - if "blackboard" in [p.name for p in condition_signature.parameters.values()]: - self.condition = functools.partial(condition, self.blackboard) - else: - self.condition = condition - - def tick(self): - """ - A decorator's tick is exactly the same as a normal proceedings for - a Behaviour's tick except that it also ticks the decorated child node. - - Yields: - :class:`~py_trees.behaviour.Behaviour`: a reference to itself or one of its children - """ - if self.logger.level < 1: - self.logger.debug("%s.tick()" % self.__class__.__name__) - - # condition check - result = self.condition() - if type(result) == common.Status: - result = False if result == common.Status.FAILURE else True - elif type(result) != bool: - error_message = "conditional check must return 'bool' or 'common.Status' [{}]".format( - type(result)) - self.logger.error("The {}".format(error_message)) - raise RuntimeError(error_message) - - if not result: - # abort, abort, the FSM is losing his noodles!!! - self.stop(common.Status.FAILURE) - yield self - else: - # normal behaviour - for node in super().tick(): - yield node - - def update(self): - """ - The update method is only ever triggered in the child's post-tick, which implies - that the condition has already been checked and passed (refer to the :meth:`tick` method). - """ - return self.decorated.status - - -class Timeout(Decorator): - """ - A decorator that applies a timeout pattern to an existing behaviour. - If the timeout is reached, the encapsulated behaviour's - :meth:`~py_trees.behaviour.Behaviour.stop` method is called with - status :data:`~py_trees.common.Status.FAILURE` otherwise it will - simply directly tick and return with the same status - as that of it's encapsulated behaviour. - """ - - def __init__( - self, - child: behaviour.Behaviour, - name: Union[str, common.Name] = common.Name.AUTO_GENERATED, - duration: float = 5.0 - ): - """ - Init with the decorated child and a timeout duration. - - Args: - child: the child behaviour or subtree - name: the decorator name - duration: timeout length in seconds - """ - super(Timeout, self).__init__(name=name, child=child) - self.duration = duration - self.finish_time = None - - def initialise(self): - """ - Reset the feedback message and finish time on behaviour entry. - """ - self.finish_time = time.monotonic() + self.duration - self.feedback_message = "" - - def update(self): - """ - Terminate the child and return :data:`~py_trees.common.Status.FAILURE` - if the timeout is exceeded. - """ - current_time = time.monotonic() - if self.decorated.status == common.Status.RUNNING and current_time > self.finish_time: - if self.logger.level < 1: - self.feedback_message = "timed out" - self.logger.debug("{}.update() {}".format( - self.__class__.__name__, self.feedback_message)) - # invalidate the decorated (i.e. cancel it), could also put this logic in a terminate() method - self.decorated.stop(common.Status.INVALID) - return common.Status.FAILURE - if self.logger.level < 1: - if self.decorated.status == common.Status.RUNNING: - self.feedback_message = "time still ticking ... [remaining: {}s]".format( - self.finish_time - current_time - ) - else: - self.feedback_message = "child finished before timeout triggered" - return self.decorated.status - - -class OneShot(Decorator): - """ - A decorator that implements the oneshot pattern. - - This decorator ensures that the underlying child is ticked through - to completion just once and while doing so, will return - with the same status as it's child. Thereafter it will return - with the final status of the underlying child. - - Completion status is determined by the policy given on construction. - - * With policy :data:`~py_trees.common.OneShotPolicy.ON_SUCCESSFUL_COMPLETION`, the oneshot will activate only when the underlying child returns :data:`~py_trees.common.Status.SUCCESS` (i.e. it permits retries). - * With policy :data:`~py_trees.common.OneShotPolicy.ON_COMPLETION`, the oneshot will activate when the child returns :data:`~py_trees.common.Status.SUCCESS` || :data:`~py_trees.common.Status.FAILURE`. - - .. seealso:: :meth:`py_trees.idioms.oneshot` - """ - - def __init__(self, child, - name=common.Name.AUTO_GENERATED, - policy=common.OneShotPolicy.ON_SUCCESSFUL_COMPLETION): - """ - Init with the decorated child. - - Args: - name (:obj:`str`): the decorator name - child (:class:`~py_trees.behaviour.Behaviour`): behaviour to time - policy (:class:`~py_trees.common.OneShotPolicy`): policy determining when the oneshot should activate - """ - super(OneShot, self).__init__(name=name, child=child) - self.final_status = None - self.policy = policy - - def update(self): - """ - Bounce if the child has already successfully completed. - """ - if self.final_status: - if self.logger.level < 1: - self.logger.debug("{}.update()[bouncing]".format(self.__class__.__name__)) - return self.final_status - return self.decorated.status - - def tick(self): - """ - Select between decorator (single child) and behaviour (no children) style - ticks depending on whether or not the underlying child has been ticked - successfully to completion previously. - """ - if self.final_status: - # ignore the child - for node in behaviour.Behaviour.tick(self): - yield node - else: - # tick the child - for node in Decorator.tick(self): - yield node - - def terminate(self, new_status): - """ - If returning :data:`~py_trees.common.Status.SUCCESS` for the first time, - flag it so future ticks will block entry to the child. - """ - if not self.final_status and new_status in self.policy.value: - if self.logger.level < 1: - self.logger.debug("{}.terminate({})[oneshot completed]".format( - self.__class__.__name__, new_status)) - self.feedback_message = "oneshot completed" - self.final_status = new_status - else: - if self.logger.level < 1: - self.logger.debug("{}.terminate({})".format(self.__class__.__name__, new_status)) - - -class Inverter(Decorator): - """ - A decorator that inverts the result of a class's update function. - """ - - def __init__(self, child=None, name=common.Name.AUTO_GENERATED): - """ - Init with the decorated child. - - Args: - child (:class:`~py_trees.behaviour.Behaviour`): behaviour to time - name (:obj:`str`): the decorator name - """ - super(Inverter, self).__init__(name=name, child=child) - - def update(self): - """ - Flip :data:`~py_trees.common.Status.FAILURE` and - :data:`~py_trees.common.Status.SUCCESS` - - Returns: - :class:`~py_trees.common.Status`: the behaviour's new status :class:`~py_trees.common.Status` - """ - if self.decorated.status == common.Status.SUCCESS: - if self.logger.level < 1: - self.feedback_message = "success -> failure" - return common.Status.FAILURE - elif self.decorated.status == common.Status.FAILURE: - if self.logger.level < 1: - self.feedback_message = "failure -> success" - return common.Status.SUCCESS - self.feedback_message = self.decorated.feedback_message - return self.decorated.status - - -class RunningIsFailure(Decorator): - """ - Got to be snappy! We want results...yesterday! - """ - - def update(self): - """ - Return the decorated child's status unless it is - :data:`~py_trees.common.Status.RUNNING` in which case, return - :data:`~py_trees.common.Status.FAILURE`. - - Returns: - :class:`~py_trees.common.Status`: the behaviour's new status :class:`~py_trees.common.Status` - """ - if self.decorated.status == common.Status.RUNNING: - if self.logger.level < 1: - self.feedback_message = "running is failure" + \ - (" [%s]" % self.decorated.feedback_message if self.decorated.feedback_message else "") - return common.Status.FAILURE - else: - self.feedback_message = self.decorated.feedback_message - return self.decorated.status - - -class RunningIsSuccess(Decorator): - """ - Don't hang around... - """ - - def update(self): - """ - Return the decorated child's status unless it is - :data:`~py_trees.common.Status.RUNNING` in which case, return - :data:`~py_trees.common.Status.SUCCESS`. - - Returns: - :class:`~py_trees.common.Status`: the behaviour's new status :class:`~py_trees.common.Status` - """ - if self.decorated.status == common.Status.RUNNING: - if self.logger.level < 1: - self.feedback_message = "running is success" + \ - (" [%s]" % self.decorated.feedback_message if self.decorated.feedback_message else "") - return common.Status.SUCCESS - self.feedback_message = self.decorated.feedback_message - return self.decorated.status - - -class FailureIsSuccess(Decorator): - """ - Be positive, always succeed. - """ - - def update(self): - """ - Return the decorated child's status unless it is - :data:`~py_trees.common.Status.FAILURE` in which case, return - :data:`~py_trees.common.Status.SUCCESS`. - - Returns: - :class:`~py_trees.common.Status`: the behaviour's new status :class:`~py_trees.common.Status` - """ - if self.decorated.status == common.Status.FAILURE: - if self.logger.level < 1: - self.feedback_message = "failure is success" + \ - (" [%s]" % self.decorated.feedback_message if self.decorated.feedback_message else "") - return common.Status.SUCCESS - self.feedback_message = self.decorated.feedback_message - return self.decorated.status - - -class FailureIsRunning(Decorator): - """ - Dont stop running. - """ - - def update(self): - """ - Return the decorated child's status unless it is - :data:`~py_trees.common.Status.FAILURE` in which case, return - :data:`~py_trees.common.Status.RUNNING`. - - Returns: - :class:`~py_trees.common.Status`: the behaviour's new status :class:`~py_trees.common.Status` - """ - if self.decorated.status == common.Status.FAILURE: - if self.logger.level < 1: - self.feedback_message = "failure is running" + \ - (" [%s]" % self.decorated.feedback_message if self.decorated.feedback_message else "") - return common.Status.RUNNING - self.feedback_message = self.decorated.feedback_message - return self.decorated.status - - -class SuccessIsFailure(Decorator): - """ - Be depressed, always fail. - """ - - def update(self): - """ - Return the decorated child's status unless it is - :data:`~py_trees.common.Status.SUCCESS` in which case, return - :data:`~py_trees.common.Status.FAILURE`. - - Returns: - :class:`~py_trees.common.Status`: the behaviour's new status :class:`~py_trees.common.Status` - """ - if self.decorated.status == common.Status.SUCCESS: - if self.logger.level < 1: - self.feedback_message = "success is failure" + \ - (" [%s]" % self.decorated.feedback_message if self.decorated.feedback_message else "") - return common.Status.FAILURE - self.feedback_message = self.decorated.feedback_message - return self.decorated.status - - -class SuccessIsRunning(Decorator): - """ - It never ends... - """ - - def update(self): - """ - Return the decorated child's status unless it is - :data:`~py_trees.common.Status.SUCCESS` in which case, return - :data:`~py_trees.common.Status.RUNNING`. - - Returns: - :class:`~py_trees.common.Status`: the behaviour's new status :class:`~py_trees.common.Status` - """ - if self.decorated.status == common.Status.SUCCESS: - if self.logger.level < 1: - self.feedback_message = "success is running [%s]" % self.decorated.feedback_message - return common.Status.RUNNING - self.feedback_message = self.decorated.feedback_message - return self.decorated.status - - -class Condition(Decorator): - """ - Encapsulates a behaviour and wait for it's status to flip to the - desired state. This behaviour will tick with - :data:`~py_trees.common.Status.RUNNING` while waiting and - :data:`~py_trees.common.Status.SUCCESS` when the flip occurs. - """ - - def __init__(self, - child, - name=common.Name.AUTO_GENERATED, - status=common.Status.SUCCESS): - """ - Initialise with child and optional name, status variables. - - Args: - child (:class:`~py_trees.behaviour.Behaviour`): the child to be decorated - name (:obj:`str`): the decorator name (can be None) - status (:class:`~py_trees.common.Status`): the desired status to watch for - """ - super(Condition, self).__init__(child, name) - self.succeed_status = status - - def update(self): - """ - :data:`~py_trees.common.Status.SUCCESS` if the decorated child has returned - the specified status, otherwise :data:`~py_trees.common.Status.RUNNING`. - This decorator will never return :data:`~py_trees.common.Status.FAILURE` - - Returns: - :class:`~py_trees.common.Status`: the behaviour's new status :class:`~py_trees.common.Status` - """ - if self.logger.level < 1: - self.logger.debug("%s.update()" % self.__class__.__name__) - self.feedback_message = "'{0}' has status {1}, waiting for {2}".format( - self.decorated.name, self.decorated.status, self.succeed_status) - if self.decorated.status == self.succeed_status: - return common.Status.SUCCESS - return common.Status.RUNNING diff --git a/kios_bt_planning/backups/dynamic_bt/py_trees/display.py b/kios_bt_planning/backups/dynamic_bt/py_trees/display.py deleted file mode 100644 index 3571a953..00000000 --- a/kios_bt_planning/backups/dynamic_bt/py_trees/display.py +++ /dev/null @@ -1,1008 +0,0 @@ -#!/usr/bin/env python -# -# License: BSD -# https://raw.githubusercontent.com/splintered-reality/py_trees/devel/LICENSE -# -############################################################################## -# Documentation -############################################################################## - -""" -Behaviour trees are significantly easier to design, monitor and debug -with visualisations. Py Trees does provide minimal assistance to render -trees to various simple output formats. Currently this includes dot graphs, -strings or stdout. -""" - -############################################################################## -# Imports -############################################################################## - -import os -import pydot -import typing -import uuid - -from py_trees import behaviour -from py_trees import blackboard -from py_trees import common -from py_trees import composites -from py_trees import console -from py_trees import decorators -from py_trees import utilities - -############################################################################## -# Symbols -############################################################################## - -Symbols = typing.Dict[typing.Any, str] - -unicode_symbols = { - 'space': ' ', - 'left_arrow': console.left_arrow, - 'right_arrow': console.right_arrow, - 'left_right_arrow': console.left_right_arrow, - 'bold': console.bold, - 'bold_reset': console.reset, - 'memory': 'm', - 'sequence_with_memory': u'{-}', - 'selector_with_memory': u'{o}', - composites.Sequence: u'[-]', - composites.Selector: u'[o]', - composites.Parallel: u'/_/', - decorators.Decorator: u'-^-', - behaviour.Behaviour: u'-->', - common.Status.SUCCESS: console.green + console.check_mark + console.reset, - common.Status.FAILURE: console.red + console.multiplication_x + console.reset, - common.Status.INVALID: console.yellow + u'-' + console.reset, - common.Status.RUNNING: console.blue + u'*' + console.reset -} -"""Symbols for a unicode, escape sequence capable console.""" - -ascii_symbols = { - 'space': ' ', - 'left_arrow': '<-', - 'right_arrow': '->', - 'left_right_arrow': '<->', - 'bold': console.bold, - 'bold_reset': console.reset, - 'memory': 'M', - 'sequence_with_memory': '{-}', - 'selector_with_memory': '{o}', - composites.Sequence: "[-]", - composites.Selector: "[o]", - composites.Parallel: "/_/", - decorators.Decorator: "-^-", - behaviour.Behaviour: "-->", - common.Status.SUCCESS: console.green + 'o' + console.reset, - common.Status.FAILURE: console.red + 'x' + console.reset, - common.Status.INVALID: console.yellow + '-' + console.reset, - common.Status.RUNNING: console.blue + '*' + console.reset -} -"""Symbols for a non-unicode, non-escape sequence capable console.""" -xhtml_symbols = { - #   is not valid xhtml, see http://www.fileformat.info/info/unicode/char/00a0/index.htm - 'space': ' ', - 'left_arrow': '←', - 'right_arrow': '→', - 'left_right_arrow': '↔', - 'bold': '', - 'bold_reset': '', - 'memory': 'Ⓜ', - 'sequence_with_memory': '{-}', - 'selector_with_memory': '{o}', - composites.Sequence: '[-]', - composites.Selector: '[o]', - composites.Parallel: '/_/', - decorators.Decorator: '-^-', - behaviour.Behaviour: '-->', - common.Status.SUCCESS: '✓', # c.f. console.check_mark - common.Status.FAILURE: '✕', # c.f. console.multiplication_x - common.Status.INVALID: '-', - common.Status.RUNNING: '*' -} -"""Symbols for embedding in html.""" - -############################################################################## -# Trees -############################################################################## - - -def _generate_text_tree( - root, - show_only_visited=False, - show_status=False, - visited={}, - previously_visited={}, - indent=0, - symbols=None): - """ - Generate a text tree utilising the specified symbol formatter. - - Args: - root (:class:`~py_trees.behaviour.Behaviour`): the root of the tree, or subtree you want to show - show_only_visited (:obj:`bool`): show only visited behaviours - show_status (:obj:`bool`): always show status and feedback message (i.e. for every element, - not just those visited) - visited (dict): dictionary of (uuid.UUID) and status (:class:`~py_trees.common.Status`) pairs - for behaviours visited on the current tick - previously_visited (dict): dictionary of behaviour id/status pairs from the previous tree tick - indent (:obj:`int`): the number of characters to indent the tree - symbols (dict, optional): dictates formatting style - (one of :data:`py_trees.display.unicode_symbols` || :data:`py_trees.display.ascii_symbols` || :data:`py_trees.display.xhtml_symbols`), - defaults to unicode if stdout supports it, ascii otherwise - - Returns: - :obj:`str`: a text-based representation of the behaviour tree - - .. seealso:: :meth:`py_trees.display.ascii_tree`, :meth:`py_trees.display.unicode_tree`, :meth:`py_trees.display.xhtml_tree` - """ - # default to unicode if stdout supports it, ascii otherwise - if symbols is None: - symbols = unicode_symbols if console.has_unicode() else ascii_symbols - tip_id = root.tip().id if root.tip() else None - - def get_behaviour_type(b): - if isinstance(b, composites.Parallel): - bt_type = composites.Parallel - if isinstance(b, decorators.Decorator): - bt_type = decorators.Decorator - if isinstance(b, composites.Sequence): - bt_type = "sequence_with_memory" if b.memory else composites.Sequence - if isinstance(b, composites.Selector): - bt_type = "selector_with_memory" if b.memory else composites.Selector - if isinstance(b, composites.ShrinkedTree): - bt_type = composites.ShrinkedTree - else: - bt_type = behaviour.Behaviour - return bt_type - - def style(s, font_weight=False): - """ - Because the way the shell escape sequences reset everything, this needs to get used on any - single block of formatted text. - """ - if font_weight: - return symbols['bold'] + s + symbols['bold_reset'] - else: - return s - - def generate_lines(root, internal_indent): - - def assemble_single_line(b): - # [Matteo:] I don't know why this is done, better set it to False! - font_weight = True if b.id == tip_id else False - font_weight = False - s = "" - s += symbols['space'] * 4 * internal_indent - s += style(symbols[get_behaviour_type(b)], font_weight) - s += " " - - if show_status or b.id in visited.keys(): - s += style("{} [".format(b.name.replace('\n', ' ')), font_weight) - s += style("{}".format(symbols[b.status]), font_weight) - message = "" if not b.feedback_message else " -- " + b.feedback_message - s += style("]" + message, font_weight) - elif (b.id in previously_visited.keys() and - b.id not in visited.keys() and - previously_visited[b.id] == common.Status.RUNNING): - s += style("{} [".format(b.name.replace('\n', ' ')), font_weight) - s += style("{}".format(symbols[b.status]), font_weight) - s += style("]", font_weight) - else: - s += style("{}".format(b.name.replace('\n', ' ')), font_weight) - return s - - if internal_indent == indent: - # Root - yield assemble_single_line(root) - internal_indent += 1 - for child in root.children: - yield assemble_single_line(child) - if child.children != []: - if not show_only_visited or child.id in visited.keys(): - for line in generate_lines(child, internal_indent + 1): - yield line - else: - yield "{}...".format(symbols['space'] * 4 * (internal_indent + 1)) - s = "" - for line in generate_lines(root, indent): - if line: - s += "%s\n" % line - return s - - -def ascii_tree( - root, - show_only_visited=False, - show_status=False, - visited={}, - previously_visited={}, - indent=0): - """ - Graffiti your console with ascii art for your trees. - - Args: - root (:class:`~py_trees.behaviour.Behaviour`): the root of the tree, or subtree you want to show - show_only_visited (:obj:`bool`) : show only visited behaviours - show_status (:obj:`bool`): always show status and feedback message (i.e. for every element, not just those visited) - visited (dict): dictionary of (uuid.UUID) and status (:class:`~py_trees.common.Status`) pairs for behaviours visited on the current tick - previously_visited (dict): dictionary of behaviour id/status pairs from the previous tree tick - indent (:obj:`int`): the number of characters to indent the tree - - Returns: - :obj:`str`: an ascii tree (i.e. in string form) - - .. seealso:: :meth:`py_trees.display.xhtml_tree`, :meth:`py_trees.display.unicode_tree` - - Examples: - - Use the :class:`~py_trees.visitors.SnapshotVisitor` - and :class:`~py_trees.trees.BehaviourTree` - to generate snapshot information at each tick and feed that to - a post tick handler that will print the traversed ascii tree - complete with status and feedback messages. - - .. image:: images/ascii_tree.png - :width: 200px - :align: right - - .. code-block:: python - - def post_tick_handler(snapshot_visitor, behaviour_tree): - print( - py_trees.display.unicode_tree( - behaviour_tree.root, - visited=snapshot_visitor.visited, - previously_visited=snapshot_visitor.visited - ) - ) - - root = py_trees.composites.Sequence("Sequence") - for action in ["Action 1", "Action 2", "Action 3"]: - b = py_trees.behaviours.Count( - name=action, - fail_until=0, - running_until=1, - success_until=10) - root.add_child(b) - behaviour_tree = py_trees.trees.BehaviourTree(root) - snapshot_visitor = py_trees.visitors.SnapshotVisitor() - behaviour_tree.add_post_tick_handler( - functools.partial(post_tick_handler, - snapshot_visitor)) - behaviour_tree.visitors.append(snapshot_visitor) - """ - lines = _generate_text_tree( - root, - show_only_visited, - show_status, - visited, - previously_visited, - indent, - symbols=ascii_symbols - ) - return lines - - -def unicode_tree( - root, - show_only_visited=False, - show_status=False, - visited={}, - previously_visited={}, - indent=0): - """ - Graffiti your console with unicode art for your trees. - - Args: - root (:class:`~py_trees.behaviour.Behaviour`): the root of the tree, or subtree you want to show - show_only_visited (:obj:`bool`) : show only visited behaviours - show_status (:obj:`bool`): always show status and feedback message (i.e. for every element, not just those visited) - visited (dict): dictionary of (uuid.UUID) and status (:class:`~py_trees.common.Status`) pairs for behaviours visited on the current tick - previously_visited (dict): dictionary of behaviour id/status pairs from the previous tree tick - indent (:obj:`int`): the number of characters to indent the tree - - Returns: - :obj:`str`: a unicode tree (i.e. in string form) - - .. seealso:: :meth:`py_trees.display.ascii_tree`, :meth:`py_trees.display.xhtml_tree` - - """ - lines = _generate_text_tree( - root, - show_only_visited, - show_status, - visited, - previously_visited, - indent, - symbols=ascii_symbols - ) - return lines - - -def xhtml_tree( - root, - show_only_visited=False, - show_status=False, - visited={}, - previously_visited={}, - indent=0): - """ - Paint your tree on an xhtml snippet. - - Args: - root (:class:`~py_trees.behaviour.Behaviour`): the root of the tree, or subtree you want to show - show_only_visited (:obj:`bool`) : show only visited behaviours - show_status (:obj:`bool`): always show status and feedback message (i.e. for every element, not just those visited) - visited (dict): dictionary of (uuid.UUID) and status (:class:`~py_trees.common.Status`) pairs for behaviours visited on the current tick - previously_visited (dict): dictionary of behaviour id/status pairs from the previous tree tick - indent (:obj:`int`): the number of characters to indent the tree - - Returns: - :obj:`str`: an ascii tree (i.e. as a xhtml snippet) - - .. seealso:: :meth:`py_trees.display.ascii_tree`, :meth:`py_trees.display.unicode_tree` - - Examples: - - .. code-block:: python - - import py_trees - a = py_trees.behaviours.Success() - b = py_trees.behaviours.Success() - c = c = py_trees.composites.Sequence(children=[a, b]) - c.tick_once() - - f = open('testies.html', 'w') - f.write('Foo') - f.write(py_trees.display.xhtml_tree(c, show_status=True)) - f.write("") - """ - lines = _generate_text_tree( - root, - show_only_visited, - show_status, - visited, - previously_visited, - indent, - symbols=xhtml_symbols - ) - lines = lines.replace("\n", "
\n") - return "\n" + lines + "" - - -def dot_tree( - root: behaviour.Behaviour, - visibility_level: common.VisibilityLevel = common.VisibilityLevel.DETAIL, - collapse_decorators: bool = False, - with_blackboard_variables: bool = False, - with_qualified_names: bool = False, - static: bool = True, -): - """ - Paint your tree on a pydot graph. - - .. seealso:: :py:func:`render_dot_tree`. - - Args: - root (:class:`~py_trees.behaviour.Behaviour`): the root of a tree, or subtree - visibility_level (optional): collapse subtrees at or under this level - collapse_decorators (optional): only show the decorator (not the child), defaults to False - with_blackboard_variables (optional): add nodes for the blackboard variables - with_qualified_names (optional): print the class information for each behaviour in each node, defaults to False - static (optional): if False, it prints the tree with the color code depending on the running status - Returns: - pydot.Dot: graph - - Examples: - - .. code-block:: python - - # convert the pydot graph to a string object - print("{}".format(py_trees.display.dot_graph(root).to_string())) - """ - def get_node_attributes(node, static=True): - blackbox_font_colours = {common.BlackBoxLevel.DETAIL: "dodgerblue", - common.BlackBoxLevel.COMPONENT: "lawngreen", - common.BlackBoxLevel.BIG_PICTURE: "white" - } - if isinstance(node, composites.Sequence) or node.name.startswith('Sequence'): - # In this case we get also custom RSequence - if not static: - color_ = get_status_color(node) - attributes = ('box', color_, 'black') - else: - attributes = ('box', 'orange', 'black') - elif isinstance(node, composites.Selector): - if not static: - color_ = get_status_color(node) - attributes = ('octagon', color_, 'black') - else: - attributes = ('octagon', 'cyan', 'black') - elif isinstance(node, composites.Parallel): - attributes = ('parallelogram', 'gold', 'black') - elif isinstance(node, decorators.Decorator): - attributes = ('diamond', 'gold', 'black') - elif isinstance(node, composites.ShrinkedTree): - attributes = ('box', 'white', 'black') - else: # actions and conditions - if not static: - color_ = get_status_color(node) - attributes = ('ellipse', color_, 'black') - else: - attributes = ('ellipse', 'lightgray', 'black') - try: - if node.blackbox_level != common.BlackBoxLevel.NOT_A_BLACKBOX: - attributes = (attributes[0], 'gray20', blackbox_font_colours[node.blackbox_level]) - except AttributeError: - # it's a blackboard client, not a behaviour, just pass - pass - return attributes - - def get_status_color(node): - """ - Return a color based on the node state. - """ - if node.status.value == 'SUCCESS': - color = 'green' - elif node.status.value == 'FAILURE': - color = 'red' - elif node.status.value == 'RUNNING': - color = 'yellow' - else: # INVALID - color = 'lightgray' - - return color - - def get_node_label(node_name, behaviour): - """ - This extracts a more detailed string (when applicable) to append to - that which will be used for the node name. - """ - # Custom handling of composites provided by this library. Not currently - # providing a generic mechanism for others to customise visualisations - # for their derived composites. - prefix = "" - policy = "" - if isinstance(behaviour, composites.Composite): - try: - if behaviour.memory: - prefix += 'm' - except AttributeError: - pass - try: - if behaviour.policy.synchronise: - pass - # prefix += console.lightning_bolt - except AttributeError: - pass - try: - policy = behaviour.policy.__class__.__name__ - except AttributeError: - pass - try: - indices = [str(behaviour.children.index(child)) - for child in behaviour.policy.children] - policy += "({})".format(', '.join(sorted(indices))) - except AttributeError: - pass - node_label = f"{prefix} {node_name}" if prefix else node_name - if policy: - pass - # node_label += f"\n{str(policy)}" - if with_qualified_names: - node_label += f"\n({utilities.get_fully_qualified_name(behaviour)})" - return node_label - - fontsize = 9 - blackboard_colour = "blue" # "dimgray" - graph = pydot.Dot(graph_type='digraph', ordering="out") - # consider making this unique to the tree sometime, e.g. based on the root name - graph.set_name("pastafarianism") - # fonts: helvetica, times-bold, arial (times-roman is the default, but this helps some viewers, like kgraphviewer) - # splines='curved' is buggy on 16.04, but would be nice to have - graph.set_graph_defaults(fontname='times-roman') - graph.set_node_defaults(fontname='times-roman') - graph.set_edge_defaults(fontname='times-roman') - (node_shape, node_colour, node_font_colour) = get_node_attributes(root, static) - node_name = root.name - node_root = pydot.Node( - name=root.name, - label=get_node_label(root.name, root), - shape=node_shape, - style="filled", - fillcolor=node_colour, - fontsize=fontsize, - fontcolor=node_font_colour, - ) - graph.add_node(node_root) - behaviour_id_name_map = {root.id: root.name} - - def add_children_and_edges(root, root_node, root_dot_name, visibility_level, collapse_decorators, static): - if isinstance(root, decorators.Decorator) and collapse_decorators: - return - if visibility_level < root.blackbox_level: - node_names = [] - for c in root.children: - (node_shape, node_colour, node_font_colour) = get_node_attributes(c, static) - node_name = c.name - node_label = node_name - while node_name in behaviour_id_name_map.values(): - node_name += " " - behaviour_id_name_map[c.id] = node_name - # Node attributes can be found on page 5 of - # https://graphviz.gitlab.io/_pages/pdf/dot.1.pdf - # Attributes that may be useful: tooltip, xlabel - node = pydot.Node( - name=node_name, - label=get_node_label(node_label, c), - shape=node_shape, - style="filled", - fillcolor=node_colour, - fontsize=fontsize, - fontcolor=node_font_colour, - ) - node_names.append(node_name) - graph.add_node(node) - edge = pydot.Edge(root_dot_name, node_name) - graph.add_edge(edge) - if c.children != []: - add_children_and_edges(c, node, node_name, visibility_level, - collapse_decorators, static) - - add_children_and_edges(root, node_root, root.name, visibility_level, - collapse_decorators, static) - - def create_blackboard_client_node(client_name: str): - return pydot.Node( - name=client_name, - label=client_name, - shape="ellipse", - style="filled", - color=blackboard_colour, - fillcolor="gray", - fontsize=fontsize - 2, - fontcolor=blackboard_colour, - ) - - def add_blackboard_nodes(blackboard_id_name_map: typing.Dict[uuid.UUID, str]): - data = blackboard.Blackboard.storage - metadata = blackboard.Blackboard.metadata - clients = blackboard.Blackboard.clients - # add client (that are not behaviour) nodes - subgraph = pydot.Subgraph( - graph_name="Blackboard", - id="Blackboard", - label="Blackboard", - rank="sink", - ) - - for unique_identifier, client_name in clients.items(): - if unique_identifier not in blackboard_id_name_map: - subgraph.add_node( - create_blackboard_client_node(client_name) - ) - # add key nodes - for key in blackboard.Blackboard.keys(): - try: - value = utilities.truncate(str(data[key]), 20) - label = key + ": " + "{}".format(value) - except KeyError: - label = key + ": " + "-" - blackboard_node = pydot.Node( - key, - label=label, - shape='box', - style="filled", - color=blackboard_colour, - fillcolor='white', - fontsize=fontsize - 1, - fontcolor=blackboard_colour, - width=0, height=0, fixedsize=False, # only big enough to fit text - ) - subgraph.add_node(blackboard_node) - for unique_identifier in metadata[key].read: - try: - edge = pydot.Edge( - blackboard_node, - blackboard_id_name_map[unique_identifier], - color="green", - constraint=False, - weight=0, - ) - except KeyError: - edge = pydot.Edge( - blackboard_node, - clients[unique_identifier], - color="green", - constraint=False, - weight=0, - ) - graph.add_edge(edge) - for unique_identifier in metadata[key].write: - try: - edge = pydot.Edge( - blackboard_id_name_map[unique_identifier], - blackboard_node, - color=blackboard_colour, - constraint=False, - weight=0, - ) - except KeyError: - edge = pydot.Edge( - clients[unique_identifier], - blackboard_node, - color=blackboard_colour, - constraint=False, - weight=0, - ) - graph.add_edge(edge) - graph.add_subgraph(subgraph) - - if with_blackboard_variables: - blackboard_id_name_map = {} - for b in root.iterate(): - for bb in b.blackboards: - blackboard_id_name_map[bb.id()] = behaviour_id_name_map[b.id] - add_blackboard_nodes(blackboard_id_name_map) - - return graph - - -def render_dot_tree( - root: behaviour.Behaviour, - visibility_level: common.VisibilityLevel = common.VisibilityLevel.DETAIL, - collapse_decorators: bool = False, - name: str = None, - target_directory: str = os.getcwd(), - with_blackboard_variables: bool = False, - with_qualified_names: bool = False, - static: bool = True -): - """ - Render the dot tree to .dot, .svg, .png. files in the current - working directory. These will be named with the root behaviour name. - Args: - root: the root of a tree, or subtree - visibility_level: collapse subtrees at or under this level - collapse_decorators: only show the decorator (not the child) - name: name to use for the created files (defaults to the root behaviour name) - target_directory: default is to use the current working directory, set this to redirect elsewhere - with_blackboard_variables: add nodes for the blackboard variables - with_qualified_names: print the class names of each behaviour in the dot node - static (optional): if False, it prints the tree with the color code depending on the running status - Example: - Render a simple tree to dot/svg/png file: - .. graphviz:: dot/sequence.dot - .. code-block:: python - root = py_trees.composites.Sequence("Sequence") - for job in ["Action 1", "Action 2", "Action 3"]: - success_after_two = py_trees.behaviours.Count(name=job, - fail_until=0, - running_until=1, - success_until=10) - root.add_child(success_after_two) - py_trees.display.render_dot_tree(root) - .. tip:: - A good practice is to provide a command line argument for optional rendering of a program so users - can quickly visualise what tree the program will execute. - """ - graph = dot_tree( - root, visibility_level, collapse_decorators, - with_blackboard_variables=with_blackboard_variables, - with_qualified_names=with_qualified_names, static=static) - filename_wo_extension_to_convert = root.name if name is None else name - filename_wo_extension = utilities.get_valid_filename(filename_wo_extension_to_convert) - filenames = {} - for extension, writer in {"png": graph.write_png, "svg": graph.write_svg}.items(): - filename = filename_wo_extension + '.' + extension - pathname = os.path.join(target_directory, filename) - print("Writing {}".format(pathname)) - writer(pathname) - filenames[extension] = pathname - return filenames - -############################################################################## -# Blackboards -############################################################################## - - -def _generate_text_blackboard( - key_filter: typing.Union[typing.Set[str], typing.List[str]] = None, - regex_filter: str = None, - client_filter: typing.Union[typing.Set[uuid.UUID], typing.List[uuid.UUID]] = None, - keys_to_highlight: typing.List[str] = [], - display_only_key_metadata: bool = False, - indent: int = 0, - symbols: typing.Optional[Symbols] = None) -> str: - """ - Generate a text blackboard. - Args: - key_filter: filter on a set/list of blackboard keys - regex_filter: filter on a python regex str - client_filter: filter on a set/list of client uuids - keys_to_highlight: list of keys to highlight - display_only_key_metadata: (read/write access, ...) instead of values - indent: the number of characters to indent the blackboard - symbols: dictates formatting style - (one of :data:`py_trees.display.unicode_symbols` || :data:`py_trees.display.ascii_symbols` || :data:`py_trees.display.xhtml_symbols`), - defaults to unicode if stdout supports it, ascii otherwise - Returns: - a text-based representation of the behaviour tree - .. seealso:: :meth:`py_trees.display.unicode_blackboard` - """ - if symbols is None: - symbols = unicode_symbols if console.has_unicode() else ascii_symbols - - def style(s, font_weight=False): - if font_weight: - return symbols['bold'] + s + symbols['bold_reset'] - else: - return s - - def generate_lines(storage, metadata, indent): - def assemble_value_line(key, value, apply_highlight, indent, key_width): - s = "" - lines = ('{0}'.format(value)).split('\n') - if len(lines) > 1: - s += console.cyan + indent + \ - '{0: <{1}}'.format(key, key_width) + console.white + ":\n" - for line in lines: - s += console.yellow + indent + " {0}\n".format(line) - else: - s += console.cyan + indent + \ - '{0: <{1}}'.format(key, key_width) + console.white + ": " + \ - console.yellow + '{0}\n'.format(value) + console.reset - return style(s, apply_highlight) + console.reset - - def assemble_metadata_line(key, metadata, apply_highlight, indent, key_width): - s = "" - s += console.cyan + indent + '{0: <{1}}'.format(key, key_width + 1) + ": " - client_uuids = list(set(metadata.read) | set(metadata.write) | set(metadata.exclusive)) - prefix = '' - metastrings = [] - for client_uuid in client_uuids: - metastring = prefix + '{0}'.format( - utilities.truncate( - blackboard.Blackboard.clients[client_uuid], 11 - ) - ) - metastring += ' (' - if client_uuid in metadata.read: - metastring += 'r' - if client_uuid in metadata.write: - metastring += 'w' - if client_uuid in metadata.exclusive: - metastring += 'x' - metastring += ')' - metastrings.append(metastring) - s += console.yellow + "{}\n".format(', '.join(metastrings)) - return style(s, apply_highlight) + console.reset - - text_indent = symbols['space'] * (4 + indent) - key_width = 0 - for key in storage.keys(): - key_width = len(key) if len(key) > key_width else key_width - for key in sorted(storage.keys()): - if metadata is not None: - yield assemble_metadata_line( - key=key, - metadata=metadata[key], - apply_highlight=key in keys_to_highlight, - indent=text_indent, - key_width=key_width) - else: - yield assemble_value_line( - key=key, - value=storage[key], - apply_highlight=key in keys_to_highlight, - indent=text_indent, - key_width=key_width) - - blackboard_metadata = blackboard.Blackboard.metadata if display_only_key_metadata else None - - if key_filter: - if isinstance(key_filter, list): - key_filter = set(key_filter) - all_keys = blackboard.Blackboard.keys() & key_filter - elif regex_filter: - all_keys = blackboard.Blackboard.keys_filtered_by_regex(regex_filter) - elif client_filter: - all_keys = blackboard.Blackboard.keys_filtered_by_clients(client_filter) - else: - all_keys = blackboard.Blackboard.keys() - blackboard_storage = {} - for key in all_keys: - try: - blackboard_storage[key] = blackboard.Blackboard.storage[key] - except KeyError: - blackboard_storage[key] = "-" - - title = "Clients" if display_only_key_metadata else "Data" - s = console.green + symbols['space'] * indent + "Blackboard {}\n".format(title) + console.reset - if key_filter: - s += symbols['space'] * (indent + 2) + "Filter: '{}'\n".format(key_filter) - elif regex_filter: - s += symbols['space'] * (indent + 2) + "Filter: '{}'\n".format(regex_filter) - elif client_filter: - s += symbols['space'] * (indent + 2) + "Filter: {}\n".format(str(client_filter)) - for line in generate_lines(blackboard_storage, blackboard_metadata, indent): - s += "{}".format(line) - return s - - -def ascii_blackboard( - key_filter: typing.Union[typing.Set[str], typing.List[str]] = None, - regex_filter: str = None, - client_filter: typing.Optional[typing.Union[typing.Set[uuid.UUID], - typing.List[uuid.UUID]]] = None, - keys_to_highlight: typing.List[str] = [], - display_only_key_metadata: bool = False, - indent: int = 0) -> str: - """ - Graffiti your console with ascii art for your blackboard. - Args: - key_filter: filter on a set/list of blackboard keys - regex_filter: filter on a python regex str - client_filter: filter on a set/list of client uuids - keys_to_highlight: list of keys to highlight - display_only_key_metadata: read/write access, ... instead of values - indent: the number of characters to indent the blackboard - Returns: - a unicoded blackboard (i.e. in string form) - .. seealso:: :meth:`py_trees.display.unicode_blackboard` - .. note:: registered variables that have not yet been set are marked with a '-' - """ - lines = _generate_text_blackboard( - key_filter=key_filter, - regex_filter=regex_filter, - client_filter=client_filter, - keys_to_highlight=keys_to_highlight, - display_only_key_metadata=display_only_key_metadata, - indent=indent, - symbols=ascii_symbols - ) - return lines - - -def unicode_blackboard( - key_filter: typing.Union[typing.Set[str], typing.List[str]] = None, - regex_filter: str = None, - client_filter: typing.Optional[typing.Union[typing.Set[uuid.UUID], - typing.List[uuid.UUID]]] = None, - keys_to_highlight: typing.List[str] = [], - display_only_key_metadata: bool = False, - indent: int = 0) -> str: - """ - Graffiti your console with unicode art for your blackboard. - Args: - key_filter: filter on a set/list of blackboard keys - regex_filter: filter on a python regex str - client_filter: filter on a set/list of client uuids - keys_to_highlight: list of keys to highlight - display_only_key_metadata: read/write access, ... instead of values - indent: the number of characters to indent the blackboard - Returns: - a unicoded blackboard (i.e. in string form) - .. seealso:: :meth:`py_trees.display.ascii_blackboard` - .. note:: registered variables that have not yet been set are marked with a '-' - """ - lines = _generate_text_blackboard( - key_filter=key_filter, - regex_filter=regex_filter, - client_filter=client_filter, - keys_to_highlight=keys_to_highlight, - display_only_key_metadata=display_only_key_metadata, - indent=indent, - symbols=None # defaults to unicode, falls back to ascii - ) - return lines - - -def _generate_text_activity( - activity_stream: typing.Optional[typing.List[blackboard.ActivityItem]] = None, - show_title: bool = True, - indent: int = 0, - symbols: typing.Optional[Symbols] = None -) -> str: - """ - Generator for the various formatted outputs (ascii, unicode, xhtml). - Args: - activity_stream: the log of activity, if None, get the entire activity stream - indent: the number of characters to indent the blackboard - show_title: include the title in the output - """ - if symbols is None: - symbols = unicode_symbols if console.has_unicode() else ascii_symbols - space = symbols['space'] - if activity_stream is None and blackboard.Blackboard.activity_stream is not None: - activity_stream = blackboard.Blackboard.activity_stream.data - s = "" - if show_title: - s += space * indent + console.green + "Blackboard Activity Stream" + console.reset + "\n" - if activity_stream is not None: - key_width = 0 - client_width = 0 - for item in activity_stream: - key_width = len(item.key) if len(item.key) > key_width else key_width - client_width = len(item.client_name) if len( - item.client_name) > client_width else client_width - client_width = min(client_width, 20) - type_width = len("ACCESS_DENIED") - value_width = 80 - key_width - 3 - type_width - 3 - client_width - 3 - for item in activity_stream: - s += console.cyan + space * (4 + indent) - s += "{0: <{1}}:".format(item.key, key_width + 1) + space - s += console.yellow - s += "{0: <{1}}".format(item.activity_type, type_width) + space - s += console.white + "|" + space - s += "{0: <{1}}".format( - utilities.truncate( - item.client_name.replace('\n', '_'), - client_width), - client_width) + space - s += "|" + space - if item.activity_type == blackboard.ActivityType.READ.value: - s += symbols["left_arrow"] + space + "{}\n".format( - utilities.truncate(str(item.current_value), value_width) - ) - elif item.activity_type == blackboard.ActivityType.WRITE.value: - s += console.green - s += symbols["right_arrow"] + space - s += "{}\n".format( - utilities.truncate(str(item.current_value), value_width) - ) - elif item.activity_type == blackboard.ActivityType.ACCESSED.value: - s += console.yellow - s += symbols["left_right_arrow"] + space - s += "{}\n".format( - utilities.truncate(str(item.current_value), value_width) - ) - elif item.activity_type == blackboard.ActivityType.ACCESS_DENIED.value: - s += console.red - s += console.multiplication_x + space - s += "client has no read/write access\n" - elif item.activity_type == blackboard.ActivityType.NO_KEY.value: - s += console.red - s += console.multiplication_x + space - s += "key does not yet exist\n" - elif item.activity_type == blackboard.ActivityType.NO_OVERWRITE.value: - s += console.yellow - s += console.forbidden_circle + space - s += "{}\n".format( - utilities.truncate(str(item.current_value), value_width) - ) - elif item.activity_type == blackboard.ActivityType.UNSET.value: - s += "\n" - elif item.activity_type == blackboard.ActivityType.INITIALISED.value: - s += console.green - s += symbols["right_arrow"] + space - s += "{}\n".format( - utilities.truncate(str(item.current_value), value_width) - ) - else: - s += "unknown operation\n" - s = s.rstrip("\n") - s += console.reset - return s - - -def unicode_blackboard_activity_stream( - activity_stream: typing.List[blackboard.ActivityItem] = None, - indent: int = 0, - show_title: bool = True -): - """ - Pretty print the blackboard stream to console. - Args: - activity_stream: the log of activity, if None, get the entire activity stream - indent: the number of characters to indent the blackboard - show_title: include the title in the output - """ - return _generate_text_activity( - activity_stream=activity_stream, - show_title=show_title, - indent=indent, - symbols=unicode_symbols if console.has_unicode() else ascii_symbols - ) diff --git a/kios_bt_planning/backups/dynamic_bt/py_trees/idioms.py b/kios_bt_planning/backups/dynamic_bt/py_trees/idioms.py deleted file mode 100644 index ce563241..00000000 --- a/kios_bt_planning/backups/dynamic_bt/py_trees/idioms.py +++ /dev/null @@ -1,357 +0,0 @@ -#!/usr/bin/env python -# -# License: BSD -# https://raw.githubusercontent.com/splintered-reality/py_trees/devel/LICENSE -# -############################################################################## -# Documentation -############################################################################## - -""" -A library of subtree creators that build complex patterns of behaviours -representing common behaviour tree idioms. -""" - -############################################################################## -# Imports -############################################################################## - -import operator -import typing -import uuid - -from py_trees import behaviour -from py_trees import behaviours -from py_trees import blackboard -from py_trees import common -from py_trees import composites -from py_trees import decorators - -############################################################################## -# Creational Methods -############################################################################## - - -def pick_up_where_you_left_off( - name="Pickup Where You Left Off Idiom", - tasks=[]): - """ - Rudely interrupted while enjoying a sandwich, a caveman (just because - they wore loincloths does not mean they were not civilised), picks - up his club and fends off the sabre-tooth tiger invading his sanctum - as if he were swatting away a gnat. Task accomplished, he returns - to the joys of munching through the layers of his sandwich. - - .. graphviz:: dot/pick_up_where_you_left_off.dot - - .. note:: - - There are alternative ways to accomplish this idiom with their - pros and cons. - - a) The tasks in the sequence could be replaced by a - factory behaviour that dynamically checks the state of play and - spins up the tasks required each time the task sequence is first - entered and invalidates/deletes them when it is either finished - or invalidated. That has the advantage of not requiring much of - the blackboard machinery here, but disadvantage in not making - visible the task sequence itself at all times (i.e. burying - details under the hood). - - b) A new composite which retains the index between - initialisations can also achieve the same pattern with fewer - blackboard shenanigans, but suffers from an increased - logical complexity cost for your trees (each new composite - increases decision making complexity (O(n!)). - - Args: - name (:obj:`str`): the name to use for the task sequence behaviour - tasks ([:class:`~py_trees.behaviour.Behaviour`): lists of tasks to be sequentially performed - - Returns: - :class:`~py_trees.behaviour.Behaviour`: root of the generated subtree - """ - root = composites.Sequence(name=name) - for task in tasks: - task_selector = composites.Selector(name="Do or Don't") - task_guard = behaviours.CheckBlackboardVariableValue( - name="Done?", - check=common.ComparisonExpression( - variable=task.name.lower().replace(" ", "_") + "_done", - value=True, - operator=operator.eq - ) - ) - sequence = composites.Sequence(name="Worker") - mark_task_done = behaviours.SetBlackboardVariable( - name="Mark\n" + task.name.lower().replace(" ", "_") + "_done", - variable_name=task.name.lower().replace(" ", "_") + "_done", - variable_value=True - ) - sequence.add_children([task, mark_task_done]) - task_selector.add_children([task_guard, sequence]) - root.add_child(task_selector) - for task in tasks: - clear_mark_done = behaviours.UnsetBlackboardVariable( - name="Clear\n" + task.name.lower().replace(" ", "_") + "_done", - key=task.name.lower().replace(" ", "_") + "_done" - ) - root.add_child(clear_mark_done) - return root - - -def eternal_guard( - subtree: behaviour.Behaviour, - name: str="Eternal Guard", - conditions: typing.List[behaviour.Behaviour]=[], - blackboard_namespace: str=None) -> behaviour.Behaviour: - """ - The eternal guard idiom implements a stronger :term:`guard` than the typical check at the - beginning of a sequence of tasks. Here they guard continuously while the task sequence - is being executed. While executing, if any of the guards should update with - status :data:`~common.Status.FAILURE`, then the task sequence is terminated. - - .. graphviz:: dot/idiom-eternal-guard.dot - :align: center - - Args: - subtree: behaviour(s) that actually do the work - name: the name to use on the root behaviour of the idiom subtree - conditions: behaviours on which tasks are conditional - blackboard_namespace: applied to condition variable results stored on the blackboard (default: derived from the idiom name) - - Returns: - the root of the idiom subtree - - .. seealso:: :class:`py_trees.decorators.EternalGuard` - """ - if blackboard_namespace is None: - blackboard_namespace = name.lower().replace(" ", "_") - blackboard_variable_names = [] - # construct simple, easy to read, variable names (risk of conflict) - counter = 1 - for condition in conditions: - suffix = "" if len(conditions) == 1 else "_{}".format(counter) - blackboard_variable_names.append( - blackboard.Blackboard.separator + - blackboard_namespace + - "_condition" + - suffix - ) - counter += 1 - # if there is just one blackboard name already on the blackboard, switch to unique names - conflict = False - for name in blackboard_variable_names: - try: - unused_name = blackboard.Blackboard.get(name) - conflict = True - except KeyError: - pass - if conflict: - blackboard_variable_names = [] - counter = 1 - unique_id = uuid.uuid4() - for condition in conditions: - suffix = "" if len(conditions) == 1 else "_{}".format(counter) - blackboard_variable_names.append(blackboard_namespace + "_" + str(unique_id) + "_condition" + suffix) - counter += 1 - # build the tree - root = composites.Parallel( - name=name, - policy=common.ParallelPolicy.SuccessOnAll(synchronise=False) - ) - guarded_tasks = composites.Selector(name="Guarded Tasks") - for condition, blackboard_variable_name in zip(conditions, blackboard_variable_names): - decorated_condition = decorators.StatusToBlackboard( - name="StatusToBB", - child=condition, - variable_name=blackboard_variable_name - ) - root.add_child(decorated_condition) - guarded_tasks.add_child( - behaviours.CheckBlackboardVariableValue( - name="Abort on\n{}".format(condition.name), - check=common.ComparisonExpression( - variable=blackboard_variable_name, - value=common.Status.FAILURE, - operator=operator.eq - ) - ) - ) - guarded_tasks.add_child(subtree) - root.add_child(guarded_tasks) - return root - - -def either_or( - conditions: typing.List[common.ComparisonExpression], - subtrees: typing.List[behaviour.Behaviour], - name="Either Or", - namespace: typing.Optional[str]=None -) -> behaviour.Behaviour: - """ - Often you need a kind of selector that doesn't implement prioritisations, i.e. - you would like different paths to be selected on a first-come, first-served basis. - - .. code-block:: python - - task_one = py_trees.behaviours.TickCounter(name="Subtree 1", duration=2) - task_two = py_trees.behaviours.TickCounter(name="Subtree 2", duration=2) - either_or = py_trees.idioms.either_or( - name="EitherOr", - conditions=[ - py_trees.common.ComparisonExpression("joystick_one", "enabled", operator.eq), - py_trees.common.ComparisonExpression("joystick_two", "enabled", operator.eq), - ], - subtrees=[task_one, task_two], - namespace="either_or", - ) - - .. graphviz:: dot/idiom-either-or.dot - :align: center - :caption: Idiom - Either Or - - Up front is an XOR conditional check which locks in the result on the blackboard - under the specified namespace. Locking the result in permits the conditional - variables to vary in future ticks without interrupting the execution of the - chosen subtree (an example of a conditional variable may be one that has - registered joystick button presses). - - Once the result is locked in, the relevant subtree is activated beneath the - selector. The children of the selector are, from left to right, not in any - order of priority since the previous xor choice has been locked in and isn't - revisited until the subtree executes to completion. Only one - may be active and it cannot be interrupted by the others. - - The only means of interrupting the execution is via a higher priority in the - tree that this idiom is embedded in. - - Args: - conditions: list of triggers that ultimately select the subtree to enable - subtrees: list of subtrees to tick from in the either_or operation - name: the name to use for this idiom's root behaviour - preemptible: whether the subtrees may preempt (interrupt) each other - namespace: this idiom's private variables will be put behind this namespace - - Raises: - ValueError if the number of conditions does not match the number of subtrees - - If no namespace is provided, a unique one is derived from the idiom's name. - - .. seealso:: :ref:`py-trees-demo-either-or ` - - .. todo:: a version for which other subtrees can preempt (in an unprioritised manner) the active branch - """ - if len(conditions) != len(subtrees): - raise ValueError("Must be the same number of conditions as subtrees [{} != {}]".format( - len(conditions), len(subtrees)) - ) - root = composites.Sequence(name=name) - configured_namespace: str = namespace if namespace is not None else \ - blackboard.Blackboard.separator + name.lower().replace("-", "_").replace(" ", "_") + \ - blackboard.Blackboard.separator + str(root.id).replace("-", "_").replace(" ", "_") + \ - blackboard.Blackboard.separator + "conditions" - xor = behaviours.CheckBlackboardVariableValues( - name="XOR", - checks=conditions, - operator=operator.xor, - namespace=configured_namespace - ) - chooser = composites.Selector(name="Chooser") - for counter in range(1, len(conditions) + 1): - sequence = composites.Sequence(name="Option {}".format(str(counter))) - variable_name = configured_namespace + blackboard.Blackboard.separator + str(counter) - disabled = behaviours.CheckBlackboardVariableValue( - name="Enabled?", - check=common.ComparisonExpression( - variable=variable_name, - value=True, - operator=operator.eq - ) - ) - sequence.add_children([disabled, subtrees[counter - 1]]) - chooser.add_child(sequence) - root.add_children([xor, chooser]) - return root - - -def oneshot( - behaviour: behaviour.Behaviour, - name: str="Oneshot", - variable_name: str="oneshot", - policy: common.OneShotPolicy=common.OneShotPolicy.ON_SUCCESSFUL_COMPLETION -) -> behaviour.Behaviour: - """ - Ensure that a particular pattern is executed through to - completion just once. Thereafter it will just rebound with the completion status. - - .. graphviz:: dot/oneshot.dot - - .. note:: - - Set the policy to configure the oneshot to keep trying if failing, or to abort - further attempts regardless of whether it finished with status - :data:`~py_trees.common.Status.SUCCESS`||:data:`~py_trees.common.Status.FAILURE`. - - Args: - behaviour: single behaviour or composited subtree to oneshot - name: the name to use for the oneshot root (selector) - variable_name: name for the variable used on the blackboard, may be nested - policy: execute just once regardless of success or failure, or keep trying if failing - - Returns: - :class:`~py_trees.behaviour.Behaviour`: the root of the oneshot subtree - - .. seealso:: :class:`py_trees.decorators.OneShot` - """ - subtree_root = composites.Selector(name=name) - oneshot_with_guard = composites.Sequence( - name="Oneshot w/ Guard") - check_not_done = decorators.Inverter( - name="Not Completed?", - child=behaviours.CheckBlackboardVariableExists( - name="Completed?", - variable_name=variable_name - ) - ) - set_flag_on_success = behaviours.SetBlackboardVariable( - name="Mark Done\n[SUCCESS]", - variable_name=variable_name, - variable_value=common.Status.SUCCESS - ) - # If it's a sequence, don't double-nest it in a redundant manner - if isinstance(behaviour, composites.Sequence): - behaviour.add_child(set_flag_on_success) - sequence = behaviour - else: - sequence = composites.Sequence(name="OneShot") - sequence.add_children([behaviour, set_flag_on_success]) - - oneshot_with_guard.add_child(check_not_done) - if policy == common.OneShotPolicy.ON_SUCCESSFUL_COMPLETION: - oneshot_with_guard.add_child(sequence) - else: # ON_COMPLETION (SUCCESS || FAILURE) - oneshot_handler = composites.Selector(name="Oneshot Handler") - bookkeeping = composites.Sequence(name="Bookkeeping") - set_flag_on_failure = behaviours.SetBlackboardVariable( - name="Mark Done\n[FAILURE]", - variable_name=variable_name, - variable_value=common.Status.FAILURE - ) - bookkeeping.add_children( - [set_flag_on_failure, - behaviours.Failure(name="Failure") - ]) - oneshot_handler.add_children([sequence, bookkeeping]) - oneshot_with_guard.add_child(oneshot_handler) - - oneshot_result = behaviours.CheckBlackboardVariableValue( - name="Oneshot Result", - check=common.ComparisonExpression( - variable=variable_name, - value=common.Status.SUCCESS, - operator=operator.eq - ) - ) - subtree_root.add_children([oneshot_with_guard, oneshot_result]) - return subtree_root diff --git a/kios_bt_planning/backups/dynamic_bt/py_trees/meta.py b/kios_bt_planning/backups/dynamic_bt/py_trees/meta.py deleted file mode 100644 index dd33d0f0..00000000 --- a/kios_bt_planning/backups/dynamic_bt/py_trees/meta.py +++ /dev/null @@ -1,48 +0,0 @@ -#!/usr/bin/env python -# -# License: BSD -# https://raw.githubusercontent.com/splintered-reality/py_trees/devel/LICENSE -# -############################################################################## -# Documentation -############################################################################## - -""" -Meta methods to create behaviours without needing to create the behaviours -themselves. -""" - -############################################################################## -# Imports -############################################################################## - -from py_trees import behaviour -from py_trees import common - -############################################################################## -# Utility Methods -############################################################################## - - -def create_behaviour_from_function(func): - """ - Create a behaviour from the specified function, dropping it in for - the Behaviour :meth:`~py_trees.behaviour.Behaviour.update` method. - Ths function must include the `self` - argument and return a :class:`~py_trees.behaviours.common.Status` value. - It also automatically provides a drop-in for the :meth:`~py_trees.behaviour.Behaviour.terminate` - method that clears the feedback message. Other methods are left untouched. - - Args: - func (:obj:`function`): a drop-in for the :meth:`~py_trees.behaviour.Behaviour.update` method - """ - class_name = func.__name__.capitalize() - - def init(self, name=class_name): - behaviour.Behaviour.__init__(self, name=name) - - def terminate(self, new_status): - if new_status == common.Status.INVALID: - self.feedback_message = "" - - return type(class_name, (behaviour.Behaviour,), dict(__init__=init, update=func, terminate=terminate)) diff --git a/kios_bt_planning/backups/dynamic_bt/py_trees/pt_logging.py b/kios_bt_planning/backups/dynamic_bt/py_trees/pt_logging.py deleted file mode 100644 index 022bbb2f..00000000 --- a/kios_bt_planning/backups/dynamic_bt/py_trees/pt_logging.py +++ /dev/null @@ -1,90 +0,0 @@ -#!/usr/bin/env python -# -# License: BSD -# https://raw.githubusercontent.com/splintered-reality/py_trees/devel/LICENSE -# -############################################################################## -# Documentation -############################################################################## - -""" -.. module:: loggers - :synopsis: Logging facilities in py_trees. - -Oh my spaghettified magnificence, -Bless my noggin with a tickle from your noodly appendages! -""" - -############################################################################## -# Imports -############################################################################## - -from enum import IntEnum - -from py_trees import console - -############################################################################## -# Logging -############################################################################## - -# I'd really prefer to use python logging facilities, but rospy logging -# on top of python logging kills it. -# -# Could still use it here, and would actually be useful if I could -# integrate it with nosetests, but for now, this will do. -# Note, you can get colour with python logging, but its tricky; -# -# http://stackoverflow.com/questions/384076/how-can-i-color-python-logging-output -# -# python way: -# -# import logging -# logging.getLogger("py_trees.Behaviour") -# logging.basicConfig(level=logging.DEBUG) -# -############################################################################## -# Level -############################################################################## - - -# levels -class Level(IntEnum): - """ - An enumerator representing the logging level. - Not valid if you override with your own loggers. - """ - DEBUG = 0 - INFO = 1 - WARN = 2 - ERROR = 3 - - -############################################################################## -# Logger Class -############################################################################## - - -class Logger(object): - """ - :cvar override: whether or not the default python logger has been overridden. - :vartype override: bool - """ - - def __init__(self, name=None, level=Level.INFO): - self.prefix = '{:<20}'.format(name.replace("\n", " ")) + " : " if name else "" - self.level = level - - def debug(self, msg): - if self.level < Level.INFO: - console.logdebug(self.prefix + msg) - - def info(self, msg): - if self.level < Level.WARN: - console.loginfo(self.prefix + msg) - - def warning(self, msg): - if self.level < Level.ERROR: - console.logwarn(self.prefix + msg) - - def error(self, msg): - console.logerror(self.prefix + msg) diff --git a/kios_bt_planning/backups/dynamic_bt/py_trees/syntax_highlighting.py b/kios_bt_planning/backups/dynamic_bt/py_trees/syntax_highlighting.py deleted file mode 100644 index 29cae710..00000000 --- a/kios_bt_planning/backups/dynamic_bt/py_trees/syntax_highlighting.py +++ /dev/null @@ -1,61 +0,0 @@ -#!/usr/bin/env python -# -# License: BSD -# https://raw.githubusercontent.com/splintered-reality/py_trees/devel/LICENSE -# -############################################################################## -# Documentation -############################################################################## - -""" -.. module:: syntax_highlighting - :synopsis: Syntax highlighting for various py_trees objects - -Oh my spaghettified magnificence, -Bless my noggin with a tickle from your noodly appendages! -""" - -############################################################################## -# Imports -############################################################################## - -from py_trees import common -from py_trees import console - -############################################################################## -# Syntax Highlighting -############################################################################## - - -_status_colour_strings = {common.Status.SUCCESS: console.green + str(common.Status.SUCCESS) + console.reset, - common.Status.RUNNING: console.blue + str(common.Status.RUNNING) + console.reset, - common.Status.FAILURE: console.red + str(common.Status.FAILURE) + console.reset, - common.Status.INVALID: console.yellow + str(common.Status.INVALID) + console.reset - } - -_status_colour_codes = {common.Status.SUCCESS: console.green, - common.Status.RUNNING: console.cyan, - common.Status.FAILURE: console.red, - common.Status.INVALID: console.yellow - } - - -def status(status): - """ - Syntax colouring for :py:class:`~py_trees.common.Status` - - :param :py:class:`~py_trees.common.Status` Status: behaviour status - :returns: syntax highlighted string representation of the status - """ - return _status_colour_strings[status] - - -def status_colour_code(status): - """ - Colour code associated with the :py:class:`~py_trees.common.Status` for - syntax highlighting - - :param :py:class:`~py_trees.common.Status` Status: behaviour status - :returns: console colour code associated with the status - """ - return _status_colour_codes[status] diff --git a/kios_bt_planning/backups/dynamic_bt/py_trees/tests.py b/kios_bt_planning/backups/dynamic_bt/py_trees/tests.py deleted file mode 100644 index de0f588d..00000000 --- a/kios_bt_planning/backups/dynamic_bt/py_trees/tests.py +++ /dev/null @@ -1,79 +0,0 @@ -#!/usr/bin/env python -# -# License: BSD -# https://raw.githubusercontent.com/splintered-reality/py_trees/devel/LICENSE -# -############################################################################## -# Documentation -############################################################################## - -""" -Library of common methods for the tests. - -Oh my spaghettified magnificence, -Bless my noggin with a tickle from your noodly appendages! -""" - -############################################################################## -# Imports -############################################################################## - -from py_trees import blackboard -from py_trees import console -from py_trees import display - -############################################################################## -# Methods -############################################################################## - - -def print_assert_banner(): - print(console.green + "\n--------- Assertions ---------\n" + console.reset) - - -def print_assert_details(text, expected, result): - print(console.green + text + - "." * (70 - len(text)) + - console.cyan + "{}".format(expected) + - console.yellow + " [{}]".format(result) + - console.reset) - - -def pre_tick_visitor(behaviour_tree): - print("\n--------- Run %s ---------\n" % behaviour_tree.count) - - -def tick_tree(root, - from_tick, - to_tick, - *, - visitors=[], - print_snapshot=False, - print_blackboard=False - ): - print("\n================== Iteration {}-{} ==================\n".format(from_tick, to_tick)) - for i in range(from_tick, to_tick + 1): - for visitor in visitors: - visitor.initialise() - print(("\n--------- Run %s ---------\n" % i)) - for node in root.tick(): - for visitor in visitors: - node.visit(visitor) - if print_snapshot: - print(console.green + "\nTree Snapshot" + console.reset) - print(display.unicode_tree(root=root, show_status=True)) - if print_blackboard: - print(display.unicode_blackboard()) - - -def clear_blackboard(): - # Useful between tests - blackboard.Blackboard.storage = {} - blackboard.Blackboard.clients = {} - blackboard.Blackboard.metadata = {} - - -def print_summary(nodes): - print("\n--------- Summary ---------\n") - for node in nodes: - print("%s" % node) diff --git a/kios_bt_planning/backups/dynamic_bt/py_trees/timers.py b/kios_bt_planning/backups/dynamic_bt/py_trees/timers.py deleted file mode 100644 index 61c171f2..00000000 --- a/kios_bt_planning/backups/dynamic_bt/py_trees/timers.py +++ /dev/null @@ -1,102 +0,0 @@ -#!/usr/bin/env python -# -# License: BSD -# https://raw.githubusercontent.com/splintered-reality/py_trees/devel/LICENSE -# -############################################################################## -# Documentation -############################################################################## - -""" -Time related behaviours. -""" - -############################################################################## -# Imports -############################################################################## - -import numbers -import time - -from py_trees import behaviour -from py_trees import common - -############################################################################## -# Behaviours -############################################################################## - - -class Timer(behaviour.Behaviour): - """ - Simple timer class that is :py:data:`~py_trees.common.Status.RUNNING` until the timer - runs out, at which point it is :data:`~py_trees.common.Status.SUCCESS`. This can be - used in a wide variety of situations - pause, duration, timeout depending on how - it is wired into the tree (e.g. pause in a sequence, duration/timeout in - a parallel). - - The timer gets reset either upon entry (:meth:`~py_trees.behaviour.Behaviour.initialise`) - if it hasn't already been set and gets cleared when it either runs out, or the behaviour is - interrupted by a higher priority or parent cancelling it. - - Args: - name (:obj:`str`): name of the behaviour - duration (:obj:`int`): length of time to run (in seconds) - - Raises: - TypeError: if the provided duration is not a real number - - .. note:: - This succeeds the first time the behaviour is ticked **after** the expected - finishing time. - - .. tip:: - Use the :func:`~py_trees.decorators.RunningIsFailure` decorator if you need - :data:`~py_trees.common.Status.FAILURE` until the timer finishes. - """ - def __init__(self, name="Timer", duration=5.0): - super(Timer, self).__init__(name) - if not isinstance(duration, numbers.Real): - raise TypeError("Timer: duration should be int or float, but you passed in {}".format(type(duration))) - self.duration = duration - self.finish_time = None - if self.logger.level < 1: - self.feedback_message = "duration set to '{0}'s".format(self.duration) - - def initialise(self): - """ - Store the expected finishing time. - """ - if self.logger.level < 1: - self.logger.debug("%s.initialise()" % self.__class__.__name__) - self.feedback_message = "configured to fire in '{0}' seconds".format(self.duration) - if self.finish_time is None: - self.finish_time = time.time() + self.duration - - def update(self): - """ - Check current time against the expected finishing time. If it is in excess, flip to - :data:`~py_trees.common.Status.SUCCESS`. - """ - if self.logger.level < 1: - self.logger.debug("%s.update()" % self.__class__.__name__) - current_time = time.time() - if current_time > self.finish_time: - if self.logger.level < 1: - self.feedback_message = "timer ran out [{0}]".format(self.duration) - return common.Status.SUCCESS - else: - # do not show the time, it causes the tree to be 'changed' every tick - # and we don't want to spam visualisations with almost meaningless updates - if self.logger.level < 1: - self.feedback_message = "still running" # (%s)" % (self.finish_time - current_time) - return common.Status.RUNNING - - def terminate(self, new_status): - """ - Clear the expected finishing time. - """ - if self.logger.level < 1: - self.logger.debug("%s.terminate(%s)" % (self.__class__.__name__, "%s->%s" % (self.status, new_status) if self.status != new_status else "%s" % new_status)) - # clear the time if finishing with SUCCESS or in the case of an interruption from INVALID - if new_status == common.Status.SUCCESS or new_status == common.Status.INVALID: - self.finish_time = None diff --git a/kios_bt_planning/backups/dynamic_bt/py_trees/trees.py b/kios_bt_planning/backups/dynamic_bt/py_trees/trees.py deleted file mode 100644 index 19233a7a..00000000 --- a/kios_bt_planning/backups/dynamic_bt/py_trees/trees.py +++ /dev/null @@ -1,435 +0,0 @@ -#!/usr/bin/env python -# -# License: BSD -# https://raw.githubusercontent.com/splintered-reality/py_trees/devel/LICENSE -# -############################################################################## -# Documentation -############################################################################## - -""" -While a graph of connected behaviours and composites form a tree in their own right -(i.e. it can be initialised and ticked), it is usually convenient to wrap your tree -in another class to take care of alot of the housework and provide some extra bells -and whistles that make your tree flourish. - -.. image:: images/yggdrasil.jpg - :width: 200px - :align: center - -This package provides a default reference implementation that is directly usable, but -can also be easily used as inspiration for your own tree custodians. -""" - -############################################################################## -# Imports -############################################################################## - -import functools -import os -import signal -import threading -import time -import typing - -from py_trees import behaviour -from py_trees import common -from py_trees import composites -from py_trees import visitors - -CONTINUOUS_TICK_TOCK = -1 - -############################################################################## -# Methods -############################################################################## - - -def setup(root: behaviour.Behaviour, - timeout: typing.Union[float, common.Duration]=common.Duration.INFINITE, - visitor: visitors.VisitorBase=None, - **kwargs: int): - """ - Crawls across a (sub)tree of behaviours - calling :meth:`~py_trees.behaviour.Behaviour.setup` on each behaviour. - - Visitors can optionally be provided to provide a node-by-node analysis - on the result of each node's :meth:`~py_trees.behaviour.Behaviour.setup` - before the next node's :meth:`~py_trees.behaviour.Behaviour.setup` is called. - This is useful on trees with relatively long setup times to progressively - report out on the current status of the operation. - - Args: - root: unmanaged (sub)tree root behaviour - timeout: time (s) to wait (use common.Duration.INFINITE to block indefinitely) - visitor: runnable entities on each node after it's setup - **kwargs: dictionary of arguments to distribute to all behaviours in the (sub) tree - - Raises: - Exception: be ready to catch if any of the behaviours raise an exception - RuntimeError: in case setup() times out - """ - # SIGUSR1 is a better choice since it's a user defined operation, but these - # are not available on windows, so overload one of the standard definitions - try: - _SIGNAL = signal.SIGUSR1 - except AttributeError: # windows... - # SIGINT can get you into trouble if for example, you are using a - # process manager that plays shenanigans with SIGINT. Nonetheless, - # it will work in most situations. If a windows user is running into - # problems, work with them to resolve it. - _SIGNAL = signal.SIGINT - current_behaviour_name = None - - def on_timer_timed_out(): - os.kill(os.getpid(), _SIGNAL) - - def signal_handler(unused_signum, unused_frame, original_signal_handler): - global current_behaviour_name - signal.signal(_SIGNAL, original_signal_handler) - raise RuntimeError("tree setup interrupted or timed out [{}]".format(current_behaviour_name)) - - def visited_setup(): - global current_behaviour_name - if visitor is not None: - visitor.initialise() - for node in root.iterate(): - current_behaviour_name = node.name - node.setup(**kwargs) - if visitor is not None: - node.visit(visitor) - if visitor is not None: - visitor.finalise() - - if isinstance(timeout, common.Duration): - timeout = timeout.value - if timeout == common.Duration.INFINITE.value: # math.inf - visited_setup() - else: - original_signal_handler = signal.getsignal(_SIGNAL) - signal.signal( - _SIGNAL, - functools.partial( - signal_handler, - original_signal_handler=original_signal_handler) - ) - try: - timer = threading.Timer(interval=timeout, function=on_timer_timed_out) - timer.start() - visited_setup() - finally: - timer.cancel() # this only works if the timer is still waiting - signal.signal(_SIGNAL, original_signal_handler) - -############################################################################## -# Trees -############################################################################## - - -class BehaviourTree(object): - """ - Grow, water, prune your behaviour tree with this, the default reference - implementation. It features a few enhancements to provide richer logging, - introspection and dynamic management of the tree itself: - - * Pre and post tick handlers to execute code automatically before and after a tick - * Visitor access to the parts of the tree that were traversed in a tick - * Subtree pruning and insertion operations - * Continuous tick-tock support - - .. seealso:: The :ref:`py-trees-demo-tree-stewardship-program` program demonstrates the above features. - - Args: - root (:class:`~py_trees.behaviour.Behaviour`): root node of the tree - - Attributes: - count: number of times the tree has been ticked. - root: root node of the tree - visitors: entities that visit traversed parts of the tree when it ticks - pre_tick_handlers: functions that run before the entire tree is ticked - post_tick_handlers: functions that run after the entire tree is ticked - - Raises: - TypeError: if root variable is not an instance of :class:`~py_trees.behaviour.Behaviour` - """ - def __init__(self, root: behaviour.Behaviour): - self.count: int = 0 - if not isinstance(root, behaviour.Behaviour): - raise TypeError("root node must be an instance of 'py_trees.behaviour.Behaviour' [{}]".format(type(root))) - self.root: behaviour.Behaviour = root - self.visitors: typing.List[visitors.VisitorBase] = [] - self.pre_tick_handlers: typing.List[typing.Callable[['BehaviourTree'], None]] = [] - self.post_tick_handlers: typing.List[typing.Callable[['BehaviourTree'], None]] = [] - self.interrupt_tick_tocking = False - self.tree_update_handler = None # child classes can utilise this one - - def add_pre_tick_handler(self, handler: typing.Callable[['BehaviourTree'], None]): - """ - Add a function to execute before the tree is ticked. The function must have - a single argument of type :class:`~py_trees.trees.BehaviourTree`. - - Some ideas that are often used: - - * logging (to file or stdout) - * modifications on the tree itself (e.g. starting a new plan) - - Args: - handler (:obj:`func`): function - """ - self.pre_tick_handlers.append(handler) - - def add_post_tick_handler(self, handler: typing.Callable[['BehaviourTree'], None]): - """ - Add a function to execute after the tree has ticked. The function must have - a single argument of type :class:`~py_trees.trees.BehaviourTree`. - - Some ideas that are often used: - - * logging - * modifications on the tree itself (e.g. closing down a plan) - * sending data to visualisation tools - * introspect the state of the tree to make and send reports - - Args: - handler (:obj:`func`): function - """ - self.post_tick_handlers.append(handler) - - def add_visitor(self, visitor: visitors.VisitorBase): - """ - Trees can run multiple visitors on each behaviour as they - tick through a tree. - - Args: - visitor (:class:`~py_trees.visitors.VisitorBase`): sub-classed instance of a visitor - - .. seealso:: :class:`~py_trees.visitors.DebugVisitor`, - :class:`~py_trees.visitors.SnapshotVisitor`, - :class:`~py_trees.visitors.DisplaySnapshotVisitor` - """ - self.visitors.append(visitor) - - def prune_subtree(self, unique_id): - """ - Prune a subtree given the unique id of the root of the subtree. - - Args: - unique_id (uuid.UUID): unique id of the subtree root - - Returns: - :obj:`bool`: success or failure of the operation - - Raises: - RuntimeError: if unique id is the behaviour tree's root node id - """ - # TODO: convert this to throwing exceptions instead - if self.root.id == unique_id: - raise RuntimeError("may not prune the root node") - for child in self.root.iterate(): - if child.id == unique_id: - parent = child.parent - if parent is not None: - parent.remove_child(child) - if self.tree_update_handler is not None: - self.tree_update_handler() - return True - return False - - def insert_subtree(self, child, unique_id, index): - """ - Insert a subtree as a child of the specified parent. If the parent - is found, this directly calls the parent's - :meth:`~py_trees.composites.Composite.insert_child` - method using the child and index arguments. - - Args: - child (:class:`~py_trees.behaviour.Behaviour`): subtree to insert - unique_id (uuid.UUID): unique id of the parent - index (:obj:`int`): insert the child at this index, pushing all children after it back one. - - Returns: - :obj:`bool`: suceess or failure (parent not found) of the operation - - Raises: - TypeError: if the parent is not a :class:`~py_trees.composites.Composite` - - .. todo:: - - Could use better, more informative error handling here. Especially if the insertion - has its own error handling (e.g. index out of range). Could also use a different api - that relies on the id of the sibling node it should be inserted before/after. - """ - # TODO: convert this to throwing exceptions instead - for node in self.root.iterate(): - if node.id == unique_id: - if not isinstance(node, composites.Composite): - raise TypeError("parent must be a Composite behaviour.") - node.insert_child(child, index) - if self.tree_update_handler is not None: - self.tree_update_handler() - return True - return False - - def replace_subtree(self, unique_id, subtree): - """ - Replace the subtree with the specified id for the new subtree. - This is a common pattern where we'd like to swap out a whole sub-behaviour for another one. - - Args: - unique_id (uuid.UUID): unique id of the parent - subtree (:class:`~py_trees.behaviour.Behaviour`): root behaviour of the subtree - - Raises - AssertionError: if unique id is the behaviour tree's root node id - - Returns: - :obj:`bool`: suceess or failure of the operation - """ - # TODO: convert this to throwing exceptions instead - if self.root.id == unique_id: - raise RuntimeError("may not replace the root node") - for child in self.root.iterate(): - if child.id == unique_id: - parent = child.parent - if parent is not None: - parent.replace_child(child, subtree) - if self.tree_update_handler is not None: - self.tree_update_handler() - return True - return False - - def setup(self, - timeout: typing.Union[float, common.Duration]=common.Duration.INFINITE, - visitor: visitors.VisitorBase=None, - **kwargs): - """ - Crawls across the tree calling :meth:`~py_trees.behaviour.Behaviour.setup` - on each behaviour. - - Visitors can optionally be provided to provide a node-by-node analysis - on the result of each node's :meth:`~py_trees.behaviour.Behaviour.setup` - before the next node's :meth:`~py_trees.behaviour.Behaviour.setup` is called. - This is useful on trees with relatively long setup times to progressively - report out on the current status of the operation. - - Args: - timeout (:obj:`float`): time (s) to wait (use common.Duration.INFINITE to block indefinitely) - visitor (:class:`~py_trees.visitors.VisitorBase`): runnable entities on each node after it's setup - **kwargs (:obj:`dict`): distribute arguments to this - behaviour and in turn, all of it's children - - Raises: - Exception: be ready to catch if any of the behaviours raise an exception - RuntimeError: in case setup() times out - """ - setup( - root=self.root, - timeout=timeout, - visitor=visitor, - **kwargs - ) - - def tick(self, pre_tick_handler=None, post_tick_handler=None): - """ - Tick the tree just once and run any handlers before and after the tick. - This optionally accepts some one-shot handlers (c.f. those added by - :meth:`~py_trees.trees.BehaviourTree.add_pre_tick_handler` and :meth:`~py_trees.trees.BehaviourTree.add_post_tick_handler` - which will be automatically run every time). - - The handler functions must have a single argument of type :class:`~py_trees.trees.BehaviourTree`. - - Args: - pre_tick_handler (:obj:`func`): function to execute before ticking - post_tick_handler (:obj:`func`): function to execute after ticking - """ - # pre - if pre_tick_handler is not None: - pre_tick_handler(self) - for handler in self.pre_tick_handlers: - handler(self) - for visitor in self.visitors: - visitor.initialise() - - # tick - for node in self.root.tick(): - for visitor in [visitor for visitor in self.visitors if not visitor.full]: - node.visit(visitor) - - for node in self.root.iterate(): - for visitor in [visitor for visitor in self.visitors if visitor.full]: - node.visit(visitor) - - # post - for visitor in self.visitors: - visitor.finalise() - for handler in self.post_tick_handlers: - handler(self) - if post_tick_handler is not None: - post_tick_handler(self) - self.count += 1 - - def tick_tock(self, - period_ms, - number_of_iterations=CONTINUOUS_TICK_TOCK, - pre_tick_handler=None, - post_tick_handler=None): - """ - Tick continuously with period as specified. Depending on the implementation, the - period may be more or less accurate and may drift in some cases (the default - implementation here merely assumes zero time in tick and sleeps for this duration - of time and consequently, will drift). - - This optionally accepts some handlers that will - be used for the duration of this tick tock (c.f. those added by - :meth:`~py_trees.trees.BehaviourTree.add_pre_tick_handler` and :meth:`~py_trees.trees.BehaviourTree.add_post_tick_handler` - which will be automatically run every time). - - The handler functions must have a single argument of type :class:`~py_trees.trees.BehaviourTree`. - - Args: - period_ms (:obj:`float`): sleep this much between ticks (milliseconds) - number_of_iterations (:obj:`int`): number of iterations to tick-tock - pre_tick_handler (:obj:`func`): function to execute before ticking - post_tick_handler (:obj:`func`): function to execute after ticking - """ - tick_tocks = 0 - while not self.interrupt_tick_tocking and (tick_tocks < number_of_iterations or number_of_iterations == CONTINUOUS_TICK_TOCK): - self.tick(pre_tick_handler, post_tick_handler) - try: - time.sleep(period_ms / 1000.0) - except KeyboardInterrupt: - break - tick_tocks += 1 - self.interrupt_tick_tocking = False - - def tip(self): - """ - Get the *tip* of the tree. - This corresponds to the the deepest node that was running before the - subtree traversal reversed direction and headed back to this node. - - Returns: - :class:`~py_trees.behaviour.Behaviour` or :obj:`None`: child behaviour, itself or :obj:`None` if its status is :data:`~py_trees.common.Status.INVALID` - - .. seealso:: :meth:`~py_trees.behaviour.Behaviour.tip` - """ - return self.root.tip() - - def interrupt(self): - """ - Interrupt tick-tock if it is tick-tocking. Note that this will permit a currently - executing tick to finish before interrupting the tick-tock. - """ - self.interrupt_tick_tocking = True - - def shutdown(self): - """ - Crawls across the tree calling :meth:`~py_trees.behaviour.Behaviour.shutdown` - on each behaviour. - - Raises: - Exception: be ready to catch if any of the behaviours raise an exception - """ - # TODO: this method is still quite naive .. could use similar visitors and - # timeout mechanisms as used in setup() - for node in self.root.iterate(): - node.shutdown() diff --git a/kios_bt_planning/backups/dynamic_bt/py_trees/utilities.py b/kios_bt_planning/backups/dynamic_bt/py_trees/utilities.py deleted file mode 100644 index 41d30826..00000000 --- a/kios_bt_planning/backups/dynamic_bt/py_trees/utilities.py +++ /dev/null @@ -1,170 +0,0 @@ -#!/usr/bin/env python -# -# License: BSD -# https://raw.githubusercontent.com/splintered-reality/py_trees/devel/LICENSE -# -############################################################################## -# Documentation -############################################################################## - -""" -Assorted utility functions. -""" - -############################################################################## -# Imports -############################################################################## - -import multiprocessing -import os -import re -import traceback -import typing - -############################################################################## -# Python Helpers -############################################################################## - - -def static_variables(**kwargs): - """ - This is a decorator that can be used with python methods to attach - initialised static variables to the method. - - .. code-block:: python - - @static_variables(counter=0) - def foo(): - foo.counter += 1 - print("Counter: {}".format(foo.counter)) - """ - def decorate(func): - for k in kwargs: - setattr(func, k, kwargs[k]) - return func - return decorate - - -@static_variables(primitives={bool, str, int, float}) -def is_primitive(incoming: typing.Any) -> bool: - """ - Check if an incoming argument is a primitive type with no esoteric - accessors (e.g. class attributes or container [] accessors. - - Args: - incoming: the instance to check - Returns: - True or false, depending on the check against the reserved primitives - """ - return type(incoming) in is_primitive.primitives - - -def truncate(original: str, length: int) -> str: - """ - Provide an elided version of the string for which the last three - characters are dots if the original string does not fit within the - specified length. - - Args: - original: string to elide - length: constrain the elided string to this - """ - s = (original[:length - 3] + '...') if len(original) > length else original - return s - -############################################################################## -# System Tools -############################################################################## - - -class Process(multiprocessing.Process): - def __init__(self, *args, **kwargs): - multiprocessing.Process.__init__(self, *args, **kwargs) - self._pconn, self._cconn = multiprocessing.Pipe() - self._exception = None - - def run(self): - try: - multiprocessing.Process.run(self) - self._cconn.send(None) - except Exception as e: - tb = traceback.format_exc() - self._cconn.send((e, tb)) - - @property - def exception(self): - if self._pconn.poll(): - self._exception = self._pconn.recv() - return self._exception - - -def which(program): - ''' - Wrapper around the command line 'which' program. - - Args: - program (:obj:`str`): name of the program to find. - - Returns: - :obj:`str`: path to the program or None if it doesnt exist. - ''' - def is_exe(fpath): - return os.path.isfile(fpath) and os.access(fpath, os.X_OK) - - fpath, unused_fname = os.path.split(program) - if fpath: - if is_exe(program): - return program - else: - for path in os.environ["PATH"].split(os.pathsep): - path = path.strip('"') - exe_file = os.path.join(path, program) - if is_exe(exe_file): - return exe_file - - return None - - -def get_valid_filename(s: str) -> str: - """ - Return the given string converted to a string that can be used for a clean - filename (without extension). Remove leading and trailing spaces; convert - other spaces and newlines to underscores; and remove anything that is not - an alphanumeric, dash, underscore, or dot. - - .. code-block:: python - - >>> utilities.get_valid_filename("john's portrait in 2004.jpg") - 'johns_portrait_in_2004.jpg' - - Args: - program (:obj:`str`): string to convert to a valid filename - - Returns: - :obj:`str`: a representation of the specified string as a valid filename - """ - s = str(s).strip().lower().replace(' ', '_').replace('\n', '_') - return re.sub(r'(?u)[^-\w.]', '', s) - - -def get_fully_qualified_name(instance: object) -> str: - """ - Get at the fully qualified name of an object, e.g. - an instance of a :class:`~py_trees.composites.Sequence` - becomes 'py_trees.composites.Sequence'. - - Args: - instance (:obj:`object`): an instance of any class - - Returns: - :obj:`str`: the fully qualified name - """ - module = instance.__class__.__module__ - # if there is no module, it will report builtin, get that - # string via what should remain constant, the 'str' class - # and check against that. - builtin = str.__class__.__module__ - if module is None or module == builtin: - return instance.__class__.__name__ - else: - return module + '.' + instance.__class__.__name__ diff --git a/kios_bt_planning/backups/dynamic_bt/py_trees/visitors.py b/kios_bt_planning/backups/dynamic_bt/py_trees/visitors.py deleted file mode 100644 index afa51921..00000000 --- a/kios_bt_planning/backups/dynamic_bt/py_trees/visitors.py +++ /dev/null @@ -1,193 +0,0 @@ -#!/usr/bin/env python -# -# License: BSD -# https://raw.githubusercontent.com/splintered-reality/py_trees/devel/LICENSE -# -############################################################################## -# Documentation -############################################################################## - -""" -Visitors are entities that can be passed to a tree implementation -(e.g. :class:`~py_trees.trees.BehaviourTree`) and used to either visit -each and every behaviour in the tree, or visit behaviours as the tree is -traversed in an executing tick. At each behaviour, the visitor -runs its own method on the behaviour to do as it wishes - logging, introspecting, etc. - -.. warning:: Visitors should not modify the behaviours they visit. -""" - -############################################################################## -# Imports -############################################################################## - -from py_trees import blackboard -from py_trees import display - -############################################################################## -# Visitors -############################################################################## - - -class VisitorBase(object): - """ - Parent template for visitor types. - - Visitors are primarily designed to work with :class:`~py_trees.trees.BehaviourTree` - but they can be used in the same way for other tree custodian implementations. - - Args: - full (:obj:`bool`): flag to indicate whether it should be used to visit only traversed nodes or the entire tree - - Attributes: - full (:obj:`bool`): flag to indicate whether it should be used to visit only traversed nodes or the entire tree - """ - def __init__(self, full=False): - self.full = full - - def initialise(self): - """ - Override this method if any resetting of variables needs to be - performed between ticks (i.e. visitations). - """ - pass - - def finalise(self): - """ - Override this method if any work needs to be - performed after ticks (i.e. showing data). - """ - pass - - def run(self, behaviour): - """ - This method gets run as each behaviour is ticked. Override it to - perform some activity - e.g. introspect the behaviour - to store/process logging data for visualisations. - - Args: - behaviour (:class:`~py_trees.behaviour.Behaviour`): behaviour that is ticking - """ - pass - - -class DebugVisitor(VisitorBase): - """ - Picks up and logs feedback messages and the behaviour's status. Logging is done with - the behaviour's logger. - """ - def __init__(self): - super(DebugVisitor, self).__init__(full=False) - - def run(self, behaviour): - if behaviour.logger.level < 1: - if behaviour.feedback_message: - behaviour.logger.debug("%s.run() [%s][%s]" % (self.__class__.__name__, behaviour.feedback_message, behaviour.status)) - else: - behaviour.logger.debug("%s.run() [%s]" % (self.__class__.__name__, behaviour.status)) - - -class SnapshotVisitor(VisitorBase): - """ - Visits the ticked part of a tree, checking off the status against the set of status - results recorded in the previous tick. If there has been a change, it flags it. - This is useful for determining when to trigger, e.g. logging. - - Attributes: - changed (Bool): flagged if there is a difference in the visited path or :class:`~py_trees.common.Status` of any behaviour on the path - visited (dict): dictionary of behaviour id (uuid.UUID) and status (:class:`~py_trees.common.Status`) pairs from the current tick - previously_visited (dict): dictionary of behaviour id (uuid.UUID) and status (:class:`~py_trees.common.Status`) pairs from the previous tick - running_nodes([uuid.UUID]): list of id's for behaviours which were traversed in the current tick - previously_running_nodes([uuid.UUID]): list of id's for behaviours which were traversed in the last tick - visited_blackboard_ids(typing.Set[uuid.UUID]): blackboard client id's on the visited path - visited_blackboard_keys(typing.Set[str]): blackboard variable keys on the visited path - - .. seealso:: The :ref:`py-trees-demo-logging-program` program demonstrates use of this visitor to trigger logging of a tree serialisation. - """ - def __init__(self): - super().__init__(full=False) - self.changed = False - self.visited = {} - self.previously_visited = {} - self.visited_blackboard_keys = set() - self.visited_blackboard_client_ids = set() - - def initialise(self): - """ - Switch running to previously running and then reset all other variables. This should - get called before a tree ticks. - """ - self.changed = False - self.previously_visited = self.visited - self.visited = {} - self.visited_blackboard_keys = set() - self.visited_blackboard_client_ids = set() - - def run(self, behaviour): - """ - This method gets run as each behaviour is ticked. Catch the id and status and store it. - Additionally add it to the running list if it is :data:`~py_trees.common.Status.RUNNING`. - - Args: - behaviour (:class:`~py_trees.behaviour.Behaviour`): behaviour that is ticking - """ - # behaviour status - self.visited[behaviour.id] = behaviour.status - try: - if self.visited[behaviour.id] != self.previously_visited[behaviour.id]: - self.changed = True - except KeyError: - self.changed = True - # blackboards - for blackboard in behaviour.blackboards: - self.visited_blackboard_client_ids.add(blackboard.id()) - self.visited_blackboard_keys = self.visited_blackboard_keys | blackboard.read | blackboard.write | blackboard.exclusive - - -class DisplaySnapshotVisitor(SnapshotVisitor): - """ - Visit the tree, capturing the visited path, it's changes since the last - tick and additionally print the snapshot to console. - - Args: - display_blackboard: print to the console the relevant part of the blackboard associated with behaviours on the visited path - display_activity_stream: print to the console a log of the activity on the blackboard over the last tick - """ - def __init__( - self, - display_only_visited_behaviours=False, - display_blackboard: bool=False, - display_activity_stream: bool=False - ): - super().__init__() - self.display_only_visited_behaviours = display_only_visited_behaviours - self.display_blackboard = display_blackboard - self.display_activity_stream = display_activity_stream - if self.display_activity_stream: - blackboard.Blackboard.enable_activity_stream() - - def initialise(self): - self.root = None - super().initialise() - if self.display_activity_stream: - blackboard.Blackboard.activity_stream.clear() - - def run(self, behaviour): - self.root = behaviour # last behaviour visited will always be the root - super().run(behaviour) - - def finalise(self): - print( - "\n" + - display.unicode_tree( - root=self.root, - show_only_visited=self.display_only_visited_behaviours, - show_status=False, - visited=self.visited, - previously_visited=self.previously_visited - ) - ) - if self.display_blackboard: - print(display.unicode_blackboard(key_filter=self.visited_blackboard_keys)) - if self.display_activity_stream: - print(display.unicode_blackboard_activity_stream()) diff --git a/kios_bt_planning/backups/dynamic_bt/robot_behaviors/__init__.py b/kios_bt_planning/backups/dynamic_bt/robot_behaviors/__init__.py deleted file mode 100644 index e69de29b..00000000 diff --git a/kios_bt_planning/backups/dynamic_bt/robot_behaviors/lfd_behaviors.py b/kios_bt_planning/backups/dynamic_bt/robot_behaviors/lfd_behaviors.py deleted file mode 100644 index 7dfd51b0..00000000 --- a/kios_bt_planning/backups/dynamic_bt/robot_behaviors/lfd_behaviors.py +++ /dev/null @@ -1,137 +0,0 @@ -""" -Definition of Mobile YuMi Behaviors. - -Combines YuMi behaviors with MobileYuMi behaviors. -""" - -# Copyright (c) 2022, ABB -# All rights reserved. -# -# Redistribution and use in source and binary forms, with -# or without modification, are permitted provided that -# the following conditions are met: -# -# * Redistributions of source code must retain the -# above copyright notice, this list of conditions -# and the following disclaimer. -# * Redistributions in binary form must reproduce the -# above copyright notice, this list of conditions -# and the following disclaimer in the documentation -# and/or other materials provided with the -# distribution. -# * Neither the name of ABB nor the names of its -# contributors may be used to endorse or promote -# products derived from this software without -# specific prior written permission. -# -# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" -# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE -# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE -# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE -# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL -# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR -# SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER -# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, -# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF -# THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - -from typing import Any, List, Tuple - -from behaviors.behavior_lists import BehaviorLists -from behaviors.common_behaviors import Behaviour, RandomSelector, RSequence -from bt_learning.learning_from_demo.demonstration import Demonstration -from py_trees.composites import Selector, Sequence -from robot_behaviors.hri_behaviors.hri_behaviors import HRIBehaviors -from robot_behaviors.mobile_yumi_behaviors.lfd_behaviors import MobileYuMiBehaviors -from robot_interface.interface import HRIInterface, MobileBaseInterface, YuMiInterface -import yaml - - -class LfDBehaviors(HRIBehaviors, MobileYuMiBehaviors): - """Combine YuMi behaviors with the Mobile base ones.""" - - def __init__(self, directory_path: str): - """Directory_path is the path to the directory where planner settings are saved.""" - super().__init__(directory_path) - self.behavior_list = self.get_behavior_list() - - def get_behavior_list(self) -> BehaviorLists: - """ - Return the behavior list. - - In order to avoid superclass method to load wrong behaviors for the class, - it just overrides the method (it is a copy of the same method). - """ - behavior_list = super().get_behavior_list() - return behavior_list - - def get_node( - self, - node: str, - world_interface: HRIInterface or MobileBaseInterface or YuMiInterface, - condition_parameters: Any - ) -> Tuple[Behaviour or RSequence or RandomSelector or Selector or Sequence, bool]: - """ - Link a string representation of the skill to the behavior. - - Args - ---- - string: name of the robot skill as string. - world_interface: interface to the robot hardware. - condition_parameters: pre- and post-conditions of the skill. - - Returns - ------- - node: behavior tree node, eventually inherits from py_trees - has_children: bool to determine if the node is a control node or a behavior. - - """ - has_children = False - - try: - # get nodes from HRI superclass - node, has_children = HRIBehaviors.get_node( - self, node, world_interface, condition_parameters) - except Exception: - # if it doesn't work, try with the robot instead - node, has_children = MobileYuMiBehaviors.get_node( - self, node, world_interface, condition_parameters) - - return node, has_children - - def get_actions(self, demonstrations: Demonstration) -> List[str]: - """ - Get the combined actions for YuMi and the MobileBase from a demonstration. - - Args - ---- - demonstration: the demonstration to parse. - - Returns - ------- - actions: list of the actions in the demonstration. - - """ - actions = HRIBehaviors.get_actions(self, demonstrations) +\ - MobileYuMiBehaviors.get_actions(self, demonstrations) - print(actions) - - return actions - - def get_conditions(self, demonstrations: Demonstration) -> List[str]: - """ - Get the combined conditions for YuMi and the MobileBase from a demonstration. - - Args - ---- - demonstration: the demonstration to parse. - - Returns - ------- - conditions: list of the conditions in the demonstration. - - """ - conditions = HRIBehaviors.get_conditions(self, demonstrations) +\ - MobileYuMiBehaviors.get_conditions(self, demonstrations) - - return conditions diff --git a/kios_bt_planning/backups/dynamic_bt/robot_behaviors/mobile_base_behaviors/__init__.py b/kios_bt_planning/backups/dynamic_bt/robot_behaviors/mobile_base_behaviors/__init__.py deleted file mode 100644 index e69de29b..00000000 diff --git a/kios_bt_planning/backups/dynamic_bt/robot_behaviors/mobile_base_behaviors/lfd_actions.py b/kios_bt_planning/backups/dynamic_bt/robot_behaviors/mobile_base_behaviors/lfd_actions.py deleted file mode 100644 index 594dbcbc..00000000 --- a/kios_bt_planning/backups/dynamic_bt/robot_behaviors/mobile_base_behaviors/lfd_actions.py +++ /dev/null @@ -1,101 +0,0 @@ -"""Definition of Mobile Base Actions with heuristics and pre- and post-conditions.""" - -# Copyright (c) 2022, ABB -# All rights reserved. -# -# Redistribution and use in source and binary forms, with -# or without modification, are permitted provided that -# the following conditions are met: -# -# * Redistributions of source code must retain the -# above copyright notice, this list of conditions -# and the following disclaimer. -# * Redistributions in binary form must reproduce the -# above copyright notice, this list of conditions -# and the following disclaimer in the documentation -# and/or other materials provided with the -# distribution. -# * Neither the name of ABB nor the names of its -# contributors may be used to endorse or promote -# products derived from this software without -# specific prior written permission. -# -# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" -# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE -# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE -# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE -# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL -# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR -# SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER -# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, -# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF -# THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - -from copy import deepcopy -from typing import List - -# TODO: make this a mobileyumi actions library and import from yumi -from ...bt_learning.learning_from_demo.demonstration import EquivalentAction -from ...robot_interface.demonstration import RobotAction - - -# Demonstrated Actions - - -class MoveAction(RobotAction): - """Communicates that the base frame is to be ignored for move actions.""" - - def __init__( - self, - data: dict, - all_frames: List[str], - default_frame: str, - exclude_frames: List[str], - last_action: str, - ): - super().__init__(data, all_frames, default_frame, last_action) - self.exclude_frames = exclude_frames - self.eps = 2.0 - - def heuristic(self) -> None: - """Determine what object is being carried.""" - if self.last_action is not None and len(self.last_action.parameters) > 0: - # Assign additional parameter to the action: the carried obj (the last picked obj) - self.parameters = deepcopy(self.last_action.parameters) - - def get_excluded_frames(self) -> List[str]: - """Return the frames to exclude during clustering.""" - excluded = self.exclude_frames - for param in self.parameters: - excluded.append(param) if param not in excluded else excluded - return excluded - - -# Equivalent actions - - -class EquivalentMove(EquivalentAction): - def __init__(self, *args, **kwargs): - super().__init__(*args, **kwargs) - - self.target = self.targets[0, :].reshape((3,)) - - def preconditions(self) -> List[str]: - """Return action's pre-conditions.""" - return [] - - def postconditions(self) -> List[str]: - """Return action's post-conditions.""" - return [ - str( - f"robot_at {self.target[0]} {self.target[1]} {self.target[2]}" - + f" {self.max_distance[0]} {self.actions[0].frame[0]}" - ) - ] - - def action_string(self) -> str: - """Return action's name.""" - return ( - f"{self.name} {self.target[0]} {self.target[1]} {self.target[2]}" - + f" {self.max_distance[0]} {self.actions[0].frame[0]}" - ) diff --git a/kios_bt_planning/backups/dynamic_bt/robot_behaviors/yumi_behaviors/__init__.py b/kios_bt_planning/backups/dynamic_bt/robot_behaviors/yumi_behaviors/__init__.py deleted file mode 100644 index e69de29b..00000000 diff --git a/kios_bt_planning/backups/dynamic_bt/robot_behaviors/yumi_behaviors/lfd_actions.py b/kios_bt_planning/backups/dynamic_bt/robot_behaviors/yumi_behaviors/lfd_actions.py deleted file mode 100644 index d16d653c..00000000 --- a/kios_bt_planning/backups/dynamic_bt/robot_behaviors/yumi_behaviors/lfd_actions.py +++ /dev/null @@ -1,193 +0,0 @@ -"""Definition of YuMi Actions with heuristics and pre- and post-conditions.""" - -# Copyright (c) 2022, ABB -# All rights reserved. -# -# Redistribution and use in source and binary forms, with -# or without modification, are permitted provided that -# the following conditions are met: -# -# * Redistributions of source code must retain the -# above copyright notice, this list of conditions -# and the following disclaimer. -# * Redistributions in binary form must reproduce the -# above copyright notice, this list of conditions -# and the following disclaimer in the documentation -# and/or other materials provided with the -# distribution. -# * Neither the name of ABB nor the names of its -# contributors may be used to endorse or promote -# products derived from this software without -# specific prior written permission. -# -# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" -# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE -# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE -# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE -# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL -# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR -# SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER -# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, -# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF -# THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - -from copy import deepcopy -from typing import Dict, List - -from bt_learning.learning_from_demo.demonstration import EquivalentAction -import numpy as np -from robot_interface.demonstration import RobotAction - - -# Demonstrated actions - -class PickAction(RobotAction): - """Provides heuristics for the picking action.""" - - def __init__( - self, - data: Dict, - all_frames: List[str], - default_frame: str, - exclude_frames: List[str], - last_action: str - ): - """ - Initialize the action as a Robot Action. - - Args: - ---- - - data: dictionary specifying an action. - - all_frames: list of all reference frames. - - default_frame: default frame this action takes place in. - - last_action: action performed before (to use with pre- and post-conditions). - - """ - super().__init__(data, all_frames, default_frame, last_action) - self.exclude_frames = exclude_frames - self.eps = 0.2 - - def heuristic(self) -> None: - """Determine what object is being picked and sets reference frame.""" - # Iterate over the frames and find the one where the target is closest to the origin. - # This is reasonably the object we are picking. - least_distance = np.inf - for f in self.all_frames: - # Ignore static frames - if f in self.exclude_frames: - continue - - target = self.target_position(f) - distance = np.linalg.norm(target) - if distance < least_distance: - least_distance = distance - self.frame = [f] - - # assign additional parameter to the action: what object to pick - self.parameters = deepcopy(self.frame) - - def get_excluded_frames(self) -> List[str]: - """Return the frames to exclude during clustering.""" - return self.exclude_frames - - -class PlaceAction(RobotAction): - """ - Provides heuristic for determining what object is being placed. - - Does not determine reference frame. - """ - - def __init__( - self, - data: dict, - all_frames: List[str], - default_frame: str, - exclude_frames: List[str], - last_action: str - ): - """ - Initialize the action as a Robot Action. - - Args: - ---- - - data: dictionary specifying an action. - - all_frames: list of all reference frames. - - default_frame: default frame this action takes place in. - - last_action: action performed before (to use with pre- and post-conditions). - - """ - super().__init__(data, all_frames, default_frame, last_action) - self.exclude_frames = exclude_frames - self.eps = 0.2 - - # Check if it is an actual place or drop - if self.type == 'drop': - self.place_type = 'drop' - else: - self.place_type = 'place' - - def heuristic(self) -> None: - """Determine what object is being placed.""" - if self.last_action is not None and len(self.last_action.parameters) > 0: - # assign additional parameter to the action: what object to place (the last object picked) - self.parameters = deepcopy(self.last_action.parameters) - - def get_excluded_frames(self) -> List[str]: - """Return the frames to exclude during clustering.""" - return self.parameters + self.exclude_frames - - -# Equivalent actions -class EquivalentPick(EquivalentAction): - - def preconditions(self) -> List[str]: - """Return action's pre-conditions.""" - return ['gripper_state open', f'reachable {self.actions[0].parameters[0]}'] - - def postconditions(self) -> List[str]: - """Return action's post-conditions.""" - return ['gripper_state closed', f'in_gripper {self.actions[0].parameters[0]}'] - - def action_string(self) -> str: - """Return action's name.""" - return f'{self.name} {self.actions[0].parameters[0]}' - - -class EquivalentPlace(EquivalentAction): - - def __init__(self, *args, **kwargs): - super().__init__(*args, **kwargs) - - self.object = self.actions[0].parameters[0] - self.target = self.targets[0, :].reshape((3,)) - - # If this cluster contains at least one drop, everything is drop - self.is_drop = False - for action in self.actions: - if action.place_type == 'drop': - self.is_drop = True - break - - def preconditions(self) -> List[str]: - """Return action's pre-conditions.""" - return [f'in_gripper {self.object}'] - - def postconditions(self) -> List[str]: - """Return action's post-conditions.""" - conditions = ['gripper_state open', 'in_gripper none'] - if self.is_drop: - conditions.append( - f'object_roughly_at {self.object} {self.target[0]} {self.target[1]}' + - f' {self.target[2]} {self.max_distance[0]} {self.actions[0].frame[0]}') - else: - conditions.append( - f'object_roughly_at {self.object} {self.target[0]} {self.target[1]}' + - f' {self.target[2]} {self.max_distance[0]} {self.actions[0].frame[0]}') - - return conditions - - def action_string(self) -> str: - """Return action's name.""" - return (f'{self.name} {self.object} {self.target[0]} {self.target[1]} {self.target[2]}' + - f' {self.max_distance[0]} {self.actions[0].frame[0]}') diff --git a/kios_bt_planning/backups/dynamic_bt/robot_interface/__init__.py b/kios_bt_planning/backups/dynamic_bt/robot_interface/__init__.py deleted file mode 100644 index e69de29b..00000000 diff --git a/kios_bt_planning/backups/dynamic_bt/robot_interface/demonstration.py b/kios_bt_planning/backups/dynamic_bt/robot_interface/demonstration.py deleted file mode 100644 index 0f89d2c8..00000000 --- a/kios_bt_planning/backups/dynamic_bt/robot_interface/demonstration.py +++ /dev/null @@ -1,210 +0,0 @@ -"""Realization of Demonstration and Action from Robot perspective.""" - -# Copyright (c) 2022, ABB -# All rights reserved. -# -# Redistribution and use in source and binary forms, with -# or without modification, are permitted provided that -# the following conditions are met: -# -# * Redistributions of source code must retain the -# above copyright notice, this list of conditions -# and the following disclaimer. -# * Redistributions in binary form must reproduce the -# above copyright notice, this list of conditions -# and the following disclaimer in the documentation -# and/or other materials provided with the -# distribution. -# * Neither the name of ABB nor the names of its -# contributors may be used to endorse or promote -# products derived from this software without -# specific prior written permission. -# -# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" -# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE -# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE -# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE -# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL -# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR -# SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER -# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, -# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF -# THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - -from copy import deepcopy -import glob -import itertools -import os -from typing import List - -from ..bt_learning.learning_from_demo.demonstration import Action, Demonstration -import numpy as np -import yaml - - -class RobotDemonstration(Demonstration): - """This class represents a collection of demonstrations as recorded from the robot.""" - - def __init__(self, demo: str, custom_actions: dict = {}, exclude_frames: dict = {}): - """ - Construct a set of demonstrations from the directory pointed to by demo. - - Args: - ---- - demo: path to the folder where the demonstration is stored. - custom_actions: dictionary with string-class pairs indicating cusom classes, - that can be used for different action types. - exclude_frames: fixed frames to be excluded. - - Note: All classes must be a subclass of RobotAction. - frames: list of all reference frames. - default_frame: frame that all actions are assigned to by default. - - """ - super().__init__(RobotAction) - - for action_type, cls in custom_actions.items(): - if not issubclass(cls, RobotAction): - raise ValueError( - f'Class "{cls}" of action "{action_type}" should be a subclass of RobotAction.' - ) - self.register_action(action_type, cls) - - # Folder with demonstrations must exist - if not os.path.isdir(demo): - raise FileNotFoundError('Folder "' + demo + '" does not exist.') - # Demonstrations folder has to contain information file - if not os.path.isfile(demo + '/info.yaml'): - raise FileNotFoundError('Folder "' + demo + '/info.yaml" does not exist.') - # There has to be at least one demonstration sequence - if not os.path.isdir(demo + '/demo1'): - raise FileNotFoundError( - 'Folder "' + demo + '" must contain at least one demonstration.') - - # Read information about the demonstratoins - with open(demo + '/info.yaml') as f: - info = yaml.safe_load(f) - self.frames = info['frames'] - self.default_frame = info['default_frame'] - - # self.__demonstrations contains a list of actions for each demonstration - self.__demonstrations = [] - for folder in glob.glob(demo + '/demo[0-9]*/'): - # Each demonstration has to contain at least one action - if not os.path.isfile(folder + '/data_1.yaml'): - raise FileNotFoundError( - 'Demonstration "' + folder + '/" must contain at least one action.') - actions = [] - # Read actions in numerical order - n_actions = len(glob.glob(folder + '/*.yaml')) - for action_number in range(1, n_actions + 1): - file = f'{folder}/data_{action_number}.yaml' - with open(file) as f: - data = yaml.safe_load(f) - if len(actions) == 0: - last_action = None - else: - last_action = actions[-1] - action = self.make_action( - data['type'], - data, - info['frames'], - self.default_frame, - exclude_frames[data['type']] if exclude_frames != {} else [], - last_action - ) - action.heuristic() - actions.append(action) - self.__demonstrations.append(actions) - - with open(demo + '/exclude_frames.yaml', 'w') as f: - yaml.dump(exclude_frames, f) - - def demonstrations(self) -> List[str]: - """Get the list of the actions performed in the demosntration.""" - return self.__demonstrations - - -class RobotAction(Action): - """Class that represents a single action as demonstrated.""" - - def __init__( - self, - data: dict, - all_frames: List[str], - default_frame: str, - last_action: str = None, - n_targets: int = 1 - ): - """ - Initialize a robot Action. - - Action data is specified according to the documentation. - - Args: - ---- - - data: dictionary specifying an action. - - all_frames: list of all reference frames. - - default_frame: default frame this action takes place in. - - last_action: action performed before (to use with pre- and post-conditions). - - n_targets: number of targets this action has. - - """ - super().__init__( - data['type'], - list(itertools.repeat(default_frame, n_targets)) - ) - - self.all_frames = all_frames - self.position = {} - self.orientation = {} - self.last_action = last_action - for frame in data['vec_pos']: - self.position[frame] = np.array(data['vec_pos'][frame]).reshape((-1, 3)) - for frame in data['vec_quat']: - self.orientation[frame] = np.array(data['vec_quat'][frame]).reshape((-1, 4)) - - def target_position(self, frame: str, i: int = 0) -> np.ndarray: - """ - Return the target position i of this action. - - Raises a KeyError if no position exists in frame. - This class assumes there is only one target (i=0 is the first target). - If an action has multiple targets, a subclass must be written. - - Args - ---- - frame: frame with respect to the position is defined. - i: index of the target. - - Returns - ------- - NumPy array with the position values [x, y, z]. - - """ - if i != 0: - raise ValueError(f'Action "{self.type}" has no target "{i}."') - - return self.position[frame][-1, :].reshape((3,)) - - def target_orientation(self, frame: str, i: int = 0) -> np.ndarray: - """ - Return the target orientation of this action. - - Raises a KeyError if no orientation exists in frame. - This class assumes there is only one target. - If an action has multiple targets, a subclass must be written. - - Args - ---- - frame: frame with respect to the orientation is defined. - i: index of the target. - - Returns - ------- - NumPy array with the orientation values [x, y, z, w] - - """ - if i != 0: - raise ValueError(f'Action "{self.type}" has no target "{i}."') - return self.orientation[frame][-1] diff --git a/kios_bt_planning/backups/dynamic_bt/simulation/__init__.py b/kios_bt_planning/backups/dynamic_bt/simulation/__init__.py deleted file mode 100644 index e69de29b..00000000 diff --git a/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/README.md b/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/README.md deleted file mode 100644 index dcd867f4..00000000 --- a/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/README.md +++ /dev/null @@ -1,39 +0,0 @@ -# Robotics Digital Lab - -This package contains a `python` implementation of a simulation environment based on Algoryx. - -At the current state only the lowes fidelity level of the simulation is implemented. It features a gripper that teleports above the items to pick without simulating friction or contacts. There is no slippage. - - -## Installation - -1. [Install AGX Dynamics 2.34.0.2](https://www.algoryx.se/documentation/complete/agx/tags/latest/UserManual/source/installation.html#) for your platform. See your license email for download instructions. -2. Install the `agxBrick` and `agxClick` python packages using `pip install -r requirements.txt` - * Some bug with `--extra-index-url` in `requirements.txt` might mean that you need to install them separate - * `pip install --extra-index-url https://agx-access:UymNbuiLHJ13CKGnmsoD@git.algoryx.se/api/v4/projects/270/packages/pypi/simple agxBrick==0.5.16` - * `pip install --extra-index-url https://click-access:F2q7LauW_d-HJ7bH37sV@git.algoryx.se/api/v4/projects/262/packages/pypi/simple agxClick==0.1.34` -3. Install OpenGL dependency with `pip install PyOpenGL-accelerate` -4. Install dotnet dependencies: -``` -wget -q https://packages.microsoft.com/config/ubuntu/20.04/packages-microsoft-prod.deb -O packages-microsoft-prod.deb -dpkg -i packages-microsoft-prod.deb -add-apt-repository universe -apt-get install apt-transport-https -apt-get update -apt-get install -qy dotnet-sdk-3.1 dotnet-sdk-5.0 dotnet-sdk-6.0 -``` -5. Add these lines to the `.bashrc` file, where version is e.g 2.34.0.1: - * `source /opt/Algoryx/AGX-version/setup_env.bash` - * `export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:"/opt/Algoryx/AGX-version/lib"` - - -## Run the BT Demo -To test the installation you need to source the AGX environment. See the user manual for instruction for your platform. - -You can run the BT Demo by running the demo script. -``` -python run_demo.py --model models/BTDemo.yml:CoarseGripperInLabDR --decorate --timeStep 0.05 -``` - -Note that this command works only from the folder where the `run_demo.py` script is stored. - diff --git a/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/__init__.py b/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/__init__.py deleted file mode 100644 index 8b137891..00000000 --- a/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/__init__.py +++ /dev/null @@ -1 +0,0 @@ - diff --git a/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/agx_application.py b/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/agx_application.py deleted file mode 100644 index 8b209813..00000000 --- a/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/agx_application.py +++ /dev/null @@ -1,201 +0,0 @@ -"""Application wrapper to handle interaction with AGX.""" - -# Copyright (c) 2022, ABB -# All rights reserved. -# -# Redistribution and use in source and binary forms, with -# or without modification, are permitted provided that -# the following conditions are met: -# -# * Redistributions of source code must retain the -# above copyright notice, this list of conditions -# and the following disclaimer. -# * Redistributions in binary form must reproduce the -# above copyright notice, this list of conditions -# and the following disclaimer in the documentation -# and/or other materials provided with the -# distribution. -# * Neither the name of ABB nor the names of its -# contributors may be used to endorse or promote -# products derived from this software without -# specific prior written permission. -# -# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" -# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE -# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE -# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE -# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL -# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR -# SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER -# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, -# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF -# THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - -import os -import time -import cloudpickle -import logging -import multiprocessing as mp -from typing import Any, Dict, List - -from agxClick import AgxApplication -import yaml -from bt_learning.learning_from_demo.debug import BTVisualizer -from simulation.algoryx import agx_environment -from simulation.py_trees_interface import PyTree, PyTreeParameters - -GRAPHICS = True -DUMP = False - - -class CloudpickleWrapper: - """ - Uses cloudpickle to serialize contents (otherwise multiprocessing tries to use pickle) - - :param var: the variable you wish to wrap for pickling with cloudpickle - """ - - def __init__(self, var: Any): - self.var = var - - def __getstate__(self) -> Any: - return cloudpickle.dumps(self.var) - - def __setstate__(self, var: Any) -> None: - self.var = cloudpickle.loads(var) - - -class Application(AgxApplication): - def __init__(self, args: Any) -> None: - if GRAPHICS: - super().__init__(args) - else: - super().__init__(['-a']) - self._logger = logging.getLogger(__file__) - self._env = None - self.args = args - - def bringup( - self, - data: Dict, - target: Dict = None, - visual: bool = False, - debug: bool = False - ) -> None: - """Initialize the Environment.""" - self.__init_agx_environment(data, target, visual, debug) - if DUMP: - print('Dumping AGX state') - self._env.sim.write('bringup.agx') - self.__step_ahead(1) - - def reset(self, target: Dict = None) -> None: - """Reset the simulation.""" - self._env.reset_simulation() - if target is not None: - self._env.apply_configuration(target) - self.__step_ahead(1) - - def shutdown(self) -> None: - """Shut down the Environment.""" - self._env.shutdown() - del self._env - self._env = None - self.stop() - print('Environment DOWN!') - - def step(self, time: float): - self.__step_ahead(time) - - def start_video(self): - pass - - def stop_video(self): - pass - - def __init_agx_environment( - self, - data: Dict, - target: Dict = None, - visual: bool = False, - debug: bool = False - ) -> None: - """Initialize AGX Environment.""" - env = agx_environment.AGXEnvironment(self.args, objects=data, debug=debug) - - brickSimulation = env.build_scene() - env.spawn(brickSimulation) - if target is not None: - env.apply_configuration(target) - if visual: - env.create_visual() - self._env = env - - def __step_ahead(self, time: float): - """Step forward in time.""" - t1 = self._env.sim.getTimeStamp() - while self._env.sim.getTimeStamp() - t1 < time: - self.stepApplication() - - -class RunBT(mp.Process): - def __init__( - self, - interface: Any, - data: Dict, - behaviors: Any, - bt: str or List[str], - tick_freq: int, - args - ): - - self.interface = interface - self.data = data - self.behaviors = behaviors - self.bt = bt - self.bt_tick_freq = tick_freq - - self.event = mp.Event() - - super().__init__(target=self.worker, args=args) - - def stop(self): - self.event.set() - self.join() - - def worker(self, env_fn_wrapper: CloudpickleWrapper, args: List[str]): - app = env_fn_wrapper.var(args) - app.bringup(self.data, visual=True) - self.interface.initialize() - - if type(self.bt) is str: - with open(os.path.join(self.bt, 'tree.yaml')) as f: - string_tree = yaml.safe_load(f) - else: - string_tree = self.bt - - parameters = PyTreeParameters() - parameters.behavior_lists = self.behaviors.get_behavior_list() - parameters.behaviors = self.behaviors - parameters.max_ticks = 15 - parameters.max_time = 30 - bt = PyTree( - string_tree, parameters, world_interface=self.interface) - - viz = BTVisualizer(bt) - - self.interface.remove_locks() - self.interface.preempt_skill() - self.interface.add_merger() - - self.interface.set_gripper('open') - - print('Launching tree...') - for _ in range(25): - if self.event.is_set(): - app.shutdown() - break - viz.tick() - current_time = app.app.getTimeStamp() - while app.app.getTimeStamp() < 2.5 + current_time: - app.stepApplication() diff --git a/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/agx_environment.py b/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/agx_environment.py deleted file mode 100644 index a4cb082a..00000000 --- a/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/agx_environment.py +++ /dev/null @@ -1,307 +0,0 @@ -"""Define the objects in the AGX simulaiton.""" - -# Copyright (c) 2022, ABB -# All rights reserved. -# -# Redistribution and use in source and binary forms, with -# or without modification, are permitted provided that -# the following conditions are met: -# -# * Redistributions of source code must retain the -# above copyright notice, this list of conditions -# and the following disclaimer. -# * Redistributions in binary form must reproduce the -# above copyright notice, this list of conditions -# and the following disclaimer in the documentation -# and/or other materials provided with the -# distribution. -# * Neither the name of ABB nor the names of its -# contributors may be used to endorse or promote -# products derived from this software without -# specific prior written permission. -# -# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" -# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE -# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE -# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE -# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL -# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR -# SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER -# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, -# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF -# THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - -from copy import copy -from typing import Any, Dict, List - -# AGX API -import agx -from agxBrick.brickLoaderUtils import extractModelFilePathAndName, setup_camera_and_lights -import agxOSG -from agxPythonModules.utils.environment import simulation, root, application -import numpy as np -import random - - -class AGXEnvironment(): - - def __init__(self, args: Any = None, objects: Dict = None, debug: bool = False) -> None: - self.sim = None - - self.debug = debug - - if args is not None: - self.args = args - if self.args.model == '': - print('No brick model defined!') - return - - # Load the brick model - self.file_path, self.model_name = extractModelFilePathAndName(self.args) - - self.brickSimulation = None - self.scene = None - - # parse config file where the user defines the items - # objects is a dicitonary where key is the obj type and value is a list of objs of that type - # every obj has a name and an reference location for the spawning - ex_objects = { - 'Box1': {'type': 'box', 'reference': 'Table1', 'size': 0.1, 'color': 'yellow'}, - 'Box2': {'type': 'box', 'reference': 'Table2', 'size': 0.1, 'color': 'green'}, - 'Box3': {'type': 'box', 'reference': 'Table1', 'size': 0.1, 'color': 'blue'}, - } - - # Reference objects in the scene that do not move - self.references = { - 'Table1': {'type': 'table', 'origin': [-1, 0, 0.375], 'size': [0.75, 0.75, 0.75]}, - 'Table2': {'type': 'table', 'origin': [1, 0, 0.375], 'size': [0.75, 0.75, 0.75]}, - 'Table3': {'type': 'table', 'origin': [0, 0, 0.375], 'size': [0.75, 0.75, 0.75]}, - } - - self.init_objects = ex_objects if objects is None else objects - # Keep track of the objects - self.objects = copy(self.init_objects) - - def get_objects(self) -> Dict: - """Return the objects in the scene.""" - return self.objects - - def get_bodies(self) -> List[str]: - """Return the names of the RigidBodies in the scene.""" - bodies = self.sim.getRigidBodies() - names = [] - for body in bodies: - names.append(str(body.getName())) - return names - - def get_brickSimulation(self) -> Any: - """Return the BrickSimulation object.""" - return self.brickSimulation - - def get_scene(self) -> Any: - """Return the scene object.""" - return self.scene - - def shutdown(self): - """Set internal variables to None.""" - self.sim.cleanup() - self.sim.clearContactData() - del self.sim - del self.brickSimulation - del self.scene - self.sim = None - self.brickSimulation = None - self.scene = None - print('Simulation DOWN!') - - def randomize_spawn(self, target: str, reference: str) -> np.ndarray: - """Randomize the initial XYZ pose of the target object with respect to the reference.""" - candidate_pose = self._sample_pose( - self.references[reference]['size'][0] - 3*self.objects[target]['size'], - self.references[reference]['size'][1] - 3*self.objects[target]['size'], - self.references[reference]['origin'][0:2] - ) - colliding = self._is_colliding(target, reference, candidate_pose) - - attempts = 200 - while colliding and attempts > 0: - # Sample a new pose - candidate_pose = self._sample_pose( - self.references[reference]['size'][0] - 3*self.objects[target]['size'], - self.references[reference]['size'][1] - 3*self.objects[target]['size'], - self.references[reference]['origin'][0:2] - ) - # Check again for collision - colliding = self._is_colliding(target, reference, candidate_pose) - attempts -= 1 - - z_val = (2*self.references[reference]['size'][2] + self.objects[target]['size'])/2 - pose = np.append(candidate_pose, z_val) - return pose - - def spawn_kitbox(self, reference: str) -> np.ndarray: - """Randomize the initial XYZ pose of the target object with respect to the reference.""" - candidate_pose = self.references[reference]['origin'][0:2] - - kitbox = self.sim.getRigidBody('KittingBox') - ref_size = round( - kitbox.getGeometries()[0].getShape().asBox().getHalfExtents()[0], 3)*2 - - z_val = (2*self.references[reference]['size'][2] + ref_size)/2 - pose = np.append(candidate_pose, z_val) - return pose - - def spawn(self, brickSimulation) -> None: - """Spawn the objects randomically in the environment, given their origin.""" - from Brick.Math import Vec3 - from Brick.Physics import Component - - if brickSimulation is None: - raise EnvironmentError( - 'No Brick Simulation found. Remember to build the AGX Scene first.') - - for obj in self.objects: - # print(f'\nAdding {obj}') - box_component = Component.CreateFromFile( - self.file_path, self.objects[obj]['color'].capitalize() + 'BoxComponent') - box_component.Bodies[0].Name = obj - pose = self.randomize_spawn(obj, self.objects[obj]['reference']) - # print(f'With pose {pose}') - box_component.Bodies[0].LocalTransform.Position = Vec3( - float(pose[0]), float(pose[1]), float(pose[2])) - - # Add the component to the simulation - brickSimulation.AddComponent(box_component) - - def respawn(self) -> None: - """Respawn the objects in the scene.""" - # respawn KittingBox - body = self.sim.getRigidBody('KittingBox') - table = random.choice(list(self.references.keys())) - remaining_tables = list(self.references.keys()) - remaining_tables.remove(table) - pose = self.spawn_kitbox(table) - # print(f'With pose {pose}') - body.setPosition(agx.Vec3(float(pose[0]), float(pose[1]), float(pose[2]))) - - # respawn boxes - for obj in self.objects: - # print(f'\nRespawning {obj}') - box_obj = self.sim.getRigidBody(obj) - # Choose a reference object at random among the references - reference = random.choice(remaining_tables) - self.objects[obj]['reference'] = reference - # Spawn the box in the reference - pose = self.randomize_spawn(obj, reference) - # print(f'With pose {pose}') - box_obj.setPosition(agx.Vec3(float(pose[0]), float(pose[1]), float(pose[2]))) - - def apply_configuration(self, target: Dict) -> None: - """Move objects in the given configuration.""" - for obj in target: - try: - box_obj = self.sim.getRigidBody(obj) - except AttributeError: - print('The taget item does not exist!') - try: - reference = target[obj]['reference'] - # if the item is spwan in a table, then ignore it to allow randomization - if reference.startswith('Table'): - continue - pose = target[obj]['pose'] - except AttributeError: - print('Target dictionary with wrong format!') - - ref_obj = self.sim.getRigidBody(reference) - ref_pose = [ - ref_obj.getPosition().x(), ref_obj.getPosition().y(), ref_obj.getPosition().z()] - # print(f'{reference} with pose {ref_pose}') - - # print( - # f'{obj} with pose {[pose[0] + ref_pose[0], pose[1] + ref_pose[1], pose[2] + ref_pose[2]]}') - box_obj.setPosition(agx.Vec3( - float(pose[0] + ref_pose[0]), - float(pose[1] + ref_pose[1]), - float(pose[2] + ref_pose[2]) - )) - - def build_scene(self) -> Any: - """Build the AGX scene with the given arguments.""" - # With the current Brick modules it is necessary to import the API inside the function - from Brick.AGXBrick import BrickSimulation - from Brick.Physics import Component - - self.sim = simulation() - self.sim.cleanup() - self.sim.clearContactData() - - self.scene = Component.CreateFromFile(self.file_path, self.model_name) - if self.scene is None: - raise Exception(f'Failed to load scene') - self.brickSimulation = BrickSimulation.Default - self.brickSimulation.AddComponent(self.scene) - - if self.debug: - application().setEnableDebugRenderer(True) - application().setEnableOSGRenderer(False) - - if self.args.decorate: - setup_camera_and_lights( - application(), - self.sim, - root(), - self.brickSimulation, - self.scene, - False - ) - - return self.brickSimulation - - def reset_simulation(self) -> None: - """Reset the Brick Simulation.""" - from Brick.Physics import ComponentLoader - - self.scene.UpdateParameterSpace() - ComponentLoader.RepositionComponent(self.scene) - - self.brickSimulation.ResetAgx() - self.brickSimulation.SyncOutputParameters() - - self.respawn() - - def create_visual(self) -> None: - """Enable the visualization of the environment.""" - agxOSG.createVisual(self.sim, application().getSceneRoot()) - - def _sample_pose(self, width: float, length: float, origin: np.ndarray) -> np.ndarray: - """Sample the pose from a uniform distribution.""" - x = round(random.uniform(-width/2 + 0.05, width/2 - 0.05), 3) - y = round(random.uniform(-length/2 + 0.05, length/2 - 0.05), 3) - return np.add(np.array([x, y]), origin) - - def _is_colliding( - self, - target: str, - reference: str, - target_pose: np.ndarray - ) -> bool: - """Check if the target pose is colliding with another element.""" - colliding = False - # Check for rejection sampling only the objects that share the same origin - if target not in self.objects: - return False - for obj in self.objects: - # Get object from simulation - try: - body: agx.RigidBody = self.sim.getRigidBody(obj) - body_pose = np.array([body.getPosition().x(), body.getPosition().y()]) - except AttributeError: - # In case the object doesn't exist yet, skip - continue - - if obj == target or self.objects[obj]['reference'] != reference: - continue - else: - if np.linalg.norm(target_pose - body_pose) < 0.15: - colliding = True - return colliding diff --git a/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/behaviors/__init__.py b/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/behaviors/__init__.py deleted file mode 100644 index 8b137891..00000000 --- a/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/behaviors/__init__.py +++ /dev/null @@ -1 +0,0 @@ - diff --git a/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/behaviors/agx_interface.py b/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/behaviors/agx_interface.py deleted file mode 100644 index 9374e3ee..00000000 --- a/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/behaviors/agx_interface.py +++ /dev/null @@ -1,421 +0,0 @@ -"""Definition of the low level behaviors for the AGX simulation.""" - -# Copyright (c) 2022, ABB -# All rights reserved. -# -# Redistribution and use in source and binary forms, with -# or without modification, are permitted provided that -# the following conditions are met: -# -# * Redistributions of source code must retain the -# above copyright notice, this list of conditions -# and the following disclaimer. -# * Redistributions in binary form must reproduce the -# above copyright notice, this list of conditions -# and the following disclaimer in the documentation -# and/or other materials provided with the -# distribution. -# * Neither the name of ABB nor the names of its -# contributors may be used to endorse or promote -# products derived from this software without -# specific prior written permission. -# -# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" -# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE -# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE -# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE -# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL -# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR -# SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER -# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, -# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF -# THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - -# TODO: -# search: -# - get the tables position from the sim and move the gripper closed to the tables -# - if there is a box in the neighborhood of the gripper then update position of the box in the robot's knowledge -# move: -# - move the gripper to the input pose/obj -# pick: -# - combine pre-grasp, grasp, and post-grasp motion into a single action that takes as input a pose/obj -# place: -# - combine pre-place, place, and post-place motion into a single action that takes as input a pose/obj -# recharge: -# - move the robot to a fix pose where it recharges the batteries -# localize: -# - make the robot bleief of its pose equal to the pose of the model -# tuck: -# - move the robot arm to a fix position in the robot ref frame - -# these skills should have a definition compatible with the lfd_behaviors in abb_robot_robot_behaviors !! -# so AGXInterface should have methods and attributes as the OnlineYuMiInterface and the OfflineInterface - -from copy import copy, deepcopy -import math -from typing import Any, List, Tuple - -import agx -from agxPythonModules.utils.callbacks import StepEventCallback as sec -from agxPythonModules.utils.environment import simulation -import numpy as np - - -class AGXInterface(): - """An interface for execution of YuMi behaviors in the AGX simulator.""" - - def __init__(self, task_objects: List[str]) -> None: - """Instantiate a new AGX Interface.""" - self.sim = simulation() - self.task_objects = task_objects - - # Get things from the simulation that is setup in the Brick model. - # End Effecor - self.gripper_ctrl = self.sim.getRigidBody('GripperControlBody') - self.gripper_obj = self.sim.getRigidBody('gripper') - self.gripper_offset = 0.12 - # Robot - self.robot = self.sim.getRigidBody('RobotControlBody') - # EE-Robot lock for control - self.gripper_lock = self.sim.getConstraint('controlGripperLock').asLockJoint() - - # Object to merge and split bodies: - self.with_merger = True - self.merger_name = 'Merger' - - if not self.with_merger: - self.item_locks = {} - self.__create_constraints(task_objects) - - # Gripper states - self.holding = '' - self._gripper_state = 'closed' - - # Flags - self.unstacked = False - - # Reachability Condition - self.workspace = self.workspace_limit() - self.max_z = deepcopy(round(self.gripper_ctrl.getPosition().z(), 3)) - - def preempt_skill(self) -> None: - """Preempt all planned sequences.""" - sec.instance()._clear() - - def add_merger(self) -> None: - merger = agx.MergedBody() - merger.setName(self.merger_name) - self.sim.add(merger) - - def remove_locks(self) -> None: - """Remove locks and merged items.""" - self.holding = '' - for obj in self.task_objects: - self._release(obj) - merger = self.sim.getMergedBody(self.merger_name) - if merger is not None: - success = self.sim.remove(merger) - - def get_feedback(self) -> bool: - """Dummy function to fit world interface template.""" - return (self.gripper_ctrl is not None) or (self.robot is not None) - - def send_references(self) -> None: - """Dummy function to fit world interface template.""" - pass - - def workspace_limit(self) -> Tuple[float, float]: - """Define the limit of the workspace given the two external tables.""" - table_1 = self.sim.getRigidBody('Table1') - table_2 = self.sim.getRigidBody('Table2') - table_size = round( - table_1.getGeometries()[0].getShape().asBox().getHalfExtents()[0], 3)*2 - - delta_x = round(table_2.getPosition().x() - table_1.getPosition().x(), 3) + table_size - delta_y = table_size - return delta_x, delta_y - - # SKILLS - def pick(self, target: str) -> None: - """Pick the target item.""" - target_obj = self.sim.getRigidBody(target) - if target_obj is None: - raise ValueError(f'Target item {target} does not exist!') - # Time Stamps - t0 = self.sim.getTimeStamp() + 1.0 - t1 = t0 + 0.5 - t2 = t1 + 0.1 - t3 = t2 + 0.5 - # Move gripper to pre-grasp position - size = round( - target_obj.getGeometries()[0].getShape().asBox().getHalfExtents()[0], 3)*2 - sec.callAt(t0, lambda: self._move_gripper( - target_obj.getPosition() + agx.Vec3(0, 0, 2*size))) - sec.callAt(t1, lambda: self._move_gripper( - target_obj.getPosition() + agx.Vec3(0, 0, 0.7*size))) - # Grasp - sec.callAt(t2, lambda: self._grasp(target)) - # Move gripper to post-grasp position - sec.callAt(t3, lambda: self._move_gripper( - self.gripper_ctrl.getPosition() + agx.Vec3(0, 0, 1.5*size))) - - def place(self, target: str, reference: str, pose: agx.Vec3 or np.ndarray) -> None: - """Place the target item in a posed espressed in the desired reference frame.""" - # Preparations - target_obj = self.sim.getRigidBody(target) - if target_obj is None: - raise ValueError(f'Target item {target} does not exist!') - target_size = round( - target_obj.getGeometries()[0].getShape().asBox().getHalfExtents()[0], 3)*2 - ref_obj = self.sim.getRigidBody(reference) - if ref_obj is None: - raise ValueError(f'Reference item {reference} does not exist!') - ref_size = round( - target_obj.getGeometries()[0].getShape().asBox().getHalfExtents()[0], 3)*2 - - pose = agx.Vec3(pose[0], pose[1], pose[2]) if type(pose) is np.ndarray else pose - - # Time Stamps - t0 = self.sim.getTimeStamp() + 1.0 - t1 = t0 + 0.5 - t2 = t1 + 0.1 - t3 = t2 + 0.5 - # Move gripper to pre-place position - sec.callAt(t0, lambda: self._move_gripper( - self.__round(ref_obj.getPosition() + pose + agx.Vec3(0, 0, 2*ref_size)))) - sec.callAt(t1, lambda: self._move_gripper(self.__round(ref_obj.getPosition() + pose))) - # Release - sec.callAt(t2, lambda: self._release(target)) - # Move gripper to post-release position - sec.callAt(t3, lambda: self._move_gripper( - self.__round(target_obj.getPosition() + agx.Vec3(0, 0, 3*target_size)))) - - def set_gripper(self, state: str) -> None: - """Open and closed the gripper.""" - if state == 'open': - self._gripper_state = state - self.holding = '' - for obj in self.task_objects: - self._release(obj) - elif state == 'closed': - self._gripper_state = state - else: - raise ValueError('Gripper state not valid!') - - if state == 'open' and self.holding != '': - # The robot is holding something that will be dropped - self._release(self.holding) - - def lift_gripper(self) -> None: - """Move the gripper along the Z azis.""" - t0 = self.sim.getTimeStamp() + 1.0 - sec.callAt(t0, lambda: self._move_gripper( - self.gripper_ctrl.getPosition() + agx.Vec3(0.0, 0.0, 0.3))) - - def tuck_gripper(self) -> None: - """Tuck the robot arm.""" - if self.robot is None: - raise ValueError('Robot not found!') - - tuck_position = self.robot.getFrame().transformPointToLocal(self.gripper_ctrl.getPosition()) - self.gripper_ctrl.setPosition(self.robot.getFrame().transformPointToWorld(tuck_position)) - - def move_robot(self, pos: agx.Vec3 or np.ndarray or str) -> None: - """Move the robot at the desired position.""" - if self.robot is None: - raise ValueError('Robot not found!') - - self.gripper_lock.setEnable(False) - position = agx.Vec3(pos[0], pos[1], pos[2]) if type(pos) is np.ndarray else pos - self.robot.setPosition(position) - - # TODO: implement it similarly to place, i.e. with a reference object and a pose - - def approach(self) -> None: - pass - - def navigate(self) -> None: - pass - - # CHECKS - def grasped(self, target: str) -> bool: - """Return if the target item is grasped or not.""" - if target is None or target == 'none': - return self.holding == '' - elif self.with_merger: - return False if agx.MergedBody.get(self.sim.getRigidBody(target)) is None else True - else: - return self.item_locks[target].getEnable() - - def empty_gripper(self) -> bool: - """Return true if the gripper is open.""" - return self.holding == '' - - def is_gripper_state(self, state: str) -> bool: - """Return true gripper is open.""" - if state == 'open' or state == 'closed': - return self._gripper_state == state - else: - raise ValueError('Gripper state not valid!') - - def get_item_in_frame(self, target: str, reference: str) -> Tuple[np.ndarray, np.ndarray]: - """Return if the target item is at the desired pose in the frame of the reference item.""" - w_T_t = self.sim.getRigidBody(target).getTransform() - w_T_r = self.sim.getRigidBody(reference).getTransform() - r_T_w = w_T_r.inverse() - r_T_t = r_T_w*w_T_t - position = r_T_t.getTranslate() - np_position = self.__as_array(position) - rotation = r_T_t.getRotate() - np_rotation = self.__as_array(rotation) - - return np_position.round(3), np_rotation.round(3) - - def items_unstacked(self, items: List[str]) -> bool: - """Check that the items are placed on the table and not over some other object.""" - table = self.sim.getRigidBody('Table1') - table_height = round( - table.getGeometries()[0].getShape().asBox().getHalfExtents()[0], 3)*2 - - unstacked = True - for item in items: - item_obj = self.sim.getRigidBody(item) - item_size = round( - item_obj.getGeometries()[0].getShape().asBox().getHalfExtents()[0], 3)*2 - item_height = round(item_obj.getPosition().z(), 3) - if item_height > table_height + item_size: - unstacked = False - - return unstacked - - def at_pose( - self, - target: str, - reference: str, - pose: agx.Vec3 or np.ndarray, - tolerance: float, - rough: bool = False - ) -> bool: - """Return if the target item is at the desired pose in the frame of the reference item.""" - position, _ = self.get_item_in_frame(target, reference) - # print(f'{target} at {position} in {reference} frame.') - real = self.as_vec3(position) - target = pose if type(pose) is agx.Vec3 else self.as_vec3(pose) - corrected = self.__round(agx.Vec3(target.x(), target.y(), target.z()-self.gripper_offset)) - # print(f'Checking for position {corrected}.') - distance = self._distance(real, corrected, 2) if rough else self._distance(real, corrected) - # print(f'At pose? Distance: {distance}, tolerance: {tolerance}.') - - return distance < float(tolerance) - - def robot_position(self): - pass - - def reachable(self, target: str) -> bool: - """Return True if the target is reachable.""" - max_distance = math.sqrt(self.workspace[0]**2 + self.workspace[1]**2 + self.max_z**2) - gripper_pose, _ = self.get_item_in_frame('GripperControlBody', target) - current_distance = math.sqrt(gripper_pose[0]**2 + gripper_pose[1]**2 + gripper_pose[2]**2) - if current_distance < max_distance: - return True - return False - - # AUXILIARY - - def as_vec3(self, array: np.ndarray or List[float]) -> agx.Vec3: - """Return the Numpy Array object as AGX Vec3.""" - return agx.Vec3(array[0], array[1], array[2]) - - def _distance(self, v1: agx.Vec3 or np.ndarray, v2: agx.Vec3 or np.ndarray, d: int = 3) -> float: - """Return the distance between two vectors in 2D or 3D.""" - v1_ = copy(v1) if type(v1) is agx.Vec3 else self.as_vec3(v1) - v2_ = copy(v2) if type(v2) is agx.Vec3 else self.as_vec3(v2) - - if d == 2: - v1_ = agx.Vec3(v1_.x(), v1_.y(), 0.0) - v2_ = agx.Vec3(v2_.x(), v2_.y(), 0.0) - - return round(float(agx.Vec3.distance(v1, v2)), 3) - - def _move_gripper(self, pos: agx.Vec3) -> None: - """Move the gripper to the desired position.""" - # print('Moving to: ', self.__round(pos)) - self.gripper_lock.setEnable(True) - self.gripper_ctrl.setPosition(pos) - - def _grasp(self, item: str) -> None: - """Grasp an intem by attaching it to the gripper.""" - item_body = self.sim.getRigidBody(item) - if self.with_merger: - self.__split(item_body) - else: - self.item_locks[item].setEnable(False) - - d = self._distance(self.gripper_ctrl.getPosition(), item_body.getPosition()) - - if d < 0.08: - # print(f'Grasping {item}.') - if self.with_merger: - self.__merge(item_body, self.gripper_obj) - else: - self.item_locks[item].rebind() - self.item_locks[item].setEnable(True) - self.holding = item - self._gripper_state = 'closed' - else: - if self.with_merger: - self.__split(item_body) - else: - self.item_locks[item].setEnable(False) - self.holding = '' - self._gripper_state = 'open' - - def _release(self, item: str) -> bool: - """Release the held object in the gripper.""" - # print(f'Releasing {item}.') - if self.with_merger: - self.__split(self.sim.getRigidBody(item)) - if agx.MergedBody.get(self.sim.getRigidBody(item)) is None: - self.holding = '' - self._gripper_state = 'open' - return True - else: - return False - else: - if not self.item_locks[item].getEnable(): - return False - else: - self.item_locks[item].setEnable(False) - self.holding = '' - self._gripper_state = 'open' - return True - - def __merge(self, body1: agx.RigidBody, body2: agx.RigidBody) -> None: - """Merge two bodies for faster simulation.""" - merger = self.sim.getMergedBody(self.merger_name) - merger.add(agx.MergedBodyEmptyEdgeInteraction(body1, body2)) - - def __split(self, body1: agx.RigidBody) -> None: - """Merge two bodies for faster simulation.""" - merger = self.sim.getMergedBody(self.merger_name) - if merger is not None: - success = merger.remove(body1) - - def __create_constraints(self, items: List[str]) -> None: - """Create locks so that task items can be attached to the gripper.""" - for obj in items: - joint = agx.LockJoint(self.gripper_ctrl, self.sim.getRigidBody(obj)) - self.sim.add(joint) - joint.setEnable(False) - self.item_locks[obj] = joint - - def __round(self, vec3: agx.Vec3) -> agx.Vec3: - """Round function for Vec3.""" - return agx.Vec3(round(vec3.x(), 3), round(vec3.y(), 3), round(vec3.z(), 3)) - - def __as_array(self, vec3: agx.Vec3) -> np.ndarray: - """Return the Vec3 object as Numpy Array.""" - try: - return np.array([vec3.x(), vec3.y(), vec3.z(), vec3.w()]) - except AttributeError: - return np.array([vec3.x(), vec3.y(), vec3.z()]) diff --git a/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/behaviors/sim_behaviors.py b/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/behaviors/sim_behaviors.py deleted file mode 100644 index 4ff9de51..00000000 --- a/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/behaviors/sim_behaviors.py +++ /dev/null @@ -1,469 +0,0 @@ -"""Definition of Robot Behaviors.""" - -# Copyright (c) 2022, ABB -# All rights reserved. -# -# Redistribution and use in source and binary forms, with -# or without modification, are permitted provided that -# the following conditions are met: -# -# * Redistributions of source code must retain the -# above copyright notice, this list of conditions -# and the following disclaimer. -# * Redistributions in binary form must reproduce the -# above copyright notice, this list of conditions -# and the following disclaimer in the documentation -# and/or other materials provided with the -# distribution. -# * Neither the name of ABB nor the names of its -# contributors may be used to endorse or promote -# products derived from this software without -# specific prior written permission. -# -# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" -# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE -# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE -# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE -# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL -# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR -# SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER -# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, -# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF -# THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - -import re -import os -from typing import Any, List, Tuple - -from behaviors.behavior_lists import BehaviorLists -from behaviors.common_behaviors import Behavior, RandomSelector, RSequence -from bt_learning.learning_from_demo.demonstration import Demonstration -import bt_learning.learning_from_demo.lfd_behaviors as lfd_bt -from py_trees.composites import Selector, Sequence -from simulation.algoryx.behaviors.agx_interface import AGXInterface -import simulation.algoryx.behaviors.sim_skills as skills -from simulation.algoryx.lfd.planning_itnerface import PlanningInterface -import yaml - -NUMBER_REGEX = r'[-+]?(?:(?:\d*\.\d+)|(?:\d+\.?))(?:[Ee][+-]?\d+)?' - -""" -The string representations of the behaviors are: - - pick name object - - place name object x y z tolerance frame - - open_gripper - - close_gripper - - move name x y z tolerance frame - - gripper_state open/closed - - empty_gripper - - in_gripper object - - object_at object x y z tolerance frame - - object_roughly_at object x y z tolerance frame - - robot_at x y z tolerance frame - -where name is the name of the pickle file where demonstrated actions are stored. -""" - -# TODO: add support for both Right and Left gripper - - -class RobotBehaviors(lfd_bt.Behaviors): - """Defines all executable actions and conditions of simulated Robot.""" - - def __init__(self, directory_path: str, object_data: List[str] = None): - """Directory_path is the path to the directory where planner settings are saved.""" - super().__init__(directory_path) - self.behavior_list = self.get_behavior_list() - self.object_data = object_data - - def get_behavior_list(self) -> BehaviorLists: - """Parse the yaml file and returns the behavior list.""" - # initialize the dictionary an populate it while parsing the yaml - behavior_list = super().get_behavior_list() - return behavior_list - - def unstack_bt(self, name: str, world_interface: Any, condition_parameters: Any) -> RSequence: - """Build a subtree that unstacks the items.""" - root = Selector('Fallback') - with open(os.path.join(self.directory_path, f'{name}.yaml'), 'r') as f: - bt = yaml.safe_load(f) - - subtree = RSequence('Sequence') - children = [self.get_node(x, world_interface, condition_parameters)[0] for x in bt[1:-1]] - subtree.add_children(children) - - root.add_children([ - skills.Unstack('unstacked', world_interface, self.object_data), - subtree - ]) - return root - - def pick_and_place_bt( - self, - node: str, - world_interface: Any, - name: str, - target: str, - pose: List[float], - reference: str, - tolerance: float, - drop: bool - ) -> Selector: - """Build a pick and place subtree.""" - gripper_fb = Selector('Fallback', memory=False) - gripper_fb.add_children([ - skills.GripperState('gripper_state open', world_interface, 'open'), - skills.SetGripper('open_gripper', world_interface, 'open') - ]) - pick_seq = RSequence('Sequence') - pick_seq.add_children([ - gripper_fb, - skills.PickBehavior( - f'pick{name[-1]} {target}', - self.directory_path, - f'pick{name[-1]}', - world_interface, - target - ) - ]) - pick_fb = Selector('Fallback', memory=False) - pick_fb.add_children([ - skills.InGripper(f'in_gripper {target}', world_interface, target), - pick_seq - ]) - place_seq = RSequence('Sequence') - if drop: - place_action = skills.DropBehavior( - f'drop{name[-1]} {target} {pose} {tolerance} {reference}', - self.directory_path, - f'drop{name[-1]}', - world_interface, - target, - pose, - reference, - tolerance - ) - place_condition = skills.ObjectRoughlyAt( - f'object_roughly_at {target} {pose} {tolerance} {reference}', - world_interface, - target, - pose, - reference - ) - else: - place_action = skills.PlaceBehavior( - f'place{name[-1]} {target} {pose} {tolerance} {reference}', - self.directory_path, - f'place{name[-1]}', - world_interface, - target, - pose, - reference, - tolerance - ) - place_condition = skills.ObjectAt( - f'object_at {target} {pose} {tolerance} {reference}', - world_interface, - target, - pose, - reference - ) - place_seq.add_children([pick_fb, place_action]) - - root = Selector('Fallback', memory=False) - root.add_children([ - place_condition, - place_seq - ]) - - return root - - def get_node( - self, - node: str, - world_interface: PlanningInterface or AGXInterface, - condition_parameters: Any - ) -> Tuple[Behavior or RSequence or RandomSelector or Selector or Sequence, bool]: - """ - Return the Behavior Tree node given its string representation. - - Args - ---- - string: name of the robot skill as string. - world_interface: interface to the robot hardware. - condition_parameters: pre- and post-conditions of the skill. - - Returns - ------- - node: behavior tree node, eventually inherits from py_trees - has_children: bool to determine if the node is a control node or a behavior. - - """ - has_children = False - - # This parser depends on what it is written during the demo --> defined in the Action GUI - - # Actions - if node.startswith('pick'): - # Pick is parametrized as 'pickN obj' where N is the unique number of - # the pick action and obj is the object. - match = re.match('^(pick\\d+) (.+)$', node) - node = skills.PickBehavior( - node, - self.directory_path, - match[1], - world_interface, - match[2] - ) - elif node.startswith('place'): - # Place is parameterized as 'placeN object x y z' - match_str = f'^(place\\d+) (.+) ({NUMBER_REGEX}) ({NUMBER_REGEX})' +\ - f' ({NUMBER_REGEX}) ({NUMBER_REGEX}) (.+)$' - match = re.match(match_str, node) - target_list = [float(i) for i in match.group(3, 4, 5)] - node = skills.PlaceBehavior( - node, - self.directory_path, - match[1], - world_interface, - match[2], - [round(num, 3) for num in target_list], - match[7], - match[6] - ) - elif node.startswith('drop'): - match_str = f'^(drop\\d+) (.+) ({NUMBER_REGEX}) ({NUMBER_REGEX})' +\ - f' ({NUMBER_REGEX}) ({NUMBER_REGEX}) (.+)$' - match = re.match(match_str, node) - target_list = [float(i) for i in match.group(3, 4, 5)] - node = skills.DropBehavior( - node, - self.directory_path, - match[1], - world_interface, - match[2], - [round(num, 3) for num in target_list], - match[7], - match[6] - ) - elif node.startswith('close_gripper'): - node = skills.SetGripper(node, world_interface, 'closed') - elif node.startswith('open_gripper'): - node = skills.SetGripper(node, world_interface, 'open') - elif node.startswith('navigate'): - match_str = f'^(navigate\\d+) ({NUMBER_REGEX}) ({NUMBER_REGEX}) ({NUMBER_REGEX})' +\ - f' ({NUMBER_REGEX}) (.+)$' - match = re.match(match_str, node) - node = skills.MoveTo( - node, - self.directory_path, - match[1], - world_interface, - match.group(3, 4, 5), - match[7] - ) - elif node.startswith('approach'): - parts = node.split() - if len(parts) == 2: - # Apporach object - node = skills.Approach(node, world_interface, target_object=parts[1]) - else: - # Approach position - node = skills.Approach( - node, - world_interface, - [float(parts[1]), float(parts[2]), float(parts[3])], - parts[4] - ) - elif node.startswith('move'): - drop = False - if node.startswith('move_roughly'): - match_str = f'^(move_roughly\\d+) (.+) ({NUMBER_REGEX}) ({NUMBER_REGEX})' +\ - f' ({NUMBER_REGEX}) ({NUMBER_REGEX}) (.+)$' - drop = True - else: - match_str = f'^(move\\d+) (.+) ({NUMBER_REGEX}) ({NUMBER_REGEX})' +\ - f' ({NUMBER_REGEX}) ({NUMBER_REGEX}) (.+)$' - match = re.match(match_str, node) - target_list = [float(i) for i in match.group(3, 4, 5)] - node = self.pick_and_place_bt( - node, - world_interface, - name=match[1], - target=match[2], - pose=[round(num, 3) for num in target_list], - reference=match[7], - tolerance=match[6], - drop=drop - ) - elif node.startswith('unstack_boxes'): - node = self.unstack_bt(node, world_interface, condition_parameters) - - # Conditions - elif node.startswith('unstacked'): - node = skills.Unstack(node, world_interface, self.object_data) - elif node.startswith('in_gripper'): - node = skills.InGripper(node, world_interface, node[11:]) - elif node.startswith('gripper_state'): - node = skills.GripperState(node, world_interface, node[14:]) - elif node.startswith('object_at'): - match_str = f'^object_at (.+) ({NUMBER_REGEX}) ({NUMBER_REGEX})' +\ - f' ({NUMBER_REGEX}) ({NUMBER_REGEX}) (.+)$' - match = re.match(match_str, node) - target_list = [float(i) for i in match.group(2, 3, 4)] - node = skills.ObjectAt( - node, - world_interface, - match[1], - [round(num, 3) for num in target_list], - match[6] - ) - elif node.startswith('object_roughly_at'): - match_str = f'^object_roughly_at (.+) ({NUMBER_REGEX}) ({NUMBER_REGEX})' +\ - f' ({NUMBER_REGEX}) ({NUMBER_REGEX}) (.+)$' - match = re.match(match_str, node) - target_list = [float(i) for i in match.group(2, 3, 4)] - node = skills.ObjectRoughlyAt( - node, - world_interface, - match[1], - [round(num, 3) for num in target_list], - match[6] - ) - elif node.startswith('robot_at'): - match_str = f'^robot_at ({NUMBER_REGEX}) ({NUMBER_REGEX}) ({NUMBER_REGEX})' +\ - f' ({NUMBER_REGEX}) (.+)$' - match = re.match(match_str, node) - target_list = [float(i) for i in match.group(2, 3, 4)] - node = skills.RobotAt( - node, - world_interface, - [round(num, 3) for num in target_list], - match[6] - ) - elif node.startswith('reachable'): - parts = node.split() - if len(parts) == 2: - node = skills.Reachable(node, world_interface, target_object=parts[1]) - else: - node = skills.Reachable( - node, - world_interface, - [float(parts[1]), float(parts[2]), float(parts[3])], - parts[4] - ) - else: - # get control node from the super class - node, has_children = super().get_node( - node, world_interface, condition_parameters) - - return node, has_children - - def get_actions(self, demonstrations: Demonstration) -> List[str]: - """ - Get the combined actions for the robot from a demonstration. - - Args - ---- - demonstration: the demonstration to parse. - - Returns - ------- - actions: list of the actions in the demonstration. - - """ - actions = ['open_gripper', 'close_gripper'] - - # Add approach actions dynamically to match demonstrated actions - for demo in demonstrations: - for action in demo: - name = action.action_string() - action_string = None - if name.startswith('pick'): - # Approach object - target_object = action.actions[0].parameters[0] - action_string = f'approach {target_object}' - elif name.startswith('place'): - # Approach position - action_string = f'approach {action.targets[0,0]} {action.targets[0,1]}' +\ - f' {action.targets[0,2]} {action.frames[0]}' - - if action_string is not None and action_string not in actions: - actions.append(action_string) - - return actions - - def get_conditions(self, demonstrations: Demonstration) -> List[str]: - """ - Get the combined conditions for YuMi and the MobileBase from a demonstration. - - Args - ---- - demonstration: the demonstration to parse. - - Returns - ------- - conditions: list of the conditions in the demonstration. - - """ - conditions = ['gripper_state open', 'gripper_state closed', 'in_gripper none'] - - # Add reachable conditions dynamically to match demonstrated conditions - for demo in demonstrations: - for action in demo: - name = action.action_string() - condition_string = None - if name.startswith('pick'): - # Reach object - target_object = action.actions[0].parameters[0] - condition_string = f'reachable {target_object}' - elif name.startswith('place'): - # Approach position - condition_string = f'reachable {action.targets[0, 0]}' +\ - f' {action.targets[0, 1]} {action.targets[0, 2]} {action.frames[0]}' - - if condition_string is not None and condition_string not in conditions: - conditions.append(condition_string) - - return conditions - - def compatible( - self, - condition1: str, - condition2: str - ) -> bool: - """Return True if the conditions are compatible, False otherwise.""" - parts1 = condition1.split() - parts2 = condition2.split() - # The condition type is the first "word" - type1 = parts1[0] - type2 = parts2[0] - - # Incompatible conditions of the same type - if type1 == type2: - if type1 == 'in_gripper' and parts1[1:] != parts2[1:]: - return False - elif type1 == 'gripper_state' and parts1[1:] != parts2[1:]: - return False - elif type1 == 'object_at' and parts1[1] == parts2[1] and parts1[2:] != parts2[2:]: - return False - elif type1 == 'object_roughly_at' and \ - parts1[1] == parts2[1] and parts1[2:] != parts2[2:]: - return False - elif type1 == 'robot_at' and condition1 != condition2: - return False - - # object_at and roughly_at are incompatible - if (type1 == 'object_roughly_at' and type2 == 'object_at' or - type1 == 'object_at' and type2 == 'object_roughly_at') and \ - parts1[1] == parts2[1] and parts1[2:] != parts2[2:]: - return False - - # We cannot hold something and have the gripper open at the same time - if type1 == 'in_gripper' and \ - condition1 != 'in_gripper none' and condition2 == 'gripper_state open' or \ - type2 == 'in_gripper' and\ - condition2 != 'in_gripper none' and condition1 == 'gripper_state open': - return False - - return True diff --git a/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/behaviors/sim_skills.py b/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/behaviors/sim_skills.py deleted file mode 100644 index c50c0de0..00000000 --- a/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/behaviors/sim_skills.py +++ /dev/null @@ -1,742 +0,0 @@ -"""Implementation of robot skills for the simulation.""" - -# Copyright (c) 2022, ABB -# All rights reserved. -# -# Redistribution and use in source and binary forms, with -# or without modification, are permitted provided that -# the following conditions are met: -# -# * Redistributions of source code must retain the -# above copyright notice, this list of conditions -# and the following disclaimer. -# * Redistributions in binary form must reproduce the -# above copyright notice, this list of conditions -# and the following disclaimer in the documentation -# and/or other materials provided with the -# distribution. -# * Neither the name of ABB nor the names of its -# contributors may be used to endorse or promote -# products derived from this software without -# specific prior written permission. -# -# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" -# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE -# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE -# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE -# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL -# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR -# SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER -# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, -# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF -# THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - -from copy import copy - -from simulation.algoryx.lfd.planning_itnerface import PlanningInterface -from simulation.algoryx.behaviors.agx_interface import AGXInterface -import py_trees as pt -import numpy as np -from behaviors.common_behaviors import ActionBehavior -import agx -from typing import Any, List -import random -import pickle - -VERBOSE = False - - -class PickBehavior(ActionBehavior): - def __init__( - self, - action_string: str, - directory_path: str, - name: str, - world_interface: AGXInterface or PlanningInterface, - target_object: str - ): - """ - Initialize the pick task. - - Args: - ---- - - action_string: name of the action. - - directory_path: path to the directory where the demonstration is stored. - - name: name of the action file. - - world_interface: interface to the robot. - - target_object: name for the object to pick. - - """ - self.world_interface = world_interface - - if name is not None: - try: - with open(directory_path + '/' + name + '.pkl', 'rb') as f: - # this object contains fields defined in ActionInfo - self.action_info = pickle.load(f) - except FileNotFoundError: - pass - - self.target_obj = target_object - - self.max_ticks = 5 - self.counter = 0 - - super().__init__(action_string, world_interface) - - def get_preconditions(self) -> List[str]: - """Return the pre-conditions of the action.""" - preconditions = self.action_info.additional_preconditions + ['gripper_state open'] - return preconditions - - def get_postconditions(self) -> List[str]: - """Return the post-conditions of the action.""" - postconditions = ['gripper_state closed', f'in_gripper {self.target_obj}'] - return postconditions - - def initialise(self): - """Initialize the task as a thread.""" - self.picking_task = self.world_interface.pick(self.target_obj) - self.counter = 0 - if VERBOSE: - print(self.counter) - return super().initialise() - - def update(self) -> pt.common.Status: - """Return the status of the behavior.""" - self.counter += 1 - if VERBOSE: - print(self.counter) - if self.world_interface.grasped(self.target_obj): - return pt.common.Status.SUCCESS - elif not self.world_interface.reachable(self.target_obj) or self.counter > self.max_ticks: - return pt.common.Status.FAILURE - else: - return pt.common.Status.RUNNING - - def terminate(self, new_status: pt.common.Status): - """Terminate the task thread and clear locks.""" - return super().terminate(new_status) - - def get_display_name(self) -> str: - """Returnt the action name.""" - return f'Pick {self.target_obj}' - - def cost(self) -> int: - """Define the cost of the action.""" - return 20 - - -class PlaceBehavior(ActionBehavior): - - def __init__( - self, - action_string: str, - directory_path: str, - name: str, - world_interface: AGXInterface or PlanningInterface, - target_object: str, - pose: agx.Vec3 or np.ndarray, - reference_object: str, - tolerance: float - ): - """ - Initialize the place task. - - Args: - ---- - - action_string: name of the action. - - directory_path: path to the directory where the demonstration is stored. - - name: name of the action file. - - world_interface: interface to the robot. - - target_object: name for the object to pick. - - reference_object: reference frame for the action. - - pose: target pose in the given reference frame. - - """ - self.world_interface = world_interface - - if name is not None: - try: - with open(directory_path + '/' + name + '.pkl', 'rb') as f: - # this object contains fields defined in ActionInfo - self.action_info = pickle.load(f) - except FileNotFoundError: - pass - - self.target_obj = target_object - self.holding = None - self.ref_obj = reference_object - self.pose = np.array(pose) - self.tolerance = tolerance - - self.max_ticks = 5 - self.counter = 0 - - super().__init__(action_string, world_interface) - - def get_preconditions(self) -> List[str]: - """Return the pre-conditions of the action.""" - preconditions = self.action_info.equivalent_action.preconditions_with_additional() - return preconditions - - def get_postconditions(self) -> List[str]: - """Return the post-conditions of the action.""" - postconditions = self.action_info.equivalent_action.postconditions() - return postconditions - - def initialise(self): - """Initialize the task as a thread.""" - self.placing_task = self.world_interface.place( - self.target_obj, self.ref_obj, self.pose) - self.counter = 0 - self.holding = self.world_interface.holding - if VERBOSE: - print(self.counter) - return super().initialise() - - def update(self) -> pt.common.Status: - """Return the status of the behavior.""" - self.counter += 1 - if VERBOSE: - print(self.counter) - if self.world_interface.at_pose(self.target_obj, self.ref_obj, self.pose, self.tolerance): - return pt.common.Status.SUCCESS - elif self.holding != self.target_obj or self.counter > self.max_ticks: - return pt.common.Status.FAILURE - else: - return pt.common.Status.RUNNING - - def terminate(self, new_status: pt.common.Status): - """Terminate the task thread and clear locks.""" - return super().terminate(new_status) - - def get_display_name(self) -> str: - """Returnt the action name.""" - name = 'Place %s at (%.2g, %.2g, %.2g) in %s' %\ - ( - self.target_obj, float(self.pose[0]), float(self.pose[1]), - float(self.pose[2]), self.ref_obj - ) - return name - - def cost(self) -> int: - """Define the cost of the action.""" - return 20 - - -class DropBehavior(PlaceBehavior): - - def update(self) -> pt.common.Status: - """Return the status of the behavior.""" - self.counter += 1 - if self.world_interface.at_pose( - self.target_obj, self.ref_obj, self.pose, self.tolerance, rough=True): - return pt.common.Status.SUCCESS - elif self.holding != self.target_obj or self.counter > self.max_ticks: - return pt.common.Status.FAILURE - else: - return pt.common.Status.RUNNING - - def get_display_name(self) -> str: - """Returnt the action name.""" - name = 'Drop %s at (%.2g, %.2g, %.2g) in %s' %\ - ( - self.target_obj, float(self.pose[0]), float(self.pose[1]), - float(self.pose[2]), self.ref_obj - ) - return name - - -class SetGripper(ActionBehavior): - def __init__( - self, - action_string: str, - world_interface: AGXInterface or PlanningInterface, - state: str - ): - """ - Set the state for the gripper. - - Args: - ---- - - action_string: name of the action. - - world_interface: interface to the robot. - - state: if the gripper is open or close. - - """ - super().__init__(action_string, world_interface) - - self.state = state - self.world_interface = world_interface - self.gripper_task = None - - def get_preconditions(self) -> List[str]: - """Return the pre-conditions of the action.""" - return [] - - def get_postconditions(self) -> List[str]: - """Return the post-conditions of the action.""" - if self.state == 'open': - return ['gripper_state open', 'in_gripper none'] - elif self.state == 'closed': - return ['gripper_state closed'] - else: - raise ValueError(f'Unknown gripper state "{self.state}".\ - The state must be either "open" or "closed".') - - def initialise(self): - """Initialize the task as a thread.""" - self.gripper_task = self.world_interface.set_gripper(self.state) - - def update(self) -> pt.common.Status: - """Return the status of the behavior.""" - if self.world_interface.is_gripper_state(self.state): - return pt.common.Status.SUCCESS - else: - return pt.common.Status.RUNNING - - def terminate(self, new_status: pt.common.Status): - """Terminate the task thread and clear locks.""" - return super().terminate(new_status) - - def get_display_name(self) -> str: - """Returnt the action name.""" - return f'Set gripper {self.state}' - - def cost(self) -> int: - """Define the cost of the action.""" - return 2 - - -class MoveTo(ActionBehavior): - def __init__( - self, - action_string: str, - directory_path: str, - name: str, - world_interface: AGXInterface or PlanningInterface, - target: np.ndarray, - reference: str - ): - """ - Initialize the move task. - - Args: - ---- - - action_string: name of the action. - - directory_path: path to the directory where the demonstration is stored. - - name: name of the action file. - - world_interface: interface to the robot. - - target: target goal for the robot. - - reference: reference frame for the target. - - """ - self.world_interface = world_interface - - if name is not None: - with open(directory_path + '/' + name + '.pkl', 'rb') as f: - # this object contains fields defined in ActionInfo - self.action_info = pickle.load(f) - - self.target = target - self.reference = reference - self.tolerance = 0.5 - self.moving_task = None - - super().__init__(action_string, world_interface) - - def get_preconditions(self) -> List[str]: - """Return the pre-conditions of the action.""" - return [] - - def get_postconditions(self) -> List[str]: - """Return the post-conditions of the action.""" - return [ - str(f'robot_at {self.target[0]} {self.target[1]} {self.target[2]}' + - f' {self.tolerance} {self.reference}') - ] - - def initialise(self): - """Initialize the task as a thread.""" - # Choose a goal randomically so that if the actions fails we have variation - self.moving_task = self.world_interface.navigate(self.target, self.reference) - - def update(self) -> pt.common.Status: - """Return the status of the behavior.""" - return pt.common.Status.SUCCESS - - def terminate(self, new_status: pt.common.Status): - """Terminate the task thread and clear locks.""" - return super().terminate(new_status) - - def get_display_name(self) -> str: - """Returnt the action name.""" - return 'Move to (%.2g, %.2g, %.2g) in %s' %\ - ( - float(self.target[0]), float(self.target[1]), float(self.target[2]), - self.reference - ) - - def cost(self) -> int: - """Define the cost of the action.""" - return 30 - - -class Approach(ActionBehavior): - def __init__( - self, - action_string: str, - world_interface: AGXInterface or PlanningInterface, - target: List[float] = None, - frame: str = None, - target_object: str = None - ): - """ - Approach action for the mobile platform. - - If object is not None, x, y, z, and frame are ignored. - If object is None, x, y, z, and frame must have values. - - Args: - ---- - - action_string: name of the action. - - directory_path: path to the directory where the demonstration is stored. - - name: name of the action file. - - world_interface: interface to the robot. - - target: target goal for the robot. - - reference: reference frame for the target. - - """ - self.world_interface = world_interface - - self.object = target_object - self.target = None - self.frame = world_interface.default_frame - self.moving_task = None - if self.object is None: - self.target = np.array(target) - self.frame = frame - - super().__init__(action_string, world_interface) - - def get_preconditions(self) -> List[str]: - """Return the pre-conditions of the action.""" - return [] - - def get_postconditions(self) -> List[str]: - """Return the post-conditions of the action.""" - if self.object is None: - return [f'reachable {self.target[0]} {self.target[1]} {self.target[2]} {self.frame}'] - else: - return [f'reachable {self.object}'] - - def initialise(self): - """Initialize the task as a thread.""" - if self.object is not None: - self.target, _ = self.world_interface.get_item_in_frame(self.object, self.frame) - self.moving_task = self.world_interface.approach(self.target, self.frame) - - def update(self) -> pt.common.Status: - """Return the status of the behavior.""" - return pt.common.Status.SUCCESS - - def terminate(self, new_status: pt.common.Status): - """Terminate the task thread and clear locks.""" - return super().terminate(new_status) - - def get_display_name(self) -> str: - """Returnt the action name.""" - if self.object is None: - return 'Approach (%.2g, %.2g, %.2g) in %s' %\ - (self.target[0], self.target[1], self.target[2], self.frame) - else: - return 'Approach ' + self.object - - def cost(self) -> int: - """Define the cost of the action.""" - return 30 - -# Conditions. -# Conditions don't need access to the configuration directory - - -class Unstack(pt.behaviour.Behaviour): - """Returns SUCCESS if the items are unstacked.""" - - def __init__( - self, - name: str, - world_interface: AGXInterface or PlanningInterface, - items: List[str] - ) -> None: - """Check if the input items are unstacked.""" - super().__init__(name) - - self.world_interface = world_interface - self.items = items - self.success = False - self.world_interface.unstacked = False - - def update(self) -> pt.common.Status: - """Return the status of the behavior.""" - unstacked = self.world_interface.items_unstacked(self.items) - if unstacked or self.success: - self.success = True - self.world_interface.unstacked = True - return pt.common.Status.SUCCESS - else: - return pt.common.Status.FAILURE - - def get_display_name(self) -> str: - """Returnt the condition name.""" - return f'Boxes Unstacked?' - - -class InGripper(pt.behaviour.Behaviour): - """ - Returns SUCCESS if the gripper is holding object. - - If object is None the behavior tests empty gripper. - """ - - def __init__( - self, - name: str, - world_interface: AGXInterface or PlanningInterface, - held_object: str - ) -> None: - """ - Initialize the condition. - - Args: - ---- - - name: name of the condition - - world_interface: interface to the robot. - - held_object: name for the object in the gripper. - - """ - super().__init__(name) - - self.world_interface = world_interface - self.object = held_object - - def update(self) -> pt.common.Status: - """Return the status of the behavior.""" - if self.object is None or self.object == 'none': - holding = self.world_interface.empty_gripper() - else: - holding = self.world_interface.grasped(self.object) - - if holding: - return pt.common.Status.SUCCESS - else: - return pt.common.Status.FAILURE - - def get_display_name(self) -> str: - """Returnt the condition name.""" - return f'In gripper {self.object}?' - - -class GripperState(pt.behaviour.Behaviour): - """Returns SUCCESS if the gripper is in the desired state.""" - - def __init__( - self, - name: str, - world_interface: AGXInterface or PlanningInterface, - state: Any - ) -> None: - """ - Initialize the condition. - - Args: - ---- - - name: name of the condition - - world_interface: interface to the robot. - - state: state of the gripper. - - """ - super().__init__(name) - - self.world_interface = world_interface - self.state = state - - def update(self) -> pt.common.Status: - """Return the status of the behavior.""" - if self.world_interface.is_gripper_state(self.state): - return pt.common.Status.SUCCESS - else: - return pt.common.Status.FAILURE - - def get_display_name(self) -> str: - """Returnt the condition name.""" - return f'Gripper {self.state}?' - - -class ObjectAt(pt.behaviour.Behaviour): - """Returns SUCCESS if the object is at a location within a tolerance.""" - - def __init__( - self, - name: str, - world_interface: AGXInterface or PlanningInterface, - held_object: str, - pose: agx.Vec3 or np.ndarray, - frame: str - ) -> None: - """ - Initialize the condition. - - Args: - ---- - - name: name of the condition - - world_interface: interface to the robot. - - held_object: name for the object in the gripper. - - pose: desired position of the target object - - frame: reference frame in which the position is defined. - - """ - super().__init__(name) - - self.world_interface = world_interface - self.object = held_object - self.position = np.array(pose) - self.frame = frame - self.tolerance = 0.03 - - def update(self) -> pt.common.Status: - """Return the status of the behavior.""" - pose_ok = self.world_interface.at_pose( - self.object, self.frame, self.position, self.tolerance) - if pose_ok: - return pt.common.Status.SUCCESS - else: - return pt.common.Status.FAILURE - - def get_display_name(self) -> str: - """Returnt the condition name.""" - return '%s at (%.2g, %.2g, %.2g) in %s?' %\ - (self.object, self.position[0], self.position[1], self.position[2], self.frame) - - -class ObjectRoughlyAt(ObjectAt): - - def __init__(self, *args, **kwargs) -> None: - """Initialize the condition.""" - super().__init__(*args, **kwargs) - - self.tolerance = 0.1 - - def update(self) -> pt.common.Status: - """Return the status of the behavior.""" - pose_ok = self.world_interface.at_pose( - self.object, self.frame, self.position, self.tolerance, rough=True) - if pose_ok: - return pt.common.Status.SUCCESS - else: - return pt.common.Status.FAILURE - - def get_display_name(self) -> str: - """Returnt the condition name.""" - return '%s roughly at (%.2g, %.2g, %.2g) in %s?' %\ - (self.object, self.position[0], self.position[1], self.position[2], self.frame) - - -class RobotAt(pt.behaviour.Behaviour): - """Returns SUCCESS if the object is at a location within a tolerance.""" - - def __init__( - self, - name: str, - world_interface: AGXInterface or PlanningInterface, - target: List[float], - frame: str - ) -> None: - """ - Condition to determine if the robot is a specific position. - - Args: - ---- - - name: name of the action. - - world_interface: interface to the robot. - - target: target goal for the action. - - tolerance: error in the robot position. - - frame: reference frame for robot. - - """ - super().__init__(name) - - self.world_interface = world_interface - self.position = np.array(target) - self.tolerance = 0.5 - self.frame = frame - - def update(self) -> pt.common.Status: - """Return the status of the behavior.""" - current_position = self.world_interface.robot_position(self.frame) - - distance = np.linalg.norm(self.position - current_position) - if distance <= self.tolerance: - return pt.common.Status.SUCCESS - else: - return pt.common.Status.FAILURE - - def get_display_name(self) -> str: - """Returnt the action name.""" - return 'Robot at (%.2g, %.2g, %.2g) in %s?' %\ - (self.position[0], self.position[1], self.position[2], self.frame) - - -class Reachable(pt.behaviour.Behaviour): - def __init__( - self, - name: str, - world_interface: AGXInterface or PlanningInterface, - target: List[float] = None, - frame: str = None, - target_object: str = None - ) -> None: - """ - Determine if the target pose or object is reachable. - - If object is not None, target, and frame are ignored. - If object is None, target, and frame must have values. - - Args: - ---- - - name: name of the condition. - - world_interface: interface to the robot. - - target: target goal for the action. - - frame: reference frame for the object to approach. - - target_object: name for the object to approach. - - """ - self.world_interface = world_interface - self.target = None - self.frame = world_interface.default_frame - self.object = target_object - - if self.object is None: - self.target = np.array([target]) - self.frame = frame - - super().__init__(name) - - def update(self) -> pt.common.Status: - """Return the status of the behavior.""" - if self.object is not None: - self.target, _ = self.world_interface.get_item_in_frame(self.object, self.frame) - - if self.world_interface.reachable(self.target, self.frame): - return pt.common.Status.SUCCESS - else: - return pt.common.Status.FAILURE - - def get_display_name(self) -> str: - """Returnt the action name.""" - if self.object is None: - return 'Reachable (%.2g, %.2g, %.2g) in %s?' %\ - (self.target[0], self.target[1], self.target[2], self.frame) - else: - return f'Reachable {self.object}?' diff --git a/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/combined/gui.py b/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/combined/gui.py deleted file mode 100644 index 790a12c0..00000000 --- a/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/combined/gui.py +++ /dev/null @@ -1,798 +0,0 @@ -""" -Launch the GUI to simulate the GP+LfD framework with the AGX Simulator. - -Use with: python combined/gui.py --model models/BTDemo.yml:CoarseGripperInLabDR --decorate --timeStep 0.05 -from the /algoryx folder -""" - -# Copyright (c) 2022, ABB -# All rights reserved. -# -# Redistribution and use in source and binary forms, with -# or without modification, are permitted provided that -# the following conditions are met: -# -# * Redistributions of source code must retain the -# above copyright notice, this list of conditions -# and the following disclaimer. -# * Redistributions in binary form must reproduce the -# above copyright notice, this list of conditions -# and the following disclaimer in the documentation -# and/or other materials provided with the -# distribution. -# * Neither the name of ABB nor the names of its -# contributors may be used to endorse or promote -# products derived from this software without -# specific prior written permission. -# -# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" -# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE -# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE -# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE -# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL -# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR -# SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER -# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, -# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF -# THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - -from copy import copy, deepcopy -import glob -import logging -import multiprocessing as mp -import os -import re -import subprocess -import time -from typing import Any, Dict, List - -from agxBrick.brickLoaderUtils import createArgumentParser -from behaviors.behavior_lists import BehaviorLists -from bt_learning.gp import logplot -from bt_learning.learning_from_demo.clustering import find_equivalent_actions -from bt_learning.learning_from_demo.learning import learn_tree -from bt_learning.learning_from_demo.plot_clusters import plot_clusters -import robot_behaviors.mobile_base_behaviors.lfd_actions as base_actions -import robot_behaviors.yumi_behaviors.lfd_actions as yumi_actions -from robot_interface.demonstration import RobotDemonstration -import simulation.algoryx.agx_application as app -import simulation.algoryx.combined.gui_common as func -import simulation.algoryx.combined.processes as proc -from simulation.algoryx.behaviors.sim_behaviors import RobotBehaviors -from simulation.algoryx.gp import gp_asprocess, gp_environment, gp_interface -from simulation.algoryx.lfd import lfd_asprocess, lfd_interface -from simulation.algoryx.lfd.planning_itnerface import PlanningInterface -from user_interface.agx_gui import AGXGUILayout -from user_interface.gui_ddmenu import GUIMenu -from user_interface.lfd_gui_actions import GUIActions -import yaml - -# OS dependent import -if os.name == 'nt': # Windows - import PySimpleGUIQt as gui -elif os.name == 'posix': # Linux Ubuntu - import PySimpleGUI as gui - - -class GUI(): - - def __init__(self, args: Any) -> None: - self._logger = logging.getLogger(__file__) - - self.repo_path = os.path.dirname(os.path.dirname((os.path.abspath(__file__)))) - with open(os.path.join(self.repo_path, 'config/sim_data.yaml')) as f: - self.sim_data = yaml.safe_load(f) - with open(os.path.join(self.repo_path, 'config/sim_objects.yaml')) as f: - self.obj_data = yaml.safe_load(f) - with open(os.path.join(self.repo_path, 'config/gp_targets.yaml')) as f: - self.config_targets = yaml.safe_load(f) - with open(os.path.join(self.repo_path, 'config/gp_initial.yaml')) as f: - gp_initial = yaml.safe_load(f) - - self.targets = copy(self.config_targets) - self.task_objects = list(self.obj_data.keys()) - self.ref_objects = self.sim_data['demonstration']['reference_frames'] - - # AGX Stuff - self.agx_args = args - random_bringup = self.sim_data['algoryx']['random_bringup'] - self.initial = gp_initial if random_bringup is False else None - - # BT Stuff - self.bt = None - self.bt_length = 0 - self.bt_tick_freq = self.sim_data['behavior_tree']['tick_freq'] - self.behaviors = None - - # GP Stuff - app.GRAPHICS = False - self.gp_interface = gp_interface.GPInterface(self.task_objects, self.ref_objects) - self.gp_par = func.init_gp_params(self.sim_data, self.repo_path) - self.baseline_index = self.sim_data['genetic']['baseline_index'] - if type(self.baseline_index) is not int or self.baseline_index == 'None': - self.baseline_index = None - self.fitness_coeff = func.get_fitness_coefficients() - self.visual = self.sim_data['genetic']['graphics'] - - self.log_folder = os.path.join(self.repo_path, f'logs/log_{self.gp_par.log_name}') - self.gp_process = None - self.current_gen = 0 - - # LfD Stuff - app.GRAPHICS = False - self.lfd_interface = lfd_interface.LfDInterface(self.task_objects) - self.default_frame = self.sim_data['demonstration']['default_frame'] - self.robot_frame = self.sim_data['demonstration']['robot_frame'] - self.ee_frame = self.sim_data['algoryx']['ee_name'] - - self.all_frames = self.sim_data['demonstration']['reference_frames'] + self.task_objects - if self.robot_frame not in self.all_frames: - self.all_frames.append(self.robot_frame) - if self.default_frame not in self.all_frames: - self.all_frames.append(self.default_frame) - self.pick_menu = GUIMenu('Choose the object to pick.', self.task_objects) - - self.grasping_actions = self.sim_data['demonstration']['grasping_actions'] - self.placing_actions = self.sim_data['demonstration']['placing_actions'] - self.navigation_actions = self.sim_data['demonstration']['navigation_actions'] - - self.actions = self.grasping_actions + self.placing_actions - - self.demo_folder = os.path.join(self.log_folder, 'demos') - self.lfd_tree_folder = os.path.join(self.demo_folder, 'bt1') - self.demonstrations = None - - # Action flags - self.holding = None - self.running_action = None - - self.parent_conn, self.child_conn = mp.Pipe() - - # Initialize GUI - self.agx_GUI = AGXGUILayout( - 'Simulation framework for learning Behavior Trees', - os.path.join(self.repo_path, f'logs') - ) - - def show(self, build_tree_function: Any = None) -> None: - """Define the GUI functionalities.""" - self.agx_GUI.initialize_layout() - - window = self.agx_GUI.get_window() - - # flags: - has_BT = False - has_GP = False - has_Demo = False - - env_showing = False - gp_running = False - bt_running = False - - # Interaction with the GUI Main menu - ret = 0 - while ret is not None: - ret, values = window.read() - - if ret == '__folder_display__' and os.path.isdir(values['__folder_display__']): - self.log_folder = values['__folder_display__'] - self.demo_folder = os.path.join(self.log_folder, 'demos') - self.lfd_tree_folder = os.path.join(self.demo_folder, 'bt1') - if not os.path.isdir(self.demo_folder): - os.makedirs(self.lfd_tree_folder) - try: - id = int(self.log_folder[-2:]) - self.gp_par.log_name = str(id) - except ValueError: - self.gp_par.log_name = self.log_folder[-1] - self.gp_par.behavior_lists = self._generate_behaviors(self.log_folder) - self.behaviors = RobotBehaviors(self.log_folder, self.task_objects) - try: - with open(os.path.join(self.log_folder, 'tree.yaml'), 'r') as f: - self.bt = yaml.safe_load(f) - except FileNotFoundError: - pass - try: - with open(os.path.join(self.log_folder, 'gp_targets.yaml'), 'r') as f: - self.targets = yaml.safe_load(f) - except FileNotFoundError: - pass - self._update_gp_values(window) - self._reload_demos(window, build_tree_function) - if self.current_gen > 0: - has_BT = True - has_GP = True - if self.demonstrations is not None: - has_BT = True - has_Demo = True - self.__continue_interaction(window, has_BT, has_GP, has_Demo) - - # region AGX Stuff - elif ret == '__initial__': - if not env_showing: - window.find_element('__initial__').update('Shutdown') - self.agx_GUI.disable_all_but(window, '__initial__') - app.GRAPHICS = True - sim_args = ( - app.CloudpickleWrapper(app.Application), - self.agx_args, - self.obj_data, - self.initial - ) - process = proc.SimProcess(sim_args) - process.start() - env_showing = True - else: - window.find_element('__initial__').update('Initial Configuration') - process.stop() - env_showing = False - self.__continue_interaction(window, has_BT, has_GP, has_Demo) - - elif ret == '__target__': - if not env_showing: - window.find_element('__target__').update('Shutdown') - self.agx_GUI.disable_all_but(window, '__target__') - app.GRAPHICS = True - sim_args = ( - app.CloudpickleWrapper(app.Application), - self.agx_args, - self.obj_data, - self.targets - ) - process = proc.SimProcess(sim_args) - process.start() - env_showing = True - else: - window.find_element('__target__').update('Target Configuration') - process.stop() - env_showing = False - self.__continue_interaction(window, has_BT, has_GP, has_Demo) - # endregion - - # region BT Stuff - elif ret == '__run_tree__': - if bt_running is False: - if self.bt is None: - print('No Behavior Tree to run!') - has_BT = False - self.__continue_interaction(window, has_BT, has_GP, has_Demo) - else: - # disable all elements - self.agx_GUI.disable_all_but(window, '__run_tree__') - window.find_element('__run_tree__').update('Stop') - print('Launching tree...') - bt_running = True - - app.GRAPHICS = True - self.bt_process = app.RunBT( - self.lfd_interface, - self.obj_data, - self.behaviors, - self.bt, - self.bt_tick_freq, - (app.CloudpickleWrapper(app.Application), self.agx_args) - ) - self.bt_process.start() - - else: - self.bt_process.stop() - self.bt_process.join() - bt_running = False - window.find_element('__run_tree__').update('Run') - self.__continue_interaction(window, has_BT, has_GP, has_Demo) - - elif ret == '__show_tree__': - if os.name == 'nt': # Windows - os.startfile(os.path.join(self.log_folder, 'best_individual.svg')) - else: - opener = 'xdg-open' - subprocess.call([opener, os.path.join(self.log_folder, 'best_individual.svg')]) - - elif ret == '__save_tree__': - folder = gui.popup_get_folder('Select a folder') - if folder is not None and folder != '': - if os.path.isdir(folder): - number = 1 - while os.path.isdir(os.path.join(folder, f'bt{number}')): - number += 1 - - folder = os.path.join(folder, f'bt{number}') - - os.makedirs(folder) - self._save_tree(self.bt, folder, self.behaviors) - self.__continue_interaction(window, has_BT, has_GP, has_Demo) - # endregion - - # region GP Stuff - elif ret == '__start__': - if not gp_running: - # disable all elements - window.find_element('__start__').update('Stop GP') - self.agx_GUI.disable_all_but(window, '__start__') - - # Create behaviors - if not os.path.isdir(self.log_folder): - os.makedirs(self.log_folder) - self.gp_par.behavior_lists = self._generate_behaviors(self.log_folder) - self.behaviors = RobotBehaviors(self.log_folder, self.task_objects) - pytrees_params = func.get_pytree_params( - self.gp_par.behavior_lists, self.behaviors) - self.gp_env = gp_environment.GPEnvironment( - self.gp_interface, - self.behaviors, - self.initial, - self.targets, - self.fitness_coeff, - pytrees_params - ) - - gp_running = True - hotstart = True if self.current_gen > 0 else False - app.GRAPHICS = self.visual - baseline = self._get_baseline() - baseline_index = self.baseline_index - if baseline is None: - baseline_index = None - self.gp_process = gp_asprocess.GPProcess( - os.path.join(self.repo_path, 'config'), - self.gp_env, - self.gp_par, - hotstart=hotstart, - baseline=baseline, - baseline_index=baseline_index, - visual=self.visual, - args=(app.CloudpickleWrapper(app.Application), self.agx_args) - ) - print('Starting GP Evolution:') - print(f'- hotstart: {hotstart}') - print(f'- baseline: {baseline}') - print(f'- target:\n {self.targets}') - self.gp_process.start() - - else: - gp_running = False - has_GP = True - has_BT = True - window.find_element('__start__').update('Start GP') - - self.gp_process.stop() - self.gp_process.join() - print(f'Stopped at: {self.current_gen + 1}.') - self._update_gp_values(window) - self.bt = self._get_baseline() - self.__continue_interaction(window, has_BT, has_GP, has_Demo) - - elif ret == '__plot_fitness__': - path_to_fitness = os.path.join(self.log_folder, 'Fitness.svg') - func.display(path_to_fitness) - - elif ret == '__increase__': - initial_n_gen = self.gp_par.n_generations - self.gp_par.n_generations = initial_n_gen + self.current_gen + 1 - print(f'Total number of generation: {self.gp_par.n_generations}.') - print(f'Current generation: {self.current_gen + 1}.') - # endregion - - # region LfD Stuff - elif ret == '__new_demo__': - if not os.path.isdir(self.demo_folder): - os.makedirs(self.demo_folder) - self._reload_demos(window, build_tree_function) - window.disappear() - self._new_demo(self.initial) - self._reload_demos(window, build_tree_function) - window.reappear() - has_BT = True - has_Demo = True - self.__continue_interaction(window, has_BT, has_GP, has_Demo) - - elif ret == '__new_target__': - if not os.path.isdir(self.demo_folder): - os.makedirs(self.demo_folder) - self._reload_demos(window, build_tree_function) - window.disappear() - self._new_demo(self.targets) - self._reload_demos(window, build_tree_function) - window.reappear() - if self.demonstrations is not None: - # We reached the size of the cluster so now it is safe to infer the target - self._add_target() - has_BT = True - has_Demo = True - self.__continue_interaction(window, has_BT, has_GP, has_Demo) - - elif ret == '__cluster__': - if self.demonstrations is None: - _ = self._load_demo() - plot_clusters(self.demo_folder, self.demonstrations) - print('Clusters plotted!') - # endregion - - def _update_gp_values(self, window) -> None: - try: - self.current_gen = logplot.get_state(self.gp_par.log_name)[2] - self.best_fitness = max(logplot.get_best_fitness(self.gp_par.log_name)) - self.best_individual = logplot.get_best_individual(self.gp_par.log_name) - - window.find_element('__gen__').update( - 'Current generation: %d' % (self.current_gen + 1)) - window.find_element('__fitness__').update( - 'Best fitness score: %d' % self.best_fitness) - args = ( - app.CloudpickleWrapper(app.Application), - self.agx_args, - self.log_folder, - self.best_individual, - self.obj_data, - self.behaviors, - self.gp_interface - ) - app.GRAPHICS = False - process = mp.Process(target=proc.bt_worker, args=args) - process.start() - time.sleep(5) - print('Behavior Tree generated!') - except (AttributeError, FileNotFoundError): - window.find_element('__gen__').update( - 'Current generation: %d' % (self.current_gen + 1)) - window.find_element('__fitness__').update( - 'Best fitness score: -inf') - - def _get_baseline(self) -> List[str] or None: - """Get the baseline from the given folder.""" - baseline = None - try: - with open(os.path.join(self.lfd_tree_folder, 'tree.yaml')) as f: - baseline = yaml.safe_load(f) - print('Using baseline from demonstration!') - except (FileNotFoundError, UnicodeDecodeError): - print('No baseline found!') - - return baseline - - def _generate_behaviors(self, folder: str) -> BehaviorLists or None: - """Generate behaviors for the GP parameters.""" - app.GRAPHICS = False - process = mp.Process( - target=proc.btlist_worker, - args=( - app.CloudpickleWrapper(app.Application), - self.agx_args, - self.obj_data, - self.sim_data, - folder, - self.gp_interface - ) - ) - process.start() - time.sleep(5) - print('Behaviors generated!') - try: - with open(os.path.join(folder, 'BT_SETTINGS.yaml')) as f: - settings = yaml.safe_load(f) - behavior_list = BehaviorLists( - settings['fallback_nodes'], - settings['atomic_fallback_nodes'], - settings['sequence_nodes'], - settings['atomic_sequence_nodes'], - settings['condition_nodes'], - settings['action_nodes'] - ) - return behavior_list - except FileNotFoundError: - print('Error in loading the behavior lists.') - return None - - def _add_target(self) -> None: - """Infer tha target from the learnt tree and add it to the config targets.""" - NUMBER_REGEX = r'[-+]?(?:(?:\d*\.\d+)|(?:\d+\.?))(?:[Ee][+-]?\d+)?' - with open(os.path.join(self.lfd_tree_folder, 'tree.yaml')) as f: - string_tree = yaml.safe_load(f) - for j, node in enumerate(string_tree): - if node.startswith('move'): - match_str = f'^(move\\d+) (.+) ({NUMBER_REGEX}) ({NUMBER_REGEX})' +\ - f' ({NUMBER_REGEX}) ({NUMBER_REGEX}) (.+)$' - match = re.match(match_str, node) - target_obj = match[2] - target_pose = [float(i) for i in match.group(3, 4, 5)] - try: - target_pose[2] -= self.lfd_interface.gripper_offset - except AttributeError: - target_pose[2] -= 0.12 - reference = match[7] - break - else: - continue - - if target_obj not in list(self.targets.keys()): - self.targets[target_obj] = {} - self.targets[target_obj]['pose'] = target_pose - self.targets[target_obj]['reference'] = reference - with open(os.path.join(self.log_folder, 'gp_targets.yaml'), 'w+') as f: - yaml.dump(self.targets, f, sort_keys=False) - print(f'A new target for the item {target_obj} has been set!') - - def _new_demo(self, current_config: Dict = None) -> None: - """Launch a new window to start a new demonstration.""" - action_GUI = GUIActions('New Demo', self.actions) - action_GUI.initialize_layout() - - action_window = action_GUI.get_window() - - if self.demonstrations is None: - n_demos = 0 - else: - n_demos = self.demonstrations.n_demonstrations() - - os.mkdir(os.path.join(self.demo_folder, 'demo%d' % (n_demos + 1))) - - config_path = os.path.join(self.repo_path, 'config') - app.GRAPHICS = True - self.lfd_process = lfd_asprocess.LfDProcess( - config_path, - self.lfd_interface, - (self.child_conn, app.CloudpickleWrapper(app.Application), self.agx_args, current_config) - ) - self.lfd_process.start() - - ret = 0 - while ret is not None: - ret, values = action_window.read() - action_GUI.execute_action(ret, self._manipulate, n_demos) - - self.lfd_process.stop() - self.lfd_process.join() - print('Demonstration concluded!') - - def _manipulate(self, n_demos: int, action_name: str, gripper_state: str) -> None: - """Define how to run actions when connected to the robot.""" - def get_obj_from_menu(menu: GUIMenu) -> str: - menu_ = deepcopy(menu) - key_ = menu_.initialize_layout() - window = menu_.get_window() - event, values = window.read() - obj = values[key_] - window.close() - return obj - - # Interact with the GUI Action menu - if action_name == 'pick': - # Handle dropdown menu - self.holding = get_obj_from_menu(self.pick_menu) - self.parent_conn.send(('pick', self.holding)) - print(f'Picking {self.parent_conn.recv()}.') - self._write_action(action_name, n_demos + 1, self.parent_conn) - else: - mainpulating = copy(self.holding) - print(f'Releasing {mainpulating}.') - self.parent_conn.send((action_name, ())) - print(f'Holding {self.parent_conn.recv()}.') - self.holding = None - self._write_action(action_name, n_demos + 1, self.parent_conn) - self.parent_conn.send(('lift', ())) - lifted = self.parent_conn.recv() - - def _write_action( - self, - action_type: str, - current_demo: str, - conn: mp.connection.Connection - ) -> None: - """Write the pose of the end effector in all available frames as a new action.""" - action_data = {'type': action_type, 'vec_pos': {}, 'vec_quat': {}} - current_demo_dir = os.path.join(self.demo_folder, 'demo%d' % current_demo) - if self.demonstrations is None: - frames = self.all_frames - else: - frames = self.demonstrations.frames - - for frame in frames: - # remove the default frame for now - conn.send(('get_sensor_data', (self.ee_frame, frame))) - position, orientation = conn.recv() - - action_data['vec_pos'][frame] = position.tolist() - action_data['vec_quat'][frame] = orientation.tolist() - - action_number = len(glob.glob(os.path.join(current_demo_dir, 'data_*.yaml'))) + 1 - with open(os.path.join(current_demo_dir, f'data_{action_number}.yaml'), 'w') as f: - yaml.dump(action_data, f, default_flow_style=None) - - def _reload_demos(self, window: gui.Window, build_tree_function) -> None: - """Update the demonstration history and build the tree.""" - print('Reloading Demos...') - n_demos = self._load_demo() - window.find_element('__n_demos__').update('Number of demonstrations: %d' % n_demos) - window.find_element('__new_demo__').update(disabled=False) - - if self.demonstrations is None: - return - - if build_tree_function is None: - self._build_tree(window) - else: - build_tree_function(window) - - def _load_demo(self) -> int: - """Load demonstration and return number of demonstrations.""" - info_file = os.path.join(self.demo_folder, 'info.yaml') - if not os.path.isfile(info_file): - with open(info_file, 'w') as f: - yaml.dump( - { - 'frames': self.all_frames, - 'robot': self.robot_frame, - 'default_frame': self.default_frame - }, f) - self.demonstrations = None - return 0 - elif len(glob.glob(os.path.join(self.demo_folder, 'demo*'))) == 0: - self.demonstrations = None - return 0 - else: - # Remove empty folders - for folder in glob.glob(self.demo_folder + '/demo[0-9]*/'): - # Each demonstration has to contain at least one action - if not os.path.isfile(folder + '/data_1.yaml'): - print(f'Folder {folder} missing data file, removing.') - os.rmdir(folder) - self.demonstrations = RobotDemonstration( - self.demo_folder, - custom_actions={ - 'pick': yumi_actions.PickAction, - 'place': yumi_actions.PlaceAction, - 'drop': yumi_actions.PlaceAction, - 'move': base_actions.MoveAction - }, - exclude_frames={ - 'pick': [self.robot_frame, self.default_frame], - 'place': [], - 'drop': [], - 'move': [self.robot_frame] - } - ) - return self.demonstrations.n_demonstrations() - - def _save_tree( - self, - string_tree: str, - target_directory: str, - behaviors: Any - ) -> None: - """Prepare the BT data for visualization and execution.""" - args = ( - self.child_conn, - app.CloudpickleWrapper(app.Application), - self.agx_args, - target_directory, - string_tree, - self.obj_data, - behaviors, - self.lfd_interface - ) - app.GRAPHICS = False - process = mp.Process(target=proc.bt_saver, args=args) - process.start() - self.parent_conn.send('') - self.bt_length = self.parent_conn.recv() - process.join() - print('Behavior Tree generated!') - - def _get_tree_from_file(self) -> None: - """Load an existing BT from a file.""" - # Use a learnt BT - bt_description = os.path.join(self.lfd_tree_folder, 'tree.yaml') - try: - with open(bt_description) as f: - string_tree = yaml.safe_load(f) - behaviors = RobotBehaviors(os.path.join( - self.lfd_tree_folder, 'settings'), self.task_objects) - self._save_tree(string_tree, self.lfd_tree_folder, behaviors) - except FileNotFoundError: - print('Error in loading BT description.') - - def _learn_tree(self, pbar: gui.ProgressBar) -> str: - """ - Prepare the Behavior Tree and return its string representation. - - The argument post_processing is a function that modifies the learnt BT. - """ - # Learn the BT - if not os.path.isdir(self.lfd_tree_folder): - os.mkdir(self.lfd_tree_folder) - settings_dir = os.path.join(self.lfd_tree_folder, 'settings') - - behaviors = RobotBehaviors(settings_dir, self.task_objects) - # Send the offline interface to the planner to expand the BT - offline_interface = PlanningInterface( - available_objects=self.task_objects, - frames=self.demonstrations.frames, - default_frame=self.demonstrations.default_frame, - random_events=False, - robot_frame=self.robot_frame - ) - - equivalent = find_equivalent_actions( - self.demonstrations, - { - 'pick': yumi_actions.EquivalentPick, - 'place': yumi_actions.EquivalentPlace, - 'drop': yumi_actions.EquivalentPlace, - 'move': base_actions.EquivalentMove - } - ) - - # tree is a PyTree object! - tree = learn_tree( - settings_dir, - equivalent, - behaviors, - offline_interface, - iterations=50, - callback=pbar.update_bar - ) - - string_tree = tree.bt.bt - print(string_tree) - - # extract the subtree moving the object and make it as a GP behavior - if self.initial is not None: - gp_tree = func.process_lfd_bt( - string_tree, - self.gp_par.behavior_lists, - shrink=True, - name='unstack_boxes', - location=self.log_folder - ) - elif self.demonstrations.n_demonstrations() >= 3: - gp_tree = func.process_lfd_bt(string_tree, self.gp_par.behavior_lists) - - # in subtree, the string representation of the tree is stored - with open(os.path.join(self.lfd_tree_folder, 'tree.yaml'), 'w') as f: - yaml.dump(gp_tree, f) - - self._save_tree(string_tree, self.lfd_tree_folder, behaviors) - - def _build_tree( - self, - window: gui.Window, - from_file: bool = False, - ) -> None: - """ - Build the Behavior Tree once it is loaded. - - The argument post_processing is a function that modifies the learnt BT. - """ - pbar = window.find_element('__progress__') - pbar.update(visible=True) - window.read(timeout=0) - - if from_file: - self._get_tree_from_file() - else: - self._learn_tree(pbar) - - window.find_element('__n_nodes__').update('Number of nodes: %d' % self.bt_length) - pbar.update(visible=False) - - def __continue_interaction( - self, - window: Any, - has_BT: bool, - has_GP: bool, - has_Demo: bool - ) -> None: - """Some prinouts and reset after interaction.""" - self.agx_GUI.enable_from_flags(window, has_BT, has_GP, has_Demo) - print('Select your next functionality...') - - -def main() -> None: - parser, args, leftover_args = createArgumentParser() - - gui = GUI(args) - gui.show() - - -if __name__ == '__main__': - main() diff --git a/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/combined/gui_common.py b/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/combined/gui_common.py deleted file mode 100644 index bbd88fc0..00000000 --- a/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/combined/gui_common.py +++ /dev/null @@ -1,190 +0,0 @@ -"""Functionalities for the GP+LfD framework.""" - -# Copyright (c) 2022, ABB -# All rights reserved. -# -# Redistribution and use in source and binary forms, with -# or without modification, are permitted provided that -# the following conditions are met: -# -# * Redistributions of source code must retain the -# above copyright notice, this list of conditions -# and the following disclaimer. -# * Redistributions in binary form must reproduce the -# above copyright notice, this list of conditions -# and the following disclaimer in the documentation -# and/or other materials provided with the -# distribution. -# * Neither the name of ABB nor the names of its -# contributors may be used to endorse or promote -# products derived from this software without -# specific prior written permission. -# -# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" -# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE -# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE -# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE -# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL -# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR -# SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER -# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, -# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF -# THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - -import os -import re -import subprocess -from typing import Any, Dict, List - -from behaviors.behavior_lists import BehaviorLists -import bt_learning.gp.genetic_programming as gp -from bt_learning.metalearning.hardcode_metadata import find_task_trees -from simulation.algoryx.gp.fitness_function import Coefficients -from simulation.py_trees_interface import PyTreeParameters -import yaml - - -def init_gp_params(data: Dict, log_path: str) -> gp.GpParameters: - gp_par = gp.GpParameters() - gp_par.save_interval = data['genetic']['save_interval'] - gp_par.ind_start_length = data['genetic']['ind_length'] - gp_par.n_population = data['genetic']['n_population'] - gp_par.n_generations = data['genetic']['n_generations'] - gp_par.rerun_fitness = 0 - gp_par.f_crossover = data['genetic']['p_crossover'] - gp_par.n_offspring_crossover = 2 - gp_par.replace_crossover = False - gp_par.f_mutation = data['genetic']['p_mutation'] - gp_par.n_offspring_mutation = 2 - gp_par.parent_selection = get_selection_from_data( - data['genetic']['selection']) - gp_par.survivor_selection = get_selection_from_data( - data['genetic']['selection']) - gp_par.f_elites = data['genetic']['p_elite'] - gp_par.f_parents = gp_par.f_elites - gp_par.max_multi_mutation = data['genetic']['mutations_per_individual'] - gp_par.mutate_co_offspring = False - gp_par.mutate_co_parents = True - gp_par.mutation_p_add = data['genetic']['mutation_add'] - gp_par.mutation_p_delete = data['genetic']['mutation_delete'] - gp_par.mutation_p_variable = 0.0 - gp_par.mutation_p_replace = 0.0 - gp_par.mutation_p_swap = 0.0 - gp_par.allow_identical = False - gp_par.plot = True - gp_par.verbose = True - gp_par.fig_last_gen = False - gp_par.log_name = get_log_number(log_path) - - return gp_par - - -def get_fitness_coefficients() -> Coefficients: - coeff = Coefficients() - coeff.task_completion = 1000.0 - coeff.task_unstacked = 500.0 - coeff.pos_acc = 0.0 - coeff.depth = 0.0 - coeff.length = 10.0 - coeff.ticks = 0.0 - coeff.failed = 50.0 - coeff.timeout = 30.0 - coeff.hand_not_empty = 0.0 - - return coeff - - -def get_pytree_params(behavior_lists: Any, behaviors: Any) -> PyTreeParameters: - params = PyTreeParameters() - params.behavior_lists = behavior_lists - params.behaviors = behaviors - params.max_ticks = 15 - params.sim_tick_time = 2.5 - params.max_time = 50 - params.max_fails = 1 - params.successes_required = 2 - params.show_world = False - params.verbose = False - - return params - - -def get_selection_from_data(data: str) -> gp.SelectionMethods: - """Read the string and return correct type.""" - if data == 'tournament': - return gp.SelectionMethods.TOURNAMENT - elif data == 'rank': - return gp.SelectionMethods.RANK - elif data == 'elitism': - return gp.SelectionMethods.ELITISM - elif data == 'random': - return gp.SelectionMethods.RANDOM - - -def get_log_number(log_path: str) -> str: - """Return a name for the log folder that is not currently used.""" - number = 1 - while os.path.isdir(os.path.join(log_path, f'logs/log_{number}')): - number += 1 - - return str(number) - - -def display(path_to_img: str) -> None: - """Open the image specified by the path.""" - if os.name == 'nt': # Windows - os.startfile(path_to_img) - else: - opener = 'xdg-open' - subprocess.call([opener, path_to_img]) - - -def process_lfd_bt( - bt: List[str], - bt_list: BehaviorLists = None, - shrink: bool = False, - name: str = 'shrunken_bt', - location: str = os.path.dirname((os.path.abspath(__file__))) -) -> List[str]: - """Extract the subtree moving the object and replace it with a behavior as in the GP pool.""" - subtrees = find_task_trees(bt) - goals = [tree[1] for tree in subtrees] - - move_bts = [] - idx = 1 - - for i, goal in enumerate(goals): - if goal.startswith('object_at'): - match_str = f'^object_at (.+) (.+) (.+)$' - match = re.match(match_str, goal) - action = 'place' - - move_bts += ['move0' + ' ' + str(match[1]) + ' ' + str(match[2]) + ' ' + str(match[3])] - idx += len(subtrees[i]) - new_btlist = BehaviorLists(atomic_fallback_nodes=move_bts) - - elif goal.startswith('object_roughly_at'): - match_str = f'^object_roughly_at (.+) (.+) (.+)$' - match = re.match(match_str, goal) - action = 'drop' - - move_bts += ['move0' + ' ' + str(match[1]) + ' ' + str(match[2]) + ' ' + str(match[3])] - idx += len(subtrees[i]) - new_btlist = BehaviorLists(atomic_fallback_nodes=move_bts) - - # recompose the LfD tree with the changed behavior - gp_bt = ['s('] - gp_bt += move_bts - # close - gp_bt.append(')') - - if shrink: - with open(os.path.join(location, f'{name}.yaml'), 'w+') as f: - yaml.dump(gp_bt, f) - gp_bt = ['s(', f'{name}', ')'] - new_btlist = BehaviorLists(atomic_fallback_nodes=[f'{name}']) - - if bt_list is not None: - bt_list.merge_behaviors(new_btlist) - - return gp_bt diff --git a/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/combined/processes.py b/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/combined/processes.py deleted file mode 100644 index 84b4ac31..00000000 --- a/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/combined/processes.py +++ /dev/null @@ -1,183 +0,0 @@ -"""Script to collect process classes and workers for the GP+LfD framework.""" - -# Copyright (c) 2022, ABB -# All rights reserved. -# -# Redistribution and use in source and binary forms, with -# or without modification, are permitted provided that -# the following conditions are met: -# -# * Redistributions of source code must retain the -# above copyright notice, this list of conditions -# and the following disclaimer. -# * Redistributions in binary form must reproduce the -# above copyright notice, this list of conditions -# and the following disclaimer in the documentation -# and/or other materials provided with the -# distribution. -# * Neither the name of ABB nor the names of its -# contributors may be used to endorse or promote -# products derived from this software without -# specific prior written permission. -# -# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" -# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE -# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE -# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE -# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL -# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR -# SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER -# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, -# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF -# THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - -import multiprocessing as mp -import os -from typing import Any, Dict, List - -import bt_learning.learning_from_demo.render_tree as tree_render -from py_trees.display import render_dot_tree -from simulation.algoryx.agx_application import CloudpickleWrapper -from simulation.algoryx.gp.gp_environment import GPEnvironment -from simulation.py_trees_interface import PyTree, PyTreeParameters -import yaml - - -def btlist_worker( - env_fn_wrapper: CloudpickleWrapper, - agx_args: List[str], - obj_data: Dict, - sim_data: Dict, - target_dir: str, - interface: Any -) -> None: - app = env_fn_wrapper.var(agx_args) - app.bringup(obj_data, visual=False) - actions = sim_data['demonstration']['grasping_actions'] +\ - sim_data['demonstration']['placing_actions'] - random_bringup = sim_data['algoryx']['random_bringup'] - # _ = interface.generate_behavior_list(target_dir, actions) - _ = interface.generate_pickplace_list(target_dir, random_bringup) - app.shutdown() - - -def bt_worker( - env_fn_wrapper: CloudpickleWrapper, - agx_args: List[str], - target_directory: str, - string_tree: str, - data: Dict, - behaviors: Any, - interface: Any -) -> None: - app = env_fn_wrapper.var(agx_args) - app.bringup(data, visual=False) - interface.initialize() - parameters = PyTreeParameters() - parameters.behavior_lists = behaviors.get_behavior_list() - parameters.behaviors = behaviors - bt = PyTree( - string_tree, parameters, world_interface=interface) - - with open( - os.path.join(target_directory, 'tree.yaml'), 'w') as f: - yaml.dump(bt.bt.bt, f) - app.shutdown() - - -def bt_saver( - conn: mp.connection.Connection, - env_fn_wrapper: CloudpickleWrapper, - agx_args: List[str], - target_directory: str, - string_tree: str, - data: Dict, - behaviors: Any, - interface: Any -) -> None: - app = env_fn_wrapper.var(agx_args) - app.bringup(data, visual=False) - interface.initialize() - parameters = PyTreeParameters() - parameters.behavior_lists = behaviors.get_behavior_list() - parameters.behaviors = behaviors - bt = PyTree( - string_tree, parameters, world_interface=interface) - - render_dot_tree( - bt.root, - name='debug_pytree', - target_directory=target_directory, - with_blackboard_variables=True - ) - - if conn.poll(): - data = conn.recv() - conn.send(bt.bt.length()) - - tree_render.dot_graph(bt).write_svg( - os.path.join(target_directory, 'full.svg'), encoding='utf-8') - positions = tree_render.write_tikz_tree( - bt, os.path.join(target_directory, 'full.tex')) - tree_render.py_trees_dot(bt).write_svg( - os.path.join(target_directory, 'full_pytrees.svg'), encoding='utf-8') - - with open(os.path.join(target_directory, 'positions.yaml'), 'w') as f: - yaml.dump(positions, f) - - print('BT data saved!') - app.shutdown() - - -class SimProcess(mp.Process): - def __init__(self, args: Any = None) -> None: - self.event = mp.Event() - super().__init__(target=self.worker, args=args) - - def stop(self) -> None: - self.event.set() - self.join() - - def worker( - self, - env_fn_wrapper: CloudpickleWrapper, - agx_args: List[str], - data: Dict, - target: Dict = None - ) -> None: - """Show the simulation environment.""" - app = env_fn_wrapper.var(agx_args) - app.bringup(data, target, True) - while True: - if self.event.is_set(): - app.shutdown() - break - continue - - -class GetFitness(mp.Process): - def __init__( - self, - environment: GPEnvironment, - args: Any = None - ) -> None: - self.data = mp.Value('d', float('-inf')) - self.env = environment - super().__init__(target=self.worker, args=args) - - def get_value(self) -> float: - return self.data.value - - def worker( - self, - env_fn_wrapper: CloudpickleWrapper, - agx_args: List[str], - data: Dict, - individual: List[str] - ) -> None: - """Show the simulation environment.""" - app = env_fn_wrapper.var(agx_args) - app.bringup(data, visual=False) - self.env.set_application(app) - - self.data.value = self.env.get_fitness(individual) diff --git a/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/config/gp_initial.yaml b/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/config/gp_initial.yaml deleted file mode 100644 index 958a6208..00000000 --- a/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/config/gp_initial.yaml +++ /dev/null @@ -1,15 +0,0 @@ -# Stack -KittingBox: - pose: [0.0, 0.0, 0.377] - reference: 'Table3' -YellowBox: - pose: [0.0, 0.0, 0.425] - reference: 'Table1' -BlueBox: - pose: [0.0, 0.0, 0.1] - reference: 'YellowBox' -GreenBox: - pose: [0.0, 0.0, 0.1] - reference: 'BlueBox' - - diff --git a/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/config/gp_targets.yaml b/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/config/gp_targets.yaml deleted file mode 100644 index e0642f1f..00000000 --- a/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/config/gp_targets.yaml +++ /dev/null @@ -1,32 +0,0 @@ -# Stack -YellowBox: - pose: [0.0, 0.0, 0.052] - reference: 'KittingBox' -GreenBox: - pose: [0.0, 0.0, 0.1] - reference: 'YellowBox' -BlueBox: - pose: [0.0, 0.0, 0.1] - reference: 'GreenBox' - -# Sort -# YellowBox: -# pose: [-0.133, 0.0, 0.052] -# reference: 'KittingBox' -# BlueBox: -# pose: [0.0, 0.0, 0.052] -# reference: 'KittingBox' -# GreenBox: -# pose: [0.133, 0.0, 0.052] -# reference: 'KittingBox' - -# Pyramid -# YellowBox: -# pose: [-0.133, 0.0, 0.052] -# reference: 'KittingBox' -# BlueBox: -# pose: [0.0, 0.0, 0.052] -# reference: 'KittingBox' -# GreenBox: -# pose: [-0.065, 0.0, 0.152] -# reference: 'KittingBox' \ No newline at end of file diff --git a/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/config/sim_data.yaml b/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/config/sim_data.yaml deleted file mode 100644 index 959846fd..00000000 --- a/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/config/sim_data.yaml +++ /dev/null @@ -1,32 +0,0 @@ - -demonstration: - default_frame: 'base_link' - robot_frame: 'base_link' - reference_frames: ['KittingBox'] - base_frame: '' - grasping_actions: ['pick'] - placing_actions: ['place'] - navigation_actions: [''] - -genetic: - n_generations: 100 - n_population: 32 - save_interval: 20 - ind_length: 4 - p_crossover: 0.25 - p_mutation: 0.75 - mutation_add: 0.1 - mutation_delete: 0.3 - mutations_per_individual: 3 - p_elite: 0.1 - selection: 'tournament' - baseline_index: 1 - graphics: False - -behavior_tree: - tick_freq: 1.0 - -algoryx: - ee_name: 'GripperControlBody' - robot_name: 'RobotControlBody' - random_bringup: False \ No newline at end of file diff --git a/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/config/sim_objects.yaml b/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/config/sim_objects.yaml deleted file mode 100644 index a428b4e5..00000000 --- a/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/config/sim_objects.yaml +++ /dev/null @@ -1,15 +0,0 @@ -YellowBox: - type: 'box' - reference: 'Table1' - size: 0.1 - color: 'yellow' -GreenBox: - type: 'box' - reference: 'Table2' - size: 0.1 - color: 'green' -BlueBox: - type: 'box' - reference: 'Table1' - size: 0.1 - color: 'blue' \ No newline at end of file diff --git a/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/gp/BT_SETTINGS.yaml b/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/gp/BT_SETTINGS.yaml deleted file mode 100644 index c11081a0..00000000 --- a/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/gp/BT_SETTINGS.yaml +++ /dev/null @@ -1,72 +0,0 @@ -action_nodes: -- close_gripper -- open_gripper -atomic_fallback_nodes: -- move0 YellowBox -0.12 0.0 0.12 0.02 GreenBox -- move0 YellowBox 0.12 0.0 0.12 0.02 GreenBox -- move0 YellowBox 0.0 -0.12 0.12 0.02 GreenBox -- move0 YellowBox 0.0 0.12 0.12 0.02 GreenBox -- move0 YellowBox 0.0 0.0 0.22 0.02 GreenBox -- move0 YellowBox -0.12 0.0 0.12 0.02 BlueBox -- move0 YellowBox 0.12 0.0 0.12 0.02 BlueBox -- move0 YellowBox 0.0 -0.12 0.12 0.02 BlueBox -- move0 YellowBox 0.0 0.12 0.12 0.02 BlueBox -- move0 YellowBox 0.0 0.0 0.22 0.02 BlueBox -- move0 GreenBox -0.12 0.0 0.12 0.02 YellowBox -- move0 GreenBox 0.12 0.0 0.12 0.02 YellowBox -- move0 GreenBox 0.0 -0.12 0.12 0.02 YellowBox -- move0 GreenBox 0.0 0.12 0.12 0.02 YellowBox -- move0 GreenBox 0.0 0.0 0.22 0.02 YellowBox -- move0 GreenBox -0.12 0.0 0.12 0.02 BlueBox -- move0 GreenBox 0.12 0.0 0.12 0.02 BlueBox -- move0 GreenBox 0.0 -0.12 0.12 0.02 BlueBox -- move0 GreenBox 0.0 0.12 0.12 0.02 BlueBox -- move0 GreenBox 0.0 0.0 0.22 0.02 BlueBox -- move0 BlueBox -0.12 0.0 0.12 0.02 YellowBox -- move0 BlueBox 0.12 0.0 0.12 0.02 YellowBox -- move0 BlueBox 0.0 -0.12 0.12 0.02 YellowBox -- move0 BlueBox 0.0 0.12 0.12 0.02 YellowBox -- move0 BlueBox 0.0 0.0 0.22 0.02 YellowBox -- move0 BlueBox -0.12 0.0 0.12 0.02 GreenBox -- move0 BlueBox 0.12 0.0 0.12 0.02 GreenBox -- move0 BlueBox 0.0 -0.12 0.12 0.02 GreenBox -- move0 BlueBox 0.0 0.12 0.12 0.02 GreenBox -- move0 BlueBox 0.0 0.0 0.22 0.02 GreenBox -- move0 YellowBox -0.133 -0.133 0.175 0.02 KittingBox -- move0 YellowBox -0.133 0.0 0.175 0.02 KittingBox -- move0 YellowBox -0.133 0.133 0.175 0.02 KittingBox -- move0 YellowBox 0.0 -0.133 0.175 0.02 KittingBox -- move0 YellowBox 0.0 0.0 0.175 0.02 KittingBox -- move0 YellowBox 0.0 0.133 0.175 0.02 KittingBox -- move0 YellowBox 0.133 -0.133 0.175 0.02 KittingBox -- move0 YellowBox 0.133 0.0 0.175 0.02 KittingBox -- move0 YellowBox 0.133 0.133 0.175 0.02 KittingBox -- move0 GreenBox -0.133 -0.133 0.175 0.02 KittingBox -- move0 GreenBox -0.133 0.0 0.175 0.02 KittingBox -- move0 GreenBox -0.133 0.133 0.175 0.02 KittingBox -- move0 GreenBox 0.0 -0.133 0.175 0.02 KittingBox -- move0 GreenBox 0.0 0.0 0.175 0.02 KittingBox -- move0 GreenBox 0.0 0.133 0.175 0.02 KittingBox -- move0 GreenBox 0.133 -0.133 0.175 0.02 KittingBox -- move0 GreenBox 0.133 0.0 0.175 0.02 KittingBox -- move0 GreenBox 0.133 0.133 0.175 0.02 KittingBox -- move0 BlueBox -0.133 -0.133 0.175 0.02 KittingBox -- move0 BlueBox -0.133 0.0 0.175 0.02 KittingBox -- move0 BlueBox -0.133 0.133 0.175 0.02 KittingBox -- move0 BlueBox 0.0 -0.133 0.175 0.02 KittingBox -- move0 BlueBox 0.0 0.0 0.175 0.02 KittingBox -- move0 BlueBox 0.0 0.133 0.175 0.02 KittingBox -- move0 BlueBox 0.133 -0.133 0.175 0.02 KittingBox -- move0 BlueBox 0.133 0.0 0.175 0.02 KittingBox -- move0 BlueBox 0.133 0.133 0.175 0.02 KittingBox -atomic_sequence_nodes: [] -condition_nodes: -- gripper_state closed -- in_gripper none -- gripper_state open -fallback_nodes: -- f( -sequence_nodes: -- s( -up_node: -- ) diff --git a/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/gp/__init__.py b/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/gp/__init__.py deleted file mode 100644 index 8b137891..00000000 --- a/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/gp/__init__.py +++ /dev/null @@ -1 +0,0 @@ - diff --git a/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/gp/fitness_function.py b/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/gp/fitness_function.py deleted file mode 100644 index 6ef6a1c4..00000000 --- a/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/gp/fitness_function.py +++ /dev/null @@ -1,105 +0,0 @@ -"""Task dependent cost function.""" - -# Copyright (c) 2022, ABB -# All rights reserved. -# -# Redistribution and use in source and binary forms, with -# or without modification, are permitted provided that -# the following conditions are met: -# -# * Redistributions of source code must retain the -# above copyright notice, this list of conditions -# and the following disclaimer. -# * Redistributions in binary form must reproduce the -# above copyright notice, this list of conditions -# and the following disclaimer in the documentation -# and/or other materials provided with the -# distribution. -# * Neither the name of ABB nor the names of its -# contributors may be used to endorse or promote -# products derived from this software without -# specific prior written permission. -# -# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" -# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE -# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE -# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE -# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL -# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR -# SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER -# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, -# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF -# THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - -from dataclasses import dataclass -from typing import Any, Dict - -from simulation.py_trees_interface import PyTree -from simulation.algoryx.gp.gp_interface import GPInterface - - -@dataclass -class Coefficients: - """Coefficients for tuning the cost function.""" - task_completion: float = 1000.0 - task_unstacked = 0.0 - pos_acc: float = 0.0 - depth: float = 0.0 - length: float = 10.0 - ticks: float = 0.0 - failed: float = 50.0 - timeout: float = 10.0 - hand_not_empty: float = 0.0 - - -def compute_fitness( - world_interface: GPInterface, - behavior_tree: PyTree, - ticks: int, - targets: Dict, - coeff: Coefficients = None, - verbose: bool = False -): - # pylint: disable=too-many-arguments - """Retrieve values and compute cost.""" - if coeff is None: - coeff = Coefficients() - - depth = behavior_tree.bt.depth() - length = behavior_tree.bt.length() - - cost = coeff.length * length + \ - coeff.depth * depth + \ - coeff.ticks * ticks - if verbose: - print('Cost from length:', cost) - for i, target in enumerate(targets.keys()): - target_pose = world_interface.as_vec3(targets[target]['pose']) - position, _ = world_interface.get_item_in_frame(target, targets[target]['reference']) - pose = world_interface.as_vec3(position) - cost += coeff.task_completion * \ - max(0, world_interface._distance(target_pose, pose) - coeff.pos_acc) - if verbose: - print('Cost:', cost) - print(i, ': ', position) - print('Distance', world_interface._distance(target_pose, pose)) - - if not world_interface.unstacked: - cost += coeff.task_unstacked - if verbose: - print('Failed: ', cost) - - if behavior_tree.failed: - cost += coeff.failed - if verbose: - print('Failed: ', cost) - if behavior_tree.timeout: - cost += coeff.timeout - if verbose: - print('Timed out: ', cost) - if world_interface.holding is not None and world_interface.holding != '': - cost += coeff.hand_not_empty - if verbose: - print('Hand not empty: ', cost) - fitness = -cost - return fitness diff --git a/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/gp/gp_asprocess.py b/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/gp/gp_asprocess.py deleted file mode 100644 index d42491d4..00000000 --- a/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/gp/gp_asprocess.py +++ /dev/null @@ -1,298 +0,0 @@ -"""A genetic programming algorithm with many possible settings.""" - -# Copyright (c) 2022, ABB -# All rights reserved. -# -# Redistribution and use in source and binary forms, with -# or without modification, are permitted provided that -# the following conditions are met: -# -# * Redistributions of source code must retain the -# above copyright notice, this list of conditions -# and the following disclaimer. -# * Redistributions in binary form must reproduce the -# above copyright notice, this list of conditions -# and the following disclaimer in the documentation -# and/or other materials provided with the -# distribution. -# * Neither the name of ABB nor the names of its -# contributors may be used to endorse or promote -# products derived from this software without -# specific prior written permission. -# -# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" -# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE -# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE -# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE -# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL -# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR -# SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER -# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, -# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF -# THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - -import multiprocessing as mp -import os -from typing import Any, List - -from bt_learning.gp import logplot -import bt_learning.gp.genetic_programming as gp -from bt_learning.gp.hash_table import HashTable -from simulation.algoryx.agx_application import CloudpickleWrapper -import yaml - - -class GPProcess(mp.Process): - - def __init__( - self, - config_folder: str, - environment: Any, - gp_par: gp.GpParameters, - hotstart: bool = False, - baseline: Any = None, - baseline_index: int = None, - visual: bool = False, - args: Any = None - ) -> None: - - with open(os.path.join(config_folder, 'sim_data.yaml')) as f: - sim_data = yaml.safe_load(f) - self.ref_objects = sim_data['demonstration']['reference_frames'] - with open(os.path.join(config_folder, 'sim_objects.yaml')) as f: - self.obj_data = yaml.safe_load(f) - - self.gp_par = gp_par - self.hotstart = hotstart - self.baseline = baseline - self.baseline_index = baseline_index - self.visual = visual - - self.event = mp.Event() - self.data = mp.Value('i', 0) - - self.envinronment = environment - - super().__init__(target=self.worker, args=args) - - def get_current(self): - return self.data.value - - def stop(self): - self.event.set() - self.join() - - def worker(self, env_fn_wrapper: CloudpickleWrapper, args: List[str]) -> None: - # pylint: disable=too-many-statements, too-many-locals, too-many-branches - """Run the genetic programming algorithm.""" - app = env_fn_wrapper.var(args) - app.bringup(self.obj_data, visual=self.visual) - self.envinronment.set_application(app) - - hash_table = HashTable(self.gp_par.hash_table_size, self.gp_par.log_name) - - if self.hotstart: - best_fitness, n_episodes, last_generation, population =\ - gp.load_state(self.gp_par.log_name, hash_table) - else: - population = gp.create_population( - self.gp_par.n_population, - self.gp_par.ind_start_length, - self.gp_par.mutation_p_leaf, - self.gp_par.behavior_lists - ) - # logplot.clear_logs(self.gp_par.log_name) - best_fitness = [] - n_episodes = [] - n_episodes.append(hash_table.n_values) - last_generation = 0 - - if self.baseline is not None: - population[0] = self.baseline - baseline_index = 0 - - fitness = [] - for individual in population: - if self.gp_par.verbose: - print(f'Individual: {individual}') - fitness.append(gp.get_fitness( - individual, - hash_table, - self.envinronment, - rerun_fitness=0, - min_episodes=self.gp_par.min_episodes - )) - if self.gp_par.verbose: - print(f'Fitness: {fitness[-1]}') - - if not self.hotstart: - best_fitness.append(max(fitness)) - - if self.gp_par.verbose: - gp.print_population(population, fitness, last_generation) - print('\n') - print('Generation: ', last_generation, ' Best fitness: ', best_fitness[-1]) - print('\n') - - logplot.log_fitness(self.gp_par.log_name, fitness) - logplot.log_population(self.gp_par.log_name, population) - - generation = self.gp_par.n_generations - 1 # In case loop is skipped due to hotstart - for generation in range(last_generation + 1, self.gp_par.n_generations): - self.data.value = generation - if self.gp_par.keep_baseline: - if self.baseline is not None and self.baseline not in population: - # Make sure we are able to source from baseline - population.append(self.baseline) - - if generation > 1: - fitness = [] - for index, individual in enumerate(population): - if self.gp_par.verbose: - print(f'Individual: {individual}') - fitness.append(gp.get_fitness( - individual, - hash_table, - self.envinronment, - self.gp_par.rerun_fitness, - self.gp_par.min_episodes - )) - if self.gp_par.verbose: - print(f'Fitness: {fitness[-1]}') - if self.baseline is not None and individual == self.baseline: - baseline_index = index - if self.gp_par.keep_baseline and self.gp_par.boost_baseline and self.baseline is not None: - baseline_fitness = fitness[baseline_index] - fitness[baseline_index] = max(fitness) - - co_parents = gp.crossover_parent_selection(population, fitness, self.gp_par) - co_offspring = gp.crossover( - population, co_parents, self.gp_par, self.baseline, self.baseline_index) - for offspring in co_offspring: - if self.gp_par.verbose: - print(f'Individual: {offspring}') - fitness.append(gp.get_fitness( - offspring, - hash_table, - self.envinronment, - self.gp_par.rerun_fitness, - self.gp_par.min_episodes - )) - if self.gp_par.verbose: - print(f'Fitness: {fitness[-1]}') - - if self.gp_par.boost_baseline and self.gp_par.boost_baseline_only_co and \ - self.baseline is not None: - # Restore original fitness for survivor selection - fitness[baseline_index] = baseline_fitness - - mutation_parents = gp.mutation_parent_selection( - population, fitness, co_parents, co_offspring, self.gp_par) - mutated_offspring = gp.mutation( - population + co_offspring, mutation_parents, self.gp_par) - for offspring in mutated_offspring: - if self.gp_par.verbose: - print(f'Individual: {offspring}') - fitness.append(gp.get_fitness( - offspring, - hash_table, - self.envinronment, - self.gp_par.rerun_fitness, - self.gp_par.min_episodes - )) - if self.gp_par.verbose: - print(f'Fitness: {fitness[-1]}') - - if self.gp_par.boost_baseline and self.baseline is not None: - # Restore original fitness for survivor selection - fitness[baseline_index] = baseline_fitness - - population, fitness = gp.survivor_selection( - population, fitness, co_offspring, mutated_offspring, self.gp_par) - - best_fitness.append(max(fitness)) - n_episodes.append(hash_table.n_values) - - logplot.log_fitness(self.gp_par.log_name, fitness) - logplot.log_population(self.gp_par.log_name, population) - - if self.gp_par.verbose: - print('\n') - print( - 'Generation: ', generation, - ' Fitness: ', fitness, - ' Best fitness: ', best_fitness[generation] - ) - print('\n') - - if self.event.is_set(): - print('\n') - print( - 'Generation: ', generation, - ' Fitness: ', fitness, - ' Best fitness: ', best_fitness[generation] - ) - print('\n') - gp.save_state( - self.gp_par, - population, - None, - best_fitness, - n_episodes, - self.baseline, - generation, - hash_table - ) - best_individual = gp.selection( - population, fitness, 1, gp.SelectionMethods.ELITISM)[0] - self.__make_plots(n_episodes, population, best_individual, best_fitness) - app.shutdown() - break - - if (generation + 1) % self.gp_par.save_interval == 0 and\ - generation < self.gp_par.n_generations - 1: # Last is saved later - gp.save_state( - self.gp_par, - population, - None, - best_fitness, - n_episodes, - self.baseline, - generation, - hash_table - ) - - print('\nFINAL POPULATION: ') - gp.print_population(population, fitness, generation) - - best_individual = gp.selection(population, fitness, 1, gp.SelectionMethods.ELITISM)[0] - - gp.save_state( - self.gp_par, - population, - best_individual, - best_fitness, - n_episodes, - self.baseline, - generation, - hash_table - ) - - self.__make_plots(n_episodes, population, best_individual, best_fitness) - - def __make_plots(self, n_episodes, population, best_individual, best_fitness): - if self.gp_par.plot: - logplot.plot_fitness(self.gp_par.log_name, best_fitness, n_episodes) - if self.gp_par.fig_best: - self.envinronment.plot_individual( - logplot.get_log_folder(self.gp_par.log_name), - 'best individual', - best_individual - ) - if self.gp_par.fig_last_gen: - for i in range(self.gp_par.n_population): - self.envinronment.plot_individual( - logplot.get_log_folder(self.gp_par.log_name), - 'individual_' + str(i), - population[i] - ) diff --git a/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/gp/gp_environment.py b/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/gp/gp_environment.py deleted file mode 100644 index 67eccb86..00000000 --- a/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/gp/gp_environment.py +++ /dev/null @@ -1,134 +0,0 @@ -"""Define the objects in the AGX simulaiton.""" - -# Copyright (c) 2022, ABB -# All rights reserved. -# -# Redistribution and use in source and binary forms, with -# or without modification, are permitted provided that -# the following conditions are met: -# -# * Redistributions of source code must retain the -# above copyright notice, this list of conditions -# and the following disclaimer. -# * Redistributions in binary form must reproduce the -# above copyright notice, this list of conditions -# and the following disclaimer in the documentation -# and/or other materials provided with the -# distribution. -# * Neither the name of ABB nor the names of its -# contributors may be used to endorse or promote -# products derived from this software without -# specific prior written permission. -# -# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" -# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE -# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE -# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE -# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL -# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR -# SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER -# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, -# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF -# THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - -from typing import Any, Dict, List, Tuple - -from simulation.py_trees_interface import PyTree, PyTreeParameters -from simulation.algoryx.agx_application import Application -from simulation.algoryx.behaviors.agx_interface import AGXInterface -from simulation.algoryx.gp import fitness_function as fit -import time - - -class GPEnvironment(): - - def __init__( - self, - world_interface: AGXInterface, - behaviors: Any, - initial: Dict, - targets: Dict, - fitness_coeff: fit.Coefficients = None, - pytrees_param: PyTreeParameters = None, - verbose: bool = False - ) -> None: - """ - Interface to the simulation environment for the GP algorithm. - - Args: - ---- - - world_interface: interface to the simulation environment where skills are defined - - behaviors: behaviors that allow the robot to act on the environment - - initial: initial configuration of the objects in the environment - - targets: target configuration of the objects in the environment - - fitness_coeff: coefficient to compute the fitness function - - pytrees_param: parameters to compute BT-related weights in the fitness function - - verbose: toggle printouts - """ - - self.world_interface = world_interface - self.behaviors = behaviors - self.initial = initial - self.targets = targets - self.verbose = verbose - self.fitness_coeff = fitness_coeff - self.pytrees_param = pytrees_param - if self.pytrees_param is not None: - self.pytrees_param.verbose = verbose - - self.app = None - - def set_application(self, application: Application) -> None: - """Set the AGX application to allow control of the simulator.""" - self.app = application - - def get_fitness(self, individual: List[str], seed: int = None) -> float: - """ - Run the simulation and return the fitness. - - In case of error, restarts world_interface and tries again. - """ - self.world_interface.initialize() - if self.pytrees_param is None: - self.pytrees_param = PyTreeParameters() - self.pytrees_param.behavior_lists = self.behaviors.get_behavior_list() - self.pytrees_param.behaviors = self.behaviors - self.pytrees_param.max_ticks = 15 - self.pytrees_param.max_time = 30 - pytree = PyTree( - individual[:], parameters=self.pytrees_param, world_interface=self.world_interface) - - status_ok = True - fitness = None - - self.app.reset(self.initial) - while fitness is None: - if self.verbose: - self.app.start_video() - - ticks, _ = pytree.run_bt(simulator=self.app) - - if self.verbose: - time.sleep(1) - self.app.stop_video() - - fitness = fit.compute_fitness( - self.world_interface, - pytree, - ticks, - self.targets, - self.fitness_coeff, - self.verbose - ) - - return fitness - - def plot_individual(self, path: str, plot_name: str, individual: List[str]) -> None: - """Saves a graphical representation of the individual""" - if self.pytrees_param is None: - self.pytrees_param = PyTreeParameters() - self.pytrees_param.behavior_lists = self.behaviors.get_behavior_list() - self.pytrees_param.behaviors = self.behaviors - pytree = PyTree( - individual[:], parameters=self.pytrees_param, world_interface=self.world_interface) - pytree.save_fig(path, name=plot_name) diff --git a/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/gp/gp_gui.py b/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/gp/gp_gui.py deleted file mode 100644 index 6bafa2b8..00000000 --- a/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/gp/gp_gui.py +++ /dev/null @@ -1,328 +0,0 @@ -""" -Launch the GUI to simulate the GP framework with the AGX Simulator. - -Use with: python gp/gp_gui.py --model models/BTDemo.yml:CoarseGripperInLabDR --decorate --timeStep 0.05 -from the /algoryx folder -""" - -# Copyright (c) 2022, ABB -# All rights reserved. -# -# Redistribution and use in source and binary forms, with -# or without modification, are permitted provided that -# the following conditions are met: -# -# * Redistributions of source code must retain the -# above copyright notice, this list of conditions -# and the following disclaimer. -# * Redistributions in binary form must reproduce the -# above copyright notice, this list of conditions -# and the following disclaimer in the documentation -# and/or other materials provided with the -# distribution. -# * Neither the name of ABB nor the names of its -# contributors may be used to endorse or promote -# products derived from this software without -# specific prior written permission. -# -# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" -# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE -# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE -# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE -# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL -# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR -# SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER -# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, -# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF -# THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - -import logging -import multiprocessing as mp -import os -import subprocess -import time -from typing import Any, List - -from agxBrick.brickLoaderUtils import createArgumentParser -from behaviors.behavior_lists import BehaviorLists -from bt_learning.gp import logplot -import bt_learning.gp.genetic_programming as gp -import simulation.algoryx.agx_application as app -from simulation.algoryx.behaviors.sim_behaviors import RobotBehaviors -from simulation.algoryx.gp import gp_asprocess, gp_environment, gp_interface -from simulation.py_trees_interface import PyTree, PyTreeParameters -from user_interface.gp_gui import GPGUILayout -import yaml - - -class GP_GUI(): - - def __init__(self, args: Any) -> None: - self._logger = logging.getLogger(__file__) - - self.repo_path = os.path.dirname(os.path.dirname((os.path.abspath(__file__)))) - with open(os.path.join(self.repo_path, 'config/sim_data.yaml')) as f: - self.sim_data = yaml.safe_load(f) - with open(os.path.join(self.repo_path, 'config/sim_objects.yaml')) as f: - self.obj_data = yaml.safe_load(f) - with open(os.path.join(self.repo_path, 'config/gp_targets.yaml')) as f: - targets = yaml.safe_load(f) - - self.task_objects = list(self.obj_data.keys()) - self.ref_objects = self.sim_data['demonstration']['reference_frames'] - app.GRAPHICS = False - self.interface = gp_interface.GPInterface(self.task_objects, self.ref_objects) - - self.agx_GUI = GPGUILayout('Algoryx Simulation for Genetic Programming') - - # bt flags - self.tree_dir = None - self.running = False - self.bt = None - self.viz = None - self.bt_tick_freq = self.sim_data['behavior_tree']['tick_freq'] - - # GP parameters - self.gp_par = gp.GpParameters() - self.gp_par.save_interval = self.sim_data['genetic']['save_interval'] - self.gp_par.ind_start_length = self.sim_data['genetic']['ind_length'] - self.gp_par.n_population = self.sim_data['genetic']['n_population'] - self.gp_par.n_generations = self.sim_data['genetic']['n_generations'] - self.gp_par.f_crossover = self.sim_data['genetic']['p_crossover'] - self.gp_par.n_offspring_crossover = 2 - self.gp_par.replace_crossover = False - self.gp_par.f_mutation = self.sim_data['genetic']['p_mutation'] - self.gp_par.n_offspring_mutation = 2 - self.gp_par.parent_selection = self.__get_selection_from_data( - self.sim_data['genetic']['selection']) - self.gp_par.survivor_selection = self.__get_selection_from_data( - self.sim_data['genetic']['selection']) - self.gp_par.f_elites = self.sim_data['genetic']['p_elite'] - self.gp_par.f_parents = self.gp_par.f_elites - self.gp_par.max_multi_mutation = self.sim_data['genetic']['mutations_per_individual'] - self.gp_par.mutate_co_offspring = False - self.gp_par.mutate_co_parents = True - self.gp_par.mutation_p_add = self.sim_data['genetic']['mutation_add'] - self.gp_par.mutation_p_delete = self.sim_data['genetic']['mutation_delete'] - self.gp_par.mutation_p_variable = 0.0 - self.gp_par.mutation_p_replace = 0.0 - self.gp_par.mutation_p_swap = 0.0 - self.gp_par.rerun_fitness = 0 - self.gp_par.allow_identical = False - self.gp_par.plot = True - self.gp_par.verbose = True - self.gp_par.fig_last_gen = False - self.gp_par.log_name = self.__get_log_number() - - self.args = (app.CloudpickleWrapper(app.Application), args) - - self.gp_path = os.path.join(self.repo_path, 'gp') - # Create behaviors - self.gp_par.behavior_lists = self.generate_behaviors() - - self.behaviors = RobotBehaviors(self.gp_path) - - self.gp_env = gp_environment.GPEnvironment( - self.interface, self.behaviors, targets) - - self.gp_process = None - self.current_gen = 0 - - self.visual = self.sim_data['genetic']['graphics'] - - def show(self) -> None: - """Define the GUI functionalities.""" - self.agx_GUI.initialize_layout() - - window = self.agx_GUI.get_window() - - # Interaction with the GUI Main menu - ret = 0 - while ret is not None: - ret, values = window.read() - - if ret == '__folder_display__' and os.path.isdir(values['__folder_display__']): - try: - id = int(values['__folder_display__'][-2:]) - self.gp_par.log_name = str(id) - except ValueError: - self.gp_par.log_name = values['__folder_display__'][-1] - self.update_gp_values(window) - window.find_element('__run__').update(disabled=False) - window.find_element('__show_tree__').update(disabled=False) - - elif ret == '__start__': - if not self.running: - # disable all elements - window.find_element('__start__').update('Stop GP') - window.find_element('__run__').update(disabled=True) - window.find_element('__show_tree__').update(disabled=True) - - self.running = True - hotstart = True if self.current_gen > 0 else False - app.GRAPHICS = self.visual - self.gp_process = gp_asprocess.GPProcess( - os.path.join(self.repo_path, 'config'), - self.gp_env, - self.gp_par, - hotstart=hotstart, - baseline=self.get_baseline(), - visual=self.visual, - args=self.args - ) - self.gp_process.start() - - else: - self.running = False - window.find_element('__start__').update('Start GP') - window.find_element('__run__').update(disabled=False) - window.find_element('__show_tree__').update(disabled=False) - - self.gp_process.stop() - self.gp_process.join() - print(f'Stopped at: {self.current_gen}.') - self.update_gp_values(window) - - elif ret == '__run__': - if self.running is False: - # disable all elements - window.find_element('__start__').update(disabled=True) - window.find_element('__show_tree__').update(disabled=True) - - window.find_element('__run__').update('Stop') - print('Launching tree...') - self.running = True - - app.GRAPHICS = True - self.bt_process = app.RunBT( - self.interface, - self.obj_data, - self.behaviors, - os.path.join(self.repo_path, f'logs/log_{self.gp_par.log_name}'), - self.bt_tick_freq, - self.args - ) - self.bt_process.start() - - else: - self.running = False - self.bt_process.stop() - self.bt_process.join() - window.find_element('__run__').update('Run') - window.find_element('__start__').update(disabled=False) - window.find_element('__show_tree__').update(disabled=False) - - elif ret == '__show_tree__': - path_to_best = os.path.join( - self.repo_path, f'logs/log_{self.gp_par.log_name}/best_individual.svg') - if os.name == 'nt': # Windows - os.startfile(path_to_best) - else: - opener = 'xdg-open' - subprocess.call([opener, path_to_best]) - - def update_gp_values(self, window) -> None: - try: - self.current_gen = logplot.get_state(self.gp_par.log_name)[2] - self.best_fitness = max(logplot.get_best_fitness(self.gp_par.log_name)) - self.best_individual = logplot.get_best_individual(self.gp_par.log_name) - - window.find_element('__gen__').update( - 'Current generation: %d' % self.current_gen) - window.find_element('__fitness__').update( - 'Best fitness score: %d' % self.best_fitness) - - proc = mp.Process(target=self._bt_worker, args=self.args) - proc.start() - time.sleep(5) - print('Behavior Tree generated!') - except AttributeError: - window.find_element('__gen__').update( - 'Current generation: %d' % self.current_gen) - window.find_element('__fitness__').update( - 'Best fitness score: -inf') - - def get_baseline(self) -> List[str] or None: - """Get the baseline from the given folder.""" - try: - best_individual = logplot.get_best_individual(self.gp_par.log_name) - return best_individual - except (FileNotFoundError, UnicodeDecodeError): - print('Error in loading the baseline.') - return None - - def generate_behaviors(self) -> BehaviorLists or None: - """Generate behaviors for the GP parameters.""" - proc = mp.Process(target=self._btlist_worker, args=self.args) - proc.start() - time.sleep(5) - print('Behaviors generated!') - try: - with open(os.path.join(self.gp_path, 'BT_SETTINGS.yaml')) as f: - settings = yaml.safe_load(f) - behavior_list = BehaviorLists( - settings['fallback_nodes'], - settings['atomic_fallback_nodes'], - settings['sequence_nodes'], - settings['atomic_sequence_nodes'], - settings['condition_nodes'], - settings['action_nodes'] - ) - return behavior_list - except FileNotFoundError: - print('Error in loading the behavior lists.') - return None - - def _bt_worker(self, env_fn_wrapper: app.CloudpickleWrapper, args: List[str]) -> None: - app = env_fn_wrapper.var(args) - app.bringup(self.obj_data, visual=False) - self.interface.initialize() - parameters = PyTreeParameters() - parameters.behavior_lists = self.behaviors.get_behavior_list() - parameters.behaviors = self.behaviors - bt = PyTree( - self.best_individual, parameters, world_interface=self.interface) - with open( - os.path.join( - self.repo_path, f'logs/log_{self.gp_par.log_name}/tree.yaml'), 'w') as f: - yaml.dump(bt.bt.bt, f) - app.shutdown() - - def _btlist_worker(self, env_fn_wrapper: app.CloudpickleWrapper, args: List[str]) -> None: - app = env_fn_wrapper.var(args) - app.bringup(self.obj_data, visual=False) - actions = self.sim_data['demonstration']['grasping_actions'] + \ - self.sim_data['demonstration']['placing_actions'] - # _ = self.interface.generate_behavior_list(self.gp_path, actions) - _ = self.interface.generate_pickplace_list(self.gp_path) - app.shutdown() - - def __get_selection_from_data(self, data: str) -> gp.SelectionMethods: - """Read the string and return correct type.""" - if data == 'tournament': - return gp.SelectionMethods.TOURNAMENT - elif data == 'rank': - return gp.SelectionMethods.RANK - elif data == 'elitism': - return gp.SelectionMethods.ELITISM - elif data == 'random': - return gp.SelectionMethods.RANDOM - - def __get_log_number(self) -> str: - """Return a name for the log folder that is not currently used.""" - number = 1 - while os.path.isdir(os.path.join(self.repo_path, f'logs/log_{number}')): - number += 1 - - return str(number) - - -def main() -> None: - parser, args, leftover_args = createArgumentParser() - - gui = GP_GUI(args) - gui.show() - - -if __name__ == '__main__': - main() diff --git a/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/gp/gp_interface.py b/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/gp/gp_interface.py deleted file mode 100644 index 692c953b..00000000 --- a/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/gp/gp_interface.py +++ /dev/null @@ -1,307 +0,0 @@ -"""Interface to generate poses in AGX for the GP.""" - -# Copyright (c) 2022, ABB -# All rights reserved. -# -# Redistribution and use in source and binary forms, with -# or without modification, are permitted provided that -# the following conditions are met: -# -# * Redistributions of source code must retain the -# above copyright notice, this list of conditions -# and the following disclaimer. -# * Redistributions in binary form must reproduce the -# above copyright notice, this list of conditions -# and the following disclaimer in the documentation -# and/or other materials provided with the -# distribution. -# * Neither the name of ABB nor the names of its -# contributors may be used to endorse or promote -# products derived from this software without -# specific prior written permission. -# -# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" -# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE -# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE -# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE -# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL -# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR -# SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER -# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, -# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF -# THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - -from typing import List - -import agx -from behaviors.behavior_lists import BehaviorLists -import numpy as np -from simulation.algoryx.behaviors.agx_interface import AGXInterface -import yaml - - -""" -The string representations of the behaviors are: - - pick object - - place object x y z tolerance frame - - open_gripper - - close_gripper - - move x y z tolerance frame - - gripper_state open/closed - - empty_gripper - - in_gripper object - - object_at object x y z tolerance frame - - object_roughly_at object x y z tolerance frame - - robot_at x y z tolerance frame - -""" - - -class GPInterface(AGXInterface): - """An interface for generation of skills in AGX.""" - - def __init__(self, task_objects: List[str], ref_objects: List[str]) -> None: - """Instantiate a new AGX Interface.""" - self.movable = task_objects - self.fixed = ref_objects - - def initialize(self): - super().__init__(self.movable) - - def generate_pickplace_list( - self, - file_path: str, - random: bool = True - ) -> BehaviorLists: - """Generate a behavior list where every element moves a block.""" - super().__init__(self.movable) - new_list = BehaviorLists() - pickplace_list = self.pickplace_list() - new_list.merge_behaviors(pickplace_list) - - if not random: - unstack_list = BehaviorLists( - condition_nodes=['unstacked'] - ) - new_list.merge_behaviors(unstack_list) - - settings = { - 'fallback_nodes': new_list.fallback_nodes, - 'atomic_fallback_nodes': new_list.atomic_fallback_nodes, - 'sequence_nodes': new_list.sequence_nodes, - 'atomic_sequence_nodes': new_list.atomic_sequence_nodes, - 'condition_nodes': new_list.condition_nodes, - 'action_nodes': new_list.action_nodes, - 'up_node': new_list.up_node - } - # Use another destination than the one where the LfD stuff is stored - file_name = file_path + '/BT_SETTINGS.yaml' - with open(file_name, 'w+') as f: - yaml.dump(settings, f) - - return new_list - - def generate_behavior_list( - self, - file_path: str, - actions: List[str], - old_behavior_list: BehaviorLists = None - ) -> BehaviorLists: - """Extend the behavior list by generating poses for each type of behavior.""" - super().__init__(self.movable) - new_list = BehaviorLists() - # Note: merge behaviors removes duplicates - if 'pick' in actions: - pick_list = self.pick_list() - new_list.merge_behaviors(pick_list) - if 'place' in actions: - place_list = self.place_list() - new_list.merge_behaviors(place_list) - if 'drop' in actions: - drop_list = self.place_list(rough=True) - new_list.merge_behaviors(drop_list) - if 'move' in actions: - move_list = self.move_list() - new_list.merge_behaviors(move_list) - - if old_behavior_list is not None: - new_list.merge_behaviors(old_behavior_list) - - settings = { - 'fallback_nodes': new_list.fallback_nodes, - 'sequence_nodes': new_list.sequence_nodes, - 'condition_nodes': new_list.condition_nodes, - 'action_nodes': new_list.action_nodes, - 'up_node': new_list.up_node - } - # Use another destination than the one where the LfD stuff is stored - file_name = file_path + '/BT_SETTINGS.yaml' - with open(file_name, 'w+') as f: - yaml.dump(settings, f) - - return new_list - - def pick_list(self) -> BehaviorLists: - """Create behaviors for picking in the LfD fashion.""" - action_list = [] - condition_list = [] - for name in self.movable: - action_list.append(str('pick0 ' + name)) - condition_list.append(str('in_gripper ' + name)) - - action_list.append('open_gripper') - condition_list.append('gripper_state closed') - - return BehaviorLists(action_nodes=action_list, condition_nodes=condition_list) - - def place_list(self, rough: bool = False) -> BehaviorLists: - """ - Create behaviors for placing in the LfD fashion. - - If rough, behaviors for dropping are created instead. - """ - action_list = [] - condition_list = [] - for target in self.movable: - for reference in self.movable: - if target == reference: - continue - poses = self._sample_cube(reference, rough) - for pose in poses: - if rough: - action_list.append(str('drop0 ' + target + pose + reference)) - condition_list.append( - str('object_roughly_at ' + target + pose + reference)) - else: - action_list.append(str('place0 ' + target + pose + reference)) - condition_list.append(str('object_at ' + target + pose + reference)) - - for target in self.movable: - for reference in self.fixed: - poses = self._sample_box(target, reference, rough) - for pose in poses: - if rough: - action_list.append(str('drop0 ' + target + pose + reference)) - condition_list.append( - str('object_roughly_at ' + target + pose + reference)) - else: - action_list.append(str('place0 ' + target + pose + reference)) - condition_list.append(str('object_at ' + target + pose + reference)) - - action_list.append('close_gripper') - condition_list.append('in_gripper none') - condition_list.append('gripper_state open') - - return BehaviorLists(action_nodes=action_list, condition_nodes=condition_list) - - def pickplace_list(self, rough: bool = False) -> BehaviorLists: - """ - Create behaviors for placing in the LfD fashion. - - If rough, behaviors for dropping are created instead. - """ - atomic_fallback_nodes = [] - for target in self.movable: - for reference in self.movable: - if target == reference: - continue - poses = self._sample_cube(reference, rough) - for pose in poses: - if rough: - atomic_fallback_nodes.append( - str('move_roughly0 ' + target + pose + reference)) - else: - atomic_fallback_nodes.append(str('move0 ' + target + pose + reference)) - - for target in self.movable: - for reference in self.fixed: - poses = self._sample_box(target, reference, rough) - for pose in poses: - if rough: - atomic_fallback_nodes.append( - str('move_roughly0 ' + target + pose + reference)) - else: - atomic_fallback_nodes.append(str('move0 ' + target + pose + reference)) - - action_list = ['close_gripper', 'open_gripper'] - condition_list = ['gripper_state closed', 'in_gripper none', 'gripper_state open'] - - return BehaviorLists( - atomic_fallback_nodes=atomic_fallback_nodes, - condition_nodes=condition_list, - action_nodes=action_list) - - def move_list(self) -> BehaviorLists: - pass - - def _sample_cube(self, reference: str, rough: bool = False) -> List[str]: - """ - Sample 5 poses adjacent to the reference object. - - Above, Left, Below, Right, On Top. - """ - obj_ = self.sim.getRigidBody(reference) - # origin = self.__round(obj_.getPosition()) - size = obj_.getGeometries()[0].getShape().asBox().getHalfExtents()[0]*2 - tolerance = 0.02 - if rough: - tolerance = 0.05 - - side_offset = 0.02 - - pose_list = [] - for x_step in [-1, 1]: - pose = self.__round( - agx.Vec3(x_step*(size + side_offset), 0, self.gripper_offset)) - pose_list.append(' ' + ' '.join(str(x) for x in pose) + ' ' + str(tolerance) + ' ') - for y_step in [-1, 1]: - pose = self.__round( - agx.Vec3(0, y_step*(size + side_offset), self.gripper_offset)) - pose_list.append(' ' + ' '.join(str(x) for x in pose) + ' ' + str(tolerance) + ' ') - - plus_z = self.__round(agx.Vec3(0, 0, size + self.gripper_offset)) - pose_list.append(' ' + ' '.join(str(x) for x in plus_z) + ' ' + str(tolerance) + ' ') - - return pose_list - - def _sample_box(self, target: str, reference: str, rough: bool = False) -> List[str]: - """ - Sample 5 poses adjacent to the reference kit box. - - Above, Left, Below, Right, On Top. - """ - tg_obj_ = self.sim.getRigidBody(target) - size = tg_obj_.getGeometries()[0].getShape().asBox().getHalfExtents()[0]*2 - ref_obj_ = self.sim.getRigidBody(reference) - bottom_plate_geom = ref_obj_.getGeometries()[0].getShape().asBox().getHalfExtents() - length = bottom_plate_geom[0]*2 - depth = bottom_plate_geom[1]*2 - height = bottom_plate_geom[2]*2 - dx = round(length/3, 3) - dy = round(depth/3, 3) - dz = round(height + size/2, 3) - tolerance = 0.02 - if rough: - tolerance = 0.05 - - pose_list = [] - for x_step in [-1, 0, 1]: - for y_step in [-1, 0, 1]: - pose = self.__round(agx.Vec3(x_step*dx, y_step*dy, self.gripper_offset + dz)) - pose_list.append(' ' + ' '.join(str(x) for x in pose) + ' ' + str(tolerance) + ' ') - - return pose_list - - def __get_fixed_objs(self) -> List[str]: - """Return the names of the RigidBodies in the scene.""" - # This is the same function in the Environment class - bodies = self.sim.getRigidBodies() - names = [] - for body in bodies: - if body not in self.movable: - names.append(str(body.getName())) - return names - - def __round(self, vec3: agx.Vec3) -> agx.Vec3: - """Round function for Vec3.""" - return agx.Vec3(round(vec3.x(), 3), round(vec3.y(), 3), round(vec3.z(), 3)) diff --git a/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/lfd/__init__.py b/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/lfd/__init__.py deleted file mode 100644 index 8b137891..00000000 --- a/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/lfd/__init__.py +++ /dev/null @@ -1 +0,0 @@ - diff --git a/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/lfd/lfd_asprocess.py b/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/lfd/lfd_asprocess.py deleted file mode 100644 index 768ccc3f..00000000 --- a/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/lfd/lfd_asprocess.py +++ /dev/null @@ -1,118 +0,0 @@ -"""A LfD application with Algoryx through multiprocessing.""" - -# Copyright (c) 2022, ABB -# All rights reserved. -# -# Redistribution and use in source and binary forms, with -# or without modification, are permitted provided that -# the following conditions are met: -# -# * Redistributions of source code must retain the -# above copyright notice, this list of conditions -# and the following disclaimer. -# * Redistributions in binary form must reproduce the -# above copyright notice, this list of conditions -# and the following disclaimer in the documentation -# and/or other materials provided with the -# distribution. -# * Neither the name of ABB nor the names of its -# contributors may be used to endorse or promote -# products derived from this software without -# specific prior written permission. -# -# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" -# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE -# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE -# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE -# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL -# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR -# SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER -# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, -# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF -# THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - -import multiprocessing as mp -import os -from typing import Any, Dict, List - -from simulation.algoryx.agx_application import CloudpickleWrapper -from simulation.algoryx.lfd import user_interaction -import yaml - - -class LfDProcess(mp.Process): - - def __init__( - self, - config_folder: str, - interface: Any, - args: Any = None - ) -> None: - - with open(os.path.join(config_folder, 'sim_data.yaml')) as f: - sim_data = yaml.safe_load(f) - self.ref_objects = sim_data['demonstration']['reference_frames'] - with open(os.path.join(config_folder, 'sim_objects.yaml')) as f: - self.obj_data = yaml.safe_load(f) - - self.event = mp.Event() - - self.interface = interface - - super().__init__(target=self.worker, args=args) - - def stop(self): - self.event.set() - self.join() - - def worker( - self, - conn: mp.connection.Connection, - env_fn_wrapper: CloudpickleWrapper, - args: List[str], - targets: Dict = None - ) -> None: - """Run the genetic programming algorithm.""" - def user_control(app: Any, holding: str) -> None: - self.user_input.running = True - self.user_input.add_text() - self.user_input.setup_keyboard_listener() - while self.user_input.running: - app.step(0.5) - self.interface._release(holding) - return None - - app = env_fn_wrapper.var(args) - app.bringup(self.obj_data, targets, visual=True) - app.reset(targets) - self.interface.initialize() - - self.interface.remove_locks() - self.interface.preempt_skill() - self.interface.add_merger() - - self.user_input = user_interaction.UserControls(self.interface, app) - - running = True - while running: - if self.event.is_set(): - app.shutdown() - break - if conn.poll(): - data = conn.recv() - if data[0] == 'get_sensor_data': - pos, rot = self.interface.get_item_in_frame(data[1][0], data[1][1]) - conn.send((pos, rot)) - elif data[0] == 'pick': - self.interface.pick(data[1]) - app.step(3.0) - conn.send(self.interface.holding) - elif data[0] == 'lift': - self.interface.lift_gripper() - app.step(3.0) - conn.send(True) - else: - user_control(app, self.interface.holding) - app.step(3.0) - self.user_input.clear_text() - conn.send(None) diff --git a/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/lfd/lfd_gui.py b/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/lfd/lfd_gui.py deleted file mode 100644 index 2d3b73e6..00000000 --- a/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/lfd/lfd_gui.py +++ /dev/null @@ -1,517 +0,0 @@ -""" -Launch the GUI to simulate the LfD framework with the AGX Simulator. - -Use with: python lfd/lfd_gui.py --model models/BTDemo.yml:CoarseGripperInLabDR --decorate --timeStep 0.05 -from the /algoryx folder -""" - -# Copyright (c) 2022, ABB -# All rights reserved. -# -# Redistribution and use in source and binary forms, with -# or without modification, are permitted provided that -# the following conditions are met: -# -# * Redistributions of source code must retain the -# above copyright notice, this list of conditions -# and the following disclaimer. -# * Redistributions in binary form must reproduce the -# above copyright notice, this list of conditions -# and the following disclaimer in the documentation -# and/or other materials provided with the -# distribution. -# * Neither the name of ABB nor the names of its -# contributors may be used to endorse or promote -# products derived from this software without -# specific prior written permission. -# -# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" -# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE -# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE -# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE -# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL -# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR -# SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER -# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, -# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF -# THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - -from copy import copy, deepcopy -from distutils.dir_util import copy_tree -import glob -import logging -import os -import subprocess -from tempfile import TemporaryDirectory -import time -from typing import Any - -from agxBrick.brickLoaderUtils import createArgumentParser -from agxClick import AgxApplication -from bt_learning.learning_from_demo.clustering import find_equivalent_actions -from bt_learning.learning_from_demo.debug import BTVisualizer -from bt_learning.learning_from_demo.learning import learn_tree -from bt_learning.learning_from_demo.plot_clusters import plot_clusters -import bt_learning.learning_from_demo.render_tree as tree_render -from py_trees.display import render_dot_tree -import robot_behaviors.mobile_base_behaviors.lfd_actions as base_actions -import robot_behaviors.yumi_behaviors.lfd_actions as yumi_actions -from robot_interface.demonstration import RobotDemonstration -from simulation.algoryx import agx_environment -from simulation.algoryx.behaviors import agx_interface -from simulation.algoryx.behaviors.sim_behaviors import RobotBehaviors -from simulation.algoryx.lfd import user_interaction -from simulation.algoryx.lfd.planning_itnerface import PlanningInterface -from simulation.py_trees_interface import PyTree, PyTreeParameters -from user_interface.gui_ddmenu import GUIMenu -from user_interface.lfd_gui import LfDGUILayout -from user_interface.lfd_gui_actions import GUIActions -import yaml - - -# OS dependent import -if os.name == 'nt': # Windows - import PySimpleGUIQt as gui -elif os.name == 'posix': # Linux Ubuntu - import PySimpleGUI as gui - - -class LFD_GUI(AgxApplication): - - def __init__(self, args: Any): - super().__init__(args) - self._logger = logging.getLogger(__file__) - - repo_path = os.path.dirname(os.path.dirname((os.path.abspath(__file__)))) - with open(os.path.join(repo_path, 'config/sim_data.yaml')) as f: - sim_data = yaml.safe_load(f) - with open(os.path.join(repo_path, 'config/sim_objects.yaml')) as f: - obj_data = yaml.safe_load(f) - - self._env = agx_environment.AGXEnvironment(args, objects=obj_data) - - brickSimulation = self._env.build_scene() - self._env.spawn(brickSimulation) - self._env.create_visual() - self.__step_ahead(2.0) - - self.objects = list(obj_data.keys()) - self.robot_interface = agx_interface.AGXInterface(self.objects) - - self.default_frame = sim_data['demonstration']['default_frame'] - self.robot_frame = sim_data['demonstration']['robot_frame'] - self.ee_frame = sim_data['algoryx']['ee_name'] - - self.agx_GUI = LfDGUILayout('Algoryx Simulation for Learning from Demonstration') - self.all_frames = sim_data['demonstration']['reference_frames'] + self.objects - if self.robot_frame not in self.all_frames: - self.all_frames.append(self.robot_frame) - if self.default_frame not in self.all_frames: - self.all_frames.append(self.default_frame) - self.pick_menu = GUIMenu('Choose the object to pick.', self.objects) - - self.grasping_actions = sim_data['demonstration']['grasping_actions'] - self.placing_actions = sim_data['demonstration']['placing_actions'] - self.navigation_actions = sim_data['demonstration']['navigation_actions'] - - self.actions = self.grasping_actions + self.placing_actions - - # demo flags - self.demo_dir = None - self.demonstrations = None - - # bt flags - self.running = False - self.bt = None - self.viz = None - self.bt_tick_freq = sim_data['behavior_tree']['tick_freq'] - - # action flags - self.holding = None - self.running_action = None - - def show(self, build_tree_function: Any = None): - """Define the GUI functionalities.""" - self.agx_GUI.initialize_layout() - - window = self.agx_GUI.get_window() - - # Interaction with the GUI Main menu - ret = 0 - while ret is not None: - ret, values = window.read() - - if ret == '__folder_display__' and os.path.isdir(values['__folder_display__']): - self.demo_dir = values['__folder_display__'] - self.__reload_demos(window, build_tree_function) - - elif ret == '__new_demo__': - window.disappear() - self.__new_demo() - self.__reload_demos(window, build_tree_function) - window.reappear() - - elif ret == '__run__': - if self.running is False: - # disable all elements - window.find_element('__new_demo__').update(disabled=True) - window.find_element('__demo_folder__').update(disabled=True) - - self.viz = BTVisualizer(self.bt) - - self._env.reset_simulation() - self.robot_interface.set_gripper('open') - - window.find_element('__run__').update('Stop') - print('Launching tree...') - self.running = True - # Run the BT (without ROS so it's hacky) - # The Stop button will not work. - last = time.time() - for _ in range(25): - next = last + self.bt_tick_freq - time.sleep(next - time.time()) # it's ok to sleep negative time - last = next - self.viz.tick() - for _ in range(50): - self.stepApplication() - else: - self.running = False - self._env.reset_simulation() - window.find_element('__new_demo__').update(disabled=False) - window.find_element('__demo_folder__').update(disabled=False) - window.find_element('__run__').update('Run') - - elif ret == '__show_tree__': - if os.name == 'nt': # Windows - os.startfile(os.path.join(self.tree_dir.name, 'full.svg')) - else: - opener = 'xdg-open' - subprocess.call([opener, os.path.join(self.tree_dir.name, 'full.svg')]) - - elif ret == '__save_tree__': - folder = gui.popup_get_folder('Select a folder') - if folder is not None and folder != '': - if os.path.isdir(folder): - number = 1 - while os.path.isdir(os.path.join(folder, f'bt{number}')): - number += 1 - - folder = os.path.join(folder, f'bt{number}') - - os.makedirs(folder) - copy_tree(self.tree_dir.name, folder) - print('Tree saved!') - - elif ret == '__load_tree__': - folder = gui.popup_get_folder('Select a folder') - if folder is not None and folder != '': - if os.path.isdir(folder): - self.tree_dir = folder - - if build_tree_function is None: - self.__build_tree(window, from_file=True) - else: - build_tree_function(window, from_file=True) - - elif ret == '__plot_cluster__': - if self.demonstrations is None: - _ = self.__load_demo() - plot_clusters(self.demo_dir, self.demonstrations) - - def __step_ahead(self, time: float): - """Step forward in time.""" - t1 = self.sim.getTimeStamp() - while self.sim.getTimeStamp() - t1 < time: - self.stepApplication() - - def __new_demo(self): - """Launch a new window to start a new demonstration.""" - action_GUI = GUIActions('New demo', self.actions) - action_GUI.initialize_layout() - - action_window = action_GUI.get_window() - - if self.demonstrations is None: - n_demos = 0 - else: - n_demos = self.demonstrations.n_demonstrations() - - os.mkdir(os.path.join(self.demo_dir, 'demo%d' % (n_demos + 1))) - - self.robot_interface.remove_locks() - self.robot_interface.preempt_skill() - self.robot_interface.add_merger() - - ret = 0 - while ret is not None: - ret, values = action_window.read() - action_GUI.execute_action(ret, self.__manipulate, n_demos) - - def __manipulate(self, n_demos: int, action_name: str, gripper_state: str): - """Define how to run actions when connected to the robot.""" - def get_obj_from_menu(menu: GUIMenu) -> str: - menu_ = deepcopy(menu) - key_ = menu_.initialize_layout() - window = menu_.get_window() - event, values = window.read() - obj = values[key_] - window.close() - return obj - - def user_control(): - user_input = user_interaction.UserControls(self.robot_interface) - user_input.setup_keyboard_listener() - while user_input.running: - self.__step_ahead(0.5) - self.robot_interface._release(self.holding) - self.holding = None - - # Interact with the GUI Action menu - if action_name == 'pick': - # Handle dropdown menu - self.holding = get_obj_from_menu(self.pick_menu) - print(f'Picking {self.holding}.') - self.running_action = self.robot_interface.pick(self.holding) - self.__step_ahead(3.0) - self.__write_action(action_name, n_demos + 1) - elif action_name == 'place': - mainpulating = copy(self.holding) - user_control() - print(f'Placing {mainpulating}.') - self.__write_action(action_name, n_demos + 1) - self.robot_interface.lift_gripper() - self.__step_ahead(3.0) - elif action_name == 'drop': - mainpulating = copy(self.holding) - user_control() - print(f'Dropping {mainpulating}.') - self.__write_action(action_name, n_demos + 1) - self.robot_interface.lift_gripper() - self.__step_ahead(3.0) - - self.robot_interface.set_gripper(gripper_state) - self.robot_interface.preempt_skill() - - def __write_action(self, action_type: str, current_demo: str): - """Write the pose of the end effector in all available frames as a new action.""" - action_data = {'type': action_type, 'vec_pos': {}, 'vec_quat': {}} - current_demo_dir = os.path.join(self.demo_dir, 'demo%d' % current_demo) - if self.demonstrations is None: - frames = self.all_frames - else: - frames = self.demonstrations.frames - - for frame in frames: - # remove the default frame for now - position, orientation = self.robot_interface.get_item_in_frame(self.ee_frame, frame) - - action_data['vec_pos'][frame] = position.tolist() - action_data['vec_quat'][frame] = orientation.tolist() - - action_number = len(glob.glob(os.path.join(current_demo_dir, 'data_*.yaml'))) + 1 - with open(os.path.join(current_demo_dir, f'data_{action_number}.yaml'), 'w') as f: - yaml.dump(action_data, f, default_flow_style=None) - - def __reload_demos(self, window: gui.Window, build_tree_function): - """Update the demonstration history and build the tree.""" - n_demos = self.__load_demo() - window.find_element('__n_demos__').update('Number of demonstrations: %d' % n_demos) - window.find_element('__new_demo__').update(disabled=False) - - if self.demonstrations is None: - return - - if build_tree_function is None: - self.__build_tree(window) - else: - build_tree_function(window) - - def __load_demo(self) -> int: - """Load demonstration and return number of demonstrations.""" - info_file = os.path.join(self.demo_dir, 'info.yaml') - if not os.path.isfile(info_file): - with open(info_file, 'w') as f: - yaml.dump( - { - 'frames': self.all_frames, - 'robot': self.robot_frame, - 'default_frame': self.default_frame - }, f) - self.demonstrations = None - return 0 - elif len(glob.glob(os.path.join(self.demo_dir, 'demo*'))) == 0: - self.demonstrations = None - return 0 - else: - # Remove empty folders - for folder in glob.glob(self.demo_dir + '/demo[0-9]*/'): - # Each demonstration has to contain at least one action - if not os.path.isfile(folder + '/data_1.yaml'): - print(f'Folder {folder} missing data file, removing.') - os.rmdir(folder) - self.demonstrations = RobotDemonstration( - self.demo_dir, - custom_actions={ - 'pick': yumi_actions.PickAction, - 'place': yumi_actions.PlaceAction, - 'drop': yumi_actions.PlaceAction, - 'move': base_actions.MoveAction - }, - exclude_frames={ - 'pick': [self.robot_frame, self.default_frame], - 'place': [], - 'drop': [], - 'move': [self.robot_frame] - } - ) - return self.demonstrations.n_demonstrations() - - def __save_tree( - self, - string_tree: str, - target_directory: str, - behaviors: Any - ): - """Prepare the BT data for visualization and execution.""" - parameters = PyTreeParameters() - parameters.behavior_lists = behaviors.get_behavior_list() - parameters.behaviors = behaviors - self.bt = PyTree( - string_tree, parameters, world_interface=self.robot_interface) - - # Print the BT with the Blackboard variables - render_dot_tree( - self.bt.root, - name='debug_pytree', - target_directory=target_directory, - with_blackboard_variables=True - ) - - tree_render.dot_graph(self.bt).write_svg( - os.path.join(target_directory, 'full.svg'), encoding='utf-8') - positions = tree_render.write_tikz_tree( - self.bt, os.path.join(target_directory, 'full.tex')) - tree_render.py_trees_dot(self.bt).write_svg( - os.path.join(target_directory, 'full_pytrees.svg'), encoding='utf-8') - - with open(os.path.join(target_directory, 'positions.yaml'), 'w') as f: - yaml.dump(positions, f) - - def __prelearning_handler(self, window: gui.Window) -> gui.ProgressBar: - """Update window behavior before learning the tree.""" - window.find_element('__run__').update('Building tree...', disabled=True) - window.find_element('__show_tree__').update(disabled=True) - window.find_element('__save_tree__').update(disabled=True) - window.find_element('__load_tree__').update(disabled=True) - window.find_element('__plot_cluster__').update(disabled=True) - pbar = window.find_element('__progress__') - pbar.update(visible=True) - window.read(timeout=0) - - return pbar - - def __postlearning_handler(self, window: gui.Window, pbar: gui.ProgressBar): - """Update window behavior before learning the tree.""" - window.find_element('__run__').update('Run', disabled=False) - window.find_element('__show_tree__').update(disabled=False) - window.find_element('__save_tree__').update(disabled=False) - window.find_element('__load_tree__').update(disabled=False) - window.find_element('__plot_cluster__').update(disabled=False) - window.find_element('__n_nodes__').update('Number of nodes: %d' % self.bt.bt.length()) - pbar.update(visible=False) - - def __get_tree_from_file(self): - """Load an existing BT from a file.""" - # Use a learnt BT - bt_description = os.path.join(self.tree_dir, 'tree.yaml') - try: - with open(bt_description) as f: - string_tree = yaml.safe_load(f) - behaviors = RobotBehaviors(os.path.join(self.tree_dir, 'settings')) - self.__save_tree(string_tree, self.tree_dir, behaviors) - except FileNotFoundError: - print('Error in loading BT description.') - - def __learn_tree(self, pbar: gui.ProgressBar, post_processing: Any = None) -> str: - """ - Prepare the Behavior Tree and return its string representation. - - The argument post_processing is a function that modifies the learnt BT. - """ - # Learn the BT - self.tree_dir = TemporaryDirectory() - settings_dir = os.path.join(self.tree_dir.name, 'settings') - - behaviors = RobotBehaviors(settings_dir) - # Send the offline interface to the planner to expand the BT - offline_interface = PlanningInterface( - available_objects=self.objects, - frames=self.demonstrations.frames, - default_frame=self.demonstrations.default_frame, - random_events=False, - robot_frame=self.robot_frame - ) - - equivalent = find_equivalent_actions( - self.demonstrations, - { - 'pick': yumi_actions.EquivalentPick, - 'place': yumi_actions.EquivalentPlace, - 'drop': yumi_actions.EquivalentPlace, - 'move': base_actions.EquivalentMove - } - ) - - # tree is a PyTree object! - tree = learn_tree( - settings_dir, - equivalent, - behaviors, - offline_interface, - iterations=50, - callback=pbar.update_bar - ) - - string_tree = tree.bt.bt - - if post_processing is not None: - string_tree, behaviors = post_processing( - settings_dir, equivalent, string_tree, behaviors) - - # in string_tree, the string representation of the tree is stored - with open(os.path.join(self.tree_dir.name, 'tree.yaml'), 'w') as f: - yaml.dump(string_tree, f) - - self.__save_tree(string_tree, self.tree_dir.name, behaviors) - - def __build_tree( - self, - window: gui.Window, - from_file: bool = False, - post_processing: Any = None - ): - """ - Build the Behavior Tree once it is loaded. - - The argument post_processing is a function that modifies the learnt BT. - """ - pbar = self.__prelearning_handler(window) - - if from_file: - self.__get_tree_from_file() - else: - self.__learn_tree(pbar, post_processing) - - self.__postlearning_handler(window, pbar) - - -def main(): - parser, args, leftover_args = createArgumentParser() - - gui = LFD_GUI(args) - gui.show() - - -if __name__ == '__main__': - main() diff --git a/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/lfd/lfd_gui_mp.py b/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/lfd/lfd_gui_mp.py deleted file mode 100644 index 12486b9b..00000000 --- a/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/lfd/lfd_gui_mp.py +++ /dev/null @@ -1,520 +0,0 @@ -""" -Launch the GUI to simulate the LfD framework with the AGX Simulator. - -Use with: python lfd/lfd_gui_mp.py --model models/BTDemo.yml:CoarseGripperInLabDR --decorate --timeStep 0.05 -from the /algoryx folder -""" - -# Copyright (c) 2022, ABB -# All rights reserved. -# -# Redistribution and use in source and binary forms, with -# or without modification, are permitted provided that -# the following conditions are met: -# -# * Redistributions of source code must retain the -# above copyright notice, this list of conditions -# and the following disclaimer. -# * Redistributions in binary form must reproduce the -# above copyright notice, this list of conditions -# and the following disclaimer in the documentation -# and/or other materials provided with the -# distribution. -# * Neither the name of ABB nor the names of its -# contributors may be used to endorse or promote -# products derived from this software without -# specific prior written permission. -# -# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" -# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE -# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE -# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE -# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL -# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR -# SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER -# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, -# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF -# THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - -from copy import copy, deepcopy -from distutils.dir_util import copy_tree -import glob -import logging -import multiprocessing as mp -import os -import subprocess -import time -from typing import Any, List - -from agxBrick.brickLoaderUtils import createArgumentParser -from bt_learning.learning_from_demo.clustering import find_equivalent_actions -from bt_learning.learning_from_demo.learning import learn_tree -from bt_learning.learning_from_demo.plot_clusters import plot_clusters -import bt_learning.learning_from_demo.render_tree as tree_render -from py_trees.display import render_dot_tree -import robot_behaviors.mobile_base_behaviors.lfd_actions as base_actions -import robot_behaviors.yumi_behaviors.lfd_actions as yumi_actions -from robot_interface.demonstration import RobotDemonstration -import simulation.algoryx.agx_application as app -from simulation.algoryx.behaviors.sim_behaviors import RobotBehaviors -from simulation.algoryx.lfd import lfd_asprocess, lfd_interface -from simulation.algoryx.lfd.planning_itnerface import PlanningInterface -from simulation.py_trees_interface import PyTree, PyTreeParameters -from user_interface.gui_ddmenu import GUIMenu -from user_interface.lfd_gui import LfDGUILayout -from user_interface.lfd_gui_actions import GUIActions -import yaml - - -# OS dependent import -if os.name == 'nt': # Windows - import PySimpleGUIQt as gui -elif os.name == 'posix': # Linux Ubuntu - import PySimpleGUI as gui - - -class LFD_GUI(): - - def __init__(self, args: Any): - self._logger = logging.getLogger(__file__) - - self.repo_path = os.path.dirname(os.path.dirname((os.path.abspath(__file__)))) - with open(os.path.join(self.repo_path, 'config/sim_data.yaml')) as f: - sim_data = yaml.safe_load(f) - with open(os.path.join(self.repo_path, 'config/sim_objects.yaml')) as f: - self.obj_data = yaml.safe_load(f) - - self.objects = list(self.obj_data.keys()) - - self.default_frame = sim_data['demonstration']['default_frame'] - self.robot_frame = sim_data['demonstration']['robot_frame'] - self.ee_frame = sim_data['algoryx']['ee_name'] - - self.agx_GUI = LfDGUILayout('Algoryx Simulation for Learning from Demonstration') - self.all_frames = sim_data['demonstration']['reference_frames'] + self.objects - if self.robot_frame not in self.all_frames: - self.all_frames.append(self.robot_frame) - if self.default_frame not in self.all_frames: - self.all_frames.append(self.default_frame) - self.pick_menu = GUIMenu('Choose the object to pick.', self.objects) - - self.grasping_actions = sim_data['demonstration']['grasping_actions'] - self.placing_actions = sim_data['demonstration']['placing_actions'] - self.navigation_actions = sim_data['demonstration']['navigation_actions'] - - self.actions = self.grasping_actions + self.placing_actions - - # demo flags - self.demo_dir = None - self.demonstrations = None - - # bt flags - self.running = False - self.bt_length = 0 - self.bt_tick_freq = sim_data['behavior_tree']['tick_freq'] - - # action flags - self.holding = None - self.running_action = None - - self.args = args - - self.interface = lfd_interface.LfDInterface(self.objects) - self.parent_conn, self.child_conn = mp.Pipe() - - def show(self, build_tree_function: Any = None): - """Define the GUI functionalities.""" - self.agx_GUI.initialize_layout() - - window = self.agx_GUI.get_window() - - # Interaction with the GUI Main menu - ret = 0 - while ret is not None: - ret, values = window.read() - - if ret == '__folder_display__' and os.path.isdir(values['__folder_display__']): - self.demo_dir = values['__folder_display__'] - self.__reload_demos(window, build_tree_function) - - elif ret == '__new_demo__': - window.disappear() - self.__new_demo() - self.__reload_demos(window, build_tree_function) - window.reappear() - - elif ret == '__run__': - if self.running is False: - # disable all elements - window.find_element('__show_tree__').update(disabled=True) - window.find_element('__run__').update('Stop') - print('Launching tree...') - self.running = True - - behaviors = RobotBehaviors(os.path.join(self.tree_dir, 'settings')) - self.bt_process = app.RunBT( - self.interface, - self.obj_data, - behaviors, - self.tree_dir, - self.bt_tick_freq, - (app.CloudpickleWrapper(app.Application), self.args) - ) - self.bt_process.start() - - else: - self.running = False - window.find_element('__run__').update('Run') - self.bt_process.stop() - self.bt_process.join() - window.find_element('__show_tree__').update(disabled=False) - - elif ret == '__show_tree__': - if os.name == 'nt': # Windows - os.startfile(os.path.join(self.tree_dir, 'full.svg')) - else: - opener = 'xdg-open' - subprocess.call([opener, os.path.join(self.tree_dir, 'full.svg')]) - - elif ret == '__save_tree__': - folder = gui.popup_get_folder('Select a folder') - if folder is not None and folder != '': - if os.path.isdir(folder): - number = 1 - while os.path.isdir(os.path.join(folder, f'bt{number}')): - number += 1 - - folder = os.path.join(folder, f'bt{number}') - - os.makedirs(folder) - copy_tree(self.tree_dir, folder) - print('Tree saved!') - - elif ret == '__load_tree__': - folder = gui.popup_get_folder('Select a folder') - if folder is not None and folder != '': - if os.path.isdir(folder): - self.tree_dir = folder - - if build_tree_function is None: - self.__build_tree(window, from_file=True) - else: - build_tree_function(window, from_file=True) - - elif ret == '__plot_cluster__': - if self.demonstrations is None: - _ = self.__load_demo() - plot_clusters(self.demo_dir, self.demonstrations) - - def __new_demo(self): - """Launch a new window to start a new demonstration.""" - action_GUI = GUIActions('New demo', self.actions) - action_GUI.initialize_layout() - - action_window = action_GUI.get_window() - - if self.demonstrations is None: - n_demos = 0 - else: - n_demos = self.demonstrations.n_demonstrations() - - os.mkdir(os.path.join(self.demo_dir, 'demo%d' % (n_demos + 1))) - - config_path = os.path.join(self.repo_path, 'config') - self.lfd_process = lfd_asprocess.LfDProcess( - config_path, - self.interface, - (self.child_conn, app.CloudpickleWrapper(app.Application), self.args) - ) - self.lfd_process.start() - - ret = 0 - while ret is not None: - ret, values = action_window.read() - action_GUI.execute_action(ret, self.__manipulate, n_demos) - - self.lfd_process.stop() - self.lfd_process.join() - - def __manipulate(self, n_demos: int, action_name: str, gripper_state: str): - """Define how to run actions when connected to the robot.""" - def get_obj_from_menu(menu: GUIMenu) -> str: - menu_ = deepcopy(menu) - key_ = menu_.initialize_layout() - window = menu_.get_window() - event, values = window.read() - obj = values[key_] - window.close() - return obj - - # Interact with the GUI Action menu - if action_name == 'pick': - # Handle dropdown menu - self.holding = get_obj_from_menu(self.pick_menu) - self.parent_conn.send(('pick', self.holding)) - print(f'Picking {self.parent_conn.recv()}.') - self.__write_action(action_name, n_demos + 1, self.parent_conn) - else: - mainpulating = copy(self.holding) - print(f'Releasing {mainpulating}.') - self.parent_conn.send((action_name, ())) - print(f'Holding {self.parent_conn.recv()}.') - self.__write_action(action_name, n_demos + 1, self.parent_conn) - self.parent_conn.send(('lift', ())) - - def __write_action(self, action_type: str, current_demo: str, conn: mp.connection.Connection): - """Write the pose of the end effector in all available frames as a new action.""" - action_data = {'type': action_type, 'vec_pos': {}, 'vec_quat': {}} - current_demo_dir = os.path.join(self.demo_dir, 'demo%d' % current_demo) - if self.demonstrations is None: - frames = self.all_frames - else: - frames = self.demonstrations.frames - - for frame in frames: - # remove the default frame for now - conn.send(('get_sensor_data', (self.ee_frame, frame))) - position, orientation = conn.recv() - - action_data['vec_pos'][frame] = position.tolist() - action_data['vec_quat'][frame] = orientation.tolist() - - action_number = len(glob.glob(os.path.join(current_demo_dir, 'data_*.yaml'))) + 1 - with open(os.path.join(current_demo_dir, f'data_{action_number}.yaml'), 'w') as f: - yaml.dump(action_data, f, default_flow_style=None) - - def __reload_demos(self, window: gui.Window, build_tree_function): - """Update the demonstration history and build the tree.""" - n_demos = self.__load_demo() - window.find_element('__n_demos__').update('Number of demonstrations: %d' % n_demos) - window.find_element('__new_demo__').update(disabled=False) - - if self.demonstrations is None: - return - - if build_tree_function is None: - self.__build_tree(window) - else: - build_tree_function(window) - - def __load_demo(self) -> int: - """Load demonstration and return number of demonstrations.""" - info_file = os.path.join(self.demo_dir, 'info.yaml') - if not os.path.isfile(info_file): - with open(info_file, 'w') as f: - yaml.dump( - { - 'frames': self.all_frames, - 'robot': self.robot_frame, - 'default_frame': self.default_frame - }, f) - self.demonstrations = None - return 0 - elif len(glob.glob(os.path.join(self.demo_dir, 'demo*'))) == 0: - self.demonstrations = None - return 0 - else: - # Remove empty folders - for folder in glob.glob(self.demo_dir + '/demo[0-9]*/'): - # Each demonstration has to contain at least one action - if not os.path.isfile(folder + '/data_1.yaml'): - print(f'Folder {folder} missing data file, removing.') - os.rmdir(folder) - self.demonstrations = RobotDemonstration( - self.demo_dir, - custom_actions={ - 'pick': yumi_actions.PickAction, - 'place': yumi_actions.PlaceAction, - 'drop': yumi_actions.PlaceAction, - 'move': base_actions.MoveAction - }, - exclude_frames={ - 'pick': [self.robot_frame, self.default_frame], - 'place': [], - 'drop': [], - 'move': [self.robot_frame] - } - ) - return self.demonstrations.n_demonstrations() - - def __bt_worker( - self, - conn: mp.connection.Connection, - target_directory: str, - string_tree: str, - behaviors: Any, - env_fn_wrapper: app.CloudpickleWrapper, - args: List[str] - ): - app = env_fn_wrapper.var(args) - app.bringup(self.obj_data, visual=False) - self.interface.initialize() - parameters = PyTreeParameters() - parameters.behavior_lists = behaviors.get_behavior_list() - parameters.behaviors = behaviors - bt = PyTree( - string_tree, parameters, world_interface=self.interface) - - render_dot_tree( - bt.root, - name='debug_pytree', - target_directory=target_directory, - with_blackboard_variables=True - ) - - if conn.poll(): - data = conn.recv() - conn.send(bt.bt.length()) - - tree_render.dot_graph(bt).write_svg( - os.path.join(target_directory, 'full.svg'), encoding='utf-8') - positions = tree_render.write_tikz_tree( - bt, os.path.join(target_directory, 'full.tex')) - tree_render.py_trees_dot(bt).write_svg( - os.path.join(target_directory, 'full_pytrees.svg'), encoding='utf-8') - - with open(os.path.join(target_directory, 'positions.yaml'), 'w') as f: - yaml.dump(positions, f) - - print('BT data saved!') - - app.shutdown() - - def __save_tree( - self, - string_tree: str, - target_directory: str, - behaviors: Any - ): - """Prepare the BT data for visualization and execution.""" - args = ( - self.child_conn, - target_directory, - string_tree, - behaviors, - app.CloudpickleWrapper(app.Application), - self.args - ) - proc = mp.Process(target=self.__bt_worker, args=args) - proc.start() - time.sleep(2) - self.parent_conn.send('') - self.bt_length = self.parent_conn.recv() - time.sleep(2) - proc.join() - print('Behavior Tree generated!') - - def __prelearning_handler(self, window: gui.Window) -> gui.ProgressBar: - """Update window behavior before learning the tree.""" - window.find_element('__run__').update('Building tree...', disabled=True) - window.find_element('__show_tree__').update(disabled=True) - window.find_element('__save_tree__').update(disabled=True) - window.find_element('__load_tree__').update(disabled=True) - window.find_element('__plot_cluster__').update(disabled=True) - pbar = window.find_element('__progress__') - pbar.update(visible=True) - window.read(timeout=0) - - return pbar - - def __postlearning_handler(self, window: gui.Window, pbar: gui.ProgressBar): - """Update window behavior before learning the tree.""" - window.find_element('__run__').update('Run', disabled=False) - window.find_element('__show_tree__').update(disabled=False) - window.find_element('__save_tree__').update(disabled=False) - window.find_element('__load_tree__').update(disabled=False) - window.find_element('__plot_cluster__').update(disabled=False) - window.find_element('__n_nodes__').update('Number of nodes: %d' % self.bt_length) - pbar.update(visible=False) - - def __get_tree_from_file(self): - """Load an existing BT from a file.""" - # Use a learnt BT - bt_description = os.path.join(self.tree_dir, 'tree.yaml') - try: - with open(bt_description) as f: - string_tree = yaml.safe_load(f) - behaviors = RobotBehaviors(os.path.join(self.tree_dir, 'settings')) - self.__save_tree(string_tree, self.tree_dir, behaviors) - except FileNotFoundError: - print('Error in loading BT description.') - - def __learn_tree(self, pbar: gui.ProgressBar) -> str: - """ - Prepare the Behavior Tree and return its string representation. - - The argument post_processing is a function that modifies the learnt BT. - """ - # Learn the BT - self.tree_dir = os.path.join(self.repo_path, 'lfd/demos_mp/tmp_dir') - if not os.path.isdir(self.tree_dir): - os.mkdir(self.tree_dir) - settings_dir = os.path.join(self.tree_dir, 'settings') - - behaviors = RobotBehaviors(settings_dir) - # Send the offline interface to the planner to expand the BT - offline_interface = PlanningInterface( - available_objects=self.objects, - frames=self.demonstrations.frames, - default_frame=self.demonstrations.default_frame, - random_events=False, - robot_frame=self.robot_frame - ) - - equivalent = find_equivalent_actions( - self.demonstrations, - { - 'pick': yumi_actions.EquivalentPick, - 'place': yumi_actions.EquivalentPlace, - 'drop': yumi_actions.EquivalentPlace, - 'move': base_actions.EquivalentMove - } - ) - - # tree is a PyTree object! - tree = learn_tree( - settings_dir, - equivalent, - behaviors, - offline_interface, - iterations=50, - callback=pbar.update_bar - ) - - string_tree = tree.bt.bt - - # in string_tree, the string representation of the tree is stored - with open(os.path.join(self.tree_dir, 'tree.yaml'), 'w') as f: - yaml.dump(string_tree, f) - - self.__save_tree(string_tree, self.tree_dir, behaviors) - - def __build_tree( - self, - window: gui.Window, - from_file: bool = False, - ): - """ - Build the Behavior Tree once it is loaded. - - The argument post_processing is a function that modifies the learnt BT. - """ - pbar = self.__prelearning_handler(window) - - if from_file: - self.__get_tree_from_file() - else: - self.__learn_tree(pbar) - - self.__postlearning_handler(window, pbar) - - -def main(): - parser, args, leftover_args = createArgumentParser() - - gui = LFD_GUI(args) - gui.show() - - -if __name__ == '__main__': - main() diff --git a/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/lfd/lfd_interface.py b/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/lfd/lfd_interface.py deleted file mode 100644 index 765884ee..00000000 --- a/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/lfd/lfd_interface.py +++ /dev/null @@ -1,47 +0,0 @@ -"""Interface to generate poses in AGX for the GP.""" - -# Copyright (c) 2022, ABB -# All rights reserved. -# -# Redistribution and use in source and binary forms, with -# or without modification, are permitted provided that -# the following conditions are met: -# -# * Redistributions of source code must retain the -# above copyright notice, this list of conditions -# and the following disclaimer. -# * Redistributions in binary form must reproduce the -# above copyright notice, this list of conditions -# and the following disclaimer in the documentation -# and/or other materials provided with the -# distribution. -# * Neither the name of ABB nor the names of its -# contributors may be used to endorse or promote -# products derived from this software without -# specific prior written permission. -# -# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" -# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE -# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE -# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE -# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL -# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR -# SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER -# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, -# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF -# THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - -from typing import List - -from simulation.algoryx.behaviors.agx_interface import AGXInterface - - -class LfDInterface(AGXInterface): - """An interface wrapper for LfD.""" - - def __init__(self, task_objects: List[str]) -> None: - """Instantiate a new AGX Interface.""" - self.objs = task_objects - - def initialize(self): - super().__init__(self.objs) diff --git a/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/lfd/planning_itnerface.py b/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/lfd/planning_itnerface.py deleted file mode 100644 index 301875a3..00000000 --- a/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/lfd/planning_itnerface.py +++ /dev/null @@ -1,298 +0,0 @@ -""" -Offline Interface for execution of behaviors. - -Used by the planner in LfD framework to expand the BT. -""" - -# Copyright (c) 2022, ABB -# All rights reserved. -# -# Redistribution and use in source and binary forms, with -# or without modification, are permitted provided that -# the following conditions are met: -# -# * Redistributions of source code must retain the -# above copyright notice, this list of conditions -# and the following disclaimer. -# * Redistributions in binary form must reproduce the -# above copyright notice, this list of conditions -# and the following disclaimer in the documentation -# and/or other materials provided with the -# distribution. -# * Neither the name of ABB nor the names of its -# contributors may be used to endorse or promote -# products derived from this software without -# specific prior written permission. -# -# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" -# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE -# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE -# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE -# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL -# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR -# SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER -# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, -# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF -# THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - -import math -import random -from typing import Dict, List, Tuple - -import numpy as np - - -class PlanningInterface(): - """An interface for the planner to execute robot behaviors.""" - - # Limits for position initialization - WORLD_LIMITS_XY = (-5, 5) - WORLD_LIMITS_Z = (0, 1) - # Probability of a random event - RONDOM_EVENT_PROB = 0.3 - - def __init__( - self, - available_objects: List[str], - frames: List[str], - default_frame: str, - random_events: bool = True, - init_state: Dict = {}, - robot_frame: str = 'base' - ): - """ - Instantiate a new PlanningInterface with random initial conditions. - - Args: - ---- - - available_objects: list of the objects that can be manipulated. - - frames: list of reference frames. - - default_frame: used frame if nothing else is specified. - - random_events: if True, random disturbances might occur during execution. - - init_state: state of the robot. - - robot_frame: name of the end effector frame of the robot. - - """ - self.default_frame = default_frame - self.all_frames = frames - if robot_frame not in self.all_frames: - self.all_frames.append(robot_frame) - self.objects = available_objects - self.random_events = random_events - self.init_state = init_state - self.robot_frame = robot_frame - - # Dictionary of frames and their position in default_frame - self.frames = {} - - # Set initial conditions - self.reset() - - def pick(self, item: str): - """Simulate a picking task.""" - self.gripper = 'closed' - self.holding = item - self.__random_event() - - def place( - self, - target: str, - reference: str, - position: np.ndarray - ): - """ - Simulate a picking task. - - Args - ---- - - target: target object. - - reference: reference frame for the object. - - pose: position of the target object. - - """ - if reference == self.default_frame: - self.frames[self.holding] = position - else: - # Transform frame - self.frames[self.holding] = position + self.frames[reference] - - self.gripper = 'open' - self.holding = '' - - self.__random_event() - - def grasped(self, held_object: str) -> bool: - """Return true if the object is held by the robot.""" - return self.holding == held_object - - def is_gripper_state(self, state: str) -> bool: - """Return true if gripper is open.""" - return False - - def empty_gripper(self) -> bool: - """Return true if gripper is open.""" - return self.holding == '' - - def get_item_in_frame( - self, - target_object: str, - frame: str - ) -> Tuple[np.ndarray, np.ndarray]: - """Return the pose of object in reference frame.""" - orientation = np.array([0, 0, 0, 1]) - if frame == self.default_frame: - position = self.frames[target_object] - else: - position = self.frames[target_object] - self.frames[frame] - return position, orientation - - def at_pose( - self, - target: str, - reference: str, - pose: np.ndarray, - tolerance: float, - rough: bool = False - ) -> Tuple[np.ndarray, np.ndarray]: - """Return the pose of object in reference frame.""" - position, _ = self.get_item_in_frame(target, reference) - - diff = np.abs(position[:2] - pose[:2]) if rough else np.abs(position - pose) - distance = math.sqrt(sum([x*x for x in diff])) - - return float(distance) < float(tolerance) - - def set_gripper(self, state: str): - """Set the gripper state.""" - self.gripper = state - if state == 'open' and self.holding != '': - # The robot is holding something that will be dropped at - # a random location - self.frames[self.holding] = self.__random_location() - self.holding = '' - self.__random_event() - - def navigate(self, goal: np.ndarray, ref_frame: str): - """Simulate a move action for the mobile manipulator.""" - # TODO: make it not dependent on Action - if ref_frame == self.default_frame: - self.frames[self.robot_frame] = goal - else: - # Transform frame - self.frames[self.robot_frame] = goal + self.frames[ref_frame] - - self.__random_event() - - def robot_position(self, frame: str) -> np.ndarray: - """Return the position of the robot End Effector.""" - if frame == self.default_frame: - return self.frames[self.robot_frame] - else: - return self.frames[self.robot_frame] - self.frames[frame] - - def approach( - self, - target: str, - frame: str - ): - """Move closer to target so it is reachable with the gripper.""" - target = target.copy() - if frame != self.default_frame: - target += self.frames[frame] - - # Sample a point uniformely from the 1.5m radius disk centered at target - r = np.random.uniform(low=0, high=1.5, size=(1,)).item() - theta = np.random.uniform(low=0, high=np.pi, size=(1,)).item() - self.frames[self.robot_frame] = np.array( - [r*np.cos(theta) + target[0], r*np.sin(theta) + target[1], 0.1] - ) - - self.__random_event() - - def reachable( - self, - target: str, - frame: str = None - ) -> bool: - """Target is reachable if it is within 1.5m of the robot. Orientation is not considered.""" - if isinstance(target, str): - target = self.frames[target] - else: - target = target.copy() - if frame != self.default_frame: - target += self.frames[frame] - - return np.linalg.norm(target[0:2] - self.frames[self.robot_frame][0:2]) <= 1.5 - - def reset(self): - """Reset the internal state to a random initial state.""" - for f in self.all_frames: - if f == self.default_frame: - self.frames[f] = np.zeros((3,)) - elif f == self.robot_frame: - # EE frame cannot have arbitrary z component - position = self.__random_location() - position[2] = 0.1 - self.frames[f] = position - else: - self.frames[f] = self.__random_location() - - for f in self.init_state: - self.frames[f] = self.init_state[f] - - if 'holding' in self.init_state: - self.holding = self.init_state['holding'] - else: - self.holding = random.choice(self.objects + ['']) - - if self.holding != '': - self.gripper = 'closed' - else: - self.gripper = random.choice(['open', 'closed']) - - def __random_location(self) -> np.ndarray: - """Generate a random position.""" - return np.random.uniform( - (self.WORLD_LIMITS_XY[0], self.WORLD_LIMITS_XY[0], self.WORLD_LIMITS_Z[0]), - (self.WORLD_LIMITS_XY[1], self.WORLD_LIMITS_XY[1], self.WORLD_LIMITS_Z[1]), - (3,) - ) - - def __random_event(self): - """ - Produce a random event with probability RANDOM_EVENT_PROB. - - An example is dropping what is being held. - """ - if not self.random_events: - return - - if np.random.rand(1).item() > self.RONDOM_EVENT_PROB: - # No random event - return - - # Possible random events are change gripper state and randomly move an object - events = ['gripper', 'displace'] - if self.holding != '': - # If the robot is holding something it can also drop it - events.append('drop') - - event = random.choice(events) - - if event == 'gripper': - self.gripper = 'closed' if self.gripper == 'open' else 'open' - # If we were holding something when the gripper opened we drop it at a random location - if self.gripper == 'open' and self.holding != '': - self.frames[self.holding] = self.__random_location() - self.holding = '' - elif event == 'displace': - target_object = random.choice(self.objects) - if target_object == self.holding: - # If the robot is holding the robot it means it has been dropped - # without opening the gripper - self.holding == '' - self.frames[target_object] = self.__random_location() - elif event == 'drop': - # Drop without opening gripper - self.frames[self.holding] = self.__random_location() - self.holding = '' diff --git a/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/lfd/user_interaction.py b/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/lfd/user_interaction.py deleted file mode 100644 index 528d06bc..00000000 --- a/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/lfd/user_interaction.py +++ /dev/null @@ -1,141 +0,0 @@ -"""Interface to allow a user to interact with the simulator and demonstrate a task.""" - -# Copyright (c) 2022, ABB -# All rights reserved. -# -# Redistribution and use in source and binary forms, with -# or without modification, are permitted provided that -# the following conditions are met: -# -# * Redistributions of source code must retain the -# above copyright notice, this list of conditions -# and the following disclaimer. -# * Redistributions in binary form must reproduce the -# above copyright notice, this list of conditions -# and the following disclaimer in the documentation -# and/or other materials provided with the -# distribution. -# * Neither the name of ABB nor the names of its -# contributors may be used to endorse or promote -# products derived from this software without -# specific prior written permission. -# -# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" -# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE -# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE -# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE -# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL -# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR -# SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER -# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, -# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF -# THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - -import agx -import agxSDK -import agxRender - -from agxPythonModules.utils.callbacks import StepEventCallback as sec -from agxPythonModules.utils.callbacks import KeyboardCallback as Input -from agxPythonModules.utils.environment import application, simulation -from simulation.algoryx.behaviors.agx_interface import AGXInterface -import simulation.algoryx.agx_application as AgxApplication - - -class UserControls(): - """This class allows the user to control the End Effector during a demonstration.""" - - def __init__(self, behaviors: AGXInterface, app: AgxApplication = None) -> None: - - self.color = agxRender.Color.Black() - self.behaviors = behaviors - self.step = 0.01 - self.running = True - self.with_cam = False - self.app = app.app if app is not None else application() - - # self.add_text() - - def add_text(self) -> None: - """Define the text to display.""" - self.app.getSceneDecorator().setText( - 1, 'Keyboard controls:', self.color) - self.app.getSceneDecorator().setText( - 2, '- Left/Right Arrows control X dimension.', self.color) - self.app.getSceneDecorator().setText( - 3, '- Up/Down Arrows control Y dimension.', self.color) - self.app.getSceneDecorator().setText( - 4, '- PageUp/PageDown Keys control Z dimension.', self.color) - self.app.getSceneDecorator().setText( - 5, f'- +/- change the movement step: current = {self.step}', self.color) - self.app.getSceneDecorator().setText( - 6, f'- Back Space to confirm placing position and release object.', self.color) - if self.with_cam: - self.app.getSceneDecorator().setText( - 7, f'Camera Eye: {self.app.getCameraData().eye}', self.color) - self.app.getSceneDecorator().setText( - 8, f'Camera Center: {self.app.getCameraData().center}', self.color) - self.app.getSceneDecorator().setText( - 9, f'Camera Up: {self.app.getCameraData().up}', self.color) - - def update_step(self) -> None: - """Update the step in the text section.""" - self.app.getSceneDecorator().setText( - 5, f'- +/- change the movement step: current = {self.step}', self.color) - if self.with_cam: - self.app.getSceneDecorator().setText( - 7, f'Camera Eye: {self.app.getCameraData().eye}', self.color) - self.app.getSceneDecorator().setText( - 8, f'Camera Center: {self.app.getCameraData().center}', self.color) - self.app.getSceneDecorator().setText( - 9, f'Camera Up: {self.app.getCameraData().up}', self.color) - - def clear_text(self) -> None: - """Clear the text after the demonstration.""" - self.app.getSceneDecorator().clearText() - - def handle_key(self, data) -> None: - """Callback function for keyboard events related to the gripper.""" - - t0 = simulation().getTimeStamp() + 0.2 - if data.isKeyDown: - if data.key == agxSDK.GuiEventListener.KEY_Left: - sec.callAt(t0, lambda: self.behaviors._move_gripper( - self.behaviors.gripper_ctrl.getPosition() + agx.Vec3(-self.step, 0.0, 0.0))) - if data.key == agxSDK.GuiEventListener.KEY_Right: - sec.callAt(t0, lambda: self.behaviors._move_gripper( - self.behaviors.gripper_ctrl.getPosition() + agx.Vec3(self.step, 0.0, 0.0))) - if data.key == agxSDK.GuiEventListener.KEY_Page_Up: - sec.callAt(t0, lambda: self.behaviors._move_gripper( - self.behaviors.gripper_ctrl.getPosition() + agx.Vec3(0.0, -self.step, 0.0))) - if data.key == agxSDK.GuiEventListener.KEY_Page_Down: - sec.callAt(t0, lambda: self.behaviors._move_gripper( - self.behaviors.gripper_ctrl.getPosition() + agx.Vec3(0.0, self.step, 0.0))) - if data.key == agxSDK.GuiEventListener.KEY_Up: - sec.callAt(t0, lambda: self.behaviors._move_gripper( - self.behaviors.gripper_ctrl.getPosition() + agx.Vec3(0.0, 0.0, self.step))) - if data.key == agxSDK.GuiEventListener.KEY_Down: - sec.callAt(t0, lambda: self.behaviors._move_gripper( - self.behaviors.gripper_ctrl.getPosition() + agx.Vec3(0.0, 0.0, -self.step))) - if data.key == agxSDK.GuiEventListener.KEY_KP_Add: - self.step = round((self.step + 0.01), 3) - self.update_step() - if data.key == agxSDK.GuiEventListener.KEY_KP_Subtract: - self.step = round((self.step - 0.01), 3) - self.update_step() - if data.key == agxSDK.GuiEventListener.KEY_BackSpace: - self.clear_text() - self.running = False - - def setup_keyboard_listener(self) -> None: - """Sets up listener for keyboard commands to control gripper.""" - Input.bind(name='Left', key=agxSDK.GuiEventListener.KEY_Left, callback=self.handle_key) - Input.bind(name='Right', key=agxSDK.GuiEventListener.KEY_Right, callback=self.handle_key) - Input.bind(name='Back', key=agxSDK.GuiEventListener.KEY_Up, callback=self.handle_key) - Input.bind(name='Forward', key=agxSDK.GuiEventListener.KEY_Down, callback=self.handle_key) - Input.bind(name='Up', key=agxSDK.GuiEventListener.KEY_Page_Up, callback=self.handle_key) - Input.bind(name='Down', key=agxSDK.GuiEventListener.KEY_Page_Down, callback=self.handle_key) - Input.bind(name='+', key=agxSDK.GuiEventListener.KEY_KP_Add, callback=self.handle_key) - Input.bind(name='-', key=agxSDK.GuiEventListener.KEY_KP_Subtract, callback=self.handle_key) - Input.bind( - name='Enter', key=agxSDK.GuiEventListener.KEY_BackSpace, callback=self.handle_key) diff --git a/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/plot_logs.py b/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/plot_logs.py deleted file mode 100644 index 31875d2a..00000000 --- a/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/plot_logs.py +++ /dev/null @@ -1,72 +0,0 @@ -"""Plot logs from the GP approach.""" - -# Copyright (c) 2022, ABB -# All rights reserved. -# -# Redistribution and use in source and binary forms, with -# or without modification, are permitted provided that -# the following conditions are met: -# -# * Redistributions of source code must retain the -# above copyright notice, this list of conditions -# and the following disclaimer. -# * Redistributions in binary form must reproduce the -# above copyright notice, this list of conditions -# and the following disclaimer in the documentation -# and/or other materials provided with the -# distribution. -# * Neither the name of ABB nor the names of its -# contributors may be used to endorse or promote -# products derived from this software without -# specific prior written permission. -# -# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" -# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE -# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE -# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE -# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL -# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR -# SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER -# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, -# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF -# THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - -from bt_learning.gp import logplot - - -def main() -> None: - batch_size = 10 - - # first plot - params = logplot.PlotParameters() - params.label = 'GP with LfD solution' - params.xlabel_fontsize = 12 - params.save_fig = False - params.mean_color = 'b' - params.std_color = 'b' - params.x_max = 2300 - params.extrapolate_y = True - params.plot_horizontal = False - logs = [str(x + 1) for x in range(batch_size)] - logplot.plot_learning_curves(logs, params) - - # second plot - params = logplot.PlotParameters() - params.title = 'Fitness score for Experiment 2' - params.title_fontsize = 18 - params.label = 'GP standalone' - params.xlabel_fontsize = 12 - params.xlabel = 'Learning Episodes' - params.legend_fontsize = 14 - params.save_fig = True - params.mean_color = 'r' - params.std_color = 'r' - params.x_max = 2300 - params.extrapolate_y = True - params.plot_horizontal = False - logs = [str(x + 1 + batch_size) for x in range(batch_size)] - logplot.plot_learning_curves(logs, params) - - -if __name__ == '__main__': - main() diff --git a/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/requirements-dev.txt b/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/requirements-dev.txt deleted file mode 100644 index 3bde1a8a..00000000 --- a/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/requirements-dev.txt +++ /dev/null @@ -1,3 +0,0 @@ --r requirements.txt - -pytest==6.2 \ No newline at end of file diff --git a/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/requirements.txt b/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/requirements.txt deleted file mode 100644 index 42d974ee..00000000 --- a/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/requirements.txt +++ /dev/null @@ -1,7 +0,0 @@ ---extra-index-url https://agx-access:UymNbuiLHJ13CKGnmsoD@git.algoryx.se/api/v4/projects/270/packages/pypi/simple ---extra-index-url https://click-access:F2q7LauW_d-HJ7bH37sV@git.algoryx.se/api/v4/projects/262/packages/pypi/simple - - -agxBrick==0.5.15 -agxClick==0.1.32 -numpy diff --git a/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/tutorials/__init__.py b/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/tutorials/__init__.py deleted file mode 100644 index 8b137891..00000000 --- a/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/tutorials/__init__.py +++ /dev/null @@ -1 +0,0 @@ - diff --git a/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/tutorials/bt_application.py b/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/tutorials/bt_application.py deleted file mode 100644 index 8df38bfb..00000000 --- a/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/tutorials/bt_application.py +++ /dev/null @@ -1,104 +0,0 @@ -"""Behavior Tree Application to run the AGX Simulation.""" - -# Copyright (c) 2022, ABB -# All rights reserved. -# -# Redistribution and use in source and binary forms, with -# or without modification, are permitted provided that -# the following conditions are met: -# -# * Redistributions of source code must retain the -# above copyright notice, this list of conditions -# and the following disclaimer. -# * Redistributions in binary form must reproduce the -# above copyright notice, this list of conditions -# and the following disclaimer in the documentation -# and/or other materials provided with the -# distribution. -# * Neither the name of ABB nor the names of its -# contributors may be used to endorse or promote -# products derived from this software without -# specific prior written permission. -# -# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" -# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE -# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE -# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE -# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL -# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR -# SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER -# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, -# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF -# THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - -import logging -from typing import Dict, List - -import agx -from agxClick import AgxApplication -from simulation.algoryx import agx_environment -import py_trees as pt - - -class BTApplication(AgxApplication): - - def __init__(self, args: List[str]) -> None: - super().__init__(args) - self._logger = logging.getLogger(__file__) - self._env = None - - self.buffer = [] - self.last_status = pt.common.Status.INVALID - - def init_agx_environment(self, args: List[str], data: Dict) -> None: - """Initialize AGX Environment.""" - env = agx_environment.AGXEnvironment(args, objects=data) - - brickSimulation = env.build_scene() - env.spawn(brickSimulation) - env.create_visual() - self._env = env - - def run_bt(self, behavior_tree: pt.trees.BehaviourTree, freq_ratio: int = 50) -> None: - """ - Run episode by ticking the Behavior Tree. - - Args - ---- - behavior_tree: the BT to run. - freq_ratio: how many simulation steps per BT tick. - - """ - done = False - freq_ratio = 50 if freq_ratio < 50 else freq_ratio - ticks = 0 - while not done: - ticks += 1 - print(f'Current tick: {ticks}') - # print(behavior_tree.root.status) - behavior_tree.root.tick_once() - # Add some little delay - for _ in range(freq_ratio): - self.stepApplication() - - if self.bt_done(behavior_tree, 4): - done = True - print(f'Ticks: {ticks}') - - def bt_done(self, behavior_tree: pt.trees.BehaviourTree, consecutive: int = 2) -> bool: - """Return True if the BT returns N consecutive SUCCESS or FAILURE.""" - is_failing = True if behavior_tree.root.status is pt.common.Status.FAILURE else False - is_successful = True if behavior_tree.root.status is pt.common.Status.SUCCESS else False - done = False - if behavior_tree.root.status == self.last_status and is_successful: - self.buffer.append(behavior_tree.root.status) - if len(self.buffer) >= consecutive: - done = all(self.buffer) - else: - self.buffer = [] - - self.last_status = behavior_tree.root.status - return done - - def reset(self) -> None: - self._env.reset_simulation() diff --git a/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/tutorials/bt_wrapper.py b/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/tutorials/bt_wrapper.py deleted file mode 100644 index b9e63f34..00000000 --- a/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/tutorials/bt_wrapper.py +++ /dev/null @@ -1,173 +0,0 @@ -"""Behavior Tree Application to run the AGX Simulation.""" - -# Copyright (c) 2022, ABB -# All rights reserved. -# -# Redistribution and use in source and binary forms, with -# or without modification, are permitted provided that -# the following conditions are met: -# -# * Redistributions of source code must retain the -# above copyright notice, this list of conditions -# and the following disclaimer. -# * Redistributions in binary form must reproduce the -# above copyright notice, this list of conditions -# and the following disclaimer in the documentation -# and/or other materials provided with the -# distribution. -# * Neither the name of ABB nor the names of its -# contributors may be used to endorse or promote -# products derived from this software without -# specific prior written permission. -# -# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" -# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE -# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE -# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE -# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL -# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR -# SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER -# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, -# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF -# THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - -from copy import copy -from typing import Any - -import agx -import numpy as np -import py_trees as pt - -VERBOSE = True - - -class PickBehavior(pt.behaviour.Behaviour): - def __init__(self, name: str, world_interface: Any, target_object: str) -> None: - """Initialize the pick task.""" - super().__init__(name) - - self.world_interface = world_interface - self.target_obj = target_object - - self.max_ticks = 10 - self.counter = 0 - - def initialise(self) -> None: - self.picking_task = self.world_interface.pick(self.target_obj) - self.counter = 0 - if VERBOSE: - print(self.counter) - return super().initialise() - - def update(self) -> pt.common.Status: - """Return the status of the behavior.""" - self.counter += 1 - if VERBOSE: - print(self.counter) - if self.world_interface.grasped(self.target_obj): - return pt.common.Status.SUCCESS - elif not self.world_interface.reachable(self.target_obj) or self.counter > self.max_ticks: - return pt.common.Status.FAILURE - else: - return pt.common.Status.RUNNING - - def terminate(self, new_status) -> None: - # self.world_interface.preempt_skill() - return super().terminate(new_status) - - -class PlaceBehavior(pt.behaviour.Behaviour): - def __init__( - self, - name: str, - world_interface: Any, - target_object: str, - reference_object: str, - pose: agx.Vec3 or np.ndarray - ) -> None: - """Initialize the place task.""" - super().__init__(name) - - self.world_interface = world_interface - self.target_obj = target_object - self.holding = None - self.ref_obj = reference_object - self.pose = pose - - self.max_ticks = 10 - self.counter = 0 - - def initialise(self) -> None: - self.placing_task = self.world_interface.place( - self.target_obj, self.ref_obj, self.pose) - self.counter = 0 - if VERBOSE: - print(self.counter) - self.holding = self.world_interface.holding - return super().initialise() - - def update(self) -> pt.common.Status: - """Return the status of the behavior.""" - self.counter += 1 - if VERBOSE: - print(self.counter) - if self.world_interface.at_pose(self.target_obj, self.ref_obj, self.pose, 0.5): - return pt.common.Status.SUCCESS - elif self.holding != self.target_obj or self.counter > self.max_ticks: - return pt.common.Status.FAILURE - else: - return pt.common.Status.RUNNING - - def terminate(self, new_status) -> None: - # self.world_interface.preempt_skill() - return super().terminate(new_status) - - -class InGripper(pt.behaviour.Behaviour): - """Returns SUCCESS if the gripper is holding object.""" - - def __init__( - self, name: str, world_interface: Any, held_object: str) -> None: - """Initialize the condition.""" - super().__init__(name) - - self.world_interface = world_interface - self.object = held_object - - def update(self) -> pt.common.Status: - """Return the status of the behavior.""" - holding = self.world_interface.grasped(self.object) - - if holding: - return pt.common.Status.SUCCESS - else: - return pt.common.Status.FAILURE - - -class AtPose(pt.behaviour.Behaviour): - """Returns SUCCESS if the target object is at given pose.""" - - def __init__( - self, - name: str, - world_interface: Any, - target: str, - reference: str, - pose: agx.Vec3 or np.ndarray - ) -> None: - """Initialize the condition.""" - super().__init__(name) - - self.world_interface = world_interface - self.target = target - self.reference = reference - self.pose = pose - - def update(self) -> pt.common.Status: - """Return the status of the behavior.""" - pose_ok = self.world_interface.at_pose(self.target, self.reference, self.pose, 0.5) - - if pose_ok: - return pt.common.Status.SUCCESS - else: - return pt.common.Status.FAILURE diff --git a/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/tutorials/merging.py b/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/tutorials/merging.py deleted file mode 100644 index edc4a39c..00000000 --- a/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/tutorials/merging.py +++ /dev/null @@ -1,89 +0,0 @@ -""" -This tutorial merges two bodies in the simulation for faster manipulation. - -Use with: agxViewer tutorials/merging.py -from the /algoryx folder -""" - -# Copyright (c) 2022, ABB -# All rights reserved. -# -# Redistribution and use in source and binary forms, with -# or without modification, are permitted provided that -# the following conditions are met: -# -# * Redistributions of source code must retain the -# above copyright notice, this list of conditions -# and the following disclaimer. -# * Redistributions in binary form must reproduce the -# above copyright notice, this list of conditions -# and the following disclaimer in the documentation -# and/or other materials provided with the -# distribution. -# * Neither the name of ABB nor the names of its -# contributors may be used to endorse or promote -# products derived from this software without -# specific prior written permission. -# -# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" -# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE -# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE -# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE -# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL -# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR -# SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER -# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, -# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF -# THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - -import agx -import agxOSG -import agxCollide - -from agxPythonModules.utils.environment import init_app, simulation, application, root -from agxPythonModules.utils.callbacks import KeyboardCallback as kc - - -def buildScene(): - """Merge with right arrow key and split with left arrow key.""" - ground = agxCollide.Geometry(agxCollide.Box(10, 10, 1)) - ground.setPosition(0, 0, -1) - simulation().add(ground) - - box1 = agx.RigidBody(agxCollide.Geometry(agxCollide.Box(1, 1, 1))) - box1.setPosition(-1.4, 0, 1) - simulation().add(box1) - - box2 = agx.RigidBody(agxCollide.Geometry(agxCollide.Box(1, 1, 1))) - box2.setPosition(1.4, 0, 1) - simulation().add(box2) - - mb = agx.MergedBody() - simulation().add(mb) - - def merge(t): - mb.add(agx.MergedBodyEmptyEdgeInteraction(box1, box2)) - - def split(t): - mb.remove(box1) - - kc.bind( - name='merge', - key=kc.KEY_Right, - mode=kc.Mode.NATIVE, - callback=merge - ) - kc.bind( - name='split', - key=kc.KEY_Left, - mode=kc.Mode.NATIVE, - callback=split - ) - - application().setEnableDebugRenderer(True) - - -init_app( - name=__name__, - scenes=['buildScene', '1'] -) diff --git a/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/tutorials/run_bt_demo.py b/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/tutorials/run_bt_demo.py deleted file mode 100644 index e34ba932..00000000 --- a/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/tutorials/run_bt_demo.py +++ /dev/null @@ -1,199 +0,0 @@ -""" -This script is to demo the Brick models in the file `models/BTDemo.yml` - -To test the automatic spawning of items run: -`python tutorials/run_bt_demo.py --model models/BTDemo.yml:CoarseGripperInLabDR --decorate --timeStep 0.05` -from the /algoryx folder -""" - -# Copyright (c) 2022, ABB -# All rights reserved. -# -# Redistribution and use in source and binary forms, with -# or without modification, are permitted provided that -# the following conditions are met: -# -# * Redistributions of source code must retain the -# above copyright notice, this list of conditions -# and the following disclaimer. -# * Redistributions in binary form must reproduce the -# above copyright notice, this list of conditions -# and the following disclaimer in the documentation -# and/or other materials provided with the -# distribution. -# * Neither the name of ABB nor the names of its -# contributors may be used to endorse or promote -# products derived from this software without -# specific prior written permission. -# -# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" -# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE -# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE -# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE -# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL -# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR -# SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER -# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, -# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF -# THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - -import functools -import os -from typing import Any -import yaml - -import agx -from agxBrick.brickLoaderUtils import createArgumentParser -from simulation.algoryx.behaviors.agx_interface import AGXInterface -from behaviors.common_behaviors import RSequence -from simulation.algoryx.tutorials.bt_application import BTApplication -from simulation.algoryx.tutorials import bt_wrapper as bt -import numpy as np -import py_trees - - -def post_tick_handler(snapshot_visitor, behavior_tree) -> None: - """Prints an ascii tree with the current snapshot status.""" - print( - '\n' - + py_trees.display.unicode_tree( - root=behavior_tree.root, - visited=snapshot_visitor.visited, - previously_visited=snapshot_visitor.visited, - ) - ) - name_ = 'root' + str(behavior_tree.count) - - -def pick_and_place_BT( - target: str, - reference: str, - pose: agx.Vec3 or np.ndarray, - world_interface: Any -) -> py_trees.composites.Selector or py_trees.composites.Sequence: - """Behavior Tree template for pick and place.""" - pick_box = py_trees.composites.Selector(name='Fallback') - pick_box.add_children( - [ - bt.InGripper( - name=f'{target} Grasped?', - world_interface=world_interface, - held_object=target, - ), - bt.PickBehavior( - name=f'Pick {target}!', - world_interface=world_interface, - target_object=target, - ), - ] - ) - - place_box = RSequence(name='Sequence') - place_box.add_children( - [ - pick_box, - bt.PlaceBehavior( - name=f'Place {target}!', - world_interface=world_interface, - target_object=target, - reference_object=reference, - pose=pose, - ), - ] - ) - - task = py_trees.composites.Selector(name='Fallback') - task.add_children( - [ - bt.AtPose( - name=f'{target} on {reference}?', - world_interface=world_interface, - target=target, - reference=reference, - pose=pose, - ), - place_box - ] - ) - - return task - - -class BTDemo: - def __init__(self, world_interface: Any) -> None: - - self.world_interface = world_interface - self.root = None - - def build_tree(self) -> None: - """Hand code the BT.""" - - box1 = pick_and_place_BT( - 'YellowBox', 'KittingBox', agx.Vec3(0, 0, 0.12), self.world_interface) - box2 = pick_and_place_BT( - 'GreenBox', 'YellowBox', agx.Vec3(0, 0, 0.18), self.world_interface) - box3 = pick_and_place_BT( - 'BlueBox', 'GreenBox', agx.Vec3(0, 0, 0.18), self.world_interface) - - self.root = RSequence(name='Sequence') - self.root.add_children([box1, box2, box3]) - self.tree = py_trees.trees.BehaviourTree(self.root) - - def get_root(self) -> py_trees.composites.Selector: - return self.root - - def visualize(self, bt_name: str) -> None: - """Compute the number of nodes and transition in a BT and save it as figure.""" - py_trees.display.render_dot_tree( - self.tree.root, - name=bt_name, - target_directory=os.path.dirname(os.path.abspath(__file__)) - ) - - def run(self, simulation_app: BTApplication) -> None: - """The BT execution is visualized in the terminal.""" - self.tree.visitors.append(py_trees.visitors.DebugVisitor()) - snapshot_visitor = py_trees.visitors.SnapshotVisitor() - self.tree.add_post_tick_handler( - functools.partial(post_tick_handler, snapshot_visitor) - ) - self.tree.visitors.append(snapshot_visitor) - self.tree.setup(timeout=15) - - self.world_interface.remove_locks() - self.world_interface.preempt_skill() - self.world_interface.add_merger() - - simulation_app.run_bt(self.tree) - - self.tree.root.terminate(py_trees.common.Status.INVALID) - self.tree.shutdown() - self.tree.root = None - - -def main() -> None: - parser, args, leftover_args = createArgumentParser() - repo_path = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) - with open(os.path.join(repo_path, 'config/sim_objects.yaml')) as f: - data = yaml.safe_load(f) - - bt_app = BTApplication(args) - bt_app.init_agx_environment(args, data) - - agx_interface = AGXInterface(list(data.keys())) - - behavior_tree = BTDemo(agx_interface) - behavior_tree.build_tree() - behavior_tree.visualize('Task_BT') - - for _ in range(2): - behavior_tree.build_tree() - behavior_tree.run(bt_app) - bt_app.reset() - - del bt_app - bt_app = None - - -if __name__ == "__main__": - main() diff --git a/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/tutorials/run_demo.py b/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/tutorials/run_demo.py deleted file mode 100644 index 831c21e0..00000000 --- a/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/tutorials/run_demo.py +++ /dev/null @@ -1,142 +0,0 @@ -""" -This script is to demo the Brick models in the file `models/BTDemo.yml` - -To test the automatic spawning of items run: -`python tutorials/run_demo.py --model models/BTDemo.yml:CoarseGripperInLabDR --decorate --timeStep 0.05` -from the /algoryx folder -""" - -# Copyright (c) 2022, ABB -# All rights reserved. -# -# Redistribution and use in source and binary forms, with -# or without modification, are permitted provided that -# the following conditions are met: -# -# * Redistributions of source code must retain the -# above copyright notice, this list of conditions -# and the following disclaimer. -# * Redistributions in binary form must reproduce the -# above copyright notice, this list of conditions -# and the following disclaimer in the documentation -# and/or other materials provided with the -# distribution. -# * Neither the name of ABB nor the names of its -# contributors may be used to endorse or promote -# products derived from this software without -# specific prior written permission. -# -# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" -# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE -# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE -# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE -# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL -# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR -# SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER -# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, -# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF -# THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - -import logging -from typing import Dict, List - -import agx -from agxBrick.brickLoaderUtils import createArgumentParser -from agxClick import AgxApplication -from agxPythonModules.utils.environment import init_app, simulation -from agxPythonModules.utils.callbacks import StepEventCallback as sec -import numpy as np -from simulation.algoryx import agx_environment -from simulation.algoryx.behaviors import agx_interface - -import os -import yaml - - -class RunDemo(AgxApplication): - - def __init__(self, args: List[str]) -> None: - super().__init__(args) - self._logger = logging.getLogger(__file__) - self._env = None - - def init_agx_environment(self, args: List[str], data: Dict) -> None: - """Initialize AGX Environment.""" - env = agx_environment.AGXEnvironment(args, objects=data) - - brickSimulation = env.build_scene() - env.spawn(brickSimulation) - env.create_visual() - self._env = env - - def reset(self) -> None: - self._env.reset_simulation() - - def run_episode(self, behaviors) -> None: - behaviors.pick('YellowBox') - self.__step_ahead(3.0) - behaviors.place('YellowBox', 'KittingBox', agx.Vec3(0, 0, 0.12)) - self.__step_ahead(3.0) - behaviors.pick('GreenBox') - self.__step_ahead(3.0) - behaviors.place('GreenBox', 'YellowBox', agx.Vec3(0, 0, 0.18)) - self.__step_ahead(3.0) - behaviors.pick('BlueBox') - self.__step_ahead(3.0) - behaviors.place('BlueBox', 'GreenBox', agx.Vec3(0, 0, 0.18)) - self.__step_ahead(3.0) - - def __step_ahead(self, time: float): - """Step forward in time.""" - t1 = self._env.sim.getTimeStamp() - while self._env.sim.getTimeStamp() - t1 < time: - self.stepApplication() - - -def print_poses(interface: agx_interface.AGXInterface): - y_pos, _ = interface.get_item_in_frame('YellowBox', 'KittingBox') - g_pos, _ = interface.get_item_in_frame('GreenBox', 'YellowBox') - b_pos, _ = interface.get_item_in_frame('BlueBox', 'GreenBox') - print('Yellow:', y_pos) - print('Green:', g_pos) - print('Blue:', b_pos) - - -def main() -> None: - parser, args, leftover_args = createArgumentParser() - repo_path = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) - with open(os.path.join(repo_path, 'config/sim_objects.yaml')) as f: - data = yaml.safe_load(f) - - demo = RunDemo(args) - demo.init_agx_environment(args, data) - - print(demo._env.get_bodies()) - w_T_yb = demo._env.sim.getRigidBody('YellowBox').getTransform() - pos_yb = w_T_yb.getTranslate() - w_T_ee = demo._env.sim.getRigidBody('GripperControlBody').getTransform() - pos_ee = w_T_ee.getTranslate() - - # method1 - print(pos_ee - pos_yb) - # method2 - yb_T_ee = w_T_yb.inverse()*w_T_ee - print(yb_T_ee.getTranslate()) - - # KitBox geometry - bottom_plate = demo._env.sim.getRigidBody('KittingBox').getGeometries()[0] - print(bottom_plate.getShape().asBox().getHalfExtents()) - - behaviors = agx_interface.AGXInterface(list(data.keys())) - - for _ in range(2): - demo.run_episode(behaviors) - print_poses(behaviors) - demo.reset() - - # del demo - # demo = None - - -if __name__ == "__main__": - main() diff --git a/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/tutorials/run_interaction.py b/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/tutorials/run_interaction.py deleted file mode 100644 index f523d720..00000000 --- a/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/tutorials/run_interaction.py +++ /dev/null @@ -1,86 +0,0 @@ -""" -This script is to demo the Brick models in the file `models/BTDemo.yml` - -To test the automatic spawning of items run: -`python tutorials/run_interaction.py --model models/BTDemo.yml:CoarseGripperInLabDR --decorate --timeStep 0.05` -from the /algoryx folder -""" - -# Copyright (c) 2022, ABB -# All rights reserved. -# -# Redistribution and use in source and binary forms, with -# or without modification, are permitted provided that -# the following conditions are met: -# -# * Redistributions of source code must retain the -# above copyright notice, this list of conditions -# and the following disclaimer. -# * Redistributions in binary form must reproduce the -# above copyright notice, this list of conditions -# and the following disclaimer in the documentation -# and/or other materials provided with the -# distribution. -# * Neither the name of ABB nor the names of its -# contributors may be used to endorse or promote -# products derived from this software without -# specific prior written permission. -# -# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" -# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE -# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE -# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE -# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL -# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR -# SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER -# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, -# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF -# THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - -import agx -from agxBrick.brickLoaderUtils import createArgumentParser -from agxPythonModules.utils.environment import init_app, simulation -from agxPythonModules.utils.callbacks import StepEventCallback as sec - -from simulation.algoryx import agx_environment -from simulation.algoryx.behaviors import agx_interface -from simulation.algoryx.lfd import user_interaction - - -import os -import yaml - - -def buildScene() -> None: - repo_path = os.path.dirname(os.path.dirname((os.path.abspath(__file__)))) - with open(os.path.join(repo_path, 'config/sim_objects.yaml')) as f: - data = yaml.safe_load(f) - - env = agx_environment.AGXEnvironment(args, objects=data) - - brickSimulation = env.build_scene() - env.spawn(brickSimulation) - env.create_visual() - - behaviors = agx_interface.AGXInterface(list(data.keys())) - - user_input = user_interaction.UserControls(behaviors) - user_input.setup_keyboard_listener() - - brickSimulation.ConnectToROS() - - -try: - args -except: - args = None - -if args is None: - parser, args, leftover_args = createArgumentParser() - - -init = init_app( - name=__name__, - scenes=[('buildScene', '1')], - autoStepping=True -) diff --git a/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/tutorials/run_useless_bt.py b/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/tutorials/run_useless_bt.py deleted file mode 100644 index cb0db674..00000000 --- a/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/tutorials/run_useless_bt.py +++ /dev/null @@ -1,142 +0,0 @@ -""" -This script is to demo the Brick models in the file `models/BTDemo.yml` - -To test the automatic spawning of items run: -`python tutorials/test_useless_bt.py --model models/BTDemo.yml:CoarseGripperInLabDR --decorate --timeStep 0.05` -from the /algoryx folder -""" - -# Copyright (c) 2022, ABB -# All rights reserved. -# -# Redistribution and use in source and binary forms, with -# or without modification, are permitted provided that -# the following conditions are met: -# -# * Redistributions of source code must retain the -# above copyright notice, this list of conditions -# and the following disclaimer. -# * Redistributions in binary form must reproduce the -# above copyright notice, this list of conditions -# and the following disclaimer in the documentation -# and/or other materials provided with the -# distribution. -# * Neither the name of ABB nor the names of its -# contributors may be used to endorse or promote -# products derived from this software without -# specific prior written permission. -# -# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" -# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE -# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE -# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE -# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL -# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR -# SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER -# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, -# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF -# THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - -import functools -import os -from typing import Any -import yaml - -import agx -from agxBrick.brickLoaderUtils import createArgumentParser -from simulation.algoryx.behaviors.agx_interface import AGXInterface -from behaviors.common_behaviors import RSequence -from simulation.algoryx.tutorials.bt_application import BTApplication -from simulation.algoryx.tutorials import bt_wrapper as bt -import numpy as np -import py_trees - - -def post_tick_handler(snapshot_visitor, behavior_tree) -> None: - """Prints an ascii tree with the current snapshot status.""" - print( - '\n' - + py_trees.display.unicode_tree( - root=behavior_tree.root, - visited=snapshot_visitor.visited, - previously_visited=snapshot_visitor.visited, - ) - ) - name_ = 'root' + str(behavior_tree.count) - - -class BTDemo: - def __init__(self, world_interface: Any) -> None: - - self.world_interface = world_interface - self.root = None - - def build_tree(self) -> None: - """Hand code the BT.""" - - self.root = py_trees.composites.Selector(name='Fallback') - self.root.add_children( - [ - bt.PlaceBehavior( - name=f'Place BlueBox!', - world_interface=self.world_interface, - target_object='BlueBox', - reference_object='KittingBox', - pose=agx.Vec3(0, 0, 0.12), - ), - bt.PlaceBehavior( - name=f'Place BlueBox!', - world_interface=self.world_interface, - target_object='BlueBox', - reference_object='YellowBox', - pose=agx.Vec3(0, 0, 0.18), - ), - ] - ) - self.tree = py_trees.trees.BehaviourTree(self.root) - - def get_root(self) -> py_trees.composites.Selector: - return self.root - - def run(self, simulation_app: BTApplication) -> None: - """The BT execution is visualized in the terminal.""" - self.tree.visitors.append(py_trees.visitors.DebugVisitor()) - snapshot_visitor = py_trees.visitors.SnapshotVisitor() - self.tree.add_post_tick_handler( - functools.partial(post_tick_handler, snapshot_visitor) - ) - self.tree.visitors.append(snapshot_visitor) - self.tree.setup(timeout=15) - - simulation_app.run_bt(self.tree) - - self.tree.root.terminate(py_trees.common.Status.INVALID) - self.tree.shutdown() - self.tree.root = None - - -def main() -> None: - parser, args, leftover_args = createArgumentParser() - repo_path = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) - with open(os.path.join(repo_path, 'config/sim_objects.yaml')) as f: - data = yaml.safe_load(f) - - bt_app = BTApplication(args) - bt_app.init_agx_environment(args, data) - - agx_interface = AGXInterface(list(data.keys())) - - behavior_tree = BTDemo(agx_interface) - behavior_tree.build_tree() - - for _ in range(2): - behavior_tree.build_tree() - behavior_tree.run(bt_app) - bt_app.reset() - - del bt_app - bt_app = None - - -if __name__ == "__main__": - main() diff --git a/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/tutorials/startstop_gui.py b/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/tutorials/startstop_gui.py deleted file mode 100644 index 1f0e097d..00000000 --- a/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/tutorials/startstop_gui.py +++ /dev/null @@ -1,198 +0,0 @@ -""" -GUI to test start - restart - reset functionalities. - -Use with: python tutorials/startstop_gui.py --model models/BTDemo.yml:CoarseGripperInLabDR --decorate --timeStep 0.05 -from the /algoryx folder -""" - -# Copyright (c) 2022, ABB -# All rights reserved. -# -# Redistribution and use in source and binary forms, with -# or without modification, are permitted provided that -# the following conditions are met: -# -# * Redistributions of source code must retain the -# above copyright notice, this list of conditions -# and the following disclaimer. -# * Redistributions in binary form must reproduce the -# above copyright notice, this list of conditions -# and the following disclaimer in the documentation -# and/or other materials provided with the -# distribution. -# * Neither the name of ABB nor the names of its -# contributors may be used to endorse or promote -# products derived from this software without -# specific prior written permission. -# -# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" -# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE -# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE -# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE -# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL -# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR -# SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER -# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, -# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF -# THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - -import logging -import os -import time -from typing import Any, Dict, List - -from agxBrick.brickLoaderUtils import createArgumentParser -from agxClick import AgxApplication -from simulation.algoryx.agx_application import Application as App -import yaml - - -# OS dependent import -if os.name == 'nt': # Windows - import PySimpleGUIQt as gui -elif os.name == 'posix': # Linux Ubuntu - import PySimpleGUI as gui - - -class SandS_GUI(): - - def __init__(self, args: Any): - self._logger = logging.getLogger(__file__) - - repo_path = os.path.dirname(os.path.dirname((os.path.abspath(__file__)))) - with open(os.path.join(repo_path, 'config/sim_objects.yaml')) as f: - obj_data = yaml.safe_load(f) - - self._env = None - self._app = None - self.args = args - self.objects = obj_data - - self.gui = Layout('Start and Stop the AGX Simulation') - - def show(self): - """Define the GUI functionalities.""" - self.gui.initialize_layout() - - window = self.gui.get_window() - - # Interaction with the GUI Main menu - ret = 0 - while ret is not None: - ret, values = window.read() - - if ret == '__start__': - self.bringup() - - elif ret == '__close__': - self.shutdown() - - elif ret == '__restart__': - self.shutdown() - time.sleep(3) - self.bringup() - - elif ret == '__reset__': - self._app.reset() - - def bringup(self): - self._app = App(self.args) - self._app.bringup(self.objects, visual=True) - - def shutdown(self): - self._app.shutdown() - del self._app - self._app = None - time.sleep(3) - print('Application DOWN!') - - -class Layout(): - - def __init__(self, title: str): - """Initialize some internal parameters.""" - self.title = title - - self.window = None - self.layout = [] - - if os.name == 'nt': # Windows - self.size_parameter = 500 - elif os.name == 'posix': # Linux Ubuntu - self.size_parameter = 60 - - self.button = 36 - self.space = 5 - self.button_space = self.button*2 + self.space - - def get_window(self) -> gui.Window: - """Return the interface as window.""" - self.window = gui.Window(self.title, self.layout) - return self.window - - def initialize_layout(self): - """Initialize the layout with the standard LfD functionalities.""" - self.layout = [ - [ - gui.Button( - 'Start AGX', - disabled=False, - size=(None, None), - key='__start__' - ), - gui.Button( - 'Close AGX', - disabled=False, - size=(None, None), - key='__close__' - ), - ], - [ - gui.Button( - 'Reset Simulation', - disabled=False, - size=(None, None), - key='__reset__' - ), - gui.Button( - 'Restart AGX', - disabled=False, - size=(None, None), - key='__restart__' - ), - ] - ] - self.reshape([2, 2]) - - def reshape(self, shape: List[int]) -> None: - """Reshape the buttons as in the given input shape.""" - if type(sum(shape)) != int: - raise TypeError('The shape has not the correct data type: it requires INT.') - - buttons = [] - for row in self.layout: - for item in row: - if type(item) is gui.Button: - buttons.append(item) - - self.layout = [] - rows = len(shape) - for i in range(rows): - self.layout.append([]) - button_size = (self.button_space - (shape[i] - 1)*self.space)//(shape[i]) - for item in buttons[:shape[i]]: - item.Size = (button_size, None) - self.layout[-1].append(item) - # Remove appended buttons to from the list of buttons. - buttons = buttons[shape[i]:] - - -def main(): - parser, args, leftover_args = createArgumentParser() - - gui = SandS_GUI(args) - gui.show() - - -if __name__ == '__main__': - main() diff --git a/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/tutorials/task_bt.png b/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/tutorials/task_bt.png deleted file mode 100644 index ab1c691b..00000000 Binary files a/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/tutorials/task_bt.png and /dev/null differ diff --git a/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/tutorials/task_bt.svg b/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/tutorials/task_bt.svg deleted file mode 100644 index 56d9d0fe..00000000 --- a/kios_bt_planning/backups/dynamic_bt/simulation/algoryx/tutorials/task_bt.svg +++ /dev/null @@ -1,271 +0,0 @@ - - - - - - -pastafarianism - - - -Sequence - -Sequence - - - -Fallback - -Fallback - - - -Sequence->Fallback - - - - - -Fallback   - -Fallback - - - -Sequence->Fallback   - - - - - -Fallback     - -Fallback - - - -Sequence->Fallback     - - - - - -YellowBox on KittingBox? - -YellowBox on KittingBox? - - - -Fallback->YellowBox on KittingBox? - - - - - -Sequence - -Sequence - - - -Fallback->Sequence - - - - - -Fallback - -Fallback - - - -Sequence ->Fallback - - - - - -Place YellowBox! - -Place YellowBox! - - - -Sequence ->Place YellowBox! - - - - - -YellowBox Grasped? - -YellowBox Grasped? - - - -Fallback ->YellowBox Grasped? - - - - - -Pick YellowBox! - -Pick YellowBox! - - - -Fallback ->Pick YellowBox! - - - - - -GreenBox on YellowBox? - -GreenBox on YellowBox? - - - -Fallback  ->GreenBox on YellowBox? - - - - - -Sequence   - -Sequence - - - -Fallback  ->Sequence   - - - - - -Fallback    - -Fallback - - - -Sequence  ->Fallback    - - - - - -Place GreenBox! - -Place GreenBox! - - - -Sequence  ->Place GreenBox! - - - - - -GreenBox Grasped? - -GreenBox Grasped? - - - -Fallback   ->GreenBox Grasped? - - - - - -Pick GreenBox! - -Pick GreenBox! - - - -Fallback   ->Pick GreenBox! - - - - - -BlueBox on GreenBox? - -BlueBox on GreenBox? - - - -Fallback    ->BlueBox on GreenBox? - - - - - -Sequence    - -Sequence - - - -Fallback    ->Sequence    - - - - - -Fallback      - -Fallback - - - -Sequence   ->Fallback      - - - - - -Place BlueBox! - -Place BlueBox! - - - -Sequence   ->Place BlueBox! - - - - - -BlueBox Grasped? - -BlueBox Grasped? - - - -Fallback     ->BlueBox Grasped? - - - - - -Pick BlueBox! - -Pick BlueBox! - - - -Fallback     ->Pick BlueBox! - - - - - diff --git a/kios_bt_planning/backups/dynamic_bt/simulation/py_trees_interface.py b/kios_bt_planning/backups/dynamic_bt/simulation/py_trees_interface.py deleted file mode 100644 index 68959e44..00000000 --- a/kios_bt_planning/backups/dynamic_bt/simulation/py_trees_interface.py +++ /dev/null @@ -1,245 +0,0 @@ -"""Interfaces to py_trees from behavior tree strings.""" - -# Copyright (c) 2022, ABB -# All rights reserved. -# -# Redistribution and use in source and binary forms, with -# or without modification, are permitted provided that -# the following conditions are met: -# -# * Redistributions of source code must retain the -# above copyright notice, this list of conditions -# and the following disclaimer. -# * Redistributions in binary form must reproduce the -# above copyright notice, this list of conditions -# and the following disclaimer in the documentation -# and/or other materials provided with the -# distribution. -# * Neither the name of ABB nor the names of its -# contributors may be used to endorse or promote -# products derived from this software without -# specific prior written permission. -# -# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" -# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE -# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE -# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE -# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL -# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR -# SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER -# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, -# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF -# THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - -from dataclasses import dataclass -import time -from typing import Any, List, Tuple - -from behaviors.behavior_lists import ParameterizedNode -from behaviors.behavior_tree import BT -from behaviors import common_behaviors -import py_trees as pt - - -@dataclass -class PyTreeParameters: - """Data class for parameters for the PyTree run.""" - - behavior_lists: Any = None # Lists of the types of behaviors - behaviors: Any = None # Module containing actual behaviors - max_ticks: int = 200 # Maximum number of ticks to run - sim_tick_time: int = 2.5 # If the simulator is given, how much time in the simulator every tick takes - max_time: float = 10000.0 # Maximum time in s to run - max_fails: int = 1 # Maximum number of failure states before breaking - successes_required: int = 2 # Number of success states required before breaking - show_world: bool = False # Animate the run - verbose: bool = False # Extra prints - - -class PyTree(pt.trees.BehaviourTree): - """A class containing a behavior tree. Inherits from the py tree BehaviorTree class.""" - - def __init__( - self, - node_list: List[ParameterizedNode], - parameters: PyTreeParameters = None, - world_interface: Any = None, - root: Any = None - ): - if parameters is not None: - self.par = parameters - else: - self.par = PyTreeParameters() - if root is not None: - self.root = root - node_list = self.get_bt_from_root() - - self.bt = BT(node_list, self.par.behavior_lists) - self.behaviors = self.par.behaviors - self.world_interface = world_interface - self.failed = False - self.timeout = False - - if root is None: - self.root, has_children = common_behaviors.get_node( - node_list[0], world_interface, self.par.verbose) - node_list.pop(0) - else: - has_children = False - - super().__init__(root=self.root) - if has_children: - self.create_from_list(node_list, self.root) - - def get_bt_from_root(self) -> List[str]: - """ - Return bt string from py tree root by cleaning the ascii tree from py trees. - - Not complete or beautiful by any means but works for many trees. - """ - string = pt.display.ascii_tree(self.root) - string = string.replace('[o] ', '') - string = string.replace('[-] ', '') - string = string.replace('\t', '') - string = string.replace('-->', '') - string = string.replace('Fallback', 'f(') - string = string.replace('Sequence', 's(') - bt = string.split('\n') - bt = bt[:-1] # Remove empty element because of final newline - - prev_leading_spaces = 999999 - for i in range(len(bt) - 1, -1, -1): - leading_spaces = len(bt[i]) - len(bt[i].lstrip(' ')) - bt[i] = bt[i].lstrip(' ') - if leading_spaces > prev_leading_spaces: - for _ in range(round((leading_spaces - prev_leading_spaces) / 4)): - bt.insert(i + 1, ')') - prev_leading_spaces = leading_spaces - - bt_obj = BT(bt, self.par.behavior_lists) - bt_obj.close() - - return bt_obj.bt - - def create_from_list( - self, - node_list: List[str], - node: pt.composites.Composite - ) -> pt.composites.Composite: - """Recursive function to generate the tree from a list.""" - while len(node_list) > 0: - if node_list[0] == ')': - node_list.pop(0) - return node - - newnode, has_children = self.behaviors.get_node( - node_list[0], self.world_interface, self.par.verbose) - node_list.pop(0) - if has_children: - # Node is a control node or decorator with children. - # Add subtree via string and then add to parent - newnode = self.create_from_list(node_list, newnode) - node.add_child(newnode) - else: - # Node is a leaf/action node - add to parent, then keep looking for siblings - node.add_child(newnode) - - # This return is only reached if there are too few up nodes - return node - - def run_bt(self, simulator=None) -> Tuple[int, bool]: - """Run the behavior tree.""" - ticks = 0 - straight_fails = 0 - successes = 0 - status_ok = True - - start = time.time() - if simulator is not None: - self.world_interface.remove_locks() - self.world_interface.preempt_skill() - self.world_interface.add_merger() - - while (self.root.status is not pt.common.Status.FAILURE or - straight_fails < self.par.max_fails) and\ - (self.root.status is not pt.common.Status.SUCCESS or - successes < self.par.successes_required) and\ - ticks < self.par.max_ticks and status_ok: - - status_ok = self.world_interface.get_feedback() # Wait for connection - - if status_ok: - if self.par.verbose: - print('Tick', ticks) - print(self.root.status) - self.root.tick_once() - self.world_interface.send_references() - # HACK!! - if simulator is not None: - current_time = simulator.app.getTimeStamp() - while simulator.app.getTimeStamp() < self.par.sim_tick_time + current_time: - simulator.stepApplication() - - if self.par.show_world: - world.animate_state(self.world_interface.state) - - ticks += 1 - if self.root.status is pt.common.Status.SUCCESS: - successes += 1 - else: - successes = 0 - - if self.root.status is pt.common.Status.FAILURE: - straight_fails += 1 - else: - straight_fails = 0 - - if time.time() - start > self.par.max_time: - status_ok = False - self.timeout = True - print('Max time expired') - - if self.par.verbose: - print('Total episode ticks: ', ticks) - print('Total episode time: ', time.time() - start) - - if self.par.show_world: - world.animate() - world.save_world('testworld') - - if ticks >= self.par.max_ticks: - self.timeout = True - if straight_fails >= self.par.max_fails: - self.failed = True - return ticks, status_ok - - def step_bt(self, simulator=None) -> bool: - """Step the BT one step.""" - status_ok = True - - status_ok = self.world_interface.get_feedback() # Wait for connection - - if status_ok: - self.root.tick_once() - self.world_interface.send_references() - # HACK!! - if simulator is not None: - simulator.__step_ahead(3) - - return status_ok - - def save_fig( - self, - path: str, - name: str = 'Behavior tree', - static: bool = True, - blackboard: bool = False - ) -> None: - """Save the tree as a figure.""" - pt.display.render_dot_tree( - self.root, - name=name, - target_directory=path, - static=static, - with_blackboard_variables=blackboard - ) diff --git a/kios_bt_planning/backups/dynamic_bt/user_interface/__init__.py b/kios_bt_planning/backups/dynamic_bt/user_interface/__init__.py deleted file mode 100644 index 5f282702..00000000 --- a/kios_bt_planning/backups/dynamic_bt/user_interface/__init__.py +++ /dev/null @@ -1 +0,0 @@ - \ No newline at end of file diff --git a/kios_bt_planning/backups/dynamic_bt/user_interface/agx_gui.py b/kios_bt_planning/backups/dynamic_bt/user_interface/agx_gui.py deleted file mode 100644 index 31f56c0e..00000000 --- a/kios_bt_planning/backups/dynamic_bt/user_interface/agx_gui.py +++ /dev/null @@ -1,304 +0,0 @@ -"""Main node launching the GUI for the GP+LfD framework.""" - -# Copyright (c) 2022, ABB -# All rights reserved. -# -# Redistribution and use in source and binary forms, with -# or without modification, are permitted provided that -# the following conditions are met: -# -# * Redistributions of source code must retain the -# above copyright notice, this list of conditions -# and the following disclaimer. -# * Redistributions in binary form must reproduce the -# above copyright notice, this list of conditions -# and the following disclaimer in the documentation -# and/or other materials provided with the -# distribution. -# * Neither the name of ABB nor the names of its -# contributors may be used to endorse or promote -# products derived from this software without -# specific prior written permission. -# -# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" -# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE -# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE -# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE -# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL -# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR -# SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER -# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, -# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF -# THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - -import os -from typing import List - -from user_interface.simple_gui import SimpleGUI - -# OS dependent import -if os.name == 'nt': # Windows - import PySimpleGUIQt as gui -elif os.name == 'posix': # Linux Ubuntu - import PySimpleGUI as gui - - -class AGXGUILayout(SimpleGUI): - """Meta class to handle more easily the Simple GUI layout.""" - - def __init__(self, title: str, root_folder: str = None) -> None: - """Initialize some internal parameters.""" - super().__init__(title) - - self.root_folder = root_folder - self.keys = [] - - def initialize_layout(self) -> None: - """Initialize the layout with the standard LfD functionalities.""" - self.layout = [ - [ - gui.Input( - 'Select a folder to load past runs...', - key='__folder_display__', - enable_events=True, - disabled=True, - size=(self.size_parameter, None) - ), - gui.FolderBrowse( - 'Select log folder', - target='__folder_display__', - initial_folder=self.root_folder, - key='__log_folder__' - ) - ], - [ - gui.Frame( - ' Simulation Environment ', - self.get_agx_layout(), - expand_x=True, - expand_y=True, - title_location=gui.TITLE_LOCATION_TOP - ) - ], - [ - gui.Frame( - ' Behavior Tree ', - self.get_bt_layout(), - expand_x=True, - expand_y=True, - title_location=gui.TITLE_LOCATION_TOP - ) - ], - [ - gui.Frame( - ' Genetic Programming ', - self.get_gp_layout(), - expand_x=True, - expand_y=True, - title_location=gui.TITLE_LOCATION_TOP - ) - ], - [ - gui.Frame( - ' Learning from Demonstration ', - self.get_lfd_layout(), - expand_x=True, - expand_y=True, - title_location=gui.TITLE_LOCATION_TOP - ) - ] - ] - - self.keys += ['__log_folder__'] - - def get_agx_layout(self) -> List[List[gui.Element]]: - """Layout for BT related stuff.""" - agx_layout = [ - [ - gui.Button( - 'Initial Configuration', - tooltip='Show the initial configuration of the simulation environment.', - disabled=False, - size=(None, None), - key='__initial__' - ), - gui.Button( - 'Target Configuration', - tooltip='Show the final configuration of the simulation environment.', - disabled=False, - size=(None, None), - key='__target__' - ), - ] - ] - self.keys += ['__initial__', '__target__'] - self.reshape(agx_layout, [2]) - return agx_layout - - def get_bt_layout(self) -> List[List[gui.Element]]: - """Layout for BT related stuff.""" - bt_layout = [ - [ - gui.Button( - 'Run', - tooltip='Run the loaded Behavior Tree and execute the learnt task.', - disabled=True, - size=(None, None), - key='__run_tree__' - ), - gui.Button( - 'Show', - tooltip='Display the Behavior Tree.', - disabled=True, - size=(None, None), - key='__show_tree__' - ), - gui.Button( - 'Save', - tooltip='Save the Behavior Tree for future reference.', - disabled=True, - size=(None, None), - key='__save_tree__' - ), - ] - ] - self.keys += ['__run_tree__', '__show_tree__', '__save_tree__'] - self.reshape(bt_layout, [3]) - return bt_layout - - def get_gp_layout(self) -> List[List[gui.Element]]: - """Layout for GP related stuff.""" - gp_layout = [ - [gui.Text( - 'Current generation:', key='__gen__', size=(self.size_parameter, None) - )], - [gui.Text('Best fitness score:', key='__fitness__', size=(self.size_parameter, None))], - [ - gui.Button( - 'Start', - tooltip='Start the evolution process with Genetic Programming.', - size=(None, None), - key='__start__' - ), - gui.Button( - 'Plot Fitness', - tooltip='Plot the fitness function.', - disabled=True, - size=(None, None), - key='__plot_fitness__' - ), - gui.Button( - 'Increase Generations', - tooltip='Increase the number of generations by the default value.', - disabled=True, - size=(None, None), - key='__increase__' - ), - ] - ] - self.keys += ['__start__', '__plot_fitness__', '__increase__'] - self.reshape(gp_layout[2:], [3]) - return gp_layout - - def get_lfd_layout(self) -> List[List[gui.Element]]: - """Layout for LfD related stuff.""" - lfd_layout = [ - [gui.Text( - 'Number of demonstrations:', key='__n_demos__', size=(self.size_parameter, None) - )], - [gui.Text('Number of nodes:', key='__n_nodes__', size=(self.size_parameter, None))], - [gui.ProgressBar(20, size=(52, 3), key='__progress__', visible=False)], - [ - gui.Button( - 'Add Demonstration', - tooltip='Teach the robot a new task by guiding its arm.', - disabled=False, - size=(None, None), - key='__new_demo__' - ), - gui.Button( - 'Add Target', - tooltip='Demonstrate a new step in the task from the current taget configuration.', - disabled=False, - size=(None, None), - key='__new_target__' - ), - gui.Button( - 'Plot Clusters', - tooltip='Plot the clusters for frame inference.', - disabled=True, - size=(None, None), - key='__cluster__' - ), - ] - ] - self.keys += ['__new_demo__', '__new_target__', '__cluster__'] - self.reshape(lfd_layout[3:], [3]) - return lfd_layout - - def reshape(self, layout: List[List[gui.Element]], shape: List[int]) -> List[List[gui.Element]]: - """ - Reshape the buttons as in the given input shape. - - Args - ---- - - shape: list where its size indicates the number of rows and every element is - the number of buttons in that row. - """ - if type(sum(shape)) != int: - raise TypeError('The shape has not the correct data type: it requires INT.') - - buttons = [] - for row in layout: - for item in row: - if type(item) is gui.Button: - buttons.append(item) - - new_layout = [] - rows = len(shape) - for i in range(rows): - new_layout.append([]) - button_size = (self.button_space - (shape[i] - 1)*self.space)//(shape[i]) - for item in buttons[:shape[i]]: - item.Size = (button_size, None) - new_layout[-1].append(item) - # Remove appended buttons to from the list of buttons. - buttons = buttons[shape[i]:] - - return new_layout - - def disable_all_but(self, window: gui.Window, key: str) -> None: - """Disable some buttons if the GUI is run offline.""" - window.read(timeout=0) - for val in self.keys: - if val == key: - continue - window.find_element(val).update(disabled=True) - - def enable_all_but(self, window: gui.Window, key: str = None) -> None: - """Disable some buttons if the GUI is run offline.""" - window.read(timeout=0) - for val in self.keys: - if val == key: - continue - window.find_element(val).update(disabled=False) - - def enable_from_flags( - self, - window: gui.Window, - has_BT: bool, - has_GP: bool, - has_Demo: bool - ) -> None: - """Disable some buttons if the GUI is run offline.""" - window.read(timeout=0) - self.enable_all_but(window) - if not has_BT: - window.find_element('__run_tree__').update(disabled=True) - window.find_element('__show_tree__').update(disabled=True) - window.find_element('__save_tree__').update(disabled=True) - if not has_GP: - window.find_element('__plot_fitness__').update(disabled=True) - window.find_element('__increase__').update(disabled=True) - if not has_Demo: - window.find_element('__cluster__').update(disabled=True) diff --git a/kios_bt_planning/backups/dynamic_bt/user_interface/agx_gui_show.py b/kios_bt_planning/backups/dynamic_bt/user_interface/agx_gui_show.py deleted file mode 100644 index ecd16644..00000000 --- a/kios_bt_planning/backups/dynamic_bt/user_interface/agx_gui_show.py +++ /dev/null @@ -1,80 +0,0 @@ -"""Visualize the GUI to simulate the LfD+GP framework with the AGX Simulator.""" - -# Copyright (c) 2022, ABB -# All rights reserved. -# -# Redistribution and use in source and binary forms, with -# or without modification, are permitted provided that -# the following conditions are met: -# -# * Redistributions of source code must retain the -# above copyright notice, this list of conditions -# and the following disclaimer. -# * Redistributions in binary form must reproduce the -# above copyright notice, this list of conditions -# and the following disclaimer in the documentation -# and/or other materials provided with the -# distribution. -# * Neither the name of ABB nor the names of its -# contributors may be used to endorse or promote -# products derived from this software without -# specific prior written permission. -# -# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" -# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE -# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE -# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE -# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL -# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR -# SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER -# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, -# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF -# THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - -import os - -from typing import Any - -from user_interface.agx_gui import AGXGUILayout - - -class GUI(): - - def __init__(self): - - self.agx_GUI = AGXGUILayout('Algoryx Simulation for Learning Behavior Trees') - self.running = False - - def show(self, build_tree_function: Any = None): - """Define the GUI functionalities.""" - self.agx_GUI.initialize_layout() - - window = self.agx_GUI.get_window() - - # Interaction with the GUI Main menu - ret = 0 - while ret is not None: - ret, values = window.read() - - if ret == '__folder_display__' and os.path.isdir(values['__folder_display__']): - pass - - if ret == '__start__': - if not self.running: - self.agx_GUI.disable_all_but(window, '__start__') - window.find_element('__start__').update('Stop') - self.running = True - else: - self.agx_GUI.enable_all_but(window) - window.find_element('__start__').update('Start') - self.running = False - - -def main(): - - gui = GUI() - gui.show() - - -if __name__ == '__main__': - main() diff --git a/kios_bt_planning/backups/dynamic_bt/user_interface/gp_gui.py b/kios_bt_planning/backups/dynamic_bt/user_interface/gp_gui.py deleted file mode 100644 index c36e20e2..00000000 --- a/kios_bt_planning/backups/dynamic_bt/user_interface/gp_gui.py +++ /dev/null @@ -1,109 +0,0 @@ -""" -Main node launching the GUI for the LfD framework. - -In this node: - - the Mobile Base is NOT enabled. - - the perception uses MaskRCNN for object detection. - - it is possible to enable HRI to disambiguate the scene. -""" - -# Copyright (c) 2022, ABB -# All rights reserved. -# -# Redistribution and use in source and binary forms, with -# or without modification, are permitted provided that -# the following conditions are met: -# -# * Redistributions of source code must retain the -# above copyright notice, this list of conditions -# and the following disclaimer. -# * Redistributions in binary form must reproduce the -# above copyright notice, this list of conditions -# and the following disclaimer in the documentation -# and/or other materials provided with the -# distribution. -# * Neither the name of ABB nor the names of its -# contributors may be used to endorse or promote -# products derived from this software without -# specific prior written permission. -# -# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" -# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE -# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE -# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE -# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL -# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR -# SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER -# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, -# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF -# THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - -import os -from typing import List - -from user_interface.simple_gui import SimpleGUI - -# OS dependent import -if os.name == 'nt': # Windows - import PySimpleGUIQt as gui -elif os.name == 'posix': # Linux Ubuntu - import PySimpleGUI as gui - - -class GPGUILayout(SimpleGUI): - """Meta class to handle more easily the Simple GUI layout.""" - - def __init__(self, title: str): - """Initialize some internal parameters.""" - super().__init__(title) - - def initialize_layout(self): - """Initialize the layout with the standard LfD functionalities.""" - self.layout = [ - [ - gui.Input( - 'No folder selected', - key='__folder_display__', - enable_events=True, - disabled=True, - size=(self.size_parameter, None) - ), - gui.FolderBrowse( - 'Select log folder', - target='__folder_display__', - key='__log_folder__' - ) - ], - [gui.Text( - 'Current generation:', key='__gen__', size=(self.size_parameter, None) - )], - [gui.Text('Best fitness score:', key='__fitness__', size=(self.size_parameter, None))], - [gui.ProgressBar(20, size=(52, 3), key='__progress__', visible=False)], - [ - gui.Button( - 'Start GP', - tooltip='Start the evolution process with Genetic Programming.', - size=(None, None), - key='__start__' - ), - gui.Button( - 'Run tree', - tooltip='Run the best Behavior Tree.', - disabled=True, - size=(None, None), - key='__run__' - ), - gui.Button( - 'Show tree', - tooltip='Display the best Behavior Tree.', - disabled=True, - size=(None, None), - key='__show_tree__' - ), - ] - ] - - self.reshape([3]) - - def reshape(self, shape: List[int]) -> None: - return super().reshape(shape, 4) diff --git a/kios_bt_planning/backups/dynamic_bt/user_interface/gui_ddmenu.py b/kios_bt_planning/backups/dynamic_bt/user_interface/gui_ddmenu.py deleted file mode 100644 index 27454faa..00000000 --- a/kios_bt_planning/backups/dynamic_bt/user_interface/gui_ddmenu.py +++ /dev/null @@ -1,98 +0,0 @@ -"""Define a dropdown menu in the GUI.""" - -# Copyright (c) 2022, ABB -# All rights reserved. -# -# Redistribution and use in source and binary forms, with -# or without modification, are permitted provided that -# the following conditions are met: -# -# * Redistributions of source code must retain the -# above copyright notice, this list of conditions -# and the following disclaimer. -# * Redistributions in binary form must reproduce the -# above copyright notice, this list of conditions -# and the following disclaimer in the documentation -# and/or other materials provided with the -# distribution. -# * Neither the name of ABB nor the names of its -# contributors may be used to endorse or promote -# products derived from this software without -# specific prior written permission. -# -# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" -# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE -# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE -# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE -# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL -# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR -# SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER -# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, -# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF -# THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - -import logging -import os -from typing import List - -# OS dependent import -if os.name == 'nt': # Windows - import PySimpleGUIQt as gui -elif os.name == 'posix': # Linux Ubuntu - import PySimpleGUI as gui - -logging.basicConfig(level=logging.NOTSET) - - -class GUIMenu(): - - def __init__( - self, - title: str = 'Simple Menu', - option_list: List[str] = ['1', '2'], - length: int = 15 - ): - self.title = title - self.options = option_list - - if os.name == 'nt': # Windows - self.length = None - elif os.name == 'posix': # Linux Ubuntu - self.length = length - - self.layout = [] - self.window = None - - def set_title(self, title: str) -> None: - """Set the title for the menu.""" - self.title = title - - def set_options(self, option_list: List[str]) -> None: - """Set the options to choose from in the menu.""" - self.options = option_list - - def get_layout(self) -> List[List[gui.Button]]: - """Return the layout of this window.""" - return self.layout - - def get_window(self) -> gui.Window: - """Return the interface as window.""" - self.window = gui.Window('Options', self.layout) - return self.window - - def initialize_layout(self) -> str: - """Define the layout for the GUI window where the user can select some options.""" - menu_def = ['Options', [self.options]] - - key_ = '__menu__' - - self.layout.append( - [gui.ButtonMenu( - self.title, - menu_def=menu_def, - border_width=self.length, - key=key_ - )] - ) - - return key_ diff --git a/kios_bt_planning/backups/dynamic_bt/user_interface/lfd_gui.py b/kios_bt_planning/backups/dynamic_bt/user_interface/lfd_gui.py deleted file mode 100644 index d02342eb..00000000 --- a/kios_bt_planning/backups/dynamic_bt/user_interface/lfd_gui.py +++ /dev/null @@ -1,138 +0,0 @@ -""" -Main node launching the GUI for the LfD framework. - -In this node: - - the Mobile Base is NOT enabled. - - the perception uses MaskRCNN for object detection. - - it is possible to enable HRI to disambiguate the scene. -""" - -# Copyright (c) 2022, ABB -# All rights reserved. -# -# Redistribution and use in source and binary forms, with -# or without modification, are permitted provided that -# the following conditions are met: -# -# * Redistributions of source code must retain the -# above copyright notice, this list of conditions -# and the following disclaimer. -# * Redistributions in binary form must reproduce the -# above copyright notice, this list of conditions -# and the following disclaimer in the documentation -# and/or other materials provided with the -# distribution. -# * Neither the name of ABB nor the names of its -# contributors may be used to endorse or promote -# products derived from this software without -# specific prior written permission. -# -# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" -# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE -# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE -# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE -# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL -# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR -# SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER -# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, -# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF -# THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - -import os -from typing import List - -from user_interface.simple_gui import SimpleGUI - -# OS dependent import -if os.name == 'nt': # Windows - import PySimpleGUIQt as gui -elif os.name == 'posix': # Linux Ubuntu - import PySimpleGUI as gui - - -class LfDGUILayout(SimpleGUI): - """Meta class to handle more easily the Simple GUI layout.""" - - def __init__(self, title: str): - """Initialize some internal parameters.""" - super().__init__(title) - - def initialize_layout(self): - """Initialize the layout with the standard LfD functionalities.""" - self.layout = [ - [ - gui.Input( - 'No folder selected', - key='__folder_display__', - enable_events=True, - disabled=True, - size=(self.size_parameter, None) - ), - gui.FolderBrowse( - 'Select demo folder', - target='__folder_display__', - key='__demo_folder__' - ) - ], - [gui.Text( - 'Number of demonstrations:', key='__n_demos__', size=(self.size_parameter, None) - )], - [gui.Text('Number of nodes:', key='__n_nodes__', size=(self.size_parameter, None))], - [gui.ProgressBar(20, size=(52, 3), key='__progress__', visible=False)], - [ - gui.Button( - 'Add demonstration', - tooltip='Teach the robot a new task by guiding its arm.', - size=(None, None), - key='__new_demo__' - ), - gui.Button( - 'Run', - tooltip='Run the loaded Behavior Tree and execute the learnt task.', - disabled=True, - size=(None, None), - key='__run__' - ) - ], - [ - gui.Button( - 'Show tree', - tooltip='Display the Behavior Tree.', - disabled=True, - size=(None, None), - key='__show_tree__' - ), - gui.Button( - 'Save tree', - tooltip='Save the Behavior Tree for future reference.', - disabled=True, - size=(None, None), - key='__save_tree__' - ), - gui.Button( - 'Plot clusters', - tooltip='Plot the clusters for frame inference.', - disabled=True, - size=(None, None), - key='__plot_cluster__' - ), - gui.Button( - 'Load tree', - tooltip='Load a learnt tree and run it.', - disabled=True, - size=(None, None), - key='__load_tree__' - ) - ] - ] - - self.reshape([2, 2, 2]) - - def reshape(self, shape: List[int]) -> None: - return super().reshape(shape, 4) - - def display_offline_layout(self, window: gui.Window) -> None: - """Disable some buttons if the GUI is run offline.""" - window.read(timeout=0) - window.find_element('__new_demo__').update(visible=False) - window.find_element('__run__').update(visible=False) diff --git a/kios_bt_planning/backups/dynamic_bt/user_interface/lfd_gui_actions.py b/kios_bt_planning/backups/dynamic_bt/user_interface/lfd_gui_actions.py deleted file mode 100644 index eb983810..00000000 --- a/kios_bt_planning/backups/dynamic_bt/user_interface/lfd_gui_actions.py +++ /dev/null @@ -1,143 +0,0 @@ -"""Enable robot actions through the GUI.""" - -# Copyright (c) 2022, ABB -# All rights reserved. -# -# Redistribution and use in source and binary forms, with -# or without modification, are permitted provided that -# the following conditions are met: -# -# * Redistributions of source code must retain the -# above copyright notice, this list of conditions -# and the following disclaimer. -# * Redistributions in binary form must reproduce the -# above copyright notice, this list of conditions -# and the following disclaimer in the documentation -# and/or other materials provided with the -# distribution. -# * Neither the name of ABB nor the names of its -# contributors may be used to endorse or promote -# products derived from this software without -# specific prior written permission. -# -# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" -# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE -# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE -# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE -# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL -# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR -# SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER -# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, -# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF -# THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - -from copy import copy -import logging -import os -from typing import Any, List - -# OS dependent import -if os.name == 'nt': # Windows - import PySimpleGUIQt as gui -elif os.name == 'posix': # Linux Ubuntu - import PySimpleGUI as gui - -logging.basicConfig(level=logging.NOTSET) - - -class GUIActions(): - - def __init__(self, title: str, action_list: List[str], button_length: int = 15): - self.title = title - self.action_list = action_list - - if os.name == 'nt': # Windows - self.button_length = None - elif os.name == 'posix': # Linux Ubuntu - self.button_length = button_length - - self.layout = [] - self.window = None - - def get_layout(self) -> List[List[gui.Button]]: - """Return the layout of this window.""" - return self.layout - - def get_window(self) -> gui.Window: - """Return the interface as window.""" - self.window = gui.Window(self.title, self.layout) - return self.window - - def add_action_button(self, action_name: str, tooltip: str = None) -> None: - """Add a custom action to the GUI.""" - action_key = action_name.lower() - action_key = '__' + action_key + '__' - self.layout.append( - [gui.Button( - button_text=action_name, - tooltip=tooltip, - size=(self.button_length, None), - key=str(action_key) - )] - ) - - def initialize_layout(self) -> None: - """Define the layout for the GUI window where the user can select the actions.""" - actions_to_add = copy(self.action_list) - if 'pick' in actions_to_add: - self.layout.append( - [gui.Button( - 'Pick', - tooltip='Close the grippers around the target object.', - size=(self.button_length, None), - key='__pick__' - )] - ) - actions_to_add.pop(actions_to_add.index('pick')) - if 'place' in actions_to_add: - self.layout.append( - [gui.Button( - 'Fine Place', - tooltip='Open the grippers. Use when the target position has to be precise.', - size=(self.button_length, None), - key='__place__' - )] - ) - actions_to_add.pop(actions_to_add.index('place')) - if 'drop' in actions_to_add: - self.layout.append( - [gui.Button( - 'Drop', - tooltip='Open the grippers. Use when the target position does not matter much.', - size=(self.button_length, None), - key='__drop__' - )] - ) - actions_to_add.pop(actions_to_add.index('drop')) - - for action in actions_to_add: - action_key = '__' + action + '__' - self.layout.append( - [gui.Button( - action.capitalize(), - tooltip=f'Perform the {action} action.', - size=(self.button_length, None), - key=action_key - )] - ) - actions_to_add.pop(actions_to_add.index(action)) - - def execute_action(self, event: str, action_function: Any, n_demos: int, *kwargs: Any) -> None: - """Map the button to the robot action.""" - if event == '__pick__': - action_function(n_demos, action_name='pick', gripper_state='closed') - elif event == '__place__': - action_function(n_demos, action_name='place', gripper_state='open') - elif event == '__drop__': - action_function(n_demos, action_name='drop', gripper_state='open') - elif event is None: - pass - else: - action_function(n_demos, action_name=event[2:-2], *kwargs) - - # TODO: add MoveBase buttons diff --git a/kios_bt_planning/backups/dynamic_bt/user_interface/simple_gui.py b/kios_bt_planning/backups/dynamic_bt/user_interface/simple_gui.py deleted file mode 100644 index b5d6ffef..00000000 --- a/kios_bt_planning/backups/dynamic_bt/user_interface/simple_gui.py +++ /dev/null @@ -1,147 +0,0 @@ -"""Common GUI functionalities.""" - -# Copyright (c) 2022, ABB -# All rights reserved. -# -# Redistribution and use in source and binary forms, with -# or without modification, are permitted provided that -# the following conditions are met: -# -# * Redistributions of source code must retain the -# above copyright notice, this list of conditions -# and the following disclaimer. -# * Redistributions in binary form must reproduce the -# above copyright notice, this list of conditions -# and the following disclaimer in the documentation -# and/or other materials provided with the -# distribution. -# * Neither the name of ABB nor the names of its -# contributors may be used to endorse or promote -# products derived from this software without -# specific prior written permission. -# -# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" -# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE -# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE -# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE -# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL -# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR -# SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER -# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, -# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF -# THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - -import os -from typing import List - -# OS dependent import -if os.name == 'nt': # Windows - import PySimpleGUIQt as gui -elif os.name == 'posix': # Linux Ubuntu - import PySimpleGUI as gui - - -class SimpleGUI(): - """Meta class to handle more easily the Simple GUI layout.""" - - def __init__(self, title: str): - """Initialize some internal parameters.""" - self.title = title - - self.window = None - self.layout = [] - - if os.name == 'nt': # Windows - self.size_parameter = 500 - elif os.name == 'posix': # Linux Ubuntu - self.size_parameter = 60 - - self.button = 36 - self.space = 5 - self.button_space = self.button*2 + self.space - - def get_window(self) -> gui.Window: - """Return the interface as window.""" - self.window = gui.Window(self.title, self.layout) - return self.window - - def get_layout(self) -> List[List[gui.Button]]: - """Return the layout of this window.""" - return self.layout - - def add_button( - self, - label: str, - tooltip: str, - disabled: bool, - key: str, - size: int = None, - row: int = None - ) -> None: - """Add a button to the layout.""" - if size is None or size > self.button_space: - size = self.button_space - - if row == None: - self.add_row() - row = -1 - - self.layout[row].append( - gui.Button( - button_text=label, - tooltip=tooltip, - disabled=disabled, - size=(size, None), - key=key - ) - ) - - def add_row(self) -> None: - """Add a row to the layout.""" - self.layout.append([]) - - def reshape(self, shape: List[int], button_idx: int) -> None: - """ - Reshape the buttons as in the given input shape. - - Args - ---- - - shape: list where its size indicates the number of rows and every element is - the number of buttons in that row. - - button_idx: index of the layout where the buttons are placed. - """ - if type(sum(shape)) != int: - raise TypeError('The shape has not the correct data type: it requires INT.') - - buttons = [] - for row in self.layout[button_idx:]: - for item in row: - if type(item) is gui.Button: - buttons.append(item) - - self.layout = self.layout[:button_idx] - - rows = len(shape) - for i in range(rows): - self.add_row() - button_size = (self.button_space - (shape[i] - 1)*self.space)//(shape[i]) - for item in buttons[:shape[i]]: - item.Size = (button_size, None) - self.layout[-1].append(item) - # Remove appended buttons to from the list of buttons. - buttons = buttons[shape[i]:] - - def hide_button(self, window: gui.Window, key: str) -> None: - """Hide a button through its key.""" - window.read(timeout=0) - window.find_element(key).update(visible=False) - - def disable_button(self, window: gui.Window, key: str) -> None: - """Deactivate a button through its key.""" - window.read(timeout=0) - window.find_element(key).update(disabled=True) - - def enable_button(self, window: gui.Window, key: str) -> None: - """Activate a button through its key.""" - window.read(timeout=0) - window.find_element(key).update(disabled=False) diff --git a/kios_bt_planning/backups/json_io_test.py b/kios_bt_planning/backups/json_io_test.py deleted file mode 100644 index 2f099d9c..00000000 --- a/kios_bt_planning/backups/json_io_test.py +++ /dev/null @@ -1,204 +0,0 @@ -import json -import os -from typing import Set, Dict, List, Any - -# mios_parameters example -mios_parameters = { - "skill_type": "BBCartesianMove", - "skill_objects": {"CartesianMove": None}, - "skill_parameters": { - "skill": { - "objects": None, - "time_max": 30, - "BBCartesianMove": { - "dX_d": [0.2, 0.2], - "ddX_d": [0.1, 0.1], - "DeltaX": [0, 0, 0, 0, 0, 0], - "K_x": [1500, 1500, 1500, 600, 600, 600], - }, - }, - "control": {"control_mode": 0}, - "user": { - "env_X": [0.01, 0.01, 0.002, 0.05, 0.05, 0.05], - "env_dX": [0.001, 0.001, 0.001, 0.005, 0.005, 0.005], - }, - }, -} - -# an action example -action_template = { - # * this is the name of the action. This should be unique and is used as the identifier of the action. - "name": "move", - # * this is the variables that the action and its conditions needs. The variables need to be grounded later. - "entities": { - "entity_names", # e.g. robot desk cabinet ball ... - }, - # * this is the mios parameters that are necessary for robot interfaces. - "mios_parameters": None, # this part is skipped for now - # * this is the precondition of the action. This should be checked before the action is executed. - # * They are explicitly implemented in the subtree as condition nodes in the sequence. - "preconditions": { - "true": { - "object_name": { # e.g. cabinet - "property": "optional_object_name", # e.g. open: "", on: "desk" - }, - }, - "false": { - "object_name": { - "property": "optional_object_name", - } - }, - "propositions": Set[ # * this is for compensating the informations. should be used as less as possible. - str - ], - }, - # * this is the effect of the action. This should be exerted to the world by the action - # * node. All the effects are considered here, not only the purposes but also the side effects. - "effects": { - "true": { - "object_name": { # e.g. cabinet - "property": "optional_object_name", # e.g. open: "", on: "desk" - }, - }, - "false": { - "object_name": { - "property": "optional_object_name", - } - }, - "propositions": Set[ # * this is for compensating the informations. should be used as less as possible. - str - ], - }, - # * this is the reason to conduct this action. This should be checked after - # * the action is executed, which is explicitly implemented in the subtree as a condition - # * node in the selector. - "purposes": { - "true": { - "object_name": { # e.g. cabinet - "property": "optional_object_name", - # e.g. open: "", this is a property of the object - # e.g. on: "desk" this is a directedrelation between objects - }, - }, - "false": { - "object_name": { - "property": "optional_object_name", - } - }, - "propositions": Set[ # * this is for compensating the informations. should be used as less as possible. - str - ], - }, -} - -# an action example -action_template = { - # * this is the name of the action. This should be unique and is used as the identifier of the action. - "name": "move", - # * this is the variables that the action and its conditions needs. The variables need to be grounded later. - "entities": { - "entity_names", # e.g. robot desk cabinet ball ... - }, - # * this is the mios parameters that are necessary for robot interfaces. - "mios_parameters": None, # this part is skipped for now - # * this is the precondition of the action. This should be checked before the action is executed. - # * They are explicitly implemented in the subtree as condition nodes in the sequence. - "preconditions": { - "true": { - "object_name": { # e.g. cabinet - "property": "optional_object_name", # e.g. open: "", on: "desk" - }, - }, - "false": { - "object_name": { - "property": "optional_object_name", - } - }, - "propositions": Set[ # * this is for compensating the informations. should be used as less as possible. - str - ], - }, - # * this is the effect of the action. This should be exerted to the world by the action - # * node. All the effects are considered here, not only the purposes but also the side effects. - "effects": { - "true": { - "object_name": { # e.g. cabinet - "property": "optional_object_name", # e.g. open: "", on: "desk" - }, - }, - "false": { - "object_name": { - "property": "optional_object_name", - } - }, - "propositions": Set[ # * this is for compensating the informations. should be used as less as possible. - str - ], - }, - # * this is the reason to conduct this action. This should be checked after - # * the action is executed, which is explicitly implemented in the subtree as a condition - # * node in the selector. - "purposes": { - "true": { - "object_name": { # e.g. cabinet - "property": "optional_object_name", - # e.g. open: "", this is a property of the object - # e.g. on: "desk" this is a directedrelation between objects - }, - }, - "false": { - "object_name": { - "property": "optional_object_name", - } - }, - "propositions": Set[ # * this is for compensating the informations. should be used as less as possible. - str - ], - }, -} - - -# Write variables to JSON file -actions = { - "move": move, -} - -# condition examples - -is_in_hand = { - "name": "is_in_hand", - "variables": {"tool": None}, -} - -is_in_tool = { - "name": "is_in_hand", - "variables": {"tool": None, "part": None}, -} - -conditions = { - "is_in_hand": is_in_hand, - "is_in_tool": is_in_tool, -} - -# Define the file path -# Get the current working directory -current_dir = os.getcwd() -folder_path = os.path.join(current_dir, "src/kios/kios_bt_planning/bt_settings") -# Create the folder if it doesn't exist -os.makedirs(folder_path, exist_ok=True) - -actions_file_path = os.path.join(folder_path, "actions.json") - -# Write variables to JSON file -with open(actions_file_path, "w") as json_file: - json.dump(actions, json_file, indent=4) - -conditions_file_path = os.path.join(folder_path, "conditions.json") - -# Write variables to JSON file -with open(conditions_file_path, "w") as json_file: - json.dump(conditions, json_file, indent=4) - -# # Read variables from JSON file -# with open(actions_file_path, "r") as json_file: -# behavior_lists = json.load(json_file) diff --git a/kios_bt_planning/backups/kios_llama/ai_model.py b/kios_bt_planning/backups/kios_llama/ai_model.py deleted file mode 100644 index fa807f4d..00000000 --- a/kios_bt_planning/backups/kios_llama/ai_model.py +++ /dev/null @@ -1,305 +0,0 @@ -from typing import List, Optional - -import fire - -from llama import Llama, Dialog - -import os -import re -import json - -enc = tiktoken.get_encoding("cl100k_base") - -script_dir = os.path.dirname(__file__) -upper_level = os.path.dirname(os.path.dirname(script_dir)) - -secret_path = os.path.join(upper_level, "secrets.json") - -# with open("../../secrets.json") as f: -with open(secret_path) as f: - credentials = json.load(f) - # print(credentials) - - -# dir_system = "./system" -dir_system = os.path.join(script_dir, "system") -# dir_prompt = "./prompt" -dir_prompt = os.path.join(script_dir, "prompt") -# dir_query = "./query" -dir_query = os.path.join(script_dir, "query") - -prompt_load_order = [ - "prompt_role", # your are a good interpreter ... Done - "prompt_domain", # domain knowledge ... Done - "prompt_problem", # how the problem is provided ... Done - "prompt_environment", # how to express the environment ... Done - "prompt_behaviortree", # how to construct the behavior tree ... Done - "prompt_output_format", # the output format ... Done - "prompt_example", # some examples ... Done -] - - -class LlamaAgent: - problem: str = None - """ - the problem description - """ - - query: str = None - """ - the query request: start working ... - """ - instruction: str = None - """ - the member variable to keep the latest instruction - """ - system_message: dict = None - """ - the "system" prompt part - """ - messages: list = None - """ - The list of messages for prompting. - including: the required format (in prompt folder), the last response as assistant prompt - """ - - def __init__(self, prompt_load_order=None) -> None: - self.messages = [] - self.max_token_length = 25000 - self.max_completion_length = 4000 - self.last_response = None - self.query = "" - self.instruction = "" - # load prompt file - # * system - fp_system = os.path.join(dir_system, "system.txt") - with open(fp_system) as f: - data = f.read() - self.system_message = {"role": "system", "content": data} - - # load prompt file - for prompt_name in prompt_load_order: - fp_prompt = os.path.join(dir_prompt, prompt_name + ".txt") - with open(fp_prompt) as f: - data = f.read() - data_spilit = re.split(r"\[user\]\n|\[assistant\]\n", data) - data_spilit = [item for item in data_spilit if len(item) != 0] - # it start with user and ends with system - assert len(data_spilit) % 2 == 0 - for i, item in enumerate(data_spilit): - if i % 2 == 0: - self.messages.append({"sender": "user", "text": item}) - else: - self.messages.append({"sender": "assistant", "text": item}) - fp_query = os.path.join(dir_query, "query.txt") - with open(fp_query) as f: - self.query = f.read() - - def create_prompt(self): - """ - create the prompt for gpt api - """ - prompt = [] # list of prompt entries - # load the system prompt - prompt.append(self.system_message) - # load the knowledge and the last response - for message in self.messages: - prompt.append({"role": message["sender"], "content": message["text"]}) - - # * truncate the prompt if it is too long - prompt_content = "" # this is only for checking the length of the prompt - for message in prompt: - prompt_content += message["content"] - print("prompt length: " + str(len(enc.encode(prompt_content)))) - if ( - len(enc.encode(prompt_content)) - > self.max_token_length - self.max_completion_length - ): - print("prompt too long. truncated.") - # * truncate the prompt by removing the oldest two messages - self.messages = self.messages[2:] - prompt = self.create_prompt() - return prompt - - def extract_json_part(self, text): - """ - extract the markdown code block part from the text - """ - if text.find("```") == -1: - return text - text_json = text[text.find("```") + 3 : text.find("```", text.find("```") + 3)] - return text_json - - def generate_with_problem(self, problem, is_user_feedback=False): - """ - runtime_instruction: the instruction from the problem/user - """ - if is_user_feedback: - pass - else: - # get the query template - text_base = self.query - - # * bb - if text_base.find("[PROBLEM]") != -1: - text_base = text_base.replace("[PROBLEM]", json.dumps(problem)) - - # finally, add the text_base to the user query message - self.messages.append({"sender": "user", "text": text_base}) - - # * request the gpt to response - response = openai.chat.completions.create( - # model="gpt-3.5-turbo-16k-0613", - # model="gpt-4-0613", # not this - model="gpt-4-1106-preview", - # "gpt-4" is available, too. Check the available models in https://platform.openai.com/docs/models/ - messages=self.create_prompt(), - temperature=0.1, - max_tokens=self.max_completion_length, - top_p=0.5, - frequency_penalty=0.0, - presence_penalty=0.0, - ) - - generator = Llama.build( - ckpt_dir=ckpt_dir, - tokenizer_path=tokenizer_path, - max_seq_len=max_seq_len, - max_batch_size=max_batch_size, - ) - - results = generator.chat_completion( - dialogs, # type: ignore - max_gen_len=max_gen_len, - temperature=temperature, - top_p=top_p, - ) - - text = response.choices[0].message.content - - print(text) - self.last_response = text - self.last_response = self.extract_json_part(self.last_response) - self.last_response = self.last_response.replace("'", '"') - # dump to a text file - with open("last_response.txt", "w") as f: - f.write(self.last_response) - try: - self.json_dict = json.loads(self.last_response, strict=False) - # update the environment after the execution of the action - self.environment = self.json_dict["environment_after"] - except json.JSONDecodeError as e: - print("Error decoding JSON:", e) - print("Problematic part:", self.last_response[e.pos - 10 : e.pos + 10]) - - # * use the last response as the assistant prompt for the next round - if len(self.messages) > 0 and self.last_response is not None: - self.messages.append({"sender": "assistant", "text": self.last_response}) - - return self.json_dict - - def dump_json(self, dump_name=None): - """ - dump the json dictionary to a file, used to save into a json file - """ - if dump_name is not None: - # dump the dictionary to json file dump 1, 2, ... - fp = os.path.join(dump_name + ".json") - with open(fp, "w") as f: - json.dump(self.json_dict, f, indent=4) - - -def main( - ckpt_dir: str, - tokenizer_path: str, - temperature: float = 0.6, - top_p: float = 0.9, - max_seq_len: int = 512, - max_batch_size: int = 8, - max_gen_len: Optional[int] = None, -): - """ - Entry point of the program for generating text using a pretrained model. - - Args: - ckpt_dir (str): The directory containing checkpoint files for the pretrained model. - tokenizer_path (str): The path to the tokenizer model used for text encoding/decoding. - temperature (float, optional): The temperature value for controlling randomness in generation. - Defaults to 0.6. - top_p (float, optional): The top-p sampling parameter for controlling diversity in generation. - Defaults to 0.9. - max_seq_len (int, optional): The maximum sequence length for input prompts. Defaults to 512. - max_batch_size (int, optional): The maximum batch size for generating sequences. Defaults to 8. - max_gen_len (int, optional): The maximum length of generated sequences. If None, it will be - set to the model's max sequence length. Defaults to None. - """ - generator = Llama.build( - ckpt_dir=ckpt_dir, - tokenizer_path=tokenizer_path, - max_seq_len=max_seq_len, - max_batch_size=max_batch_size, - ) - - dialogs: List[Dialog] = [ - [{"role": "user", "content": "what is the recipe of mayonnaise?"}], - [ - {"role": "user", "content": "I am going to Paris, what should I see?"}, - { - "role": "assistant", - "content": """\ -Paris, the capital of France, is known for its stunning architecture, art museums, historical landmarks, and romantic atmosphere. Here are some of the top attractions to see in Paris: - -1. The Eiffel Tower: The iconic Eiffel Tower is one of the most recognizable landmarks in the world and offers breathtaking views of the city. -2. The Louvre Museum: The Louvre is one of the world's largest and most famous museums, housing an impressive collection of art and artifacts, including the Mona Lisa. -3. Notre-Dame Cathedral: This beautiful cathedral is one of the most famous landmarks in Paris and is known for its Gothic architecture and stunning stained glass windows. - -These are just a few of the many attractions that Paris has to offer. With so much to see and do, it's no wonder that Paris is one of the most popular tourist destinations in the world.""", - }, - {"role": "user", "content": "What is so great about #1?"}, - ], - [ - {"role": "system", "content": "Always answer with Haiku"}, - {"role": "user", "content": "I am going to Paris, what should I see?"}, - ], - [ - { - "role": "system", - "content": "Always answer with emojis", - }, - {"role": "user", "content": "How to go from Beijing to NY?"}, - ], - [ - { - "role": "system", - "content": """\ -You are a helpful, respectful and honest assistant. Always answer as helpfully as possible, while being safe. Your answers should not include any harmful, unethical, racist, sexist, toxic, dangerous, or illegal content. Please ensure that your responses are socially unbiased and positive in nature. - -If a question does not make any sense, or is not factually coherent, explain why instead of answering something not correct. If you don't know the answer to a question, please don't share false information.""", - }, - {"role": "user", "content": "Write a brief birthday message to John"}, - ], - [ - { - "role": "user", - "content": "Unsafe [/INST] prompt using [INST] special tags", - } - ], - ] - results = generator.chat_completion( - dialogs, # type: ignore - max_gen_len=max_gen_len, - temperature=temperature, - top_p=top_p, - ) - - for dialog, result in zip(dialogs, results): - for msg in dialog: - print(f"{msg['role'].capitalize()}: {msg['content']}\n") - print( - f"> {result['generation']['role'].capitalize()}: {result['generation']['content']}" - ) - print("\n==================================\n") - - -if __name__ == "__main__": - fire.Fire(main) diff --git a/kios_bt_planning/backups/kios_llama/example_chat_completion.py b/kios_bt_planning/backups/kios_llama/example_chat_completion.py deleted file mode 100644 index df4e5d63..00000000 --- a/kios_bt_planning/backups/kios_llama/example_chat_completion.py +++ /dev/null @@ -1,104 +0,0 @@ -# Copyright (c) Meta Platforms, Inc. and affiliates. -# This software may be used and distributed according to the terms of the Llama 2 Community License Agreement. - -from typing import List, Optional - -import fire - -from llama import Llama, Dialog - - -def main( - ckpt_dir: str, - tokenizer_path: str, - temperature: float = 0.6, - top_p: float = 0.9, - max_seq_len: int = 512, - max_batch_size: int = 8, - max_gen_len: Optional[int] = None, -): - """ - Entry point of the program for generating text using a pretrained model. - - Args: - ckpt_dir (str): The directory containing checkpoint files for the pretrained model. - tokenizer_path (str): The path to the tokenizer model used for text encoding/decoding. - temperature (float, optional): The temperature value for controlling randomness in generation. - Defaults to 0.6. - top_p (float, optional): The top-p sampling parameter for controlling diversity in generation. - Defaults to 0.9. - max_seq_len (int, optional): The maximum sequence length for input prompts. Defaults to 512. - max_batch_size (int, optional): The maximum batch size for generating sequences. Defaults to 8. - max_gen_len (int, optional): The maximum length of generated sequences. If None, it will be - set to the model's max sequence length. Defaults to None. - """ - generator = Llama.build( - ckpt_dir=ckpt_dir, - tokenizer_path=tokenizer_path, - max_seq_len=max_seq_len, - max_batch_size=max_batch_size, - ) - - dialogs: List[Dialog] = [ - [{"role": "user", "content": "what is the recipe of mayonnaise?"}], - [ - {"role": "user", "content": "I am going to Paris, what should I see?"}, - { - "role": "assistant", - "content": """\ -Paris, the capital of France, is known for its stunning architecture, art museums, historical landmarks, and romantic atmosphere. Here are some of the top attractions to see in Paris: - -1. The Eiffel Tower: The iconic Eiffel Tower is one of the most recognizable landmarks in the world and offers breathtaking views of the city. -2. The Louvre Museum: The Louvre is one of the world's largest and most famous museums, housing an impressive collection of art and artifacts, including the Mona Lisa. -3. Notre-Dame Cathedral: This beautiful cathedral is one of the most famous landmarks in Paris and is known for its Gothic architecture and stunning stained glass windows. - -These are just a few of the many attractions that Paris has to offer. With so much to see and do, it's no wonder that Paris is one of the most popular tourist destinations in the world.""", - }, - {"role": "user", "content": "What is so great about #1?"}, - ], - [ - {"role": "system", "content": "Always answer with Haiku"}, - {"role": "user", "content": "I am going to Paris, what should I see?"}, - ], - [ - { - "role": "system", - "content": "Always answer with emojis", - }, - {"role": "user", "content": "How to go from Beijing to NY?"}, - ], - [ - { - "role": "system", - "content": """\ -You are a helpful, respectful and honest assistant. Always answer as helpfully as possible, while being safe. Your answers should not include any harmful, unethical, racist, sexist, toxic, dangerous, or illegal content. Please ensure that your responses are socially unbiased and positive in nature. - -If a question does not make any sense, or is not factually coherent, explain why instead of answering something not correct. If you don't know the answer to a question, please don't share false information.""", - }, - {"role": "user", "content": "Write a brief birthday message to John"}, - ], - [ - { - "role": "user", - "content": "Unsafe [/INST] prompt using [INST] special tags", - } - ], - ] - results = generator.chat_completion( - dialogs, # type: ignore - max_gen_len=max_gen_len, - temperature=temperature, - top_p=top_p, - ) - - for dialog, result in zip(dialogs, results): - for msg in dialog: - print(f"{msg['role'].capitalize()}: {msg['content']}\n") - print( - f"> {result['generation']['role'].capitalize()}: {result['generation']['content']}" - ) - print("\n==================================\n") - - -if __name__ == "__main__": - fire.Fire(main) diff --git a/kios_bt_planning/backups/kios_llama/example_text_completion.py b/kios_bt_planning/backups/kios_llama/example_text_completion.py deleted file mode 100755 index 0d60b9c9..00000000 --- a/kios_bt_planning/backups/kios_llama/example_text_completion.py +++ /dev/null @@ -1,69 +0,0 @@ -# Copyright (c) Meta Platforms, Inc. and affiliates. -# This software may be used and distributed according to the terms of the Llama 2 Community License Agreement. - -import fire - -from llama import Llama -from typing import List - -def main( - ckpt_dir: str, - tokenizer_path: str, - temperature: float = 0.6, - top_p: float = 0.9, - max_seq_len: int = 128, - max_gen_len: int = 64, - max_batch_size: int = 4, -): - """ - Entry point of the program for generating text using a pretrained model. - - Args: - ckpt_dir (str): The directory containing checkpoint files for the pretrained model. - tokenizer_path (str): The path to the tokenizer model used for text encoding/decoding. - temperature (float, optional): The temperature value for controlling randomness in generation. - Defaults to 0.6. - top_p (float, optional): The top-p sampling parameter for controlling diversity in generation. - Defaults to 0.9. - max_seq_len (int, optional): The maximum sequence length for input prompts. Defaults to 128. - max_gen_len (int, optional): The maximum length of generated sequences. Defaults to 64. - max_batch_size (int, optional): The maximum batch size for generating sequences. Defaults to 4. - """ - generator = Llama.build( - ckpt_dir=ckpt_dir, - tokenizer_path=tokenizer_path, - max_seq_len=max_seq_len, - max_batch_size=max_batch_size, - ) - - prompts: List[str] = [ - # For these prompts, the expected answer is the natural continuation of the prompt - "I believe the meaning of life is", - "Simply put, the theory of relativity states that ", - """A brief message congratulating the team on the launch: - - Hi everyone, - - I just """, - # Few shot prompt (providing a few examples before asking model to complete more); - """Translate English to French: - - sea otter => loutre de mer - peppermint => menthe poivrée - plush girafe => girafe peluche - cheese =>""", - ] - results = generator.text_completion( - prompts, - max_gen_len=max_gen_len, - temperature=temperature, - top_p=top_p, - ) - for prompt, result in zip(prompts, results): - print(prompt) - print(f"> {result['generation']}") - print("\n==================================\n") - - -if __name__ == "__main__": - fire.Fire(main) diff --git a/kios_bt_planning/backups/kios_llama/prompt/old_prompt_example.txt b/kios_bt_planning/backups/kios_llama/prompt/old_prompt_example.txt deleted file mode 100644 index 8d71b9b6..00000000 --- a/kios_bt_planning/backups/kios_llama/prompt/old_prompt_example.txt +++ /dev/null @@ -1,232 +0,0 @@ -[user] -I will give you some examples of the input and the output you will generate. -Example 1: -""" -- Input: -{"assets":["
", "", "", "", ""], -"asset_states": {"": "on_something(
)", - "": "on_something()"}, -"objects": ["", ""], -"object_states": {"": "on_something(
)", - "": "on_something()"}, -"instruction": "Put the juice on top of the shelf"} -- Output: -``` -{"task_cohesion": { - "task_sequence": [ - "move_hand()", - "grasp_object()", - "detach_from_plane()", - "move_object()", - "move_object()", - "move_object()", - "attach_to_plane()", - "release_object()" - ], - "step_instructions": [ - "move the hand near the juice", - "grasp the juice", - "pick up the juice", - "take the juice from the bottom of the shelf", - "move the juice upward", - "move the juice above the upper shelf", - "place the juice", - "release the juice" - ], - "object_name": ""}, -"environment_before": {"assets":["
", "", "", "", ""], - "asset_states": {"": "on_something(
)", - "": "on_something()"}, - "objects": ["", ""], - "object_states": {"": "on_something(
)", - "": "on_something()"}}, -"environment_after": {"assets":["
", "", "", "", ""], - "asset_states": {"": "on_something(
)", - "": "on_something()"}, - "objects": ["", ""], - "object_states": {"": "on_something(
)", - "": "on_something()"}}, -"instruction_summary": "put the juice on top of the shelf", -"question":""} -``` -""" -Example 2: -""" -- Input: -{"assets":["
", "", "", "", ""], -"asset_states": {"": "on_something(
)", - "": "on_something()"}, -"objects": ["", ""], -"object_states": {"": "on_something(
)", - "": "on_something()"}, -"instruction": "Throw away the spam into the trash bin."} -- Output: -``` -{"task_cohesion": { - "task_sequence": [ - "move_hand()", - "grasp_object()", - "detach_from_plane()", - "move_object()", - "release_object()" - ], - "step_instructions": [ - "move your hand near the spam", - "grasp the spam", - "pick up the spam", - "bring the spam above the trash bin", - "release the spam to drop it in the trash bin" - ], - "object_name": ""}, -"environment_before": {"assets":["
", "", "", "", ""], - "asset_states": {"": "on_something(
)", - "": "on_something()"}, - "objects": ["", ""], - "object_states": {"": "on_something(
)", - "": "on_something()"}}, -"environment_after": {"assets":["
", "", "", "", ""], - "asset_states": {"": "on_something(
)", - "": "on_something()"}, - "objects": ["", ""], - "object_states": {"": "inside_something()", - "": "on_something()"}}, -"instruction_summary": "throw away the spam", -"question":""} -``` -""" -Example 3: -""" --Input: -{"assets":["", ""], -"asset_states": {"": "on_something()"}, -"objects": [""], -"object_states": {"": "closed()"}, -"instruction": "Open the fridge."} -- Output: -``` -{"task_cohesion": { - "task_sequence": [ - "move_hand()", - "grasp_object()", - "open_by_rotate()", - "release_object()" - ], - "step_instructions": [ - "move the hand near the fridge handle", - "grasp the handle", - "open the fridge by pulling the handle", - "release the handle" - ], - "object_name": ""}, -"environment_before": {"assets":["", ""], - "asset_states": {"": "on_something()"}, - "objects": [""], - "object_states": {"": "closed()"}}, -"environment_after": {"assets":["", ""], - "asset_states": {"": "on_something()"}, - "objects": [""], - "object_states": {"": "open()"}}, -"instruction_summary": "open a fridge", -"question":""} -``` -""" -Example 4: -""" --Input: -{"assets":["
", ""], -"asset_states": {"
": "next_to()"}, -"objects": [""], -"object_states": {"": "on_something(
)"}, -"instruction": "Wipe the window with the sponge. Then, put the sponge on the table."} -- Output: -``` -{"task_cohesion": { - "task_sequence": [ - "move_hand()", - "grasp_object()", - "detach_from_plane()", - "move_object()", - "attach_to_plane()", - "wipe_on_plane()", - "detach_from_plane()", - "move_object()", - "attach_to_plane()", - "release_object()" - ], - "step_instructions": [ - "move the hand near the sponge", - "grasp the sponge", - "pick up the sponge", - "bring the sponge above the window", - "attach the sponge to the window", - "wipe the window with the sponge", - "detach the sponge from the window", - "bring the sponge above the table", - "put down the sponge", - "release the sponge" - ], - "object_name": ""}, -"environment_before": {"assets":["
", ""], - "asset_states": {"
": "next_to()"}, - "objects": [""], - "object_states": {"": "on_something(
)"},} -"environment_after": {"assets":["
", ""], - "asset_states": {"
": "next_to()"}, - "objects": [""], - "object_states": {"": "on_something(
)"}}, -"instruction_summary": "wipe the window with the sponge, then put the sponge on the table", -"question":""} -``` -""" -Example 5: -""" --Input: -{"assets":["
"], -"asset_states": {"": "can_manipulate()"}, -"objects": ["", "", ""], -"object_states": {"": "on_something(
)", "": "on_something(
), "": "can_manipulate()"}, -"instruction": "screw the nut into the bolt. Tool should be used and be put back at the end."} -- Output: -``` -{"task_cohesion": { - "task_sequence": [ - "load_tool()", - "approach()", - "grasp_object()", - "pick_up()", - "approach()", - "insert()", - "screw()", - "release_object()", - "unload_tool()" - ], - "step_instructions": [ - "load the parallel_toolbox", - "move the hand close to the nut", - "grasp the nut", - "pick up the nut", - "move the hand close to the bolt", - "insert the nut onto the bolt", - "screw the nut into the bold", - "release the nut", - "unload the parallel_toolbox" - ], - "object_name": ""}, -"environment_before": {"assets":["
"], - "asset_states": {"": "can_manipulate()"}, - "objects": ["", "", ""], - "object_states": {"": "on_something(
)", "": "on_something(
)"},} -"environment_after": {"assets":["
"], - "asset_states": {"": "can_manipulate()"}, - "objects": ["", "", ""], - "object_states": {"": "screwed_into()", "": "on_something(
)"}, -"instruction_summary": "screw the nut into the bolt. Tool should be used and be put back at the end.", -"question":""} -} -``` -""" -From these examples, learn that some robotic actions have dependencies with the actions before and after them. -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/kios_bt_planning/backups/kios_llama/prompt/old_prompt_output_format.txt b/kios_bt_planning/backups/kios_llama/prompt/old_prompt_output_format.txt deleted file mode 100644 index 7be3d38b..00000000 --- a/kios_bt_planning/backups/kios_llama/prompt/old_prompt_output_format.txt +++ /dev/null @@ -1,20 +0,0 @@ -[user] -You divide the actions given in the text into detailed robot actions and put them together as a python dictionary. -The dictionary has five keys. -""" -- dictionary["task_cohesion"]: A dictionary containing information about the robot's actions that have been split up. -- dictionary["environment_before"]: The state of the environment before the manipulation. -- dictionary["environment_after"]: The state of the environment after the manipulation. -- dictionary["instruction_summary"]: contains a brief summary of the given sentence. -- dictionary["question"]: If you cannot understand the given sentence, you can ask the user to rephrase the sentence. Leave this key empty if you can understand the given sentence. -""" -Three keys exist in dictionary["task_cohesion"]. -""" -- dictionary["task_cohesion"]["task_sequence"]: Contains a list of robot actions. Only the behaviors defined in the "ROBOT ACTION LIST" will be used. -- dictionary["task_cohesion"]["step_instructions"]: contains a list of instructions corresponding to dictionary["task_cohesion"]["task_sequence"]. -- dictionary["task_cohesion"]["object_name"]: The name of the manipulated object. Only objects defined in the input dictionary will be used for the object name. -""" -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/kios_bt_planning/backups/kios_llama/prompt/oldprompt_environment.txt b/kios_bt_planning/backups/kios_llama/prompt/oldprompt_environment.txt deleted file mode 100644 index ac8293ab..00000000 --- a/kios_bt_planning/backups/kios_llama/prompt/oldprompt_environment.txt +++ /dev/null @@ -1,28 +0,0 @@ -[user] -Information about world states are given as python dictionary. Example: -""" -{"environment":{"assets":["
", "", "", "", ""], - "asset_states": {"": "on_something(
)", - "": "on_something()"}, - "objects": ["", ""], - "object_states": {"": "on_something(
)", - "": "on_something()"}}} -""" -Asset states and object states are represented using those state sets: -""" -"STATE LIST" -- on_something(): Object is located on -- inside_something(): Object is located inside -- inside_hand(): Object is being grasped by a robot hand -- closed(): Object can be opened -- open(): Object can be closed or kept opened -- can_manipulate(): Object can be used to grasp and manipulate -- equipped(): Object is being equipped by the robot hand. -- screwed_into(): Object is screwed into by robot hand -- inserted_into(): Object is inserted into by robot hand -""" - should be one of the assets or objects in the environment. -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/kios_bt_planning/backups/kios_llama/prompt/prompt_behaviortree.txt b/kios_bt_planning/backups/kios_llama/prompt/prompt_behaviortree.txt deleted file mode 100644 index d3734671..00000000 --- a/kios_bt_planning/backups/kios_llama/prompt/prompt_behaviortree.txt +++ /dev/null @@ -1,117 +0,0 @@ -[user] -The behavior tree should be in python dictionary and should have the following structure: -""" -{ - "name": "Pick Up Apple", - "identifier": 0, - "type_name": "selector", - "children": [ - { - "summary": "check if the apple is in hand", - "name": "check apple in hand", - "identifier": 1, - "type_name": "condition", - "conditions": [ - { - "object_name": "apple", - "property_name": "in", - "property_value": "hand", - "status": True, - } - ], - }, - { - "name": "Pick Up Sequence", - "identifier": 2, - "type_name": "sequence", - "children": [ - { - "summary": "check if the apple is on the ground", - "name": "check apple on the ground", - "identifier": 3, - "type_name": "condition", - "conditions": [ - { - "object_name": "apple", - "property_name": "on_the_ground", - "property_value": None, - "status": True, - } - ], - }, - { - "summary": "check if the hand is free", - "name": "check hand free", - "identifier": 4, - "type_name": "condition", - "conditions": [ - { - "object_name": "hand", - "property_name": "free", - "property_value": None, - "status": True, - } - ], - }, - { - "summary": "pick up the apple", - "name": "pick_up", - "identifier": 5, - "type_name": "action", - "effects": [ - { - "object_name": "apple", - "property_name": "on_the_ground", - "property_value": None, - "status": False, - }, - { - "object_name": "apple", - "property_name": "in", - "property_value": "hand", - "status": True, - }, - { - "object_name": "hand", - "property_name": "free", - "property_value": None, - "status": False, - }, - ], - }, - ], - }, - ], -} -""" -The dictionary that you return should be formatted as python dictionary. Follow these rules: -1. Every node should have three basic keys: - - "name": the name of the node - - "identifier": the identifier of the node, which should be unique - - "type_name": the type of the node, its value can be: "selector", "sequence", "condition", "action" -2. The "selector" and "sequence" nodes can (and should) have a key "children" with a list of children nodes. -3. The "action" and "condition" nodes should have a key "summary" with a short description of the action/condition. -4. The "condition" nodes should have a key "conditions" with a list of conditions to be checked. Each condition should have the following keys: - - "object_name": the object to check - - "property_name": the property/relation of the object to be checked - - "property_value": this should be the other object if this is a relation. For property it should be None - - "status": the boolean status of the property/relation the condition expects, True if the property/relation should be True, False if the property/relation should be False -5. The "action" nodes should have a key "effects" with a list of effects to be exerted after the execution of the action. Each effect should have the following keys: - - "object_name": the object to be affected - - "property_name": the property/relation of the object to be affected - - "property_value": this should be the other object if this is a relation. For property it should be None - - "status": True if the effect is to add this property/relation. False if is to remove. -6. The behavior tree is used to represent an action in the action sequence. Its basic structure is: - - A selector as the root node. - - (Usually one) (Goal-)Condition node as the first child of the selector. It should check the goal conditions the action intends to achieve. So if the goal condition is met, the selector will directly return success. - - A sequence node as the last child of the selector node. The aim of the sequence is to achieve the goal conditions. - - (Pre-)Condition nodes as the front children of the sequence node. They should check the preconditions of the action, which is the last child node of the sequence node. So if any of the preconditions is not met, the sequence node will return failure. - - Only one action node as the last child of the sequence node. Its preconditions are checked by the condition nodes in the front of it. Its effects are exerted after the execution of the action, and should be able to satisfy the goal condition(s), which is the first child of the selector node. -7. When there is any (Pre-)condition node that is not satisfied, a behavior tree can be constructed according to the rules above, which replaces the unsatisfied (Pre-)condition, takes that (Pre-)condition as the (Goal-)condition to achieve and includes the action to achieve it. -8. The selector can have multiple (goal-)condition nodes as children, but must have only and no more than one sequence node as the last child. -9. The sequence node can have multiple (pre-)condition nodes (or selectors) as children, but must have only and no more than one action node as the last child. -10. The behavior tree should be constructed based on the action sequence you planned and the action definitions from the domain knowledge. -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/kios_bt_planning/backups/kios_llama/prompt/prompt_domain.txt b/kios_bt_planning/backups/kios_llama/prompt/prompt_domain.txt deleted file mode 100644 index 66ea6005..00000000 --- a/kios_bt_planning/backups/kios_llama/prompt/prompt_domain.txt +++ /dev/null @@ -1,89 +0,0 @@ -[user] -Following is the domain knowledge you need, given as a pddl domain file, which includes: -- the types of objects in the world, -- the predicates that can be used to describe the states of the world, -- the actions that you can apply, including their precondtions and effects. -""" -(define (domain robot_assembly_problem-domain) - (:requirements :strips :typing) - (:types - thing - object - tool part hand - thing - ) - (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part)) - (:action pickup - :parameters ( ?hand - hand ?part - part ?tool - tool) - :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part)) - :effect (and (hold ?tool ?part) (not (is_free ?tool)))) - (:action putdown - :parameters ( ?hand - hand ?tool - tool ?part - part) - :precondition (and (hold ?tool ?part) (hold ?hand ?tool)) - :effect (and (not (hold ?hand ?part)) (is_free ?tool))) - (:action place - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2))) - (:action detach - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2)))) - (:action insert - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2))) - (:action pull - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2)))) - (:action screw - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2))) - (:action unscrew - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2)))) - (:action load_tool - :parameters ( ?hand - hand ?tool - tool) - :precondition (and (is_equippable ?tool) (is_free ?hand)) - :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool))) - (:action unload_tool - :parameters ( ?hand - hand ?tool - tool) - :precondition (and (hold ?hand ?tool) (is_free ?tool)) - :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool)))) -) -""" -The precidates are explained as follows: -""" -"PREDICATE_LIST" -- is_free(): is free and can hold something. (Initially defaulted to be true) -- is_equippable(): is ready to be eqquipped by a robot hand. (Initially defaulted to be true) -- can_manipulate(, ): can be used to manipulate (manipulation constraint) (Initially defaulted to be false) -- can_insert_to(, ): can be inserted into (insertion assembly constraint) (Initially defaulted to be false) -- can_screw_to(, ): can be screwed into (screw assembly constraint) (Initially defaulted to be false) -- can_place_to(, ): can be placed into (placement assembly constraint) (Initially defaulted to be false) -- hold(, ): is being held by (Initially defaulted to be false) -- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (Initially defaulted to be false) -- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (Initially defaulted to be false) -- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (Initially defaulted to be false) -""" - -The actions are explained as follows: -""" -"ROBOT_ACTION_LIST" -- pickup(, , ): use to pickup -- putdown(, , ): use to putdown -- place(, , , ): use to place to (to satisfy the placement assembly constraint) -- detach(, , , ): use to detach from (to lift the existing placement assembly constraint) -- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint) -- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint) -- screw(, , , ): use to screw into (to satisfy the screw assembly constraint) -- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint) -- load_tool(, ): equips to manipulate other objects -- unload_tool(, ): unloads to manipulate other objects -""" -the items in <> indicate the type of the instances needed for the predicates and actions. -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. diff --git a/kios_bt_planning/backups/kios_llama/prompt/prompt_environment.txt b/kios_bt_planning/backups/kios_llama/prompt/prompt_environment.txt deleted file mode 100644 index 1d371a46..00000000 --- a/kios_bt_planning/backups/kios_llama/prompt/prompt_environment.txt +++ /dev/null @@ -1,31 +0,0 @@ -[user] -Information about world states are given as python dictionary. Example: -""" -{ - "objects": [ - {"name": "leg1", "properties": []}, - {"name": "seat", "properties": []}, - {"name": "cabinet", "properties": ["open"]}, - {"name": "parallel_box1", "properties": []}, - {"name": "table", "properties": []}, - ], - "constraints": [ - {"source": "leg1", "name": "can_screw_to", "target": "seat"}, - {"source": "parallel_box1", "name": "can_manipulate", "target": "leg1"}, - {"source": "cabinet", "name": "on_something", "target": "table"}, - ], - "relations": [ - {"source": "leg1", "name": "is_screwed_to", "target": "seat"}, - ], -} -""" -It contains three lists: - - objects are the objects and their properties (as lists of strings) in the environment. The properties can be changed by robot actions. - - constraints are the relations that are enforced (which are truth knowledge or defined by the user in the current contexts). They are given as a part of the problem and should not be changed. - - relations are the relations that are observed in the environment. They can be changed by robot actions. -The available properties, constraints and relations can be found in the [PREDICATE_LIST]. -You should only concentrate on those objects, constraints and relations that are relevant for the current task. -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/kios_bt_planning/backups/kios_llama/prompt/prompt_example.txt b/kios_bt_planning/backups/kios_llama/prompt/prompt_example.txt deleted file mode 100644 index 53b7752c..00000000 --- a/kios_bt_planning/backups/kios_llama/prompt/prompt_example.txt +++ /dev/null @@ -1,294 +0,0 @@ -[user] -I will give you some examples of the input and the output you will generate. -Example 1: -""" -- Input: -(define (problem robot_assembly_problem-problem) - (:domain robot_assembly_problem-domain) - (:objects - parallel_box1 parallel_box2 inward_claw outward_claw no_tool - tool - gear1 gear2 gear3 shaft1 shaft2 base - part - left_hand - hand - ) - (:init (can_manipulate parallel_box1 gear1) (can_manipulate outward_claw gear2) (can_manipulate inward_claw gear3) (can_manipulate parallel_box2 shaft1) (can_manipulate no_tool shaft2) (can_screw_to leg1 seat) (can_screw_to leg2 seat) (can_insert_to back seat) (can_screw_to nut1 seat) (can_screw_to nut2 seat) (can_screw_to blub base) (can_place_to lamp blub) (can_insert_to shaft1 base) (can_insert_to shaft2 base) (can_insert_to gear3 shaft2) (can_insert_to gear2 base) (can_insert_to gear1 shaft1) (is_inserted_to shaft1 base) (is_free left_hand) (is_free parallel_box1) (is_free parallel_box2) (is_free inward_claw) (is_free outward_claw) (is_free no_tool) (is_free gear1) (is_free gear2) (is_free gear3) (is_free shaft1) (is_free shaft2) (is_free base) (is_equippable parallel_box1) (is_equippable parallel_box2) (is_equippable inward_claw) (is_equippable outward_claw) (is_equippable no_tool)) - (:goal (and (is_inserted_to gear1 shaft1))) -) -- Output: -``` -{"task_cohesion": { - "action_sequence": [ - "load_tool(left_hand, parallel_box1)", - "pickup(left_hand, gear1, parallel_box1)", - "insert(left_hand, parallel_box1, gear1, shaft1)" - ], - "step_instructions": [ - "equip the parallel_box1 on the left hand", - "pick up the gear1 with the left hand", - "insert the gear1 into the shaft1 using the left hand with the parallel_box1" - ], - "object_name": ["left_hand", "gear1", "parallel_box1", "shaft1"], - "behavior_tree": { - "name": "insert selector", - "identifier": 13, - "type_name": "selector", - "children": [ - { - "summary": "check if gear1 is inserted to shaft1", - "name": "check gear1 is_inserted_to shaft1", - "identifier": 14, - "type_name": "condition", - "conditions": [ - { - "object_name": "gear1", - "property_name": "is_inserted_to", - "property_value": "shaft1", - "status": True, - } - ], - }, - { - "name": "insert sequence", - "identifier": 15, - "type_name": "sequence", - "children": [ - { - "name": "load_tool selector", - "identifier": 0, - "type_name": "selector", - "children": [ - { - "summary": "check if parallel_box1 is hold", - "name": "check left_hand hold parallel_box1", - "identifier": 1, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "parallel_box1", - "status": True, - } - ], - }, - { - "name": "load_tool Sequence", - "identifier": 2, - "type_name": "sequence", - "children": [ - { - "summary": "check if parallel_box1 is equippable", - "name": "check parallel_box1 is_equippable", - "identifier": 3, - "type_name": "condition", - "conditions": [ - { - "object_name": "parallel_box1", - "property_name": "is_equippable", - "property_value": None, - "status": True, - } - ], - }, - { - "summary": "check if left hand is free", - "name": "check left_hand is_free", - "identifier": 4, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "is_free", - "property_value": None, - "status": True, - } - ], - }, - { - "summary": "equip parallel_box1 to left hand", - "name": "load_tool(left_hand, parallel_box1)", - "identifier": 5, - "type_name": "action", - "effects": [ - { - "object_name": "hand", - "property_name": "is_free", - "property_value": None, - "status": False, - }, - { - "object_name": "parallel_box1", - "property_name": "is_equippable", - "property_value": None, - "status": False, - }, - { - "object_name": "hand", - "property_name": "hold", - "property_value": "parallel_box1", - "status": True, - }, - ], - }, - ], - }, - ], - }, - { - "name": "pick_up selector", - "identifier": 6, - "type_name": "selector", - "children": [ - { - "summary": "check if parallel_box1 holds gear1", - "name": "check parallel_box1 hold gear1", - "identifier": 7, - "type_name": "condition", - "conditions": [ - { - "object_name": "parallel_box1", - "property_name": "hold", - "property_value": "gear1", - "status": True, - } - ], - }, - { - "name": "pick_up Sequence", - "identifier": 8, - "type_name": "sequence", - "children": [ - { - "summary": "check if parallel_box1 is free", - "name": "check parallel_box1 is_free", - "identifier": 9, - "type_name": "condition", - "conditions": [ - { - "object_name": "parallel_box1", - "property_name": "is_free", - "property_value": None, - "status": True, - } - ], - }, - { - "summary": "check if left hand holds parallel_box1", - "name": "check left_hand hold parallel_box1", - "identifier": 16, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "parallel_box1", - "status": True, - } - ], - }, - { - "summary": "pick up gear1 using parallel_box1", - "name": "pick_up(left_hand, parallel_box1, gear1)", - "identifier": 12, - "type_name": "action", - "effects": [ - { - "object_name": "parallel_box1", - "property_name": "hold", - "property_value": "gear1", - "status": True, - }, - { - "object_name": "parallel_box1", - "property_name": "is_free", - "property_value": None, - "status": False, - }, - ], - }, - ], - }, - ], - }, - { - "summary": "check if gear1 can be inserted to shaft1", - "name": "check gear1 can_insert_to shaft1", - "identifier": 18, - "type_name": "condition", - "conditions": [ - { - "object_name": "gear1", - "property_name": "can_insert_to", - "property_value": "shaft1", - "status": True, - } - ], - }, - { - "summary": "insert gear1 to shaft1", - "name": "insert(left_hand, parallel_box1, gear1, shaft1)", - "identifier": 19, - "type_name": "action", - "effects": [ - { - "object_name": "parallel_box1", - "property_name": "hold", - "property_value": "gear1", - "status": False, - }, - { - "object_name": "parallel_box1", - "property_name": "is_free", - "property_value": None, - "status": True, - }, - { - "object_name": "gear1", - "property_name": "is_inserted_to", - "property_value": "shaft1", - "status": True, - }, - ], - }, - ], - }, - ], -} -}, -"environment_before": {"objects": [ - {"name": "gear1", "properties": []}, - {"name": "shaft1", "properties": []}, - {"name": "parallel_box1", "properties": ["is_equippable", "is_free"]}, - {"name": "left_hand", "properties": ["is_free"]}, - ], - "constraints": [ - {"source": "gear1", "name": "can_insert_to", "target": "shaft1"}, - {"source": "parallel_box1", "name": "can_manipulate", "target": "gear1"}, - ], - "relations": [ - - ], -}, -"environment_after": {"objects": [ - {"name": "gear1", "properties": []}, - {"name": "shaft1", "properties": []}, - {"name": "parallel_box1", "properties": ["is_equippable", "is_free"]}, - {"name": "left_hand", "properties": []}, - ], - "constraints": [ - {"source": "gear1", "name": "can_insert_to", "target": "shaft1"}, - {"source": "parallel_box1", "name": "can_manipulate", "target": "gear1"}, - ], - "relations": [ - {"source": "gear1", "name": "is_inserted_to", "target": "shaft1"}, - {"source": "left_hand", "name": "hold", "target": "parallel_box1"}, - ], -}, -"instruction_summary": "insert gear1 into shaft1", -"question":""} -``` -""" -From these examples, learn that some robotic actions have dependencies with the actions before and after them. -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/kios_bt_planning/backups/kios_llama/prompt/prompt_output_format.txt b/kios_bt_planning/backups/kios_llama/prompt/prompt_output_format.txt deleted file mode 100644 index 0558fb35..00000000 --- a/kios_bt_planning/backups/kios_llama/prompt/prompt_output_format.txt +++ /dev/null @@ -1,21 +0,0 @@ -[user] -You divide the actions given in the text into detailed robot actions and contruct a behavior tree based on it in python dictionary format. -The dictionary has five keys. -""" -- dictionary["task_cohesion"]: A dictionary containing information about the robot's actions that have been split up. -- dictionary["environment_before"]: The environment state before the manipulation. -- dictionary["environment_after"]: The environment state after the manipulation. -- dictionary["instruction_summary"]: contains a brief summary of the given sentence. -- dictionary["question"]: If you cannot understand the given sentence, you can ask the user to rephrase the sentence. Leave this key empty if you can understand the given sentence. -""" -Three keys exist in dictionary["task_cohesion"]. -""" -- dictionary["task_cohesion"]["action_sequence"]: Contains a list of robot actions (with the parameters as defined in pddl domain knowledge). Only the actions defined in the "ROBOT_ACTION LIST" will be used. -- dictionary["task_cohesion"]["action_instructions"]: contains a list of instructions corresponding to dictionary["task_cohesion"]["action_sequence"] to explain the action. -- dictionary["task_cohesion"]["manipulated_objects"]: The list of the names of the manipulated object. Only objects defined in the input dictionary will be used for the object name. -- dictionary["task_cohesion"]["behavior_tree"]: contains a dictionary of the behavior tree constructed according to the dictionary ["task_cohesion"]["action_sequence"] -""" -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/kios_bt_planning/backups/kios_llama/prompt/prompt_problem.txt b/kios_bt_planning/backups/kios_llama/prompt/prompt_problem.txt deleted file mode 100644 index c22df152..00000000 --- a/kios_bt_planning/backups/kios_llama/prompt/prompt_problem.txt +++ /dev/null @@ -1,26 +0,0 @@ -[user] -Information about the problem is given as pddl problem file that corresponds to the domain file. -It includes the following information: -- Objects in the world -- Initial states you start with -- Goal states you need to achieve -Example: -""" -(define (problem robot_assembly_problem-problem) - (:domain robot_assembly_problem-domain) - (:objects - parallel_box1 parallel_box2 inward_claw outward_claw no_tool - tool - gear1 gear2 gear3 shaft1 shaft2 base - part - left_hand - hand - ) - (:init (can_manipulate parallel_box1 gear1) (can_manipulate outward_claw gear2) (can_manipulate inward_claw gear3) (can_manipulate parallel_box2 shaft1) (can_manipulate no_tool shaft2) (can_screw_to leg1 seat) (can_screw_to leg2 seat) (can_insert_to back seat) (can_screw_to nut1 seat) (can_screw_to nut2 seat) (can_screw_to blub base) (can_place_to lamp blub) (can_insert_to shaft1 base) (can_insert_to shaft2 base) (can_insert_to gear3 shaft2) (can_insert_to gear2 base) (can_insert_to gear1 shaft1) (is_free left_hand) (is_free parallel_box1) (is_free parallel_box2) (is_free inward_claw) (is_free outward_claw) (is_free no_tool) (is_free gear1) (is_free gear2) (is_free gear3) (is_free shaft1) (is_free shaft2) (is_free base) (is_equippable parallel_box1) (is_equippable parallel_box2) (is_equippable inward_claw) (is_equippable outward_claw) (is_equippable no_tool)) - (:goal (and (is_inserted_to shaft1 base))) -) -""" -Objects in the world are all instances of the types defined in the domain file. For example, the object "parallel_box1" is an instance of the type "tool". The object "gear1" is an instance of the type "part". -Initial states are represented using the [PREDICATE_LIST] from the domain file. For example, the initial state "can_manipulate parallel_box1 gear1" means that the object "parallel_box1" can be used to manipulate the object "gear1". -Goal states are represented using the [PREDICATE_LIST] from the domain file. For example, the goal state "is_inserted_to shaft1 base" means that the object "shaft1" is inserted to the object "base". -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/kios_bt_planning/backups/kios_llama/prompt/prompt_role.txt b/kios_bt_planning/backups/kios_llama/prompt/prompt_role.txt deleted file mode 100644 index 90b48291..00000000 --- a/kios_bt_planning/backups/kios_llama/prompt/prompt_role.txt +++ /dev/null @@ -1,5 +0,0 @@ -[user] -You are an excellent interpreter of human instructions for robotic assembly tasks. Given the information about the world states and some instructions, you break it down and construct robot task plans represented by behaiovr trees based on the action sequences. -Please do not begin working until I say "Start working." Instead, simply output the message "Waiting for next input." Understood? -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/kios_bt_planning/backups/kios_llama/query/old_query.txt b/kios_bt_planning/backups/kios_llama/query/old_query.txt deleted file mode 100644 index f652fe76..00000000 --- a/kios_bt_planning/backups/kios_llama/query/old_query.txt +++ /dev/null @@ -1,23 +0,0 @@ -Start working. Resume from the environment below. -""" -{"environment":[ENVIRONMENT]} -""" -The problem is as follows: -""" -{"problem": [PROBLEM]} -""" -The instruction is as follows: -""" -{"instruction": [INSTRUCTION]} -""" -The dictionary that you return should be formatted as python dictionary. Follow these rules: -1. Always equip the proper tools before performing specific object manipulaion. The robot should not hold any tool or object at the end. -2. Make sure that each element of the ["action_instructions"] explains corresponding element of the ["action_sequence"]. Refer to the "ROBOT ACTION LIST" to understand the elements of ["action_sequence"]. -3, The length of the ["action_instructions"] list must be the same as the length of the ["action_sequence"] list. -4. Never left ',' at the end of the list. -5. Keep track of all items listed in the "objects" section of the "environment_before" field. Please ensure that you fill out both the "objects" and "object_states" sections for all listed items. -6. Use the "PREDICATE_LIST" for the "object states" field. -7. All keys of the dictionary should be double-quoted. -8. Insert ``` at the beginning and the end of the dictionary to separate it from the rest of your response. -9. Make sure that you output a consistent manipultation as a single arm robot. For example, grasping an object should not occur in successive steps. -Adhere to the output format I defined above. Follow the nine rules. Think step by step. \ No newline at end of file diff --git a/kios_bt_planning/backups/kios_llama/query/query.txt b/kios_bt_planning/backups/kios_llama/query/query.txt deleted file mode 100644 index 530173f3..00000000 --- a/kios_bt_planning/backups/kios_llama/query/query.txt +++ /dev/null @@ -1,16 +0,0 @@ -Start working. Resume from problem below. -""" -{"problem": [PROBLEM]} -""" -The dictionary that you return should be formatted as python dictionary. Follow these rules: -1. Always equip the proper tools before performing specific object manipulaion. -2. Make sure that each element of the ["action_instructions"] explains corresponding element of the ["action_sequence"]. Refer to the "ROBOT ACTION LIST" to understand the elements of ["action_sequence"]. -3, The length of the ["action_instructions"] list must be the same as the length of the ["action_sequence"] list. -4. Never left ',' at the end of the list. -5. Keep track of all items listed in the "objects" section of the "environment_before" field. Please make sure you fill out the "objects" section for all listed items. -6. Use the "PREDICATE_LIST" for the "object states" field. -7. All keys of the dictionary should be double-quoted. -8. Insert ``` at the beginning and the end of the dictionary to separate it from the rest of your response. -9. Make sure that you output a consistent manipultation as a single arm robot. For example, grasping an object should not occur in successive steps. -10. For boolean values don't use capital letters. Use "true" or "false" instead of "True" or "False". -Adhere to the output format I defined above. Follow the rules above. Think step by step. \ No newline at end of file diff --git a/kios_bt_planning/backups/kios_llama/system/system.txt b/kios_bt_planning/backups/kios_llama/system/system.txt deleted file mode 100644 index eab24f94..00000000 --- a/kios_bt_planning/backups/kios_llama/system/system.txt +++ /dev/null @@ -1 +0,0 @@ -You are an excellent interpreter of instructions for robotic assembly tasks. Given the information about the world states and some instructions, you break it down and construct robot task plans represented by behaiovr trees based on the action sequences. \ No newline at end of file diff --git a/kios_bt_planning/backups/kios_llm/aimodel.py b/kios_bt_planning/backups/kios_llm/aimodel.py deleted file mode 100644 index cee696a3..00000000 --- a/kios_bt_planning/backups/kios_llm/aimodel.py +++ /dev/null @@ -1,399 +0,0 @@ -import openai -import tiktoken -import json -import os -import re -import argparse -import sys - - -enc = tiktoken.get_encoding("cl100k_base") - -script_dir = os.path.dirname(__file__) -secret_path = os.path.join(script_dir, "secrets.json") - -# with open("../../secrets.json") as f: -with open(secret_path) as f: - credentials = json.load(f) - print(credentials) - -dir_system = "./system" -dir_prompt = "./prompt" -dir_query = "./query" -prompt_load_order = [ - "prompt_role", - "prompt_function", - "prompt_environment", - "prompt_output_format", - "prompt_example", -] - - -# azure openai api has changed from '2023-05-15' to '2023-05-15' -# if you are using a 0301 version, use '2022-12-01' -# Otherwise, use '2023-05-15' - - -class ChatGPT: - VALID_API_VERSIONS = ["2022-12-01", "2023-05-15"] - - def __init__( - self, credentials, prompt_load_order, use_azure=False, api_version="2023-05-15" - ): - self.use_azure = use_azure - if self.use_azure: - openai.api_key = credentials["azureopenai"]["AZURE_OPENAI_KEY"] - openai.api_base = credentials["azureopenai"]["AZURE_OPENAI_ENDPOINT"] - openai.api_type = "azure" - if api_version not in self.VALID_API_VERSIONS: - raise ValueError( - f"api_version must be one of {self.VALID_API_VERSIONS}" - ) - openai.api_version = api_version - else: - openai.organization = credentials["openai"]["YOUR_ORG_ID"] - openai.api_key = credentials["openai"]["OPENAI_API_KEY"] - - self.credentials = credentials - self.messages = [] - self.max_token_length = 8000 - self.max_completion_length = 1000 - self.last_response = None - self.query = "" - self.instruction = "" - # load prompt file - # * system - fp_system = os.path.join(dir_system, "system.txt") - with open(fp_system) as f: - data = f.read() - self.system_message = {"role": "system", "content": data} - - # load prompt file - for prompt_name in prompt_load_order: - fp_prompt = os.path.join(dir_prompt, prompt_name + ".txt") - with open(fp_prompt) as f: - data = f.read() - data_spilit = re.split(r"\[user\]\n|\[assistant\]\n", data) - data_spilit = [item for item in data_spilit if len(item) != 0] - # it start with user and ends with system - assert len(data_spilit) % 2 == 0 - for i, item in enumerate(data_spilit): - if i % 2 == 0: - self.messages.append({"sender": "user", "text": item}) - else: - self.messages.append({"sender": "assistant", "text": item}) - fp_query = os.path.join(dir_query, "query.txt") - with open(fp_query) as f: - self.query = f.read() - - # See - # https://learn.microsoft.com/en-us/azure/cognitive-services/openai/how-to/chatgpt#chatml - def create_prompt(self): - prompt = "" - if self.use_azure and openai.api_version == "2022-12-01": - prompt = "<|im_start|>system\n" - prompt += self.system_message["content"] - prompt += "\n<|im_end|>\n" - for message in self.messages: - prompt += ( - f"\n<|im_start|>{message['sender']}\n{message['text']}\n<|im_end|>" - ) - prompt += "\n<|im_start|>assistant\n" - print("prompt length: " + str(len(enc.encode(prompt)))) - if ( - len(enc.encode(prompt)) - > self.max_token_length - self.max_completion_length - ): - print("prompt too long. truncated.") - # truncate the prompt by removing the oldest two messages - self.messages = self.messages[2:] - prompt = self.create_prompt() - else: - prompt = [] - prompt.append(self.system_message) - for message in self.messages: - prompt.append({"role": message["sender"], "content": message["text"]}) - prompt_content = "" - for message in prompt: - prompt_content += message["content"] - print("prompt length: " + str(len(enc.encode(prompt_content)))) - if ( - len(enc.encode(prompt_content)) - > self.max_token_length - self.max_completion_length - ): - print("prompt too long. truncated.") - # truncate the prompt by removing the oldest two messages - self.messages = self.messages[2:] - prompt = self.create_prompt() - return prompt - - def extract_json_part(self, text): - # because the json part is in the middle of the text, we need to extract it. - # json part is between ``` and ```. - # skip if there is no json part - if text.find("```") == -1: - return text - text_json = text[text.find("```") + 3 : text.find("```", text.find("```") + 3)] - return text_json - - def generate(self, message, environment, is_user_feedback=False): - if is_user_feedback: - self.messages.append( - {"sender": "user", "text": message + "\n" + self.instruction} - ) - else: - text_base = self.query - if text_base.find("[ENVIRONMENT]") != -1: - text_base = text_base.replace("[ENVIRONMENT]", json.dumps(environment)) - if text_base.find("[INSTRUCTION]") != -1: - text_base = text_base.replace("[INSTRUCTION]", message) - self.instruction = text_base - self.messages.append({"sender": "user", "text": text_base}) - - if self.use_azure and openai.api_version == "2022-12-01": - # Remove unsafe user inputs. May need further refinement in the - # future. - if message.find("<|im_start|>") != -1: - message = message.replace("<|im_start|>", "") - if message.find("<|im_end|>") != -1: - message = message.replace("<|im_end|>", "") - deployment_name = self.credentials["azureopenai"][ - "AZURE_OPENAI_DEPLOYMENT_NAME_CHATGPT" - ] - response = openai.Completion.create( - engine=deployment_name, - prompt=self.create_prompt(), - temperature=0.1, - max_tokens=self.max_completion_length, - top_p=0.5, - frequency_penalty=0.0, - presence_penalty=0.0, - stop=["<|im_end|>"], - ) - text = response["choices"][0]["text"] - elif self.use_azure and openai.api_version == "2023-05-15": - deployment_name = self.credentials["azureopenai"][ - "AZURE_OPENAI_DEPLOYMENT_NAME_CHATGPT" - ] - response = openai.ChatCompletion.create( - engine=deployment_name, - messages=self.create_prompt(), - temperature=0.1, - max_tokens=self.max_completion_length, - top_p=0.5, - frequency_penalty=0.0, - presence_penalty=0.0, - ) - text = response["choices"][0]["message"]["content"] - else: - response = openai.ChatCompletion.create( - model="gpt-3.5-turbo-16k", - # "gpt-4" is available, too. Check the available models in https://platform.openai.com/docs/models/ - messages=self.create_prompt(), - temperature=0.1, - max_tokens=self.max_completion_length, - top_p=0.5, - frequency_penalty=0.0, - presence_penalty=0.0, - ) - text = response["choices"][0].message.content - print(text) - self.last_response = text - self.last_response = self.extract_json_part(self.last_response) - self.last_response = self.last_response.replace("'", '"') - # dump to a text file - with open("last_response.txt", "w") as f: - f.write(self.last_response) - try: - self.json_dict = json.loads(self.last_response, strict=False) - self.environment = self.json_dict["environment_after"] - except BaseException: - self.json_dict = None - import pdb - - pdb.set_trace() - - if len(self.messages) > 0 and self.last_response is not None: - self.messages.append({"sender": "assistant", "text": self.last_response}) - - return self.json_dict - - def dump_json(self, dump_name=None): - """ - dump the json dictionary to a file - """ - if dump_name is not None: - # dump the dictionary to json file dump 1, 2, ... - fp = os.path.join(dump_name + ".json") - with open(fp, "w") as f: - json.dump(self.json_dict, f, indent=4) - - -if __name__ == "__main__": - parser = argparse.ArgumentParser() - parser.add_argument( - "--scenario", - type=str, - required=True, - help="scenario name (see the code for details)", - ) - args = parser.parse_args() - scenario_name = args.scenario - # 1. example of moving objects on the table and the shelf - if scenario_name == "shelf": - environment = { - "assets": [ - "
", - "", - "", - "", - "", - ], - "asset_states": { - "": "on_something(
)", - "": "on_something()", - }, - "objects": ["", ""], - "object_states": { - "": "on_something(
)", - "": "on_something()", - }, - } - instructions = [ - "Put the juice on top of the shelf", - "Throw away the spam into the trash bin", - "Move the juice on top of the table", - "Throw away the juice", - ] - # 2. example of opening and closing the fridge, and putting the juice on - # the floor - elif scenario_name == "fridge": - environment = { - "assets": ["", ""], - "asset_states": {"": "on_something()"}, - "objects": ["", ""], - "object_states": { - "": "closed()", - "": "inside_something()", - }, - } - instructions = [ - "Open the fridge half way", - "Open the fridge wider", - "Take the juice in the fridge and put it on the floor", - "Close the fridge", - ] - # 3. example of opening and closing the drawer - elif scenario_name == "drawer": - environment = { - "assets": ["", ""], - "asset_states": {"": "on_something()"}, - "objects": [""], - "object_states": {"": "closed()"}, - } - instructions = [ - "Open the drawer widely", - "Close the drawer half way", - "Close the drawer fully", - ] - # 4. example of wiping the table - elif scenario_name == "table": - environment = { - "assets": ["", "", "", ""], - "asset_states": { - "": "next_to()", - "": "on_something()", - }, - "objects": [""], - "object_states": {"": "on_something()"}, - } - instructions = [ - "Put the sponge on the table2", - "Wipe the table2 with the sponge", - ] - # 5. example of wiping the window - elif scenario_name == "window": - environment = { - "assets": ["
", "", "", ""], - "asset_states": { - "
": "next_to()", - "": "on_something()", - }, - "objects": [""], - "object_states": {"": "on_something(
)"}, - } - instructions = [ - "Get the sponge from the table and wipe the window with it. After that, put the sponge back on the table", - "Throw away the sponge on the table", - ] - elif scenario_name == "chair_assembly": - environment = { - "assets": [ - "
", - ], - "asset_states": { - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "can_manipulate(, )", - "": "can_manipulate(, )", - }, - "objects": [ - "", - "", - "", - "", - "", - "", - "", - "", - "", - ], - "object_states": { - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - }, - } - instructions = [ - "Screw the leg1 to the seat", - "Screw the leg2 to the seat", - "Insert the seat to the back", - "Screw the nut1 to the seat", - "Screw the nut2 to the seat", - ] - - else: - parser.error("Invalid scenario name:" + scenario_name) - - aimodel = ChatGPT( - credentials, - prompt_load_order=prompt_load_order, - use_azure=False, - api_version="2022-12-01", - ) - - if not os.path.exists("./out/" + scenario_name): - os.makedirs("./out/" + scenario_name) - for i, instruction in enumerate(instructions): - print(json.dumps(environment)) - text = aimodel.generate(instruction, environment, is_user_feedback=False) - while True: - user_feedback = input("user feedback (return empty if satisfied): ") - if user_feedback == "q": - exit() - if user_feedback != "": - text = aimodel.generate( - user_feedback, environment, is_user_feedback=True - ) - else: - # update the current environment - environment = aimodel.environment - break - aimodel.dump_json(f"./out/{scenario_name}/{i}") diff --git a/kios_bt_planning/backups/kios_llm/last_response.txt b/kios_bt_planning/backups/kios_llm/last_response.txt deleted file mode 100644 index 96deca94..00000000 --- a/kios_bt_planning/backups/kios_llm/last_response.txt +++ /dev/null @@ -1,28 +0,0 @@ -{"task_cohesion": { - "task_sequence": [ - "move_hand()", - "grasp_object()", - "detach_from_plane()", - "move_object()", - "attach_to_plane()", - "screw()", - "release_object()" - ], - "step_instructions": [ - "move the hand near leg2", - "grasp leg2", - "pick up leg2", - "bring leg2 near the seat", - "attach leg2 to the seat", - "screw leg2 into the seat", - "release leg2" - ], - "object_name": ""}, -"environment_before": {"assets": ["
"], - "asset_states": {"": "on_something(
)", "": "on_something(
)", "": "on_something(
)", "": "on_something(
)", "": "on_something(
)", "": "on_something(
)", "": "can_manipulate(, )", "": "can_manipulate(, )"}, - "objects": ["", "", "", "", "", "", "", "", ""], - "object_states": {"": "screwed_into()", "": "on_something(
)", "": "on_something(
)", "": "on_something(
)", "": "on_something(
)", "": "on_something(
)"}},"environment_after": {"assets": ["
"], - "asset_states": {"": "on_something(
)", "": "on_something(
)", "": "on_something(
)", "": "on_something(
)", "": "on_something(
)", "": "on_something(
)", "": "can_manipulate(, )", "": "can_manipulate(, )"}, - "objects": ["", "", "", "", "", "", "", "", ""], - "object_states": {"": "screwed_into()", "": "screwed_into()", "": "on_something(
)", "": "on_something(
)", "": "on_something(
)", "": "on_something(
)"}},"instruction_summary": "screw leg2 to the seat", -"question":""} \ No newline at end of file diff --git a/kios_bt_planning/backups/kios_llm/out/chair_assembly/0.json b/kios_bt_planning/backups/kios_llm/out/chair_assembly/0.json deleted file mode 100644 index 3ccb8747..00000000 --- a/kios_bt_planning/backups/kios_llm/out/chair_assembly/0.json +++ /dev/null @@ -1,93 +0,0 @@ -{ - "task_cohesion": { - "task_sequence": [ - "move_hand()", - "grasp_object()", - "detach_from_plane()", - "move_object()", - "attach_to_plane()", - "screw()", - "release_object()" - ], - "step_instructions": [ - "move the hand near leg1", - "grasp leg1", - "pick up leg1", - "bring leg1 near the seat", - "attach leg1 to the seat", - "screw leg1 into the seat", - "release leg1" - ], - "object_name": "" - }, - "environment_before": { - "assets": [ - "
" - ], - "asset_states": { - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "can_manipulate(, )", - "": "can_manipulate(, )" - }, - "objects": [ - "", - "", - "", - "", - "", - "", - "", - "", - "" - ], - "object_states": { - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)" - } - }, - "environment_after": { - "assets": [ - "
" - ], - "asset_states": { - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "can_manipulate(, )", - "": "can_manipulate(, )" - }, - "objects": [ - "", - "", - "", - "", - "", - "", - "", - "", - "" - ], - "object_states": { - "": "screwed_into()", - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)" - } - }, - "instruction_summary": "screw leg1 to the seat", - "question": "" -} \ No newline at end of file diff --git a/kios_bt_planning/backups/kios_llm/out/chair_assembly/1.json b/kios_bt_planning/backups/kios_llm/out/chair_assembly/1.json deleted file mode 100644 index 2a19da24..00000000 --- a/kios_bt_planning/backups/kios_llm/out/chair_assembly/1.json +++ /dev/null @@ -1,93 +0,0 @@ -{ - "task_cohesion": { - "task_sequence": [ - "move_hand()", - "grasp_object()", - "detach_from_plane()", - "move_object()", - "attach_to_plane()", - "screw()", - "release_object()" - ], - "step_instructions": [ - "move the hand near leg2", - "grasp leg2", - "pick up leg2", - "bring leg2 near the seat", - "attach leg2 to the seat", - "screw leg2 into the seat", - "release leg2" - ], - "object_name": "" - }, - "environment_before": { - "assets": [ - "
" - ], - "asset_states": { - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "can_manipulate(, )", - "": "can_manipulate(, )" - }, - "objects": [ - "", - "", - "", - "", - "", - "", - "", - "", - "" - ], - "object_states": { - "": "screwed_into()", - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)" - } - }, - "environment_after": { - "assets": [ - "
" - ], - "asset_states": { - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "can_manipulate(, )", - "": "can_manipulate(, )" - }, - "objects": [ - "", - "", - "", - "", - "", - "", - "", - "", - "" - ], - "object_states": { - "": "screwed_into()", - "": "screwed_into()", - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)" - } - }, - "instruction_summary": "screw leg2 to the seat", - "question": "" -} \ No newline at end of file diff --git a/kios_bt_planning/backups/kios_llm/out/chair_assembly/2.json b/kios_bt_planning/backups/kios_llm/out/chair_assembly/2.json deleted file mode 100644 index c09e717f..00000000 --- a/kios_bt_planning/backups/kios_llm/out/chair_assembly/2.json +++ /dev/null @@ -1,93 +0,0 @@ -{ - "task_cohesion": { - "task_sequence": [ - "move_hand()", - "grasp_object()", - "detach_from_plane()", - "move_object()", - "attach_to_plane()", - "insert()", - "release_object()" - ], - "step_instructions": [ - "move the hand near the seat", - "grasp the seat", - "pick up the seat", - "bring the seat near the back", - "attach the seat to the back", - "insert the seat into the back", - "release the seat" - ], - "object_name": "" - }, - "environment_before": { - "assets": [ - "
" - ], - "asset_states": { - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "can_manipulate(, )", - "": "can_manipulate(, )" - }, - "objects": [ - "", - "", - "", - "", - "", - "", - "", - "", - "" - ], - "object_states": { - "": "screwed_into()", - "": "screwed_into()", - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)" - } - }, - "environment_after": { - "assets": [ - "
" - ], - "asset_states": { - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "inside_something()", - "": "on_something(
)", - "": "on_something(
)", - "": "can_manipulate(, )", - "": "can_manipulate(, )" - }, - "objects": [ - "", - "", - "", - "", - "", - "", - "", - "", - "" - ], - "object_states": { - "": "screwed_into()", - "": "screwed_into()", - "": "on_something(
)", - "": "inside_something()", - "": "on_something(
)", - "": "on_something(
)" - } - }, - "instruction_summary": "insert the seat to the back", - "question": "" -} \ No newline at end of file diff --git a/kios_bt_planning/backups/kios_llm/out/chair_assembly/3.json b/kios_bt_planning/backups/kios_llm/out/chair_assembly/3.json deleted file mode 100644 index 0a948eef..00000000 --- a/kios_bt_planning/backups/kios_llm/out/chair_assembly/3.json +++ /dev/null @@ -1,93 +0,0 @@ -{ - "task_cohesion": { - "task_sequence": [ - "move_hand()", - "grasp_object()", - "detach_from_plane()", - "move_object()", - "attach_to_plane()", - "screw()", - "release_object()" - ], - "step_instructions": [ - "move the hand near nut1", - "grasp nut1", - "pick up nut1", - "bring nut1 near the seat", - "attach nut1 to the seat", - "screw nut1 into the seat", - "release nut1" - ], - "object_name": "" - }, - "environment_before": { - "assets": [ - "
" - ], - "asset_states": { - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "inside_something()", - "": "on_something(
)", - "": "on_something(
)", - "": "can_manipulate(, )", - "": "can_manipulate(, )" - }, - "objects": [ - "", - "", - "", - "", - "", - "", - "", - "", - "" - ], - "object_states": { - "": "screwed_into()", - "": "screwed_into()", - "": "on_something(
)", - "": "inside_something()", - "": "on_something(
)", - "": "on_something(
)" - } - }, - "environment_after": { - "assets": [ - "
" - ], - "asset_states": { - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "inside_something()", - "": "screwed_into()", - "": "on_something(
)", - "": "can_manipulate(, )", - "": "can_manipulate(, )" - }, - "objects": [ - "", - "", - "", - "", - "", - "", - "", - "", - "" - ], - "object_states": { - "": "screwed_into()", - "": "screwed_into()", - "": "on_something(
)", - "": "inside_something()", - "": "screwed_into()", - "": "on_something(
)" - } - }, - "instruction_summary": "screw nut1 to the seat", - "question": "" -} \ No newline at end of file diff --git a/kios_bt_planning/backups/kios_llm/out/drawer/0.json b/kios_bt_planning/backups/kios_llm/out/drawer/0.json deleted file mode 100644 index 3c6551a8..00000000 --- a/kios_bt_planning/backups/kios_llm/out/drawer/0.json +++ /dev/null @@ -1,49 +0,0 @@ -{ - "task_cohesion": { - "task_sequence": [ - "move_hand()", - "grasp_object()", - "open_by_slide()", - "release_object()" - ], - "step_instructions": [ - "move the hand near the drawer handle", - "grasp the drawer handle", - "pull the drawer handle to open the drawer", - "release the drawer handle" - ], - "object_name": "" - }, - "environment_before": { - "assets": [ - "", - "" - ], - "asset_states": { - "": "on_something()" - }, - "objects": [ - "" - ], - "object_states": { - "": "closed()" - } - }, - "environment_after": { - "assets": [ - "", - "" - ], - "asset_states": { - "": "on_something()" - }, - "objects": [ - "" - ], - "object_states": { - "": "open()" - } - }, - "instruction_summary": "open the drawer widely", - "question": "" -} \ No newline at end of file diff --git a/kios_bt_planning/backups/kios_llm/out/drawer/1.json b/kios_bt_planning/backups/kios_llm/out/drawer/1.json deleted file mode 100644 index 3d8e80c5..00000000 --- a/kios_bt_planning/backups/kios_llm/out/drawer/1.json +++ /dev/null @@ -1,49 +0,0 @@ -{ - "task_cohesion": { - "task_sequence": [ - "move_hand()", - "grasp_object()", - "adjust_by_slide()", - "release_object()" - ], - "step_instructions": [ - "move the hand near the drawer handle", - "grasp the drawer handle", - "move the drawer handle to close the drawer halfway", - "release the drawer handle" - ], - "object_name": "" - }, - "environment_before": { - "assets": [ - "", - "" - ], - "asset_states": { - "": "on_something()" - }, - "objects": [ - "" - ], - "object_states": { - "": "open()" - } - }, - "environment_after": { - "assets": [ - "", - "" - ], - "asset_states": { - "": "on_something()" - }, - "objects": [ - "" - ], - "object_states": { - "": "open()" - } - }, - "instruction_summary": "close the drawer halfway", - "question": "" -} \ No newline at end of file diff --git a/kios_bt_planning/backups/kios_llm/out/drawer/2.json b/kios_bt_planning/backups/kios_llm/out/drawer/2.json deleted file mode 100644 index c18ca676..00000000 --- a/kios_bt_planning/backups/kios_llm/out/drawer/2.json +++ /dev/null @@ -1,49 +0,0 @@ -{ - "task_cohesion": { - "task_sequence": [ - "move_hand()", - "grasp_object()", - "close_by_slide()", - "release_object()" - ], - "step_instructions": [ - "move the hand near the drawer handle", - "grasp the drawer handle", - "move the drawer handle to close the drawer fully", - "release the drawer handle" - ], - "object_name": "" - }, - "environment_before": { - "assets": [ - "", - "" - ], - "asset_states": { - "": "on_something()" - }, - "objects": [ - "" - ], - "object_states": { - "": "open()" - } - }, - "environment_after": { - "assets": [ - "", - "" - ], - "asset_states": { - "": "on_something()" - }, - "objects": [ - "" - ], - "object_states": { - "": "closed()" - } - }, - "instruction_summary": "close the drawer fully", - "question": "" -} \ No newline at end of file diff --git a/kios_bt_planning/backups/kios_llm/out/fridge/0.json b/kios_bt_planning/backups/kios_llm/out/fridge/0.json deleted file mode 100644 index cf4c0d36..00000000 --- a/kios_bt_planning/backups/kios_llm/out/fridge/0.json +++ /dev/null @@ -1,53 +0,0 @@ -{ - "task_cohesion": { - "task_sequence": [ - "move_hand()", - "grasp_object()", - "open_by_rotate()", - "release_object()" - ], - "step_instructions": [ - "move the hand near the fridge handle", - "grasp the handle", - "open the fridge halfway", - "release the handle" - ], - "object_name": "" - }, - "environment_before": { - "assets": [ - "", - "" - ], - "asset_states": { - "": "on_something()" - }, - "objects": [ - "", - "" - ], - "object_states": { - "": "closed()", - "": "inside_something()" - } - }, - "environment_after": { - "assets": [ - "", - "" - ], - "asset_states": { - "": "on_something()" - }, - "objects": [ - "", - "" - ], - "object_states": { - "": "open()", - "": "inside_something()" - } - }, - "instruction_summary": "open the fridge halfway", - "question": "" -} \ No newline at end of file diff --git a/kios_bt_planning/backups/kios_llm/out/fridge/1.json b/kios_bt_planning/backups/kios_llm/out/fridge/1.json deleted file mode 100644 index cb3a4afb..00000000 --- a/kios_bt_planning/backups/kios_llm/out/fridge/1.json +++ /dev/null @@ -1,53 +0,0 @@ -{ - "task_cohesion": { - "task_sequence": [ - "move_hand()", - "grasp_object()", - "adjust_by_rotate()", - "release_object()" - ], - "step_instructions": [ - "move the hand near the fridge handle", - "grasp the handle", - "open the fridge wider", - "release the handle" - ], - "object_name": "" - }, - "environment_before": { - "assets": [ - "", - "" - ], - "asset_states": { - "": "on_something()" - }, - "objects": [ - "", - "" - ], - "object_states": { - "": "open()", - "": "inside_something()" - } - }, - "environment_after": { - "assets": [ - "", - "" - ], - "asset_states": { - "": "on_something()" - }, - "objects": [ - "", - "" - ], - "object_states": { - "": "open()", - "": "inside_something()" - } - }, - "instruction_summary": "open the fridge wider", - "question": "" -} \ No newline at end of file diff --git a/kios_bt_planning/backups/kios_llm/out/fridge/2.json b/kios_bt_planning/backups/kios_llm/out/fridge/2.json deleted file mode 100644 index cc7bdfa2..00000000 --- a/kios_bt_planning/backups/kios_llm/out/fridge/2.json +++ /dev/null @@ -1,61 +0,0 @@ -{ - "task_cohesion": { - "task_sequence": [ - "move_hand()", - "grasp_object()", - "detach_from_plane()", - "move_object()", - "move_hand()", - "move_object()", - "attach_to_plane()", - "release_object()" - ], - "step_instructions": [ - "move the hand near the juice", - "grasp the juice", - "pick up the juice from the top of the fridge", - "move the juice to the floor", - "move the hand near the juice", - "move the juice to the floor", - "place the juice on the floor", - "release the juice" - ], - "object_name": "" - }, - "environment_before": { - "assets": [ - "", - "" - ], - "asset_states": { - "": "on_something()" - }, - "objects": [ - "", - "" - ], - "object_states": { - "": "open()", - "": "inside_something()" - } - }, - "environment_after": { - "assets": [ - "", - "" - ], - "asset_states": { - "": "on_something()" - }, - "objects": [ - "", - "" - ], - "object_states": { - "": "open()", - "": "on_something()" - } - }, - "instruction_summary": "take the juice from the fridge and put it on the floor", - "question": "" -} \ No newline at end of file diff --git a/kios_bt_planning/backups/kios_llm/out/fridge/3.json b/kios_bt_planning/backups/kios_llm/out/fridge/3.json deleted file mode 100644 index 9b483592..00000000 --- a/kios_bt_planning/backups/kios_llm/out/fridge/3.json +++ /dev/null @@ -1,53 +0,0 @@ -{ - "task_cohesion": { - "task_sequence": [ - "move_hand()", - "grasp_object()", - "move_object()", - "release_object()" - ], - "step_instructions": [ - "move the hand near the carton box", - "grasp the carton box", - "move the carton box away from the fridge", - "release the carton box" - ], - "object_name": "" - }, - "environment_before": { - "assets": [ - "", - "" - ], - "asset_states": { - "": "on_something()" - }, - "objects": [ - "", - "" - ], - "object_states": { - "": "open()", - "": "on_something()" - } - }, - "environment_after": { - "assets": [ - "", - "" - ], - "asset_states": { - "": "on_something()" - }, - "objects": [ - "", - "" - ], - "object_states": { - "": "open()", - "": "on_something()" - } - }, - "instruction_summary": "move the carton box away from the fridge", - "question": "" -} \ No newline at end of file diff --git a/kios_bt_planning/backups/kios_llm/out/shelf/0.json b/kios_bt_planning/backups/kios_llm/out/shelf/0.json deleted file mode 100644 index 16bc759e..00000000 --- a/kios_bt_planning/backups/kios_llm/out/shelf/0.json +++ /dev/null @@ -1,67 +0,0 @@ -{ - "task_cohesion": { - "task_sequence": [ - "move_hand()", - "grasp_object()", - "detach_from_plane()", - "move_object()", - "attach_to_plane()", - "open_by_rotate()", - "release_object()" - ], - "step_instructions": [ - "move the hand near the spam", - "grasp the spam", - "pick up the spam", - "bring the spam above the trash bin", - "attach the spam to the trash bin", - "open the spam to check the expiration date", - "release the spam" - ], - "object_name": "" - }, - "environment_before": { - "assets": [ - "
", - "", - "", - "", - "" - ], - "asset_states": { - "": "on_something(
)", - "": "on_something()" - }, - "objects": [ - "", - "" - ], - "object_states": { - "": "on_something(
)", - "": "on_something()" - } - }, - "environment_after": { - "assets": [ - "
", - "", - "", - "", - "" - ], - "asset_states": { - "": "on_something(
)", - "": "on_something()" - }, - "objects": [ - "", - "" - ], - "object_states": { - "": "inside_something()", - "": "on_something()" - } - }, - "instruction_summary": "take the spam and throw it away if the out-of-date date is expired, otherwise put it on the shelf", - "question": "" -} \ No newline at end of file diff --git a/kios_bt_planning/backups/kios_llm/out/shelf/1.json b/kios_bt_planning/backups/kios_llm/out/shelf/1.json deleted file mode 100644 index c26e27b8..00000000 --- a/kios_bt_planning/backups/kios_llm/out/shelf/1.json +++ /dev/null @@ -1,63 +0,0 @@ -{ - "task_cohesion": { - "task_sequence": [ - "move_hand()", - "grasp_object()", - "detach_from_plane()", - "move_object()", - "release_object()" - ], - "step_instructions": [ - "move your hand near the spam", - "grasp the spam", - "pick up the spam", - "bring the spam above the trash bin", - "release the spam to drop it in the trash bin" - ], - "object_name": "" - }, - "environment_before": { - "assets": [ - "
", - "", - "", - "", - "" - ], - "asset_states": { - "": "on_something(
)", - "": "on_something()" - }, - "objects": [ - "", - "" - ], - "object_states": { - "": "on_something(
)", - "": "on_something()" - } - }, - "environment_after": { - "assets": [ - "
", - "", - "", - "", - "" - ], - "asset_states": { - "": "on_something(
)", - "": "on_something()" - }, - "objects": [ - "", - "" - ], - "object_states": { - "": "inside_something()", - "": "on_something()" - } - }, - "instruction_summary": "throw away the spam", - "question": "" -} \ No newline at end of file diff --git a/kios_bt_planning/backups/kios_llm/out/shelf/2.json b/kios_bt_planning/backups/kios_llm/out/shelf/2.json deleted file mode 100644 index 3b00dcb0..00000000 --- a/kios_bt_planning/backups/kios_llm/out/shelf/2.json +++ /dev/null @@ -1,69 +0,0 @@ -{ - "task_cohesion": { - "task_sequence": [ - "move_hand()", - "grasp_object()", - "detach_from_plane()", - "move_object()", - "move_object()", - "move_object()", - "attach_to_plane()", - "release_object()" - ], - "step_instructions": [ - "move the hand near the juice", - "grasp the juice", - "pick up the juice", - "take the juice from the top of the shelf", - "move the juice downward", - "move the juice above the table", - "place the juice", - "release the juice" - ], - "object_name": "" - }, - "environment_before": { - "assets": [ - "
", - "", - "", - "", - "" - ], - "asset_states": { - "": "on_something(
)", - "": "on_something()" - }, - "objects": [ - "", - "" - ], - "object_states": { - "": "inside_something()", - "": "on_something()" - } - }, - "environment_after": { - "assets": [ - "
", - "", - "", - "", - "" - ], - "asset_states": { - "": "on_something(
)", - "": "on_something()" - }, - "objects": [ - "", - "" - ], - "object_states": { - "": "inside_something()", - "": "on_something(
)" - } - }, - "instruction_summary": "move the juice on top of the table", - "question": "" -} \ No newline at end of file diff --git a/kios_bt_planning/backups/kios_llm/out/shelf/3.json b/kios_bt_planning/backups/kios_llm/out/shelf/3.json deleted file mode 100644 index 92a955b4..00000000 --- a/kios_bt_planning/backups/kios_llm/out/shelf/3.json +++ /dev/null @@ -1,63 +0,0 @@ -{ - "task_cohesion": { - "task_sequence": [ - "move_hand()", - "grasp_object()", - "detach_from_plane()", - "move_object()", - "release_object()" - ], - "step_instructions": [ - "move your hand near the juice", - "grasp the juice", - "pick up the juice", - "bring the juice above the trash bin", - "release the juice to drop it in the trash bin" - ], - "object_name": "" - }, - "environment_before": { - "assets": [ - "
", - "", - "", - "", - "" - ], - "asset_states": { - "": "on_something(
)", - "": "on_something()" - }, - "objects": [ - "", - "" - ], - "object_states": { - "": "inside_something()", - "": "on_something(
)" - } - }, - "environment_after": { - "assets": [ - "
", - "", - "", - "", - "" - ], - "asset_states": { - "": "on_something(
)", - "": "on_something()" - }, - "objects": [ - "", - "" - ], - "object_states": { - "": "inside_something()", - "": "inside_something()" - } - }, - "instruction_summary": "throw away the juice", - "question": "" -} \ No newline at end of file diff --git a/kios_bt_planning/backups/kios_llm/out/table/0.json b/kios_bt_planning/backups/kios_llm/out/table/0.json deleted file mode 100644 index ddf8b608..00000000 --- a/kios_bt_planning/backups/kios_llm/out/table/0.json +++ /dev/null @@ -1,61 +0,0 @@ -{ - "task_cohesion": { - "task_sequence": [ - "move_hand()", - "grasp_object()", - "detach_from_plane()", - "move_object()", - "move_object()", - "attach_to_plane()", - "release_object()" - ], - "step_instructions": [ - "move the hand near the sponge", - "grasp the sponge", - "pick up the sponge", - "take the sponge from the first table", - "move the sponge to the second table", - "place the sponge on the second table", - "release the sponge" - ], - "object_name": "" - }, - "environment_before": { - "assets": [ - "", - "", - "", - "" - ], - "asset_states": { - "": "next_to()", - "": "on_something()" - }, - "objects": [ - "" - ], - "object_states": { - "": "on_something()" - } - }, - "environment_after": { - "assets": [ - "", - "", - "", - "" - ], - "asset_states": { - "": "next_to()", - "": "on_something()" - }, - "objects": [ - "" - ], - "object_states": { - "": "on_something()" - } - }, - "instruction_summary": "put the sponge on the table2", - "question": "" -} \ No newline at end of file diff --git a/kios_bt_planning/backups/kios_llm/out/table/1.json b/kios_bt_planning/backups/kios_llm/out/table/1.json deleted file mode 100644 index 4b1f7029..00000000 --- a/kios_bt_planning/backups/kios_llm/out/table/1.json +++ /dev/null @@ -1,55 +0,0 @@ -{ - "task_cohesion": { - "task_sequence": [ - "move_hand()", - "grasp_object()", - "wipe_on_plane()", - "release_object()" - ], - "step_instructions": [ - "move the hand near the sponge on the second table", - "grasp the sponge", - "wipe the second table with the sponge", - "release the sponge" - ], - "object_name": "" - }, - "environment_before": { - "assets": [ - "", - "", - "", - "" - ], - "asset_states": { - "": "next_to()", - "": "on_something()" - }, - "objects": [ - "" - ], - "object_states": { - "": "on_something()" - } - }, - "environment_after": { - "assets": [ - "", - "", - "", - "" - ], - "asset_states": { - "": "next_to()", - "": "on_something()" - }, - "objects": [ - "" - ], - "object_states": { - "": "on_something()" - } - }, - "instruction_summary": "wipe the table2 with the sponge", - "question": "" -} \ No newline at end of file diff --git a/kios_bt_planning/backups/kios_llm/out/window/0.json b/kios_bt_planning/backups/kios_llm/out/window/0.json deleted file mode 100644 index e8172cb3..00000000 --- a/kios_bt_planning/backups/kios_llm/out/window/0.json +++ /dev/null @@ -1,67 +0,0 @@ -{ - "task_cohesion": { - "task_sequence": [ - "move_hand()", - "grasp_object()", - "detach_from_plane()", - "move_object()", - "attach_to_plane()", - "wipe_on_plane()", - "detach_from_plane()", - "move_object()", - "attach_to_plane()", - "release_object()" - ], - "step_instructions": [ - "move the hand near the sponge", - "grasp the sponge", - "pick up the sponge", - "bring the sponge above the window", - "attach the sponge to the window", - "wipe the window with the sponge", - "detach the sponge from the window", - "bring the sponge above the table", - "put down the sponge", - "release the sponge" - ], - "object_name": "" - }, - "environment_before": { - "assets": [ - "
", - "", - "", - "" - ], - "asset_states": { - "
": "next_to()", - "": "on_something()" - }, - "objects": [ - "" - ], - "object_states": { - "": "on_something(
)" - } - }, - "environment_after": { - "assets": [ - "
", - "", - "", - "" - ], - "asset_states": { - "
": "next_to()", - "": "on_something()" - }, - "objects": [ - "" - ], - "object_states": { - "": "on_something(
)" - } - }, - "instruction_summary": "get the sponge from the table, wipe the window with it, and put the sponge back on the table", - "question": "" -} \ No newline at end of file diff --git a/kios_bt_planning/backups/kios_llm/out/window/1.json b/kios_bt_planning/backups/kios_llm/out/window/1.json deleted file mode 100644 index 6b660c25..00000000 --- a/kios_bt_planning/backups/kios_llm/out/window/1.json +++ /dev/null @@ -1,57 +0,0 @@ -{ - "task_cohesion": { - "task_sequence": [ - "move_hand()", - "grasp_object()", - "detach_from_plane()", - "move_object()", - "release_object()" - ], - "step_instructions": [ - "move the hand near the sponge", - "grasp the sponge", - "pick up the sponge", - "bring the sponge above the trash bin", - "release the sponge and drop it in the trash bin" - ], - "object_name": "" - }, - "environment_before": { - "assets": [ - "
", - "", - "", - "" - ], - "asset_states": { - "
": "next_to()", - "": "on_something()" - }, - "objects": [ - "" - ], - "object_states": { - "": "on_something(
)" - } - }, - "environment_after": { - "assets": [ - "
", - "", - "", - "" - ], - "asset_states": { - "
": "next_to()", - "": "on_something()" - }, - "objects": [ - "" - ], - "object_states": { - "": "inside_something()" - } - }, - "instruction_summary": "throw away the sponge", - "question": "" -} \ No newline at end of file diff --git a/kios_bt_planning/backups/kios_llm/prompt/prompt_environment.txt b/kios_bt_planning/backups/kios_llm/prompt/prompt_environment.txt deleted file mode 100644 index 8a980ec7..00000000 --- a/kios_bt_planning/backups/kios_llm/prompt/prompt_environment.txt +++ /dev/null @@ -1,28 +0,0 @@ -[user] -Information about environments and objects are given as python dictionary. Example: -""" -{"environment":{"assets":["
", "", "", "", ""], - "asset_states": {"": "on_something(
)", - "": "on_something()"}, - "objects": ["", ""], - "object_states": {"": "on_something(
)", - "": "on_something()"}}} -""" -Asset states and object states are represented using those state sets: -""" -"STATE LIST" -- on_something(): Object is located on -- inside_something(): Object is located inside -- inside_hand(): Object is being grasped by a robot hand -- closed(): Object can be opened -- open(): Object can be closed or kept opened -- can_manipulate(): Object can be used to grasp and manipulate -- equipped(): Object is being equipped by the robot hand. -- screwed_into(): Object is screwed into by robot hand -- inserted_into(): Object is inserted into by robot hand -""" - should be one of the assets or objects in the environment. -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/kios_bt_planning/backups/kios_llm/prompt/prompt_example.txt b/kios_bt_planning/backups/kios_llm/prompt/prompt_example.txt deleted file mode 100644 index 8d71b9b6..00000000 --- a/kios_bt_planning/backups/kios_llm/prompt/prompt_example.txt +++ /dev/null @@ -1,232 +0,0 @@ -[user] -I will give you some examples of the input and the output you will generate. -Example 1: -""" -- Input: -{"assets":["
", "", "", "", ""], -"asset_states": {"": "on_something(
)", - "": "on_something()"}, -"objects": ["", ""], -"object_states": {"": "on_something(
)", - "": "on_something()"}, -"instruction": "Put the juice on top of the shelf"} -- Output: -``` -{"task_cohesion": { - "task_sequence": [ - "move_hand()", - "grasp_object()", - "detach_from_plane()", - "move_object()", - "move_object()", - "move_object()", - "attach_to_plane()", - "release_object()" - ], - "step_instructions": [ - "move the hand near the juice", - "grasp the juice", - "pick up the juice", - "take the juice from the bottom of the shelf", - "move the juice upward", - "move the juice above the upper shelf", - "place the juice", - "release the juice" - ], - "object_name": ""}, -"environment_before": {"assets":["
", "", "", "", ""], - "asset_states": {"": "on_something(
)", - "": "on_something()"}, - "objects": ["", ""], - "object_states": {"": "on_something(
)", - "": "on_something()"}}, -"environment_after": {"assets":["
", "", "", "", ""], - "asset_states": {"": "on_something(
)", - "": "on_something()"}, - "objects": ["", ""], - "object_states": {"": "on_something(
)", - "": "on_something()"}}, -"instruction_summary": "put the juice on top of the shelf", -"question":""} -``` -""" -Example 2: -""" -- Input: -{"assets":["
", "", "", "", ""], -"asset_states": {"": "on_something(
)", - "": "on_something()"}, -"objects": ["", ""], -"object_states": {"": "on_something(
)", - "": "on_something()"}, -"instruction": "Throw away the spam into the trash bin."} -- Output: -``` -{"task_cohesion": { - "task_sequence": [ - "move_hand()", - "grasp_object()", - "detach_from_plane()", - "move_object()", - "release_object()" - ], - "step_instructions": [ - "move your hand near the spam", - "grasp the spam", - "pick up the spam", - "bring the spam above the trash bin", - "release the spam to drop it in the trash bin" - ], - "object_name": ""}, -"environment_before": {"assets":["
", "", "", "", ""], - "asset_states": {"": "on_something(
)", - "": "on_something()"}, - "objects": ["", ""], - "object_states": {"": "on_something(
)", - "": "on_something()"}}, -"environment_after": {"assets":["
", "", "", "", ""], - "asset_states": {"": "on_something(
)", - "": "on_something()"}, - "objects": ["", ""], - "object_states": {"": "inside_something()", - "": "on_something()"}}, -"instruction_summary": "throw away the spam", -"question":""} -``` -""" -Example 3: -""" --Input: -{"assets":["", ""], -"asset_states": {"": "on_something()"}, -"objects": [""], -"object_states": {"": "closed()"}, -"instruction": "Open the fridge."} -- Output: -``` -{"task_cohesion": { - "task_sequence": [ - "move_hand()", - "grasp_object()", - "open_by_rotate()", - "release_object()" - ], - "step_instructions": [ - "move the hand near the fridge handle", - "grasp the handle", - "open the fridge by pulling the handle", - "release the handle" - ], - "object_name": ""}, -"environment_before": {"assets":["", ""], - "asset_states": {"": "on_something()"}, - "objects": [""], - "object_states": {"": "closed()"}}, -"environment_after": {"assets":["", ""], - "asset_states": {"": "on_something()"}, - "objects": [""], - "object_states": {"": "open()"}}, -"instruction_summary": "open a fridge", -"question":""} -``` -""" -Example 4: -""" --Input: -{"assets":["
", ""], -"asset_states": {"
": "next_to()"}, -"objects": [""], -"object_states": {"": "on_something(
)"}, -"instruction": "Wipe the window with the sponge. Then, put the sponge on the table."} -- Output: -``` -{"task_cohesion": { - "task_sequence": [ - "move_hand()", - "grasp_object()", - "detach_from_plane()", - "move_object()", - "attach_to_plane()", - "wipe_on_plane()", - "detach_from_plane()", - "move_object()", - "attach_to_plane()", - "release_object()" - ], - "step_instructions": [ - "move the hand near the sponge", - "grasp the sponge", - "pick up the sponge", - "bring the sponge above the window", - "attach the sponge to the window", - "wipe the window with the sponge", - "detach the sponge from the window", - "bring the sponge above the table", - "put down the sponge", - "release the sponge" - ], - "object_name": ""}, -"environment_before": {"assets":["
", ""], - "asset_states": {"
": "next_to()"}, - "objects": [""], - "object_states": {"": "on_something(
)"},} -"environment_after": {"assets":["
", ""], - "asset_states": {"
": "next_to()"}, - "objects": [""], - "object_states": {"": "on_something(
)"}}, -"instruction_summary": "wipe the window with the sponge, then put the sponge on the table", -"question":""} -``` -""" -Example 5: -""" --Input: -{"assets":["
"], -"asset_states": {"": "can_manipulate()"}, -"objects": ["", "", ""], -"object_states": {"": "on_something(
)", "": "on_something(
), "": "can_manipulate()"}, -"instruction": "screw the nut into the bolt. Tool should be used and be put back at the end."} -- Output: -``` -{"task_cohesion": { - "task_sequence": [ - "load_tool()", - "approach()", - "grasp_object()", - "pick_up()", - "approach()", - "insert()", - "screw()", - "release_object()", - "unload_tool()" - ], - "step_instructions": [ - "load the parallel_toolbox", - "move the hand close to the nut", - "grasp the nut", - "pick up the nut", - "move the hand close to the bolt", - "insert the nut onto the bolt", - "screw the nut into the bold", - "release the nut", - "unload the parallel_toolbox" - ], - "object_name": ""}, -"environment_before": {"assets":["
"], - "asset_states": {"": "can_manipulate()"}, - "objects": ["", "", ""], - "object_states": {"": "on_something(
)", "": "on_something(
)"},} -"environment_after": {"assets":["
"], - "asset_states": {"": "can_manipulate()"}, - "objects": ["", "", ""], - "object_states": {"": "screwed_into()", "": "on_something(
)"}, -"instruction_summary": "screw the nut into the bolt. Tool should be used and be put back at the end.", -"question":""} -} -``` -""" -From these examples, learn that some robotic actions have dependencies with the actions before and after them. -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/kios_bt_planning/backups/kios_llm/prompt/prompt_function.txt b/kios_bt_planning/backups/kios_llm/prompt/prompt_function.txt deleted file mode 100644 index a5e4fdd9..00000000 --- a/kios_bt_planning/backups/kios_llm/prompt/prompt_function.txt +++ /dev/null @@ -1,25 +0,0 @@ -[user] -Necessary and sufficient robot actions are defined as follows: -""" -"ROBOT ACTION LIST" -- load_tool(): Equip the robot hand with a toolbox in order to manipulate some objects later. This action can only be performed when no tool is being equipped. -- unload_tool(): This action can only be performed when a tool is being equipped by the robot hand. The opposite action of load_tool(). -- move_hand(): Move the robot hand from one position to another with/without grasping an object. -- approach(): Move the robot hand to get close to an object. -- grasp_object(): Grab an object with robot hand. The robot hand must equips the proper toolbox for manipulating the object. -- release_object(): Release an object in the robot hand when the robot is grasping something. -- move_object(): Move the object grabbed by the robot hand from one position to another. move_object() is allowed only when the object is not physically constrained by the environment. For example, if the robot hand moves an object on the floor to another location, move_object() is not allowed because the object is constrained by the floor. -- pick_up(): This action can only be performed if an object is grabbed. Move the grabbed object from a state in which it is constrained by a plane to a state in which it is not constrained by any plane. For example, pick_up() is used when a robot hand picks up an object on a table. -- put_down(): This action can only be performed if an object is grabbed. The opposite operation of pick_up(). -- insert(): This action can only be performed if an object is in hand. Insert the grasped object into a hole of another object. -- screw(): This action can only be performed if an object is grabbed and has been inserted into the hole. Screw the grabbed object into the hole or onto a bolt-like structure of another obejct. This can only be conducted when the object and the hole has an screw thread assembly constraint. -- adjust_by_rotate(): This action can only be performed if an object is grabbed. Rotate an object that is rotationally constrained by its environment along its rotation. For example, when adjusting the temperature of a refrigerator, the temperature knob makes this motion. -- close_by_rotate(): This action can only be performed if an object is grabbed. The opposite operation of open_by_rotate(). -- open_by_slide(): This action can only be performed if an object is grabbed. Moves an object that is translationally constrained in two axes from its environment along one unbounded axis. For example, when opening a sliding door or drawer, the handle makes this movement. -- adjust_by_slide(): This action can only be performed if an object is grabbed. Slide an object that is translationally constrained in two axes from its environment along one unbounded axis. For example, when widen the gap between a sliding door and the wall, the handle makes this movement. -- close_by_slide(): This action can only be performed if an object is grabbed. The opposite operation of open_by_slide(). -""" -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/kios_bt_planning/backups/kios_llm/prompt/prompt_output_format.txt b/kios_bt_planning/backups/kios_llm/prompt/prompt_output_format.txt deleted file mode 100644 index 7be3d38b..00000000 --- a/kios_bt_planning/backups/kios_llm/prompt/prompt_output_format.txt +++ /dev/null @@ -1,20 +0,0 @@ -[user] -You divide the actions given in the text into detailed robot actions and put them together as a python dictionary. -The dictionary has five keys. -""" -- dictionary["task_cohesion"]: A dictionary containing information about the robot's actions that have been split up. -- dictionary["environment_before"]: The state of the environment before the manipulation. -- dictionary["environment_after"]: The state of the environment after the manipulation. -- dictionary["instruction_summary"]: contains a brief summary of the given sentence. -- dictionary["question"]: If you cannot understand the given sentence, you can ask the user to rephrase the sentence. Leave this key empty if you can understand the given sentence. -""" -Three keys exist in dictionary["task_cohesion"]. -""" -- dictionary["task_cohesion"]["task_sequence"]: Contains a list of robot actions. Only the behaviors defined in the "ROBOT ACTION LIST" will be used. -- dictionary["task_cohesion"]["step_instructions"]: contains a list of instructions corresponding to dictionary["task_cohesion"]["task_sequence"]. -- dictionary["task_cohesion"]["object_name"]: The name of the manipulated object. Only objects defined in the input dictionary will be used for the object name. -""" -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/kios_bt_planning/backups/kios_llm/prompt/prompt_role.txt b/kios_bt_planning/backups/kios_llm/prompt/prompt_role.txt deleted file mode 100644 index 06feafda..00000000 --- a/kios_bt_planning/backups/kios_llm/prompt/prompt_role.txt +++ /dev/null @@ -1,5 +0,0 @@ -[user] -You are an excellent interpreter of human instructions for robotic assembly tasks. Given an instruction and information about the working environment, you break it down into a sequence of robotic actions. -Please do not begin working until I say "Start working." Instead, simply output the message "Waiting for next input." Understood? -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/kios_bt_planning/backups/kios_llm/query/query.txt b/kios_bt_planning/backups/kios_llm/query/query.txt deleted file mode 100644 index ab5f24f2..00000000 --- a/kios_bt_planning/backups/kios_llm/query/query.txt +++ /dev/null @@ -1,27 +0,0 @@ -Start working. Resume from the environment below. -""" -{"environment":[ENVIRONMENT]} -""" -The instruction is as follows: -""" -{"instruction": [INSTRUCTION]} -""" -The dictionary that you return should be formatted as python dictionary. Follow these rules: -1. The first element should be move_hand() to move the robot hand closer to the object. Always end with releasing the object. -2. Make sure that each element of the ["step_instructions"] explains corresponding element of the ["task_sequence"]. Refer to the "ROBOT ACTION LIST" to understand the elements of ["task_sequence"]. -3, The length of the ["step_instructions"] list must be the same as the length of the ["task_sequence"] list. -4. Never left ',' at the end of the list. -5. Keep track of all items listed in the "objects" section of the "environment_before" field. Please ensure that you fill out both the "objects" and "object_states" sections for all listed items. -6. Use the "STATE LIST" for the "object states" field. -""" -"STATE LIST" -- on_something(): Object is located on -- inside_something(): Object is located inside -- inside_hand(): Object is being grasped by a robot hand -- closed(): Object can be opened -- open(): Object can be closed or kept opened -""" -7. All keys of the dictionary should be double-quoted. -8. Insert ``` at the beginning and the end of the dictionary to separate it from the rest of your response. -9. Make sure that you output a consistent manipultation as a single arm robot. For example, grasping an object should not occur in successive steps. -Adhere to the output format I defined above. Follow the nine rules. Think step by step. \ No newline at end of file diff --git a/kios_bt_planning/backups/kios_llm/system/system.txt b/kios_bt_planning/backups/kios_llm/system/system.txt deleted file mode 100644 index 4cef77dd..00000000 --- a/kios_bt_planning/backups/kios_llm/system/system.txt +++ /dev/null @@ -1 +0,0 @@ -You are an excellent interpreter of instructions for robotic assembly tasks. Given an instruction and information about the working environment, you break it down into a sequence of robotic actions. \ No newline at end of file diff --git a/kios_bt_planning/backups/kios_llm_action_sequence/aimodel.py b/kios_bt_planning/backups/kios_llm_action_sequence/aimodel.py deleted file mode 100644 index 23d28ef3..00000000 --- a/kios_bt_planning/backups/kios_llm_action_sequence/aimodel.py +++ /dev/null @@ -1,446 +0,0 @@ -import openai -import tiktoken -import json -import os -import re -import argparse -import sys - - -enc = tiktoken.get_encoding("cl100k_base") - -script_dir = os.path.dirname(__file__) -secret_path = os.path.join(script_dir, "secrets.json") - -# with open("../../secrets.json") as f: -with open(secret_path) as f: - credentials = json.load(f) - print(credentials) - -# dir_system = "./system" -dir_system = os.path.join(script_dir, "system") -# dir_prompt = "./prompt" -dir_prompt = os.path.join(script_dir, "prompt") -# dir_query = "./query" -dir_query = os.path.join(script_dir, "query") - -prompt_load_order = [ - "prompt_role", - "prompt_function", - "prompt_environment", - "prompt_output_format", - "prompt_example", -] - - -# azure openai api has changed from '2023-05-15' to '2023-05-15' -# if you are using a 0301 version, use '2022-12-01' -# Otherwise, use '2023-05-15' - - -class ChatGPT: - query: str = None - """ - the query request: start working ... - """ - instruction: str = None - """ - the member variable to keep the latest instruction - """ - system_message: dict = None - """ - the "system" prompt part - """ - messages: list = None - """ - The list of messages for prompting. - including: the required format (in prompt folder), the last response as assistant prompt - """ - - VALID_API_VERSIONS = ["2022-12-01", "2023-05-15"] - - def __init__( - self, credentials, prompt_load_order, use_azure=False, api_version="2023-05-15" - ): - # initialize the openai api - self.use_azure = use_azure - if self.use_azure: - openai.api_key = credentials["azureopenai"]["AZURE_OPENAI_KEY"] - openai.api_base = credentials["azureopenai"]["AZURE_OPENAI_ENDPOINT"] - openai.api_type = "azure" - if api_version not in self.VALID_API_VERSIONS: - raise ValueError( - f"api_version must be one of {self.VALID_API_VERSIONS}" - ) - openai.api_version = api_version - else: - openai.organization = credentials["openai"]["YOUR_ORG_ID"] - openai.api_key = credentials["openai"]["OPENAI_API_KEY"] - - self.credentials = credentials - self.messages = [] - self.max_token_length = 8000 - self.max_completion_length = 1000 - self.last_response = None - self.query = "" - self.instruction = "" - # load prompt file - # * system - fp_system = os.path.join(dir_system, "system.txt") - with open(fp_system) as f: - data = f.read() - self.system_message = {"role": "system", "content": data} - - # load prompt file - for prompt_name in prompt_load_order: - fp_prompt = os.path.join(dir_prompt, prompt_name + ".txt") - with open(fp_prompt) as f: - data = f.read() - data_spilit = re.split(r"\[user\]\n|\[assistant\]\n", data) - data_spilit = [item for item in data_spilit if len(item) != 0] - # it start with user and ends with system - assert len(data_spilit) % 2 == 0 - for i, item in enumerate(data_spilit): - if i % 2 == 0: - self.messages.append({"sender": "user", "text": item}) - else: - self.messages.append({"sender": "assistant", "text": item}) - fp_query = os.path.join(dir_query, "query.txt") - with open(fp_query) as f: - self.query = f.read() - - def create_prompt(self): - """ - create the prompt for gpt api - """ - prompt = "" - if self.use_azure and openai.api_version == "2022-12-01": - prompt = "<|im_start|>system\n" - prompt += self.system_message["content"] - prompt += "\n<|im_end|>\n" - for message in self.messages: - prompt += ( - f"\n<|im_start|>{message['sender']}\n{message['text']}\n<|im_end|>" - ) - prompt += "\n<|im_start|>assistant\n" - print("prompt length: " + str(len(enc.encode(prompt)))) - if ( - len(enc.encode(prompt)) - > self.max_token_length - self.max_completion_length - ): - print("prompt too long. truncated.") - # * truncate the prompt by removing the oldest two messages - self.messages = self.messages[2:] - prompt = self.create_prompt() - else: - prompt = [] # list of prompt entries - # load the system prompt - prompt.append(self.system_message) - # load the knowledge and the last response - for message in self.messages: - prompt.append({"role": message["sender"], "content": message["text"]}) - - # * truncate the prompt if it is too long - prompt_content = "" # this is only for checking the length of the prompt - for message in prompt: - prompt_content += message["content"] - print("prompt length: " + str(len(enc.encode(prompt_content)))) - if ( - len(enc.encode(prompt_content)) - > self.max_token_length - self.max_completion_length - ): - print("prompt too long. truncated.") - # * truncate the prompt by removing the oldest two messages - self.messages = self.messages[2:] - prompt = self.create_prompt() - return prompt - - def extract_json_part(self, text): - """ - extract the markdown code block part from the text - """ - if text.find("```") == -1: - return text - text_json = text[text.find("```") + 3 : text.find("```", text.find("```") + 3)] - return text_json - - def generate(self, runtime_instruction, environment, is_user_feedback=False): - """ - runtime_instruction: the instruction from the problem/user - """ - if is_user_feedback: - self.messages.append( - { - "sender": "user", - "text": runtime_instruction + "\n" + self.instruction, - } - ) - else: - # get the query template - text_base = self.query - # replace the environment part of the template - if text_base.find("[ENVIRONMENT]") != -1: - text_base = text_base.replace("[ENVIRONMENT]", json.dumps(environment)) - # replace the instruction part of the template - if text_base.find("[INSTRUCTION]") != -1: - text_base = text_base.replace("[INSTRUCTION]", runtime_instruction) - # update the instruction of the chatbot - self.instruction = text_base - # finally, add the text_base to the user query message - self.messages.append({"sender": "user", "text": text_base}) - - if self.use_azure and openai.api_version == "2022-12-01": - # Remove unsafe user inputs. May need further refinement in the - # future. - if runtime_instruction.find("<|im_start|>") != -1: - runtime_instruction = runtime_instruction.replace("<|im_start|>", "") - if runtime_instruction.find("<|im_end|>") != -1: - runtime_instruction = runtime_instruction.replace("<|im_end|>", "") - deployment_name = self.credentials["azureopenai"][ - "AZURE_OPENAI_DEPLOYMENT_NAME_CHATGPT" - ] - response = openai.Completion.create( - engine=deployment_name, - prompt=self.create_prompt(), - temperature=0.1, - max_tokens=self.max_completion_length, - top_p=0.5, - frequency_penalty=0.0, - presence_penalty=0.0, - stop=["<|im_end|>"], - ) - text = response["choices"][0]["text"] - elif self.use_azure and openai.api_version == "2023-05-15": - deployment_name = self.credentials["azureopenai"][ - "AZURE_OPENAI_DEPLOYMENT_NAME_CHATGPT" - ] - response = openai.ChatCompletion.create( - engine=deployment_name, - messages=self.create_prompt(), - temperature=0.1, - max_tokens=self.max_completion_length, - top_p=0.5, - frequency_penalty=0.0, - presence_penalty=0.0, - ) - text = response["choices"][0]["message"]["content"] - else: - # * request the gpt to response - response = openai.ChatCompletion.create( - model="gpt-3.5-turbo-16k", - # "gpt-4" is available, too. Check the available models in https://platform.openai.com/docs/models/ - messages=self.create_prompt(), - temperature=0.1, - max_tokens=self.max_completion_length, - top_p=0.5, - frequency_penalty=0.0, - presence_penalty=0.0, - ) - text = response["choices"][0].message.content - print(text) - self.last_response = text - self.last_response = self.extract_json_part(self.last_response) - self.last_response = self.last_response.replace("'", '"') - # dump to a text file - with open("last_response.txt", "w") as f: - f.write(self.last_response) - try: - self.json_dict = json.loads(self.last_response, strict=False) - # update the environment after the execution of the action - self.environment = self.json_dict["environment_after"] - except BaseException: - self.json_dict = None - import pdb - - pdb.set_trace() - - # * use the last response as the assistant prompt for the next round - if len(self.messages) > 0 and self.last_response is not None: - self.messages.append({"sender": "assistant", "text": self.last_response}) - - return self.json_dict - - def dump_json(self, dump_name=None): - """ - dump the json dictionary to a file, used to save into a json file - """ - if dump_name is not None: - # dump the dictionary to json file dump 1, 2, ... - fp = os.path.join(dump_name + ".json") - with open(fp, "w") as f: - json.dump(self.json_dict, f, indent=4) - - -if __name__ == "__main__": - parser = argparse.ArgumentParser() - parser.add_argument( - "--scenario", - type=str, - required=False, # ! CHEAT - help="scenario name (see the code for details)", - ) - args = parser.parse_args() - scenario_name = args.scenario - - # ! CHEAT - scenario_name = "chair_assembly" - - # 1. example of moving objects on the table and the shelf - if scenario_name == "shelf": - environment = { - "assets": [ - "
", - "", - "", - "", - "", - ], - "asset_states": { - "": "on_something(
)", - "": "on_something()", - }, - "objects": ["", ""], - "object_states": { - "": "on_something(
)", - "": "on_something()", - }, - } - instructions = [ - "Put the juice on top of the shelf", - "Throw away the spam into the trash bin", - "Move the juice on top of the table", - "Throw away the juice", - ] - # 2. example of opening and closing the fridge, and putting the juice on - # the floor - elif scenario_name == "fridge": - environment = { - "assets": ["", ""], - "asset_states": {"": "on_something()"}, - "objects": ["", ""], - "object_states": { - "": "closed()", - "": "inside_something()", - }, - } - instructions = [ - "Open the fridge half way", - "Open the fridge wider", - "Take the juice in the fridge and put it on the floor", - "Close the fridge", - ] - # 3. example of opening and closing the drawer - elif scenario_name == "drawer": - environment = { - "assets": ["", ""], - "asset_states": {"": "on_something()"}, - "objects": [""], - "object_states": {"": "closed()"}, - } - instructions = [ - "Open the drawer widely", - "Close the drawer half way", - "Close the drawer fully", - ] - # 4. example of wiping the table - elif scenario_name == "table": - environment = { - "assets": ["", "", "", ""], - "asset_states": { - "": "next_to()", - "": "on_something()", - }, - "objects": [""], - "object_states": {"": "on_something()"}, - } - instructions = [ - "Put the sponge on the table2", - "Wipe the table2 with the sponge", - ] - # 5. example of wiping the window - elif scenario_name == "window": - environment = { - "assets": ["
", "", "", ""], - "asset_states": { - "
": "next_to()", - "": "on_something()", - }, - "objects": [""], - "object_states": {"": "on_something(
)"}, - } - instructions = [ - "Get the sponge from the table and wipe the window with it. After that, put the sponge back on the table", - "Throw away the sponge on the table", - ] - elif scenario_name == "chair_assembly": - environment = { - "assets": [ - "
", - ], - "asset_states": { - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "can_manipulate(, )", - "": "can_manipulate(, )", - }, - "objects": [ - "", - "", - "", - "", - "", - "", - "", - "", - "", - ], - "object_states": { - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - }, - } - instructions = [ - "Screw the leg1 to the seat", - "Screw the leg2 to the seat", - "Insert the seat to the back", - "Screw the nut1 to the seat", - "Screw the nut2 to the seat", - ] - - else: - parser.error("Invalid scenario name:" + scenario_name) - - aimodel = ChatGPT( - credentials, - prompt_load_order=prompt_load_order, - use_azure=False, - api_version="2023-05-15", - ) - - if not os.path.exists("./out/" + scenario_name): - os.makedirs("./out/" + scenario_name) - # run the instructions one by one - for i, instruction in enumerate(instructions): - print(json.dumps(environment)) - text = aimodel.generate(instruction, environment, is_user_feedback=False) - while True: - user_feedback = input("user feedback (return empty if satisfied): ") - if user_feedback == "q": - exit() - if user_feedback != "": - text = aimodel.generate( - user_feedback, environment, is_user_feedback=True - ) - else: - # update the current environment - environment = aimodel.environment - break - aimodel.dump_json(f"./out/{scenario_name}/{i}") diff --git a/kios_bt_planning/backups/kios_llm_action_sequence/last_response.txt b/kios_bt_planning/backups/kios_llm_action_sequence/last_response.txt deleted file mode 100644 index 96deca94..00000000 --- a/kios_bt_planning/backups/kios_llm_action_sequence/last_response.txt +++ /dev/null @@ -1,28 +0,0 @@ -{"task_cohesion": { - "task_sequence": [ - "move_hand()", - "grasp_object()", - "detach_from_plane()", - "move_object()", - "attach_to_plane()", - "screw()", - "release_object()" - ], - "step_instructions": [ - "move the hand near leg2", - "grasp leg2", - "pick up leg2", - "bring leg2 near the seat", - "attach leg2 to the seat", - "screw leg2 into the seat", - "release leg2" - ], - "object_name": ""}, -"environment_before": {"assets": ["
"], - "asset_states": {"": "on_something(
)", "": "on_something(
)", "": "on_something(
)", "": "on_something(
)", "": "on_something(
)", "": "on_something(
)", "": "can_manipulate(, )", "": "can_manipulate(, )"}, - "objects": ["", "", "", "", "", "", "", "", ""], - "object_states": {"": "screwed_into()", "": "on_something(
)", "": "on_something(
)", "": "on_something(
)", "": "on_something(
)", "": "on_something(
)"}},"environment_after": {"assets": ["
"], - "asset_states": {"": "on_something(
)", "": "on_something(
)", "": "on_something(
)", "": "on_something(
)", "": "on_something(
)", "": "on_something(
)", "": "can_manipulate(, )", "": "can_manipulate(, )"}, - "objects": ["", "", "", "", "", "", "", "", ""], - "object_states": {"": "screwed_into()", "": "screwed_into()", "": "on_something(
)", "": "on_something(
)", "": "on_something(
)", "": "on_something(
)"}},"instruction_summary": "screw leg2 to the seat", -"question":""} \ No newline at end of file diff --git a/kios_bt_planning/backups/kios_llm_action_sequence/out/chair_assembly/0.json b/kios_bt_planning/backups/kios_llm_action_sequence/out/chair_assembly/0.json deleted file mode 100644 index 3ccb8747..00000000 --- a/kios_bt_planning/backups/kios_llm_action_sequence/out/chair_assembly/0.json +++ /dev/null @@ -1,93 +0,0 @@ -{ - "task_cohesion": { - "task_sequence": [ - "move_hand()", - "grasp_object()", - "detach_from_plane()", - "move_object()", - "attach_to_plane()", - "screw()", - "release_object()" - ], - "step_instructions": [ - "move the hand near leg1", - "grasp leg1", - "pick up leg1", - "bring leg1 near the seat", - "attach leg1 to the seat", - "screw leg1 into the seat", - "release leg1" - ], - "object_name": "" - }, - "environment_before": { - "assets": [ - "
" - ], - "asset_states": { - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "can_manipulate(, )", - "": "can_manipulate(, )" - }, - "objects": [ - "", - "", - "", - "", - "", - "", - "", - "", - "" - ], - "object_states": { - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)" - } - }, - "environment_after": { - "assets": [ - "
" - ], - "asset_states": { - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "can_manipulate(, )", - "": "can_manipulate(, )" - }, - "objects": [ - "", - "", - "", - "", - "", - "", - "", - "", - "" - ], - "object_states": { - "": "screwed_into()", - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)" - } - }, - "instruction_summary": "screw leg1 to the seat", - "question": "" -} \ No newline at end of file diff --git a/kios_bt_planning/backups/kios_llm_action_sequence/out/chair_assembly/1.json b/kios_bt_planning/backups/kios_llm_action_sequence/out/chair_assembly/1.json deleted file mode 100644 index 2a19da24..00000000 --- a/kios_bt_planning/backups/kios_llm_action_sequence/out/chair_assembly/1.json +++ /dev/null @@ -1,93 +0,0 @@ -{ - "task_cohesion": { - "task_sequence": [ - "move_hand()", - "grasp_object()", - "detach_from_plane()", - "move_object()", - "attach_to_plane()", - "screw()", - "release_object()" - ], - "step_instructions": [ - "move the hand near leg2", - "grasp leg2", - "pick up leg2", - "bring leg2 near the seat", - "attach leg2 to the seat", - "screw leg2 into the seat", - "release leg2" - ], - "object_name": "" - }, - "environment_before": { - "assets": [ - "
" - ], - "asset_states": { - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "can_manipulate(, )", - "": "can_manipulate(, )" - }, - "objects": [ - "", - "", - "", - "", - "", - "", - "", - "", - "" - ], - "object_states": { - "": "screwed_into()", - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)" - } - }, - "environment_after": { - "assets": [ - "
" - ], - "asset_states": { - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "can_manipulate(, )", - "": "can_manipulate(, )" - }, - "objects": [ - "", - "", - "", - "", - "", - "", - "", - "", - "" - ], - "object_states": { - "": "screwed_into()", - "": "screwed_into()", - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)" - } - }, - "instruction_summary": "screw leg2 to the seat", - "question": "" -} \ No newline at end of file diff --git a/kios_bt_planning/backups/kios_llm_action_sequence/out/chair_assembly/2.json b/kios_bt_planning/backups/kios_llm_action_sequence/out/chair_assembly/2.json deleted file mode 100644 index c09e717f..00000000 --- a/kios_bt_planning/backups/kios_llm_action_sequence/out/chair_assembly/2.json +++ /dev/null @@ -1,93 +0,0 @@ -{ - "task_cohesion": { - "task_sequence": [ - "move_hand()", - "grasp_object()", - "detach_from_plane()", - "move_object()", - "attach_to_plane()", - "insert()", - "release_object()" - ], - "step_instructions": [ - "move the hand near the seat", - "grasp the seat", - "pick up the seat", - "bring the seat near the back", - "attach the seat to the back", - "insert the seat into the back", - "release the seat" - ], - "object_name": "" - }, - "environment_before": { - "assets": [ - "
" - ], - "asset_states": { - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "can_manipulate(, )", - "": "can_manipulate(, )" - }, - "objects": [ - "", - "", - "", - "", - "", - "", - "", - "", - "" - ], - "object_states": { - "": "screwed_into()", - "": "screwed_into()", - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)" - } - }, - "environment_after": { - "assets": [ - "
" - ], - "asset_states": { - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "inside_something()", - "": "on_something(
)", - "": "on_something(
)", - "": "can_manipulate(, )", - "": "can_manipulate(, )" - }, - "objects": [ - "", - "", - "", - "", - "", - "", - "", - "", - "" - ], - "object_states": { - "": "screwed_into()", - "": "screwed_into()", - "": "on_something(
)", - "": "inside_something()", - "": "on_something(
)", - "": "on_something(
)" - } - }, - "instruction_summary": "insert the seat to the back", - "question": "" -} \ No newline at end of file diff --git a/kios_bt_planning/backups/kios_llm_action_sequence/out/chair_assembly/3.json b/kios_bt_planning/backups/kios_llm_action_sequence/out/chair_assembly/3.json deleted file mode 100644 index 0a948eef..00000000 --- a/kios_bt_planning/backups/kios_llm_action_sequence/out/chair_assembly/3.json +++ /dev/null @@ -1,93 +0,0 @@ -{ - "task_cohesion": { - "task_sequence": [ - "move_hand()", - "grasp_object()", - "detach_from_plane()", - "move_object()", - "attach_to_plane()", - "screw()", - "release_object()" - ], - "step_instructions": [ - "move the hand near nut1", - "grasp nut1", - "pick up nut1", - "bring nut1 near the seat", - "attach nut1 to the seat", - "screw nut1 into the seat", - "release nut1" - ], - "object_name": "" - }, - "environment_before": { - "assets": [ - "
" - ], - "asset_states": { - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "inside_something()", - "": "on_something(
)", - "": "on_something(
)", - "": "can_manipulate(, )", - "": "can_manipulate(, )" - }, - "objects": [ - "", - "", - "", - "", - "", - "", - "", - "", - "" - ], - "object_states": { - "": "screwed_into()", - "": "screwed_into()", - "": "on_something(
)", - "": "inside_something()", - "": "on_something(
)", - "": "on_something(
)" - } - }, - "environment_after": { - "assets": [ - "
" - ], - "asset_states": { - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "inside_something()", - "": "screwed_into()", - "": "on_something(
)", - "": "can_manipulate(, )", - "": "can_manipulate(, )" - }, - "objects": [ - "", - "", - "", - "", - "", - "", - "", - "", - "" - ], - "object_states": { - "": "screwed_into()", - "": "screwed_into()", - "": "on_something(
)", - "": "inside_something()", - "": "screwed_into()", - "": "on_something(
)" - } - }, - "instruction_summary": "screw nut1 to the seat", - "question": "" -} \ No newline at end of file diff --git a/kios_bt_planning/backups/kios_llm_action_sequence/out/drawer/0.json b/kios_bt_planning/backups/kios_llm_action_sequence/out/drawer/0.json deleted file mode 100644 index 3c6551a8..00000000 --- a/kios_bt_planning/backups/kios_llm_action_sequence/out/drawer/0.json +++ /dev/null @@ -1,49 +0,0 @@ -{ - "task_cohesion": { - "task_sequence": [ - "move_hand()", - "grasp_object()", - "open_by_slide()", - "release_object()" - ], - "step_instructions": [ - "move the hand near the drawer handle", - "grasp the drawer handle", - "pull the drawer handle to open the drawer", - "release the drawer handle" - ], - "object_name": "" - }, - "environment_before": { - "assets": [ - "", - "" - ], - "asset_states": { - "": "on_something()" - }, - "objects": [ - "" - ], - "object_states": { - "": "closed()" - } - }, - "environment_after": { - "assets": [ - "", - "" - ], - "asset_states": { - "": "on_something()" - }, - "objects": [ - "" - ], - "object_states": { - "": "open()" - } - }, - "instruction_summary": "open the drawer widely", - "question": "" -} \ No newline at end of file diff --git a/kios_bt_planning/backups/kios_llm_action_sequence/out/drawer/1.json b/kios_bt_planning/backups/kios_llm_action_sequence/out/drawer/1.json deleted file mode 100644 index 3d8e80c5..00000000 --- a/kios_bt_planning/backups/kios_llm_action_sequence/out/drawer/1.json +++ /dev/null @@ -1,49 +0,0 @@ -{ - "task_cohesion": { - "task_sequence": [ - "move_hand()", - "grasp_object()", - "adjust_by_slide()", - "release_object()" - ], - "step_instructions": [ - "move the hand near the drawer handle", - "grasp the drawer handle", - "move the drawer handle to close the drawer halfway", - "release the drawer handle" - ], - "object_name": "" - }, - "environment_before": { - "assets": [ - "", - "" - ], - "asset_states": { - "": "on_something()" - }, - "objects": [ - "" - ], - "object_states": { - "": "open()" - } - }, - "environment_after": { - "assets": [ - "", - "" - ], - "asset_states": { - "": "on_something()" - }, - "objects": [ - "" - ], - "object_states": { - "": "open()" - } - }, - "instruction_summary": "close the drawer halfway", - "question": "" -} \ No newline at end of file diff --git a/kios_bt_planning/backups/kios_llm_action_sequence/out/drawer/2.json b/kios_bt_planning/backups/kios_llm_action_sequence/out/drawer/2.json deleted file mode 100644 index c18ca676..00000000 --- a/kios_bt_planning/backups/kios_llm_action_sequence/out/drawer/2.json +++ /dev/null @@ -1,49 +0,0 @@ -{ - "task_cohesion": { - "task_sequence": [ - "move_hand()", - "grasp_object()", - "close_by_slide()", - "release_object()" - ], - "step_instructions": [ - "move the hand near the drawer handle", - "grasp the drawer handle", - "move the drawer handle to close the drawer fully", - "release the drawer handle" - ], - "object_name": "" - }, - "environment_before": { - "assets": [ - "", - "" - ], - "asset_states": { - "": "on_something()" - }, - "objects": [ - "" - ], - "object_states": { - "": "open()" - } - }, - "environment_after": { - "assets": [ - "", - "" - ], - "asset_states": { - "": "on_something()" - }, - "objects": [ - "" - ], - "object_states": { - "": "closed()" - } - }, - "instruction_summary": "close the drawer fully", - "question": "" -} \ No newline at end of file diff --git a/kios_bt_planning/backups/kios_llm_action_sequence/out/fridge/0.json b/kios_bt_planning/backups/kios_llm_action_sequence/out/fridge/0.json deleted file mode 100644 index cf4c0d36..00000000 --- a/kios_bt_planning/backups/kios_llm_action_sequence/out/fridge/0.json +++ /dev/null @@ -1,53 +0,0 @@ -{ - "task_cohesion": { - "task_sequence": [ - "move_hand()", - "grasp_object()", - "open_by_rotate()", - "release_object()" - ], - "step_instructions": [ - "move the hand near the fridge handle", - "grasp the handle", - "open the fridge halfway", - "release the handle" - ], - "object_name": "" - }, - "environment_before": { - "assets": [ - "", - "" - ], - "asset_states": { - "": "on_something()" - }, - "objects": [ - "", - "" - ], - "object_states": { - "": "closed()", - "": "inside_something()" - } - }, - "environment_after": { - "assets": [ - "", - "" - ], - "asset_states": { - "": "on_something()" - }, - "objects": [ - "", - "" - ], - "object_states": { - "": "open()", - "": "inside_something()" - } - }, - "instruction_summary": "open the fridge halfway", - "question": "" -} \ No newline at end of file diff --git a/kios_bt_planning/backups/kios_llm_action_sequence/out/fridge/1.json b/kios_bt_planning/backups/kios_llm_action_sequence/out/fridge/1.json deleted file mode 100644 index cb3a4afb..00000000 --- a/kios_bt_planning/backups/kios_llm_action_sequence/out/fridge/1.json +++ /dev/null @@ -1,53 +0,0 @@ -{ - "task_cohesion": { - "task_sequence": [ - "move_hand()", - "grasp_object()", - "adjust_by_rotate()", - "release_object()" - ], - "step_instructions": [ - "move the hand near the fridge handle", - "grasp the handle", - "open the fridge wider", - "release the handle" - ], - "object_name": "" - }, - "environment_before": { - "assets": [ - "", - "" - ], - "asset_states": { - "": "on_something()" - }, - "objects": [ - "", - "" - ], - "object_states": { - "": "open()", - "": "inside_something()" - } - }, - "environment_after": { - "assets": [ - "", - "" - ], - "asset_states": { - "": "on_something()" - }, - "objects": [ - "", - "" - ], - "object_states": { - "": "open()", - "": "inside_something()" - } - }, - "instruction_summary": "open the fridge wider", - "question": "" -} \ No newline at end of file diff --git a/kios_bt_planning/backups/kios_llm_action_sequence/out/fridge/2.json b/kios_bt_planning/backups/kios_llm_action_sequence/out/fridge/2.json deleted file mode 100644 index cc7bdfa2..00000000 --- a/kios_bt_planning/backups/kios_llm_action_sequence/out/fridge/2.json +++ /dev/null @@ -1,61 +0,0 @@ -{ - "task_cohesion": { - "task_sequence": [ - "move_hand()", - "grasp_object()", - "detach_from_plane()", - "move_object()", - "move_hand()", - "move_object()", - "attach_to_plane()", - "release_object()" - ], - "step_instructions": [ - "move the hand near the juice", - "grasp the juice", - "pick up the juice from the top of the fridge", - "move the juice to the floor", - "move the hand near the juice", - "move the juice to the floor", - "place the juice on the floor", - "release the juice" - ], - "object_name": "" - }, - "environment_before": { - "assets": [ - "", - "" - ], - "asset_states": { - "": "on_something()" - }, - "objects": [ - "", - "" - ], - "object_states": { - "": "open()", - "": "inside_something()" - } - }, - "environment_after": { - "assets": [ - "", - "" - ], - "asset_states": { - "": "on_something()" - }, - "objects": [ - "", - "" - ], - "object_states": { - "": "open()", - "": "on_something()" - } - }, - "instruction_summary": "take the juice from the fridge and put it on the floor", - "question": "" -} \ No newline at end of file diff --git a/kios_bt_planning/backups/kios_llm_action_sequence/out/fridge/3.json b/kios_bt_planning/backups/kios_llm_action_sequence/out/fridge/3.json deleted file mode 100644 index 9b483592..00000000 --- a/kios_bt_planning/backups/kios_llm_action_sequence/out/fridge/3.json +++ /dev/null @@ -1,53 +0,0 @@ -{ - "task_cohesion": { - "task_sequence": [ - "move_hand()", - "grasp_object()", - "move_object()", - "release_object()" - ], - "step_instructions": [ - "move the hand near the carton box", - "grasp the carton box", - "move the carton box away from the fridge", - "release the carton box" - ], - "object_name": "" - }, - "environment_before": { - "assets": [ - "", - "" - ], - "asset_states": { - "": "on_something()" - }, - "objects": [ - "", - "" - ], - "object_states": { - "": "open()", - "": "on_something()" - } - }, - "environment_after": { - "assets": [ - "", - "" - ], - "asset_states": { - "": "on_something()" - }, - "objects": [ - "", - "" - ], - "object_states": { - "": "open()", - "": "on_something()" - } - }, - "instruction_summary": "move the carton box away from the fridge", - "question": "" -} \ No newline at end of file diff --git a/kios_bt_planning/backups/kios_llm_action_sequence/out/shelf/0.json b/kios_bt_planning/backups/kios_llm_action_sequence/out/shelf/0.json deleted file mode 100644 index 16bc759e..00000000 --- a/kios_bt_planning/backups/kios_llm_action_sequence/out/shelf/0.json +++ /dev/null @@ -1,67 +0,0 @@ -{ - "task_cohesion": { - "task_sequence": [ - "move_hand()", - "grasp_object()", - "detach_from_plane()", - "move_object()", - "attach_to_plane()", - "open_by_rotate()", - "release_object()" - ], - "step_instructions": [ - "move the hand near the spam", - "grasp the spam", - "pick up the spam", - "bring the spam above the trash bin", - "attach the spam to the trash bin", - "open the spam to check the expiration date", - "release the spam" - ], - "object_name": "" - }, - "environment_before": { - "assets": [ - "
", - "", - "", - "", - "" - ], - "asset_states": { - "": "on_something(
)", - "": "on_something()" - }, - "objects": [ - "", - "" - ], - "object_states": { - "": "on_something(
)", - "": "on_something()" - } - }, - "environment_after": { - "assets": [ - "
", - "", - "", - "", - "" - ], - "asset_states": { - "": "on_something(
)", - "": "on_something()" - }, - "objects": [ - "", - "" - ], - "object_states": { - "": "inside_something()", - "": "on_something()" - } - }, - "instruction_summary": "take the spam and throw it away if the out-of-date date is expired, otherwise put it on the shelf", - "question": "" -} \ No newline at end of file diff --git a/kios_bt_planning/backups/kios_llm_action_sequence/out/shelf/1.json b/kios_bt_planning/backups/kios_llm_action_sequence/out/shelf/1.json deleted file mode 100644 index c26e27b8..00000000 --- a/kios_bt_planning/backups/kios_llm_action_sequence/out/shelf/1.json +++ /dev/null @@ -1,63 +0,0 @@ -{ - "task_cohesion": { - "task_sequence": [ - "move_hand()", - "grasp_object()", - "detach_from_plane()", - "move_object()", - "release_object()" - ], - "step_instructions": [ - "move your hand near the spam", - "grasp the spam", - "pick up the spam", - "bring the spam above the trash bin", - "release the spam to drop it in the trash bin" - ], - "object_name": "" - }, - "environment_before": { - "assets": [ - "
", - "", - "", - "", - "" - ], - "asset_states": { - "": "on_something(
)", - "": "on_something()" - }, - "objects": [ - "", - "" - ], - "object_states": { - "": "on_something(
)", - "": "on_something()" - } - }, - "environment_after": { - "assets": [ - "
", - "", - "", - "", - "" - ], - "asset_states": { - "": "on_something(
)", - "": "on_something()" - }, - "objects": [ - "", - "" - ], - "object_states": { - "": "inside_something()", - "": "on_something()" - } - }, - "instruction_summary": "throw away the spam", - "question": "" -} \ No newline at end of file diff --git a/kios_bt_planning/backups/kios_llm_action_sequence/out/shelf/2.json b/kios_bt_planning/backups/kios_llm_action_sequence/out/shelf/2.json deleted file mode 100644 index 3b00dcb0..00000000 --- a/kios_bt_planning/backups/kios_llm_action_sequence/out/shelf/2.json +++ /dev/null @@ -1,69 +0,0 @@ -{ - "task_cohesion": { - "task_sequence": [ - "move_hand()", - "grasp_object()", - "detach_from_plane()", - "move_object()", - "move_object()", - "move_object()", - "attach_to_plane()", - "release_object()" - ], - "step_instructions": [ - "move the hand near the juice", - "grasp the juice", - "pick up the juice", - "take the juice from the top of the shelf", - "move the juice downward", - "move the juice above the table", - "place the juice", - "release the juice" - ], - "object_name": "" - }, - "environment_before": { - "assets": [ - "
", - "", - "", - "", - "" - ], - "asset_states": { - "": "on_something(
)", - "": "on_something()" - }, - "objects": [ - "", - "" - ], - "object_states": { - "": "inside_something()", - "": "on_something()" - } - }, - "environment_after": { - "assets": [ - "
", - "", - "", - "", - "" - ], - "asset_states": { - "": "on_something(
)", - "": "on_something()" - }, - "objects": [ - "", - "" - ], - "object_states": { - "": "inside_something()", - "": "on_something(
)" - } - }, - "instruction_summary": "move the juice on top of the table", - "question": "" -} \ No newline at end of file diff --git a/kios_bt_planning/backups/kios_llm_action_sequence/out/shelf/3.json b/kios_bt_planning/backups/kios_llm_action_sequence/out/shelf/3.json deleted file mode 100644 index 92a955b4..00000000 --- a/kios_bt_planning/backups/kios_llm_action_sequence/out/shelf/3.json +++ /dev/null @@ -1,63 +0,0 @@ -{ - "task_cohesion": { - "task_sequence": [ - "move_hand()", - "grasp_object()", - "detach_from_plane()", - "move_object()", - "release_object()" - ], - "step_instructions": [ - "move your hand near the juice", - "grasp the juice", - "pick up the juice", - "bring the juice above the trash bin", - "release the juice to drop it in the trash bin" - ], - "object_name": "" - }, - "environment_before": { - "assets": [ - "
", - "", - "", - "", - "" - ], - "asset_states": { - "": "on_something(
)", - "": "on_something()" - }, - "objects": [ - "", - "" - ], - "object_states": { - "": "inside_something()", - "": "on_something(
)" - } - }, - "environment_after": { - "assets": [ - "
", - "", - "", - "", - "" - ], - "asset_states": { - "": "on_something(
)", - "": "on_something()" - }, - "objects": [ - "", - "" - ], - "object_states": { - "": "inside_something()", - "": "inside_something()" - } - }, - "instruction_summary": "throw away the juice", - "question": "" -} \ No newline at end of file diff --git a/kios_bt_planning/backups/kios_llm_action_sequence/out/table/0.json b/kios_bt_planning/backups/kios_llm_action_sequence/out/table/0.json deleted file mode 100644 index ddf8b608..00000000 --- a/kios_bt_planning/backups/kios_llm_action_sequence/out/table/0.json +++ /dev/null @@ -1,61 +0,0 @@ -{ - "task_cohesion": { - "task_sequence": [ - "move_hand()", - "grasp_object()", - "detach_from_plane()", - "move_object()", - "move_object()", - "attach_to_plane()", - "release_object()" - ], - "step_instructions": [ - "move the hand near the sponge", - "grasp the sponge", - "pick up the sponge", - "take the sponge from the first table", - "move the sponge to the second table", - "place the sponge on the second table", - "release the sponge" - ], - "object_name": "" - }, - "environment_before": { - "assets": [ - "", - "", - "", - "" - ], - "asset_states": { - "": "next_to()", - "": "on_something()" - }, - "objects": [ - "" - ], - "object_states": { - "": "on_something()" - } - }, - "environment_after": { - "assets": [ - "", - "", - "", - "" - ], - "asset_states": { - "": "next_to()", - "": "on_something()" - }, - "objects": [ - "" - ], - "object_states": { - "": "on_something()" - } - }, - "instruction_summary": "put the sponge on the table2", - "question": "" -} \ No newline at end of file diff --git a/kios_bt_planning/backups/kios_llm_action_sequence/out/table/1.json b/kios_bt_planning/backups/kios_llm_action_sequence/out/table/1.json deleted file mode 100644 index 4b1f7029..00000000 --- a/kios_bt_planning/backups/kios_llm_action_sequence/out/table/1.json +++ /dev/null @@ -1,55 +0,0 @@ -{ - "task_cohesion": { - "task_sequence": [ - "move_hand()", - "grasp_object()", - "wipe_on_plane()", - "release_object()" - ], - "step_instructions": [ - "move the hand near the sponge on the second table", - "grasp the sponge", - "wipe the second table with the sponge", - "release the sponge" - ], - "object_name": "" - }, - "environment_before": { - "assets": [ - "", - "", - "", - "" - ], - "asset_states": { - "": "next_to()", - "": "on_something()" - }, - "objects": [ - "" - ], - "object_states": { - "": "on_something()" - } - }, - "environment_after": { - "assets": [ - "", - "", - "", - "" - ], - "asset_states": { - "": "next_to()", - "": "on_something()" - }, - "objects": [ - "" - ], - "object_states": { - "": "on_something()" - } - }, - "instruction_summary": "wipe the table2 with the sponge", - "question": "" -} \ No newline at end of file diff --git a/kios_bt_planning/backups/kios_llm_action_sequence/out/window/0.json b/kios_bt_planning/backups/kios_llm_action_sequence/out/window/0.json deleted file mode 100644 index e8172cb3..00000000 --- a/kios_bt_planning/backups/kios_llm_action_sequence/out/window/0.json +++ /dev/null @@ -1,67 +0,0 @@ -{ - "task_cohesion": { - "task_sequence": [ - "move_hand()", - "grasp_object()", - "detach_from_plane()", - "move_object()", - "attach_to_plane()", - "wipe_on_plane()", - "detach_from_plane()", - "move_object()", - "attach_to_plane()", - "release_object()" - ], - "step_instructions": [ - "move the hand near the sponge", - "grasp the sponge", - "pick up the sponge", - "bring the sponge above the window", - "attach the sponge to the window", - "wipe the window with the sponge", - "detach the sponge from the window", - "bring the sponge above the table", - "put down the sponge", - "release the sponge" - ], - "object_name": "" - }, - "environment_before": { - "assets": [ - "
", - "", - "", - "" - ], - "asset_states": { - "
": "next_to()", - "": "on_something()" - }, - "objects": [ - "" - ], - "object_states": { - "": "on_something(
)" - } - }, - "environment_after": { - "assets": [ - "
", - "", - "", - "" - ], - "asset_states": { - "
": "next_to()", - "": "on_something()" - }, - "objects": [ - "" - ], - "object_states": { - "": "on_something(
)" - } - }, - "instruction_summary": "get the sponge from the table, wipe the window with it, and put the sponge back on the table", - "question": "" -} \ No newline at end of file diff --git a/kios_bt_planning/backups/kios_llm_action_sequence/out/window/1.json b/kios_bt_planning/backups/kios_llm_action_sequence/out/window/1.json deleted file mode 100644 index 6b660c25..00000000 --- a/kios_bt_planning/backups/kios_llm_action_sequence/out/window/1.json +++ /dev/null @@ -1,57 +0,0 @@ -{ - "task_cohesion": { - "task_sequence": [ - "move_hand()", - "grasp_object()", - "detach_from_plane()", - "move_object()", - "release_object()" - ], - "step_instructions": [ - "move the hand near the sponge", - "grasp the sponge", - "pick up the sponge", - "bring the sponge above the trash bin", - "release the sponge and drop it in the trash bin" - ], - "object_name": "" - }, - "environment_before": { - "assets": [ - "
", - "", - "", - "" - ], - "asset_states": { - "
": "next_to()", - "": "on_something()" - }, - "objects": [ - "" - ], - "object_states": { - "": "on_something(
)" - } - }, - "environment_after": { - "assets": [ - "
", - "", - "", - "" - ], - "asset_states": { - "
": "next_to()", - "": "on_something()" - }, - "objects": [ - "" - ], - "object_states": { - "": "inside_something()" - } - }, - "instruction_summary": "throw away the sponge", - "question": "" -} \ No newline at end of file diff --git a/kios_bt_planning/backups/kios_llm_action_sequence/prompt/prompt_bt.txt b/kios_bt_planning/backups/kios_llm_action_sequence/prompt/prompt_bt.txt deleted file mode 100644 index e69de29b..00000000 diff --git a/kios_bt_planning/backups/kios_llm_action_sequence/prompt/prompt_domain.txt b/kios_bt_planning/backups/kios_llm_action_sequence/prompt/prompt_domain.txt deleted file mode 100644 index 2842c621..00000000 --- a/kios_bt_planning/backups/kios_llm_action_sequence/prompt/prompt_domain.txt +++ /dev/null @@ -1,89 +0,0 @@ -[user] -Following is the domain knowledge given as pddl domain file, which includes: -- the types of objects in the world, -- the predicates that can be used to describe the states of the world, -- the actions that you can apply, including their precondtions and effects. -""" -(define (domain robot_assembly_problem-domain) - (:requirements :strips :typing) - (:types - thing - object - tool part hand - thing - ) - (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part)) - (:action pickup - :parameters ( ?hand - hand ?part - part ?tool - tool) - :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part)) - :effect (and (hold ?tool ?part) (not (is_free ?tool)))) - (:action putdown - :parameters ( ?hand - hand ?tool - tool ?part - part) - :precondition (and (hold ?tool ?part) (hold ?hand ?tool)) - :effect (and (not (hold ?hand ?part)) (is_free ?tool))) - (:action place - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2))) - (:action detach - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2)))) - (:action insert - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2))) - (:action pull - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2)))) - (:action screw - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2))) - (:action unscrew - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2)))) - (:action load_tool - :parameters ( ?hand - hand ?tool - tool) - :precondition (and (is_equippable ?tool) (is_free ?hand)) - :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool))) - (:action unload_tool - :parameters ( ?hand - hand ?tool - tool) - :precondition (and (hold ?hand ?tool) (is_free ?tool)) - :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool)))) -) -""" -The precidates are explained as follows: -""" -"PREDICATE_LIST" -- is_free(): is free and can hold something -- is_equippable(): is ready to be eqquipped by a robot hand -- can_manipulate(, ): can be used to manipulate -- can_insert_to(, ): can be inserted into (insertion assembly constraint) -- can_screw_to(, ): can be screwed into (screw assembly constraint) -- can_place_to(, ): can be placed into (placement assembly constraint) -- hold(, ): is being held by -- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) -- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) -- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) -""" - -The actions are explained as follows: -""" -"ACTION_LIST" -- pickup(, , ): use to pickup -- putdown(, , ): use to putdown -- place(, , , ): use to place to (to satisfy the placement assembly constraint) -- detach(, , , ): use to detach from (to lift the existing placement assembly constraint) -- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint) -- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint) -- screw(, , , ): use to screw into (to satisfy the screw assembly constraint) -- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint) -- load_tool(, ): equips to manipulate other objects -- unload_tool(, ): unloads to manipulate other objects -""" -the items in <> indicate the type of the instances needed for the predicates and actions. -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. diff --git a/kios_bt_planning/backups/kios_llm_action_sequence/prompt/prompt_environment.txt b/kios_bt_planning/backups/kios_llm_action_sequence/prompt/prompt_environment.txt deleted file mode 100644 index ac8293ab..00000000 --- a/kios_bt_planning/backups/kios_llm_action_sequence/prompt/prompt_environment.txt +++ /dev/null @@ -1,28 +0,0 @@ -[user] -Information about world states are given as python dictionary. Example: -""" -{"environment":{"assets":["
", "", "", "", ""], - "asset_states": {"": "on_something(
)", - "": "on_something()"}, - "objects": ["", ""], - "object_states": {"": "on_something(
)", - "": "on_something()"}}} -""" -Asset states and object states are represented using those state sets: -""" -"STATE LIST" -- on_something(): Object is located on -- inside_something(): Object is located inside -- inside_hand(): Object is being grasped by a robot hand -- closed(): Object can be opened -- open(): Object can be closed or kept opened -- can_manipulate(): Object can be used to grasp and manipulate -- equipped(): Object is being equipped by the robot hand. -- screwed_into(): Object is screwed into by robot hand -- inserted_into(): Object is inserted into by robot hand -""" - should be one of the assets or objects in the environment. -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/kios_bt_planning/backups/kios_llm_action_sequence/prompt/prompt_example.txt b/kios_bt_planning/backups/kios_llm_action_sequence/prompt/prompt_example.txt deleted file mode 100644 index 8d71b9b6..00000000 --- a/kios_bt_planning/backups/kios_llm_action_sequence/prompt/prompt_example.txt +++ /dev/null @@ -1,232 +0,0 @@ -[user] -I will give you some examples of the input and the output you will generate. -Example 1: -""" -- Input: -{"assets":["
", "", "", "", ""], -"asset_states": {"": "on_something(
)", - "": "on_something()"}, -"objects": ["", ""], -"object_states": {"": "on_something(
)", - "": "on_something()"}, -"instruction": "Put the juice on top of the shelf"} -- Output: -``` -{"task_cohesion": { - "task_sequence": [ - "move_hand()", - "grasp_object()", - "detach_from_plane()", - "move_object()", - "move_object()", - "move_object()", - "attach_to_plane()", - "release_object()" - ], - "step_instructions": [ - "move the hand near the juice", - "grasp the juice", - "pick up the juice", - "take the juice from the bottom of the shelf", - "move the juice upward", - "move the juice above the upper shelf", - "place the juice", - "release the juice" - ], - "object_name": ""}, -"environment_before": {"assets":["
", "", "", "", ""], - "asset_states": {"": "on_something(
)", - "": "on_something()"}, - "objects": ["", ""], - "object_states": {"": "on_something(
)", - "": "on_something()"}}, -"environment_after": {"assets":["
", "", "", "", ""], - "asset_states": {"": "on_something(
)", - "": "on_something()"}, - "objects": ["", ""], - "object_states": {"": "on_something(
)", - "": "on_something()"}}, -"instruction_summary": "put the juice on top of the shelf", -"question":""} -``` -""" -Example 2: -""" -- Input: -{"assets":["
", "", "", "", ""], -"asset_states": {"": "on_something(
)", - "": "on_something()"}, -"objects": ["", ""], -"object_states": {"": "on_something(
)", - "": "on_something()"}, -"instruction": "Throw away the spam into the trash bin."} -- Output: -``` -{"task_cohesion": { - "task_sequence": [ - "move_hand()", - "grasp_object()", - "detach_from_plane()", - "move_object()", - "release_object()" - ], - "step_instructions": [ - "move your hand near the spam", - "grasp the spam", - "pick up the spam", - "bring the spam above the trash bin", - "release the spam to drop it in the trash bin" - ], - "object_name": ""}, -"environment_before": {"assets":["
", "", "", "", ""], - "asset_states": {"": "on_something(
)", - "": "on_something()"}, - "objects": ["", ""], - "object_states": {"": "on_something(
)", - "": "on_something()"}}, -"environment_after": {"assets":["
", "", "", "", ""], - "asset_states": {"": "on_something(
)", - "": "on_something()"}, - "objects": ["", ""], - "object_states": {"": "inside_something()", - "": "on_something()"}}, -"instruction_summary": "throw away the spam", -"question":""} -``` -""" -Example 3: -""" --Input: -{"assets":["", ""], -"asset_states": {"": "on_something()"}, -"objects": [""], -"object_states": {"": "closed()"}, -"instruction": "Open the fridge."} -- Output: -``` -{"task_cohesion": { - "task_sequence": [ - "move_hand()", - "grasp_object()", - "open_by_rotate()", - "release_object()" - ], - "step_instructions": [ - "move the hand near the fridge handle", - "grasp the handle", - "open the fridge by pulling the handle", - "release the handle" - ], - "object_name": ""}, -"environment_before": {"assets":["", ""], - "asset_states": {"": "on_something()"}, - "objects": [""], - "object_states": {"": "closed()"}}, -"environment_after": {"assets":["", ""], - "asset_states": {"": "on_something()"}, - "objects": [""], - "object_states": {"": "open()"}}, -"instruction_summary": "open a fridge", -"question":""} -``` -""" -Example 4: -""" --Input: -{"assets":["
", ""], -"asset_states": {"
": "next_to()"}, -"objects": [""], -"object_states": {"": "on_something(
)"}, -"instruction": "Wipe the window with the sponge. Then, put the sponge on the table."} -- Output: -``` -{"task_cohesion": { - "task_sequence": [ - "move_hand()", - "grasp_object()", - "detach_from_plane()", - "move_object()", - "attach_to_plane()", - "wipe_on_plane()", - "detach_from_plane()", - "move_object()", - "attach_to_plane()", - "release_object()" - ], - "step_instructions": [ - "move the hand near the sponge", - "grasp the sponge", - "pick up the sponge", - "bring the sponge above the window", - "attach the sponge to the window", - "wipe the window with the sponge", - "detach the sponge from the window", - "bring the sponge above the table", - "put down the sponge", - "release the sponge" - ], - "object_name": ""}, -"environment_before": {"assets":["
", ""], - "asset_states": {"
": "next_to()"}, - "objects": [""], - "object_states": {"": "on_something(
)"},} -"environment_after": {"assets":["
", ""], - "asset_states": {"
": "next_to()"}, - "objects": [""], - "object_states": {"": "on_something(
)"}}, -"instruction_summary": "wipe the window with the sponge, then put the sponge on the table", -"question":""} -``` -""" -Example 5: -""" --Input: -{"assets":["
"], -"asset_states": {"": "can_manipulate()"}, -"objects": ["", "", ""], -"object_states": {"": "on_something(
)", "": "on_something(
), "": "can_manipulate()"}, -"instruction": "screw the nut into the bolt. Tool should be used and be put back at the end."} -- Output: -``` -{"task_cohesion": { - "task_sequence": [ - "load_tool()", - "approach()", - "grasp_object()", - "pick_up()", - "approach()", - "insert()", - "screw()", - "release_object()", - "unload_tool()" - ], - "step_instructions": [ - "load the parallel_toolbox", - "move the hand close to the nut", - "grasp the nut", - "pick up the nut", - "move the hand close to the bolt", - "insert the nut onto the bolt", - "screw the nut into the bold", - "release the nut", - "unload the parallel_toolbox" - ], - "object_name": ""}, -"environment_before": {"assets":["
"], - "asset_states": {"": "can_manipulate()"}, - "objects": ["", "", ""], - "object_states": {"": "on_something(
)", "": "on_something(
)"},} -"environment_after": {"assets":["
"], - "asset_states": {"": "can_manipulate()"}, - "objects": ["", "", ""], - "object_states": {"": "screwed_into()", "": "on_something(
)"}, -"instruction_summary": "screw the nut into the bolt. Tool should be used and be put back at the end.", -"question":""} -} -``` -""" -From these examples, learn that some robotic actions have dependencies with the actions before and after them. -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/kios_bt_planning/backups/kios_llm_action_sequence/prompt/prompt_function.txt b/kios_bt_planning/backups/kios_llm_action_sequence/prompt/prompt_function.txt deleted file mode 100644 index a5e4fdd9..00000000 --- a/kios_bt_planning/backups/kios_llm_action_sequence/prompt/prompt_function.txt +++ /dev/null @@ -1,25 +0,0 @@ -[user] -Necessary and sufficient robot actions are defined as follows: -""" -"ROBOT ACTION LIST" -- load_tool(): Equip the robot hand with a toolbox in order to manipulate some objects later. This action can only be performed when no tool is being equipped. -- unload_tool(): This action can only be performed when a tool is being equipped by the robot hand. The opposite action of load_tool(). -- move_hand(): Move the robot hand from one position to another with/without grasping an object. -- approach(): Move the robot hand to get close to an object. -- grasp_object(): Grab an object with robot hand. The robot hand must equips the proper toolbox for manipulating the object. -- release_object(): Release an object in the robot hand when the robot is grasping something. -- move_object(): Move the object grabbed by the robot hand from one position to another. move_object() is allowed only when the object is not physically constrained by the environment. For example, if the robot hand moves an object on the floor to another location, move_object() is not allowed because the object is constrained by the floor. -- pick_up(): This action can only be performed if an object is grabbed. Move the grabbed object from a state in which it is constrained by a plane to a state in which it is not constrained by any plane. For example, pick_up() is used when a robot hand picks up an object on a table. -- put_down(): This action can only be performed if an object is grabbed. The opposite operation of pick_up(). -- insert(): This action can only be performed if an object is in hand. Insert the grasped object into a hole of another object. -- screw(): This action can only be performed if an object is grabbed and has been inserted into the hole. Screw the grabbed object into the hole or onto a bolt-like structure of another obejct. This can only be conducted when the object and the hole has an screw thread assembly constraint. -- adjust_by_rotate(): This action can only be performed if an object is grabbed. Rotate an object that is rotationally constrained by its environment along its rotation. For example, when adjusting the temperature of a refrigerator, the temperature knob makes this motion. -- close_by_rotate(): This action can only be performed if an object is grabbed. The opposite operation of open_by_rotate(). -- open_by_slide(): This action can only be performed if an object is grabbed. Moves an object that is translationally constrained in two axes from its environment along one unbounded axis. For example, when opening a sliding door or drawer, the handle makes this movement. -- adjust_by_slide(): This action can only be performed if an object is grabbed. Slide an object that is translationally constrained in two axes from its environment along one unbounded axis. For example, when widen the gap between a sliding door and the wall, the handle makes this movement. -- close_by_slide(): This action can only be performed if an object is grabbed. The opposite operation of open_by_slide(). -""" -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/kios_bt_planning/backups/kios_llm_action_sequence/prompt/prompt_output_format.txt b/kios_bt_planning/backups/kios_llm_action_sequence/prompt/prompt_output_format.txt deleted file mode 100644 index 7be3d38b..00000000 --- a/kios_bt_planning/backups/kios_llm_action_sequence/prompt/prompt_output_format.txt +++ /dev/null @@ -1,20 +0,0 @@ -[user] -You divide the actions given in the text into detailed robot actions and put them together as a python dictionary. -The dictionary has five keys. -""" -- dictionary["task_cohesion"]: A dictionary containing information about the robot's actions that have been split up. -- dictionary["environment_before"]: The state of the environment before the manipulation. -- dictionary["environment_after"]: The state of the environment after the manipulation. -- dictionary["instruction_summary"]: contains a brief summary of the given sentence. -- dictionary["question"]: If you cannot understand the given sentence, you can ask the user to rephrase the sentence. Leave this key empty if you can understand the given sentence. -""" -Three keys exist in dictionary["task_cohesion"]. -""" -- dictionary["task_cohesion"]["task_sequence"]: Contains a list of robot actions. Only the behaviors defined in the "ROBOT ACTION LIST" will be used. -- dictionary["task_cohesion"]["step_instructions"]: contains a list of instructions corresponding to dictionary["task_cohesion"]["task_sequence"]. -- dictionary["task_cohesion"]["object_name"]: The name of the manipulated object. Only objects defined in the input dictionary will be used for the object name. -""" -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/kios_bt_planning/backups/kios_llm_action_sequence/prompt/prompt_role.txt b/kios_bt_planning/backups/kios_llm_action_sequence/prompt/prompt_role.txt deleted file mode 100644 index 06feafda..00000000 --- a/kios_bt_planning/backups/kios_llm_action_sequence/prompt/prompt_role.txt +++ /dev/null @@ -1,5 +0,0 @@ -[user] -You are an excellent interpreter of human instructions for robotic assembly tasks. Given an instruction and information about the working environment, you break it down into a sequence of robotic actions. -Please do not begin working until I say "Start working." Instead, simply output the message "Waiting for next input." Understood? -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/kios_bt_planning/backups/kios_llm_action_sequence/query/query.txt b/kios_bt_planning/backups/kios_llm_action_sequence/query/query.txt deleted file mode 100644 index ab5f24f2..00000000 --- a/kios_bt_planning/backups/kios_llm_action_sequence/query/query.txt +++ /dev/null @@ -1,27 +0,0 @@ -Start working. Resume from the environment below. -""" -{"environment":[ENVIRONMENT]} -""" -The instruction is as follows: -""" -{"instruction": [INSTRUCTION]} -""" -The dictionary that you return should be formatted as python dictionary. Follow these rules: -1. The first element should be move_hand() to move the robot hand closer to the object. Always end with releasing the object. -2. Make sure that each element of the ["step_instructions"] explains corresponding element of the ["task_sequence"]. Refer to the "ROBOT ACTION LIST" to understand the elements of ["task_sequence"]. -3, The length of the ["step_instructions"] list must be the same as the length of the ["task_sequence"] list. -4. Never left ',' at the end of the list. -5. Keep track of all items listed in the "objects" section of the "environment_before" field. Please ensure that you fill out both the "objects" and "object_states" sections for all listed items. -6. Use the "STATE LIST" for the "object states" field. -""" -"STATE LIST" -- on_something(): Object is located on -- inside_something(): Object is located inside -- inside_hand(): Object is being grasped by a robot hand -- closed(): Object can be opened -- open(): Object can be closed or kept opened -""" -7. All keys of the dictionary should be double-quoted. -8. Insert ``` at the beginning and the end of the dictionary to separate it from the rest of your response. -9. Make sure that you output a consistent manipultation as a single arm robot. For example, grasping an object should not occur in successive steps. -Adhere to the output format I defined above. Follow the nine rules. Think step by step. \ No newline at end of file diff --git a/kios_bt_planning/backups/kios_llm_action_sequence/system/system.txt b/kios_bt_planning/backups/kios_llm_action_sequence/system/system.txt deleted file mode 100644 index 4cef77dd..00000000 --- a/kios_bt_planning/backups/kios_llm_action_sequence/system/system.txt +++ /dev/null @@ -1 +0,0 @@ -You are an excellent interpreter of instructions for robotic assembly tasks. Given an instruction and information about the working environment, you break it down into a sequence of robotic actions. \ No newline at end of file diff --git a/kios_bt_planning/backups/kios_planner/__init__.py b/kios_bt_planning/backups/kios_planner/__init__.py deleted file mode 100644 index ad97f726..00000000 --- a/kios_bt_planning/backups/kios_planner/__init__.py +++ /dev/null @@ -1,3 +0,0 @@ -from colorama import Fore - -print(Fore.RED + "THIS MODULE IS CURRENTLY DISCARDED! SEE KIOS_DOMAIN INSTEAD!") diff --git a/kios_bt_planning/backups/kios_planner/domain.py b/kios_bt_planning/backups/kios_planner/domain.py deleted file mode 100644 index 99033010..00000000 --- a/kios_bt_planning/backups/kios_planner/domain.py +++ /dev/null @@ -1,292 +0,0 @@ -from unified_planning.shortcuts import * - -######################################## Type ######################################## -Location = UserType("Location") -Thing = UserType("Thing") -Tool = UserType("Tool", father=Thing) -Part = UserType("Part", father=Thing) - -######################################## Fluent ######################################## -reachable = unified_planning.model.Fluent( - "reachable", BoolType(), l_from=Location, l_to=Location -) -robot_at = unified_planning.model.Fluent("robot_at", BoolType(), l=Location) -thing_at = unified_planning.model.Fluent("thing_at", BoolType(), t=Thing, l=Location) - -in_tool = Fluent("in_tool", p=Part, tool=Tool) -in_hand = Fluent("in_hand", BoolType(), tool=Tool) - -interactive = Fluent("interactive", BoolType(), p=Part, tool=Tool) - -inserted = Fluent("inserted", BoolType(), p=Part) -screwed = Fluent("screwed", BoolType(), p=Part) -placed = Fluent("placed", BoolType(), p=Part) - -######################################## Action ######################################## - -# * move -move = InstantaneousAction("move", l_from=Location, l_to=Location) - -l_from = move.parameter("l_from") -l_to = move.parameter("l_to") - -move.add_precondition(robot_at(l_from)) -move.add_precondition(reachable(l_from, l_to)) - -move.add_effect(robot_at(l_from), False) -move.add_effect(robot_at(l_to), True) - -# * load_tool -load_tool = unified_planning.model.InstantaneousAction( - "load_tool", - tool=Tool, -) -tool = load_tool.parameter("tool") - -tool_var = Variable("tool_var", Tool) -load_tool.add_precondition(Not(Exists(in_hand(tool_var)), tool_var)) - -load_tool.add_effect(in_hand(tool), True) - -# * unload_tool -unload_tool = unified_planning.model.InstantaneousAction("unload_tool", tool=Tool) -tool = unload_tool.parameter("tool") - -unload_tool.add_precondition(in_hand(tool)) -p_var = Variable("p_var", Part) -unload_tool.add_precondition(Not(Exists(in_tool(p_var, tool), p_var))) - -unload_tool.add_effect(in_hand(tool), False) - -# * pick -pick = unified_planning.model.InstantaneousAction( - "pick", p=Part, tool=Tool, l_from=Location, l_to=Location -) -p = pick.parameter("p") -tool = pick.parameter("tool") -l_from = pick.parameter("l_from") -l_to = pick.parameter("l_to") - -pick.add_precondition(robot_at(l_from)) -pick.add_precondition(reachable(l_from, l_to)) -pick.add_precondition(thing_at(p, l_to)) -pick.add_precondition(in_hand(tool)) -p_var = Variable("p_var", Part) -pick.add_precondition(Forall(Not(in_tool(p_var, tool)), p_var)) -pick.add_precondition(interactive(p, tool)) - -pick.add_effect(robot_at(l_from), False) -pick.add_effect(robot_at(l_to), True) -pick.add_effect(thing_at(p, l_to), False) -pick.add_effect(in_tool(p, tool), True) - -# # * place -# place = unified_planning.model.InstantaneousAction( -# "place", p=Part, tool=Tool, l_from=Location, l_to=Location -# ) -# p = place.parameter("p") -# tool = place.parameter("tool") -# l_from = place.parameter("l_from") -# l_to = place.parameter("l_to") - -# place.add_precondition(robot_at(l_from)) -# place.add_precondition(reachable(l_from, l_to)) -# place.add_precondition(in_tool(p, tool)) -# place.add_precondition(in_hand(tool)) - -# place.add_effect(robot_at(l_from), False) -# place.add_effect(robot_at(l_to), True) -# place.add_effect(in_tool(p, tool), False) -# place.add_effect(thing_at(p, l_to), True) - - -# # * push, can be move - - -# # * insert -# insert = unified_planning.model.InstantaneousAction( -# "insert", p=Part, tool=Tool, l_from=Location, l_to=Location -# ) -# p = insert.parameter("p") -# tool = insert.parameter("tool") -# l_from = insert.parameter("l_from") -# l_to = insert.parameter("l_to") - -# insert.add_precondition(reachable(l_from, l_to)) -# insert.add_precondition(robot_at(l_from)) -# insert.add_precondition(in_hand(tool)) -# insert.add_precondition(in_tool(p, tool)) -# # insert.add_precondition(interactive(p, tool)) - -# insert.add_effect(robot_at(l_from), False) -# insert.add_effect(robot_at(l_to), True) -# insert.add_effect(in_tool(p, tool), False) -# insert.add_effect(thing_at(p, l_to), True) -# insert.add_effect(inserted(p), True) - -# # * pull -# pull = unified_planning.model.InstantaneousAction( -# "pull", p=Part, tool=Tool, l_from=Location, l_to=Location -# ) -# p = pull.parameter("p") -# tool = pull.parameter("tool") -# l_from = pull.parameter("l_from") -# l_to = pull.parameter("l_to") - -# pull.add_precondition(reachable(l_from, l_to)) -# pull.add_precondition(robot_at(l_from)) -# pull.add_precondition(thing_at(p, l_to)) -# pull.add_precondition(inserted(p)) -# pull.add_precondition(in_hand(tool)) -# pull.add_precondition(interactive(p, tool)) - -# pull.add_effect(robot_at(l_from), False) -# pull.add_effect(robot_at(l_to), True) -# pull.add_effect(inserted(p), False) -# pull.add_effect(thing_at(p, l_to), False) -# pull.add_effect(in_tool(p, tool), True) - -# # * screw -# screw = unified_planning.model.InstantaneousAction( -# "screw", p=Part, tool=Tool, l_from=Location, l_to=Location -# ) -# p = screw.parameter("p") -# tool = screw.parameter("tool") -# l_from = screw.parameter("l_from") -# l_to = screw.parameter("l_to") - -# screw.add_precondition(reachable(l_from, l_to)) -# screw.add_precondition(robot_at(l_from)) -# screw.add_precondition(in_hand(tool)) -# screw.add_precondition(in_tool(p, tool)) - -# screw.add_effect(robot_at(l_from), False) -# screw.add_effect(robot_at(l_to), True) -# screw.add_effect(in_tool(p, tool), False) -# screw.add_effect(thing_at(p, l_to), True) -# screw.add_effect(screwed(p), True) - -# # * unscrew -# unscrew = unified_planning.model.InstantaneousAction( -# "unscrew", p=Part, tool=Tool, l_from=Location, l_to=Location -# ) -# p = unscrew.parameter("p") -# tool = unscrew.parameter("tool") -# l_from = unscrew.parameter("l_from") -# l_to = unscrew.parameter("l_to") - -# unscrew.add_precondition(reachable(l_from, l_to)) -# unscrew.add_precondition(robot_at(l_from)) -# unscrew.add_precondition(thing_at(p, l_to)) -# unscrew.add_precondition(screwed(p)) -# unscrew.add_precondition(in_hand(tool)) -# unscrew.add_precondition(interactive(p, tool)) - -# unscrew.add_effect(robot_at(l_from), False) -# unscrew.add_effect(robot_at(l_to), True) -# unscrew.add_effect(screwed(p), False) -# unscrew.add_effect(thing_at(p, l_to), False) -# unscrew.add_effect(in_tool(p, tool), True) - -######################################## Problem ######################################## - -problem = Problem("kios_tamp_problem") - -########################## * knowledge - -problem.add_fluent(robot_at, default_initial_value=False) -problem.add_fluent(reachable, default_initial_value=True) -problem.add_fluent(thing_at, default_initial_value=False) -problem.add_fluent(in_hand, default_initial_value=False) -problem.add_fluent(in_tool, default_initial_value=False) -problem.add_fluent(inserted, default_initial_value=False) -problem.add_fluent(screwed, default_initial_value=False) -problem.add_fluent(placed, default_initial_value=False) -problem.add_fluent(interactive, default_initial_value=False) - -problem.add_action(move) -# problem.add_action(pick) -# problem.add_action(place) -# problem.add_action(insert) -# problem.add_action(pull) -# problem.add_action(screw) -# problem.add_action(unscrew) -problem.add_action(load_tool) -problem.add_action(unload_tool) - -######################## * instances -# locations -# NLOC = 10 -# locations = [unified_planning.model.Object("l%s" % i, Location) for i in range(NLOC)] -# problem.add_objects(locations) -# TODO: add locations -# ! for test -location_1 = unified_planning.model.Object("location_1", Location) -location_2 = unified_planning.model.Object("location_2", Location) -location_3 = unified_planning.model.Object("location_3", Location) -location_4 = unified_planning.model.Object("location_4", Location) - -# tools -parallel_box1 = unified_planning.model.Object("parallel_box1", Tool) -parallel_box2 = unified_planning.model.Object("parallel_box2", Tool) -inward_claw = unified_planning.model.Object("inward_claw", Tool) -outward_claw = unified_planning.model.Object("outward_claw", Tool) -no_tool = unified_planning.model.Object("no_tool", Tool) - -# parts -# * chair -leg1 = unified_planning.model.Object("leg1", Part) -leg2 = unified_planning.model.Object("leg2", Part) -nut1 = unified_planning.model.Object("nut1", Part) -nut2 = unified_planning.model.Object("nut2", Part) -seat = unified_planning.model.Object("seat", Part) -back = unified_planning.model.Object("back", Part) - -# * lamp -base = unified_planning.model.Object("base", Part) -blub = unified_planning.model.Object("blub", Part) -lamp = unified_planning.model.Object("lamp", Part) - -# * gearset -gear1 = unified_planning.model.Object("gear1", Part) -gear2 = unified_planning.model.Object("gear2", Part) -gear3 = unified_planning.model.Object("gear3", Part) -shaft1 = unified_planning.model.Object("shaft1", Part) -shaft2 = unified_planning.model.Object("shaft2", Part) - -# * tool --- part -# * chair - -problem.add_objects([location_1, location_2, location_3, location_4]) -problem.add_objects([no_tool, parallel_box1, parallel_box2, inward_claw, outward_claw]) - -########################## * initial state - -##########! for test - -problem.set_initial_value(robot_at(location_1), True) -problem.set_initial_value(in_hand(parallel_box1), True) - - -##########! - -# problem.set_initial_value(robot_at(locations[0]), True) -# for i in range(NLOC - 1): -# problem.set_initial_value(connected(locations[i], locations[i + 1]), True) - - -########################## * goal -problem.add_goal(robot_at(location_4)) -problem.add_goal(in_hand(no_tool)) -print(problem) - -# with OneshotPlanner(name="pyperplan") as planner: -# result = planner.solve(problem) -# if result.status == up.engines.PlanGenerationResultStatus.SOLVED_SATISFICING: -# print("Pyperplan returned: %s" % result.plan) -# else: -# print("No plan found.") - -with OneshotPlanner(problem_kind=problem.kind) as planner: - result = planner.solve(problem) - print("%s returned: %s" % (planner.name, result.plan)) diff --git a/kios_bt_planning/backups/kios_planner/planner.py b/kios_bt_planning/backups/kios_planner/planner.py deleted file mode 100644 index 99faabbc..00000000 --- a/kios_bt_planning/backups/kios_planner/planner.py +++ /dev/null @@ -1,305 +0,0 @@ -from unified_planning.shortcuts import * - -######################################## Type ######################################## -Location = UserType("Location") -Thing = UserType("Thing") -Tool = UserType("Tool", father=Thing) -Part = UserType("Part", father=Thing) - -######################################## Fluent ######################################## -reachable = unified_planning.model.Fluent( - "reachable", BoolType(), l_from=Location, l_to=Location -) -robot_at = unified_planning.model.Fluent("robot_at", BoolType(), l=Location) -thing_at = unified_planning.model.Fluent("thing_at", BoolType(), t=Thing, l=Location) - -in_tool = Fluent("in_tool", p=Part, tool=Tool) -in_hand = Fluent("in_hand", BoolType(), tool=Tool) - -interactive = Fluent("interactive", BoolType(), p=Part, tool=Tool) - -inserted = Fluent("inserted", BoolType(), p=Part) -screwed = Fluent("screwed", BoolType(), p=Part) -placed = Fluent("placed", BoolType(), p=Part) - -######################################## Action ######################################## - -# * move -move = InstantaneousAction("move", l_from=Location, l_to=Location) - -l_from = move.parameter("l_from") -l_to = move.parameter("l_to") - -move.add_precondition(robot_at(l_from)) -move.add_precondition(reachable(l_from, l_to)) - -move.add_effect(robot_at(l_from), False) -move.add_effect(robot_at(l_to), True) - -# * load_tool -load_tool = unified_planning.model.InstantaneousAction( - "load_tool", - tool=Tool, -) -tool = load_tool.parameter("tool") - -tool_var = Variable("tool_var", Tool) -load_tool.add_precondition(Not(Exists(in_hand(tool_var)), tool_var)) - -load_tool.add_effect(in_hand(tool), True) - -# * unload_tool -unload_tool = unified_planning.model.InstantaneousAction("unload_tool", tool=Tool) -tool = unload_tool.parameter("tool") - -unload_tool.add_precondition(in_hand(tool)) -p_var = Variable("p_var", Part) -unload_tool.add_precondition(Not(Exists(in_tool(p_var, tool), p_var))) - -unload_tool.add_effect(in_hand(tool), False) - -# * pick -pick = unified_planning.model.InstantaneousAction( - "pick", p=Part, tool=Tool, l_from=Location, l_to=Location -) -p = pick.parameter("p") -tool = pick.parameter("tool") -l_from = pick.parameter("l_from") -l_to = pick.parameter("l_to") - -pick.add_precondition(robot_at(l_from)) -pick.add_precondition(reachable(l_from, l_to)) -pick.add_precondition(thing_at(p, l_to)) -pick.add_precondition(in_hand(tool)) -p_var = Variable("p_var", Part) -pick.add_precondition(Forall(Not(in_tool(p_var, tool)), p_var)) -pick.add_precondition(interactive(p, tool)) - -pick.add_effect(robot_at(l_from), False) -pick.add_effect(robot_at(l_to), True) -pick.add_effect(thing_at(p, l_to), False) -pick.add_effect(in_tool(p, tool), True) - -# # * place -# place = unified_planning.model.InstantaneousAction( -# "place", p=Part, tool=Tool, l_from=Location, l_to=Location -# ) -# p = place.parameter("p") -# tool = place.parameter("tool") -# l_from = place.parameter("l_from") -# l_to = place.parameter("l_to") - -# place.add_precondition(robot_at(l_from)) -# place.add_precondition(reachable(l_from, l_to)) -# place.add_precondition(in_tool(p, tool)) -# place.add_precondition(in_hand(tool)) - -# place.add_effect(robot_at(l_from), False) -# place.add_effect(robot_at(l_to), True) -# place.add_effect(in_tool(p, tool), False) -# place.add_effect(thing_at(p, l_to), True) - - -# # * push, can be move - - -# # * insert -# insert = unified_planning.model.InstantaneousAction( -# "insert", p=Part, tool=Tool, l_from=Location, l_to=Location -# ) -# p = insert.parameter("p") -# tool = insert.parameter("tool") -# l_from = insert.parameter("l_from") -# l_to = insert.parameter("l_to") - -# insert.add_precondition(reachable(l_from, l_to)) -# insert.add_precondition(robot_at(l_from)) -# insert.add_precondition(in_hand(tool)) -# insert.add_precondition(in_tool(p, tool)) -# # insert.add_precondition(interactive(p, tool)) - -# insert.add_effect(robot_at(l_from), False) -# insert.add_effect(robot_at(l_to), True) -# insert.add_effect(in_tool(p, tool), False) -# insert.add_effect(thing_at(p, l_to), True) -# insert.add_effect(inserted(p), True) - -# # * pull -# pull = unified_planning.model.InstantaneousAction( -# "pull", p=Part, tool=Tool, l_from=Location, l_to=Location -# ) -# p = pull.parameter("p") -# tool = pull.parameter("tool") -# l_from = pull.parameter("l_from") -# l_to = pull.parameter("l_to") - -# pull.add_precondition(reachable(l_from, l_to)) -# pull.add_precondition(robot_at(l_from)) -# pull.add_precondition(thing_at(p, l_to)) -# pull.add_precondition(inserted(p)) -# pull.add_precondition(in_hand(tool)) -# pull.add_precondition(interactive(p, tool)) - -# pull.add_effect(robot_at(l_from), False) -# pull.add_effect(robot_at(l_to), True) -# pull.add_effect(inserted(p), False) -# pull.add_effect(thing_at(p, l_to), False) -# pull.add_effect(in_tool(p, tool), True) - -# # * screw -# screw = unified_planning.model.InstantaneousAction( -# "screw", p=Part, tool=Tool, l_from=Location, l_to=Location -# ) -# p = screw.parameter("p") -# tool = screw.parameter("tool") -# l_from = screw.parameter("l_from") -# l_to = screw.parameter("l_to") - -# screw.add_precondition(reachable(l_from, l_to)) -# screw.add_precondition(robot_at(l_from)) -# screw.add_precondition(in_hand(tool)) -# screw.add_precondition(in_tool(p, tool)) - -# screw.add_effect(robot_at(l_from), False) -# screw.add_effect(robot_at(l_to), True) -# screw.add_effect(in_tool(p, tool), False) -# screw.add_effect(thing_at(p, l_to), True) -# screw.add_effect(screwed(p), True) - -# # * unscrew -# unscrew = unified_planning.model.InstantaneousAction( -# "unscrew", p=Part, tool=Tool, l_from=Location, l_to=Location -# ) -# p = unscrew.parameter("p") -# tool = unscrew.parameter("tool") -# l_from = unscrew.parameter("l_from") -# l_to = unscrew.parameter("l_to") - -# unscrew.add_precondition(reachable(l_from, l_to)) -# unscrew.add_precondition(robot_at(l_from)) -# unscrew.add_precondition(thing_at(p, l_to)) -# unscrew.add_precondition(screwed(p)) -# unscrew.add_precondition(in_hand(tool)) -# unscrew.add_precondition(interactive(p, tool)) - -# unscrew.add_effect(robot_at(l_from), False) -# unscrew.add_effect(robot_at(l_to), True) -# unscrew.add_effect(screwed(p), False) -# unscrew.add_effect(thing_at(p, l_to), False) -# unscrew.add_effect(in_tool(p, tool), True) - -######################################## Problem ######################################## - -problem = Problem("kios_tamp_problem") - -########################## * knowledge - -problem.add_fluent(robot_at, default_initial_value=False) -problem.add_fluent(reachable, default_initial_value=True) -problem.add_fluent(thing_at, default_initial_value=False) -problem.add_fluent(in_hand, default_initial_value=False) -problem.add_fluent(in_tool, default_initial_value=False) -problem.add_fluent(inserted, default_initial_value=False) -problem.add_fluent(screwed, default_initial_value=False) -problem.add_fluent(placed, default_initial_value=False) -problem.add_fluent(interactive, default_initial_value=False) - -problem.add_action(move) -# problem.add_action(pick) -# problem.add_action(place) -# problem.add_action(insert) -# problem.add_action(pull) -# problem.add_action(screw) -# problem.add_action(unscrew) -problem.add_action(load_tool) -problem.add_action(unload_tool) - -######################## * instances -# locations -# NLOC = 10 -# locations = [unified_planning.model.Object("l%s" % i, Location) for i in range(NLOC)] -# problem.add_objects(locations) -# TODO: add locations -# ! for test -location_1 = unified_planning.model.Object("location_1", Location) -location_2 = unified_planning.model.Object("location_2", Location) -location_3 = unified_planning.model.Object("location_3", Location) -location_4 = unified_planning.model.Object("location_4", Location) - -# tools -parallel_box1 = unified_planning.model.Object("parallel_box1", Tool) -parallel_box2 = unified_planning.model.Object("parallel_box2", Tool) -inward_claw = unified_planning.model.Object("inward_claw", Tool) -outward_claw = unified_planning.model.Object("outward_claw", Tool) -no_tool = unified_planning.model.Object("no_tool", Tool) - -# parts -# * chair -leg1 = unified_planning.model.Object("leg1", Part) -leg2 = unified_planning.model.Object("leg2", Part) -nut1 = unified_planning.model.Object("nut1", Part) -nut2 = unified_planning.model.Object("nut2", Part) -seat = unified_planning.model.Object("seat", Part) -back = unified_planning.model.Object("back", Part) - -# * lamp -base = unified_planning.model.Object("base", Part) -blub = unified_planning.model.Object("blub", Part) -lamp = unified_planning.model.Object("lamp", Part) - -# * gearset -gear1 = unified_planning.model.Object("gear1", Part) -gear2 = unified_planning.model.Object("gear2", Part) -gear3 = unified_planning.model.Object("gear3", Part) -shaft1 = unified_planning.model.Object("shaft1", Part) -shaft2 = unified_planning.model.Object("shaft2", Part) - -# * tool --- part -# * chair - -problem.add_objects([location_1, location_2, location_3, location_4]) -problem.add_objects([no_tool, parallel_box1, parallel_box2, inward_claw, outward_claw]) - -########################## * initial state - -##########! for test - -problem.set_initial_value(robot_at(location_1), True) -problem.set_initial_value(in_hand(parallel_box1), True) - - -##########! - -# problem.set_initial_value(robot_at(locations[0]), True) -# for i in range(NLOC - 1): -# problem.set_initial_value(connected(locations[i], locations[i + 1]), True) - - -########################## * goal -problem.add_goal(robot_at(location_4)) -problem.add_goal(in_hand(no_tool)) -print(problem) - -# with OneshotPlanner(name="pyperplan") as planner: -# result = planner.solve(problem) -# if result.status == up.engines.PlanGenerationResultStatus.SOLVED_SATISFICING: -# print("Pyperplan returned: %s" % result.plan) -# else: -# print("No plan found.") - -with OneshotPlanner(problem_kind=problem.kind) as planner: - result = planner.solve(problem) - print("%s returned: %s" % (planner.name, result.plan)) - -# from unified_planning.engines import CompilationKind - -# with Compiler( -# problem_kind=problem.kind, -# compilation_kind=CompilationKind.QUANTIFIERS_REMOVING, -# ) as quantifiers_remover: -# # After we have the compiler, we get the compilation result -# qr_result = quantifiers_remover.compile( -# problem, CompilationKind.QUANTIFIERS_REMOVING -# ) -# qr_problem = qr_result.problem -# qr_kind = qr_problem.kind diff --git a/kios_bt_planning/backups/kios_planner/world_interface.py b/kios_bt_planning/backups/kios_planner/world_interface.py deleted file mode 100644 index a4c5bf3e..00000000 --- a/kios_bt_planning/backups/kios_planner/world_interface.py +++ /dev/null @@ -1,147 +0,0 @@ -""" -see kios_world module for the world interface -""" -# ! DISCARDED -# ! BBREMOVE - -from unified_planning.shortcuts import * - -Location = UserType("Location") - -at = Fluent("at", Location) -distance = Fluent("distance", IntType(), l1=Location, l2=Location) -battery_charge = Fluent("battery_charge", IntType(0, 100)) - -move = InstantaneousAction("move", l_from=Location, l_to=Location) -l_from = move.parameter("l_from") -l_to = move.parameter("l_to") -move.add_precondition(Equals(at, l_from)) -move.add_effect(at, l_to) -move.add_decrease_effect(battery_charge, distance(l_from, l_to)) - -l1 = Object("l1", Location) -l2 = Object("l2", Location) -l3 = Object("l3", Location) -l4 = Object("l4", Location) -l5 = Object("l5", Location) -locations = [l5, l4, l3, l2, l1] - -problem = Problem("moving_robot") -problem.add_fluent(at) -problem.add_fluent(battery_charge) -problem.add_fluent(distance, default_initial_value=101) -problem.add_action(move) -problem.add_objects(locations) - -problem.set_initial_value(at, l1) -problem.set_initial_value(battery_charge, 100) - -problem.set_initial_value(distance(l1, l2), 20) -problem.set_initial_value(distance(l2, l3), 30) -problem.set_initial_value(distance(l3, l4), 20) -problem.set_initial_value(distance(l4, l5), 30) - -problem.set_initial_value(distance(l1, l3), 60) - -problem.add_goal(Equals(at, l5)) -battery_exp = FluentExp(battery_charge) -robot_at = FluentExp(at) - -with SequentialSimulator(problem=problem) as simulator: - initial_state = simulator.get_initial_state() - for travel_location in locations: - if simulator.is_applicable(initial_state, move, (l1, travel_location)): - print(f"From l1 we can reach: {travel_location}") - - # print(initial_state) - - state_at_l3 = simulator.apply(initial_state, move, (l1, l3)) - for travel_location in locations: - if simulator.is_applicable(state_at_l3, move, (l3, travel_location)): - print(f"From l3 we can reach: {travel_location}") - state_at_l4 = simulator.apply(state_at_l3, move, (l3, l4)) - if simulator.is_applicable(state_at_l4, move, (l4, l5)): - print("Done!") - else: - print(f"Problem! Battery too low: {state_at_l4.get_value(battery_exp)}") - - print(state_at_l3) - - state_at_l2 = simulator.apply(initial_state, move, (l1, l2)) - new_state_at_l3 = simulator.apply(state_at_l2, move, (l2, l3)) - new_state_better = ( - new_state_at_l3.get_value(battery_exp) > state_at_l3.get_value(battery_exp) - ).simplify() - if new_state_better.bool_constant_value(): - print("Reaching l3 passing through l2 saves battery!") - else: - print("Can't save battery reaching l3, the problem has no solution!") - - state_at_l3 = new_state_at_l3 - state_at_l4 = simulator.apply(state_at_l3, move, (l3, l4)) - if simulator.is_applicable(state_at_l4, move, (l4, l5)): - print("Done!") - else: - print(f"Problem! Battery too low: {state_at_l4.get_value(battery_exp)}") - state_at_l5 = simulator.apply(state_at_l4, move, (l4, l5)) - - print(state_at_l5) - - from unified_planning.plans import SequentialPlan, ActionInstance - - plan = SequentialPlan( - [ - ActionInstance(move, (l1, l2)), - ActionInstance(move, (l2, l3)), - ActionInstance(move, (l3, l4)), - ActionInstance(move, (l4, l5)), - ] - ) - print( - f"Initial battery value: {initial_state.get_value(battery_exp).constant_value()}" - ) - current_state = initial_state - # We also store the states to plot the metrics later - states = [current_state] - for action_instance in plan.actions: - current_state = simulator.apply(current_state, action_instance) - if current_state is None: - print(f"Error in applying: {action_instance}") - break - states.append(current_state) - current_battery_value = current_state.get_value(battery_exp).constant_value() - # in current_battery_value we inspect the State - print(f"Battery value after {action_instance}: {current_battery_value}") - print(f"Robot is at: {current_state.get_value(robot_at)}") - - # from unified_planning.plot import plot_sequential_plan - - # # Redefine the plot package methods imported above to print the plot to a temp file - # # if the exception "could not locate runnable browser" is raised. This usually happens - # # in the Continuous Integration. - - # from inspect import getmembers, isfunction - # from unified_planning import plot - # from functools import partial - # import os, uuid, tempfile as tf - - # # Define the function that will be executed instead - # def _function(original_function, *args, **kwargs): - # try: - # original_function(*args, **kwargs) - # except Exception as e: - # if "could not locate runnable browser" in str(e): - # original_function( - # *args, - # **kwargs, - # filename=f"{os.path.join(tf.gettempdir(), str(uuid.uuid1()))}.png", - # ) - # else: - # raise e - - # # Iterate over all the functions of the plot package - # for function_name, function in getmembers(plot, isfunction): - # # Override the original function with the new one - # globals()[function_name] = partial(_function, function) - - # plot_sequential_plan(plan, problem, battery_exp, figsize=(9, 3)) diff --git a/kios_bt_planning/backups/kios_utils.py b/kios_bt_planning/backups/kios_utils.py deleted file mode 100644 index ed56e34f..00000000 --- a/kios_bt_planning/backups/kios_utils.py +++ /dev/null @@ -1,36 +0,0 @@ -from enum import Enum, auto - - -class ResultCode(Enum): - SUCCESS = auto() - FAILURE = auto() - ERROR = auto() - - -class ActionPhase(Enum): - FINISH = (999,) - CONDITION = (-9,) - - ERROR = (-1,) - INITIALIZATION = (0,) - APPROACH = (1,) - - RECOVER = (10,) - CARTESIAN_MOVE = (11,) - JOINT_MOVE = (12,) - GRIPPER_FORCE = (13,) - GRIPPER_MOVE = (14,) - CONTACT = (15,) - WIGGLE = (16,) - - TOOL_LOAD = (20,) - TOOL_UNLOAD = (21,) - TOOL_GRASP = (22,) - TOOL_RELEASE = (23,) - TOOL_PICK = (24,) - TOOL_PLACE = (25,) - - GRIPPER_RELEASE = (26,) - GRIPPER_GRASP = (27,) - GRIPPER_PICK = (28,) - GRIPPER_PLACE = (29,) diff --git a/kios_bt_planning/backups/prompt_skeletons/cot_skeleton.json b/kios_bt_planning/backups/prompt_skeletons/cot_skeleton.json deleted file mode 100644 index f815a248..00000000 --- a/kios_bt_planning/backups/prompt_skeletons/cot_skeleton.json +++ /dev/null @@ -1,16 +0,0 @@ -{ - "skeleton_name": "cot_skeleton", - "model_name": "gpt-4-1106-preview", - "version": "V0", - "prompt_dir": "prompts/cot_skeleton_chain", - "prompt_laod_order": [ - "cot_sk_role", - "cot_sk_output_format", - "cot_sk_domain", - "cot_sk_problem", - "cot_sk_state", - "cot_sk_bt", - "cot_sk_chain", - "cot_sk_example" - ] -} \ No newline at end of file diff --git a/kios_bt_planning/backups/prompt_skeletons/refine_skeleton.json b/kios_bt_planning/backups/prompt_skeletons/refine_skeleton.json deleted file mode 100644 index 2b288210..00000000 --- a/kios_bt_planning/backups/prompt_skeletons/refine_skeleton.json +++ /dev/null @@ -1,16 +0,0 @@ -{ - "skeleton_name": "refine_skeleton", - "model_name": "gpt-4-1106-preview", - "version": "V0", - "prompt_dir": "prompts/sk_refine", - "prompt_load_order": [ - "refine_role", - "refine_input_format", - "refine_state", - "refine_help", - "refine_controlflow", - "refine_condition", - "refine_action", - "refine_output_format" - ] -} diff --git a/kios_bt_planning/backups/prompts/DISCARDED_FOLDER b/kios_bt_planning/backups/prompts/DISCARDED_FOLDER deleted file mode 100644 index e69de29b..00000000 diff --git a/kios_bt_planning/backups/prompts/cot_skeleton/prompt/cot_sk_bt.txt b/kios_bt_planning/backups/prompts/cot_skeleton/prompt/cot_sk_bt.txt deleted file mode 100644 index 7219d47f..00000000 --- a/kios_bt_planning/backups/prompts/cot_skeleton/prompt/cot_sk_bt.txt +++ /dev/null @@ -1,83 +0,0 @@ -[user] -The behavior tree should be in json dictionary format and have a structure like the following example: -""" -{ - "summary": "selector to use left hand to load inward_claw", - "name": "selector: load_tool(left_hand, inward_claw)", - "children": [ - { - "summary": "condition node to check if left_hand holds inward_claw", - "name": "target: hold(left_hand, inward_claw)", - }, - { - "summary": "sequence to use left hand to load inward_claw", - "name": "sequence: load_tool(left_hand, inward_claw)", - "children": [ - { - "summary": "condition node to check if inward_claw is equippable", - "name": "precondition: is_equippable(inward_claw)", - }, - { - "summary": "condition node to check if left hand is free", - "name": "precondition: is_free(left_hand)", - }, - { - "summary": "action node to use left hand to load inward_claw", - "name": "action: load_tool(left_hand, inward_claw)", - "effects": [ - {"summary": "left hand will be not free anymore",}, - {"summary": "inward_claw will be not equippable anymore",}, - {"summary": "left_hand will hold inward_claw",}, - ], - }, - ], - }, - ], -} -""" -Follow these rules: -1. Every node are described by a summary and a name: - - The "summary" key includes a short interpretive description of the node. - - The "name" key includes the type of the node and the formal form of the node according to the domain knowledge. -2. The control flow nodes, "selector" and "sequence", can have "children", which is a list of children nodes. -3. The "condition" nodes that are children of "selectors" are targets. Those that are children of "sequences" are preconditions. -4. The "action" nodes should have a key "effects" with a list of dictionaries, which describes the to-be-exerted effects of the action. The effects are shortly described under the key "summary". -5. All the subtrees always have their corresponding action in ROBOT_ACTION_LIST. The recommended structure is: - - A selector as the root node. - - (Usually one) target condition node as the first child of the selector to check if the target of the action has been fulfilled. - - A sequence node as the last child of the selector node to achieve the target. - - (Usually a number of) precondition nodes as the front children of the sequence node to check the preconditions for executing the action. - - One action node as the last child of the sequence node, whose preconditions are checked before and will immediate start the job when being ticked. - - The action will exert its defined effects after finishing the job successfully. Those effects should fulfilled all the target conditions in the selector node. - A brief example is as follows: -{ - selector: load_tool selector, - children: [ - { - target: check if inward_claw is equipped by left hand, - }, - { - sequence: load_tool, - children: [ - {precondition: check if inward_claw is equippable}, - {precondition: check if left hand is free}, - { - action: equip inward_claw to left hand, - effects: [ - left hand not free, - inward_claw not equippable, - left_hand holds inward_claw, - ] - }, - ], - }, - ], -} - -6. When performing a new action to fulfill a unsatisfied precondition, you can construct a subtree to replace the precondition and take that unfulfilled precondition as the target of the subtree. Then you construct the sequence of the subtree according to the new action you want to perform. -7. All of the control flow nodes, namely selectors and sequences, are without memory. A new behavior tree tick will always start from the root and traverse all the nodes even if they are ticked in the last tick. - -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/cot_skeleton/prompt/cot_sk_chain.txt b/kios_bt_planning/backups/prompts/cot_skeleton/prompt/cot_sk_chain.txt deleted file mode 100644 index 8ab56a97..00000000 --- a/kios_bt_planning/backups/prompts/cot_skeleton/prompt/cot_sk_chain.txt +++ /dev/null @@ -1,15 +0,0 @@ -[user] -Follow this workflow to solve the task: - -1. Read the problem description. Parse and generate the "initial_state" in the output dictionary. -2. Interpret the goal of the problem as goal_state and write into the "instruction_summary" in the output dictionary. -3. Try to solve the problem, achieve the goal_state from the initial_state based on the domain knowledge provided. Generate a action sequence with the actions defined in and write it into dictionary["task_plan"]["action_sequence"] of the output. -4. If you cannot find a plan to solve the task, ask question with nature language in the "question" of the output dictionary. -5. Explain the actions in the action sequence and write the output dictionary["task_plan"]["action_instructions"]. -6. Collect the objects in the world that are effected by the action sequence, write into dictionary["task_plan"]["manipulated_objects"]. -7. Generate the behavior tree based on the action sequence and write into dictionary["task_plan"]["behavior_tree"]. - -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/cot_skeleton/prompt/cot_sk_domain.txt b/kios_bt_planning/backups/prompts/cot_skeleton/prompt/cot_sk_domain.txt deleted file mode 100644 index bf3c7859..00000000 --- a/kios_bt_planning/backups/prompts/cot_skeleton/prompt/cot_sk_domain.txt +++ /dev/null @@ -1,89 +0,0 @@ -[user] -Following is the domain knowledge you need in pddl domain format, which includes: -- the object types in the world, -- the predicates for describing the states of the world, -- the actions that the robot can apply, with their precondtions and effects defined. -""" -(define (domain robot_assembly_problem-domain) - (:requirements :strips :typing) - (:types - thing - object - tool part hand - thing - ) - (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part)) - (:action pick_up - :parameters ( ?hand - hand ?tool - tool ?part - part) - :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part)) - :effect (and (hold ?tool ?part) (not (is_free ?tool)))) - (:action put_down - :parameters ( ?hand - hand ?tool - tool ?part - part) - :precondition (and (hold ?tool ?part) (hold ?hand ?tool)) - :effect (and (not (hold ?hand ?part)) (is_free ?tool))) - (:action place - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2))) - (:action detach - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2)))) - (:action insert - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2))) - (:action pull - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2)))) - (:action screw - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2))) - (:action unscrew - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2)))) - (:action load_tool - :parameters ( ?hand - hand ?tool - tool) - :precondition (and (is_equippable ?tool) (is_free ?hand)) - :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool))) - (:action unload_tool - :parameters ( ?hand - hand ?tool - tool) - :precondition (and (hold ?hand ?tool) (is_free ?tool)) - :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool)))) -) -""" -The precidates are explained as follows: -""" -"PREDICATE_LIST" -- is_free( or ): or is free and can hold something. (default: true) -- is_equippable(): is ready to be eqquipped by a robot hand. (default: true) -- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false) -- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false) -- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false) -- can_place_to(, ): can be placed into (placement assembly constraint) (default: false) -- hold(, ): is holding (default: false) -- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false) -- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false) -- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false) -""" - -The actions are explained as follows: -""" -"ROBOT_ACTION_LIST" -- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something. -- put_down(, , ): use to put down . As preconditions, should be holding and should be holding . -- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding . -- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into . -- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding . -- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into . -- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding . -- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into . -- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free. -- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free. -""" -the items in <> indicate the type of the instances needed for the predicates and actions. -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. diff --git a/kios_bt_planning/backups/prompts/cot_skeleton/prompt/cot_sk_example.txt b/kios_bt_planning/backups/prompts/cot_skeleton/prompt/cot_sk_example.txt deleted file mode 100644 index 02853177..00000000 --- a/kios_bt_planning/backups/prompts/cot_skeleton/prompt/cot_sk_example.txt +++ /dev/null @@ -1,144 +0,0 @@ -[user] -I will give you some examples of the input and the output you will generate. -Example 1: -""" -- Input: -(define (problem robot_assembly_problem-problem) - (:domain robot_assembly_problem-domain) - (:objects - parallel_box1 parallel_box2 inward_claw outward_claw no_tool - tool - gear1 gear2 gear3 shaft1 shaft2 base - part - left_hand - hand - ) - (:init (can_manipulate parallel_box1 gear1) (can_manipulate outward_claw gear2) (can_manipulate inward_claw gear3) (can_manipulate parallel_box2 shaft1) (can_manipulate no_tool shaft2) (can_screw_to leg1 seat) (can_screw_to leg2 seat) (can_insert_to back seat) (can_screw_to nut1 seat) (can_screw_to nut2 seat) (can_screw_to blub base) (can_place_to lamp blub) (can_insert_to shaft1 base) (can_insert_to shaft2 base) (can_insert_to gear3 shaft2) (can_insert_to gear2 base) (can_insert_to gear1 shaft1) (is_inserted_to shaft1 base) (is_free left_hand) (is_free parallel_box1) (is_free parallel_box2) (is_free inward_claw) (is_free outward_claw) (is_free no_tool) (is_equippable parallel_box1) (is_equippable parallel_box2) (is_equippable inward_claw) (is_equippable outward_claw) (is_equippable no_tool)) - (:goal (and (is_inserted_to gear1 shaft1))) -) -- Output: -``` -{"task_plan": { - "action_sequence": [ - "load_tool(left_hand, parallel_box1)", - "pick_up(left_hand, parallel_box1, gear1)", - "insert(left_hand, parallel_box1, gear1, shaft1)" - ], - "action_instructions": [ - "equip the parallel_box1 on the left hand", - "pick up the gear1 with the parallel_box1 in the left hand", - "insert the gear1 into the shaft1 using the parallel_box1 in the left hand" - ], - "object_name": ["left_hand", "gear1", "parallel_box1", "shaft1"], - "behavior_tree": { - "summary": "selector to insert gear1 into shaft1", - "name": "selector: insert(left_hand, parallel_box1, gear1, shaft1)", - "children": [ - { - "summary": "condition node to check if gear1 is inserted to shaft1", - "name": "target: is_inserted_to(gear1, shaft1)", - }, - { - "summary": "sequence to insert gear1 into shaft1", - "name": "sequence: insert(left_hand, parallel_box1, gear1, shaft1)", - "children": [ - { - "summary": "selector to load parallel_box1", - "name": "selector: load_tool(left_hand, parallel_box1)", - "children": [ - { - "summary": "condition node to check if left_hand holds parallel_box1", - "name": "target: hold(left_hand, parallel_box1)" - }, - { - "summary": "sequence to load parallel_box1", - "name": "sequence: load_tool(left_hand, parallel_box1)", - "children": [ - { - "summary": "condition node to check if parallel_box1 is equippable", - "name": "precondition: is_equippable(parallel_box1)" - }, - { - "summary": "condition node to check if left hand is free", - "name": "precondition: is_free(left_hand)" - }, - { - "summary": "action node to equip parallel_box1 to left hand", - "name": "action: load_tool(left_hand, parallel_box1)", - "effects": [ - {"summary": "left_hand will be not free",}, - {"summary": "parallel_box1 will be not equippable",}, - {"summary": "left_hand will hold parallel_box1",}, - ], - }, - ], - }, - ], - }, - { - "summary": "selector to use left_hand with parallel_box1 to pick up gear1", - "name": "selector: pick_up(left_hand, parallel_box1, gear1)", - "children": [ - { - "summary": "condition node to check if parallel_box1 holds gear1", - "name": "target: hold(parallel_box1, gear1)" - }, - { - "summary": "sequence to use left hand with parallel_box1 to pick up gear1", - "name": "sequence: pick_up(left_hand, parallel_box1, gear1)", - "children": [ - { - "summary": "condition node to check if parallel_box1 is free", - "name": "precondition: is_free(parallel_box1)" - }, - { - "summary": "condition node to check if left hand holds parallel_box1", - "name": "precondition: hold(left_hand, parallel_box1)" - }, - { - "summary": "action node to use left hand with parallel_box1 to pick up gear1", - "name": "action: pick_up(left_hand, parallel_box1, gear1)", - "effects": [ - {"summary": "parallel_box1 will hold gear1",}, - {"summary": "parallel_box1 will be not free",}, - ], - }, - ], - }, - ], - }, - { - "summary": "condition node to check if gear1 can be inserted to shaft1", - "name": "precondition: can_insert_to(gear1, shaft1)", - }, - { - "summary": "action node to use left_hand with parallel_box1 to insert gear1 to shaft1", - "name": "action: insert(left_hand, parallel_box1, gear1, shaft1)", - "effects": [ - {"summary": "parallel_box1 will be not holding gear1",}, - {"summary": "parallel_box1 will be free",}, - {"summary": "gear1 will be inserted to shaft1",}, - ], - }, - ], - }, - ], -} -}, -"initial_state": {"objects": [ - {"name": "gear1", "properties": []}, - {"name": "shaft1", "properties": []}, - {"name": "parallel_box1", "properties": ["is_equippable", "is_free"]}, - {"name": "left_hand", "properties": ["is_free"]}, - ], - "constraints": [ - {"source": "gear1", "name": "can_insert_to", "target": "shaft1"}, - {"source": "parallel_box1", "name": "can_manipulate", "target": "gear1"}, - ], - "relations": [], -}, -"instruction_summary": "insert gear1 into shaft1", -"question":""} -``` -""" -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/cot_skeleton/prompt/cot_sk_output_format.txt b/kios_bt_planning/backups/prompts/cot_skeleton/prompt/cot_sk_output_format.txt deleted file mode 100644 index d47c6af5..00000000 --- a/kios_bt_planning/backups/prompts/cot_skeleton/prompt/cot_sk_output_format.txt +++ /dev/null @@ -1,21 +0,0 @@ -[user] -You will be provided with a problem and the necessary domain knowledge. -You generate an action sequence to solve the task and contruct a behavior tree based on it in json format. -The output dictionary has five keys. -""" -- dictionary["task_plan"]: A dictionary containing the task plan. -- dictionary["initial_state"]: The initial world state before executing the plan. -- dictionary["instruction_summary"]: A brief summary of the task plan. -- dictionary["question"]: If you don't have enough knowledge for solving the problem, you can ask questions about the knowledge you need. Leave this key empty if you can solve the task. -""" -Four keys exist in dictionary["task_plan"]. -""" -- dictionary["task_plan"]["action_sequence"]: Contains a list of robot actions. Only the actions defined in the "ROBOT_ACTION LIST" will be used. -- dictionary["task_plan"]["action_instructions"]: contains a list of instructions corresponding to dictionary["task_plan"]["action_sequence"] to explain the actions. -- dictionary["task_plan"]["manipulated_objects"]: The list of the manipulated objects. It only includes the objects that are related to the current task. -- dictionary["task_plan"]["behavior_tree"]: contains a dictionary of the behavior tree constructed according to the dictionary ["task_plan"]["action_sequence"] -""" -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/cot_skeleton/prompt/cot_sk_problem.txt b/kios_bt_planning/backups/prompts/cot_skeleton/prompt/cot_sk_problem.txt deleted file mode 100644 index 6b1ddfb7..00000000 --- a/kios_bt_planning/backups/prompts/cot_skeleton/prompt/cot_sk_problem.txt +++ /dev/null @@ -1,24 +0,0 @@ -[user] -Information about the problem is given as a pddl problem. -It includes the following information: -- objects: all the objects in the world. They should be of a type defined in the domain knowledge. For example, "parallel_box1" is an instance of the type "tool", while "gear1" is an instance of the type "part". -- init: the initial world state to start with, described with [PREDICATE_LIST] from domain knowledge. For example, "is_free hand" means the object "hand" is free and holds nothing. -- goal: the world state to achieve, described with [PREDICATE_LIST] from domain knowledge. -An example: -""" -(define (problem robot_assembly_problem-problem) - (:domain robot_assembly_problem-domain) - (:objects - parallel_box1 parallel_box2 inward_claw outward_claw no_tool - tool - gear1 gear2 gear3 shaft1 shaft2 base - part - left_hand - hand - ) - (:init (can_manipulate parallel_box1 gear1) (can_manipulate outward_claw gear2) (can_manipulate inward_claw gear3) (can_manipulate parallel_box2 shaft1) (can_manipulate no_tool shaft2) (can_screw_to leg1 seat) (can_screw_to leg2 seat) (can_insert_to back seat) (can_screw_to nut1 seat) (can_screw_to nut2 seat) (can_screw_to blub base) (can_place_to lamp blub) (can_insert_to shaft1 base) (can_insert_to shaft2 base) (can_insert_to gear3 shaft2) (can_insert_to gear2 base) (can_insert_to gear1 shaft1) (is_free left_hand) (is_free parallel_box1) (is_free parallel_box2) (is_free inward_claw) (is_free outward_claw) (is_free no_tool) (is_equippable parallel_box1) (is_equippable parallel_box2) (is_equippable inward_claw) (is_equippable outward_claw) (is_equippable no_tool)) - (:goal (and (is_inserted_to shaft1 base))) -) -""" - -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/cot_skeleton/prompt/cot_sk_role.txt b/kios_bt_planning/backups/prompts/cot_skeleton/prompt/cot_sk_role.txt deleted file mode 100644 index 6ce736a0..00000000 --- a/kios_bt_planning/backups/prompts/cot_skeleton/prompt/cot_sk_role.txt +++ /dev/null @@ -1,5 +0,0 @@ -[user] -You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you break it down and construct robot task plans represented by behaiovr trees. -Please do not begin working until I say "Start working." Instead, simply output the message "Waiting for next input." Understood? -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/cot_skeleton/prompt/cot_sk_state.txt b/kios_bt_planning/backups/prompts/cot_skeleton/prompt/cot_sk_state.txt deleted file mode 100644 index 00d748ea..00000000 --- a/kios_bt_planning/backups/prompts/cot_skeleton/prompt/cot_sk_state.txt +++ /dev/null @@ -1,28 +0,0 @@ -[user] -Information about world states should be in json dictionary format. Example: -""" -{ - "objects": [ - {"name": "leg1", "properties": []}, # leg1 is an object with no properties - {"name": "seat", "properties": []}, # seat is an object with no properties - {"name": "hand", "properties": ["is_free"]}, # hand is an object with a property "is_free" - ], - "constraints": [ - {"source": "leg1", "name": "can_screw_to", "target": "seat"}, # means leg1 can be screwed to seat - ], - "relations": [ - {"source": "leg1", "name": "is_screwed_to", "target": "seat"}, # means leg1 is screwed to seat - ], -} -""" -It contains three lists: - - "objects": the objects and their properties (as lists of strings) in the world. The properties can be changed by robot actions. - - "constraints": the constraint relations that are enforced and should not be changed. They are knowledge about the world or some constraints set by the user. - - "relations": the relations in the world. They can be changed by robot actions. -The properties, constraints and relations should always align with the [PREDICATE_LIST] in domain knowledge. -Only concentrate on the objects, constraints and relations that are related to the current task. - -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/cot_skeleton/query/old_query.txt b/kios_bt_planning/backups/prompts/cot_skeleton/query/old_query.txt deleted file mode 100644 index d9ff871a..00000000 --- a/kios_bt_planning/backups/prompts/cot_skeleton/query/old_query.txt +++ /dev/null @@ -1,16 +0,0 @@ -Start working. Resume from the problem below. -""" -{"problem": [PROBLEM]} -""" -The dictionary that you return should be formatted as json file. Follow these rules: -1. Always equip the proper tools before performing specific object manipulaion. -2. Make sure that each element of the ["action_instructions"] explains corresponding element of the ["action_sequence"]. Refer to the "ROBOT ACTION LIST" to understand the elements of ["action_sequence"]. -3, The length of the ["action_instructions"] list and the ["action_sequence"] list should be the same. -4. Never left ',' at the end of the list. Use json boolean values, "true" or "false", instead of "True" or "False". -5. Keep track of all items listed in the "objects" section of the "initial_state" field. Please make sure you fill out the "objects" section for all listed items. -6. Use the "PREDICATE_LIST" for the initial states (:init) in the problem to generate ["initial_state"]. -7. All keys of the dictionary should be double-quoted. -8. Insert ``` at the beginning and the end of the dictionary to separate it from the rest of your response. Don't write any additional things. -9. Make sure that you output a consistent manipultation as a single arm robot. For example, grasping an object should not occur in successive steps. -10. When generating the behavior tree, make sure the tree has the asked structure. -Adhere to the output format I defined above. Follow the rules above. Think step by step. \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/cot_skeleton/query/query.txt b/kios_bt_planning/backups/prompts/cot_skeleton/query/query.txt deleted file mode 100644 index cfeb4e8e..00000000 --- a/kios_bt_planning/backups/prompts/cot_skeleton/query/query.txt +++ /dev/null @@ -1,16 +0,0 @@ -Start working. Resume from the problem below. -""" -{"problem": [PROBLEM]} -""" -The dictionary that you return should be formatted as json file. Follow these rules: -1. Always equip the proper tools before performing specific object manipulaion. -2. Make sure that each element of the ["action_instructions"] explains corresponding element of the ["action_sequence"]. Refer to the "ROBOT ACTION LIST" to understand the elements of ["action_sequence"]. -3, The length of the ["action_instructions"] list and the ["action_sequence"] list should be the same. -4. Never left ',' at the end of the list. Use json boolean values, "true" or "false", instead of "True" or "False". -5. Keep track of all items listed in the "objects" section of the "initial_state" field. -6. Use the "PREDICATE_LIST" for the initial states (:init) in the problem to generate ["initial_state"]. -7. All keys of the dictionary should be double-quoted. -8. Insert ``` at the beginning and the end of the dictionary to separate it from the rest of your response. Don't write any additional things. -9. Make sure that you output a consistent manipultation as a single arm robot. For example, grasping an object should not occur in successive steps. -10. When generating the behavior tree, make sure the tree has the asked structure. -Adhere to the output format I defined above. Follow the rules above. Think step by step. \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/cot_skeleton/system/system.txt b/kios_bt_planning/backups/prompts/cot_skeleton/system/system.txt deleted file mode 100644 index 26f2afe2..00000000 --- a/kios_bt_planning/backups/prompts/cot_skeleton/system/system.txt +++ /dev/null @@ -1 +0,0 @@ -You are an excellent interpreter of instructions for robotic assembly tasks. Given the knowledge you need, the world states and some instructions, you break it down and construct robot task plans represented by behaiovr trees based on the action sequences. \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/cot_skeleton_chain/prompt/cot_sk_bt.txt b/kios_bt_planning/backups/prompts/cot_skeleton_chain/prompt/cot_sk_bt.txt deleted file mode 100644 index 7219d47f..00000000 --- a/kios_bt_planning/backups/prompts/cot_skeleton_chain/prompt/cot_sk_bt.txt +++ /dev/null @@ -1,83 +0,0 @@ -[user] -The behavior tree should be in json dictionary format and have a structure like the following example: -""" -{ - "summary": "selector to use left hand to load inward_claw", - "name": "selector: load_tool(left_hand, inward_claw)", - "children": [ - { - "summary": "condition node to check if left_hand holds inward_claw", - "name": "target: hold(left_hand, inward_claw)", - }, - { - "summary": "sequence to use left hand to load inward_claw", - "name": "sequence: load_tool(left_hand, inward_claw)", - "children": [ - { - "summary": "condition node to check if inward_claw is equippable", - "name": "precondition: is_equippable(inward_claw)", - }, - { - "summary": "condition node to check if left hand is free", - "name": "precondition: is_free(left_hand)", - }, - { - "summary": "action node to use left hand to load inward_claw", - "name": "action: load_tool(left_hand, inward_claw)", - "effects": [ - {"summary": "left hand will be not free anymore",}, - {"summary": "inward_claw will be not equippable anymore",}, - {"summary": "left_hand will hold inward_claw",}, - ], - }, - ], - }, - ], -} -""" -Follow these rules: -1. Every node are described by a summary and a name: - - The "summary" key includes a short interpretive description of the node. - - The "name" key includes the type of the node and the formal form of the node according to the domain knowledge. -2. The control flow nodes, "selector" and "sequence", can have "children", which is a list of children nodes. -3. The "condition" nodes that are children of "selectors" are targets. Those that are children of "sequences" are preconditions. -4. The "action" nodes should have a key "effects" with a list of dictionaries, which describes the to-be-exerted effects of the action. The effects are shortly described under the key "summary". -5. All the subtrees always have their corresponding action in ROBOT_ACTION_LIST. The recommended structure is: - - A selector as the root node. - - (Usually one) target condition node as the first child of the selector to check if the target of the action has been fulfilled. - - A sequence node as the last child of the selector node to achieve the target. - - (Usually a number of) precondition nodes as the front children of the sequence node to check the preconditions for executing the action. - - One action node as the last child of the sequence node, whose preconditions are checked before and will immediate start the job when being ticked. - - The action will exert its defined effects after finishing the job successfully. Those effects should fulfilled all the target conditions in the selector node. - A brief example is as follows: -{ - selector: load_tool selector, - children: [ - { - target: check if inward_claw is equipped by left hand, - }, - { - sequence: load_tool, - children: [ - {precondition: check if inward_claw is equippable}, - {precondition: check if left hand is free}, - { - action: equip inward_claw to left hand, - effects: [ - left hand not free, - inward_claw not equippable, - left_hand holds inward_claw, - ] - }, - ], - }, - ], -} - -6. When performing a new action to fulfill a unsatisfied precondition, you can construct a subtree to replace the precondition and take that unfulfilled precondition as the target of the subtree. Then you construct the sequence of the subtree according to the new action you want to perform. -7. All of the control flow nodes, namely selectors and sequences, are without memory. A new behavior tree tick will always start from the root and traverse all the nodes even if they are ticked in the last tick. - -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/cot_skeleton_chain/prompt/cot_sk_chain.txt b/kios_bt_planning/backups/prompts/cot_skeleton_chain/prompt/cot_sk_chain.txt deleted file mode 100644 index 8ab56a97..00000000 --- a/kios_bt_planning/backups/prompts/cot_skeleton_chain/prompt/cot_sk_chain.txt +++ /dev/null @@ -1,15 +0,0 @@ -[user] -Follow this workflow to solve the task: - -1. Read the problem description. Parse and generate the "initial_state" in the output dictionary. -2. Interpret the goal of the problem as goal_state and write into the "instruction_summary" in the output dictionary. -3. Try to solve the problem, achieve the goal_state from the initial_state based on the domain knowledge provided. Generate a action sequence with the actions defined in and write it into dictionary["task_plan"]["action_sequence"] of the output. -4. If you cannot find a plan to solve the task, ask question with nature language in the "question" of the output dictionary. -5. Explain the actions in the action sequence and write the output dictionary["task_plan"]["action_instructions"]. -6. Collect the objects in the world that are effected by the action sequence, write into dictionary["task_plan"]["manipulated_objects"]. -7. Generate the behavior tree based on the action sequence and write into dictionary["task_plan"]["behavior_tree"]. - -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/cot_skeleton_chain/prompt/cot_sk_domain.txt b/kios_bt_planning/backups/prompts/cot_skeleton_chain/prompt/cot_sk_domain.txt deleted file mode 100644 index bf3c7859..00000000 --- a/kios_bt_planning/backups/prompts/cot_skeleton_chain/prompt/cot_sk_domain.txt +++ /dev/null @@ -1,89 +0,0 @@ -[user] -Following is the domain knowledge you need in pddl domain format, which includes: -- the object types in the world, -- the predicates for describing the states of the world, -- the actions that the robot can apply, with their precondtions and effects defined. -""" -(define (domain robot_assembly_problem-domain) - (:requirements :strips :typing) - (:types - thing - object - tool part hand - thing - ) - (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part)) - (:action pick_up - :parameters ( ?hand - hand ?tool - tool ?part - part) - :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part)) - :effect (and (hold ?tool ?part) (not (is_free ?tool)))) - (:action put_down - :parameters ( ?hand - hand ?tool - tool ?part - part) - :precondition (and (hold ?tool ?part) (hold ?hand ?tool)) - :effect (and (not (hold ?hand ?part)) (is_free ?tool))) - (:action place - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2))) - (:action detach - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2)))) - (:action insert - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2))) - (:action pull - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2)))) - (:action screw - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2))) - (:action unscrew - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2)))) - (:action load_tool - :parameters ( ?hand - hand ?tool - tool) - :precondition (and (is_equippable ?tool) (is_free ?hand)) - :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool))) - (:action unload_tool - :parameters ( ?hand - hand ?tool - tool) - :precondition (and (hold ?hand ?tool) (is_free ?tool)) - :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool)))) -) -""" -The precidates are explained as follows: -""" -"PREDICATE_LIST" -- is_free( or ): or is free and can hold something. (default: true) -- is_equippable(): is ready to be eqquipped by a robot hand. (default: true) -- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false) -- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false) -- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false) -- can_place_to(, ): can be placed into (placement assembly constraint) (default: false) -- hold(, ): is holding (default: false) -- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false) -- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false) -- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false) -""" - -The actions are explained as follows: -""" -"ROBOT_ACTION_LIST" -- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something. -- put_down(, , ): use to put down . As preconditions, should be holding and should be holding . -- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding . -- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into . -- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding . -- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into . -- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding . -- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into . -- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free. -- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free. -""" -the items in <> indicate the type of the instances needed for the predicates and actions. -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. diff --git a/kios_bt_planning/backups/prompts/cot_skeleton_chain/prompt/cot_sk_example.txt b/kios_bt_planning/backups/prompts/cot_skeleton_chain/prompt/cot_sk_example.txt deleted file mode 100644 index 02853177..00000000 --- a/kios_bt_planning/backups/prompts/cot_skeleton_chain/prompt/cot_sk_example.txt +++ /dev/null @@ -1,144 +0,0 @@ -[user] -I will give you some examples of the input and the output you will generate. -Example 1: -""" -- Input: -(define (problem robot_assembly_problem-problem) - (:domain robot_assembly_problem-domain) - (:objects - parallel_box1 parallel_box2 inward_claw outward_claw no_tool - tool - gear1 gear2 gear3 shaft1 shaft2 base - part - left_hand - hand - ) - (:init (can_manipulate parallel_box1 gear1) (can_manipulate outward_claw gear2) (can_manipulate inward_claw gear3) (can_manipulate parallel_box2 shaft1) (can_manipulate no_tool shaft2) (can_screw_to leg1 seat) (can_screw_to leg2 seat) (can_insert_to back seat) (can_screw_to nut1 seat) (can_screw_to nut2 seat) (can_screw_to blub base) (can_place_to lamp blub) (can_insert_to shaft1 base) (can_insert_to shaft2 base) (can_insert_to gear3 shaft2) (can_insert_to gear2 base) (can_insert_to gear1 shaft1) (is_inserted_to shaft1 base) (is_free left_hand) (is_free parallel_box1) (is_free parallel_box2) (is_free inward_claw) (is_free outward_claw) (is_free no_tool) (is_equippable parallel_box1) (is_equippable parallel_box2) (is_equippable inward_claw) (is_equippable outward_claw) (is_equippable no_tool)) - (:goal (and (is_inserted_to gear1 shaft1))) -) -- Output: -``` -{"task_plan": { - "action_sequence": [ - "load_tool(left_hand, parallel_box1)", - "pick_up(left_hand, parallel_box1, gear1)", - "insert(left_hand, parallel_box1, gear1, shaft1)" - ], - "action_instructions": [ - "equip the parallel_box1 on the left hand", - "pick up the gear1 with the parallel_box1 in the left hand", - "insert the gear1 into the shaft1 using the parallel_box1 in the left hand" - ], - "object_name": ["left_hand", "gear1", "parallel_box1", "shaft1"], - "behavior_tree": { - "summary": "selector to insert gear1 into shaft1", - "name": "selector: insert(left_hand, parallel_box1, gear1, shaft1)", - "children": [ - { - "summary": "condition node to check if gear1 is inserted to shaft1", - "name": "target: is_inserted_to(gear1, shaft1)", - }, - { - "summary": "sequence to insert gear1 into shaft1", - "name": "sequence: insert(left_hand, parallel_box1, gear1, shaft1)", - "children": [ - { - "summary": "selector to load parallel_box1", - "name": "selector: load_tool(left_hand, parallel_box1)", - "children": [ - { - "summary": "condition node to check if left_hand holds parallel_box1", - "name": "target: hold(left_hand, parallel_box1)" - }, - { - "summary": "sequence to load parallel_box1", - "name": "sequence: load_tool(left_hand, parallel_box1)", - "children": [ - { - "summary": "condition node to check if parallel_box1 is equippable", - "name": "precondition: is_equippable(parallel_box1)" - }, - { - "summary": "condition node to check if left hand is free", - "name": "precondition: is_free(left_hand)" - }, - { - "summary": "action node to equip parallel_box1 to left hand", - "name": "action: load_tool(left_hand, parallel_box1)", - "effects": [ - {"summary": "left_hand will be not free",}, - {"summary": "parallel_box1 will be not equippable",}, - {"summary": "left_hand will hold parallel_box1",}, - ], - }, - ], - }, - ], - }, - { - "summary": "selector to use left_hand with parallel_box1 to pick up gear1", - "name": "selector: pick_up(left_hand, parallel_box1, gear1)", - "children": [ - { - "summary": "condition node to check if parallel_box1 holds gear1", - "name": "target: hold(parallel_box1, gear1)" - }, - { - "summary": "sequence to use left hand with parallel_box1 to pick up gear1", - "name": "sequence: pick_up(left_hand, parallel_box1, gear1)", - "children": [ - { - "summary": "condition node to check if parallel_box1 is free", - "name": "precondition: is_free(parallel_box1)" - }, - { - "summary": "condition node to check if left hand holds parallel_box1", - "name": "precondition: hold(left_hand, parallel_box1)" - }, - { - "summary": "action node to use left hand with parallel_box1 to pick up gear1", - "name": "action: pick_up(left_hand, parallel_box1, gear1)", - "effects": [ - {"summary": "parallel_box1 will hold gear1",}, - {"summary": "parallel_box1 will be not free",}, - ], - }, - ], - }, - ], - }, - { - "summary": "condition node to check if gear1 can be inserted to shaft1", - "name": "precondition: can_insert_to(gear1, shaft1)", - }, - { - "summary": "action node to use left_hand with parallel_box1 to insert gear1 to shaft1", - "name": "action: insert(left_hand, parallel_box1, gear1, shaft1)", - "effects": [ - {"summary": "parallel_box1 will be not holding gear1",}, - {"summary": "parallel_box1 will be free",}, - {"summary": "gear1 will be inserted to shaft1",}, - ], - }, - ], - }, - ], -} -}, -"initial_state": {"objects": [ - {"name": "gear1", "properties": []}, - {"name": "shaft1", "properties": []}, - {"name": "parallel_box1", "properties": ["is_equippable", "is_free"]}, - {"name": "left_hand", "properties": ["is_free"]}, - ], - "constraints": [ - {"source": "gear1", "name": "can_insert_to", "target": "shaft1"}, - {"source": "parallel_box1", "name": "can_manipulate", "target": "gear1"}, - ], - "relations": [], -}, -"instruction_summary": "insert gear1 into shaft1", -"question":""} -``` -""" -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/cot_skeleton_chain/prompt/cot_sk_output_format.txt b/kios_bt_planning/backups/prompts/cot_skeleton_chain/prompt/cot_sk_output_format.txt deleted file mode 100644 index d47c6af5..00000000 --- a/kios_bt_planning/backups/prompts/cot_skeleton_chain/prompt/cot_sk_output_format.txt +++ /dev/null @@ -1,21 +0,0 @@ -[user] -You will be provided with a problem and the necessary domain knowledge. -You generate an action sequence to solve the task and contruct a behavior tree based on it in json format. -The output dictionary has five keys. -""" -- dictionary["task_plan"]: A dictionary containing the task plan. -- dictionary["initial_state"]: The initial world state before executing the plan. -- dictionary["instruction_summary"]: A brief summary of the task plan. -- dictionary["question"]: If you don't have enough knowledge for solving the problem, you can ask questions about the knowledge you need. Leave this key empty if you can solve the task. -""" -Four keys exist in dictionary["task_plan"]. -""" -- dictionary["task_plan"]["action_sequence"]: Contains a list of robot actions. Only the actions defined in the "ROBOT_ACTION LIST" will be used. -- dictionary["task_plan"]["action_instructions"]: contains a list of instructions corresponding to dictionary["task_plan"]["action_sequence"] to explain the actions. -- dictionary["task_plan"]["manipulated_objects"]: The list of the manipulated objects. It only includes the objects that are related to the current task. -- dictionary["task_plan"]["behavior_tree"]: contains a dictionary of the behavior tree constructed according to the dictionary ["task_plan"]["action_sequence"] -""" -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/cot_skeleton_chain/prompt/cot_sk_problem.txt b/kios_bt_planning/backups/prompts/cot_skeleton_chain/prompt/cot_sk_problem.txt deleted file mode 100644 index 6b1ddfb7..00000000 --- a/kios_bt_planning/backups/prompts/cot_skeleton_chain/prompt/cot_sk_problem.txt +++ /dev/null @@ -1,24 +0,0 @@ -[user] -Information about the problem is given as a pddl problem. -It includes the following information: -- objects: all the objects in the world. They should be of a type defined in the domain knowledge. For example, "parallel_box1" is an instance of the type "tool", while "gear1" is an instance of the type "part". -- init: the initial world state to start with, described with [PREDICATE_LIST] from domain knowledge. For example, "is_free hand" means the object "hand" is free and holds nothing. -- goal: the world state to achieve, described with [PREDICATE_LIST] from domain knowledge. -An example: -""" -(define (problem robot_assembly_problem-problem) - (:domain robot_assembly_problem-domain) - (:objects - parallel_box1 parallel_box2 inward_claw outward_claw no_tool - tool - gear1 gear2 gear3 shaft1 shaft2 base - part - left_hand - hand - ) - (:init (can_manipulate parallel_box1 gear1) (can_manipulate outward_claw gear2) (can_manipulate inward_claw gear3) (can_manipulate parallel_box2 shaft1) (can_manipulate no_tool shaft2) (can_screw_to leg1 seat) (can_screw_to leg2 seat) (can_insert_to back seat) (can_screw_to nut1 seat) (can_screw_to nut2 seat) (can_screw_to blub base) (can_place_to lamp blub) (can_insert_to shaft1 base) (can_insert_to shaft2 base) (can_insert_to gear3 shaft2) (can_insert_to gear2 base) (can_insert_to gear1 shaft1) (is_free left_hand) (is_free parallel_box1) (is_free parallel_box2) (is_free inward_claw) (is_free outward_claw) (is_free no_tool) (is_equippable parallel_box1) (is_equippable parallel_box2) (is_equippable inward_claw) (is_equippable outward_claw) (is_equippable no_tool)) - (:goal (and (is_inserted_to shaft1 base))) -) -""" - -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/cot_skeleton_chain/prompt/cot_sk_role.txt b/kios_bt_planning/backups/prompts/cot_skeleton_chain/prompt/cot_sk_role.txt deleted file mode 100644 index 6ce736a0..00000000 --- a/kios_bt_planning/backups/prompts/cot_skeleton_chain/prompt/cot_sk_role.txt +++ /dev/null @@ -1,5 +0,0 @@ -[user] -You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you break it down and construct robot task plans represented by behaiovr trees. -Please do not begin working until I say "Start working." Instead, simply output the message "Waiting for next input." Understood? -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/cot_skeleton_chain/prompt/cot_sk_state.txt b/kios_bt_planning/backups/prompts/cot_skeleton_chain/prompt/cot_sk_state.txt deleted file mode 100644 index 00d748ea..00000000 --- a/kios_bt_planning/backups/prompts/cot_skeleton_chain/prompt/cot_sk_state.txt +++ /dev/null @@ -1,28 +0,0 @@ -[user] -Information about world states should be in json dictionary format. Example: -""" -{ - "objects": [ - {"name": "leg1", "properties": []}, # leg1 is an object with no properties - {"name": "seat", "properties": []}, # seat is an object with no properties - {"name": "hand", "properties": ["is_free"]}, # hand is an object with a property "is_free" - ], - "constraints": [ - {"source": "leg1", "name": "can_screw_to", "target": "seat"}, # means leg1 can be screwed to seat - ], - "relations": [ - {"source": "leg1", "name": "is_screwed_to", "target": "seat"}, # means leg1 is screwed to seat - ], -} -""" -It contains three lists: - - "objects": the objects and their properties (as lists of strings) in the world. The properties can be changed by robot actions. - - "constraints": the constraint relations that are enforced and should not be changed. They are knowledge about the world or some constraints set by the user. - - "relations": the relations in the world. They can be changed by robot actions. -The properties, constraints and relations should always align with the [PREDICATE_LIST] in domain knowledge. -Only concentrate on the objects, constraints and relations that are related to the current task. - -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/cot_skeleton_chain/query/old_query.txt b/kios_bt_planning/backups/prompts/cot_skeleton_chain/query/old_query.txt deleted file mode 100644 index d9ff871a..00000000 --- a/kios_bt_planning/backups/prompts/cot_skeleton_chain/query/old_query.txt +++ /dev/null @@ -1,16 +0,0 @@ -Start working. Resume from the problem below. -""" -{"problem": [PROBLEM]} -""" -The dictionary that you return should be formatted as json file. Follow these rules: -1. Always equip the proper tools before performing specific object manipulaion. -2. Make sure that each element of the ["action_instructions"] explains corresponding element of the ["action_sequence"]. Refer to the "ROBOT ACTION LIST" to understand the elements of ["action_sequence"]. -3, The length of the ["action_instructions"] list and the ["action_sequence"] list should be the same. -4. Never left ',' at the end of the list. Use json boolean values, "true" or "false", instead of "True" or "False". -5. Keep track of all items listed in the "objects" section of the "initial_state" field. Please make sure you fill out the "objects" section for all listed items. -6. Use the "PREDICATE_LIST" for the initial states (:init) in the problem to generate ["initial_state"]. -7. All keys of the dictionary should be double-quoted. -8. Insert ``` at the beginning and the end of the dictionary to separate it from the rest of your response. Don't write any additional things. -9. Make sure that you output a consistent manipultation as a single arm robot. For example, grasping an object should not occur in successive steps. -10. When generating the behavior tree, make sure the tree has the asked structure. -Adhere to the output format I defined above. Follow the rules above. Think step by step. \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/cot_skeleton_chain/query/query.txt b/kios_bt_planning/backups/prompts/cot_skeleton_chain/query/query.txt deleted file mode 100644 index cbe4d915..00000000 --- a/kios_bt_planning/backups/prompts/cot_skeleton_chain/query/query.txt +++ /dev/null @@ -1,16 +0,0 @@ -Start working. Resume from the problem below. ---------------- -{problem} ---------------- -The dictionary that you return should be formatted as json file. Follow these rules: -1. Always equip the proper tools before performing specific object manipulaion. -2. Make sure that each element of the ["action_instructions"] explains corresponding element of the ["action_sequence"]. Refer to the "ROBOT ACTION LIST" to understand the elements of ["action_sequence"]. -3, The length of the ["action_instructions"] list and the ["action_sequence"] list should be the same. -4. Never left ',' at the end of the list. Use json boolean values, "true" or "false", instead of "True" or "False". -5. Keep track of all items listed in the "objects" section of the "initial_state" field. -6. Use the "PREDICATE_LIST" for the initial states (:init) in the problem to generate ["initial_state"]. -7. All keys of the dictionary should be double-quoted. -8. Return the json dictionary without adding anything at the beginning and the end of it. You return should be ready to be parsed as a json object directly. -9. Make sure that you output a consistent manipultation as a single arm robot. For example, grasping an object should not occur in successive steps. -10. When generating the behavior tree, make sure the tree has the asked structure. -Adhere to the output format I defined above. Follow the rules above. Think step by step. \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/cot_skeleton_chain/system/system.txt b/kios_bt_planning/backups/prompts/cot_skeleton_chain/system/system.txt deleted file mode 100644 index 26f2afe2..00000000 --- a/kios_bt_planning/backups/prompts/cot_skeleton_chain/system/system.txt +++ /dev/null @@ -1 +0,0 @@ -You are an excellent interpreter of instructions for robotic assembly tasks. Given the knowledge you need, the world states and some instructions, you break it down and construct robot task plans represented by behaiovr trees based on the action sequences. \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/cot_sp_skeleton/prompt/cot_bt.txt b/kios_bt_planning/backups/prompts/cot_sp_skeleton/prompt/cot_bt.txt deleted file mode 100644 index e2fc475c..00000000 --- a/kios_bt_planning/backups/prompts/cot_sp_skeleton/prompt/cot_bt.txt +++ /dev/null @@ -1,68 +0,0 @@ -[user] -The behavior tree should be in json dictionary format and have a structure like the following example: -""" -{ - "summary": "selector to load inward_claw to left hand", - "children": [ - { - "summary": "condition node to check if inward_claw is held by left hand", - }, - { - "summary": "sequence to load inward_claw to left hand", - "children": [ - { - "summary": "condition node to check if inward_claw is equippable", - }, - { - "summary": "condition node to check if left hand is free", - }, - { - "summary": "action node to equip inward_claw to left hand", - "effects": [ - {"summary": "left hand will be not free anymore",}, - {"summary": "inward_claw will be not equippable anymore",}, - {"summary": "left_hand will hold inward_claw",}, - ], - }, - ], - }, - ], -} -""" -Follow these rules: -1. Every node are described by a summary, which is a short interpretive description of the node. It should include the type of the node and the purpose of the node. -2. The control flow nodes, "selector" and "sequence", can have "children", which is a list of children nodes. -3. The "condition" nodes that are children of "selectors" are "targets". Those that are children of "sequences" are preconditions. -4. The "action" nodes should have a key "effects" with a list of dictionaries, which describes the to-be-exerted effects of the action. The effects are shortly described under the key "summary". -5. All the subtrees always have their corresponding action in ROBOT_ACTION_LIST. They are used to represent the actions in the action sequence. The recommended structure is: - - A selector as the root node. - - (Usually one) target condition node as the first child of the selector to check if the target of the action has been fulfilled. - - A sequence node as the last child of the selector node to achieve the target. - - (Usually a number of) precondition nodes as the front children of the sequence node to check the preconditions for executing the action. - - One action node as the last child of the sequence node, whose preconditions are checked before and will immediate start the job when being ticked. - - The action will exert its defined effects after finishing the job successfully. Those effects should fulfilled all the target conditions in the selector node. - A brief example is as follows: -{ - selector: load_tool selector, - children: [ - { - target: check if inward_claw is equipped by left hand, - }, - { - sequence: load_tool, - children: [ - precondition: check if inward_claw is equippable, - precondition: check if left hand is free, - {action: equip inward_claw to left hand, effects: [left hand not free, inward_claw not equippable, left_hand holds inward_claw]}, - ], - }, - ], -} - -6. When performing a new action to fulfill a unsatisfied precondition, you can construct a subtree to replace the precondition and take that unfulfilled precondition as the target of the subtree. Then you construct the sequence of the subtree according to the new action you want to perform. -7. All of the control flow nodes, namely selectors and sequences, are without memory. A new behavior tree tick will always start from the root and traverse all the nodes even if they are ticked in the last tick. - -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/cot_sp_skeleton/prompt/cot_chain.txt b/kios_bt_planning/backups/prompts/cot_sp_skeleton/prompt/cot_chain.txt deleted file mode 100644 index 8ab56a97..00000000 --- a/kios_bt_planning/backups/prompts/cot_sp_skeleton/prompt/cot_chain.txt +++ /dev/null @@ -1,15 +0,0 @@ -[user] -Follow this workflow to solve the task: - -1. Read the problem description. Parse and generate the "initial_state" in the output dictionary. -2. Interpret the goal of the problem as goal_state and write into the "instruction_summary" in the output dictionary. -3. Try to solve the problem, achieve the goal_state from the initial_state based on the domain knowledge provided. Generate a action sequence with the actions defined in and write it into dictionary["task_plan"]["action_sequence"] of the output. -4. If you cannot find a plan to solve the task, ask question with nature language in the "question" of the output dictionary. -5. Explain the actions in the action sequence and write the output dictionary["task_plan"]["action_instructions"]. -6. Collect the objects in the world that are effected by the action sequence, write into dictionary["task_plan"]["manipulated_objects"]. -7. Generate the behavior tree based on the action sequence and write into dictionary["task_plan"]["behavior_tree"]. - -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/cot_sp_skeleton/prompt/cot_domain.txt b/kios_bt_planning/backups/prompts/cot_sp_skeleton/prompt/cot_domain.txt deleted file mode 100644 index bf3c7859..00000000 --- a/kios_bt_planning/backups/prompts/cot_sp_skeleton/prompt/cot_domain.txt +++ /dev/null @@ -1,89 +0,0 @@ -[user] -Following is the domain knowledge you need in pddl domain format, which includes: -- the object types in the world, -- the predicates for describing the states of the world, -- the actions that the robot can apply, with their precondtions and effects defined. -""" -(define (domain robot_assembly_problem-domain) - (:requirements :strips :typing) - (:types - thing - object - tool part hand - thing - ) - (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part)) - (:action pick_up - :parameters ( ?hand - hand ?tool - tool ?part - part) - :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part)) - :effect (and (hold ?tool ?part) (not (is_free ?tool)))) - (:action put_down - :parameters ( ?hand - hand ?tool - tool ?part - part) - :precondition (and (hold ?tool ?part) (hold ?hand ?tool)) - :effect (and (not (hold ?hand ?part)) (is_free ?tool))) - (:action place - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2))) - (:action detach - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2)))) - (:action insert - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2))) - (:action pull - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2)))) - (:action screw - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2))) - (:action unscrew - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2)))) - (:action load_tool - :parameters ( ?hand - hand ?tool - tool) - :precondition (and (is_equippable ?tool) (is_free ?hand)) - :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool))) - (:action unload_tool - :parameters ( ?hand - hand ?tool - tool) - :precondition (and (hold ?hand ?tool) (is_free ?tool)) - :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool)))) -) -""" -The precidates are explained as follows: -""" -"PREDICATE_LIST" -- is_free( or ): or is free and can hold something. (default: true) -- is_equippable(): is ready to be eqquipped by a robot hand. (default: true) -- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false) -- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false) -- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false) -- can_place_to(, ): can be placed into (placement assembly constraint) (default: false) -- hold(, ): is holding (default: false) -- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false) -- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false) -- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false) -""" - -The actions are explained as follows: -""" -"ROBOT_ACTION_LIST" -- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something. -- put_down(, , ): use to put down . As preconditions, should be holding and should be holding . -- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding . -- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into . -- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding . -- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into . -- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding . -- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into . -- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free. -- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free. -""" -the items in <> indicate the type of the instances needed for the predicates and actions. -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. diff --git a/kios_bt_planning/backups/prompts/cot_sp_skeleton/prompt/cot_example.txt b/kios_bt_planning/backups/prompts/cot_sp_skeleton/prompt/cot_example.txt deleted file mode 100644 index 3d8d6d5b..00000000 --- a/kios_bt_planning/backups/prompts/cot_sp_skeleton/prompt/cot_example.txt +++ /dev/null @@ -1,127 +0,0 @@ -[user] -I will give you some examples of the input and the output you will generate. -Example 1: -""" -- Input: -(define (problem robot_assembly_problem-problem) - (:domain robot_assembly_problem-domain) - (:objects - parallel_box1 parallel_box2 inward_claw outward_claw no_tool - tool - gear1 gear2 gear3 shaft1 shaft2 base - part - left_hand - hand - ) - (:init (can_manipulate parallel_box1 gear1) (can_manipulate outward_claw gear2) (can_manipulate inward_claw gear3) (can_manipulate parallel_box2 shaft1) (can_manipulate no_tool shaft2) (can_screw_to leg1 seat) (can_screw_to leg2 seat) (can_insert_to back seat) (can_screw_to nut1 seat) (can_screw_to nut2 seat) (can_screw_to blub base) (can_place_to lamp blub) (can_insert_to shaft1 base) (can_insert_to shaft2 base) (can_insert_to gear3 shaft2) (can_insert_to gear2 base) (can_insert_to gear1 shaft1) (is_inserted_to shaft1 base) (is_free left_hand) (is_free parallel_box1) (is_free parallel_box2) (is_free inward_claw) (is_free outward_claw) (is_free no_tool) (is_equippable parallel_box1) (is_equippable parallel_box2) (is_equippable inward_claw) (is_equippable outward_claw) (is_equippable no_tool)) - (:goal (and (is_inserted_to gear1 shaft1))) -) -- Output: -``` -{"task_plan": { - "action_sequence": [ - "load_tool(left_hand, parallel_box1)", - "pick_up(left_hand, parallel_box1, gear1)", - "insert(left_hand, parallel_box1, gear1, shaft1)" - ], - "action_instructions": [ - "equip the parallel_box1 on the left hand", - "pick up the gear1 with the parallel_box1 in the left hand", - "insert the gear1 into the shaft1 using the parallel_box1 in the left hand" - ], - "object_name": ["left_hand", "gear1", "parallel_box1", "shaft1"], - "behavior_tree": { - "summary": "selector to insert gear1 into shaft1", - "children": [ - { - "summary": "condition node to check if gear1 is inserted to shaft1", - }, - { - "summary": "sequence to insert gear1 into shaft1", - "children": [ - { - "summary": "selector to load_tool - "children": [ - { - "summary": "condition node to check if parallel_box1 is hold", - }, - { - "summary": "sequence to load_tool", - "children": [ - { - "summary": "condition node to check if parallel_box1 is equippable", - }, - { - "summary": "condition node to check if left hand is free", - }, - { - "summary": "action node to equip parallel_box1 to left hand", - "effects": [ - {"summary": "left_hand will be not free",}, - {"summary": "parallel_box1 will be not equippable",}, - {"summary": "left_hand will hold parallel_box1",}, - ], - }, - ], - }, - ], - }, - { - "summary": "selector to pick_up", - "children": [ - { - "summary": "condition node to check if parallel_box1 holds gear1", - }, - { - "summary": "sequence to pick_up", - "children": [ - { - "summary": "condition node to check if parallel_box1 is free", - }, - { - "summary": "condition node to check if left hand holds parallel_box1", - }, - { - "summary": "action node to pick up gear1 using parallel_box1", - "effects": [ - {"summary": "parallel_box1 will hold gear1",}, - {"summary": "parallel_box1 will be not free",}, - ], - }, - ], - }, - ], - }, - { - "summary": "condition node to check if gear1 can be inserted to shaft1", - }, - { - "summary": "action node to insert gear1 to shaft1", - "effects": [ - {"summary": "parallel_box1 will be not holding gear1",}, - {"summary": "parallel_box1 will be free",}, - {"summary": "gear1 will be inserted to shaft1",}, - ], - }, - ], - }, - ], -} -}, -"initial_state": {"objects": [ - {"name": "gear1", "properties": []}, - {"name": "shaft1", "properties": []}, - {"name": "parallel_box1", "properties": ["is_equippable", "is_free"]}, - {"name": "left_hand", "properties": ["is_free"]}, - ], - "constraints": [ - {"source": "gear1", "name": "can_insert_to", "target": "shaft1"}, - {"source": "parallel_box1", "name": "can_manipulate", "target": "gear1"}, - ], - "relations": [], -}, -"instruction_summary": "insert gear1 into shaft1", -"question":""} -``` -""" -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/cot_sp_skeleton/prompt/cot_output_format.txt b/kios_bt_planning/backups/prompts/cot_sp_skeleton/prompt/cot_output_format.txt deleted file mode 100644 index d47c6af5..00000000 --- a/kios_bt_planning/backups/prompts/cot_sp_skeleton/prompt/cot_output_format.txt +++ /dev/null @@ -1,21 +0,0 @@ -[user] -You will be provided with a problem and the necessary domain knowledge. -You generate an action sequence to solve the task and contruct a behavior tree based on it in json format. -The output dictionary has five keys. -""" -- dictionary["task_plan"]: A dictionary containing the task plan. -- dictionary["initial_state"]: The initial world state before executing the plan. -- dictionary["instruction_summary"]: A brief summary of the task plan. -- dictionary["question"]: If you don't have enough knowledge for solving the problem, you can ask questions about the knowledge you need. Leave this key empty if you can solve the task. -""" -Four keys exist in dictionary["task_plan"]. -""" -- dictionary["task_plan"]["action_sequence"]: Contains a list of robot actions. Only the actions defined in the "ROBOT_ACTION LIST" will be used. -- dictionary["task_plan"]["action_instructions"]: contains a list of instructions corresponding to dictionary["task_plan"]["action_sequence"] to explain the actions. -- dictionary["task_plan"]["manipulated_objects"]: The list of the manipulated objects. It only includes the objects that are related to the current task. -- dictionary["task_plan"]["behavior_tree"]: contains a dictionary of the behavior tree constructed according to the dictionary ["task_plan"]["action_sequence"] -""" -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/cot_sp_skeleton/prompt/cot_problem.txt b/kios_bt_planning/backups/prompts/cot_sp_skeleton/prompt/cot_problem.txt deleted file mode 100644 index 658940e7..00000000 --- a/kios_bt_planning/backups/prompts/cot_sp_skeleton/prompt/cot_problem.txt +++ /dev/null @@ -1,27 +0,0 @@ -[user] -Information about the problem is given as a pddl problem. -It includes the following information: -- objects: all the objects in the world. They should be of a type defined in the domain knowledge. For example, "parallel_box1" is an instance of the type "tool", while "gear1" is an instance of the type "part". -- init: the initial world state to start with, described with [PREDICATE_LIST] from domain knowledge. For example, "is_free hand" means the object "hand" is free and holds nothing. -- goal: the world state to achieve, described with [PREDICATE_LIST] from domain knowledge. -An example: -""" -(define (problem robot_assembly_problem-problem) - (:domain robot_assembly_problem-domain) - (:objects - parallel_box1 parallel_box2 inward_claw outward_claw no_tool - tool - gear1 gear2 gear3 shaft1 shaft2 base - part - left_hand - hand - ) - (:init (can_manipulate parallel_box1 gear1) (can_manipulate outward_claw gear2) (can_manipulate inward_claw gear3) (can_manipulate parallel_box2 shaft1) (can_manipulate no_tool shaft2) (can_screw_to leg1 seat) (can_screw_to leg2 seat) (can_insert_to back seat) (can_screw_to nut1 seat) (can_screw_to nut2 seat) (can_screw_to blub base) (can_place_to lamp blub) (can_insert_to shaft1 base) (can_insert_to shaft2 base) (can_insert_to gear3 shaft2) (can_insert_to gear2 base) (can_insert_to gear1 shaft1) (is_free left_hand) (is_free parallel_box1) (is_free parallel_box2) (is_free inward_claw) (is_free outward_claw) (is_free no_tool) (is_equippable parallel_box1) (is_equippable parallel_box2) (is_equippable inward_claw) (is_equippable outward_claw) (is_equippable no_tool)) - (:goal (and (is_inserted_to shaft1 base))) -) -""" - - - - -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/cot_sp_skeleton/prompt/cot_role.txt b/kios_bt_planning/backups/prompts/cot_sp_skeleton/prompt/cot_role.txt deleted file mode 100644 index 6ce736a0..00000000 --- a/kios_bt_planning/backups/prompts/cot_sp_skeleton/prompt/cot_role.txt +++ /dev/null @@ -1,5 +0,0 @@ -[user] -You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you break it down and construct robot task plans represented by behaiovr trees. -Please do not begin working until I say "Start working." Instead, simply output the message "Waiting for next input." Understood? -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/cot_sp_skeleton/prompt/cot_state.txt b/kios_bt_planning/backups/prompts/cot_sp_skeleton/prompt/cot_state.txt deleted file mode 100644 index a8ea9cd6..00000000 --- a/kios_bt_planning/backups/prompts/cot_sp_skeleton/prompt/cot_state.txt +++ /dev/null @@ -1,30 +0,0 @@ -[user] -Information about world states should be in json dictionary format. Example: -""" -{ - "objects": [ - {"name": "leg1", "properties": []}, # leg1 is an object with no properties - {"name": "seat", "properties": []}, # seat is an object with no properties - {"name": "cabinet", "properties": ["open"]}, # cabinet is an object with a property "open" - {"name": "table", "properties": []}, # table is an object with no properties - ], - "constraints": [ - {"source": "leg1", "name": "can_screw_to", "target": "seat"}, # means leg1 can be screwed to seat - {"source": "cabinet", "name": "on_something", "target": "table"}, # means cabinet is on table - ], - "relations": [ - {"source": "leg1", "name": "is_screwed_to", "target": "seat"}, # means leg1 is screwed to seat - ], -} -""" -It contains three lists: - - "objects": the objects and their properties (as lists of strings) in the world. The properties can be changed by robot actions. - - "constraints": the constraint relations that are enforced and should not be changed. They are knowledge about the world or some constraints set by the user. - - "relations": the relations in the world. They can be changed by robot actions. -The properties, constraints and relations should always align with the [PREDICATE_LIST] in domain knowledge. -Only concentrate on the objects, constraints and relations that are related to the current task. - -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/cot_sp_skeleton/query/old_query.txt b/kios_bt_planning/backups/prompts/cot_sp_skeleton/query/old_query.txt deleted file mode 100644 index d9ff871a..00000000 --- a/kios_bt_planning/backups/prompts/cot_sp_skeleton/query/old_query.txt +++ /dev/null @@ -1,16 +0,0 @@ -Start working. Resume from the problem below. -""" -{"problem": [PROBLEM]} -""" -The dictionary that you return should be formatted as json file. Follow these rules: -1. Always equip the proper tools before performing specific object manipulaion. -2. Make sure that each element of the ["action_instructions"] explains corresponding element of the ["action_sequence"]. Refer to the "ROBOT ACTION LIST" to understand the elements of ["action_sequence"]. -3, The length of the ["action_instructions"] list and the ["action_sequence"] list should be the same. -4. Never left ',' at the end of the list. Use json boolean values, "true" or "false", instead of "True" or "False". -5. Keep track of all items listed in the "objects" section of the "initial_state" field. Please make sure you fill out the "objects" section for all listed items. -6. Use the "PREDICATE_LIST" for the initial states (:init) in the problem to generate ["initial_state"]. -7. All keys of the dictionary should be double-quoted. -8. Insert ``` at the beginning and the end of the dictionary to separate it from the rest of your response. Don't write any additional things. -9. Make sure that you output a consistent manipultation as a single arm robot. For example, grasping an object should not occur in successive steps. -10. When generating the behavior tree, make sure the tree has the asked structure. -Adhere to the output format I defined above. Follow the rules above. Think step by step. \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/cot_sp_skeleton/query/query.txt b/kios_bt_planning/backups/prompts/cot_sp_skeleton/query/query.txt deleted file mode 100644 index cfeb4e8e..00000000 --- a/kios_bt_planning/backups/prompts/cot_sp_skeleton/query/query.txt +++ /dev/null @@ -1,16 +0,0 @@ -Start working. Resume from the problem below. -""" -{"problem": [PROBLEM]} -""" -The dictionary that you return should be formatted as json file. Follow these rules: -1. Always equip the proper tools before performing specific object manipulaion. -2. Make sure that each element of the ["action_instructions"] explains corresponding element of the ["action_sequence"]. Refer to the "ROBOT ACTION LIST" to understand the elements of ["action_sequence"]. -3, The length of the ["action_instructions"] list and the ["action_sequence"] list should be the same. -4. Never left ',' at the end of the list. Use json boolean values, "true" or "false", instead of "True" or "False". -5. Keep track of all items listed in the "objects" section of the "initial_state" field. -6. Use the "PREDICATE_LIST" for the initial states (:init) in the problem to generate ["initial_state"]. -7. All keys of the dictionary should be double-quoted. -8. Insert ``` at the beginning and the end of the dictionary to separate it from the rest of your response. Don't write any additional things. -9. Make sure that you output a consistent manipultation as a single arm robot. For example, grasping an object should not occur in successive steps. -10. When generating the behavior tree, make sure the tree has the asked structure. -Adhere to the output format I defined above. Follow the rules above. Think step by step. \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/cot_sp_skeleton/system/system.txt b/kios_bt_planning/backups/prompts/cot_sp_skeleton/system/system.txt deleted file mode 100644 index 26f2afe2..00000000 --- a/kios_bt_planning/backups/prompts/cot_sp_skeleton/system/system.txt +++ /dev/null @@ -1 +0,0 @@ -You are an excellent interpreter of instructions for robotic assembly tasks. Given the knowledge you need, the world states and some instructions, you break it down and construct robot task plans represented by behaiovr trees based on the action sequences. \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/end_to_end_v1/prompt/backup b/kios_bt_planning/backups/prompts/end_to_end_v1/prompt/backup deleted file mode 100644 index c4a0f6f6..00000000 --- a/kios_bt_planning/backups/prompts/end_to_end_v1/prompt/backup +++ /dev/null @@ -1,43 +0,0 @@ - -There are also some counterexamples for you to refer to: -1. One sequence should never have multiple action nodes as its children like the behavior tree below. -{ - "summary": "selector to get the apple", - "name": "Get the apple", - "type_name": "selector", - "children": [ - { - "summary": "check if the apple is in hand", - "name": "target: in_hand(apple)", - "type_name": "condition", - }, - { - "summary": "sequence to get the apple", - "name": "sequence: get apple", - "type_name": "sequence", - "children": [ - { - "summary": "check if the apple is on the table", - "name": "precondition: on(apple, table)", - "type_name": "condition", - }, - { - "summary": "go to the table", - "name": "action: go_to(table)", - "type_name": "action", - }, - { - "summary": "check if the hand is free", - "name": "precondition: is_free(hand)", - "type_name": "condition", - }, - { - "summary": "pick up the apple", - "name": "action: pick_up(apple)", - "type_name": "action", - (effect omitted) - }, - ], - }, - ], -} \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/end_to_end_v1/prompt/old_prompt_example.txt b/kios_bt_planning/backups/prompts/end_to_end_v1/prompt/old_prompt_example.txt deleted file mode 100644 index 8d71b9b6..00000000 --- a/kios_bt_planning/backups/prompts/end_to_end_v1/prompt/old_prompt_example.txt +++ /dev/null @@ -1,232 +0,0 @@ -[user] -I will give you some examples of the input and the output you will generate. -Example 1: -""" -- Input: -{"assets":["
", "", "", "", ""], -"asset_states": {"": "on_something(
)", - "": "on_something()"}, -"objects": ["", ""], -"object_states": {"": "on_something(
)", - "": "on_something()"}, -"instruction": "Put the juice on top of the shelf"} -- Output: -``` -{"task_cohesion": { - "task_sequence": [ - "move_hand()", - "grasp_object()", - "detach_from_plane()", - "move_object()", - "move_object()", - "move_object()", - "attach_to_plane()", - "release_object()" - ], - "step_instructions": [ - "move the hand near the juice", - "grasp the juice", - "pick up the juice", - "take the juice from the bottom of the shelf", - "move the juice upward", - "move the juice above the upper shelf", - "place the juice", - "release the juice" - ], - "object_name": ""}, -"environment_before": {"assets":["
", "", "", "", ""], - "asset_states": {"": "on_something(
)", - "": "on_something()"}, - "objects": ["", ""], - "object_states": {"": "on_something(
)", - "": "on_something()"}}, -"environment_after": {"assets":["
", "", "", "", ""], - "asset_states": {"": "on_something(
)", - "": "on_something()"}, - "objects": ["", ""], - "object_states": {"": "on_something(
)", - "": "on_something()"}}, -"instruction_summary": "put the juice on top of the shelf", -"question":""} -``` -""" -Example 2: -""" -- Input: -{"assets":["
", "", "", "", ""], -"asset_states": {"": "on_something(
)", - "": "on_something()"}, -"objects": ["", ""], -"object_states": {"": "on_something(
)", - "": "on_something()"}, -"instruction": "Throw away the spam into the trash bin."} -- Output: -``` -{"task_cohesion": { - "task_sequence": [ - "move_hand()", - "grasp_object()", - "detach_from_plane()", - "move_object()", - "release_object()" - ], - "step_instructions": [ - "move your hand near the spam", - "grasp the spam", - "pick up the spam", - "bring the spam above the trash bin", - "release the spam to drop it in the trash bin" - ], - "object_name": ""}, -"environment_before": {"assets":["
", "", "", "", ""], - "asset_states": {"": "on_something(
)", - "": "on_something()"}, - "objects": ["", ""], - "object_states": {"": "on_something(
)", - "": "on_something()"}}, -"environment_after": {"assets":["
", "", "", "", ""], - "asset_states": {"": "on_something(
)", - "": "on_something()"}, - "objects": ["", ""], - "object_states": {"": "inside_something()", - "": "on_something()"}}, -"instruction_summary": "throw away the spam", -"question":""} -``` -""" -Example 3: -""" --Input: -{"assets":["", ""], -"asset_states": {"": "on_something()"}, -"objects": [""], -"object_states": {"": "closed()"}, -"instruction": "Open the fridge."} -- Output: -``` -{"task_cohesion": { - "task_sequence": [ - "move_hand()", - "grasp_object()", - "open_by_rotate()", - "release_object()" - ], - "step_instructions": [ - "move the hand near the fridge handle", - "grasp the handle", - "open the fridge by pulling the handle", - "release the handle" - ], - "object_name": ""}, -"environment_before": {"assets":["", ""], - "asset_states": {"": "on_something()"}, - "objects": [""], - "object_states": {"": "closed()"}}, -"environment_after": {"assets":["", ""], - "asset_states": {"": "on_something()"}, - "objects": [""], - "object_states": {"": "open()"}}, -"instruction_summary": "open a fridge", -"question":""} -``` -""" -Example 4: -""" --Input: -{"assets":["
", ""], -"asset_states": {"
": "next_to()"}, -"objects": [""], -"object_states": {"": "on_something(
)"}, -"instruction": "Wipe the window with the sponge. Then, put the sponge on the table."} -- Output: -``` -{"task_cohesion": { - "task_sequence": [ - "move_hand()", - "grasp_object()", - "detach_from_plane()", - "move_object()", - "attach_to_plane()", - "wipe_on_plane()", - "detach_from_plane()", - "move_object()", - "attach_to_plane()", - "release_object()" - ], - "step_instructions": [ - "move the hand near the sponge", - "grasp the sponge", - "pick up the sponge", - "bring the sponge above the window", - "attach the sponge to the window", - "wipe the window with the sponge", - "detach the sponge from the window", - "bring the sponge above the table", - "put down the sponge", - "release the sponge" - ], - "object_name": ""}, -"environment_before": {"assets":["
", ""], - "asset_states": {"
": "next_to()"}, - "objects": [""], - "object_states": {"": "on_something(
)"},} -"environment_after": {"assets":["
", ""], - "asset_states": {"
": "next_to()"}, - "objects": [""], - "object_states": {"": "on_something(
)"}}, -"instruction_summary": "wipe the window with the sponge, then put the sponge on the table", -"question":""} -``` -""" -Example 5: -""" --Input: -{"assets":["
"], -"asset_states": {"": "can_manipulate()"}, -"objects": ["", "", ""], -"object_states": {"": "on_something(
)", "": "on_something(
), "": "can_manipulate()"}, -"instruction": "screw the nut into the bolt. Tool should be used and be put back at the end."} -- Output: -``` -{"task_cohesion": { - "task_sequence": [ - "load_tool()", - "approach()", - "grasp_object()", - "pick_up()", - "approach()", - "insert()", - "screw()", - "release_object()", - "unload_tool()" - ], - "step_instructions": [ - "load the parallel_toolbox", - "move the hand close to the nut", - "grasp the nut", - "pick up the nut", - "move the hand close to the bolt", - "insert the nut onto the bolt", - "screw the nut into the bold", - "release the nut", - "unload the parallel_toolbox" - ], - "object_name": ""}, -"environment_before": {"assets":["
"], - "asset_states": {"": "can_manipulate()"}, - "objects": ["", "", ""], - "object_states": {"": "on_something(
)", "": "on_something(
)"},} -"environment_after": {"assets":["
"], - "asset_states": {"": "can_manipulate()"}, - "objects": ["", "", ""], - "object_states": {"": "screwed_into()", "": "on_something(
)"}, -"instruction_summary": "screw the nut into the bolt. Tool should be used and be put back at the end.", -"question":""} -} -``` -""" -From these examples, learn that some robotic actions have dependencies with the actions before and after them. -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/end_to_end_v1/prompt/old_prompt_output_format.txt b/kios_bt_planning/backups/prompts/end_to_end_v1/prompt/old_prompt_output_format.txt deleted file mode 100644 index 7be3d38b..00000000 --- a/kios_bt_planning/backups/prompts/end_to_end_v1/prompt/old_prompt_output_format.txt +++ /dev/null @@ -1,20 +0,0 @@ -[user] -You divide the actions given in the text into detailed robot actions and put them together as a python dictionary. -The dictionary has five keys. -""" -- dictionary["task_cohesion"]: A dictionary containing information about the robot's actions that have been split up. -- dictionary["environment_before"]: The state of the environment before the manipulation. -- dictionary["environment_after"]: The state of the environment after the manipulation. -- dictionary["instruction_summary"]: contains a brief summary of the given sentence. -- dictionary["question"]: If you cannot understand the given sentence, you can ask the user to rephrase the sentence. Leave this key empty if you can understand the given sentence. -""" -Three keys exist in dictionary["task_cohesion"]. -""" -- dictionary["task_cohesion"]["task_sequence"]: Contains a list of robot actions. Only the behaviors defined in the "ROBOT ACTION LIST" will be used. -- dictionary["task_cohesion"]["step_instructions"]: contains a list of instructions corresponding to dictionary["task_cohesion"]["task_sequence"]. -- dictionary["task_cohesion"]["object_name"]: The name of the manipulated object. Only objects defined in the input dictionary will be used for the object name. -""" -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/end_to_end_v1/prompt/oldprompt_behaviortree.txt b/kios_bt_planning/backups/prompts/end_to_end_v1/prompt/oldprompt_behaviortree.txt deleted file mode 100644 index 585ecc2f..00000000 --- a/kios_bt_planning/backups/prompts/end_to_end_v1/prompt/oldprompt_behaviortree.txt +++ /dev/null @@ -1,170 +0,0 @@ -[user] -The behavior tree should be in python dictionary format and have a structure like the following example: -""" -{ - "name": "Pick Up Apple", - "identifier": 0, - "type_name": "selector", - "children": [ - { - "summary": "check if the apple is in hand", - "name": "check apple in hand", - "identifier": 1, - "type_name": "condition", - "conditions": [ - { - "object_name": "apple", - "property_name": "in", - "property_value": "hand", - "status": True, - } - ], - }, - { - "name": "Pick Up Sequence", - "identifier": 2, - "type_name": "sequence", - "children": [ - { - "summary": "check if the apple is on the ground", - "name": "check apple on the ground", - "identifier": 3, - "type_name": "condition", - "conditions": [ - { - "object_name": "apple", - "property_name": "on_the_ground", - "property_value": None, - "status": True, - } - ], - }, - { - "summary": "check if the hand is free", - "name": "check hand free", - "identifier": 4, - "type_name": "condition", - "conditions": [ - { - "object_name": "hand", - "property_name": "free", - "property_value": None, - "status": True, - } - ], - }, - { - "summary": "pick up the apple", - "name": "pick_up(apple)", - "identifier": 5, - "type_name": "action", - "effects": [ - { - "object_name": "apple", - "property_name": "on_the_ground", - "property_value": None, - "status": False, - }, - { - "object_name": "apple", - "property_name": "in", - "property_value": "hand", - "status": True, - }, - { - "object_name": "hand", - "property_name": "free", - "property_value": None, - "status": False, - }, - ], - }, - ], - }, - ], -} -""" -The dictionary that you return should be formatted as python dictionary. Follow these rules: -1. Every node should have three basic keys: - - "name": the name of the node (for action node, the name should be the same as defined in the domain knowledge) - - "identifier": the identifier of the node, which should be unique - - "type_name": the type of the node, its value can be: "selector", "sequence", "condition", "action" -2. The "selector" and "sequence" nodes can (and should) have a key "children" with a list of children nodes. -3. The "action" and "condition" nodes should have a key "summary" with a short description of the action/condition. -4. The "condition" nodes should have a key "conditions" with a list of conditions to be checked. Each condition should have the following keys: - - "object_name": the object to check - - "property_name": the property/relation of the object to be checked - - "property_value": this should be the other object if this is a relation. For property it should be None - - "status": the boolean status of the property/relation the condition expects, True if the property/relation should be True, False if the property/relation should be False -5. The "action" nodes should have a key "effects" with a list of effects to be exerted after the execution of the action. Each effect should have the following keys: - - "object_name": the object to be affected - - "property_name": the property/relation of the object to be affected - - "property_value": this should be the other object if this is a relation. For property it should be None - - "status": True if the effect is to add this property/relation. False if is to remove. -6. The behavior tree is used to represent an action in the action sequence. Its basic structure is: - - A selector as the root node. - - (Usually one) (Goal-)Condition node as the first child of the selector. It should check the goal conditions the action intends to achieve. So if the goal condition is met, the selector will directly return success. - - A sequence node as the last child of the selector node. The aim of the sequence is to achieve the goal conditions. - - (Pre-)Condition nodes as the front children of the sequence node. They should check the preconditions of the action, which is the last child node of the sequence node. So if any of the preconditions is not met, the sequence node will return failure. - - Only one action node as the last child of the sequence node. Its preconditions are checked by the condition nodes in the front of it. Its effects are exerted after the execution of the action, and should be able to satisfy the goal condition(s), which is the first child of the selector node. -7. When there is any (Pre-)condition node that is not satisfied, a behavior tree can be constructed according to the rules above, which replaces the unsatisfied (Pre-)condition, takes that (Pre-)condition as the (Goal-)condition to achieve and includes the action to achieve it. -8. The selector can have multiple (goal-)condition nodes as children, but must have only and no more than one sequence node as the last child. -9. The sequence node can have multiple (pre-)condition nodes (or selectors) as children, but must have only and no more than one action node as the last child. -10. The behavior tree should be constructed based on the action sequence you planned and the action definitions from the domain knowledge. The depth of the behavior tree is not limited, and when being expanded, it should expanded in a vertical way instead of inserting new actions into the old sequence. - -There are also some counterexamples for you to refer to: -1. One sequence should never have multiple action nodes as its children like the behavior tree below. -{ - "name": "Get the apple", - "identifier": 0, - "type_name": "selector", - "children": [ - { - "summary": "check if the apple is in hand", - "name": "check apple in hand", - "identifier": 1, - "type_name": "condition", - (condition omitted) - }, - { - "name": "Pick Up Sequence", - "identifier": 2, - "type_name": "sequence", - "children": [ - { - "summary": "check if the apple is on the table", - "name": "check apple on the table", - "identifier": 3, - "type_name": "condition", - (condition omitted) - }, - { - "summary": "go to the table", - "name": "go_to(table)", - "identifier": 4, - "type_name": "action", - (effect omitted) - }, - { - "summary": "check if the hand is free", - "name": "check hand free", - "identifier": 5, - "type_name": "condition", - (condition omitted) - }, - { - "summary": "pick up the apple", - "name": "pick_up(apple)", - "identifier": 6, - "type_name": "action", - (effect omitted) - }, - ], - }, - ], -} - -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/end_to_end_v1/prompt/oldprompt_environment.txt b/kios_bt_planning/backups/prompts/end_to_end_v1/prompt/oldprompt_environment.txt deleted file mode 100644 index ac8293ab..00000000 --- a/kios_bt_planning/backups/prompts/end_to_end_v1/prompt/oldprompt_environment.txt +++ /dev/null @@ -1,28 +0,0 @@ -[user] -Information about world states are given as python dictionary. Example: -""" -{"environment":{"assets":["
", "", "", "", ""], - "asset_states": {"": "on_something(
)", - "": "on_something()"}, - "objects": ["", ""], - "object_states": {"": "on_something(
)", - "": "on_something()"}}} -""" -Asset states and object states are represented using those state sets: -""" -"STATE LIST" -- on_something(): Object is located on -- inside_something(): Object is located inside -- inside_hand(): Object is being grasped by a robot hand -- closed(): Object can be opened -- open(): Object can be closed or kept opened -- can_manipulate(): Object can be used to grasp and manipulate -- equipped(): Object is being equipped by the robot hand. -- screwed_into(): Object is screwed into by robot hand -- inserted_into(): Object is inserted into by robot hand -""" - should be one of the assets or objects in the environment. -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/end_to_end_v1/prompt/prompt_behaviortree.txt b/kios_bt_planning/backups/prompts/end_to_end_v1/prompt/prompt_behaviortree.txt deleted file mode 100644 index 6a88afae..00000000 --- a/kios_bt_planning/backups/prompts/end_to_end_v1/prompt/prompt_behaviortree.txt +++ /dev/null @@ -1,189 +0,0 @@ -[user] -The behavior tree should be in json dictionary format and have a structure like the following example: -""" -{ - "name": "load_tool selector", - "identifier": 3, - "type_name": "selector", - "children": [ - { - "summary": "check if inward_claw is held by left hand", - "name": "check left_hand hold inward_claw", - "identifier": 4, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "inward_claw", - "status": True, - } - ], - }, - { - "name": "load_tool sequence", - "identifier": 5, - "type_name": "sequence", - "children": [ - { - "summary": "check if inward_claw is equippable", - "name": "check inward_claw is_equippable", - "identifier": 6, - "type_name": "condition", - "conditions": [ - { - "object_name": "inward_claw", - "property_name": "is_equippable", - "property_value": None, - "status": True, - } - ], - }, - { - "summary": "check if left hand is free", - "name": "check left_hand is_free", - "identifier": 7, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "is_free", - "property_value": None, - "status": True, - } - ], - }, - { - "summary": "equip inward_claw to left hand", - "name": "load_tool(left_hand, inward_claw)", - "identifier": 8, - "type_name": "action", - "effects": [ - { - "object_name": "left_hand", - "property_name": "is_free", - "property_value": None, - "status": False, - }, - { - "object_name": "inward_claw", - "property_name": "is_equippable", - "property_value": None, - "status": False, - }, - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "inward_claw", - "status": True, - }, - ], - }, - ], - }, - ], -} -""" -The dictionary that you return should be formatted as json dictionary. Follow these rules: -1. Every node should have three basic keys: - - "name": the name of the node (for action node, the name should be the same as defined in the domain knowledge) - - "identifier": the unique id of the node - - "type_name": node_type, can be: "selector", "sequence", "condition", "action" -2. The control flow node, "selector" and "sequence" nodes, can have "children", which is a list of children nodes. -3. The "action" and "condition" nodes should have "summary", which is a short description of the action/condition. -4. The "condition" nodes should have "conditions", which is a list of conditions to be checked. Each condition should have the following keys: - - "object_name": the object to check - - "property_name": the property/relation of the object to be checked - - "property_value": this should be the other object if this is a relation. For property it should be None - - "status": the boolean status of the property/relation the condition expects, True if the property/relation should be True, False otherwise -5. The "action" nodes should have a key "effects" with a list of effects to be exerted after the execution of the action. Each effect should have the following keys: - - "object_name": the affected object - - "property_name": the affected property/relation of the object - - "property_value": this should be the other object if this is a relation. For property it should be None - - "status": True if the effect is to add this property/relation. False if is to remove. -6. All the subtrees always have their corresponding action in ROBOT_ACTION_LIST. They are used to represent the actions in the action sequence. Its basic structure is: - - A selector as the root node. - - (Usually one) (Goal-)Condition node as the first child of the selector. It should check the goal conditions the action intends to achieve. So if the goal condition is met, the selector will directly return success. - - A sequence node as the last child of the selector node. The aim of the sequence is to achieve the goal conditions. - - (Pre-)Condition nodes as the front children of the sequence node. They should check the preconditions of the action node after them. - - Only one action node as the last child of the sequence node. Its preconditions are checked by the condition nodes in the front of it. Its effects are exerted after the execution of the action, and should be able to satisfy the goal condition(s). - A brief example: -{ - selector: load_tool selector, - "children": [ - { - (goal)condition: check if inward_claw is equipped by left hand, - }, - { - sequence: load_tool sequence, - children: [ - (pre)condition: check if inward_claw is equippable, (if not satisfied, this condition node can be expanded. A new subtree can be constructed to satisfy this) - (pre)condition: check if left hand is free, (if not satisfied, this condition node can be expanded. A new subtree can be constructed to satisfy this) - (more preconditions can be added here), - action: equip inward_claw to left hand, - ], - }, - ], -} - -7. When there is any (Pre-)condition node that is not satisfied, a behavior tree can be constructed according to the rules above, which replaces the unsatisfied (Pre-)condition, takes that (Pre-)condition as the (Goal-)condition to achieve and includes the action to achieve it. -8. The selector can have multiple (goal-)condition nodes as children, but must have only and no more than one sequence node as the last child. -9. The sequence node can have multiple (pre-)condition nodes (or selectors) as children, but must have only and no more than one action node as the last child. -10. The behavior tree should be constructed based on the action sequence you planned and the action definitions in ROBOT_ACTION_LIST. - -There are also some counterexamples for you to refer to: -1. One sequence should never have multiple action nodes as its children like the behavior tree below. -{ - "name": "Get the apple", - "identifier": 0, - "type_name": "selector", - "children": [ - { - "summary": "check if the apple is in hand", - "name": "check apple in hand", - "identifier": 1, - "type_name": "condition", - (condition omitted) - }, - { - "name": "Pick Up Sequence", - "identifier": 2, - "type_name": "sequence", - "children": [ - { - "summary": "check if the apple is on the table", - "name": "check apple on the table", - "identifier": 3, - "type_name": "condition", - (condition omitted) - }, - { - "summary": "go to the table", - "name": "go_to(table)", - "identifier": 4, - "type_name": "action", - (effect omitted) - }, - { - "summary": "check if the hand is free", - "name": "check hand free", - "identifier": 5, - "type_name": "condition", - (condition omitted) - }, - { - "summary": "pick up the apple", - "name": "pick_up(apple)", - "identifier": 6, - "type_name": "action", - (effect omitted) - }, - ], - }, - ], -} - -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/end_to_end_v1/prompt/prompt_bt_skeleton.txt b/kios_bt_planning/backups/prompts/end_to_end_v1/prompt/prompt_bt_skeleton.txt deleted file mode 100644 index b1225fcc..00000000 --- a/kios_bt_planning/backups/prompts/end_to_end_v1/prompt/prompt_bt_skeleton.txt +++ /dev/null @@ -1,83 +0,0 @@ -[user] -The behavior tree should be in json dictionary format and have a structure like the following example: -""" -{ - "summary": "selector to load inward_claw to left hand", - "name": "selector: load_tool", - "type_name": "selector", - "children": [ - { - "summary": "check if inward_claw is held by left hand", - "name": "target: hold(left_hand, inward_claw)", - "type_name": "condition", - }, - { - "summary": "sequence to load inward_claw to left hand", - "name": "sequence: load_tool", - "type_name": "sequence", - "children": [ - { - "summary": "check if inward_claw is equippable", - "name": "precondition: is_equippable(inward_claw)", - "type_name": "condition", - }, - { - "summary": "check if left hand is free", - "name": "precondition: is_free(left_hand)", - "type_name": "condition", - }, - { - "summary": "equip inward_claw to left hand", - "name": "action: load_tool(left_hand, inward_claw)", - "type_name": "action", - "effects": [ - {"summary": "left hand will be not free anymore",}, - {"summary": "inward_claw will be not equippable anymore",}, - {"summary": "left_hand will hold inward_claw",}, - ], - }, - ], - }, - ], -} -""" -The dictionary that you return should be formatted as json dictionary. Follow these rules: -1. Every node should have three basic keys: - - "summary": a short interpretive description of the node - - "name": the name of the node. The name should follow the definitions in the domain knowledge. - - "type_name": can be: "selector", "sequence", "condition", "action" -2. The control flow nodes, "selector" and "sequence", can have "children", which is a list of children nodes. -3. The "condition" nodes that are children of "selectors" are "targets". Those that are children of "sequences" are preconditions. -4. The "action" nodes should have a key "effects" with a list of dictionaries, which describes the to-be-exerted effects of the action. The effects are shortly described under the key "summary". -5. All the subtrees always have their corresponding action in ROBOT_ACTION_LIST. They are used to represent the actions in the action sequence. The recommended structure is: - - A selector as the root node. - - (Usually one) target condition node as the first child of the selector to check if the target of the action has been fulfilled. - - A sequence node as the last child of the selector node to achieve the target. - - (Usually a number of) precondition nodes as the front children of the sequence node to check the preconditions for executing the action. - - One action node as the last child of the sequence node, whose preconditions are checked before and will immediate start the job when being ticked. - - The action will exert its defined effects after finishing the job successfully. Those effects should fulfilled all the target conditions in the selector node. - A brief example is as follows: -{ - selector: load_tool selector, - children: [ - { - target: check if inward_claw is equipped by left hand, - }, - { - sequence: load_tool, - children: [ - precondition: check if inward_claw is equippable, - precondition: check if left hand is free, - {action: equip inward_claw to left hand, effects: [left hand not free, inward_claw not equippable, left_hand holds inward_claw]}, - ], - }, - ], -} - -6. When performing a new action to fulfill a unsatisfied precondition, you can construct a subtree to replace the precondition and take that precondition as the target of the subtree. Then you construct the sequence of the subtree according to the new action you want to perform. -7. All of the control flow nodes, namely selectors and sequences, are without memory. A new behavior tree tick will always start from the root and traverse all the nodes even if they are ticked in the last tick. - -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/end_to_end_v1/prompt/prompt_bt_skeleton_example.txt b/kios_bt_planning/backups/prompts/end_to_end_v1/prompt/prompt_bt_skeleton_example.txt deleted file mode 100644 index e84cdfb9..00000000 --- a/kios_bt_planning/backups/prompts/end_to_end_v1/prompt/prompt_bt_skeleton_example.txt +++ /dev/null @@ -1,195 +0,0 @@ -[user] -I will give you some examples of the input and the output you will generate. -Example 1: -""" -- Input: -(define (problem robot_assembly_problem-problem) - (:domain robot_assembly_problem-domain) - (:objects - parallel_box1 parallel_box2 inward_claw outward_claw no_tool - tool - gear1 gear2 gear3 shaft1 shaft2 base - part - left_hand - hand - ) - (:init (can_manipulate parallel_box1 gear1) (can_manipulate outward_claw gear2) (can_manipulate inward_claw gear3) (can_manipulate parallel_box2 shaft1) (can_manipulate no_tool shaft2) (can_screw_to leg1 seat) (can_screw_to leg2 seat) (can_insert_to back seat) (can_screw_to nut1 seat) (can_screw_to nut2 seat) (can_screw_to blub base) (can_place_to lamp blub) (can_insert_to shaft1 base) (can_insert_to shaft2 base) (can_insert_to gear3 shaft2) (can_insert_to gear2 base) (can_insert_to gear1 shaft1) (is_inserted_to shaft1 base) (is_free left_hand) (is_free parallel_box1) (is_free parallel_box2) (is_free inward_claw) (is_free outward_claw) (is_free no_tool) (is_free gear1) (is_free gear2) (is_free gear3) (is_free shaft1) (is_free shaft2) (is_free base) (is_equippable parallel_box1) (is_equippable parallel_box2) (is_equippable inward_claw) (is_equippable outward_claw) (is_equippable no_tool)) - (:goal (and (is_inserted_to gear1 shaft1))) -) -- Output: -``` -{"task_plan": { - "action_sequence": [ - "load_tool(left_hand, parallel_box1)", - "pick_up(left_hand, parallel_box1, gear1)", - "insert(left_hand, parallel_box1, gear1, shaft1)" - ], - "action_instructions": [ - "equip the parallel_box1 on the left hand", - "pick up the gear1 with the left hand", - "insert the gear1 into the shaft1 using the left hand with the parallel_box1" - ], - "object_name": ["left_hand", "gear1", "parallel_box1", "shaft1"], - "behavior_tree": { - "summary": "selector to insert gear1 into shaft1", - "name": "selector: insert gear1 into shaft1", - "type_name": "selector", - "children": [ - { - "summary": "check if gear1 is inserted to shaft1", - "name": "target: is_inserted_to(gear1, shaft1)", - "type_name": "condition", - }, - { - "summary": "sequence to insert gear1 into shaft1", - "name": "sequence: insert(gear1, shaft1)", - "type_name": "sequence", - "children": [ - { - "summary": "selector to load_tool - "name": "load_tool selector", - "type_name": "selector", - "children": [ - { - "summary": "check if parallel_box1 is hold", - "name": "target: hold(left_hand, parallel_box1)", - "type_name": "condition", - }, - { - "summary": "sequence to load_tool", - "name": "sequence: load_tool", - "type_name": "sequence", - "children": [ - { - "summary": "check if parallel_box1 is equippable", - "name": "precondition: is_equippable(parallel_box1)", - "type_name": "condition", - }, - { - "summary": "check if left hand is free", - "name": "precondition: is_free(left_hand)", - "type_name": "condition", - }, - { - "summary": "equip parallel_box1 to left hand", - "name": "action: load_tool(left_hand, parallel_box1)", - "type_name": "action", - "effects": [ - { - "summary": "left_hand will be not free", - }, - { - "summary": "parallel_box1 will be not equippable", - }, - { - "summary": "left_hand will hold parallel_box1", - }, - ], - }, - ], - }, - ], - }, - { - "summary": "selector to pick_up", - "name": "selector: pick_up", - "type_name": "selector", - "children": [ - { - "summary": "check if parallel_box1 holds gear1", - "name": "precondition: hold(parallel_box1, gear1)", - "type_name": "condition", - }, - { - "summary": "sequence to pick_up", - "name": "sequence: pick_up", - "type_name": "sequence", - "children": [ - { - "summary": "check if parallel_box1 is free", - "name": "precondition: is_free(parallel_box1)", - "type_name": "condition", - }, - { - "summary": "check if left hand holds parallel_box1", - "name": "precondition: hold(left_hand, parallel_box1)", - "type_name": "condition", - }, - { - "summary": "pick up gear1 using parallel_box1", - "name": "action: pick_up(left_hand, parallel_box1, gear1)", - "type_name": "action", - "effects": [ - { - "summary": "parallel_box1 will hold gear1", - }, - { - "summary": "parallel_box1 will be not free", - }, - ], - }, - ], - }, - ], - }, - { - "summary": "check if gear1 can be inserted to shaft1", - "name": "precondition: can_insert_to(gear1, shaft1)", - "type_name": "condition", - }, - { - "summary": "insert gear1 to shaft1", - "name": "action: insert(left_hand, parallel_box1, gear1, shaft1)", - "type_name": "action", - "effects": [ - { - "summary": "parallel_box1 will be not holding gear1", - }, - { - "summary": "parallel_box1 will be free", - }, - { - "summary": "gear1 will be inserted to shaft1", - }, - ], - }, - ], - }, - ], -} -}, -"environment_before": {"objects": [ - {"name": "gear1", "properties": []}, - {"name": "shaft1", "properties": []}, - {"name": "parallel_box1", "properties": ["is_equippable", "is_free"]}, - {"name": "left_hand", "properties": ["is_free"]}, - ], - "constraints": [ - {"source": "gear1", "name": "can_insert_to", "target": "shaft1"}, - {"source": "parallel_box1", "name": "can_manipulate", "target": "gear1"}, - ], - "relations": [ - - ], -}, -"environment_after": {"objects": [ - {"name": "gear1", "properties": []}, - {"name": "shaft1", "properties": []}, - {"name": "parallel_box1", "properties": ["is_equippable", "is_free"]}, - {"name": "left_hand", "properties": []}, - ], - "constraints": [ - {"source": "gear1", "name": "can_insert_to", "target": "shaft1"}, - {"source": "parallel_box1", "name": "can_manipulate", "target": "gear1"}, - ], - "relations": [ - {"source": "gear1", "name": "is_inserted_to", "target": "shaft1"}, - {"source": "left_hand", "name": "hold", "target": "parallel_box1"}, - ], -}, -"instruction_summary": "insert gear1 into shaft1", -"question":""} -``` -""" -From these examples, learn that some robotic actions have dependencies with the actions before and after them. -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/end_to_end_v1/prompt/prompt_bt_validation.txt b/kios_bt_planning/backups/prompts/end_to_end_v1/prompt/prompt_bt_validation.txt deleted file mode 100644 index e69de29b..00000000 diff --git a/kios_bt_planning/backups/prompts/end_to_end_v1/prompt/prompt_domain.txt b/kios_bt_planning/backups/prompts/end_to_end_v1/prompt/prompt_domain.txt deleted file mode 100644 index 670f1581..00000000 --- a/kios_bt_planning/backups/prompts/end_to_end_v1/prompt/prompt_domain.txt +++ /dev/null @@ -1,89 +0,0 @@ -[user] -Following is the domain knowledge you need in pddl domain format, which includes: -- the object types in the world, -- the predicates for describing the states of the world, -- the actions that the robot can apply, with their precondtions and effects defined. -""" -(define (domain robot_assembly_problem-domain) - (:requirements :strips :typing) - (:types - thing - object - tool part hand - thing - ) - (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part)) - (:action pick_up - :parameters ( ?hand - hand ?tool - tool ?part - part) - :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part)) - :effect (and (hold ?tool ?part) (not (is_free ?tool)))) - (:action put_down - :parameters ( ?hand - hand ?tool - tool ?part - part) - :precondition (and (hold ?tool ?part) (hold ?hand ?tool)) - :effect (and (not (hold ?hand ?part)) (is_free ?tool))) - (:action place - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2))) - (:action detach - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2)))) - (:action insert - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2))) - (:action pull - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2)))) - (:action screw - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2))) - (:action unscrew - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2)))) - (:action load_tool - :parameters ( ?hand - hand ?tool - tool) - :precondition (and (is_equippable ?tool) (is_free ?hand)) - :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool))) - (:action unload_tool - :parameters ( ?hand - hand ?tool - tool) - :precondition (and (hold ?hand ?tool) (is_free ?tool)) - :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool)))) -) -""" -The precidates are explained as follows: -""" -"PREDICATE_LIST" -- is_free(): is free and can hold something. (default: true) -- is_equippable(): is ready to be eqquipped by a robot hand. (default: true) -- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false) -- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false) -- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false) -- can_place_to(, ): can be placed into (placement assembly constraint) (default: false) -- hold(, ): is holding (default: false) -- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false) -- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false) -- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false) -""" - -The actions are explained as follows: -""" -"ROBOT_ACTION_LIST" -- pick_up(, , ): use to pick up -- put_down(, , ): use to put down -- place(, , , ): use to place to (to satisfy the placement assembly constraint) -- detach(, , , ): use to detach from (to lift the existing placement assembly constraint) -- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint) -- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint) -- screw(, , , ): use to screw into (to satisfy the screw assembly constraint) -- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint) -- load_tool(, ): equips to manipulate other objects -- unload_tool(, ): unloads to manipulate other objects -""" -the items in <> indicate the type of the instances needed for the predicates and actions. -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. diff --git a/kios_bt_planning/backups/prompts/end_to_end_v1/prompt/prompt_environment.txt b/kios_bt_planning/backups/prompts/end_to_end_v1/prompt/prompt_environment.txt deleted file mode 100644 index 4cd08f12..00000000 --- a/kios_bt_planning/backups/prompts/end_to_end_v1/prompt/prompt_environment.txt +++ /dev/null @@ -1,31 +0,0 @@ -[user] -Information about world states are given as json dictionary. Example: -""" -{ - "objects": [ - {"name": "leg1", "properties": []}, - {"name": "seat", "properties": []}, - {"name": "cabinet", "properties": ["open"]}, - {"name": "parallel_box1", "properties": []}, - {"name": "table", "properties": []}, - ], - "constraints": [ - {"source": "leg1", "name": "can_screw_to", "target": "seat"}, - {"source": "parallel_box1", "name": "can_manipulate", "target": "leg1"}, - {"source": "cabinet", "name": "on_something", "target": "table"}, - ], - "relations": [ - {"source": "leg1", "name": "is_screwed_to", "target": "seat"}, - ], -} -""" -It contains three lists: - - "objects": the objects and their properties (as lists of strings) in the environment. The properties can be changed by robot actions. - - "constraints": the constraint relations that are enforced and should not be changed. They are knowledge about the world or some constraints set by the user. - - "relations": the relations in the environment. They can be changed by robot actions. -The properties, constraints and relations should always align with the [PREDICATE_LIST] in domain knowledge. -You should only concentrate on the objects, constraints and relations that are related to (or can be effected by) the current task. -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/end_to_end_v1/prompt/prompt_example.txt b/kios_bt_planning/backups/prompts/end_to_end_v1/prompt/prompt_example.txt deleted file mode 100644 index a079e986..00000000 --- a/kios_bt_planning/backups/prompts/end_to_end_v1/prompt/prompt_example.txt +++ /dev/null @@ -1,294 +0,0 @@ -[user] -I will give you some examples of the input and the output you will generate. -Example 1: -""" -- Input: -(define (problem robot_assembly_problem-problem) - (:domain robot_assembly_problem-domain) - (:objects - parallel_box1 parallel_box2 inward_claw outward_claw no_tool - tool - gear1 gear2 gear3 shaft1 shaft2 base - part - left_hand - hand - ) - (:init (can_manipulate parallel_box1 gear1) (can_manipulate outward_claw gear2) (can_manipulate inward_claw gear3) (can_manipulate parallel_box2 shaft1) (can_manipulate no_tool shaft2) (can_screw_to leg1 seat) (can_screw_to leg2 seat) (can_insert_to back seat) (can_screw_to nut1 seat) (can_screw_to nut2 seat) (can_screw_to blub base) (can_place_to lamp blub) (can_insert_to shaft1 base) (can_insert_to shaft2 base) (can_insert_to gear3 shaft2) (can_insert_to gear2 base) (can_insert_to gear1 shaft1) (is_inserted_to shaft1 base) (is_free left_hand) (is_free parallel_box1) (is_free parallel_box2) (is_free inward_claw) (is_free outward_claw) (is_free no_tool) (is_free gear1) (is_free gear2) (is_free gear3) (is_free shaft1) (is_free shaft2) (is_free base) (is_equippable parallel_box1) (is_equippable parallel_box2) (is_equippable inward_claw) (is_equippable outward_claw) (is_equippable no_tool)) - (:goal (and (is_inserted_to gear1 shaft1))) -) -- Output: -``` -{"task_cohesion": { - "action_sequence": [ - "load_tool(left_hand, parallel_box1)", - "pick_up(left_hand, parallel_box1, gear1)", - "insert(left_hand, parallel_box1, gear1, shaft1)" - ], - "step_instructions": [ - "equip the parallel_box1 on the left hand", - "pick up the gear1 with the left hand", - "insert the gear1 into the shaft1 using the left hand with the parallel_box1" - ], - "object_name": ["left_hand", "gear1", "parallel_box1", "shaft1"], - "behavior_tree": { - "name": "insert selector", - "identifier": 13, - "type_name": "selector", - "children": [ - { - "summary": "check if gear1 is inserted to shaft1", - "name": "check gear1 is_inserted_to shaft1", - "identifier": 14, - "type_name": "condition", - "conditions": [ - { - "object_name": "gear1", - "property_name": "is_inserted_to", - "property_value": "shaft1", - "status": True, - } - ], - }, - { - "name": "insert sequence", - "identifier": 15, - "type_name": "sequence", - "children": [ - { - "name": "load_tool selector", - "identifier": 0, - "type_name": "selector", - "children": [ - { - "summary": "check if parallel_box1 is hold", - "name": "check left_hand hold parallel_box1", - "identifier": 1, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "parallel_box1", - "status": True, - } - ], - }, - { - "name": "load_tool Sequence", - "identifier": 2, - "type_name": "sequence", - "children": [ - { - "summary": "check if parallel_box1 is equippable", - "name": "check parallel_box1 is_equippable", - "identifier": 3, - "type_name": "condition", - "conditions": [ - { - "object_name": "parallel_box1", - "property_name": "is_equippable", - "property_value": None, - "status": True, - } - ], - }, - { - "summary": "check if left hand is free", - "name": "check left_hand is_free", - "identifier": 4, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "is_free", - "property_value": None, - "status": True, - } - ], - }, - { - "summary": "equip parallel_box1 to left hand", - "name": "load_tool(left_hand, parallel_box1)", - "identifier": 5, - "type_name": "action", - "effects": [ - { - "object_name": "hand", - "property_name": "is_free", - "property_value": None, - "status": False, - }, - { - "object_name": "parallel_box1", - "property_name": "is_equippable", - "property_value": None, - "status": False, - }, - { - "object_name": "hand", - "property_name": "hold", - "property_value": "parallel_box1", - "status": True, - }, - ], - }, - ], - }, - ], - }, - { - "name": "pick_up selector", - "identifier": 6, - "type_name": "selector", - "children": [ - { - "summary": "check if parallel_box1 holds gear1", - "name": "check parallel_box1 hold gear1", - "identifier": 7, - "type_name": "condition", - "conditions": [ - { - "object_name": "parallel_box1", - "property_name": "hold", - "property_value": "gear1", - "status": True, - } - ], - }, - { - "name": "pick_up Sequence", - "identifier": 8, - "type_name": "sequence", - "children": [ - { - "summary": "check if parallel_box1 is free", - "name": "check parallel_box1 is_free", - "identifier": 9, - "type_name": "condition", - "conditions": [ - { - "object_name": "parallel_box1", - "property_name": "is_free", - "property_value": None, - "status": True, - } - ], - }, - { - "summary": "check if left hand holds parallel_box1", - "name": "check left_hand hold parallel_box1", - "identifier": 16, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "parallel_box1", - "status": True, - } - ], - }, - { - "summary": "pick up gear1 using parallel_box1", - "name": "pick_up(left_hand, parallel_box1, gear1)", - "identifier": 12, - "type_name": "action", - "effects": [ - { - "object_name": "parallel_box1", - "property_name": "hold", - "property_value": "gear1", - "status": True, - }, - { - "object_name": "parallel_box1", - "property_name": "is_free", - "property_value": None, - "status": False, - }, - ], - }, - ], - }, - ], - }, - { - "summary": "check if gear1 can be inserted to shaft1", - "name": "check gear1 can_insert_to shaft1", - "identifier": 18, - "type_name": "condition", - "conditions": [ - { - "object_name": "gear1", - "property_name": "can_insert_to", - "property_value": "shaft1", - "status": True, - } - ], - }, - { - "summary": "insert gear1 to shaft1", - "name": "insert(left_hand, parallel_box1, gear1, shaft1)", - "identifier": 19, - "type_name": "action", - "effects": [ - { - "object_name": "parallel_box1", - "property_name": "hold", - "property_value": "gear1", - "status": False, - }, - { - "object_name": "parallel_box1", - "property_name": "is_free", - "property_value": None, - "status": True, - }, - { - "object_name": "gear1", - "property_name": "is_inserted_to", - "property_value": "shaft1", - "status": True, - }, - ], - }, - ], - }, - ], -} -}, -"environment_before": {"objects": [ - {"name": "gear1", "properties": []}, - {"name": "shaft1", "properties": []}, - {"name": "parallel_box1", "properties": ["is_equippable", "is_free"]}, - {"name": "left_hand", "properties": ["is_free"]}, - ], - "constraints": [ - {"source": "gear1", "name": "can_insert_to", "target": "shaft1"}, - {"source": "parallel_box1", "name": "can_manipulate", "target": "gear1"}, - ], - "relations": [ - - ], -}, -"environment_after": {"objects": [ - {"name": "gear1", "properties": []}, - {"name": "shaft1", "properties": []}, - {"name": "parallel_box1", "properties": ["is_equippable", "is_free"]}, - {"name": "left_hand", "properties": []}, - ], - "constraints": [ - {"source": "gear1", "name": "can_insert_to", "target": "shaft1"}, - {"source": "parallel_box1", "name": "can_manipulate", "target": "gear1"}, - ], - "relations": [ - {"source": "gear1", "name": "is_inserted_to", "target": "shaft1"}, - {"source": "left_hand", "name": "hold", "target": "parallel_box1"}, - ], -}, -"instruction_summary": "insert gear1 into shaft1", -"question":""} -``` -""" -From these examples, learn that some robotic actions have dependencies with the actions before and after them. -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/end_to_end_v1/prompt/prompt_output_format.txt b/kios_bt_planning/backups/prompts/end_to_end_v1/prompt/prompt_output_format.txt deleted file mode 100644 index 6eaad1e0..00000000 --- a/kios_bt_planning/backups/prompts/end_to_end_v1/prompt/prompt_output_format.txt +++ /dev/null @@ -1,21 +0,0 @@ -[user] -You generate an action sequence to solve the task and contruct a behavior tree based on it in json format. -The output dictionary has five keys. -""" -- dictionary["task_plan"]: A dictionary containing information about the task plan. -- dictionary["environment_before"]: The environment state before executing the plan. -- dictionary["environment_after"]: The environment state after executing the plan. -- dictionary["instruction_summary"]: A brief summary of the task plan. -- dictionary["question"]: If you don't have enough knowledge for solving the problem, you can ask questions about the knowledge you need. Leave this key empty if you can solve the task. -""" -Three keys exist in dictionary["task_plan"]. -""" -- dictionary["task_plan"]["action_sequence"]: Contains a list of robot actions. Only the actions defined in the "ROBOT_ACTION LIST" will be used. -- dictionary["task_plan"]["action_instructions"]: contains a list of instructions corresponding to dictionary["task_plan"]["action_sequence"] to explain the actions. -- dictionary["task_plan"]["manipulated_objects"]: The list of the names of the manipulated object. It only includes the objects that are related to the current task. -- dictionary["task_plan"]["behavior_tree"]: contains a dictionary of the behavior tree constructed according to the dictionary ["task_plan"]["action_sequence"] -""" -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/end_to_end_v1/prompt/prompt_problem.txt b/kios_bt_planning/backups/prompts/end_to_end_v1/prompt/prompt_problem.txt deleted file mode 100644 index 22734b26..00000000 --- a/kios_bt_planning/backups/prompts/end_to_end_v1/prompt/prompt_problem.txt +++ /dev/null @@ -1,23 +0,0 @@ -[user] -Information about the problem is given as a pddl problem. -It includes the following information: -- Objects in the world: all instances of the types defined in the domain knowledge. For example, "parallel_box1" is an instance of the type "tool", while "gear1" is an instance of the type "part". -- Initial states: the world states to start with, described with [PREDICATE_LIST] from domain knowledge. For example, "is_free hand" means the object "hand" is free and holds nothing. -- Goal states: the world states to achieve, described with [PREDICATE_LIST] from domain knowledge. -Example: -""" -(define (problem robot_assembly_problem-problem) - (:domain robot_assembly_problem-domain) - (:objects - parallel_box1 parallel_box2 inward_claw outward_claw no_tool - tool - gear1 gear2 gear3 shaft1 shaft2 base - part - left_hand - hand - ) - (:init (can_manipulate parallel_box1 gear1) (can_manipulate outward_claw gear2) (can_manipulate inward_claw gear3) (can_manipulate parallel_box2 shaft1) (can_manipulate no_tool shaft2) (can_screw_to leg1 seat) (can_screw_to leg2 seat) (can_insert_to back seat) (can_screw_to nut1 seat) (can_screw_to nut2 seat) (can_screw_to blub base) (can_place_to lamp blub) (can_insert_to shaft1 base) (can_insert_to shaft2 base) (can_insert_to gear3 shaft2) (can_insert_to gear2 base) (can_insert_to gear1 shaft1) (is_free left_hand) (is_free parallel_box1) (is_free parallel_box2) (is_free inward_claw) (is_free outward_claw) (is_free no_tool) (is_equippable parallel_box1) (is_equippable parallel_box2) (is_equippable inward_claw) (is_equippable outward_claw) (is_equippable no_tool)) - (:goal (and (is_inserted_to shaft1 base))) -) -""" -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/end_to_end_v1/prompt/prompt_role.txt b/kios_bt_planning/backups/prompts/end_to_end_v1/prompt/prompt_role.txt deleted file mode 100644 index 6ce736a0..00000000 --- a/kios_bt_planning/backups/prompts/end_to_end_v1/prompt/prompt_role.txt +++ /dev/null @@ -1,5 +0,0 @@ -[user] -You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you break it down and construct robot task plans represented by behaiovr trees. -Please do not begin working until I say "Start working." Instead, simply output the message "Waiting for next input." Understood? -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/end_to_end_v1/query/old_query.txt b/kios_bt_planning/backups/prompts/end_to_end_v1/query/old_query.txt deleted file mode 100644 index f652fe76..00000000 --- a/kios_bt_planning/backups/prompts/end_to_end_v1/query/old_query.txt +++ /dev/null @@ -1,23 +0,0 @@ -Start working. Resume from the environment below. -""" -{"environment":[ENVIRONMENT]} -""" -The problem is as follows: -""" -{"problem": [PROBLEM]} -""" -The instruction is as follows: -""" -{"instruction": [INSTRUCTION]} -""" -The dictionary that you return should be formatted as python dictionary. Follow these rules: -1. Always equip the proper tools before performing specific object manipulaion. The robot should not hold any tool or object at the end. -2. Make sure that each element of the ["action_instructions"] explains corresponding element of the ["action_sequence"]. Refer to the "ROBOT ACTION LIST" to understand the elements of ["action_sequence"]. -3, The length of the ["action_instructions"] list must be the same as the length of the ["action_sequence"] list. -4. Never left ',' at the end of the list. -5. Keep track of all items listed in the "objects" section of the "environment_before" field. Please ensure that you fill out both the "objects" and "object_states" sections for all listed items. -6. Use the "PREDICATE_LIST" for the "object states" field. -7. All keys of the dictionary should be double-quoted. -8. Insert ``` at the beginning and the end of the dictionary to separate it from the rest of your response. -9. Make sure that you output a consistent manipultation as a single arm robot. For example, grasping an object should not occur in successive steps. -Adhere to the output format I defined above. Follow the nine rules. Think step by step. \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/end_to_end_v1/query/query.txt b/kios_bt_planning/backups/prompts/end_to_end_v1/query/query.txt deleted file mode 100644 index 4eaf4407..00000000 --- a/kios_bt_planning/backups/prompts/end_to_end_v1/query/query.txt +++ /dev/null @@ -1,16 +0,0 @@ -Start working. Resume from problem below. -""" -{"problem": [PROBLEM]} -""" -The dictionary that you return should be formatted as json file. Follow these rules: -1. Always equip the proper tools before performing specific object manipulaion. -2. Make sure that each element of the ["action_instructions"] explains corresponding element of the ["action_sequence"]. Refer to the "ROBOT ACTION LIST" to understand the elements of ["action_sequence"]. -3, The length of the ["action_instructions"] list and the ["action_sequence"] list should be the same. -4. Never left ',' at the end of the list. Use json boolean values, "true" or "false", instead of "True" or "False". -5. Keep track of all items listed in the "objects" section of the "environment_before" field. Please make sure you fill out the "objects" section for all listed items. -6. Use the "PREDICATE_LIST" for the states in the problem. -7. All keys of the dictionary should be double-quoted. -8. Insert ``` at the beginning and the end of the dictionary to separate it from the rest of your response. -9. Make sure that you output a consistent manipultation as a single arm robot. For example, grasping an object should not occur in successive steps. -10. When generating the behavior tree, make sure the tree has the asked structure. -Adhere to the output format I defined above. Follow the rules above. Think step by step. \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/end_to_end_v1/query/second_old_query.txt b/kios_bt_planning/backups/prompts/end_to_end_v1/query/second_old_query.txt deleted file mode 100644 index 530173f3..00000000 --- a/kios_bt_planning/backups/prompts/end_to_end_v1/query/second_old_query.txt +++ /dev/null @@ -1,16 +0,0 @@ -Start working. Resume from problem below. -""" -{"problem": [PROBLEM]} -""" -The dictionary that you return should be formatted as python dictionary. Follow these rules: -1. Always equip the proper tools before performing specific object manipulaion. -2. Make sure that each element of the ["action_instructions"] explains corresponding element of the ["action_sequence"]. Refer to the "ROBOT ACTION LIST" to understand the elements of ["action_sequence"]. -3, The length of the ["action_instructions"] list must be the same as the length of the ["action_sequence"] list. -4. Never left ',' at the end of the list. -5. Keep track of all items listed in the "objects" section of the "environment_before" field. Please make sure you fill out the "objects" section for all listed items. -6. Use the "PREDICATE_LIST" for the "object states" field. -7. All keys of the dictionary should be double-quoted. -8. Insert ``` at the beginning and the end of the dictionary to separate it from the rest of your response. -9. Make sure that you output a consistent manipultation as a single arm robot. For example, grasping an object should not occur in successive steps. -10. For boolean values don't use capital letters. Use "true" or "false" instead of "True" or "False". -Adhere to the output format I defined above. Follow the rules above. Think step by step. \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/end_to_end_v1/system/system.txt b/kios_bt_planning/backups/prompts/end_to_end_v1/system/system.txt deleted file mode 100644 index 26f2afe2..00000000 --- a/kios_bt_planning/backups/prompts/end_to_end_v1/system/system.txt +++ /dev/null @@ -1 +0,0 @@ -You are an excellent interpreter of instructions for robotic assembly tasks. Given the knowledge you need, the world states and some instructions, you break it down and construct robot task plans represented by behaiovr trees based on the action sequences. \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/end_to_end_v2/prompt/backup b/kios_bt_planning/backups/prompts/end_to_end_v2/prompt/backup deleted file mode 100644 index e45ce6c2..00000000 --- a/kios_bt_planning/backups/prompts/end_to_end_v2/prompt/backup +++ /dev/null @@ -1,52 +0,0 @@ - -There are also some counterexamples for you to refer to: -1. One sequence should never have multiple action nodes as its children like the behavior tree below. -{ - "name": "Get the apple", - "identifier": 0, - "type_name": "selector", - "children": [ - { - "summary": "check if the apple is in hand", - "name": "check apple in hand", - "identifier": 1, - "type_name": "condition", - (condition omitted) - }, - { - "name": "Pick Up Sequence", - "identifier": 2, - "type_name": "sequence", - "children": [ - { - "summary": "check if the apple is on the table", - "name": "check apple on the table", - "identifier": 3, - "type_name": "condition", - (condition omitted) - }, - { - "summary": "go to the table", - "name": "go_to(table)", - "identifier": 4, - "type_name": "action", - (effect omitted) - }, - { - "summary": "check if the hand is free", - "name": "check hand free", - "identifier": 5, - "type_name": "condition", - (condition omitted) - }, - { - "summary": "pick up the apple", - "name": "pick_up(apple)", - "identifier": 6, - "type_name": "action", - (effect omitted) - }, - ], - }, - ], -} \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/end_to_end_v2/prompt/e2e_bt.txt b/kios_bt_planning/backups/prompts/end_to_end_v2/prompt/e2e_bt.txt deleted file mode 100644 index 21e56f03..00000000 --- a/kios_bt_planning/backups/prompts/end_to_end_v2/prompt/e2e_bt.txt +++ /dev/null @@ -1,140 +0,0 @@ -[user] -The behavior tree should be in json dictionary format and have a structure like the following example: -""" -{ - "summary": "selector to load inward_claw to left hand", - "name": "selector: load_tool(left_hand, inward_claw)", - "identifier": 3, - "type_name": "selector", - "children": [ - { - "summary": "check if left_hand is holding inward_claw", - "name": "target: hold(left_hand, inward_claw)", - "identifier": 4, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "inward_claw", - "status": true, - } - ], - }, - { - - "summary": "sequence to load inward_claw to left hand", - "name": "sequence: load_tool(left_hand, inward_claw)", - "identifier": 5, - "type_name": "sequence", - "children": [ - { - "summary": "check if inward_claw is equippable", - "name": "precondition: is_equippable(inward_claw)", - "identifier": 6, - "type_name": "condition", - "conditions": [ - { - "object_name": "inward_claw", - "property_name": "is_equippable", - "property_value": null, - "status": true, - } - ], - }, - { - "summary": "check if left hand is free", - "name": "precondition: is_free(left_hand)", - "identifier": 7, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "is_free", - "property_value": null, - "status": true, - } - ], - }, - { - "summary": "equip inward_claw to left hand", - "name": "action: load_tool(left_hand, inward_claw)", - "identifier": 8, - "type_name": "action", - "effects": [ - { - "object_name": "left_hand", - "property_name": "is_free", - "property_value": null, - "status": false, - }, - { - "object_name": "inward_claw", - "property_name": "is_equippable", - "property_value": null, - "status": false, - }, - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "inward_claw", - "status": true, - }, - ], - }, - ], - }, - ], -} -""" -The dictionary that you return should be formatted as json dictionary. Follow these rules: -1. Every node should have four basic keys: - - "summary": a short description of the node. - - "name": the name of the node. The name should follow the definitions in the domain knowledge. - - "identifier": the unique id of the node. - - "type_name": can be: "selector", "sequence", "condition", "action". -2. The control flow node, "selector" and "sequence" nodes, can have "children", which is a list of children nodes. -3. The "condition" nodes that are children of "selectors" are "targets". Those that are children of "sequences" are preconditions. -4. The "condition" nodes should have the key "conditions", which is a list of conditions to be checked. Each condition should have the following keys: - - "object_name": the object to check. - - "property_name": the property/relation of the object to be checked. - - "property_value": this should be the other object in the relation. For property it should be null. - - "status": true if the property/relation is expected to be true, false otherwise. -5. The "action" nodes should have a key "effects", which is a list of the to-be-exerted effects of the action. Each effect should have the following keys: - - "object_name": the affected object. - - "property_name": the affected property/relation of the object. - - "property_value": this should be the other object if this is a relation. For property it should be null. - - "status": true if the effect is to add this property/relation. false if is to remove it. -6. All the subtrees always have their corresponding action in ROBOT_ACTION_LIST. They are used to represent the actions in the action sequence. Its basic structure is: - - A selector as the root node. - - (Usually one) target condition node as the first child of the selector to check if the target of the action has been fulfilled. - - A sequence node as the last child of the selector node to achieve the target. - - (Usually a number of) precondition nodes as the front children of the sequence node to check the preconditions for executing the action. - - One action node as the last child of the sequence node, whose preconditions are checked before and will immediate start the job when being ticked. - - The action will exert its defined effects after finishing the job successfully. Those effects should fulfilled all the target conditions in the selector node. - A brief example: -{ - selector: load_tool selector, - children: [ - { - target: check if inward_claw is equipped by left hand, - }, - { - sequence: load_tool, - children: [ - precondition: check if inward_claw is equippable, - precondition: check if left hand is free, - {action: equip inward_claw to left hand, effects: [left hand not free, inward_claw not equippable, left_hand holds inward_claw]}, - ], - }, - ], -} - -7. When performing a new action to fulfill a unsatisfied precondition, you can construct a subtree to replace the precondition and take that unfulfilled precondition as the target of the subtree. Then you construct the sequence of the subtree according to the new action you want to perform. -8. The behavior tree should be constructed based on the action sequence you planned and the action definitions in ROBOT_ACTION_LIST. -9. All of the control flow nodes, namely selectors and sequences, are without memory. A new behavior tree tick will always start from the root and traverse all the nodes even if they are ticked in the last tick. - -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/end_to_end_v2/prompt/e2e_chain.txt b/kios_bt_planning/backups/prompts/end_to_end_v2/prompt/e2e_chain.txt deleted file mode 100644 index abd348a9..00000000 --- a/kios_bt_planning/backups/prompts/end_to_end_v2/prompt/e2e_chain.txt +++ /dev/null @@ -1,16 +0,0 @@ -[user] -Follow this workflow to solve the task: - -1. Read the problem description. Parse the initial state and write into the "initial_state" in the output dictionary. -2. Interpret the goal of the problem as goal_state and write into the "instruction_summary" in the output dictionary. -3. Try to solve the problem, achieve the goal_state from the initial_state based on the domain knowledge provided. Generate a action sequence with the actions defined in and write it into dictionary["task_plan"]["action_sequence"] of the output. -4. If you cannot find a plan to solve the task, ask question with nature language in the "question" of the output dictionary. -5. Explain the actions in the action sequence and write the output dictionary["task_plan"]["action_instructions"]. -6. Collect the objects in the world that are affected by the action sequence, write into dictionary["task_plan"]["manipulated_objects"]. -7. Update the "initial_state" based on the affected objects you collected in the last step so irrelevant objects and relations/constraints are removed. -8. Generate the behavior tree based on the action sequence and write into dictionary["task_plan"]["behavior_tree"]. - -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/end_to_end_v2/prompt/e2e_domain.txt b/kios_bt_planning/backups/prompts/end_to_end_v2/prompt/e2e_domain.txt deleted file mode 100644 index bf3c7859..00000000 --- a/kios_bt_planning/backups/prompts/end_to_end_v2/prompt/e2e_domain.txt +++ /dev/null @@ -1,89 +0,0 @@ -[user] -Following is the domain knowledge you need in pddl domain format, which includes: -- the object types in the world, -- the predicates for describing the states of the world, -- the actions that the robot can apply, with their precondtions and effects defined. -""" -(define (domain robot_assembly_problem-domain) - (:requirements :strips :typing) - (:types - thing - object - tool part hand - thing - ) - (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part)) - (:action pick_up - :parameters ( ?hand - hand ?tool - tool ?part - part) - :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part)) - :effect (and (hold ?tool ?part) (not (is_free ?tool)))) - (:action put_down - :parameters ( ?hand - hand ?tool - tool ?part - part) - :precondition (and (hold ?tool ?part) (hold ?hand ?tool)) - :effect (and (not (hold ?hand ?part)) (is_free ?tool))) - (:action place - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2))) - (:action detach - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2)))) - (:action insert - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2))) - (:action pull - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2)))) - (:action screw - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2))) - (:action unscrew - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2)))) - (:action load_tool - :parameters ( ?hand - hand ?tool - tool) - :precondition (and (is_equippable ?tool) (is_free ?hand)) - :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool))) - (:action unload_tool - :parameters ( ?hand - hand ?tool - tool) - :precondition (and (hold ?hand ?tool) (is_free ?tool)) - :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool)))) -) -""" -The precidates are explained as follows: -""" -"PREDICATE_LIST" -- is_free( or ): or is free and can hold something. (default: true) -- is_equippable(): is ready to be eqquipped by a robot hand. (default: true) -- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false) -- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false) -- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false) -- can_place_to(, ): can be placed into (placement assembly constraint) (default: false) -- hold(, ): is holding (default: false) -- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false) -- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false) -- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false) -""" - -The actions are explained as follows: -""" -"ROBOT_ACTION_LIST" -- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something. -- put_down(, , ): use to put down . As preconditions, should be holding and should be holding . -- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding . -- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into . -- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding . -- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into . -- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding . -- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into . -- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free. -- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free. -""" -the items in <> indicate the type of the instances needed for the predicates and actions. -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. diff --git a/kios_bt_planning/backups/prompts/end_to_end_v2/prompt/e2e_example.txt b/kios_bt_planning/backups/prompts/end_to_end_v2/prompt/e2e_example.txt deleted file mode 100644 index f0dccece..00000000 --- a/kios_bt_planning/backups/prompts/end_to_end_v2/prompt/e2e_example.txt +++ /dev/null @@ -1,281 +0,0 @@ -[user] -I will give you some examples of the input and the output you will generate. -Example 1: -""" -- Input: -(define (problem robot_assembly_problem-problem) - (:domain robot_assembly_problem-domain) - (:objects - parallel_box1 parallel_box2 inward_claw outward_claw no_tool - tool - gear1 gear2 gear3 shaft1 shaft2 base - part - left_hand - hand - ) - (:init (can_manipulate parallel_box1 gear1) (can_manipulate outward_claw gear2) (can_manipulate inward_claw gear3) (can_manipulate parallel_box2 shaft1) (can_manipulate no_tool shaft2) (can_screw_to leg1 seat) (can_screw_to leg2 seat) (can_insert_to back seat) (can_screw_to nut1 seat) (can_screw_to nut2 seat) (can_screw_to blub base) (can_place_to lamp blub) (can_insert_to shaft1 base) (can_insert_to shaft2 base) (can_insert_to gear3 shaft2) (can_insert_to gear2 base) (can_insert_to gear1 shaft1) (is_inserted_to shaft1 base) (is_free left_hand) (is_free parallel_box1) (is_free parallel_box2) (is_free inward_claw) (is_free outward_claw) (is_free no_tool) (is_equippable parallel_box1) (is_equippable parallel_box2) (is_equippable inward_claw) (is_equippable outward_claw) (is_equippable no_tool)) - (:goal (and (is_inserted_to gear1 shaft1))) -) -- Output: -``` -{"task_plan": { - "action_sequence": [ - "load_tool(left_hand, parallel_box1)", - "pick_up(left_hand, parallel_box1, gear1)", - "insert(left_hand, parallel_box1, gear1, shaft1)" - ], - "action_instructions": [ - "equip the parallel_box1 on the left hand", - "pick up the gear1 with the parallel_box1 in the left hand", - "insert the gear1 into the shaft1 using the parallel_box1 in the left hand" - ], - "object_name": ["left_hand", "gear1", "parallel_box1", "shaft1"], - "behavior_tree": { - "summary": "selector to insert gear1 into shaft1", - "name": "selector: insert gear1 into shaft1", - "identifier": 13, - "type_name": "selector", - "children": [ - { - "summary": "check if gear1 is inserted to shaft1", - "name": "target: is_inserted_to(gear1, shaft1)", - "type_name": "condition", - "conditions": [ - { - "object_name": "gear1", - "property_name": "is_inserted_to", - "property_value": "shaft1", - "status": true, - } - ], - }, - { - "summary": "sequence to insert gear1 into shaft1", - "name": "sequence: insert(gear1, shaft1)", - "identifier": 15, - "type_name": "sequence", - "children": [ - { - "summary": "selector to load parallel_box1", - "name": "selector: load_tool(left_hand, parallel_box1)", - "identifier": 0, - "type_name": "selector", - "children": [ - { - "summary": "check if parallel_box1 is hold", - "name": "target: hold(left_hand, parallel_box1)", - "identifier": 1, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "parallel_box1", - "status": true, - } - ], - }, - { - "summary": "sequence to load parallel_box1", - "name": "sequence: load_tool(left_hand, parallel_box1)", - "identifier": 2, - "type_name": "sequence", - "children": [ - { - "summary": "check if parallel_box1 is equippable", - "name": "precondition: is_equippable(parallel_box1)", - "identifier": 3, - "type_name": "condition", - "conditions": [ - { - "object_name": "parallel_box1", - "property_name": "is_equippable", - "property_value": null, - "status": true, - } - ], - }, - { - "summary": "check if left hand is free", - "name": "precondition: is_free(left_hand)", - "identifier": 4, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "is_free", - "property_value": null, - "status": true, - } - ], - }, - { - "summary": "equip parallel_box1 to left hand", - "name": "action: load_tool(left_hand, parallel_box1)", - "identifier": 5, - "type_name": "action", - "effects": [ - { - "object_name": "hand", - "property_name": "is_free", - "property_value": null, - "status": false, - }, - { - "object_name": "parallel_box1", - "property_name": "is_equippable", - "property_value": null, - "status": false, - }, - { - "object_name": "hand", - "property_name": "hold", - "property_value": "parallel_box1", - "status": true, - }, - ], - }, - ], - }, - ], - }, - { - "summary": "selector to pick_up gear1", - "name": "selector: pick_up(left_hand, parallel_box1, gear1)", - "identifier": 6, - "type_name": "selector", - "children": [ - { - "summary": "check if parallel_box1 holds gear1", - "name": "target: hold(parallel_box1, gear1)", - "identifier": 7, - "type_name": "condition", - "conditions": [ - { - "object_name": "parallel_box1", - "property_name": "hold", - "property_value": "gear1", - "status": true, - } - ], - }, - { - "summary": "sequence to pick_up gear1", - "name": "sequence: pick_up(left_hand, parallel_box1, gear1)", - "identifier": 8, - "type_name": "sequence", - "children": [ - { - "summary": "check if parallel_box1 is free", - "name": "precondition: is_free(parallel_box1)", - "identifier": 9, - "type_name": "condition", - "conditions": [ - { - "object_name": "parallel_box1", - "property_name": "is_free", - "property_value": null, - "status": true, - } - ], - }, - { - "summary": "check if left hand holds parallel_box1", - "name": "precondition: hold(left_hand, parallel_box1)", - "identifier": 16, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "parallel_box1", - "status": true, - } - ], - }, - { - "summary": "pick up gear1 using parallel_box1", - "name": "action: pick_up(left_hand, parallel_box1, gear1)", - "identifier": 12, - "type_name": "action", - "effects": [ - { - "object_name": "parallel_box1", - "property_name": "hold", - "property_value": "gear1", - "status": true, - }, - { - "object_name": "parallel_box1", - "property_name": "is_free", - "property_value": null, - "status": false, - }, - ], - }, - ], - }, - ], - }, - { - "summary": "check if gear1 can be inserted to shaft1", - "name": "precondition: can_insert_to(gear1, shaft1)", - "identifier": 18, - "type_name": "condition", - "conditions": [ - { - "object_name": "gear1", - "property_name": "can_insert_to", - "property_value": "shaft1", - "status": true, - } - ], - }, - { - "summary": "insert gear1 to shaft1", - "name": "action: insert(left_hand, parallel_box1, gear1, shaft1)", - "identifier": 19, - "type_name": "action", - "effects": [ - { - "object_name": "parallel_box1", - "property_name": "hold", - "property_value": "gear1", - "status": false, - }, - { - "object_name": "parallel_box1", - "property_name": "is_free", - "property_value": null, - "status": true, - }, - { - "object_name": "gear1", - "property_name": "is_inserted_to", - "property_value": "shaft1", - "status": true, - }, - ], - }, - ], - }, - ], -} -}, -"initial_state": {"objects": [ - {"name": "gear1", "properties": []}, - {"name": "shaft1", "properties": []}, - {"name": "parallel_box1", "properties": ["is_equippable", "is_free"]}, - {"name": "left_hand", "properties": ["is_free"]}, - ], - "constraints": [ - {"source": "gear1", "name": "can_insert_to", "target": "shaft1"}, - {"source": "parallel_box1", "name": "can_manipulate", "target": "gear1"}, - ], - "relations": [], -}, -"instruction_summary": "insert gear1 into shaft1", -"question":""} -``` -""" -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/end_to_end_v2/prompt/e2e_output_format.txt b/kios_bt_planning/backups/prompts/end_to_end_v2/prompt/e2e_output_format.txt deleted file mode 100644 index a194e955..00000000 --- a/kios_bt_planning/backups/prompts/end_to_end_v2/prompt/e2e_output_format.txt +++ /dev/null @@ -1,21 +0,0 @@ -[user] -You will be provided with a problem and the necessary domain knowledge. -You generate an action sequence to solve the task and contruct a behavior tree based on it in json format. -The output dictionary has five keys. -""" -- dictionary["task_plan"]: A dictionary containing the task plan. -- dictionary["initial_state"]: The initial world state before executing the plan. -- dictionary["instruction_summary"]: A brief summary of the task plan. -- dictionary["question"]: If you don't have enough knowledge for solving the problem, you can ask questions about the knowledge you need. Leave this key empty if you can solve the task. -""" -Three keys exist in dictionary["task_plan"]. -""" -- dictionary["task_plan"]["action_sequence"]: Contains a list of robot actions. Only the actions defined in the "ROBOT_ACTION LIST" will be used. -- dictionary["task_plan"]["action_instructions"]: contains a list of instructions corresponding to dictionary["task_plan"]["action_sequence"] to explain the actions. -- dictionary["task_plan"]["manipulated_objects"]: The list of the manipulated objects. It only includes the objects that are related to the current task. -- dictionary["task_plan"]["behavior_tree"]: contains a dictionary of the behavior tree constructed according to the dictionary ["task_plan"]["action_sequence"] -""" -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/end_to_end_v2/prompt/e2e_problem.txt b/kios_bt_planning/backups/prompts/end_to_end_v2/prompt/e2e_problem.txt deleted file mode 100644 index 6b1ddfb7..00000000 --- a/kios_bt_planning/backups/prompts/end_to_end_v2/prompt/e2e_problem.txt +++ /dev/null @@ -1,24 +0,0 @@ -[user] -Information about the problem is given as a pddl problem. -It includes the following information: -- objects: all the objects in the world. They should be of a type defined in the domain knowledge. For example, "parallel_box1" is an instance of the type "tool", while "gear1" is an instance of the type "part". -- init: the initial world state to start with, described with [PREDICATE_LIST] from domain knowledge. For example, "is_free hand" means the object "hand" is free and holds nothing. -- goal: the world state to achieve, described with [PREDICATE_LIST] from domain knowledge. -An example: -""" -(define (problem robot_assembly_problem-problem) - (:domain robot_assembly_problem-domain) - (:objects - parallel_box1 parallel_box2 inward_claw outward_claw no_tool - tool - gear1 gear2 gear3 shaft1 shaft2 base - part - left_hand - hand - ) - (:init (can_manipulate parallel_box1 gear1) (can_manipulate outward_claw gear2) (can_manipulate inward_claw gear3) (can_manipulate parallel_box2 shaft1) (can_manipulate no_tool shaft2) (can_screw_to leg1 seat) (can_screw_to leg2 seat) (can_insert_to back seat) (can_screw_to nut1 seat) (can_screw_to nut2 seat) (can_screw_to blub base) (can_place_to lamp blub) (can_insert_to shaft1 base) (can_insert_to shaft2 base) (can_insert_to gear3 shaft2) (can_insert_to gear2 base) (can_insert_to gear1 shaft1) (is_free left_hand) (is_free parallel_box1) (is_free parallel_box2) (is_free inward_claw) (is_free outward_claw) (is_free no_tool) (is_equippable parallel_box1) (is_equippable parallel_box2) (is_equippable inward_claw) (is_equippable outward_claw) (is_equippable no_tool)) - (:goal (and (is_inserted_to shaft1 base))) -) -""" - -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/end_to_end_v2/prompt/e2e_role.txt b/kios_bt_planning/backups/prompts/end_to_end_v2/prompt/e2e_role.txt deleted file mode 100644 index 6ce736a0..00000000 --- a/kios_bt_planning/backups/prompts/end_to_end_v2/prompt/e2e_role.txt +++ /dev/null @@ -1,5 +0,0 @@ -[user] -You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you break it down and construct robot task plans represented by behaiovr trees. -Please do not begin working until I say "Start working." Instead, simply output the message "Waiting for next input." Understood? -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/end_to_end_v2/prompt/e2e_state.txt b/kios_bt_planning/backups/prompts/end_to_end_v2/prompt/e2e_state.txt deleted file mode 100644 index a8ea9cd6..00000000 --- a/kios_bt_planning/backups/prompts/end_to_end_v2/prompt/e2e_state.txt +++ /dev/null @@ -1,30 +0,0 @@ -[user] -Information about world states should be in json dictionary format. Example: -""" -{ - "objects": [ - {"name": "leg1", "properties": []}, # leg1 is an object with no properties - {"name": "seat", "properties": []}, # seat is an object with no properties - {"name": "cabinet", "properties": ["open"]}, # cabinet is an object with a property "open" - {"name": "table", "properties": []}, # table is an object with no properties - ], - "constraints": [ - {"source": "leg1", "name": "can_screw_to", "target": "seat"}, # means leg1 can be screwed to seat - {"source": "cabinet", "name": "on_something", "target": "table"}, # means cabinet is on table - ], - "relations": [ - {"source": "leg1", "name": "is_screwed_to", "target": "seat"}, # means leg1 is screwed to seat - ], -} -""" -It contains three lists: - - "objects": the objects and their properties (as lists of strings) in the world. The properties can be changed by robot actions. - - "constraints": the constraint relations that are enforced and should not be changed. They are knowledge about the world or some constraints set by the user. - - "relations": the relations in the world. They can be changed by robot actions. -The properties, constraints and relations should always align with the [PREDICATE_LIST] in domain knowledge. -Only concentrate on the objects, constraints and relations that are related to the current task. - -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/end_to_end_v2/query/query.txt b/kios_bt_planning/backups/prompts/end_to_end_v2/query/query.txt deleted file mode 100644 index cfeb4e8e..00000000 --- a/kios_bt_planning/backups/prompts/end_to_end_v2/query/query.txt +++ /dev/null @@ -1,16 +0,0 @@ -Start working. Resume from the problem below. -""" -{"problem": [PROBLEM]} -""" -The dictionary that you return should be formatted as json file. Follow these rules: -1. Always equip the proper tools before performing specific object manipulaion. -2. Make sure that each element of the ["action_instructions"] explains corresponding element of the ["action_sequence"]. Refer to the "ROBOT ACTION LIST" to understand the elements of ["action_sequence"]. -3, The length of the ["action_instructions"] list and the ["action_sequence"] list should be the same. -4. Never left ',' at the end of the list. Use json boolean values, "true" or "false", instead of "True" or "False". -5. Keep track of all items listed in the "objects" section of the "initial_state" field. -6. Use the "PREDICATE_LIST" for the initial states (:init) in the problem to generate ["initial_state"]. -7. All keys of the dictionary should be double-quoted. -8. Insert ``` at the beginning and the end of the dictionary to separate it from the rest of your response. Don't write any additional things. -9. Make sure that you output a consistent manipultation as a single arm robot. For example, grasping an object should not occur in successive steps. -10. When generating the behavior tree, make sure the tree has the asked structure. -Adhere to the output format I defined above. Follow the rules above. Think step by step. \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/end_to_end_v2/system/system.txt b/kios_bt_planning/backups/prompts/end_to_end_v2/system/system.txt deleted file mode 100644 index 26f2afe2..00000000 --- a/kios_bt_planning/backups/prompts/end_to_end_v2/system/system.txt +++ /dev/null @@ -1 +0,0 @@ -You are an excellent interpreter of instructions for robotic assembly tasks. Given the knowledge you need, the world states and some instructions, you break it down and construct robot task plans represented by behaiovr trees based on the action sequences. \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/sk_refine/prompt/dicard_refine_bt.txt b/kios_bt_planning/backups/prompts/sk_refine/prompt/dicard_refine_bt.txt deleted file mode 100644 index d7233761..00000000 --- a/kios_bt_planning/backups/prompts/sk_refine/prompt/dicard_refine_bt.txt +++ /dev/null @@ -1,140 +0,0 @@ -[user] -You should refine the behavior tree skeleton into a full behavior tree, which is in json dictionary format and have a structure like the following example: -""" -{ - "summary": "selector to load inward_claw to left hand", - "name": "selector: load_tool(left_hand, inward_claw)", - "identifier": 3, - "type_name": "selector", - "children": [ - { - "summary": "check if left_hand is holding inward_claw", - "name": "target: hold(left_hand, inward_claw)", - "identifier": 4, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "inward_claw", - "status": true, - } - ], - }, - { - - "summary": "sequence to load inward_claw to left hand", - "name": "sequence: load_tool(left_hand, inward_claw)", - "identifier": 5, - "type_name": "sequence", - "children": [ - { - "summary": "check if inward_claw is equippable", - "name": "precondition: is_equippable(inward_claw)", - "identifier": 6, - "type_name": "condition", - "conditions": [ - { - "object_name": "inward_claw", - "property_name": "is_equippable", - "property_value": null, - "status": true, - } - ], - }, - { - "summary": "check if left hand is free", - "name": "precondition: is_free(left_hand)", - "identifier": 7, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "is_free", - "property_value": null, - "status": true, - } - ], - }, - { - "summary": "equip inward_claw to left hand", - "name": "action: load_tool(left_hand, inward_claw)", - "identifier": 8, - "type_name": "action", - "effects": [ - { - "object_name": "left_hand", - "property_name": "is_free", - "property_value": null, - "status": false, - }, - { - "object_name": "inward_claw", - "property_name": "is_equippable", - "property_value": null, - "status": false, - }, - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "inward_claw", - "status": true, - }, - ], - }, - ], - }, - ], -} -""" -The dictionary that you return should be formatted as json dictionary. Follow these rules: -1. Every node should have four basic keys: - - "summary": a short description of the node. - - "name": the name of the node. The name should follow the definitions in the domain knowledge. - - "identifier": the unique id of the node. - - "type_name": can be: "selector", "sequence", "condition", "action". -2. The control flow node, "selector" and "sequence" nodes, can have "children", which is a list of children nodes. -3. The "condition" nodes that are children of "selectors" are "targets". Those that are children of "sequences" are preconditions. -4. The "condition" nodes should have the key "conditions", which is a list of conditions to be checked. Each condition should have the following keys: - - "object_name": the object to check. - - "property_name": the property/relation of the object to be checked. - - "property_value": this should be the other object in the relation. For property it should be null. - - "status": true if the property/relation is expected to be true, false otherwise. -5. The "action" nodes should have a key "effects", which is a list of the to-be-exerted effects of the action. Each effect should have the following keys: - - "object_name": the affected object. - - "property_name": the affected property/relation of the object. - - "property_value": this should be the other object if this is a relation. For property it should be null. - - "status": true if the effect is to add this property/relation. false if is to remove it. -6. All the subtrees always have their corresponding action in ROBOT_ACTION_LIST. They are used to represent the actions in the action sequence. Its basic structure is: - - A selector as the root node. - - (Usually one) target condition node as the first child of the selector to check if the target of the action has been fulfilled. - - A sequence node as the last child of the selector node to achieve the target. - - (Usually a number of) precondition nodes as the front children of the sequence node to check the preconditions for executing the action. - - One action node as the last child of the sequence node, whose preconditions are checked before and will immediate start the job when being ticked. - - The action will exert its defined effects after finishing the job successfully. Those effects should fulfilled all the target conditions in the selector node. - A brief example: -{ - selector: load_tool selector, - children: [ - { - target: check if inward_claw is equipped by left hand, - }, - { - sequence: load_tool, - children: [ - precondition: check if inward_claw is equippable, - precondition: check if left hand is free, - {action: equip inward_claw to left hand, effects: [left hand not free, inward_claw not equippable, left_hand holds inward_claw]}, - ], - }, - ], -} - -7. When performing a new action to fulfill a unsatisfied precondition, you can construct a subtree to replace the precondition and take that unfulfilled precondition as the target of the subtree. Then you construct the sequence of the subtree according to the new action you want to perform. -8. The behavior tree should be constructed based on the action sequence you planned and the action definitions in ROBOT_ACTION_LIST. -9. All of the control flow nodes, namely selectors and sequences, are without memory. A new behavior tree tick will always start from the root and traverse all the nodes even if they are ticked in the last tick. - -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/sk_refine/prompt/discard_cot_bt.txt b/kios_bt_planning/backups/prompts/sk_refine/prompt/discard_cot_bt.txt deleted file mode 100644 index e2fc475c..00000000 --- a/kios_bt_planning/backups/prompts/sk_refine/prompt/discard_cot_bt.txt +++ /dev/null @@ -1,68 +0,0 @@ -[user] -The behavior tree should be in json dictionary format and have a structure like the following example: -""" -{ - "summary": "selector to load inward_claw to left hand", - "children": [ - { - "summary": "condition node to check if inward_claw is held by left hand", - }, - { - "summary": "sequence to load inward_claw to left hand", - "children": [ - { - "summary": "condition node to check if inward_claw is equippable", - }, - { - "summary": "condition node to check if left hand is free", - }, - { - "summary": "action node to equip inward_claw to left hand", - "effects": [ - {"summary": "left hand will be not free anymore",}, - {"summary": "inward_claw will be not equippable anymore",}, - {"summary": "left_hand will hold inward_claw",}, - ], - }, - ], - }, - ], -} -""" -Follow these rules: -1. Every node are described by a summary, which is a short interpretive description of the node. It should include the type of the node and the purpose of the node. -2. The control flow nodes, "selector" and "sequence", can have "children", which is a list of children nodes. -3. The "condition" nodes that are children of "selectors" are "targets". Those that are children of "sequences" are preconditions. -4. The "action" nodes should have a key "effects" with a list of dictionaries, which describes the to-be-exerted effects of the action. The effects are shortly described under the key "summary". -5. All the subtrees always have their corresponding action in ROBOT_ACTION_LIST. They are used to represent the actions in the action sequence. The recommended structure is: - - A selector as the root node. - - (Usually one) target condition node as the first child of the selector to check if the target of the action has been fulfilled. - - A sequence node as the last child of the selector node to achieve the target. - - (Usually a number of) precondition nodes as the front children of the sequence node to check the preconditions for executing the action. - - One action node as the last child of the sequence node, whose preconditions are checked before and will immediate start the job when being ticked. - - The action will exert its defined effects after finishing the job successfully. Those effects should fulfilled all the target conditions in the selector node. - A brief example is as follows: -{ - selector: load_tool selector, - children: [ - { - target: check if inward_claw is equipped by left hand, - }, - { - sequence: load_tool, - children: [ - precondition: check if inward_claw is equippable, - precondition: check if left hand is free, - {action: equip inward_claw to left hand, effects: [left hand not free, inward_claw not equippable, left_hand holds inward_claw]}, - ], - }, - ], -} - -6. When performing a new action to fulfill a unsatisfied precondition, you can construct a subtree to replace the precondition and take that unfulfilled precondition as the target of the subtree. Then you construct the sequence of the subtree according to the new action you want to perform. -7. All of the control flow nodes, namely selectors and sequences, are without memory. A new behavior tree tick will always start from the root and traverse all the nodes even if they are ticked in the last tick. - -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/sk_refine/prompt/discard_cot_example.txt b/kios_bt_planning/backups/prompts/sk_refine/prompt/discard_cot_example.txt deleted file mode 100644 index 3d8d6d5b..00000000 --- a/kios_bt_planning/backups/prompts/sk_refine/prompt/discard_cot_example.txt +++ /dev/null @@ -1,127 +0,0 @@ -[user] -I will give you some examples of the input and the output you will generate. -Example 1: -""" -- Input: -(define (problem robot_assembly_problem-problem) - (:domain robot_assembly_problem-domain) - (:objects - parallel_box1 parallel_box2 inward_claw outward_claw no_tool - tool - gear1 gear2 gear3 shaft1 shaft2 base - part - left_hand - hand - ) - (:init (can_manipulate parallel_box1 gear1) (can_manipulate outward_claw gear2) (can_manipulate inward_claw gear3) (can_manipulate parallel_box2 shaft1) (can_manipulate no_tool shaft2) (can_screw_to leg1 seat) (can_screw_to leg2 seat) (can_insert_to back seat) (can_screw_to nut1 seat) (can_screw_to nut2 seat) (can_screw_to blub base) (can_place_to lamp blub) (can_insert_to shaft1 base) (can_insert_to shaft2 base) (can_insert_to gear3 shaft2) (can_insert_to gear2 base) (can_insert_to gear1 shaft1) (is_inserted_to shaft1 base) (is_free left_hand) (is_free parallel_box1) (is_free parallel_box2) (is_free inward_claw) (is_free outward_claw) (is_free no_tool) (is_equippable parallel_box1) (is_equippable parallel_box2) (is_equippable inward_claw) (is_equippable outward_claw) (is_equippable no_tool)) - (:goal (and (is_inserted_to gear1 shaft1))) -) -- Output: -``` -{"task_plan": { - "action_sequence": [ - "load_tool(left_hand, parallel_box1)", - "pick_up(left_hand, parallel_box1, gear1)", - "insert(left_hand, parallel_box1, gear1, shaft1)" - ], - "action_instructions": [ - "equip the parallel_box1 on the left hand", - "pick up the gear1 with the parallel_box1 in the left hand", - "insert the gear1 into the shaft1 using the parallel_box1 in the left hand" - ], - "object_name": ["left_hand", "gear1", "parallel_box1", "shaft1"], - "behavior_tree": { - "summary": "selector to insert gear1 into shaft1", - "children": [ - { - "summary": "condition node to check if gear1 is inserted to shaft1", - }, - { - "summary": "sequence to insert gear1 into shaft1", - "children": [ - { - "summary": "selector to load_tool - "children": [ - { - "summary": "condition node to check if parallel_box1 is hold", - }, - { - "summary": "sequence to load_tool", - "children": [ - { - "summary": "condition node to check if parallel_box1 is equippable", - }, - { - "summary": "condition node to check if left hand is free", - }, - { - "summary": "action node to equip parallel_box1 to left hand", - "effects": [ - {"summary": "left_hand will be not free",}, - {"summary": "parallel_box1 will be not equippable",}, - {"summary": "left_hand will hold parallel_box1",}, - ], - }, - ], - }, - ], - }, - { - "summary": "selector to pick_up", - "children": [ - { - "summary": "condition node to check if parallel_box1 holds gear1", - }, - { - "summary": "sequence to pick_up", - "children": [ - { - "summary": "condition node to check if parallel_box1 is free", - }, - { - "summary": "condition node to check if left hand holds parallel_box1", - }, - { - "summary": "action node to pick up gear1 using parallel_box1", - "effects": [ - {"summary": "parallel_box1 will hold gear1",}, - {"summary": "parallel_box1 will be not free",}, - ], - }, - ], - }, - ], - }, - { - "summary": "condition node to check if gear1 can be inserted to shaft1", - }, - { - "summary": "action node to insert gear1 to shaft1", - "effects": [ - {"summary": "parallel_box1 will be not holding gear1",}, - {"summary": "parallel_box1 will be free",}, - {"summary": "gear1 will be inserted to shaft1",}, - ], - }, - ], - }, - ], -} -}, -"initial_state": {"objects": [ - {"name": "gear1", "properties": []}, - {"name": "shaft1", "properties": []}, - {"name": "parallel_box1", "properties": ["is_equippable", "is_free"]}, - {"name": "left_hand", "properties": ["is_free"]}, - ], - "constraints": [ - {"source": "gear1", "name": "can_insert_to", "target": "shaft1"}, - {"source": "parallel_box1", "name": "can_manipulate", "target": "gear1"}, - ], - "relations": [], -}, -"instruction_summary": "insert gear1 into shaft1", -"question":""} -``` -""" -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/sk_refine/prompt/discard_refine_refine.txt b/kios_bt_planning/backups/prompts/sk_refine/prompt/discard_refine_refine.txt deleted file mode 100644 index 777e8f26..00000000 --- a/kios_bt_planning/backups/prompts/sk_refine/prompt/discard_refine_refine.txt +++ /dev/null @@ -1,247 +0,0 @@ -[user] -You should refine the behavior tree skeleton into a full behavior tree in json dictionary format. - -For each node, you should: -1. add an unique integer "identifier". -2. add its "type_name" accordingly. (precondition and target are both "condition") - -For control flow nodes, namely selectors and sequences, an example is shown as follows: -the skeleton: -""" -{ - "summary": "selector to load no_tool to hand", - "name": "selector: load_tool", - "children": [...], -} - -""" -after refinement: -""" -{ - "summary": "selector to load no_tool to hand", - "name": "selector: load_tool(left_hand, no_tool)", - "identifier": 3, - "type_name": "selector", - "children": [...], -} -""" - -For condition nodes, you should: -1. refine and add the key "conditions", which is a list of conditions to be checked. Each condition should have the following keys: - - "object_name": the object to check. - - "property_name": the property/relation of the object to be checked. - - "property_value": this should be the other object in the relation. For property it should be null. - - "status": true if the property/relation is expected to be true, false otherwise. -An example is shown below: -the skeleton: -""" -{ - "summary": "check if left_hand is holding parallel_box1", - "name": "target: hold(left_hand, parallel_box1)", -} -""" -after refinement: -""" -{ - "summary": "check if left_hand is holding parallel_box1", - "name": "target: hold(left_hand, parallel_box1)", - "identifier": 4, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "parallel_box1", - "status": true, - } - ], -}, -""" - -For action nodes, you should: -1. refine and replace the summaries in the key "effects", which is a list of the to-be-exerted effects of the action. Each effect should have the following keys: - - "object_name": the affected object. - - "property_name": the affected property/relation of the object. - - "property_value": this should be the other object if this is a relation. For property it should be null. - - "status": true if the effect is to add this property/relation. false if is to remove it. -An example is shown below: -the skeleton: -""" -{ - "summary": "left hand equips tool1", - "name": "action: load_tool(left_hand, tool1)", - "effects": [ - {"summary": "left hand will be not free anymore",}, - {"summary": "tool1 will be not equippable anymore",}, - {"summary": "left_hand will hold tool1",}, - ], -}, -""" -after refinement: -""" -{ - "summary": "left hand equips tool1", - "name": "action: load_tool(left_hand, tool1)", - "identifier": 8, - "type_name": "action", - "effects": [ - { - "object_name": "left_hand", - "property_name": "is_free", - "property_value": null, - "status": false, - }, - { - "object_name": "tool1", - "property_name": "is_equippable", - "property_value": null, - "status": false, - }, - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "tool1", - "status": true, - }, - ], -}, -""" - -""" -{ - "summary": "selector to load inward_claw to left hand", - "name": "selector: load_tool(left_hand, inward_claw)", - "identifier": 3, - "type_name": "selector", - "children": [ - { - "summary": "check if left_hand is holding inward_claw", - "name": "target: hold(left_hand, inward_claw)", - "identifier": 4, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "inward_claw", - "status": true, - } - ], - }, - { - - "summary": "sequence to load inward_claw to left hand", - "name": "sequence: load_tool(left_hand, inward_claw)", - "identifier": 5, - "type_name": "sequence", - "children": [ - { - "summary": "check if inward_claw is equippable", - "name": "precondition: is_equippable(inward_claw)", - "identifier": 6, - "type_name": "condition", - "conditions": [ - { - "object_name": "inward_claw", - "property_name": "is_equippable", - "property_value": null, - "status": true, - } - ], - }, - { - "summary": "check if left hand is free", - "name": "precondition: is_free(left_hand)", - "identifier": 7, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "is_free", - "property_value": null, - "status": true, - } - ], - }, - { - "summary": "equip inward_claw to left hand", - "name": "action: load_tool(left_hand, inward_claw)", - "identifier": 8, - "type_name": "action", - "effects": [ - { - "object_name": "left_hand", - "property_name": "is_free", - "property_value": null, - "status": false, - }, - { - "object_name": "inward_claw", - "property_name": "is_equippable", - "property_value": null, - "status": false, - }, - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "inward_claw", - "status": true, - }, - ], - }, - ], - }, - ], -} -""" -The dictionary that you return should be formatted as json dictionary. Follow these rules: -1. Every node should have four basic keys: - - "summary": a short description of the node. - - "name": the name of the node. The name should follow the definitions in the domain knowledge. - - "identifier": the unique id of the node. - - "type_name": can be: "selector", "sequence", "condition", "action". -2. The control flow node, "selector" and "sequence" nodes, can have "children", which is a list of children nodes. -3. The "condition" nodes that are children of "selectors" are "targets". Those that are children of "sequences" are preconditions. -4. The "condition" nodes should have the key "conditions", which is a list of conditions to be checked. Each condition should have the following keys: - - "object_name": the object to check. - - "property_name": the property/relation of the object to be checked. - - "property_value": this should be the other object in the relation. For property it should be null. - - "status": true if the property/relation is expected to be true, false otherwise. -5. The "action" nodes should have a key "effects", which is a list of the to-be-exerted effects of the action. Each effect should have the following keys: - - "object_name": the affected object. - - "property_name": the affected property/relation of the object. - - "property_value": this should be the other object if this is a relation. For property it should be null. - - "status": true if the effect is to add this property/relation. false if is to remove it. -6. All the subtrees always have their corresponding action in ROBOT_ACTION_LIST. They are used to represent the actions in the action sequence. Its basic structure is: - - A selector as the root node. - - (Usually one) target condition node as the first child of the selector to check if the target of the action has been fulfilled. - - A sequence node as the last child of the selector node to achieve the target. - - (Usually a number of) precondition nodes as the front children of the sequence node to check the preconditions for executing the action. - - One action node as the last child of the sequence node, whose preconditions are checked before and will immediate start the job when being ticked. - - The action will exert its defined effects after finishing the job successfully. Those effects should fulfilled all the target conditions in the selector node. - A brief example: -{ - selector: load_tool selector, - children: [ - { - target: check if inward_claw is equipped by left hand, - }, - { - sequence: load_tool, - children: [ - precondition: check if inward_claw is equippable, - precondition: check if left hand is free, - {action: equip inward_claw to left hand, effects: [left hand not free, inward_claw not equippable, left_hand holds inward_claw]}, - ], - }, - ], -} - -7. When performing a new action to fulfill a unsatisfied precondition, you can construct a subtree to replace the precondition and take that unfulfilled precondition as the target of the subtree. Then you construct the sequence of the subtree according to the new action you want to perform. -8. The behavior tree should be constructed based on the action sequence you planned and the action definitions in ROBOT_ACTION_LIST. -9. All of the control flow nodes, namely selectors and sequences, are without memory. A new behavior tree tick will always start from the root and traverse all the nodes even if they are ticked in the last tick. - -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/sk_refine/prompt/refine_action.txt b/kios_bt_planning/backups/prompts/sk_refine/prompt/refine_action.txt deleted file mode 100644 index ebba9c19..00000000 --- a/kios_bt_planning/backups/prompts/sk_refine/prompt/refine_action.txt +++ /dev/null @@ -1,54 +0,0 @@ -[user] -For action nodes, you should: -1. refine and replace the summaries in the key "effects", which is a list of the to-be-exerted effects of the action. Each effect should have the following keys: - - "object_name": the affected object. - - "property_name": the affected property/relation of the object. - - "property_value": this should be the other object if this is a relation. For property it should be null. - - "status": true if the effect is to add this property/relation. false if is to remove it. -An example is shown below: -the skeleton: -""" -{ - "summary": "left hand equips tool1", - "name": "action: load_tool(left_hand, tool1)", - "effects": [ - {"summary": "left hand will be not free anymore",}, - {"summary": "tool1 will be not equippable anymore",}, - {"summary": "left_hand will hold tool1",}, - ], -}, -""" -after refinement: -""" -{ - "summary": "left hand equips tool1", - "name": "action: load_tool(left_hand, tool1)", - "identifier": 8, - "type_name": "action", - "effects": [ - { - "object_name": "left_hand", - "property_name": "is_free", - "property_value": null, - "status": false, - }, - { - "object_name": "tool1", - "property_name": "is_equippable", - "property_value": null, - "status": false, - }, - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "tool1", - "status": true, - }, - ], -}, -""" - -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/sk_refine/prompt/refine_chain.txt b/kios_bt_planning/backups/prompts/sk_refine/prompt/refine_chain.txt deleted file mode 100644 index 8ab56a97..00000000 --- a/kios_bt_planning/backups/prompts/sk_refine/prompt/refine_chain.txt +++ /dev/null @@ -1,15 +0,0 @@ -[user] -Follow this workflow to solve the task: - -1. Read the problem description. Parse and generate the "initial_state" in the output dictionary. -2. Interpret the goal of the problem as goal_state and write into the "instruction_summary" in the output dictionary. -3. Try to solve the problem, achieve the goal_state from the initial_state based on the domain knowledge provided. Generate a action sequence with the actions defined in and write it into dictionary["task_plan"]["action_sequence"] of the output. -4. If you cannot find a plan to solve the task, ask question with nature language in the "question" of the output dictionary. -5. Explain the actions in the action sequence and write the output dictionary["task_plan"]["action_instructions"]. -6. Collect the objects in the world that are effected by the action sequence, write into dictionary["task_plan"]["manipulated_objects"]. -7. Generate the behavior tree based on the action sequence and write into dictionary["task_plan"]["behavior_tree"]. - -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/sk_refine/prompt/refine_condition.txt b/kios_bt_planning/backups/prompts/sk_refine/prompt/refine_condition.txt deleted file mode 100644 index bb08d441..00000000 --- a/kios_bt_planning/backups/prompts/sk_refine/prompt/refine_condition.txt +++ /dev/null @@ -1,37 +0,0 @@ -[user] -For condition nodes, you should: -1. refine and add the key "conditions", which is a list of conditions to be checked. Each condition should have the following keys: - - "object_name": the object to check. - - "property_name": the property/relation of the object to be checked. - - "property_value": this should be the other object in the relation. For property it should be null. - - "status": true if the property/relation is expected to be true, false otherwise. -An example is shown below: -the skeleton: -""" -{ - "summary": "check if left_hand is holding parallel_box1", - "name": "target: hold(left_hand, parallel_box1)", -} -""" -after refinement: -""" -{ - "summary": "check if left_hand is holding parallel_box1", - "name": "target: hold(left_hand, parallel_box1)", - "identifier": 4, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "parallel_box1", - "status": true, - } - ], -}, -""" - -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/sk_refine/prompt/refine_controlflow.txt b/kios_bt_planning/backups/prompts/sk_refine/prompt/refine_controlflow.txt deleted file mode 100644 index d0f4c953..00000000 --- a/kios_bt_planning/backups/prompts/sk_refine/prompt/refine_controlflow.txt +++ /dev/null @@ -1,26 +0,0 @@ -[user] -For control flow nodes, namely selectors and sequences, an example is shown as follows: -the skeleton: -""" -{ - "summary": "selector to load no_tool to hand", - "name": "selector: load_tool", - "children": [...], -} - -""" -after refinement: -""" -{ - "summary": "selector to load no_tool to hand", - "name": "selector: load_tool(left_hand, no_tool)", - "identifier": 3, - "type_name": "selector", - "children": [...], -} -""" - -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/sk_refine/prompt/refine_domain.txt b/kios_bt_planning/backups/prompts/sk_refine/prompt/refine_domain.txt deleted file mode 100644 index 7f48353e..00000000 --- a/kios_bt_planning/backups/prompts/sk_refine/prompt/refine_domain.txt +++ /dev/null @@ -1,89 +0,0 @@ -[user] -Following is the domain knowledge you need in pddl domain format, which includes: -- the object types in the world, -- the predicates for describing the states of the world, -- the actions that the robot can apply, with their precondtions and effects defined. -""" -(define (domain robot_assembly_problem-domain) - (:requirements :strips :typing) - (:types - thing - object - tool part hand - thing - ) - (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part)) - (:action pick_up - :parameters ( ?hand - hand ?tool - tool ?part - part) - :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part)) - :effect (and (hold ?tool ?part) (not (is_free ?tool)))) - (:action put_down - :parameters ( ?hand - hand ?tool - tool ?part - part) - :precondition (and (hold ?tool ?part) (hold ?hand ?tool)) - :effect (and (not (hold ?hand ?part)) (is_free ?tool))) - (:action place - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2))) - (:action detach - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2)))) - (:action insert - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2))) - (:action pull - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2)))) - (:action screw - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2))) - (:action unscrew - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2)))) - (:action load_tool - :parameters ( ?hand - hand ?tool - tool) - :precondition (and (is_equippable ?tool) (is_free ?hand)) - :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool))) - (:action unload_tool - :parameters ( ?hand - hand ?tool - tool) - :precondition (and (hold ?hand ?tool) (is_free ?tool)) - :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool)))) -) -""" -The precidates are explained as follows: -""" -"PREDICATE_LIST" -- is_free( or ): or is free and can hold something. (default: true) -- is_equippable(): is ready to be eqquipped by a robot hand. (default: true) -- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false) -- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false) -- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false) -- can_place_to(, ): can be placed into (placement assembly constraint) (default: false) -- hold(, ): is holding (default: false) -- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false) -- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false) -- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false) -""" - -The actions are explained as follows: -""" -"ROBOT_ACTION_LIST" -- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something. -- put_down(, , ): use to put down . As preconditions, should be holding and should be holding . -- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding . -- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into . -- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding . -- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into . -- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding . -- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into . -- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free. -- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free. -""" -the items in <> indicate the type of the instances needed for the predicates and actions. -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. Waiting for next input. diff --git a/kios_bt_planning/backups/prompts/sk_refine/prompt/refine_help.txt b/kios_bt_planning/backups/prompts/sk_refine/prompt/refine_help.txt deleted file mode 100644 index 1ca165db..00000000 --- a/kios_bt_planning/backups/prompts/sk_refine/prompt/refine_help.txt +++ /dev/null @@ -1,11 +0,0 @@ -[user] -You should refine the behavior tree skeleton into a full behavior tree in json dictionary format. - -1. Refer to "initial_state" of input to find the necessary objects, properties and relationships. -2. Refer to the domain knowledge to understand the actions (corresponds to "ROBOT_ACTION_LIST") and conditions (corresponds to "PREDICATE_LIST"). -3. The name of the node (if it has) also provides information for its refinement. - -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/sk_refine/prompt/refine_input_format.txt b/kios_bt_planning/backups/prompts/sk_refine/prompt/refine_input_format.txt deleted file mode 100644 index 27a2fdef..00000000 --- a/kios_bt_planning/backups/prompts/sk_refine/prompt/refine_input_format.txt +++ /dev/null @@ -1,127 +0,0 @@ -[user] -The input you get and the output you return should be in json dictionary format. -The dictionary contains the following keys: -""" -- dictionary["task_plan"]: A dictionary containing the task plan. -- dictionary["initial_state"]: The initial world state before executing the plan. -- dictionary["instruction_summary"]: A brief summary of the task plan. -- dictionary["question"]: Question for more knowledge that are unknown yet but necessary for making the plan. Normally empty. -""" -Four keys exist in dictionary["task_plan"]. -""" -- dictionary["task_plan"]["action_sequence"]: Contains a list of robot actions. -- dictionary["task_plan"]["action_instructions"]: contains a list of instructions to explain the actions in dictionary["task_plan"]["action_sequence"]. -- dictionary["task_plan"]["manipulated_objects"]: The list of the objects that are related to the current task. -- dictionary["task_plan"]["behavior_tree"]: contains a dictionary of the behavior tree skeleton constructed according to the dictionary ["task_plan"]["action_sequence"] -""" -The following is an input example: -``` -{"task_plan": { - "action_sequence": [ - "load_tool(left_hand, parallel_box1)", - "pick_up(left_hand, parallel_box1, gear1)", - "insert(left_hand, parallel_box1, gear1, shaft1)" - ], - "action_instructions": [ - "equip the parallel_box1 on the left hand", - "pick up the gear1 with the parallel_box1 in the left hand", - "insert the gear1 into the shaft1 using the parallel_box1 in the left hand" - ], - "object_name": ["left_hand", "gear1", "parallel_box1", "shaft1"], - "behavior_tree": { - "summary": "selector to insert gear1 into shaft1", - "children": [ - { - "summary": "condition node to check if gear1 is inserted to shaft1", - }, - { - "summary": "sequence to insert gear1 into shaft1", - "children": [ - { - "summary": "selector to load_tool - "children": [ - { - "summary": "condition node to check if parallel_box1 is hold", - }, - { - "summary": "sequence to load_tool", - "children": [ - { - "summary": "condition node to check if parallel_box1 is equippable", - }, - { - "summary": "condition node to check if left hand is free", - }, - { - "summary": "action node to equip parallel_box1 to left hand", - "effects": [ - {"summary": "left_hand will be not free",}, - {"summary": "parallel_box1 will be not equippable",}, - {"summary": "left_hand will hold parallel_box1",}, - ], - }, - ], - }, - ], - }, - { - "summary": "selector to pick_up", - "children": [ - { - "summary": "condition node to check if parallel_box1 holds gear1", - }, - { - "summary": "sequence to pick_up", - "children": [ - { - "summary": "condition node to check if parallel_box1 is free", - }, - { - "summary": "condition node to check if left hand holds parallel_box1", - }, - { - "summary": "action node to pick up gear1 using parallel_box1", - "effects": [ - {"summary": "parallel_box1 will hold gear1",}, - {"summary": "parallel_box1 will be not free",}, - ], - }, - ], - }, - ], - }, - { - "summary": "condition node to check if gear1 can be inserted to shaft1", - }, - { - "summary": "action node to insert gear1 to shaft1", - "effects": [ - {"summary": "parallel_box1 will be not holding gear1",}, - {"summary": "parallel_box1 will be free",}, - {"summary": "gear1 will be inserted to shaft1",}, - ], - }, - ], - }, - ], -} -}, -"initial_state": {"objects": [ - {"name": "gear1", "properties": []}, - {"name": "shaft1", "properties": []}, - {"name": "parallel_box1", "properties": ["is_equippable", "is_free"]}, - {"name": "left_hand", "properties": ["is_free"]}, - ], - "constraints": [ - {"source": "gear1", "name": "can_insert_to", "target": "shaft1"}, - {"source": "parallel_box1", "name": "can_manipulate", "target": "gear1"}, - ], - "relations": [], -}, -"instruction_summary": "insert gear1 into shaft1", -"question":""} -``` -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/sk_refine/prompt/refine_output_format.txt b/kios_bt_planning/backups/prompts/sk_refine/prompt/refine_output_format.txt deleted file mode 100644 index d40e17ea..00000000 --- a/kios_bt_planning/backups/prompts/sk_refine/prompt/refine_output_format.txt +++ /dev/null @@ -1,19 +0,0 @@ -[user] -The output dictionary has four keys. -""" -- dictionary["task_plan"] -- dictionary["initial_state"] -- dictionary["instruction_summary"] -- dictionary["question"] -""" -Four keys exist in dictionary["task_plan"]. -""" -- dictionary["task_plan"]["action_sequence"] -- dictionary["task_plan"]["action_instructions"] -- dictionary["task_plan"]["manipulated_objects"] -- dictionary["task_plan"]["behavior_tree"] -""" -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/sk_refine/prompt/refine_role.txt b/kios_bt_planning/backups/prompts/sk_refine/prompt/refine_role.txt deleted file mode 100644 index de12cb04..00000000 --- a/kios_bt_planning/backups/prompts/sk_refine/prompt/refine_role.txt +++ /dev/null @@ -1,5 +0,0 @@ -[user] -You are an excellent task planner for robotic assembly tasks. Given the world state and a rough plan, you break it down and refine the plan into a well formatted and executable behavior tree. -Please do not start to work until I say "Start working." Instead, simply output the message "Waiting for next input." Understood? -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/sk_refine/prompt/refine_state.txt b/kios_bt_planning/backups/prompts/sk_refine/prompt/refine_state.txt deleted file mode 100644 index b84f1778..00000000 --- a/kios_bt_planning/backups/prompts/sk_refine/prompt/refine_state.txt +++ /dev/null @@ -1,27 +0,0 @@ -[user] -The initial state in the input is explained as follows: - - "objects": the objects and their properties (as lists of strings) in the world. The properties can be changed by robot actions. - - "constraints": the constraint relations that are enforced and should not be changed. They are knowledge about the world or some constraints set by the user. - - "relations": the relations in the world. They can be changed by robot actions. -following is another example: -""" -{ - "objects": [ - {"name": "leg1", "properties": []}, # leg1 is an object with no properties - {"name": "seat", "properties": []}, # seat is an object with no properties - {"name": "hand", "properties": ["is_free"]}, # hand is an object with a property "is_free" - ], - "constraints": [ - {"source": "leg1", "name": "can_screw_to", "target": "seat"}, # means leg1 can be screwed to seat - ], - "relations": [ - {"source": "leg1", "name": "is_screwed_to", "target": "seat"}, # means leg1 is screwed to seat - ], -} -""" -Only concentrate on the objects, constraints and relations that are related to the current task. - -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/sk_refine/query/dup_query.txt b/kios_bt_planning/backups/prompts/sk_refine/query/dup_query.txt deleted file mode 100644 index c1a7955e..00000000 --- a/kios_bt_planning/backups/prompts/sk_refine/query/dup_query.txt +++ /dev/null @@ -1,16 +0,0 @@ -Start working. Resume from the input below. -""" -{"input": [PROBLEM]} -""" -The dictionary that you return should be formatted as json file. Follow these rules: -1. Always equip the proper tools before performing specific object manipulaion. -2. Make sure that each element of the ["action_instructions"] explains corresponding element of the ["action_sequence"]. Refer to the "ROBOT ACTION LIST" to understand the elements of ["action_sequence"]. -3, The length of the ["action_instructions"] list and the ["action_sequence"] list should be the same. -4. Never left ',' at the end of the list. Use json boolean values, "true" or "false", instead of "True" or "False". -5. Keep track of all items listed in the "objects" section of the "initial_state" field. -6. Use the "PREDICATE_LIST" for the initial states (:init) in the problem to generate ["initial_state"]. -7. All keys of the dictionary should be double-quoted. -8. Insert ``` at the beginning and the end of the dictionary to separate it from the rest of your response. Don't write any additional things. -9. Make sure that you output a consistent manipultation as a single arm robot. For example, grasping an object should not occur in successive steps. -10. When generating the behavior tree, make sure the tree has the asked structure. -Adhere to the output format I defined above. Follow the rules above. Think step by step. \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/sk_refine/query/query.txt b/kios_bt_planning/backups/prompts/sk_refine/query/query.txt deleted file mode 100644 index 7abaa278..00000000 --- a/kios_bt_planning/backups/prompts/sk_refine/query/query.txt +++ /dev/null @@ -1,14 +0,0 @@ -Start working. Resume from the input below. -""" -{"input": [PROBLEM]} -""" -The dictionary that you return should be formatted as json file. Follow these rules: -1. Only refine the skeleton behavior tree in the "behavior_tree" field. Refine the tree according to rules provided before. Replace the old behavior tree with the refined behavior tree. -2. Refer to the domain knowledge, namely "ROBOT_ACTION_LIST" and "PREDICATE_LIST", to understand and refine the skeleton behavior tree. -3. Do not change other fields of the dictionary and return the new dictionary as json file. -4. Never left ',' at the end of the list. -5. Use json boolean values, "true" or "false", instead of "True" or "False". -6. All keys of the dictionary should be double-quoted. -7. Insert ``` at the beginning and the end of the dictionary to separate it from the rest of your response. Don't write any additional things. -8. When generating the behavior tree, make sure the tree has the asked structure. -Adhere to the output format I defined above. Follow the rules above. Think step by step. \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/sk_refine/system/system.txt b/kios_bt_planning/backups/prompts/sk_refine/system/system.txt deleted file mode 100644 index 5bc3c5a6..00000000 --- a/kios_bt_planning/backups/prompts/sk_refine/system/system.txt +++ /dev/null @@ -1 +0,0 @@ -You are an excellent robot behavior tree expert in robotic assembly tasks. Given the knowledge you need, the world state and a rough plan, you break it down and refine the task plan to make it well formatted and ready to be executed. \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/skeleton/prompt/backup b/kios_bt_planning/backups/prompts/skeleton/prompt/backup deleted file mode 100644 index c4a0f6f6..00000000 --- a/kios_bt_planning/backups/prompts/skeleton/prompt/backup +++ /dev/null @@ -1,43 +0,0 @@ - -There are also some counterexamples for you to refer to: -1. One sequence should never have multiple action nodes as its children like the behavior tree below. -{ - "summary": "selector to get the apple", - "name": "Get the apple", - "type_name": "selector", - "children": [ - { - "summary": "check if the apple is in hand", - "name": "target: in_hand(apple)", - "type_name": "condition", - }, - { - "summary": "sequence to get the apple", - "name": "sequence: get apple", - "type_name": "sequence", - "children": [ - { - "summary": "check if the apple is on the table", - "name": "precondition: on(apple, table)", - "type_name": "condition", - }, - { - "summary": "go to the table", - "name": "action: go_to(table)", - "type_name": "action", - }, - { - "summary": "check if the hand is free", - "name": "precondition: is_free(hand)", - "type_name": "condition", - }, - { - "summary": "pick up the apple", - "name": "action: pick_up(apple)", - "type_name": "action", - (effect omitted) - }, - ], - }, - ], -} \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/skeleton/prompt/sk_bt_skeleton.txt b/kios_bt_planning/backups/prompts/skeleton/prompt/sk_bt_skeleton.txt deleted file mode 100644 index b1225fcc..00000000 --- a/kios_bt_planning/backups/prompts/skeleton/prompt/sk_bt_skeleton.txt +++ /dev/null @@ -1,83 +0,0 @@ -[user] -The behavior tree should be in json dictionary format and have a structure like the following example: -""" -{ - "summary": "selector to load inward_claw to left hand", - "name": "selector: load_tool", - "type_name": "selector", - "children": [ - { - "summary": "check if inward_claw is held by left hand", - "name": "target: hold(left_hand, inward_claw)", - "type_name": "condition", - }, - { - "summary": "sequence to load inward_claw to left hand", - "name": "sequence: load_tool", - "type_name": "sequence", - "children": [ - { - "summary": "check if inward_claw is equippable", - "name": "precondition: is_equippable(inward_claw)", - "type_name": "condition", - }, - { - "summary": "check if left hand is free", - "name": "precondition: is_free(left_hand)", - "type_name": "condition", - }, - { - "summary": "equip inward_claw to left hand", - "name": "action: load_tool(left_hand, inward_claw)", - "type_name": "action", - "effects": [ - {"summary": "left hand will be not free anymore",}, - {"summary": "inward_claw will be not equippable anymore",}, - {"summary": "left_hand will hold inward_claw",}, - ], - }, - ], - }, - ], -} -""" -The dictionary that you return should be formatted as json dictionary. Follow these rules: -1. Every node should have three basic keys: - - "summary": a short interpretive description of the node - - "name": the name of the node. The name should follow the definitions in the domain knowledge. - - "type_name": can be: "selector", "sequence", "condition", "action" -2. The control flow nodes, "selector" and "sequence", can have "children", which is a list of children nodes. -3. The "condition" nodes that are children of "selectors" are "targets". Those that are children of "sequences" are preconditions. -4. The "action" nodes should have a key "effects" with a list of dictionaries, which describes the to-be-exerted effects of the action. The effects are shortly described under the key "summary". -5. All the subtrees always have their corresponding action in ROBOT_ACTION_LIST. They are used to represent the actions in the action sequence. The recommended structure is: - - A selector as the root node. - - (Usually one) target condition node as the first child of the selector to check if the target of the action has been fulfilled. - - A sequence node as the last child of the selector node to achieve the target. - - (Usually a number of) precondition nodes as the front children of the sequence node to check the preconditions for executing the action. - - One action node as the last child of the sequence node, whose preconditions are checked before and will immediate start the job when being ticked. - - The action will exert its defined effects after finishing the job successfully. Those effects should fulfilled all the target conditions in the selector node. - A brief example is as follows: -{ - selector: load_tool selector, - children: [ - { - target: check if inward_claw is equipped by left hand, - }, - { - sequence: load_tool, - children: [ - precondition: check if inward_claw is equippable, - precondition: check if left hand is free, - {action: equip inward_claw to left hand, effects: [left hand not free, inward_claw not equippable, left_hand holds inward_claw]}, - ], - }, - ], -} - -6. When performing a new action to fulfill a unsatisfied precondition, you can construct a subtree to replace the precondition and take that precondition as the target of the subtree. Then you construct the sequence of the subtree according to the new action you want to perform. -7. All of the control flow nodes, namely selectors and sequences, are without memory. A new behavior tree tick will always start from the root and traverse all the nodes even if they are ticked in the last tick. - -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/skeleton/prompt/sk_bt_skeleton_example.txt b/kios_bt_planning/backups/prompts/skeleton/prompt/sk_bt_skeleton_example.txt deleted file mode 100644 index af7cbe72..00000000 --- a/kios_bt_planning/backups/prompts/skeleton/prompt/sk_bt_skeleton_example.txt +++ /dev/null @@ -1,177 +0,0 @@ -[user] -I will give you some examples of the input and the output you will generate. -Example 1: -""" -- Input: -(define (problem robot_assembly_problem-problem) - (:domain robot_assembly_problem-domain) - (:objects - parallel_box1 parallel_box2 inward_claw outward_claw no_tool - tool - gear1 gear2 gear3 shaft1 shaft2 base - part - left_hand - hand - ) - (:init (can_manipulate parallel_box1 gear1) (can_manipulate outward_claw gear2) (can_manipulate inward_claw gear3) (can_manipulate parallel_box2 shaft1) (can_manipulate no_tool shaft2) (can_screw_to leg1 seat) (can_screw_to leg2 seat) (can_insert_to back seat) (can_screw_to nut1 seat) (can_screw_to nut2 seat) (can_screw_to blub base) (can_place_to lamp blub) (can_insert_to shaft1 base) (can_insert_to shaft2 base) (can_insert_to gear3 shaft2) (can_insert_to gear2 base) (can_insert_to gear1 shaft1) (is_inserted_to shaft1 base) (is_free left_hand) (is_free parallel_box1) (is_free parallel_box2) (is_free inward_claw) (is_free outward_claw) (is_free no_tool) (is_equippable parallel_box1) (is_equippable parallel_box2) (is_equippable inward_claw) (is_equippable outward_claw) (is_equippable no_tool)) - (:goal (and (is_inserted_to gear1 shaft1))) -) -- Output: -``` -{"task_plan": { - "action_sequence": [ - "load_tool(left_hand, parallel_box1)", - "pick_up(left_hand, parallel_box1, gear1)", - "insert(left_hand, parallel_box1, gear1, shaft1)" - ], - "action_instructions": [ - "equip the parallel_box1 on the left hand", - "pick up the gear1 with the parallel_box1 in the left hand", - "insert the gear1 into the shaft1 using the parallel_box1 in the left hand" - ], - "object_name": ["left_hand", "gear1", "parallel_box1", "shaft1"], - "behavior_tree": { - "summary": "selector to insert gear1 into shaft1", - "name": "selector: insert gear1 into shaft1", - "type_name": "selector", - "children": [ - { - "summary": "check if gear1 is inserted to shaft1", - "name": "target: is_inserted_to(gear1, shaft1)", - "type_name": "condition", - }, - { - "summary": "sequence to insert gear1 into shaft1", - "name": "sequence: insert(gear1, shaft1)", - "type_name": "sequence", - "children": [ - { - "summary": "selector to load parallel_box1", - "name": "selector: load_tool(left_hand, parallel_box1)", - "type_name": "selector", - "children": [ - { - "summary": "check if parallel_box1 is hold", - "name": "target: hold(left_hand, parallel_box1)", - "type_name": "condition", - }, - { - "summary": "sequence to load parallel_box1", - "name": "sequence: load_tool(left_hand, parallel_box1)", - "type_name": "sequence", - "children": [ - { - "summary": "check if parallel_box1 is equippable", - "name": "precondition: is_equippable(parallel_box1)", - "type_name": "condition", - }, - { - "summary": "check if left hand is free", - "name": "precondition: is_free(left_hand)", - "type_name": "condition", - }, - { - "summary": "equip parallel_box1 to left hand", - "name": "action: load_tool(left_hand, parallel_box1)", - "type_name": "action", - "effects": [ - { - "summary": "left_hand will be not free", - }, - { - "summary": "parallel_box1 will be not equippable", - }, - { - "summary": "left_hand will hold parallel_box1", - }, - ], - }, - ], - }, - ], - }, - { - "summary": "selector to pick_up gear1", - "name": "selector: pick_up(left_hand, parallel_box1, gear1)", - "type_name": "selector", - "children": [ - { - "summary": "check if parallel_box1 holds gear1", - "name": "target: hold(parallel_box1, gear1)", - "type_name": "condition", - }, - { - "summary": "sequence to pick_up gear1", - "name": "sequence: pick_up(left_hand, parallel_box1, gear1)", - "type_name": "sequence", - "children": [ - { - "summary": "check if parallel_box1 is free", - "name": "precondition: is_free(parallel_box1)", - "type_name": "condition", - }, - { - "summary": "check if left hand holds parallel_box1", - "name": "precondition: hold(left_hand, parallel_box1)", - "type_name": "condition", - }, - { - "summary": "pick up gear1 using parallel_box1", - "name": "action: pick_up(left_hand, parallel_box1, gear1)", - "type_name": "action", - "effects": [ - { - "summary": "parallel_box1 will hold gear1", - }, - { - "summary": "parallel_box1 will be not free", - }, - ], - }, - ], - }, - ], - }, - { - "summary": "check if gear1 can be inserted to shaft1", - "name": "precondition: can_insert_to(gear1, shaft1)", - "type_name": "condition", - }, - { - "summary": "insert gear1 to shaft1", - "name": "action: insert(left_hand, parallel_box1, gear1, shaft1)", - "type_name": "action", - "effects": [ - { - "summary": "parallel_box1 will be not holding gear1", - }, - { - "summary": "parallel_box1 will be free", - }, - { - "summary": "gear1 will be inserted to shaft1", - }, - ], - }, - ], - }, - ], -} -}, -"environment_before": {"objects": [ - {"name": "gear1", "properties": []}, - {"name": "shaft1", "properties": []}, - {"name": "parallel_box1", "properties": ["is_equippable", "is_free"]}, - {"name": "left_hand", "properties": ["is_free"]}, - ], - "constraints": [ - {"source": "gear1", "name": "can_insert_to", "target": "shaft1"}, - {"source": "parallel_box1", "name": "can_manipulate", "target": "gear1"}, - ], - "relations": [], -}, -"instruction_summary": "insert gear1 into shaft1", -"question":""} -``` -""" -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/skeleton/prompt/sk_bt_validation.txt b/kios_bt_planning/backups/prompts/skeleton/prompt/sk_bt_validation.txt deleted file mode 100644 index e69de29b..00000000 diff --git a/kios_bt_planning/backups/prompts/skeleton/prompt/sk_domain.txt b/kios_bt_planning/backups/prompts/skeleton/prompt/sk_domain.txt deleted file mode 100644 index bf3c7859..00000000 --- a/kios_bt_planning/backups/prompts/skeleton/prompt/sk_domain.txt +++ /dev/null @@ -1,89 +0,0 @@ -[user] -Following is the domain knowledge you need in pddl domain format, which includes: -- the object types in the world, -- the predicates for describing the states of the world, -- the actions that the robot can apply, with their precondtions and effects defined. -""" -(define (domain robot_assembly_problem-domain) - (:requirements :strips :typing) - (:types - thing - object - tool part hand - thing - ) - (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part)) - (:action pick_up - :parameters ( ?hand - hand ?tool - tool ?part - part) - :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part)) - :effect (and (hold ?tool ?part) (not (is_free ?tool)))) - (:action put_down - :parameters ( ?hand - hand ?tool - tool ?part - part) - :precondition (and (hold ?tool ?part) (hold ?hand ?tool)) - :effect (and (not (hold ?hand ?part)) (is_free ?tool))) - (:action place - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2))) - (:action detach - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2)))) - (:action insert - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2))) - (:action pull - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2)))) - (:action screw - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2))) - (:action unscrew - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2)))) - (:action load_tool - :parameters ( ?hand - hand ?tool - tool) - :precondition (and (is_equippable ?tool) (is_free ?hand)) - :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool))) - (:action unload_tool - :parameters ( ?hand - hand ?tool - tool) - :precondition (and (hold ?hand ?tool) (is_free ?tool)) - :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool)))) -) -""" -The precidates are explained as follows: -""" -"PREDICATE_LIST" -- is_free( or ): or is free and can hold something. (default: true) -- is_equippable(): is ready to be eqquipped by a robot hand. (default: true) -- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false) -- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false) -- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false) -- can_place_to(, ): can be placed into (placement assembly constraint) (default: false) -- hold(, ): is holding (default: false) -- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false) -- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false) -- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false) -""" - -The actions are explained as follows: -""" -"ROBOT_ACTION_LIST" -- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something. -- put_down(, , ): use to put down . As preconditions, should be holding and should be holding . -- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding . -- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into . -- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding . -- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into . -- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding . -- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into . -- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free. -- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free. -""" -the items in <> indicate the type of the instances needed for the predicates and actions. -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. diff --git a/kios_bt_planning/backups/prompts/skeleton/prompt/sk_environment.txt b/kios_bt_planning/backups/prompts/skeleton/prompt/sk_environment.txt deleted file mode 100644 index 4cd08f12..00000000 --- a/kios_bt_planning/backups/prompts/skeleton/prompt/sk_environment.txt +++ /dev/null @@ -1,31 +0,0 @@ -[user] -Information about world states are given as json dictionary. Example: -""" -{ - "objects": [ - {"name": "leg1", "properties": []}, - {"name": "seat", "properties": []}, - {"name": "cabinet", "properties": ["open"]}, - {"name": "parallel_box1", "properties": []}, - {"name": "table", "properties": []}, - ], - "constraints": [ - {"source": "leg1", "name": "can_screw_to", "target": "seat"}, - {"source": "parallel_box1", "name": "can_manipulate", "target": "leg1"}, - {"source": "cabinet", "name": "on_something", "target": "table"}, - ], - "relations": [ - {"source": "leg1", "name": "is_screwed_to", "target": "seat"}, - ], -} -""" -It contains three lists: - - "objects": the objects and their properties (as lists of strings) in the environment. The properties can be changed by robot actions. - - "constraints": the constraint relations that are enforced and should not be changed. They are knowledge about the world or some constraints set by the user. - - "relations": the relations in the environment. They can be changed by robot actions. -The properties, constraints and relations should always align with the [PREDICATE_LIST] in domain knowledge. -You should only concentrate on the objects, constraints and relations that are related to (or can be effected by) the current task. -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/skeleton/prompt/sk_output_format.txt b/kios_bt_planning/backups/prompts/skeleton/prompt/sk_output_format.txt deleted file mode 100644 index c7e7c60c..00000000 --- a/kios_bt_planning/backups/prompts/skeleton/prompt/sk_output_format.txt +++ /dev/null @@ -1,20 +0,0 @@ -[user] -You generate an action sequence to solve the task and contruct a behavior tree based on it in json format. -The output dictionary has five keys. -""" -- dictionary["task_plan"]: A dictionary containing information about the task plan. -- dictionary["environment_before"]: The environment state before executing the plan, described by the problem provided. -- dictionary["instruction_summary"]: A brief summary of the task plan. -- dictionary["question"]: If you don't have enough knowledge for solving the problem, you can ask questions about the knowledge you need. Leave this key empty if you can solve the task. -""" -Three keys exist in dictionary["task_plan"]. -""" -- dictionary["task_plan"]["action_sequence"]: Contains a list of robot actions. Only the actions defined in the "ROBOT_ACTION LIST" will be used. -- dictionary["task_plan"]["action_instructions"]: contains a list of instructions corresponding to dictionary["task_plan"]["action_sequence"] to explain the actions. -- dictionary["task_plan"]["manipulated_objects"]: The list of the names of the manipulated object. It only includes the objects that are related to the current task. -- dictionary["task_plan"]["behavior_tree"]: contains a dictionary of the behavior tree constructed according to the dictionary ["task_plan"]["action_sequence"] -""" -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/skeleton/prompt/sk_problem.txt b/kios_bt_planning/backups/prompts/skeleton/prompt/sk_problem.txt deleted file mode 100644 index ec47c472..00000000 --- a/kios_bt_planning/backups/prompts/skeleton/prompt/sk_problem.txt +++ /dev/null @@ -1,23 +0,0 @@ -[user] -Information about the problem is given as a pddl problem. -It includes the following information: -- Objects in the world: all instances of the types defined in the domain knowledge. For example, "parallel_box1" is an instance of the type "tool", while "gear1" is an instance of the type "part". -- Initial states: the world states to start with, described with [PREDICATE_LIST] from domain knowledge. For example, "is_free hand" means the object "hand" is free and holds nothing. -- Goal states: the world states to achieve, described with [PREDICATE_LIST] from domain knowledge. -Example: -""" -(define (problem robot_assembly_problem-problem) - (:domain robot_assembly_problem-domain) - (:objects - parallel_box1 parallel_box2 inward_claw outward_claw no_tool - tool - gear1 gear2 gear3 shaft1 shaft2 base - part - left_hand - hand - ) - (:init (can_manipulate parallel_box1 gear1) (can_manipulate outward_claw gear2) (can_manipulate inward_claw gear3) (can_manipulate parallel_box2 shaft1) (can_manipulate no_tool shaft2) (can_screw_to leg1 seat) (can_screw_to leg2 seat) (can_insert_to back seat) (can_screw_to nut1 seat) (can_screw_to nut2 seat) (can_screw_to blub base) (can_place_to lamp blub) (can_insert_to shaft1 base) (can_insert_to shaft2 base) (can_insert_to gear3 shaft2) (can_insert_to gear2 base) (can_insert_to gear1 shaft1) (is_free left_hand) (is_free parallel_box1) (is_free parallel_box2) (is_free inward_claw) (is_free outward_claw) (is_free no_tool) 1) (is_equippable parallel_box1) (is_equippable parallel_box2) (is_equippable inward_claw) (is_equippable outward_claw) (is_equippable no_tool)) - (:goal (and (is_inserted_to shaft1 base))) -) -""" -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/skeleton/prompt/sk_role.txt b/kios_bt_planning/backups/prompts/skeleton/prompt/sk_role.txt deleted file mode 100644 index 6ce736a0..00000000 --- a/kios_bt_planning/backups/prompts/skeleton/prompt/sk_role.txt +++ /dev/null @@ -1,5 +0,0 @@ -[user] -You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you break it down and construct robot task plans represented by behaiovr trees. -Please do not begin working until I say "Start working." Instead, simply output the message "Waiting for next input." Understood? -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/skeleton/query/query.txt b/kios_bt_planning/backups/prompts/skeleton/query/query.txt deleted file mode 100644 index 8ec7310d..00000000 --- a/kios_bt_planning/backups/prompts/skeleton/query/query.txt +++ /dev/null @@ -1,16 +0,0 @@ -Start working. Resume from the problem below. -""" -{"problem": [PROBLEM]} -""" -The dictionary that you return should be formatted as json file. Follow these rules: -1. Always equip the proper tools before performing specific object manipulaion. -2. Make sure that each element of the ["action_instructions"] explains corresponding element of the ["action_sequence"]. Refer to the "ROBOT ACTION LIST" to understand the elements of ["action_sequence"]. -3, The length of the ["action_instructions"] list and the ["action_sequence"] list should be the same. -4. Never left ',' at the end of the list. Use json boolean values, "true" or "false", instead of "True" or "False". -5. Keep track of all items listed in the "objects" section of the "environment_before" field. Please make sure you fill out the "objects" section for all listed items. -6. Use the "PREDICATE_LIST" for the states in the problem. -7. All keys of the dictionary should be double-quoted. -8. Insert ``` at the beginning and the end of the dictionary to separate it from the rest of your response. -9. Make sure that you output a consistent manipultation as a single arm robot. For example, grasping an object should not occur in successive steps. -10. When generating the behavior tree, make sure the tree has the asked structure. -Adhere to the output format I defined above. Follow the rules above. Think step by step. \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/skeleton/system/system.txt b/kios_bt_planning/backups/prompts/skeleton/system/system.txt deleted file mode 100644 index 26f2afe2..00000000 --- a/kios_bt_planning/backups/prompts/skeleton/system/system.txt +++ /dev/null @@ -1 +0,0 @@ -You are an excellent interpreter of instructions for robotic assembly tasks. Given the knowledge you need, the world states and some instructions, you break it down and construct robot task plans represented by behaiovr trees based on the action sequences. \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/spsk_refine/prompt/dicard_refine_bt.txt b/kios_bt_planning/backups/prompts/spsk_refine/prompt/dicard_refine_bt.txt deleted file mode 100644 index d7233761..00000000 --- a/kios_bt_planning/backups/prompts/spsk_refine/prompt/dicard_refine_bt.txt +++ /dev/null @@ -1,140 +0,0 @@ -[user] -You should refine the behavior tree skeleton into a full behavior tree, which is in json dictionary format and have a structure like the following example: -""" -{ - "summary": "selector to load inward_claw to left hand", - "name": "selector: load_tool(left_hand, inward_claw)", - "identifier": 3, - "type_name": "selector", - "children": [ - { - "summary": "check if left_hand is holding inward_claw", - "name": "target: hold(left_hand, inward_claw)", - "identifier": 4, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "inward_claw", - "status": true, - } - ], - }, - { - - "summary": "sequence to load inward_claw to left hand", - "name": "sequence: load_tool(left_hand, inward_claw)", - "identifier": 5, - "type_name": "sequence", - "children": [ - { - "summary": "check if inward_claw is equippable", - "name": "precondition: is_equippable(inward_claw)", - "identifier": 6, - "type_name": "condition", - "conditions": [ - { - "object_name": "inward_claw", - "property_name": "is_equippable", - "property_value": null, - "status": true, - } - ], - }, - { - "summary": "check if left hand is free", - "name": "precondition: is_free(left_hand)", - "identifier": 7, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "is_free", - "property_value": null, - "status": true, - } - ], - }, - { - "summary": "equip inward_claw to left hand", - "name": "action: load_tool(left_hand, inward_claw)", - "identifier": 8, - "type_name": "action", - "effects": [ - { - "object_name": "left_hand", - "property_name": "is_free", - "property_value": null, - "status": false, - }, - { - "object_name": "inward_claw", - "property_name": "is_equippable", - "property_value": null, - "status": false, - }, - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "inward_claw", - "status": true, - }, - ], - }, - ], - }, - ], -} -""" -The dictionary that you return should be formatted as json dictionary. Follow these rules: -1. Every node should have four basic keys: - - "summary": a short description of the node. - - "name": the name of the node. The name should follow the definitions in the domain knowledge. - - "identifier": the unique id of the node. - - "type_name": can be: "selector", "sequence", "condition", "action". -2. The control flow node, "selector" and "sequence" nodes, can have "children", which is a list of children nodes. -3. The "condition" nodes that are children of "selectors" are "targets". Those that are children of "sequences" are preconditions. -4. The "condition" nodes should have the key "conditions", which is a list of conditions to be checked. Each condition should have the following keys: - - "object_name": the object to check. - - "property_name": the property/relation of the object to be checked. - - "property_value": this should be the other object in the relation. For property it should be null. - - "status": true if the property/relation is expected to be true, false otherwise. -5. The "action" nodes should have a key "effects", which is a list of the to-be-exerted effects of the action. Each effect should have the following keys: - - "object_name": the affected object. - - "property_name": the affected property/relation of the object. - - "property_value": this should be the other object if this is a relation. For property it should be null. - - "status": true if the effect is to add this property/relation. false if is to remove it. -6. All the subtrees always have their corresponding action in ROBOT_ACTION_LIST. They are used to represent the actions in the action sequence. Its basic structure is: - - A selector as the root node. - - (Usually one) target condition node as the first child of the selector to check if the target of the action has been fulfilled. - - A sequence node as the last child of the selector node to achieve the target. - - (Usually a number of) precondition nodes as the front children of the sequence node to check the preconditions for executing the action. - - One action node as the last child of the sequence node, whose preconditions are checked before and will immediate start the job when being ticked. - - The action will exert its defined effects after finishing the job successfully. Those effects should fulfilled all the target conditions in the selector node. - A brief example: -{ - selector: load_tool selector, - children: [ - { - target: check if inward_claw is equipped by left hand, - }, - { - sequence: load_tool, - children: [ - precondition: check if inward_claw is equippable, - precondition: check if left hand is free, - {action: equip inward_claw to left hand, effects: [left hand not free, inward_claw not equippable, left_hand holds inward_claw]}, - ], - }, - ], -} - -7. When performing a new action to fulfill a unsatisfied precondition, you can construct a subtree to replace the precondition and take that unfulfilled precondition as the target of the subtree. Then you construct the sequence of the subtree according to the new action you want to perform. -8. The behavior tree should be constructed based on the action sequence you planned and the action definitions in ROBOT_ACTION_LIST. -9. All of the control flow nodes, namely selectors and sequences, are without memory. A new behavior tree tick will always start from the root and traverse all the nodes even if they are ticked in the last tick. - -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/spsk_refine/prompt/discard_cot_bt.txt b/kios_bt_planning/backups/prompts/spsk_refine/prompt/discard_cot_bt.txt deleted file mode 100644 index e2fc475c..00000000 --- a/kios_bt_planning/backups/prompts/spsk_refine/prompt/discard_cot_bt.txt +++ /dev/null @@ -1,68 +0,0 @@ -[user] -The behavior tree should be in json dictionary format and have a structure like the following example: -""" -{ - "summary": "selector to load inward_claw to left hand", - "children": [ - { - "summary": "condition node to check if inward_claw is held by left hand", - }, - { - "summary": "sequence to load inward_claw to left hand", - "children": [ - { - "summary": "condition node to check if inward_claw is equippable", - }, - { - "summary": "condition node to check if left hand is free", - }, - { - "summary": "action node to equip inward_claw to left hand", - "effects": [ - {"summary": "left hand will be not free anymore",}, - {"summary": "inward_claw will be not equippable anymore",}, - {"summary": "left_hand will hold inward_claw",}, - ], - }, - ], - }, - ], -} -""" -Follow these rules: -1. Every node are described by a summary, which is a short interpretive description of the node. It should include the type of the node and the purpose of the node. -2. The control flow nodes, "selector" and "sequence", can have "children", which is a list of children nodes. -3. The "condition" nodes that are children of "selectors" are "targets". Those that are children of "sequences" are preconditions. -4. The "action" nodes should have a key "effects" with a list of dictionaries, which describes the to-be-exerted effects of the action. The effects are shortly described under the key "summary". -5. All the subtrees always have their corresponding action in ROBOT_ACTION_LIST. They are used to represent the actions in the action sequence. The recommended structure is: - - A selector as the root node. - - (Usually one) target condition node as the first child of the selector to check if the target of the action has been fulfilled. - - A sequence node as the last child of the selector node to achieve the target. - - (Usually a number of) precondition nodes as the front children of the sequence node to check the preconditions for executing the action. - - One action node as the last child of the sequence node, whose preconditions are checked before and will immediate start the job when being ticked. - - The action will exert its defined effects after finishing the job successfully. Those effects should fulfilled all the target conditions in the selector node. - A brief example is as follows: -{ - selector: load_tool selector, - children: [ - { - target: check if inward_claw is equipped by left hand, - }, - { - sequence: load_tool, - children: [ - precondition: check if inward_claw is equippable, - precondition: check if left hand is free, - {action: equip inward_claw to left hand, effects: [left hand not free, inward_claw not equippable, left_hand holds inward_claw]}, - ], - }, - ], -} - -6. When performing a new action to fulfill a unsatisfied precondition, you can construct a subtree to replace the precondition and take that unfulfilled precondition as the target of the subtree. Then you construct the sequence of the subtree according to the new action you want to perform. -7. All of the control flow nodes, namely selectors and sequences, are without memory. A new behavior tree tick will always start from the root and traverse all the nodes even if they are ticked in the last tick. - -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/spsk_refine/prompt/discard_cot_example.txt b/kios_bt_planning/backups/prompts/spsk_refine/prompt/discard_cot_example.txt deleted file mode 100644 index 3d8d6d5b..00000000 --- a/kios_bt_planning/backups/prompts/spsk_refine/prompt/discard_cot_example.txt +++ /dev/null @@ -1,127 +0,0 @@ -[user] -I will give you some examples of the input and the output you will generate. -Example 1: -""" -- Input: -(define (problem robot_assembly_problem-problem) - (:domain robot_assembly_problem-domain) - (:objects - parallel_box1 parallel_box2 inward_claw outward_claw no_tool - tool - gear1 gear2 gear3 shaft1 shaft2 base - part - left_hand - hand - ) - (:init (can_manipulate parallel_box1 gear1) (can_manipulate outward_claw gear2) (can_manipulate inward_claw gear3) (can_manipulate parallel_box2 shaft1) (can_manipulate no_tool shaft2) (can_screw_to leg1 seat) (can_screw_to leg2 seat) (can_insert_to back seat) (can_screw_to nut1 seat) (can_screw_to nut2 seat) (can_screw_to blub base) (can_place_to lamp blub) (can_insert_to shaft1 base) (can_insert_to shaft2 base) (can_insert_to gear3 shaft2) (can_insert_to gear2 base) (can_insert_to gear1 shaft1) (is_inserted_to shaft1 base) (is_free left_hand) (is_free parallel_box1) (is_free parallel_box2) (is_free inward_claw) (is_free outward_claw) (is_free no_tool) (is_equippable parallel_box1) (is_equippable parallel_box2) (is_equippable inward_claw) (is_equippable outward_claw) (is_equippable no_tool)) - (:goal (and (is_inserted_to gear1 shaft1))) -) -- Output: -``` -{"task_plan": { - "action_sequence": [ - "load_tool(left_hand, parallel_box1)", - "pick_up(left_hand, parallel_box1, gear1)", - "insert(left_hand, parallel_box1, gear1, shaft1)" - ], - "action_instructions": [ - "equip the parallel_box1 on the left hand", - "pick up the gear1 with the parallel_box1 in the left hand", - "insert the gear1 into the shaft1 using the parallel_box1 in the left hand" - ], - "object_name": ["left_hand", "gear1", "parallel_box1", "shaft1"], - "behavior_tree": { - "summary": "selector to insert gear1 into shaft1", - "children": [ - { - "summary": "condition node to check if gear1 is inserted to shaft1", - }, - { - "summary": "sequence to insert gear1 into shaft1", - "children": [ - { - "summary": "selector to load_tool - "children": [ - { - "summary": "condition node to check if parallel_box1 is hold", - }, - { - "summary": "sequence to load_tool", - "children": [ - { - "summary": "condition node to check if parallel_box1 is equippable", - }, - { - "summary": "condition node to check if left hand is free", - }, - { - "summary": "action node to equip parallel_box1 to left hand", - "effects": [ - {"summary": "left_hand will be not free",}, - {"summary": "parallel_box1 will be not equippable",}, - {"summary": "left_hand will hold parallel_box1",}, - ], - }, - ], - }, - ], - }, - { - "summary": "selector to pick_up", - "children": [ - { - "summary": "condition node to check if parallel_box1 holds gear1", - }, - { - "summary": "sequence to pick_up", - "children": [ - { - "summary": "condition node to check if parallel_box1 is free", - }, - { - "summary": "condition node to check if left hand holds parallel_box1", - }, - { - "summary": "action node to pick up gear1 using parallel_box1", - "effects": [ - {"summary": "parallel_box1 will hold gear1",}, - {"summary": "parallel_box1 will be not free",}, - ], - }, - ], - }, - ], - }, - { - "summary": "condition node to check if gear1 can be inserted to shaft1", - }, - { - "summary": "action node to insert gear1 to shaft1", - "effects": [ - {"summary": "parallel_box1 will be not holding gear1",}, - {"summary": "parallel_box1 will be free",}, - {"summary": "gear1 will be inserted to shaft1",}, - ], - }, - ], - }, - ], -} -}, -"initial_state": {"objects": [ - {"name": "gear1", "properties": []}, - {"name": "shaft1", "properties": []}, - {"name": "parallel_box1", "properties": ["is_equippable", "is_free"]}, - {"name": "left_hand", "properties": ["is_free"]}, - ], - "constraints": [ - {"source": "gear1", "name": "can_insert_to", "target": "shaft1"}, - {"source": "parallel_box1", "name": "can_manipulate", "target": "gear1"}, - ], - "relations": [], -}, -"instruction_summary": "insert gear1 into shaft1", -"question":""} -``` -""" -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/spsk_refine/prompt/discard_refine_refine.txt b/kios_bt_planning/backups/prompts/spsk_refine/prompt/discard_refine_refine.txt deleted file mode 100644 index 777e8f26..00000000 --- a/kios_bt_planning/backups/prompts/spsk_refine/prompt/discard_refine_refine.txt +++ /dev/null @@ -1,247 +0,0 @@ -[user] -You should refine the behavior tree skeleton into a full behavior tree in json dictionary format. - -For each node, you should: -1. add an unique integer "identifier". -2. add its "type_name" accordingly. (precondition and target are both "condition") - -For control flow nodes, namely selectors and sequences, an example is shown as follows: -the skeleton: -""" -{ - "summary": "selector to load no_tool to hand", - "name": "selector: load_tool", - "children": [...], -} - -""" -after refinement: -""" -{ - "summary": "selector to load no_tool to hand", - "name": "selector: load_tool(left_hand, no_tool)", - "identifier": 3, - "type_name": "selector", - "children": [...], -} -""" - -For condition nodes, you should: -1. refine and add the key "conditions", which is a list of conditions to be checked. Each condition should have the following keys: - - "object_name": the object to check. - - "property_name": the property/relation of the object to be checked. - - "property_value": this should be the other object in the relation. For property it should be null. - - "status": true if the property/relation is expected to be true, false otherwise. -An example is shown below: -the skeleton: -""" -{ - "summary": "check if left_hand is holding parallel_box1", - "name": "target: hold(left_hand, parallel_box1)", -} -""" -after refinement: -""" -{ - "summary": "check if left_hand is holding parallel_box1", - "name": "target: hold(left_hand, parallel_box1)", - "identifier": 4, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "parallel_box1", - "status": true, - } - ], -}, -""" - -For action nodes, you should: -1. refine and replace the summaries in the key "effects", which is a list of the to-be-exerted effects of the action. Each effect should have the following keys: - - "object_name": the affected object. - - "property_name": the affected property/relation of the object. - - "property_value": this should be the other object if this is a relation. For property it should be null. - - "status": true if the effect is to add this property/relation. false if is to remove it. -An example is shown below: -the skeleton: -""" -{ - "summary": "left hand equips tool1", - "name": "action: load_tool(left_hand, tool1)", - "effects": [ - {"summary": "left hand will be not free anymore",}, - {"summary": "tool1 will be not equippable anymore",}, - {"summary": "left_hand will hold tool1",}, - ], -}, -""" -after refinement: -""" -{ - "summary": "left hand equips tool1", - "name": "action: load_tool(left_hand, tool1)", - "identifier": 8, - "type_name": "action", - "effects": [ - { - "object_name": "left_hand", - "property_name": "is_free", - "property_value": null, - "status": false, - }, - { - "object_name": "tool1", - "property_name": "is_equippable", - "property_value": null, - "status": false, - }, - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "tool1", - "status": true, - }, - ], -}, -""" - -""" -{ - "summary": "selector to load inward_claw to left hand", - "name": "selector: load_tool(left_hand, inward_claw)", - "identifier": 3, - "type_name": "selector", - "children": [ - { - "summary": "check if left_hand is holding inward_claw", - "name": "target: hold(left_hand, inward_claw)", - "identifier": 4, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "inward_claw", - "status": true, - } - ], - }, - { - - "summary": "sequence to load inward_claw to left hand", - "name": "sequence: load_tool(left_hand, inward_claw)", - "identifier": 5, - "type_name": "sequence", - "children": [ - { - "summary": "check if inward_claw is equippable", - "name": "precondition: is_equippable(inward_claw)", - "identifier": 6, - "type_name": "condition", - "conditions": [ - { - "object_name": "inward_claw", - "property_name": "is_equippable", - "property_value": null, - "status": true, - } - ], - }, - { - "summary": "check if left hand is free", - "name": "precondition: is_free(left_hand)", - "identifier": 7, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "is_free", - "property_value": null, - "status": true, - } - ], - }, - { - "summary": "equip inward_claw to left hand", - "name": "action: load_tool(left_hand, inward_claw)", - "identifier": 8, - "type_name": "action", - "effects": [ - { - "object_name": "left_hand", - "property_name": "is_free", - "property_value": null, - "status": false, - }, - { - "object_name": "inward_claw", - "property_name": "is_equippable", - "property_value": null, - "status": false, - }, - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "inward_claw", - "status": true, - }, - ], - }, - ], - }, - ], -} -""" -The dictionary that you return should be formatted as json dictionary. Follow these rules: -1. Every node should have four basic keys: - - "summary": a short description of the node. - - "name": the name of the node. The name should follow the definitions in the domain knowledge. - - "identifier": the unique id of the node. - - "type_name": can be: "selector", "sequence", "condition", "action". -2. The control flow node, "selector" and "sequence" nodes, can have "children", which is a list of children nodes. -3. The "condition" nodes that are children of "selectors" are "targets". Those that are children of "sequences" are preconditions. -4. The "condition" nodes should have the key "conditions", which is a list of conditions to be checked. Each condition should have the following keys: - - "object_name": the object to check. - - "property_name": the property/relation of the object to be checked. - - "property_value": this should be the other object in the relation. For property it should be null. - - "status": true if the property/relation is expected to be true, false otherwise. -5. The "action" nodes should have a key "effects", which is a list of the to-be-exerted effects of the action. Each effect should have the following keys: - - "object_name": the affected object. - - "property_name": the affected property/relation of the object. - - "property_value": this should be the other object if this is a relation. For property it should be null. - - "status": true if the effect is to add this property/relation. false if is to remove it. -6. All the subtrees always have their corresponding action in ROBOT_ACTION_LIST. They are used to represent the actions in the action sequence. Its basic structure is: - - A selector as the root node. - - (Usually one) target condition node as the first child of the selector to check if the target of the action has been fulfilled. - - A sequence node as the last child of the selector node to achieve the target. - - (Usually a number of) precondition nodes as the front children of the sequence node to check the preconditions for executing the action. - - One action node as the last child of the sequence node, whose preconditions are checked before and will immediate start the job when being ticked. - - The action will exert its defined effects after finishing the job successfully. Those effects should fulfilled all the target conditions in the selector node. - A brief example: -{ - selector: load_tool selector, - children: [ - { - target: check if inward_claw is equipped by left hand, - }, - { - sequence: load_tool, - children: [ - precondition: check if inward_claw is equippable, - precondition: check if left hand is free, - {action: equip inward_claw to left hand, effects: [left hand not free, inward_claw not equippable, left_hand holds inward_claw]}, - ], - }, - ], -} - -7. When performing a new action to fulfill a unsatisfied precondition, you can construct a subtree to replace the precondition and take that unfulfilled precondition as the target of the subtree. Then you construct the sequence of the subtree according to the new action you want to perform. -8. The behavior tree should be constructed based on the action sequence you planned and the action definitions in ROBOT_ACTION_LIST. -9. All of the control flow nodes, namely selectors and sequences, are without memory. A new behavior tree tick will always start from the root and traverse all the nodes even if they are ticked in the last tick. - -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/spsk_refine/prompt/refine_action.txt b/kios_bt_planning/backups/prompts/spsk_refine/prompt/refine_action.txt deleted file mode 100644 index 06e74ce2..00000000 --- a/kios_bt_planning/backups/prompts/spsk_refine/prompt/refine_action.txt +++ /dev/null @@ -1,54 +0,0 @@ -[user] -For action nodes, you should: -1. refine and replace the summaries in the key "effects", which is a list of the to-be-exerted effects of the action. Each effect should have the following keys: - - "object_name": the affected object. - - "property_name": the affected property/relation of the object. - - "property_value": this should be the other object if this is a relation. For property it should be null. - - "status": true if the effect is to add this property/relation. false if is to remove it. -An example is shown below: -the skeleton: -""" -{ - "summary": "left hand equips tool1", - "name": "action: load_tool(left_hand, tool1)", - "effects": [ - {"summary": "left hand will be not free anymore",}, - {"summary": "tool1 will be not equippable anymore",}, - {"summary": "left_hand will hold tool1",}, - ], -}, -""" -after refinement: -""" -{ - "summary": "left hand equips tool1", - "name": "action: load_tool(left_hand, tool1)", - "identifier": 8, - "type_name": "action", - "effects": [ - { - "object_name": "left_hand", - "property_name": "is_free", - "property_value": null, - "status": false, - }, - { - "object_name": "tool1", - "property_name": "is_equippable", - "property_value": null, - "status": false, - }, - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "tool1", - "status": true, - }, - ], -}, -""" - -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/spsk_refine/prompt/refine_chain.txt b/kios_bt_planning/backups/prompts/spsk_refine/prompt/refine_chain.txt deleted file mode 100644 index 8ab56a97..00000000 --- a/kios_bt_planning/backups/prompts/spsk_refine/prompt/refine_chain.txt +++ /dev/null @@ -1,15 +0,0 @@ -[user] -Follow this workflow to solve the task: - -1. Read the problem description. Parse and generate the "initial_state" in the output dictionary. -2. Interpret the goal of the problem as goal_state and write into the "instruction_summary" in the output dictionary. -3. Try to solve the problem, achieve the goal_state from the initial_state based on the domain knowledge provided. Generate a action sequence with the actions defined in and write it into dictionary["task_plan"]["action_sequence"] of the output. -4. If you cannot find a plan to solve the task, ask question with nature language in the "question" of the output dictionary. -5. Explain the actions in the action sequence and write the output dictionary["task_plan"]["action_instructions"]. -6. Collect the objects in the world that are effected by the action sequence, write into dictionary["task_plan"]["manipulated_objects"]. -7. Generate the behavior tree based on the action sequence and write into dictionary["task_plan"]["behavior_tree"]. - -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/spsk_refine/prompt/refine_condition.txt b/kios_bt_planning/backups/prompts/spsk_refine/prompt/refine_condition.txt deleted file mode 100644 index ff0f396c..00000000 --- a/kios_bt_planning/backups/prompts/spsk_refine/prompt/refine_condition.txt +++ /dev/null @@ -1,37 +0,0 @@ -[user] -For condition nodes, you should: -1. refine and add the key "conditions", which is a list of conditions to be checked. Each condition should have the following keys: - - "object_name": the object to check. - - "property_name": the property/relation of the object to be checked. - - "property_value": this should be the other object in the relation. For property it should be null. - - "status": true if the property/relation is expected to be true, false otherwise. -An example is shown below: -the skeleton: -""" -{ - "summary": "check if left_hand is holding parallel_box1", - "name": "target: hold(left_hand, parallel_box1)", -} -""" -after refinement: -""" -{ - "summary": "check if left_hand is holding parallel_box1", - "name": "target: hold(left_hand, parallel_box1)", - "identifier": 4, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "parallel_box1", - "status": true, - } - ], -}, -""" - -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/spsk_refine/prompt/refine_controlflow.txt b/kios_bt_planning/backups/prompts/spsk_refine/prompt/refine_controlflow.txt deleted file mode 100644 index 28828ac2..00000000 --- a/kios_bt_planning/backups/prompts/spsk_refine/prompt/refine_controlflow.txt +++ /dev/null @@ -1,26 +0,0 @@ -[user] -For control flow nodes, namely selectors and sequences, an example is shown as follows: -the skeleton: -""" -{ - "summary": "selector to load no_tool to hand", - "name": "selector: load_tool", - "children": [...], -} - -""" -after refinement: -""" -{ - "summary": "selector to load no_tool to hand", - "name": "selector: load_tool(left_hand, no_tool)", - "identifier": 3, - "type_name": "selector", - "children": [...], -} -""" - -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/spsk_refine/prompt/refine_domain.txt b/kios_bt_planning/backups/prompts/spsk_refine/prompt/refine_domain.txt deleted file mode 100644 index bf3c7859..00000000 --- a/kios_bt_planning/backups/prompts/spsk_refine/prompt/refine_domain.txt +++ /dev/null @@ -1,89 +0,0 @@ -[user] -Following is the domain knowledge you need in pddl domain format, which includes: -- the object types in the world, -- the predicates for describing the states of the world, -- the actions that the robot can apply, with their precondtions and effects defined. -""" -(define (domain robot_assembly_problem-domain) - (:requirements :strips :typing) - (:types - thing - object - tool part hand - thing - ) - (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part)) - (:action pick_up - :parameters ( ?hand - hand ?tool - tool ?part - part) - :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part)) - :effect (and (hold ?tool ?part) (not (is_free ?tool)))) - (:action put_down - :parameters ( ?hand - hand ?tool - tool ?part - part) - :precondition (and (hold ?tool ?part) (hold ?hand ?tool)) - :effect (and (not (hold ?hand ?part)) (is_free ?tool))) - (:action place - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2))) - (:action detach - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2)))) - (:action insert - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2))) - (:action pull - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2)))) - (:action screw - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2))) - (:action unscrew - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2)))) - (:action load_tool - :parameters ( ?hand - hand ?tool - tool) - :precondition (and (is_equippable ?tool) (is_free ?hand)) - :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool))) - (:action unload_tool - :parameters ( ?hand - hand ?tool - tool) - :precondition (and (hold ?hand ?tool) (is_free ?tool)) - :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool)))) -) -""" -The precidates are explained as follows: -""" -"PREDICATE_LIST" -- is_free( or ): or is free and can hold something. (default: true) -- is_equippable(): is ready to be eqquipped by a robot hand. (default: true) -- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false) -- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false) -- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false) -- can_place_to(, ): can be placed into (placement assembly constraint) (default: false) -- hold(, ): is holding (default: false) -- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false) -- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false) -- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false) -""" - -The actions are explained as follows: -""" -"ROBOT_ACTION_LIST" -- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something. -- put_down(, , ): use to put down . As preconditions, should be holding and should be holding . -- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding . -- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into . -- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding . -- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into . -- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding . -- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into . -- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free. -- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free. -""" -the items in <> indicate the type of the instances needed for the predicates and actions. -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. diff --git a/kios_bt_planning/backups/prompts/spsk_refine/prompt/refine_help.txt b/kios_bt_planning/backups/prompts/spsk_refine/prompt/refine_help.txt deleted file mode 100644 index c81cfca9..00000000 --- a/kios_bt_planning/backups/prompts/spsk_refine/prompt/refine_help.txt +++ /dev/null @@ -1,11 +0,0 @@ -[user] -You should refine the behavior tree skeleton into a full behavior tree in json dictionary format. - -1. Refer to "initial_state" of input to find the necessary objects, properties and relationships. -2. Refer to the domain knowledge to understand the actions (corresponds to "ROBOT_ACTION_LIST") and conditions (corresponds to "PREDICATE_LIST"). -3. The name of the node (if it has) also provides information for its refinement. - -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/spsk_refine/prompt/refine_input_format.txt b/kios_bt_planning/backups/prompts/spsk_refine/prompt/refine_input_format.txt deleted file mode 100644 index 27a2fdef..00000000 --- a/kios_bt_planning/backups/prompts/spsk_refine/prompt/refine_input_format.txt +++ /dev/null @@ -1,127 +0,0 @@ -[user] -The input you get and the output you return should be in json dictionary format. -The dictionary contains the following keys: -""" -- dictionary["task_plan"]: A dictionary containing the task plan. -- dictionary["initial_state"]: The initial world state before executing the plan. -- dictionary["instruction_summary"]: A brief summary of the task plan. -- dictionary["question"]: Question for more knowledge that are unknown yet but necessary for making the plan. Normally empty. -""" -Four keys exist in dictionary["task_plan"]. -""" -- dictionary["task_plan"]["action_sequence"]: Contains a list of robot actions. -- dictionary["task_plan"]["action_instructions"]: contains a list of instructions to explain the actions in dictionary["task_plan"]["action_sequence"]. -- dictionary["task_plan"]["manipulated_objects"]: The list of the objects that are related to the current task. -- dictionary["task_plan"]["behavior_tree"]: contains a dictionary of the behavior tree skeleton constructed according to the dictionary ["task_plan"]["action_sequence"] -""" -The following is an input example: -``` -{"task_plan": { - "action_sequence": [ - "load_tool(left_hand, parallel_box1)", - "pick_up(left_hand, parallel_box1, gear1)", - "insert(left_hand, parallel_box1, gear1, shaft1)" - ], - "action_instructions": [ - "equip the parallel_box1 on the left hand", - "pick up the gear1 with the parallel_box1 in the left hand", - "insert the gear1 into the shaft1 using the parallel_box1 in the left hand" - ], - "object_name": ["left_hand", "gear1", "parallel_box1", "shaft1"], - "behavior_tree": { - "summary": "selector to insert gear1 into shaft1", - "children": [ - { - "summary": "condition node to check if gear1 is inserted to shaft1", - }, - { - "summary": "sequence to insert gear1 into shaft1", - "children": [ - { - "summary": "selector to load_tool - "children": [ - { - "summary": "condition node to check if parallel_box1 is hold", - }, - { - "summary": "sequence to load_tool", - "children": [ - { - "summary": "condition node to check if parallel_box1 is equippable", - }, - { - "summary": "condition node to check if left hand is free", - }, - { - "summary": "action node to equip parallel_box1 to left hand", - "effects": [ - {"summary": "left_hand will be not free",}, - {"summary": "parallel_box1 will be not equippable",}, - {"summary": "left_hand will hold parallel_box1",}, - ], - }, - ], - }, - ], - }, - { - "summary": "selector to pick_up", - "children": [ - { - "summary": "condition node to check if parallel_box1 holds gear1", - }, - { - "summary": "sequence to pick_up", - "children": [ - { - "summary": "condition node to check if parallel_box1 is free", - }, - { - "summary": "condition node to check if left hand holds parallel_box1", - }, - { - "summary": "action node to pick up gear1 using parallel_box1", - "effects": [ - {"summary": "parallel_box1 will hold gear1",}, - {"summary": "parallel_box1 will be not free",}, - ], - }, - ], - }, - ], - }, - { - "summary": "condition node to check if gear1 can be inserted to shaft1", - }, - { - "summary": "action node to insert gear1 to shaft1", - "effects": [ - {"summary": "parallel_box1 will be not holding gear1",}, - {"summary": "parallel_box1 will be free",}, - {"summary": "gear1 will be inserted to shaft1",}, - ], - }, - ], - }, - ], -} -}, -"initial_state": {"objects": [ - {"name": "gear1", "properties": []}, - {"name": "shaft1", "properties": []}, - {"name": "parallel_box1", "properties": ["is_equippable", "is_free"]}, - {"name": "left_hand", "properties": ["is_free"]}, - ], - "constraints": [ - {"source": "gear1", "name": "can_insert_to", "target": "shaft1"}, - {"source": "parallel_box1", "name": "can_manipulate", "target": "gear1"}, - ], - "relations": [], -}, -"instruction_summary": "insert gear1 into shaft1", -"question":""} -``` -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/spsk_refine/prompt/refine_output_format.txt b/kios_bt_planning/backups/prompts/spsk_refine/prompt/refine_output_format.txt deleted file mode 100644 index d40e17ea..00000000 --- a/kios_bt_planning/backups/prompts/spsk_refine/prompt/refine_output_format.txt +++ /dev/null @@ -1,19 +0,0 @@ -[user] -The output dictionary has four keys. -""" -- dictionary["task_plan"] -- dictionary["initial_state"] -- dictionary["instruction_summary"] -- dictionary["question"] -""" -Four keys exist in dictionary["task_plan"]. -""" -- dictionary["task_plan"]["action_sequence"] -- dictionary["task_plan"]["action_instructions"] -- dictionary["task_plan"]["manipulated_objects"] -- dictionary["task_plan"]["behavior_tree"] -""" -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/spsk_refine/prompt/refine_role.txt b/kios_bt_planning/backups/prompts/spsk_refine/prompt/refine_role.txt deleted file mode 100644 index de12cb04..00000000 --- a/kios_bt_planning/backups/prompts/spsk_refine/prompt/refine_role.txt +++ /dev/null @@ -1,5 +0,0 @@ -[user] -You are an excellent task planner for robotic assembly tasks. Given the world state and a rough plan, you break it down and refine the plan into a well formatted and executable behavior tree. -Please do not start to work until I say "Start working." Instead, simply output the message "Waiting for next input." Understood? -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/spsk_refine/prompt/refine_state.txt b/kios_bt_planning/backups/prompts/spsk_refine/prompt/refine_state.txt deleted file mode 100644 index c6ce4d9d..00000000 --- a/kios_bt_planning/backups/prompts/spsk_refine/prompt/refine_state.txt +++ /dev/null @@ -1,27 +0,0 @@ -[user] -The initial state in the input is explained as follows: - - "objects": the objects and their properties (as lists of strings) in the world. The properties can be changed by robot actions. - - "constraints": the constraint relations that are enforced and should not be changed. They are knowledge about the world or some constraints set by the user. - - "relations": the relations in the world. They can be changed by robot actions. -following is another example: -""" -{ - "objects": [ - {"name": "leg1", "properties": []}, # leg1 is an object with no properties - {"name": "seat", "properties": []}, # seat is an object with no properties - {"name": "hand", "properties": ["is_free"]}, # hand is an object with a property "is_free" - ], - "constraints": [ - {"source": "leg1", "name": "can_screw_to", "target": "seat"}, # means leg1 can be screwed to seat - ], - "relations": [ - {"source": "leg1", "name": "is_screwed_to", "target": "seat"}, # means leg1 is screwed to seat - ], -} -""" -Only concentrate on the objects, constraints and relations that are related to the current task. - -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/spsk_refine/query/dup_query.txt b/kios_bt_planning/backups/prompts/spsk_refine/query/dup_query.txt deleted file mode 100644 index c1a7955e..00000000 --- a/kios_bt_planning/backups/prompts/spsk_refine/query/dup_query.txt +++ /dev/null @@ -1,16 +0,0 @@ -Start working. Resume from the input below. -""" -{"input": [PROBLEM]} -""" -The dictionary that you return should be formatted as json file. Follow these rules: -1. Always equip the proper tools before performing specific object manipulaion. -2. Make sure that each element of the ["action_instructions"] explains corresponding element of the ["action_sequence"]. Refer to the "ROBOT ACTION LIST" to understand the elements of ["action_sequence"]. -3, The length of the ["action_instructions"] list and the ["action_sequence"] list should be the same. -4. Never left ',' at the end of the list. Use json boolean values, "true" or "false", instead of "True" or "False". -5. Keep track of all items listed in the "objects" section of the "initial_state" field. -6. Use the "PREDICATE_LIST" for the initial states (:init) in the problem to generate ["initial_state"]. -7. All keys of the dictionary should be double-quoted. -8. Insert ``` at the beginning and the end of the dictionary to separate it from the rest of your response. Don't write any additional things. -9. Make sure that you output a consistent manipultation as a single arm robot. For example, grasping an object should not occur in successive steps. -10. When generating the behavior tree, make sure the tree has the asked structure. -Adhere to the output format I defined above. Follow the rules above. Think step by step. \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/spsk_refine/query/query.txt b/kios_bt_planning/backups/prompts/spsk_refine/query/query.txt deleted file mode 100644 index 7abaa278..00000000 --- a/kios_bt_planning/backups/prompts/spsk_refine/query/query.txt +++ /dev/null @@ -1,14 +0,0 @@ -Start working. Resume from the input below. -""" -{"input": [PROBLEM]} -""" -The dictionary that you return should be formatted as json file. Follow these rules: -1. Only refine the skeleton behavior tree in the "behavior_tree" field. Refine the tree according to rules provided before. Replace the old behavior tree with the refined behavior tree. -2. Refer to the domain knowledge, namely "ROBOT_ACTION_LIST" and "PREDICATE_LIST", to understand and refine the skeleton behavior tree. -3. Do not change other fields of the dictionary and return the new dictionary as json file. -4. Never left ',' at the end of the list. -5. Use json boolean values, "true" or "false", instead of "True" or "False". -6. All keys of the dictionary should be double-quoted. -7. Insert ``` at the beginning and the end of the dictionary to separate it from the rest of your response. Don't write any additional things. -8. When generating the behavior tree, make sure the tree has the asked structure. -Adhere to the output format I defined above. Follow the rules above. Think step by step. \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/spsk_refine/system/system.txt b/kios_bt_planning/backups/prompts/spsk_refine/system/system.txt deleted file mode 100644 index 5bc3c5a6..00000000 --- a/kios_bt_planning/backups/prompts/spsk_refine/system/system.txt +++ /dev/null @@ -1 +0,0 @@ -You are an excellent robot behavior tree expert in robotic assembly tasks. Given the knowledge you need, the world state and a rough plan, you break it down and refine the task plan to make it well formatted and ready to be executed. \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/super_skeleton/prompt/backup b/kios_bt_planning/backups/prompts/super_skeleton/prompt/backup deleted file mode 100644 index c4a0f6f6..00000000 --- a/kios_bt_planning/backups/prompts/super_skeleton/prompt/backup +++ /dev/null @@ -1,43 +0,0 @@ - -There are also some counterexamples for you to refer to: -1. One sequence should never have multiple action nodes as its children like the behavior tree below. -{ - "summary": "selector to get the apple", - "name": "Get the apple", - "type_name": "selector", - "children": [ - { - "summary": "check if the apple is in hand", - "name": "target: in_hand(apple)", - "type_name": "condition", - }, - { - "summary": "sequence to get the apple", - "name": "sequence: get apple", - "type_name": "sequence", - "children": [ - { - "summary": "check if the apple is on the table", - "name": "precondition: on(apple, table)", - "type_name": "condition", - }, - { - "summary": "go to the table", - "name": "action: go_to(table)", - "type_name": "action", - }, - { - "summary": "check if the hand is free", - "name": "precondition: is_free(hand)", - "type_name": "condition", - }, - { - "summary": "pick up the apple", - "name": "action: pick_up(apple)", - "type_name": "action", - (effect omitted) - }, - ], - }, - ], -} \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/super_skeleton/prompt/prompt_bt_skeleton.txt b/kios_bt_planning/backups/prompts/super_skeleton/prompt/prompt_bt_skeleton.txt deleted file mode 100644 index 4b713321..00000000 --- a/kios_bt_planning/backups/prompts/super_skeleton/prompt/prompt_bt_skeleton.txt +++ /dev/null @@ -1,68 +0,0 @@ -[user] -The behavior tree should be in json dictionary format and have a structure like the following example: -""" -{ - "summary": "selector to load inward_claw to left hand", - "children": [ - { - "summary": "condition node to check if inward_claw is held by left hand", - }, - { - "summary": "sequence to load inward_claw to left hand", - "children": [ - { - "summary": "condition node to check if inward_claw is equippable", - }, - { - "summary": "condition node to check if left hand is free", - }, - { - "summary": "action node to equip inward_claw to left hand", - "effects": [ - {"summary": "left hand will be not free anymore",}, - {"summary": "inward_claw will be not equippable anymore",}, - {"summary": "left_hand will hold inward_claw",}, - ], - }, - ], - }, - ], -} -""" -Follow these rules: -1. Every node are described by a summary, which is a short interpretive description of the node. It should include the type of the node and the purpose of the node. -2. The control flow nodes, "selector" and "sequence", can have "children", which is a list of children nodes. -3. The "condition" nodes that are children of "selectors" are "targets". Those that are children of "sequences" are preconditions. -4. The "action" nodes should have a key "effects" with a list of dictionaries, which describes the to-be-exerted effects of the action. The effects are shortly described under the key "summary". -5. All the subtrees always have their corresponding action in ROBOT_ACTION_LIST. They are used to represent the actions in the action sequence. The recommended structure is: - - A selector as the root node. - - (Usually one) target condition node as the first child of the selector to check if the target of the action has been fulfilled. - - A sequence node as the last child of the selector node to achieve the target. - - (Usually a number of) precondition nodes as the front children of the sequence node to check the preconditions for executing the action. - - One action node as the last child of the sequence node, whose preconditions are checked before and will immediate start the job when being ticked. - - The action will exert its defined effects after finishing the job successfully. Those effects should fulfilled all the target conditions in the selector node. - A brief example is as follows: -{ - selector: load_tool selector, - children: [ - { - target: check if inward_claw is equipped by left hand, - }, - { - sequence: load_tool, - children: [ - precondition: check if inward_claw is equippable, - precondition: check if left hand is free, - {action: equip inward_claw to left hand, effects: [left hand not free, inward_claw not equippable, left_hand holds inward_claw]}, - ], - }, - ], -} - -6. When performing a new action to fulfill a unsatisfied precondition, you can construct a subtree to replace the precondition and take that precondition as the target of the subtree. Then you construct the sequence of the subtree according to the new action you want to perform. -7. All of the control flow nodes, namely selectors and sequences, are without memory. A new behavior tree tick will always start from the root and traverse all the nodes even if they are ticked in the last tick. - -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/super_skeleton/prompt/prompt_bt_skeleton_example.txt b/kios_bt_planning/backups/prompts/super_skeleton/prompt/prompt_bt_skeleton_example.txt deleted file mode 100644 index 6f832f67..00000000 --- a/kios_bt_planning/backups/prompts/super_skeleton/prompt/prompt_bt_skeleton_example.txt +++ /dev/null @@ -1,143 +0,0 @@ -[user] -I will give you some examples of the input and the output you will generate. -Example 1: -""" -- Input: -(define (problem robot_assembly_problem-problem) - (:domain robot_assembly_problem-domain) - (:objects - parallel_box1 parallel_box2 inward_claw outward_claw no_tool - tool - gear1 gear2 gear3 shaft1 shaft2 base - part - left_hand - hand - ) - (:init (can_manipulate parallel_box1 gear1) (can_manipulate outward_claw gear2) (can_manipulate inward_claw gear3) (can_manipulate parallel_box2 shaft1) (can_manipulate no_tool shaft2) (can_screw_to leg1 seat) (can_screw_to leg2 seat) (can_insert_to back seat) (can_screw_to nut1 seat) (can_screw_to nut2 seat) (can_screw_to blub base) (can_place_to lamp blub) (can_insert_to shaft1 base) (can_insert_to shaft2 base) (can_insert_to gear3 shaft2) (can_insert_to gear2 base) (can_insert_to gear1 shaft1) (is_inserted_to shaft1 base) (is_free left_hand) (is_free parallel_box1) (is_free parallel_box2) (is_free inward_claw) (is_free outward_claw) (is_free no_tool) (is_free gear1) (is_free gear2) (is_free gear3) (is_free shaft1) (is_free shaft2) (is_free base) (is_equippable parallel_box1) (is_equippable parallel_box2) (is_equippable inward_claw) (is_equippable outward_claw) (is_equippable no_tool)) - (:goal (and (is_inserted_to gear1 shaft1))) -) -- Output: -``` -{"task_plan": { - "action_sequence": [ - "load_tool(left_hand, parallel_box1)", - "pick_up(left_hand, parallel_box1, gear1)", - "insert(left_hand, parallel_box1, gear1, shaft1)" - ], - "action_instructions": [ - "equip the parallel_box1 on the left hand", - "pick up the gear1 with the parallel_box1 in the left hand", - "insert the gear1 into the shaft1 using the parallel_box1 in the left hand" - ], - "object_name": ["left_hand", "gear1", "parallel_box1", "shaft1"], - "behavior_tree": { - "summary": "selector to insert gear1 into shaft1", - "children": [ - { - "summary": "condition node to check if gear1 is inserted to shaft1", - }, - { - "summary": "sequence to insert gear1 into shaft1", - "children": [ - { - "summary": "selector to load_tool - "children": [ - { - "summary": "condition node to check if parallel_box1 is hold", - }, - { - "summary": "sequence to load_tool", - "children": [ - { - "summary": "condition node to check if parallel_box1 is equippable", - }, - { - "summary": "condition node to check if left hand is free", - }, - { - "summary": "action node to equip parallel_box1 to left hand", - "effects": [ - { - "summary": "left_hand will be not free", - }, - { - "summary": "parallel_box1 will be not equippable", - }, - { - "summary": "left_hand will hold parallel_box1", - }, - ], - }, - ], - }, - ], - }, - { - "summary": "selector to pick_up", - "children": [ - { - "summary": "condition node to check if parallel_box1 holds gear1", - }, - { - "summary": "sequence to pick_up", - "children": [ - { - "summary": "condition node to check if parallel_box1 is free", - }, - { - "summary": "condition node to check if left hand holds parallel_box1", - }, - { - "summary": "action node to pick up gear1 using parallel_box1", - "effects": [ - { - "summary": "parallel_box1 will hold gear1", - }, - { - "summary": "parallel_box1 will be not free", - }, - ], - }, - ], - }, - ], - }, - { - "summary": "condition node to check if gear1 can be inserted to shaft1", - }, - { - "summary": "action node to insert gear1 to shaft1", - "effects": [ - { - "summary": "parallel_box1 will be not holding gear1", - }, - { - "summary": "parallel_box1 will be free", - }, - { - "summary": "gear1 will be inserted to shaft1", - }, - ], - }, - ], - }, - ], -} -}, -"environment_before": {"objects": [ - {"name": "gear1", "properties": []}, - {"name": "shaft1", "properties": []}, - {"name": "parallel_box1", "properties": ["is_equippable", "is_free"]}, - {"name": "left_hand", "properties": ["is_free"]}, - ], - "constraints": [ - {"source": "gear1", "name": "can_insert_to", "target": "shaft1"}, - {"source": "parallel_box1", "name": "can_manipulate", "target": "gear1"}, - ], - "relations": [], -}, -"instruction_summary": "insert gear1 into shaft1", -"question":""} -``` -""" -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/super_skeleton/prompt/prompt_bt_validation.txt b/kios_bt_planning/backups/prompts/super_skeleton/prompt/prompt_bt_validation.txt deleted file mode 100644 index e69de29b..00000000 diff --git a/kios_bt_planning/backups/prompts/super_skeleton/prompt/prompt_domain.txt b/kios_bt_planning/backups/prompts/super_skeleton/prompt/prompt_domain.txt deleted file mode 100644 index bf3c7859..00000000 --- a/kios_bt_planning/backups/prompts/super_skeleton/prompt/prompt_domain.txt +++ /dev/null @@ -1,89 +0,0 @@ -[user] -Following is the domain knowledge you need in pddl domain format, which includes: -- the object types in the world, -- the predicates for describing the states of the world, -- the actions that the robot can apply, with their precondtions and effects defined. -""" -(define (domain robot_assembly_problem-domain) - (:requirements :strips :typing) - (:types - thing - object - tool part hand - thing - ) - (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part)) - (:action pick_up - :parameters ( ?hand - hand ?tool - tool ?part - part) - :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part)) - :effect (and (hold ?tool ?part) (not (is_free ?tool)))) - (:action put_down - :parameters ( ?hand - hand ?tool - tool ?part - part) - :precondition (and (hold ?tool ?part) (hold ?hand ?tool)) - :effect (and (not (hold ?hand ?part)) (is_free ?tool))) - (:action place - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2))) - (:action detach - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2)))) - (:action insert - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2))) - (:action pull - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2)))) - (:action screw - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2))) - (:action unscrew - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2)))) - (:action load_tool - :parameters ( ?hand - hand ?tool - tool) - :precondition (and (is_equippable ?tool) (is_free ?hand)) - :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool))) - (:action unload_tool - :parameters ( ?hand - hand ?tool - tool) - :precondition (and (hold ?hand ?tool) (is_free ?tool)) - :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool)))) -) -""" -The precidates are explained as follows: -""" -"PREDICATE_LIST" -- is_free( or ): or is free and can hold something. (default: true) -- is_equippable(): is ready to be eqquipped by a robot hand. (default: true) -- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false) -- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false) -- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false) -- can_place_to(, ): can be placed into (placement assembly constraint) (default: false) -- hold(, ): is holding (default: false) -- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false) -- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false) -- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false) -""" - -The actions are explained as follows: -""" -"ROBOT_ACTION_LIST" -- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something. -- put_down(, , ): use to put down . As preconditions, should be holding and should be holding . -- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding . -- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into . -- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding . -- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into . -- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding . -- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into . -- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free. -- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free. -""" -the items in <> indicate the type of the instances needed for the predicates and actions. -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. diff --git a/kios_bt_planning/backups/prompts/super_skeleton/prompt/prompt_environment.txt b/kios_bt_planning/backups/prompts/super_skeleton/prompt/prompt_environment.txt deleted file mode 100644 index 656fd60c..00000000 --- a/kios_bt_planning/backups/prompts/super_skeleton/prompt/prompt_environment.txt +++ /dev/null @@ -1,31 +0,0 @@ -[user] -Information about world states should be in json dictionary format. Example: -""" -{ - "objects": [ - {"name": "leg1", "properties": []}, # leg1 is an object with no properties - {"name": "seat", "properties": []}, # seat is an object with no properties - {"name": "cabinet", "properties": ["open"]}, # cabinet is an object with a property "open" - {"name": "parallel_box1", "properties": []}, # parallel_box1 is an object with no properties - {"name": "table", "properties": []}, # table is an object with no properties - ], - "constraints": [ - {"source": "leg1", "name": "can_screw_to", "target": "seat"}, # means leg1 can be screwed to seat - {"source": "parallel_box1", "name": "can_manipulate", "target": "leg1"}, # means parallel_box1 can manipulate leg1 - {"source": "cabinet", "name": "on_something", "target": "table"}, # means cabinet is on table - ], - "relations": [ - {"source": "leg1", "name": "is_screwed_to", "target": "seat"}, # means leg1 is screwed to seat - ], -} -""" -It contains three lists: - - "objects": the objects and their properties (as lists of strings) in the environment. The properties can be changed by robot actions. - - "constraints": the constraint relations that are enforced and should not be changed. They are knowledge about the world or some constraints set by the user. - - "relations": the relations in the environment. They can be changed by robot actions. -The properties, constraints and relations should always align with the [PREDICATE_LIST] in domain knowledge. -You should only concentrate on the objects, constraints and relations that are related to (or can be effected by) the current task. -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/super_skeleton/prompt/prompt_output_format.txt b/kios_bt_planning/backups/prompts/super_skeleton/prompt/prompt_output_format.txt deleted file mode 100644 index 83c5c6f7..00000000 --- a/kios_bt_planning/backups/prompts/super_skeleton/prompt/prompt_output_format.txt +++ /dev/null @@ -1,20 +0,0 @@ -[user] -You generate an action sequence to solve the task and contruct a behavior tree based on it in json format. -The output dictionary has five keys. -""" -- dictionary["task_plan"]: A dictionary containing information about the task plan. -- dictionary["environment_before"]: The environment state before executing the plan. This should be consistent with the initial states in the provided problem. -- dictionary["instruction_summary"]: A brief summary of the task plan. -- dictionary["question"]: If you don't have enough knowledge for solving the problem, you can ask questions about the knowledge you need. Leave this key empty if you can solve the task. -""" -Three keys exist in dictionary["task_plan"]. -""" -- dictionary["task_plan"]["action_sequence"]: Contains a list of robot actions. Only the actions defined in the "ROBOT_ACTION LIST" will be used. -- dictionary["task_plan"]["action_instructions"]: contains a list of instructions corresponding to dictionary["task_plan"]["action_sequence"] to explain the actions. -- dictionary["task_plan"]["manipulated_objects"]: The list of the names of the manipulated object. It only includes the objects that are related to the current task. -- dictionary["task_plan"]["behavior_tree"]: contains a dictionary of the behavior tree constructed according to the dictionary ["task_plan"]["action_sequence"] -""" -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/super_skeleton/prompt/prompt_problem.txt b/kios_bt_planning/backups/prompts/super_skeleton/prompt/prompt_problem.txt deleted file mode 100644 index 22734b26..00000000 --- a/kios_bt_planning/backups/prompts/super_skeleton/prompt/prompt_problem.txt +++ /dev/null @@ -1,23 +0,0 @@ -[user] -Information about the problem is given as a pddl problem. -It includes the following information: -- Objects in the world: all instances of the types defined in the domain knowledge. For example, "parallel_box1" is an instance of the type "tool", while "gear1" is an instance of the type "part". -- Initial states: the world states to start with, described with [PREDICATE_LIST] from domain knowledge. For example, "is_free hand" means the object "hand" is free and holds nothing. -- Goal states: the world states to achieve, described with [PREDICATE_LIST] from domain knowledge. -Example: -""" -(define (problem robot_assembly_problem-problem) - (:domain robot_assembly_problem-domain) - (:objects - parallel_box1 parallel_box2 inward_claw outward_claw no_tool - tool - gear1 gear2 gear3 shaft1 shaft2 base - part - left_hand - hand - ) - (:init (can_manipulate parallel_box1 gear1) (can_manipulate outward_claw gear2) (can_manipulate inward_claw gear3) (can_manipulate parallel_box2 shaft1) (can_manipulate no_tool shaft2) (can_screw_to leg1 seat) (can_screw_to leg2 seat) (can_insert_to back seat) (can_screw_to nut1 seat) (can_screw_to nut2 seat) (can_screw_to blub base) (can_place_to lamp blub) (can_insert_to shaft1 base) (can_insert_to shaft2 base) (can_insert_to gear3 shaft2) (can_insert_to gear2 base) (can_insert_to gear1 shaft1) (is_free left_hand) (is_free parallel_box1) (is_free parallel_box2) (is_free inward_claw) (is_free outward_claw) (is_free no_tool) (is_equippable parallel_box1) (is_equippable parallel_box2) (is_equippable inward_claw) (is_equippable outward_claw) (is_equippable no_tool)) - (:goal (and (is_inserted_to shaft1 base))) -) -""" -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/super_skeleton/prompt/prompt_role.txt b/kios_bt_planning/backups/prompts/super_skeleton/prompt/prompt_role.txt deleted file mode 100644 index 6ce736a0..00000000 --- a/kios_bt_planning/backups/prompts/super_skeleton/prompt/prompt_role.txt +++ /dev/null @@ -1,5 +0,0 @@ -[user] -You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you break it down and construct robot task plans represented by behaiovr trees. -Please do not begin working until I say "Start working." Instead, simply output the message "Waiting for next input." Understood? -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/super_skeleton/query/query.txt b/kios_bt_planning/backups/prompts/super_skeleton/query/query.txt deleted file mode 100644 index 674cfab0..00000000 --- a/kios_bt_planning/backups/prompts/super_skeleton/query/query.txt +++ /dev/null @@ -1,16 +0,0 @@ -Start working. Resume from the problem below. -""" -{"problem": [PROBLEM]} -""" -The dictionary that you return should be formatted as json file. Follow these rules: -1. Always equip the proper tools before performing specific object manipulaion. -2. Make sure that each element of the ["action_instructions"] explains corresponding element of the ["action_sequence"]. Refer to the "ROBOT ACTION LIST" to understand the elements of ["action_sequence"]. -3, The length of the ["action_instructions"] list and the ["action_sequence"] list should be the same. -4. Never left ',' at the end of the list. Use json boolean values, "true" or "false", instead of "True" or "False". -5. Keep track of all items listed in the "objects" section of the "environment_before" field. Please make sure you fill out the "objects" section for all listed items. -6. Use the "PREDICATE_LIST" for the initial states (:init) in the problem to generate ["environment_before"]. -7. All keys of the dictionary should be double-quoted. -8. Insert ``` at the beginning and the end of the dictionary to separate it from the rest of your response. Don't write any additional things. -9. Make sure that you output a consistent manipultation as a single arm robot. For example, grasping an object should not occur in successive steps. -10. When generating the behavior tree, make sure the tree has the asked structure. -Adhere to the output format I defined above. Follow the rules above. Think step by step. \ No newline at end of file diff --git a/kios_bt_planning/backups/prompts/super_skeleton/system/system.txt b/kios_bt_planning/backups/prompts/super_skeleton/system/system.txt deleted file mode 100644 index 26f2afe2..00000000 --- a/kios_bt_planning/backups/prompts/super_skeleton/system/system.txt +++ /dev/null @@ -1 +0,0 @@ -You are an excellent interpreter of instructions for robotic assembly tasks. Given the knowledge you need, the world states and some instructions, you break it down and construct robot task plans represented by behaiovr trees based on the action sequences. \ No newline at end of file diff --git a/kios_bt_planning/backups/pybt_io.py b/kios_bt_planning/backups/pybt_io.py deleted file mode 100644 index 0a30db8e..00000000 --- a/kios_bt_planning/backups/pybt_io.py +++ /dev/null @@ -1,173 +0,0 @@ -""" -BT in/out in various formats - -""" -# ! Discarded, BBREMOVE - -import xml.etree.ElementTree as ET - -import json -import os - - -class ExtendedAction: - name: str - variables: dict - mios_parameters: list - preconditions: dict - effects: dict - - -class ExtendedCondition: - name: str - variables: dict - - -class BehaviorTreeTemplates: - """ - behavior tree templates manager - """ - - def __init__(self): - pass - self.conditions = [] - self.actions = [] - folder = os.getcwd() - self.directory = os.path.join(folder, "src/kios/kios_bt_planning/bt_settings") - self.actions_file_path = os.path.join(self.directory, "actions.json") - self.conditions_file_path = os.path.join(self.directory, "conditions.json") - - def update_directory(self, directory): - self.directory = directory - self.actions_file_path = os.path.join(self.directory, "actions.json") - self.conditions_file_path = os.path.join(self.directory, "conditions.json") - - def initialize(self): - """ - default initialization - """ - self.read_actions() - self.read_conditions() - - def read_actions(self, file_path=None): - if file_path is None: - file_path = self.actions_file_path - with open(file_path) as json_file: - self.actions = json.load(json_file) - - def write_actions(self, file_path=None): - if file_path is None: - file_path = self.actions_file_path - with open(file_path, "w") as json_file: - json.dump(self.actions, json_file, indent=4) - - def read_conditions(self, file_path=None): - if file_path is None: - file_path = self.conditions_file_path - with open(file_path) as json_file: - self.conditions = json.load(json_file) - - def write_conditions(self, file_path=None): - if file_path is None: - file_path = self.conditions_file_path - with open(file_path, "w") as json_file: - json.dump(self.conditions, json_file, indent=4) - - def get_action(self, action_name) -> dict: - if action_name in self.actions: - return self.actions[action_name] - else: - return None - - def get_condition(self, condition_name): - if condition_name in self.conditions: - return self.conditions[condition_name] - else: - return None - - -# XML test part -# ! unfinished - - -class BehaviorTreeFactory: - def __init__(self): - self.root = ET.Element("root", BTCPP_format="4") - self.main_tree = ET.SubElement(self.root, "BehaviorTree", ID="MainTree") - self.current_parent = ET.SubElement( - self.main_tree, "ReactiveSequence", name="root_sequence" - ) - self.subtrees = {} - - def add_node(self, node_type, name, **attributes): - if node_type == "SubTree": - subtree_id = attributes.get("ID") - if subtree_id in self.subtrees: - subtree_copy = self._deepcopy(self.subtrees[subtree_id]) - self.current_parent.append(subtree_copy) - else: - print(f"SubTree with ID {subtree_id} not found!") - return - else: - node = ET.SubElement( - self.current_parent, node_type, name=name, **attributes - ) - if node_type in ["ReactiveSequence", "Fallback", "Sequence"]: - self.current_parent = node - - def define_subtree(self, subtree_id, node_type, name): - subtree = ET.Element(node_type, name=name) - self.subtrees[subtree_id] = subtree - return subtree - - def _deepcopy(self, element): - """Creates a deep copy of an XML element and its children.""" - copy_elem = ET.Element(element.tag, attrib=element.attrib) - for child in element: - copy_elem.append(self._deepcopy(child)) - return copy_elem - - def to_string(self): - return ET.tostring(self.root, encoding="utf-8", method="xml").decode("utf-8") - - def to_pretty_string(self): - # For pretty printing - import xml.dom.minidom - - xml_str = ET.tostring(self.root, encoding="utf-8", method="xml") - dom = xml.dom.minidom.parseString(xml_str) - return dom.toprettyxml() - - -def test_xml(): - # Usage - tree_factory = BehaviorTreeFactory() - - tree_factory.add_node("CheckFinished", "check_finished") - tree_factory.add_node("SubTree", "", ID="push_peg_z") - - # Peg align hole - align_subtree = tree_factory.add_node("ReactiveSequence", "peg_align_hole") - tree_factory.add_node("CheckAlign", "check_align") - tree_factory.add_node("Align", "align") - - # Peg fit hole - fit_subtree = tree_factory.add_node("ReactiveSequence", "peg_fit_hole") - tree_factory.add_node("CheckFit", "check_fit") - tree_factory.add_node("Fit", "fit_hole") - - # Peg reach hole - reach_subtree = tree_factory.add_node("ReactiveSequence", "peg_reach_hole") - tree_factory.add_node("CheckReach", "check_reach") - tree_factory.add_node("ReachHole", "reach_hole") - - tree_factory.add_node("Approach", "approach") - - # Defining the push_peg_z subtree - push_peg_subtree = tree_factory.define_subtree( - "push_peg_z", "ReactiveSequence", "push_peg" - ) - ET.SubElement(push_peg_subtree, "CheckPush", name="check_push") - ET.SubElement(push_peg_subtree, "Push", name="push_peg") - - print(tree_factory.to_pretty_string()) diff --git a/kios_bt_planning/backups/pytree_mod_test.py b/kios_bt_planning/backups/pytree_mod_test.py deleted file mode 100644 index fd49b3e6..00000000 --- a/kios_bt_planning/backups/pytree_mod_test.py +++ /dev/null @@ -1,151 +0,0 @@ -import py_trees - -from kios_bt.behavior_nodes import ActionNode, ConditionNode - -from kios_bt.data_types import ( - Action, - Condition, - ObjectProperty, -) - -from kios_utils.pybt_test import ( - generate_bt_stewardship, - tick_once_test, - render_dot_tree, -) - -from kios_bt.bt_factory import BehaviorTreeFactory - -action = Action( - summary="pick up apple", - identifier=0, - name="pick_up", - effects=[ - ObjectProperty( - object_name="apple", - property_name="on_the_ground", - property_value=None, - status=False, - ), - ObjectProperty( - object_name="apple", - property_name="in", - property_value="hand", - status=True, - ), - ObjectProperty( - object_name="hand", - property_name="free", - property_value=None, - status=False, - ), - ], -) - -precondition1 = Condition( - summary="check if the apple is on the ground", - identifier=1, - name="apple on the ground", - conditions=[ - ObjectProperty( - object_name="apple", - property_name="on_the_ground", - property_value=None, - status=True, - ), - ], -) - -precondition2 = Condition( - summary="check if the hand is free", - identifier=2, - name="hand free", - conditions=[ - ObjectProperty( - object_name="hand", - property_name="free", - property_value=None, - status=True, - ), - ], -) - -effect = Condition( - summary="check if the apple is in hand", - identifier=3, - name="apple in hand", - conditions=[ - ObjectProperty( - object_name="apple", - property_name="in", - property_value="hand", - status=True, - ), - ], -) - -another_action = Action( - summary="put down apple", - identifier=4, - name="put_down", - effects=[ - ObjectProperty( - object_name="apple", - property_name="on_the_ground", - property_value=None, - status=True, - ), - ObjectProperty( - object_name="apple", - property_name="in", - property_value="hand", - status=False, - ), - ObjectProperty( - object_name="hand", - property_name="free", - property_value=None, - status=True, - ), - ], -) - -bt_factory = BehaviorTreeFactory() - -precondition1_node = bt_factory.generate_condition_node(precondition1) -precondition2_node = bt_factory.generate_condition_node(precondition2) -effect_node = bt_factory.generate_condition_node(effect) -action_node = bt_factory.generate_fake_action_node(action) - -another_action_node = bt_factory.generate_fake_action_node(another_action) - -another_action_node1 = bt_factory.generate_fake_action_node(another_action) - -subtree = bt_factory.generate_subtree( - preconditions=[precondition1_node, precondition2_node], - action=action_node, - effects=[effect_node], -) - -behaviortree = generate_bt_stewardship(subtree) - -render_dot_tree(behaviortree) - -tick_once_test(behaviortree) - -# try the mod methods in pytree - -behaviortree.replace_subtree(action_node.id, another_action_node) - -tick_once_test(behaviortree) - -behaviortree.insert_subtree( - another_action_node1, - another_action_node.parent.id, - index=1, -) - -behaviortree.setup(timeout=15) - - -tick_once_test(behaviortree) diff --git a/kios_bt_planning/backups/robot_actuator.py b/kios_bt_planning/backups/robot_actuator.py deleted file mode 100644 index 3fb4b828..00000000 --- a/kios_bt_planning/backups/robot_actuator.py +++ /dev/null @@ -1,324 +0,0 @@ -# from kios_utils.task import * -# import time -# import json - -# from kios_robot.data_types import MiosInterfaceResponse, MiosTaskResult - -# # ! discarded - - -# class RobotActuator: -# robot_address: str = None -# robot_port: int = None - -# def __init__(self, robot_address: str, robot_port: int): -# if robot_address is not None: -# self.robot_address = robot_address -# else: -# raise Exception("robot_address is not set") - -# if robot_port is not None: -# self.robot_port = robot_port -# else: -# raise Exception("robot_port is not set") - -# # move to pose cartesian -# def cartesian_move(self, object: str): -# context = { -# "skill": { -# "p0": { -# "dX_d": [0.1, 0.5], -# "ddX_d": [0.5, 1], -# "K_x": [1500, 1500, 1500, 150, 150, 150], -# }, -# "objects": {"GoalPose": object}, -# }, -# "control": {"control_mode": 0}, -# } -# t = Task(self.robot_address) -# t.add_skill("move", "TaxMove", context) -# t.start() -# result = t.wait() -# mios_response = MiosInterfaceResponse.from_json(result["result"]) -# # print("Result: " + str(result)) -# print("\033[92mMios replied: ") -# print(mios_response) -# print("\033[0m") # Reset color to default - -# # ! NOT IN USE YET -# def joint_move(self, joint_location: str): -# """ -# * available. debug: task context error, "skill", "control" and "user" are on the same level. -# move to a joint position. -# Args: -# joint_location (str): the stored Taskframe_HomogeneousT_EE_goal in mongo DB -# """ -# context = { -# "skill": { -# "speed": 0.5, -# "acc": 1, -# "q_g": [0, 0, 0, 0, 0, 0, 0], -# "q_g_offset": [0, 0, 0, 0, 0, 0, 0], -# "objects": {"goal_pose": joint_location}, -# }, -# "control": {"control_mode": 3}, -# "user": {"env_X": [0.005, 0.005, 0.005, 0.0175, 0.0175, 0.0175]}, -# } -# t = Task(self.robot_address) -# t.add_skill("move", "MoveToPoseJoint", context) -# t.start() -# result = t.wait() -# mios_response = MiosInterfaceResponse.from_json(result["result"]) -# # print("Result: " + str(result)) -# print("\033[92mMios replied: ") -# print(mios_response) -# print("\033[0m") # Reset color to default - -# def gripper_grasp( -# self, width=0.01, speed=0.05, force=50, epsilon_inner=0.05, epsilon_outer=0.05 -# ): -# payload = { -# "width": 0.01, -# "speed": 0.01, -# "force": force, -# "epsilon_inner": 0.05, -# "epsilon_outer": 0.05, -# } -# return call_method(self.robot_address, self.robot_port, "grasp", payload) - -# def gripper_move(self, width: float, speed: float = 0.05): -# payload = { -# "width": width, -# "speed": speed, -# } -# return call_method(self.robot_address, self.robot_port, "move_gripper", payload) - -# # # ? how should we make use of the object in the system? -# # def gripper_grasp_object(self, force=60): -# # payload = { -# # "object": "NoneObject", -# # "width": 0.0, -# # "speed": 0.05, -# # "force": force, -# # "check_widtch": False, -# # } -# # return call_method(self.robot_address, self.robot_port, "grasp_object", payload) - -# def gripper_home(self): -# payload = {} -# return call_method(self.robot_address, self.robot_port, "home_gripper", payload) - -# # preserved for future use -# def equip_tool(self): -# # move finger to the right position -# self.gripper_move(0.042) - -# # preserved for future use -# def lift_tool(self): -# # move finger to the right position -# self.gripper_move(0.08) - -# # # ! don't use this. -# # def moveJ(self, q_g): -# # """ -# # call mios for moving the lefthand to desired joint position - -# # Paramter -# # -------- -# # q_g: list, len(7) -# # """ -# # parameters = { -# # "parameters": { -# # "pose": "NoneObject", -# # "q_g": q_g, -# # "speed": 0.5, -# # "acc": 0.7, -# # } -# # } -# # return start_task_and_wait( -# # self.robot_address, "MoveToJointPose", parameters, False -# # ) - -# # * modify this later -# def insertion(self): -# call_method( -# self.robot_address, -# self.robot_port, -# "set_grasped_object", -# {"object": "ring"}, -# ) -# content = { -# "skill": { -# "objects": { -# "Container": "housing", -# "Approach": "app1", -# "Insertable": "ring", -# }, -# "time_max": 17, -# "p0": { -# "dX_d": [0.1, 1], -# "ddX_d": [0.5, 4], -# "DeltaX": [0, 0, 0, 0, 0, 0], -# "K_x": [1500, 1500, 1500, 600, 600, 600], -# }, -# "p1": { -# "dX_d": [0.03, 0.1], -# "ddX_d": [0.5, 0.1], -# "K_x": [500, 500, 500, 600, 600, 600], -# }, -# "p2": { -# # "search_a": [10, 10, 0, 2, 2, 0], -# # "search_f": [1, 1, 0, 1.2, 1.2, 0], -# "search_a": [5, 5, 0, 2, 2, 0], -# "search_f": [1, 1, 0, 1.2, 1.2, 0], -# "search_phi": [ -# 0, -# 3.14159265358979323846 / 2, -# 0, -# 3.14159265358979323846 / 2, -# 0, -# 0, -# ], -# "K_x": [500, 500, 500, 800, 800, 800], -# "f_push": [0, 0, 7, 0, 0, 0], -# # "dX_d": [0.1, 0.5], -# # "ddX_d": [0.5, 1], -# "dX_d": [0.08, 0.5], -# "ddX_d": [0.3, 1], -# }, -# "p3": { -# "dX_d": [0.1, 0.5], -# "ddX_d": [0.5, 1], -# "f_push": 7, -# "K_x": [500, 500, 0, 800, 800, 800], -# }, -# }, -# "control": {"control_mode": 0}, -# "user": { -# "env_X": [0.01, 0.01, 0.002, 0.05, 0.05, 0.05], -# "env_dX": [0.001, 0.001, 0.001, 0.005, 0.005, 0.005], -# "F_ext_contact": [3.0, 2.0], -# }, -# } -# t = Task(self.robot_address) -# t.add_skill("insertion", "TaxInsertion", content) -# t.start() -# time.sleep(0.5) -# result = t.wait() -# print("Result: " + str(result)) - -# def load_tool(self, tool_name: str): -# if tool_name is None: -# raise Exception("tool_name is not set!") - -# # you should also check if this exists in the scene - -# payload = { -# "skill": { -# "objects": { -# "Tool": tool_name, -# }, -# "time_max": 30, -# "MoveAbove": { -# "dX_d": [0.2, 0.2], -# "ddX_d": [0.2, 0.2], -# "DeltaX": [0, 0, 0, 0, 0, 0], -# "K_x": [1500, 1500, 1500, 600, 600, 600], -# }, -# "MoveIn": { -# "dX_d": [0.4, 0.4], -# "ddX_d": [0.1, 0.1], -# "DeltaX": [0, 0, 0, 0, 0, 0], -# "K_x": [1500, 1500, 1500, 600, 600, 600], -# }, -# "GripperMove": { -# "width": 0.043, -# "speed": 0.1, -# "force": 70, -# "K_x": [1500, 1500, 1500, 100, 100, 100], -# "eps_in": 0.05, -# "eps_out": 0.05, -# }, -# "Retreat": { -# "dX_d": [0.4, 0.4], -# "ddX_d": [0.1, 0.1], -# "DeltaX": [0, 0, 0, 0, 0, 0], -# "K_x": [1500, 1500, 1500, 600, 600, 600], -# }, -# }, -# "control": {"control_mode": 0}, -# "user": { -# "env_X": [0.01, 0.01, 0.002, 0.05, 0.05, 0.05], -# "env_dX": [0.001, 0.001, 0.001, 0.005, 0.005, 0.005], -# "F_ext_contact": [3.0, 2.0], -# }, -# } - -# t = Task(self.robot_address) -# t.add_skill("load_tool", "KiosLoadTool", payload) -# t.start() -# time.sleep(0.5) -# result = t.wait() -# mios_response = MiosInterfaceResponse.from_json(result["result"]) -# # print("Result: " + str(result)) -# print("\033[92mMios replied: ") -# print(mios_response) -# print("\033[0m") # Reset color to default - -# def unload_tool(self, tool_name: str): -# if tool_name is None: -# raise Exception("tool_name is not set!") - -# # you should also check if this exists in the scene - -# payload = { -# "skill": { -# "objects": { -# "Tool": tool_name, -# }, -# "time_max": 30, -# "MoveAbove": { -# "dX_d": [0.2, 0.2], -# "ddX_d": [0.2, 0.2], -# "DeltaX": [0, 0, 0, 0, 0, 0], -# "K_x": [1500, 1500, 1500, 600, 600, 600], -# }, -# "MoveIn": { -# "dX_d": [0.4, 0.4], -# "ddX_d": [0.1, 0.1], -# "DeltaX": [0, 0, 0, 0, 0, 0], -# "K_x": [1500, 1500, 1500, 600, 600, 600], -# }, -# "GripperMove": { -# "width": 0.08, -# "speed": 0.1, -# "force": 70, -# "K_x": [1500, 1500, 1500, 100, 100, 100], -# "eps_in": 0.05, -# "eps_out": 0.05, -# }, -# "Retreat": { -# "dX_d": [0.4, 0.4], -# "ddX_d": [0.1, 0.1], -# "DeltaX": [0, 0, 0, 0, 0, 0], -# "K_x": [1500, 1500, 1500, 600, 600, 600], -# }, -# }, -# "control": {"control_mode": 0}, -# "user": { -# "env_X": [0.01, 0.01, 0.002, 0.05, 0.05, 0.05], -# "env_dX": [0.001, 0.001, 0.001, 0.005, 0.005, 0.005], -# "F_ext_contact": [3.0, 2.0], -# }, -# } - -# t = Task(self.robot_address) -# t.add_skill("unload_tool", "KiosLoadTool", payload) -# t.start() -# time.sleep(0.5) -# result = t.wait() -# mios_response = MiosInterfaceResponse.from_json(result["result"]) -# # print("Result: " + str(result)) -# print("\033[92mMios replied: ") -# print(mios_response) -# print("\033[0m") # Reset color to default diff --git a/kios_bt_planning/backups/robot_skill_engine.py b/kios_bt_planning/backups/robot_skill_engine.py deleted file mode 100644 index f8bab8cb..00000000 --- a/kios_bt_planning/backups/robot_skill_engine.py +++ /dev/null @@ -1,51 +0,0 @@ -# from kios_robot.robot_actuator import RobotActuator -# from kios_robot.robot_proprioceptor import RobotProprioceptor -# import numpy as np - -# # ! DISCARDED. -# ! BBREMOVE - -# class RobotSkillEngine: -# robot_actuator: RobotActuator = None -# robot_proprioceptor: RobotProprioceptor = None - -# def __init__( -# self, -# robot_actuator: RobotActuator, -# robot_proprioceptor: RobotProprioceptor, -# ): -# if robot_actuator is not None: -# self.robot_actuator = robot_actuator -# else: -# raise Exception("robot_actuator is not set") - -# if robot_proprioceptor is not None: -# self.robot_proprioceptor = robot_proprioceptor -# else: -# raise Exception("robot_proprioceptor is not set") - -# def create_guidance_pose(self, object_name: str, DeltaHT: np.ndarray): -# # get the object pose -# pass - -# def load_tool(self, tool_name: str): -# print("todo: check the tool in the scene.") -# self.robot_actuator.load_tool(tool_name=tool_name) -# print("todo: change robot status TCP") - -# def unload_tool(self, tool_name: str): -# print("todo: check the tool in the scene.") -# self.robot_actuator.unload_tool(tool_name=tool_name) -# print("todo: change robot status TCP") - -# # def cartesian_move() - -# def pick(self, object_name: str): -# # move onto the object -# self.robot_actuator.cartesian_move() -# # move into the object -# self.robot_actuator.cartesian_move() -# # grasp the object -# self.robot_actuator.close_gripper() -# # retreat back highly -# self.robot_actuator.cartesian_move() diff --git a/kios_bt_planning/backups/robot_status.py b/kios_bt_planning/backups/robot_status.py deleted file mode 100644 index ec3d2e23..00000000 --- a/kios_bt_planning/backups/robot_status.py +++ /dev/null @@ -1,22 +0,0 @@ -# import numpy as np -# from typing import List, Optional, Dict, Any - -# """ -# robot status class, tailored for mios interface. -# designed for indicating some status of the robot for the adjustments in skills. -# for example, whether the robot is holding a tool or not can affects the TCP of the EE. -# """ - -# # ! This design is a bit weird. -# # ! Use mios memory or mongodb instead. - - -# class RobotStatus: -# O_T_EE: np.ndarray -# EE_T_TCP: np.ndarray = np.eye( -# 4 -# ) # * default value is identity matrix, which means no tool is attached. -# q: List[float] = [0, 0, 0, 0, 0, 0, 0] - -# def __init__(self): -# pass diff --git a/kios_bt_planning/backups/subtree_test.py b/kios_bt_planning/backups/subtree_test.py deleted file mode 100644 index ada6ef17..00000000 --- a/kios_bt_planning/backups/subtree_test.py +++ /dev/null @@ -1,27 +0,0 @@ -# from kios_bt.bt_factory import SubtreeFactory -# from kios_bt.data_types import * -# from kios_bt.pybt_io import BehaviorTreeTemplates -# from kios_utils.pybt_test import ( -# generate_bt_stewardship, -# tick_once_test, -# render_dot_tree, -# ) - -# move = ActionInstance( -# tag="move to bedroom", -# name="move", -# variables={"l_from": "kitchen", "l_to": "bedroom"}, -# ) - -# bt_templates = BehaviorTreeTemplates() -# bt_templates.initialize() - -# subtree_factory = SubtreeFactory(bt_templates) - -# test_tree = subtree_factory.generate_standard_subtree(move) - -# bt = generate_bt_stewardship(test_tree) - -# render_dot_tree(bt) - -# tick_once_test(bt) diff --git a/kios_bt_planning/backups/xml_test.py b/kios_bt_planning/backups/xml_test.py deleted file mode 100644 index ba16035d..00000000 --- a/kios_bt_planning/backups/xml_test.py +++ /dev/null @@ -1,87 +0,0 @@ -""" -BT in/out in XML format -""" - -import xml.etree.ElementTree as ET - - -class BehaviorTreeFactory: - def __init__(self): - self.root = ET.Element("root", BTCPP_format="4") - self.main_tree = ET.SubElement(self.root, "BehaviorTree", ID="MainTree") - self.current_parent = ET.SubElement( - self.main_tree, "ReactiveSequence", name="root_sequence" - ) - self.subtrees = {} - - def add_node(self, node_type, name, **attributes): - if node_type == "SubTree": - subtree_id = attributes.get("ID") - if subtree_id in self.subtrees: - subtree_copy = self._deepcopy(self.subtrees[subtree_id]) - self.current_parent.append(subtree_copy) - else: - print(f"SubTree with ID {subtree_id} not found!") - return - else: - node = ET.SubElement( - self.current_parent, node_type, name=name, **attributes - ) - if node_type in ["ReactiveSequence", "Fallback", "Sequence"]: - self.current_parent = node - - def define_subtree(self, subtree_id, node_type, name): - subtree = ET.Element(node_type, name=name) - self.subtrees[subtree_id] = subtree - return subtree - - def _deepcopy(self, element): - """Creates a deep copy of an XML element and its children.""" - copy_elem = ET.Element(element.tag, attrib=element.attrib) - for child in element: - copy_elem.append(self._deepcopy(child)) - return copy_elem - - def to_string(self): - return ET.tostring(self.root, encoding="utf-8", method="xml").decode("utf-8") - - def to_pretty_string(self): - # For pretty printing - import xml.dom.minidom - - xml_str = ET.tostring(self.root, encoding="utf-8", method="xml") - dom = xml.dom.minidom.parseString(xml_str) - return dom.toprettyxml() - - -# Usage -tree_factory = BehaviorTreeFactory() - -tree_factory.add_node("CheckFinished", "check_finished") -tree_factory.add_node("SubTree", "", ID="push_peg_z") - -# Peg align hole -align_subtree = tree_factory.add_node("ReactiveSequence", "peg_align_hole") -tree_factory.add_node("CheckAlign", "check_align") -tree_factory.add_node("Align", "align") - -# Peg fit hole -fit_subtree = tree_factory.add_node("ReactiveSequence", "peg_fit_hole") -tree_factory.add_node("CheckFit", "check_fit") -tree_factory.add_node("Fit", "fit_hole") - -# Peg reach hole -reach_subtree = tree_factory.add_node("ReactiveSequence", "peg_reach_hole") -tree_factory.add_node("CheckReach", "check_reach") -tree_factory.add_node("ReachHole", "reach_hole") - -tree_factory.add_node("Approach", "approach") - -# Defining the push_peg_z subtree -push_peg_subtree = tree_factory.define_subtree( - "push_peg_z", "ReactiveSequence", "push_peg" -) -ET.SubElement(push_peg_subtree, "CheckPush", name="check_push") -ET.SubElement(push_peg_subtree, "Push", name="push_peg") - -print(tree_factory.to_pretty_string()) diff --git a/kios_bt_planning/kios_agent/RAG_chain_tutorial.py b/kios_bt_planning/kios_agent/RAG_chain_tutorial.py deleted file mode 100644 index f9099bc9..00000000 --- a/kios_bt_planning/kios_agent/RAG_chain_tutorial.py +++ /dev/null @@ -1,123 +0,0 @@ -import os -from operator import itemgetter -from typing import List, Tuple - -""" -RAG tutorial from the langchain community -""" - -from langchain.schema import AIMessage, HumanMessage, format_document -from langchain_community.chat_models import ChatOpenAI -from langchain_community.embeddings import OpenAIEmbeddings -from langchain_community.vectorstores import Pinecone -from langchain_core.output_parsers import StrOutputParser -from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder -from langchain_core.prompts.prompt import PromptTemplate -from langchain_core.pydantic_v1 import BaseModel, Field -from langchain_core.runnables import ( - RunnableBranch, - RunnableLambda, - RunnableParallel, - RunnablePassthrough, -) - -if os.environ.get("PINECONE_API_KEY", None) is None: - raise Exception("Missing `PINECONE_API_KEY` environment variable.") - -if os.environ.get("PINECONE_ENVIRONMENT", None) is None: - raise Exception("Missing `PINECONE_ENVIRONMENT` environment variable.") - -PINECONE_INDEX_NAME = os.environ.get("PINECONE_INDEX", "langchain-test") - -### Ingest code - you may need to run this the first time -# # Load -# from langchain_community.document_loaders import WebBaseLoader -# loader = WebBaseLoader("https://lilianweng.github.io/posts/2023-06-23-agent/") -# data = loader.load() - -# # Split -# from langchain.text_splitter import RecursiveCharacterTextSplitter -# text_splitter = RecursiveCharacterTextSplitter(chunk_size=500, chunk_overlap=0) -# all_splits = text_splitter.split_documents(data) - -# # Add to vectorDB -# vectorstore = Pinecone.from_documents( -# documents=all_splits, embedding=OpenAIEmbeddings(), index_name=PINECONE_INDEX_NAME -# ) -# retriever = vectorstore.as_retriever() - -vectorstore = Pinecone.from_existing_index(PINECONE_INDEX_NAME, OpenAIEmbeddings()) -retriever = vectorstore.as_retriever() - -# Condense a chat history and follow-up question into a standalone question -_template = """Given the following conversation and a follow up question, rephrase the follow up question to be a standalone question, in its original language. -Chat History: -{chat_history} -Follow Up Input: {question} -Standalone question:""" # noqa: E501 -CONDENSE_QUESTION_PROMPT = PromptTemplate.from_template(_template) - -# RAG answer synthesis prompt -template = """Answer the question based only on the following context: - -{context} -""" -ANSWER_PROMPT = ChatPromptTemplate.from_messages( - [ - ("system", template), - MessagesPlaceholder(variable_name="chat_history"), - ("user", "{question}"), - ] -) - -# Conversational Retrieval Chain -DEFAULT_DOCUMENT_PROMPT = PromptTemplate.from_template(template="{page_content}") - - -def _combine_documents( - docs, document_prompt=DEFAULT_DOCUMENT_PROMPT, document_separator="\n\n" -): - doc_strings = [format_document(doc, document_prompt) for doc in docs] - return document_separator.join(doc_strings) - - -def _format_chat_history(chat_history: List[Tuple[str, str]]) -> List: - buffer = [] - for human, ai in chat_history: - buffer.append(HumanMessage(content=human)) - buffer.append(AIMessage(content=ai)) - return buffer - - -# User input -class ChatHistory(BaseModel): - chat_history: List[Tuple[str, str]] = Field(..., extra={"widget": {"type": "chat"}}) - question: str - - -_search_query = RunnableBranch( - # If input includes chat_history, we condense it with the follow-up question - ( - RunnableLambda(lambda x: bool(x.get("chat_history"))).with_config( - run_name="HasChatHistoryCheck" - ), # Condense follow-up question and chat into a standalone_question - RunnablePassthrough.assign( - chat_history=lambda x: _format_chat_history(x["chat_history"]) - ) - | CONDENSE_QUESTION_PROMPT - | ChatOpenAI(temperature=0) - | StrOutputParser(), - ), - # Else, we have no chat history, so just pass through the question - RunnableLambda(itemgetter("question")), -) - -_inputs = RunnableParallel( - { - "question": lambda x: x["question"], - "chat_history": lambda x: _format_chat_history(x["chat_history"]), - "context": _search_query | retriever | _combine_documents, - } -).with_types(input_type=ChatHistory) - -chain = _inputs | ANSWER_PROMPT | ChatOpenAI() | StrOutputParser() diff --git a/kios_bt_planning/kios_agent/__init__.py b/kios_bt_planning/kios_agent/__init__.py deleted file mode 100644 index 8b137891..00000000 --- a/kios_bt_planning/kios_agent/__init__.py +++ /dev/null @@ -1 +0,0 @@ - diff --git a/kios_bt_planning/kios_agent/data_types.py b/kios_bt_planning/kios_agent/data_types.py deleted file mode 100644 index e81dd495..00000000 --- a/kios_bt_planning/kios_agent/data_types.py +++ /dev/null @@ -1,47 +0,0 @@ -from dataclasses import dataclass, field -from typing import List, Optional, Dict, Any - - -@dataclass -class AgentResponse: - """Response from llm agent""" - - action_sequence: List[str] - ... - - @staticmethod - def from_json(json: Dict[str, Any]) -> "AgentResponse": - return AgentResponse( - action_sequence=json["action_sequence"], - ) - - -@dataclass -class KiosPromptSkeleton: - """Kios prompt template""" - - skeleton_name: str - model_name: str - version: str - - prompt_dir: str - prompt_load_order: List[str] - - fullname: str = field(default=None) - - @staticmethod - def from_json(json: Dict[str, Any]) -> "KiosPromptSkeleton": - skeleton_name = json["skeleton_name"] - prompt_dir = json["prompt_dir"] - prompt_load_order = json["prompt_load_order"] - version = json["version"] if "version" in json else "VX" - model_name = json["model_name"] if "model_name" in json else "gptX" - fullname = f"{skeleton_name}_{version}_{model_name}" - return KiosPromptSkeleton( - skeleton_name=skeleton_name, - model_name=model_name, - version=version, - prompt_dir=prompt_dir, - prompt_load_order=prompt_load_order, - fullname=fullname, - ) diff --git a/kios_bt_planning/kios_agent/kios_graph.py b/kios_bt_planning/kios_agent/kios_graph.py deleted file mode 100644 index 8dafb88f..00000000 --- a/kios_bt_planning/kios_agent/kios_graph.py +++ /dev/null @@ -1,309 +0,0 @@ -import json -import os - -# os.environ["LANGCHAIN_TRACING_V2"] = "true" -# os.environ["LANGCHAIN_ENDPOINT"] = "https://api.smith.langchain.com" -# os.environ["LANGCHAIN_PROJECT"] = "kios_human_in_the_loop" - - -from kios_agent.llm_supporter import KiosLLMSupporter -from kios_agent.data_types import KiosPromptSkeleton - - -from dotenv import load_dotenv - - -from langchain_openai import ChatOpenAI - -from langchain.chains.openai_functions import ( - create_structured_output_runnable, -) - -from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder -from langchain_core.output_parsers import JsonOutputParser, StrOutputParser -from langchain_core.pydantic_v1 import BaseModel, Field - -from langchain.prompts.pipeline import PipelinePromptTemplate -from langchain.prompts.prompt import PromptTemplate - - -load_dotenv() - -####################### dirs -current_dir = os.path.dirname(os.path.abspath(__file__)) -scene_path = os.path.join(current_dir, "scene.json") -# bt_json_file_path = os.path.join(current_dir, "behavior_tree.json") -world_state_path = os.path.join(current_dir, "world_state.json") -problem_path = os.path.join(current_dir, "gearset.problem") -domain_knowledge_path = os.path.join(current_dir, "domain_knowledge.txt") - -# * kios data prompt skeleton dir -data_dir = os.environ.get("KIOS_DATA_DIR").format(username=os.getlogin()) -print(data_dir) -prompt_sk_dir = os.path.join(data_dir, "prompt_skeletons") -prompt_dir = os.path.join(data_dir, "prompts") - - -########## *sequential planner for BT - -template_file = os.path.join(prompt_dir, "seq_planner/template.txt") -task_file = os.path.join(prompt_dir, "seq_planner/task.txt") -system_file = os.path.join(prompt_dir, "seq_planner/system.txt") -domain_file = os.path.join(prompt_dir, "seq_planner/domain.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {template} - """ -) - -seq_planner_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ], -) - -seq_planner_chain = ( - seq_planner_ppt_ppl - | ChatOpenAI(model="gpt-4-turbo-preview", temperature=0) - | StrOutputParser() -) - - -# * output schema of the recursive_sk_generator -class SolutionBehaviortree(BaseModel): - """the fixed behavior tree""" - - behavior_tree: dict = Field( - description="the behavior tree that adheres the required format." - ) - - -behaviortree_file = os.path.join(prompt_dir, "rec_sk_gen/behaviortree.txt") -template_file = os.path.join(prompt_dir, "rec_sk_gen/template.txt") -task_file = os.path.join(prompt_dir, "rec_sk_gen/task.txt") -system_file = os.path.join(prompt_dir, "rec_sk_gen/system.txt") -object_file = os.path.join(prompt_dir, "rec_sk_gen/object.txt") -domain_file = os.path.join(prompt_dir, "rec_sk_gen/domain.txt") -example_file = os.path.join(prompt_dir, "rec_sk_gen/example.txt") -state_file = os.path.join(prompt_dir, "rec_sk_gen/state.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) -with open(object_file, "r") as f: - object_ppt = PromptTemplate.from_template(f.read()) -with open(behaviortree_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - behaviortree_ppt = ppt_tmp.partial(input=f.read()) - -with open(state_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - state_ppt = ppt_tmp.partial(input=f.read()) - -with open(example_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - example_ppt = ppt_tmp.partial(input=f.read()) -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {state} - - {object} - - {behaviortree} - - {example} - - {template} - """ -) - -re_sk_gen_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ("object", object_ppt), - ("behaviortree", behaviortree_ppt), - ("example", example_ppt), - ("state", state_ppt), - ], -) - -rec_sk_gen_chain = ( - re_sk_gen_ppt_ppl - | ChatOpenAI(model="gpt-4-turbo-preview", temperature=0) - | JsonOutputParser() -) - - -##################################################### * planner -# * output schema of the planner -class Plan(BaseModel): - """Plan to follow in future""" - - steps: list[str] = Field( - description="a list of different steps to follow, should be in sorted order" - ) - - -template_file = os.path.join(prompt_dir, "planner/template.txt") -task_file = os.path.join(prompt_dir, "planner/task.txt") -system_file = os.path.join(prompt_dir, "planner/system.txt") -domain_file = os.path.join(prompt_dir, "planner/domain.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {template} - """ -) - -planner_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ], -) - -planner = create_structured_output_runnable( - Plan, ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), planner_ppt_ppl -) - -##################################################### * plan_updaterner - -template_file = os.path.join(prompt_dir, "plan_updater/template.txt") -task_file = os.path.join(prompt_dir, "plan_updater/task.txt") -system_file = os.path.join(prompt_dir, "plan_updater/system.txt") -domain_file = os.path.join(prompt_dir, "plan_updater/domain.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) - -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {template} - """ -) - -plan_updater_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ], -) - -plan_updater = create_structured_output_runnable( - Plan, ChatOpenAI(model="gpt-4-turbo-preview", temperature=0), plan_updater_ppt_ppl -) - -# * human instruction helped behavior tree generation -behaviortree_file = os.path.join(prompt_dir, "human_instruction/behaviortree.txt") -template_file = os.path.join(prompt_dir, "human_instruction/template.txt") -task_file = os.path.join(prompt_dir, "human_instruction/task.txt") -system_file = os.path.join(prompt_dir, "human_instruction/system.txt") -domain_file = os.path.join(prompt_dir, "human_instruction/domain.txt") -with open(template_file, "r") as f: - template_ppt = PromptTemplate.from_template(f.read()) -with open(task_file, "r") as f: - task_ppt = PromptTemplate.from_template(f.read()) -with open(system_file, "r") as f: - system_ppt = PromptTemplate.from_template(f.read()) -with open(domain_file, "r") as f: - domain_ppt = PromptTemplate.from_template(f.read()) -with open(behaviortree_file, "r") as f: - ppt_tmp = PromptTemplate.from_template("{input}") - behaviortree_ppt = ppt_tmp.partial(input=f.read()) - -# with open(state_file, "r") as f: -# ppt_tmp = PromptTemplate.from_template("{input}") -# state_ppt = ppt_tmp.partial(input=f.read()) - -# with open(example_file, "r") as f: -# ppt_tmp = PromptTemplate.from_template("{input}") -# example_ppt = ppt_tmp.partial(input=f.read()) -full_template_ppt = ChatPromptTemplate.from_template( - """{system} - - {task} - - {domain} - - {behaviortree} - - {template} - """ -) - -human_instruction_ppt_ppl = PipelinePromptTemplate( - final_prompt=full_template_ppt, - pipeline_prompts=[ - ("template", template_ppt), - ("task", task_ppt), - ("system", system_ppt), - ("domain", domain_ppt), - ("behaviortree", behaviortree_ppt), - ], -) - -human_instruction_chain = ( - human_instruction_ppt_ppl - | ChatOpenAI(model="gpt-4-turbo-preview", temperature=0) - | JsonOutputParser() -) diff --git a/kios_bt_planning/kios_agent/kios_langchain/TT/add_history.py b/kios_bt_planning/kios_agent/kios_langchain/TT/add_history.py deleted file mode 100644 index b7de1632..00000000 --- a/kios_bt_planning/kios_agent/kios_langchain/TT/add_history.py +++ /dev/null @@ -1,155 +0,0 @@ -""" -the application needs some sort of “memory” of past questions and answers, -and some logic for incorporating those into its current thinking. - -use a chain to handle the chat history and the latest user question to -reformulate the question to be standalone. -then feed the standalone question to the rag_chain to get the answer. - -not the true memory. -need reroutine. - -""" - -import bs4 -from langchain import hub -from langchain.text_splitter import RecursiveCharacterTextSplitter -from langchain_community.document_loaders import WebBaseLoader -from langchain_community.vectorstores import Chroma -from langchain_core.output_parsers import StrOutputParser -from langchain_core.runnables import RunnablePassthrough -from langchain_openai import ChatOpenAI, OpenAIEmbeddings - -import os - -os.environ["LANGCHAIN_TRACING_V2"] = "true" -os.environ["LANGCHAIN_ENDPOINT"] = "https://api.smith.langchain.com" -os.environ["LANGCHAIN_PROJECT"] = "kios_agent" - -# Load, chunk and index the contents of the blog. -loader = WebBaseLoader( - web_paths=("https://lilianweng.github.io/posts/2023-06-23-agent/",), - bs_kwargs=dict( - parse_only=bs4.SoupStrainer( - class_=("post-content", "post-title", "post-header") - ) - ), -) -docs = loader.load() - -text_splitter = RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=200) -splits = text_splitter.split_documents(docs) -vectorstore = Chroma.from_documents(documents=splits, embedding=OpenAIEmbeddings()) - -# Retrieve and generate using the relevant snippets of the blog. -retriever = vectorstore.as_retriever() -prompt = hub.pull("rlm/rag-prompt") -llm = ChatOpenAI(model_name="gpt-3.5-turbo", temperature=0) - - -def format_docs(docs): - return "\n\n".join(doc.page_content for doc in docs) - - -# rag_chain = ( -# {"context": retriever | format_docs, "question": RunnablePassthrough()} -# | prompt -# | llm -# | StrOutputParser() -# ) - -""" - a sub-chain that takes historical messages and the latest user question, - and reformulates the question to be standalone -""" - -from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder - -contextualize_q_system_prompt = """Given a chat history and the latest user question \ -which might reference context in the chat history, formulate a standalone question \ -which can be understood without the chat history. Do NOT answer the question, \ -just reformulate it if needed and otherwise return it as is.""" - -# * the prompt template to make a standalone question -contextualize_q_prompt = ChatPromptTemplate.from_messages( - [ - ("system", contextualize_q_system_prompt), - MessagesPlaceholder(variable_name="chat_history"), - ("human", "{question}"), - ] -) -contextualize_q_chain = contextualize_q_prompt | llm | StrOutputParser() - -""" Example usage -""" - -from langchain_core.messages import AIMessage, HumanMessage - -# the context chain to make a standalone question -contextualize_q_chain.invoke( # * invoke needs to fill out the missing part of the prompt. - { - # * so this part is a hand-crafted example of a chat history and a question - "chat_history": [ - HumanMessage(content="What does LLM stand for?"), - AIMessage(content="Large language model"), # answered by the user - ], - "question": "What is meant by large", - } -) - -# * the full chain - -qa_system_prompt = """You are an assistant for question-answering tasks. \ -Use the following pieces of retrieved context to answer the question. \ -If you don't know the answer, just say that you don't know. \ -Use three sentences maximum and keep the answer concise.\ - -{context}""" - -# * the template for the prompt -qa_prompt = ChatPromptTemplate.from_messages( - [ - ("system", qa_system_prompt), - MessagesPlaceholder(variable_name="chat_history"), - ("human", "{question}"), - ] -) - - -""" -reroutine the rag chain. -if the input has a chat history, it will use the contextualize_q_chain to make a standalone question. -if not, the question should be the standalone one. it will use the question as is. -""" - - -def contextualized_question(input: dict): - if input.get( - "chat_history" - ): # * In Python, an empty list [] is considered False in a boolean context. - return ( - contextualize_q_chain # in which you need the chat history and the question - ) - else: - return input["question"] - - -# * the chain -rag_chain = ( - RunnablePassthrough.assign( - context=contextualized_question | retriever | format_docs - ) - | qa_prompt - | llm -) - -chat_history = [] - -# * the first question -question = "What is Task Decomposition?" -ai_msg = rag_chain.invoke({"question": question, "chat_history": chat_history}) -chat_history.extend([HumanMessage(content=question), ai_msg]) - -# * the second question -second_question = "What are common ways of doing it?" -rag_chain.invoke({"question": second_question, "chat_history": chat_history}) diff --git a/kios_bt_planning/kios_agent/kios_langchain/TT/citation.py b/kios_bt_planning/kios_agent/kios_langchain/TT/citation.py deleted file mode 100644 index b1346af3..00000000 --- a/kios_bt_planning/kios_agent/kios_langchain/TT/citation.py +++ /dev/null @@ -1,315 +0,0 @@ -""" -extracting citations, showing which from the source document are used for -generating the answer and which are not. - -"which parts are truly in use?" - -this is quite critical when retrieving from online sources like wiki, where -the retrieved snippets are not always used for generating the answer. - -""" - -# ! skipped Direct prompting which is a hand-crafted citation example - -# * or we can do post processing after retrieving the sources, -# * to select the most relevant snippets to use for generating the answer. -# * this asks for another splitting, embedding and retrieval, -# * which is not efficient but can save tokens. - - -# * Another approach is to post-process our model generation. -# * get answer first, and ask the model to annotate the sources used. -# ! this will cost a lot of tokens!!! - -import bs4 -from langchain import hub -from langchain.text_splitter import RecursiveCharacterTextSplitter -from langchain_community.document_loaders import WebBaseLoader -from langchain_community.vectorstores import Chroma -from langchain_core.output_parsers import StrOutputParser -from langchain_core.runnables import RunnablePassthrough -from langchain_openai import ChatOpenAI, OpenAIEmbeddings - -import os - -os.environ["LANGCHAIN_TRACING_V2"] = "true" -os.environ["LANGCHAIN_ENDPOINT"] = "https://api.smith.langchain.com" -os.environ["LANGCHAIN_PROJECT"] = "kios_agent" - -from langchain_community.retrievers import WikipediaRetriever -from langchain_core.prompts import ChatPromptTemplate - -llm = ChatOpenAI(model="gpt-3.5-turbo", temperature=0) -wiki = WikipediaRetriever(top_k_results=6, doc_content_chars_max=2000) -prompt = ChatPromptTemplate.from_messages( - [ - ( - "system", - "You're a helpful AI assistant. Given a user question and some Wikipedia article snippets, answer the user question. If none of the articles answer the question, just say you don't know.\n\nHere are the Wikipedia articles:{context}", - ), - ("human", "{question}"), - ] -) - -# prompt.pretty_print() - -from operator import itemgetter -from typing import List - -from langchain_core.documents import Document -from langchain_core.output_parsers import StrOutputParser -from langchain_core.runnables import ( - RunnableLambda, - RunnableParallel, - RunnablePassthrough, -) - - -def format_docs(docs: List[Document]) -> str: - """Convert Documents to a single string.:""" - formatted = [ - f"Article Title: {doc.metadata['title']}\nArticle Snippet: {doc.page_content}" - for doc in docs - ] - return "\n\n" + "\n\n".join(formatted) - - -# * callable operator to extract the docs and format them -format = itemgetter("docs") | RunnableLambda(format_docs) -# * subchain for generating an answer once we've done retrieval -answer = prompt | llm | StrOutputParser() -# complete chain that calls wiki -> formats docs to string -> runs answer subchain -> returns just the answer and retrieved docs. -chain = ( - RunnableParallel(question=RunnablePassthrough(), docs=wiki) - .assign(context=format) # * assign chains to keys - .assign(answer=answer) # * assign answer chain to the answer key - .pick(["answer", "docs"]) # * pick only the answer and the docs as the outputs -) - -# chain.invoke("How fast are cheetahs?") - -######################## -# * use function-calling - -from langchain_core.pydantic_v1 import BaseModel, Field - - -class cited_answer(BaseModel): - """Answer the user question based only on the given sources, and cite the sources used.""" - - answer: str = Field( - ..., - description="The answer to the user question, which is based only on the given sources.", - ) - citations: List[int] = Field( - ..., - description="The integer IDs of the SPECIFIC sources which justify the answer.", - ) - - -# * use tool "class" -llm_with_tool = llm.bind_tools( - [cited_answer], - tool_choice="cited_answer", -) -example_q = """What Brian's height? - -Source: 1 -Information: Suzy is 6'2" - -Source: 2 -Information: Jeremiah is blonde - -Source: 3 -Information: Brian is 3 inches shorted than Suzy""" - -# llm_with_tool.invoke(example_q) # * default is "question" - - -# * output parser to extract the cited answer -from langchain.output_parsers.openai_tools import JsonOutputKeyToolsParser - -output_parser = JsonOutputKeyToolsParser(key_name="cited_answer", return_single=True) -# (llm_with_tool | output_parser).invoke(example_q) - - -def format_docs_with_id(docs: List[Document]) -> str: - """ - get the source id, article title and article snippet of the documents - return as a string - """ - formatted = [ - f"Source ID: {i}\nArticle Title: {doc.metadata['title']}\nArticle Snippet: {doc.page_content}" - for i, doc in enumerate(docs) - ] - return "\n\n" + "\n\n".join(formatted) - - -# format chain -format_1 = itemgetter("docs") | RunnableLambda(format_docs_with_id) -# answer chain -answer_1 = prompt | llm_with_tool | output_parser -# the full chain -chain_1 = ( - RunnableParallel(question=RunnablePassthrough(), docs=wiki) - .assign(context=format_1) - .assign(cited_answer=answer_1) - .pick(["cited_answer", "docs"]) -) - -# for chunk in chain_1.stream("How fast are cheetahs?"): -# print(chunk, end="", flush=True) - -# * cite snippets - - -class Citation(BaseModel): - source_id: int = Field( - ..., - description="The integer ID of a SPECIFIC source which justifies the answer.", - ) - quote: str = Field( - ..., - description="The VERBATIM quote from the specified source that justifies the answer.", - ) - - -class quoted_answer(BaseModel): - """Answer the user question based only on the given sources, and cite the sources used.""" - - answer: str = Field( - ..., - description="The answer to the user question, which is based only on the given sources.", - ) - citations: List[Citation] = Field( - ..., description="Citations from the given sources that justify the answer." - ) - - -output_parser_2 = JsonOutputKeyToolsParser(key_name="quoted_answer", return_single=True) -llm_with_tool_2 = llm.bind_tools( - [quoted_answer], - tool_choice="quoted_answer", -) -format_2 = itemgetter("docs") | RunnableLambda(format_docs_with_id) -answer_2 = prompt | llm_with_tool_2 | output_parser_2 -chain_2 = ( - RunnableParallel(question=RunnablePassthrough(), docs=wiki) - .assign(context=format_2) - .assign(quoted_answer=answer_2) - .pick(["quoted_answer", "docs"]) -) - -# for chunk in chain_2.stream("How fast are cheetahs?"): -# print(chunk, end="", flush=True) - -# * post processing - -from langchain.retrievers.document_compressors import EmbeddingsFilter -from langchain.text_splitter import RecursiveCharacterTextSplitter -from langchain_openai import OpenAIEmbeddings - -splitter = RecursiveCharacterTextSplitter( - chunk_size=400, - chunk_overlap=0, - separators=["\n\n", "\n", ".", " "], - keep_separator=False, -) -compressor = EmbeddingsFilter(embeddings=OpenAIEmbeddings(), k=10) - - -def split_and_filter(input) -> List[Document]: - docs = input["docs"] - question = input["question"] - split_docs = splitter.split_documents(docs) - stateful_docs = compressor.compress_documents(split_docs, question) - return [stateful_doc for stateful_doc in stateful_docs] - - -retrieve = ( - RunnableParallel(question=RunnablePassthrough(), docs=wiki) | split_and_filter -) -docs = retrieve.invoke("How fast are cheetahs?") -# for doc in docs: -# print(doc.page_content) -# print("\n\n") - -chain_4 = ( - RunnableParallel(question=RunnablePassthrough(), docs=retrieve) - .assign(context=format) - .assign(answer=answer) - .pick(["answer", "docs"]) -) - -# Note the documents have an article "summary" in the metadata that is now much longer than the -# actual document page content. This summary isn't actually passed to the model. -# chain_4.invoke("How fast are cheetahs?") - -# * Generation post-processing - - -class Citation(BaseModel): - source_id: int = Field( - ..., - description="The integer ID of a SPECIFIC source which justifies the answer.", - ) - quote: str = Field( - ..., - description="The VERBATIM quote from the specified source that justifies the answer.", - ) - - -class annotated_answer(BaseModel): - """Annotate the answer to the user question with quote citations that justify the answer.""" - - citations: List[Citation] = Field( - ..., description="Citations from the given sources that justify the answer." - ) - - -llm_with_tools_5 = llm.bind_tools( - [annotated_answer], - tool_choice="annotated_answer", -) - -from langchain_core.prompts import MessagesPlaceholder - -prompt_5 = ChatPromptTemplate.from_messages( - [ - ( - "system", - "You're a helpful AI assistant. Given a user question and some Wikipedia article snippets, answer the user question. If none of the articles answer the question, just say you don't know.\n\nHere are the Wikipedia articles:{context}", - ), - ("human", "{question}"), - MessagesPlaceholder("chat_history", optional=True), # * optional chat history - ] -) -answer_5 = prompt_5 | llm # * answer chain needs llm -annotation_chain = ( - prompt_5 - | llm_with_tools_5 - | JsonOutputKeyToolsParser(key_name="annotated_answer", return_single=True) - | itemgetter("citations") -) -# * the assign are executed one by one, you can check it in langsmith monitor. -chain_5 = ( - RunnableParallel(question=RunnablePassthrough(), docs=wiki) - .assign( - context=format - ) # * retrieve the sources and format them, assign to the context key - .assign( - ai_message=answer_5 - ) # * generate the answer and annotate the sources used, assign to the ai_message key - .assign( - chat_history=(lambda x: [x["ai_message"]]), - answer=(lambda x: x["ai_message"].content), - ) # * assign the chat history and the answer to the chat_history and answer keys - .assign( - annotations=annotation_chain - ) # * annotate the sources used, assign to the annotations key - .pick( - ["answer", "docs", "annotations"] - ) # * pick the answer, the sources and the annotations as the outputs -) - -chain_5.invoke("How fast are cheetahs?") diff --git a/kios_bt_planning/kios_agent/kios_langchain/TT/multi_user.py b/kios_bt_planning/kios_agent/kios_langchain/TT/multi_user.py deleted file mode 100644 index ca420e98..00000000 --- a/kios_bt_planning/kios_agent/kios_langchain/TT/multi_user.py +++ /dev/null @@ -1,3 +0,0 @@ -""" -skipped -""" diff --git a/kios_bt_planning/kios_agent/kios_langchain/TT/returning_sources.py b/kios_bt_planning/kios_agent/kios_langchain/TT/returning_sources.py deleted file mode 100644 index efa662c6..00000000 --- a/kios_bt_planning/kios_agent/kios_langchain/TT/returning_sources.py +++ /dev/null @@ -1,64 +0,0 @@ -import os - -""" -show users the sources that were used to generate the answer -""" - -import bs4 -from langchain import hub -from langchain.text_splitter import RecursiveCharacterTextSplitter -from langchain_community.document_loaders import WebBaseLoader -from langchain_community.vectorstores import Chroma -from langchain_core.output_parsers import StrOutputParser -from langchain_core.runnables import RunnablePassthrough -from langchain_openai import ChatOpenAI, OpenAIEmbeddings - -os.environ["LANGCHAIN_TRACING_V2"] = "true" -os.environ["LANGCHAIN_ENDPOINT"] = "https://api.smith.langchain.com" -os.environ["LANGCHAIN_PROJECT"] = "kios_agent" - -# Load, chunk and index the contents of the blog. -bs_strainer = bs4.SoupStrainer(class_=("post-content", "post-title", "post-header")) -loader = WebBaseLoader( - web_paths=("https://lilianweng.github.io/posts/2023-06-23-agent/",), - bs_kwargs={"parse_only": bs_strainer}, -) -docs = loader.load() - -text_splitter = RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=200) -splits = text_splitter.split_documents(docs) -vectorstore = Chroma.from_documents(documents=splits, embedding=OpenAIEmbeddings()) - -# Retrieve and generate using the relevant snippets of the blog. -retriever = vectorstore.as_retriever() -prompt = hub.pull("rlm/rag-prompt") -llm = ChatOpenAI(model_name="gpt-3.5-turbo", temperature=0) - - -def format_docs(docs): - return "\n\n".join(doc.page_content for doc in docs) - - -# rag_chain = ( -# {"context": retriever | format_docs, "question": RunnablePassthrough()} -# | prompt -# | llm -# | StrOutputParser() -# ) - -# rag_chain.invoke("What is Task Decomposition?") - -from langchain_core.runnables import RunnableParallel - -rag_chain_from_docs = ( - RunnablePassthrough.assign(context=(lambda x: format_docs(x["context"]))) - | prompt - | llm - | StrOutputParser() -) - -rag_chain_with_source = RunnableParallel( - {"context": retriever, "question": RunnablePassthrough()} -).assign(answer=rag_chain_from_docs) - -rag_chain_with_source.invoke("What is Task Decomposition") diff --git a/kios_bt_planning/kios_agent/kios_langchain/TT/streaming.py b/kios_bt_planning/kios_agent/kios_langchain/TT/streaming.py deleted file mode 100644 index 8fb91f7e..00000000 --- a/kios_bt_planning/kios_agent/kios_langchain/TT/streaming.py +++ /dev/null @@ -1,73 +0,0 @@ -import bs4 -from langchain import hub -from langchain.text_splitter import RecursiveCharacterTextSplitter -from langchain_community.document_loaders import WebBaseLoader -from langchain_community.vectorstores import Chroma -from langchain_core.output_parsers import StrOutputParser -from langchain_core.runnables import RunnableParallel, RunnablePassthrough -from langchain_openai import ChatOpenAI, OpenAIEmbeddings - -""" -Streaming final outputs -""" - -# Load, chunk and index the contents of the blog. -bs_strainer = bs4.SoupStrainer(class_=("post-content", "post-title", "post-header")) -loader = WebBaseLoader( - web_paths=("https://lilianweng.github.io/posts/2023-06-23-agent/",), - bs_kwargs={"parse_only": bs_strainer}, -) -docs = loader.load() - -text_splitter = RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=200) -splits = text_splitter.split_documents(docs) -vectorstore = Chroma.from_documents(documents=splits, embedding=OpenAIEmbeddings()) - -# Retrieve and generate using the relevant snippets of the blog. -retriever = vectorstore.as_retriever() -prompt = hub.pull("rlm/rag-prompt") -llm = ChatOpenAI(model_name="gpt-3.5-turbo", temperature=0) - - -def format_docs(docs): - return "\n\n".join(doc.page_content for doc in docs) - - -rag_chain_from_docs = ( - RunnablePassthrough.assign(context=(lambda x: format_docs(x["context"]))) - | prompt - | llm - | StrOutputParser() -) - -rag_chain_with_source = RunnableParallel( - {"context": retriever, "question": RunnablePassthrough()} -).assign(answer=rag_chain_from_docs) - - -# * stream method of the chain -# for chunk in rag_chain_with_source.stream("What is Task Decomposition"): -# print(chunk) - -output = {} -curr_key = None - -""" -only print each key once, and then print the values -""" - -for chunk in rag_chain_with_source.stream("What is Task Decomposition"): - for key in chunk: - if key not in output: - output[key] = chunk[key] - else: - output[key] += chunk[key] - if key != curr_key: - print(f"\n\n{key}: {chunk[key]}", end="", flush=True) - else: - print(chunk[key], end="", flush=True) - curr_key = key -output - - -# ! skipped the steps for streaming the middle steps diff --git a/kios_bt_planning/kios_agent/kios_langchain/TT/using_agent.py b/kios_bt_planning/kios_agent/kios_langchain/TT/using_agent.py deleted file mode 100644 index 998988bb..00000000 --- a/kios_bt_planning/kios_agent/kios_langchain/TT/using_agent.py +++ /dev/null @@ -1,34 +0,0 @@ -# # * get the documents -# from langchain_community.document_loaders import TextLoader - -# loader = TextLoader("../../modules/state_of_the_union.txt") -# documents = loader.load() -# ! where is this file? - -# # * create_retriever -# from langchain.text_splitter import CharacterTextSplitter -# from langchain_community.vectorstores import FAISS -# from langchain_openai import OpenAIEmbeddings - -# text_splitter = CharacterTextSplitter(chunk_size=1000, chunk_overlap=0) -# texts = text_splitter.split_documents(documents) -# embeddings = OpenAIEmbeddings() -# db = FAISS.from_documents(texts, embeddings) - -# retriever = db.as_retriever() - -# # * create_retriever_tool -# from langchain.tools.retriever import create_retriever_tool - -# tool = create_retriever_tool( -# retriever, -# "search_state_of_union", -# "Searches and returns excerpts from the 2022 State of the Union.", -# ) -# tools = [tool] - -from langchain import hub - -prompt = hub.pull("hwchase17/openai-tools-agent") -print(prompt.messages) -# prompt.messages diff --git a/kios_bt_planning/kios_agent/kios_langchain/kios_agent.py b/kios_bt_planning/kios_agent/kios_langchain/kios_agent.py deleted file mode 100644 index e69de29b..00000000 diff --git a/kios_bt_planning/kios_agent/kios_langchain/test.py b/kios_bt_planning/kios_agent/kios_langchain/test.py deleted file mode 100644 index 530c4905..00000000 --- a/kios_bt_planning/kios_agent/kios_langchain/test.py +++ /dev/null @@ -1,3 +0,0 @@ -from langchain import hub - -obj = hub.pull("homanp/superagent") diff --git a/kios_bt_planning/kios_agent/kios_langchain/test_chain.py b/kios_bt_planning/kios_agent/kios_langchain/test_chain.py deleted file mode 100644 index ff616a3d..00000000 --- a/kios_bt_planning/kios_agent/kios_langchain/test_chain.py +++ /dev/null @@ -1,52 +0,0 @@ -import bs4 -import os -import getpass -from langchain import hub -from langchain.text_splitter import RecursiveCharacterTextSplitter -from langchain_community.document_loaders import WebBaseLoader -from langchain_community.vectorstores import Chroma -from langchain_core.output_parsers import StrOutputParser -from langchain_core.runnables import RunnablePassthrough -from langchain_openai import ChatOpenAI, OpenAIEmbeddings - -# * langsmith tracing -os.environ["LANGCHAIN_TRACING_V2"] = "true" -os.environ["LANGCHAIN_ENDPOINT"] = "https://api.smith.langchain.com" -# terminal input -# os.environ["LANGCHAIN_API_KEY"] = getpass.getpass() -os.environ["LANGCHAIN_PROJECT"] = "kios_agent" - -# Load, chunk and index the contents of the blog. -loader = WebBaseLoader( - web_paths=("https://lilianweng.github.io/posts/2023-06-23-agent/",), - bs_kwargs=dict( - parse_only=bs4.SoupStrainer( - class_=("post-content", "post-title", "post-header") - ) - ), -) -docs = loader.load() - -text_splitter = RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=200) -splits = text_splitter.split_documents(docs) -vectorstore = Chroma.from_documents(documents=splits, embedding=OpenAIEmbeddings()) - -# Retrieve and generate using the relevant snippets of the blog. -retriever = vectorstore.as_retriever() -prompt = hub.pull("rlm/rag-prompt") -llm = ChatOpenAI(model_name="gpt-3.5-turbo", temperature=0) - - -def format_docs(docs): - return "\n\n".join(doc.page_content for doc in docs) - - -rag_chain = ( - {"context": retriever | format_docs, "question": RunnablePassthrough()} - | prompt - | llm - | StrOutputParser() -) - -for chunk in rag_chain.stream("What is a task in task composition?"): - print(chunk, end="", flush=True) diff --git a/kios_bt_planning/kios_agent/kios_langchain/test_smith.py b/kios_bt_planning/kios_agent/kios_langchain/test_smith.py deleted file mode 100644 index a7b36ac2..00000000 --- a/kios_bt_planning/kios_agent/kios_langchain/test_smith.py +++ /dev/null @@ -1,12 +0,0 @@ -import os -from uuid import uuid4 -import getpass - -unique_id = uuid4().hex[0:8] -os.environ["LANGCHAIN_TRACING_V2"] = "true" -os.environ["LANGCHAIN_PROJECT"] = f"Tracing Walkthrough - {unique_id}" -os.environ["LANGCHAIN_ENDPOINT"] = "https://api.smith.langchain.com" -os.environ["LANGCHAIN_API_KEY"] = "" # Update to your API key - -# Used by the agent in this tutorial -os.environ["OPENAI_API_KEY"] = getpass.getpass() diff --git a/kios_bt_planning/kios_agent/kios_llm.py b/kios_bt_planning/kios_agent/kios_llm.py deleted file mode 100644 index 30a05dff..00000000 --- a/kios_bt_planning/kios_agent/kios_llm.py +++ /dev/null @@ -1,312 +0,0 @@ -from typing import Any, List, Dict # ! use embedded typing in python from 3.10. -import openai -import tiktoken -import json -import os -import re -import ast -import argparse -import sys -import textwrap - -from langsmith import traceable -from langsmith.wrappers import wrap_openai - -""" -the llm class for prompt testing. -# ! now use chain. -""" - -# langsmith tracing -os.environ["LANGCHAIN_TRACING_V2"] = "true" -os.environ["LANGCHAIN_ENDPOINT"] = "https://api.smith.langchain.com" -os.environ["LANGCHAIN_PROJECT"] = "kios_agent" - - -class KiosLLM: - encoder: tiktoken.Encoding = None - prompt_directories: Dict[str, str] = None - - ######################################## * - max_token_length: int - max_completion_length: int - last_response: str = None - - ######################################## * prompts - prompt_dir: str = None - prompt_load_order: List[str] = None - - # the problem description - problem: str = None - - # the query template - query: str = None - - # variable to keep the latest instruction - instruction: str = None - - # the "system" prompt part - system_message: dict = None - - # The list of messages for prompting, will be sent to api - # including: the required format (in prompt folder), the last response as assistant prompt - messages: List[Dict[str, str]] = None - - ######################################## * dir - history_dir: str = None - - def __init__(self, openai_api_key: str = None): - if openai_api_key is not None: - openai.api_key = openai_api_key - else: - try: - openai.api_key = os.environ["OPENAI_API_KEY"] - except KeyError: - raise Exception( - "OPENAI_API_KEY is not given or set in the environment!" - ) - - # ! extend this method later. - def initialize(self, prompt_dir: str = None, prompt_load_order: List[str] = None): - self.encoder = tiktoken.get_encoding("cl100k_base") - script_dir = os.path.dirname(__file__) - if prompt_dir is not None: - self.prompt_dir = os.path.join(script_dir, prompt_dir) - else: - self.prompt_dir = os.path.join(script_dir, "prompts") - self.prompt_directories = { - "system": os.path.join(self.prompt_dir, "system"), - "query": os.path.join(self.prompt_dir, "query"), - "prompt": os.path.join(self.prompt_dir, "prompt"), - } - - # * check the prompt directories - print("prompt directories:") - for key, value in self.prompt_directories.items(): - print(f'["{key}"] = {value}') - - # default prompt load order - if prompt_load_order is not None: - self.prompt_load_order = prompt_load_order - else: - self.prompt_load_order = [ - "prompt_role", # your are a good interpreter ... Done - "prompt_domain", # domain knowledge ... Done - "prompt_problem", # how the problem is provided ... Done - "prompt_environment", # how to express the environment ... Done - # "prompt_behaviortree", # how to construct the behavior tree ... Done - "prompt_bt_skeleton", # how to construct the skeleton behavior tree ... Done - "prompt_bt_skeleton_example", # some skeleton examples ... Done - "prompt_output_format", # the output format ... Done - # "prompt_example", # some examples ... Done - ] - - # history directory - if not os.path.exists(os.path.join(script_dir, "query_history")): - os.makedirs(os.path.join(script_dir, "query_history")) - self.history_dir = os.path.join(script_dir, "query_history") - - # default - self.max_token_length: int = 16000 - self.max_completion_length: int = 4000 - self.last_response = None - - # prompt initialization - self.messages = [] - self.query = "" - self.instruction = "" - - # *load prompt file - # system - fp_system = os.path.join(self.prompt_directories["system"], "system.txt") - with open(fp_system) as f: - data = f.read() - self.system_message = {"role": "system", "content": data} - - # prompt for domain knowledge, user: brabrabra, assistant: do nothing and wait for the next prompt - for prompt_name in self.prompt_load_order: - fp_prompt = os.path.join( - self.prompt_directories["prompt"], prompt_name + ".txt" - ) - with open(fp_prompt) as f: - data = f.read() - data_spilit = re.split(r"\[user\]\n|\[assistant\]\n", data) - data_spilit = [item for item in data_spilit if len(item) != 0] - assert len(data_spilit) % 2 == 0 - # load sparately to user and assistant - for i, item in enumerate(data_spilit): - if i % 2 == 0: - self.messages.append({"sender": "user", "text": item}) - else: - self.messages.append({"sender": "assistant", "text": item}) - - # load the query template - fp_query = os.path.join(self.prompt_directories["query"], "query.txt") - with open(fp_query) as f: - self.query = f.read() - - # print("KiosLLM initialized") - # print(self.query) - # print(self.messages) - - def setup_model(self): - pass - - def model_query(self) -> json: - pass - - def parse_response(self, response: json) -> Any: - pass - - def create_prompt(self): - """ - create the prompt from messages for gpt api. - if too long, truncate the prompt and call this recursively - """ - prompt = [] # list of prompt entries - # load the system prompt - prompt.append(self.system_message) - # load the knowledge and the last response - for message in self.messages: - prompt.append({"role": message["sender"], "content": message["text"]}) - - # * truncate the prompt if it is too long - prompt_content = "" # this is only for checking the length of the prompt - for message in prompt: - prompt_content += message["content"] - print("prompt length: " + str(len(self.encoder.encode(prompt_content)))) - if ( - len(self.encoder.encode(prompt_content)) - > self.max_token_length - self.max_completion_length - ): - print("prompt too long. truncated.") - # * truncate the prompt by removing the oldest two messages from front - self.messages = self.messages[2:] - prompt = self.create_prompt() - return prompt - - def query_llm(self, problem: str, problem_name: str, model: str = None) -> dict: - """ - runtime_instruction: the instruction from the problem/user - """ - - # get the query template - text_base = self.query - - # replace problem part - if text_base.find("[PROBLEM]") != -1: - text_base = text_base.replace("[PROBLEM]", json.dumps(problem)) - - # finally, add the text_base to the user query message - self.messages.append({"sender": "user", "text": text_base}) - - # print("messages:") - # # print(self.messages) - # for message in self.create_prompt(): - # print(message.get("role")) - - # * Substitue the openai.Client() with wrap_openai(openai.Client()) to enable tracing - client = wrap_openai(openai.Client()) - - # * request the gpt to response - if model is None: - model = "gpt-4-1106-preview" - - response = client.chat.completions.create( - # model="gpt-3.5-turbo-16k-0613", - # model="gpt-4-0613", # not this - model=model, - messages=self.create_prompt(), - temperature=0.0, - max_tokens=self.max_completion_length, - top_p=0.3, - frequency_penalty=0.0, - presence_penalty=0.3, - ) - - text = response.choices[0].message.content - print(text) - - self.last_response = text - self.last_response = self.extract_json_part(self.last_response) - self.last_response = self.last_response.replace("'", '"') - - self.record_history( - query=text_base, response=self.last_response, problem_name=problem_name - ) - - self.json_dict = json.loads(self.last_response, strict=False) - - return self.json_dict - - def extract_json_part(self, text): - """ - extract the markdown code block part from the text - """ - if text.find("```") == -1: - return text - text_json = text[text.find("```") + 3 : text.find("```", text.find("```") + 3)] - return text_json - - def dump_json(self, dump_name=None): - """ - dump the json dictionary to a file, used to save into a json file - """ - if dump_name is not None: - # dump the dictionary to json file dump 1, 2, ... - fp = os.path.join(dump_name + ".json") - with open(fp, "w") as f: - json.dump(self.json_dict, f, indent=4) - - def record_history(self, query: str, response: str, problem_name: str): - - while not os.path.exists(os.path.join(self.history_dir, problem_name)): - os.makedirs(os.path.join(self.history_dir, problem_name)) - - problem_dir = os.path.join(self.history_dir, problem_name) - - i = 0 - while os.path.exists(os.path.join(problem_dir, str(i))): - i = i + 1 - os.makedirs(os.path.join(problem_dir, str(i))) - this_problem_dir = os.path.join(problem_dir, str(i)) - - # get the json response - json_response = response - - # dump to a text file - with open(os.path.join(this_problem_dir, "response.txt"), "w") as f: - f.write(json_response) - - # write the query problem - with open(os.path.join(this_problem_dir, "query.txt"), "w") as f: - f.write(query) - - # copy the prompts - with open(os.path.join(this_problem_dir, "message.txt"), "w") as f: - f.write(str(self.messages)) - - -def test_llm(): - problem_name = "gearset_skeleton_v1" - problem = "(define (problem robot_assembly_problem-problem)\ - (:domain robot_assembly_problem-domain)\ - (:objects\ - parallel_box1 parallel_box2 inward_claw outward_claw no_tool - tool\ - gear1 gear2 gear3 shaft1 shaft2 base - part\ - left_hand - hand\ - )\ - (:init (can_manipulate parallel_box1 gear1) (can_manipulate outward_claw gear2) (can_manipulate inward_claw gear3) (can_manipulate parallel_box2 shaft1) (can_manipulate no_tool shaft2) (can_screw_to leg1 seat) (can_screw_to leg2 seat) (can_insert_to back seat) (can_screw_to nut1 seat) (can_screw_to nut2 seat) (can_screw_to blub base) (can_place_to lamp blub) (can_insert_to shaft1 base) (can_insert_to shaft2 base) (can_insert_to gear3 shaft2) (can_insert_to gear2 base) (can_insert_to gear1 shaft1) (is_inserted_to shaft1 base) (is_free left_hand) (is_free parallel_box1) (is_free parallel_box2) (is_free inward_claw) (is_free outward_claw) (is_free no_tool) (is_free gear1) (is_free gear2) (is_free gear3) (is_free shaft1) (is_free shaft2) (is_free base) (is_equippable parallel_box1) (is_equippable parallel_box2) (is_equippable inward_claw) (is_equippable outward_claw) (is_equippable no_tool))\ - (:goal (and (is_inserted_to gear1 shaft1)))\ - )" # ! CHEAT - - llm_model = KiosLLM() - llm_model.initialize() - - # run the instructions one by one - response = llm_model.query_llm(problem, problem_name) - - -if __name__ == "__main__": - # test_llm() - pass diff --git a/kios_bt_planning/kios_agent/kios_llm_bt/aimodel.py b/kios_bt_planning/kios_agent/kios_llm_bt/aimodel.py deleted file mode 100644 index 1b0fbd71..00000000 --- a/kios_bt_planning/kios_agent/kios_llm_bt/aimodel.py +++ /dev/null @@ -1,419 +0,0 @@ -import openai -import tiktoken -import json -import os -import re -import argparse -import sys -import textwrap - -# ! code is still dirty. need to be cleaned up and wrapped into a class KiosAgent - -enc = tiktoken.get_encoding("cl100k_base") - -script_dir = os.path.dirname(__file__) -print(script_dir) # output: src/kios/kios_bt_planning/kios_agent/kios_llm_bt -upper_level = os.path.dirname(script_dir) -print(upper_level) # output: src/kios/kios_bt_planning - -secret_path = os.path.join(upper_level, "secrets.json") - -# with open("../../secrets.json") as f: -with open(secret_path) as f: - credentials = json.load(f) - # print(credentials) - - -# dir_system = "./system" -dir_system = os.path.join(script_dir, "system") -# dir_prompt = "./prompt" -dir_prompt = os.path.join(script_dir, "prompt") -# dir_query = "./query" -dir_query = os.path.join(script_dir, "query") - -prompt_load_order = [ - "prompt_role", # your are a good interpreter ... Done - "prompt_domain", # domain knowledge ... Done - "prompt_problem", # how the problem is provided ... Done - "prompt_environment", # how to express the environment ... Done - "prompt_behaviortree", # how to construct the behavior tree ... Done - "prompt_output_format", # the output format ... Done - "prompt_example", # some examples ... Done -] - -# and also, query: the user request and the rules for the generation of the response ... Done -# ! still not rules about the behavior tree yet. - - -# azure openai api has changed from '2023-05-15' to '2023-05-15' -# if you are using a 0301 version, use '2022-12-01' -# Otherwise, use '2023-05-15' - - -class ChatGPT: - problem: str = None - """ - the problem description - """ - - query: str = None - """ - the query request: start working ... - """ - instruction: str = None - """ - the member variable to keep the latest instruction - """ - system_message: dict = None - """ - the "system" prompt part - """ - messages: list = None - """ - The list of messages for prompting. - including: the required format (in prompt folder), the last response as assistant prompt - """ - - VALID_API_VERSIONS = ["2022-12-01", "2023-05-15"] - - def __init__( - self, credentials, prompt_load_order, use_azure=False, api_version="2023-05-15" - ): - # initialize the openai api - self.use_azure = use_azure - if self.use_azure: - openai.api_key = credentials["azureopenai"]["AZURE_OPENAI_KEY"] - openai.api_base = credentials["azureopenai"]["AZURE_OPENAI_ENDPOINT"] - openai.api_type = "azure" - if api_version not in self.VALID_API_VERSIONS: - raise ValueError( - f"api_version must be one of {self.VALID_API_VERSIONS}" - ) - openai.api_version = api_version - else: - openai.organization = credentials["openai"]["YOUR_ORG_ID"] - openai.api_key = credentials["openai"]["OPENAI_API_KEY"] - - self.credentials = credentials - self.messages = [] - self.max_token_length = 25000 - self.max_completion_length = 4000 - self.last_response = None - self.query = "" - self.instruction = "" - # load prompt file - # * system - fp_system = os.path.join(dir_system, "system.txt") - with open(fp_system) as f: - data = f.read() - self.system_message = {"role": "system", "content": data} - - # load prompt file - for prompt_name in prompt_load_order: - fp_prompt = os.path.join(dir_prompt, prompt_name + ".txt") - with open(fp_prompt) as f: - data = f.read() - data_spilit = re.split(r"\[user\]\n|\[assistant\]\n", data) - data_spilit = [item for item in data_spilit if len(item) != 0] - # it start with user and ends with system - assert len(data_spilit) % 2 == 0 - for i, item in enumerate(data_spilit): - if i % 2 == 0: - self.messages.append({"sender": "user", "text": item}) - else: - self.messages.append({"sender": "assistant", "text": item}) - fp_query = os.path.join(dir_query, "query.txt") - with open(fp_query) as f: - self.query = f.read() - - def create_prompt(self): - """ - create the prompt for gpt api - """ - prompt = "" - if self.use_azure and openai.api_version == "2022-12-01": - prompt = "<|im_start|>system\n" - prompt += self.system_message["content"] - prompt += "\n<|im_end|>\n" - for message in self.messages: - prompt += ( - f"\n<|im_start|>{message['sender']}\n{message['text']}\n<|im_end|>" - ) - prompt += "\n<|im_start|>assistant\n" - print("prompt length: " + str(len(enc.encode(prompt)))) - if ( - len(enc.encode(prompt)) - > self.max_token_length - self.max_completion_length - ): - print("prompt too long. truncated.") - # * truncate the prompt by removing the oldest two messages - self.messages = self.messages[2:] - prompt = self.create_prompt() - else: - prompt = [] # list of prompt entries - # load the system prompt - prompt.append(self.system_message) - # load the knowledge and the last response - for message in self.messages: - prompt.append({"role": message["sender"], "content": message["text"]}) - - # * truncate the prompt if it is too long - prompt_content = "" # this is only for checking the length of the prompt - for message in prompt: - prompt_content += message["content"] - print("prompt length: " + str(len(enc.encode(prompt_content)))) - if ( - len(enc.encode(prompt_content)) - > self.max_token_length - self.max_completion_length - ): - print("prompt too long. truncated.") - # * truncate the prompt by removing the oldest two messages - self.messages = self.messages[2:] - prompt = self.create_prompt() - return prompt - - def extract_json_part(self, text): - """ - extract the markdown code block part from the text - """ - if text.find("```") == -1: - return text - text_json = text[text.find("```") + 3 : text.find("```", text.find("```") + 3)] - return text_json - - def generate(self, runtime_instruction, environment, is_user_feedback=False): - """ - runtime_instruction: the instruction from the problem/user - """ - if is_user_feedback: - self.messages.append( - { - "sender": "user", - "text": runtime_instruction + "\n" + self.instruction, - } - ) - else: - # get the query template - text_base = self.query - - # # * bb - # if text_base.find("[PROBLEM]") != -1: - # text_base = text_base.replace("[PROBLEM]", json.dumps(environment)) - - # replace the environment part of the template - if text_base.find("[ENVIRONMENT]") != -1: - text_base = text_base.replace("[ENVIRONMENT]", json.dumps(environment)) - # replace the instruction part of the template - if text_base.find("[INSTRUCTION]") != -1: - text_base = text_base.replace("[INSTRUCTION]", runtime_instruction) - # update the instruction of the chatbot - self.instruction = text_base - # finally, add the text_base to the user query message - self.messages.append({"sender": "user", "text": text_base}) - - if self.use_azure and openai.api_version == "2022-12-01": - # Remove unsafe user inputs. May need further refinement in the - # future. - if runtime_instruction.find("<|im_start|>") != -1: - runtime_instruction = runtime_instruction.replace("<|im_start|>", "") - if runtime_instruction.find("<|im_end|>") != -1: - runtime_instruction = runtime_instruction.replace("<|im_end|>", "") - deployment_name = self.credentials["azureopenai"][ - "AZURE_OPENAI_DEPLOYMENT_NAME_CHATGPT" - ] - response = openai.Completion.create( - engine=deployment_name, - prompt=self.create_prompt(), - temperature=0.1, - max_tokens=self.max_completion_length, - top_p=0.5, - frequency_penalty=0.0, - presence_penalty=0.0, - stop=["<|im_end|>"], - ) - text = response["choices"][0]["text"] - elif self.use_azure and openai.api_version == "2023-05-15": - deployment_name = self.credentials["azureopenai"][ - "AZURE_OPENAI_DEPLOYMENT_NAME_CHATGPT" - ] - response = openai.ChatCompletion.create( - engine=deployment_name, - messages=self.create_prompt(), - temperature=0.1, - max_tokens=self.max_completion_length, - top_p=0.5, - frequency_penalty=0.0, - presence_penalty=0.0, - ) - text = response["choices"][0]["message"]["content"] - else: - # * request the gpt to response - response = openai.ChatCompletion.create( - model="gpt-3.5-turbo-16k", - # "gpt-4" is available, too. Check the available models in https://platform.openai.com/docs/models/ - messages=self.create_prompt(), - temperature=0.1, - max_tokens=self.max_completion_length, - top_p=0.5, - frequency_penalty=0.0, - presence_penalty=0.0, - ) - text = response["choices"][0].message.content - print(text) - self.last_response = text - self.last_response = self.extract_json_part(self.last_response) - self.last_response = self.last_response.replace("'", '"') - # dump to a text file - with open("last_response.txt", "w") as f: - f.write(self.last_response) - try: - self.json_dict = json.loads(self.last_response, strict=False) - # update the environment after the execution of the action - self.environment = self.json_dict["environment_after"] - except BaseException: - self.json_dict = None - import pdb - - pdb.set_trace() - - # * use the last response as the assistant prompt for the next round - if len(self.messages) > 0 and self.last_response is not None: - self.messages.append({"sender": "assistant", "text": self.last_response}) - - return self.json_dict - - def generate_with_problem(self, problem, is_user_feedback=False): - """ - runtime_instruction: the instruction from the problem/user - """ - if is_user_feedback: - pass - else: - # get the query template - text_base = self.query - - # * bb - if text_base.find("[PROBLEM]") != -1: - text_base = text_base.replace("[PROBLEM]", json.dumps(problem)) - - # finally, add the text_base to the user query message - self.messages.append({"sender": "user", "text": text_base}) - - # * request the gpt to response - response = openai.chat.completions.create( - # model="gpt-3.5-turbo-16k-0613", - # model="gpt-4-0613", # not this - model="gpt-4-1106-preview", - # "gpt-4" is available, too. Check the available models in https://platform.openai.com/docs/models/ - messages=self.create_prompt(), - temperature=0.1, - max_tokens=self.max_completion_length, - top_p=0.3, - frequency_penalty=0.0, - presence_penalty=0.3, - ) - text = response.choices[0].message.content - - print(text) - self.last_response = text - self.last_response = self.extract_json_part(self.last_response) - self.last_response = self.last_response.replace("'", '"') - # dump to a text file - with open("last_response.txt", "w") as f: - f.write(self.last_response) - try: - self.json_dict = json.loads(self.last_response, strict=False) - # update the environment after the execution of the action - self.environment = self.json_dict["environment_after"] - except json.JSONDecodeError as e: - print("Error decoding JSON:", e) - print("Problematic part:", self.last_response[e.pos - 10 : e.pos + 10]) - - # self.json_dict = None - # import pdb - - # pdb.set_trace() - - # * use the last response as the assistant prompt for the next round - if len(self.messages) > 0 and self.last_response is not None: - self.messages.append({"sender": "assistant", "text": self.last_response}) - - return self.json_dict - - def dump_json(self, dump_name=None): - """ - dump the json dictionary to a file, used to save into a json file - """ - if dump_name is not None: - # dump the dictionary to json file dump 1, 2, ... - fp = os.path.join(dump_name + ".json") - with open(fp, "w") as f: - json.dump(self.json_dict, f, indent=4) - - -if __name__ == "__main__": - parser = argparse.ArgumentParser() - parser.add_argument( - "--scenario", - type=str, - required=False, # ! CHEAT - help="scenario name (see the code for details)", - ) - args = parser.parse_args() - scenario_name = args.scenario - - # ! CHEAT - scenario_name = "gearset_1" - problem = None - - if scenario_name == "chair_assembly_bt": - problem = "(define (problem robot_assembly_problem-problem)\ - (:domain robot_assembly_problem-domain)\ - (:objects\ - parallel_box1 parallel_box2 inward_claw outward_claw no_tool - tool\ - gear1 gear2 gear3 shaft1 shaft2 base - part\ - left_hand - hand\ - )\ - (:init (can_manipulate parallel_box1 gear1) (can_manipulate outward_claw gear2) (can_manipulate inward_claw gear3) (can_manipulate parallel_box2 shaft1) (can_manipulate no_tool shaft2) (can_screw_to leg1 seat) (can_screw_to leg2 seat) (can_insert_to back seat) (can_screw_to nut1 seat) (can_screw_to nut2 seat) (can_screw_to blub base) (can_place_to lamp blub) (can_insert_to shaft1 base) (can_insert_to shaft2 base) (can_insert_to gear3 shaft2) (can_insert_to gear2 base) (can_insert_to gear1 shaft1) (is_inserted_to shaft2 base) (is_free left_hand) (is_free parallel_box1) (is_free parallel_box2) (is_free inward_claw) (is_free outward_claw) (is_free no_tool) (is_free gear1) (is_free gear2) (is_free gear3) (is_free shaft1) (is_free shaft2) (is_free base) (is_equippable parallel_box1) (is_equippable parallel_box2) (is_equippable inward_claw) (is_equippable outward_claw) (is_equippable no_tool))\ - (:goal (and (is_inserted_to gear3 shaft2)))\ - )\ - " - if scenario_name == "gearset_1": - problem = "(define (problem robot_assembly_problem-problem)\ - (:domain robot_assembly_problem-domain)\ - (:objects\ - parallel_box1 parallel_box2 inward_claw outward_claw no_tool - tool\ - gear1 gear2 gear3 shaft1 shaft2 base - part\ - left_hand - hand\ - )\ - (:init (can_manipulate parallel_box1 gear1) (can_manipulate outward_claw gear2) (can_manipulate inward_claw gear3) (can_manipulate parallel_box2 shaft1) (can_manipulate no_tool shaft2) (can_screw_to leg1 seat) (can_screw_to leg2 seat) (can_insert_to back seat) (can_screw_to nut1 seat) (can_screw_to nut2 seat) (can_screw_to blub base) (can_place_to lamp blub) (can_insert_to shaft1 base) (can_insert_to shaft2 base) (can_insert_to gear3 shaft2) (can_insert_to gear2 base) (can_insert_to gear1 shaft1) (is_inserted_to shaft1 base) (is_free left_hand) (is_free parallel_box1) (is_free parallel_box2) (is_free inward_claw) (is_free outward_claw) (is_free no_tool) (is_free gear1) (is_free gear2) (is_free gear3) (is_free shaft1) (is_free shaft2) (is_free base) (is_equippable parallel_box1) (is_equippable parallel_box2) (is_equippable inward_claw) (is_equippable outward_claw) (is_equippable no_tool))\ - (:goal (and (is_inserted_to gear1 shaft1)))\ - )" # ! CHEAT - - # formatted_problem = textwrap.dedent(problem) - # print(formatted_problem) - else: - parser.error("Invalid scenario name:" + scenario_name) - - aimodel = ChatGPT( - credentials, - prompt_load_order=prompt_load_order, - use_azure=False, - api_version="2023-05-15", - ) - - if not os.path.exists("./out/" + scenario_name): - os.makedirs("./out/" + scenario_name) - # run the instructions one by one - text = aimodel.generate_with_problem(problem, is_user_feedback=False) - while True: - user_feedback = input("user feedback (return empty if satisfied): ") - if user_feedback == "q": - exit() - if user_feedback != "": - text = aimodel.generate(user_feedback, environment, is_user_feedback=True) - else: - # update the current environment - environment = aimodel.environment - break - aimodel.dump_json(f"./out/{scenario_name}/0") diff --git a/kios_bt_planning/kios_agent/kios_llm_bt/last_response.txt b/kios_bt_planning/kios_agent/kios_llm_bt/last_response.txt deleted file mode 100644 index 0ecc0ea4..00000000 --- a/kios_bt_planning/kios_agent/kios_llm_bt/last_response.txt +++ /dev/null @@ -1,273 +0,0 @@ - -{ - "task_cohesion": { - "action_sequence": [ - "load_tool(left_hand, inward_claw)", - "pickup(left_hand, gear3, inward_claw)", - "insert(left_hand, inward_claw, gear3, shaft2)" - ], - "action_instructions": [ - "Equip the inward_claw on the left hand.", - "Pick up the gear3 with the left hand using the inward_claw.", - "Insert the gear3 into the shaft2 using the left hand equipped with the inward_claw." - ], - "manipulated_objects": ["left_hand", "gear3", "inward_claw", "shaft2"], - "behavior_tree": { - "name": "insert gear3 into shaft2 selector", - "identifier": 0, - "type_name": "selector", - "children": [ - { - "summary": "check if gear3 is inserted to shaft2", - "name": "check gear3 is_inserted_to shaft2", - "identifier": 1, - "type_name": "condition", - "conditions": [ - { - "object_name": "gear3", - "property_name": "is_inserted_to", - "property_value": "shaft2", - "status": true - } - ] - }, - { - "name": "insert gear3 into shaft2 sequence", - "identifier": 2, - "type_name": "sequence", - "children": [ - { - "name": "load_tool selector", - "identifier": 3, - "type_name": "selector", - "children": [ - { - "summary": "check if inward_claw is held by left hand", - "name": "check left_hand hold inward_claw", - "identifier": 4, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "inward_claw", - "status": true - } - ] - }, - { - "name": "load_tool sequence", - "identifier": 5, - "type_name": "sequence", - "children": [ - { - "summary": "check if inward_claw is equippable", - "name": "check inward_claw is_equippable", - "identifier": 6, - "type_name": "condition", - "conditions": [ - { - "object_name": "inward_claw", - "property_name": "is_equippable", - "property_value": null, - "status": true - } - ] - }, - { - "summary": "check if left hand is free", - "name": "check left_hand is_free", - "identifier": 7, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "is_free", - "property_value": null, - "status": true - } - ] - }, - { - "summary": "equip inward_claw to left hand", - "name": "load_tool(left_hand, inward_claw)", - "identifier": 8, - "type_name": "action", - "effects": [ - { - "object_name": "left_hand", - "property_name": "is_free", - "property_value": null, - "status": false - }, - { - "object_name": "inward_claw", - "property_name": "is_equippable", - "property_value": null, - "status": false - }, - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "inward_claw", - "status": true - } - ] - } - ] - } - ] - }, - { - "name": "pickup selector", - "identifier": 9, - "type_name": "selector", - "children": [ - { - "summary": "check if inward_claw holds gear3", - "name": "check inward_claw hold gear3", - "identifier": 10, - "type_name": "condition", - "conditions": [ - { - "object_name": "inward_claw", - "property_name": "hold", - "property_value": "gear3", - "status": true - } - ] - }, - { - "name": "pickup sequence", - "identifier": 11, - "type_name": "sequence", - "children": [ - { - "summary": "check if inward_claw is free", - "name": "check inward_claw is_free", - "identifier": 12, - "type_name": "condition", - "conditions": [ - { - "object_name": "inward_claw", - "property_name": "is_free", - "property_value": null, - "status": true - } - ] - }, - { - "summary": "check if inward_claw can manipulate gear3", - "name": "check inward_claw can_manipulate gear3", - "identifier": 13, - "type_name": "condition", - "conditions": [ - { - "object_name": "inward_claw", - "property_name": "can_manipulate", - "property_value": "gear3", - "status": true - } - ] - }, - { - "summary": "pick up gear3 using inward_claw", - "name": "pickup(left_hand, gear3, inward_claw)", - "identifier": 14, - "type_name": "action", - "effects": [ - { - "object_name": "inward_claw", - "property_name": "hold", - "property_value": "gear3", - "status": true - }, - { - "object_name": "inward_claw", - "property_name": "is_free", - "property_value": null, - "status": false - } - ] - } - ] - } - ] - }, - { - "summary": "check if gear3 can be inserted to shaft2", - "name": "check gear3 can_insert_to shaft2", - "identifier": 15, - "type_name": "condition", - "conditions": [ - { - "object_name": "gear3", - "property_name": "can_insert_to", - "property_value": "shaft2", - "status": true - } - ] - }, - { - "summary": "insert gear3 to shaft2", - "name": "insert(left_hand, inward_claw, gear3, shaft2)", - "identifier": 16, - "type_name": "action", - "effects": [ - { - "object_name": "inward_claw", - "property_name": "hold", - "property_value": "gear3", - "status": false - }, - { - "object_name": "inward_claw", - "property_name": "is_free", - "property_value": null, - "status": true - }, - { - "object_name": "gear3", - "property_name": "is_inserted_to", - "property_value": "shaft2", - "status": true - } - ] - } - ] - } - ] - } - }, - "environment_before": { - "objects": [ - {"name": "gear3", "properties": ["is_free"]}, - {"name": "shaft2", "properties": ["is_free"]}, - {"name": "inward_claw", "properties": ["is_equippable", "is_free"]}, - {"name": "left_hand", "properties": ["is_free"]} - ], - "constraints": [ - {"source": "gear3", "name": "can_insert_to", "target": "shaft2"}, - {"source": "inward_claw", "name": "can_manipulate", "target": "gear3"} - ], - "relations": [] - }, - "environment_after": { - "objects": [ - {"name": "gear3", "properties": []}, - {"name": "shaft2", "properties": []}, - {"name": "inward_claw", "properties": ["is_equippable", "is_free"]}, - {"name": "left_hand", "properties": []} - ], - "constraints": [ - {"source": "gear3", "name": "can_insert_to", "target": "shaft2"}, - {"source": "inward_claw", "name": "can_manipulate", "target": "gear3"} - ], - "relations": [ - {"source": "gear3", "name": "is_inserted_to", "target": "shaft2"}, - {"source": "left_hand", "name": "hold", "target": "inward_claw"} - ] - }, - "instruction_summary": "insert gear3 into shaft2", - "question": "" -} diff --git a/kios_bt_planning/kios_agent/kios_llm_bt/out/chair_assembly/0.json b/kios_bt_planning/kios_agent/kios_llm_bt/out/chair_assembly/0.json deleted file mode 100644 index 3ccb8747..00000000 --- a/kios_bt_planning/kios_agent/kios_llm_bt/out/chair_assembly/0.json +++ /dev/null @@ -1,93 +0,0 @@ -{ - "task_cohesion": { - "task_sequence": [ - "move_hand()", - "grasp_object()", - "detach_from_plane()", - "move_object()", - "attach_to_plane()", - "screw()", - "release_object()" - ], - "step_instructions": [ - "move the hand near leg1", - "grasp leg1", - "pick up leg1", - "bring leg1 near the seat", - "attach leg1 to the seat", - "screw leg1 into the seat", - "release leg1" - ], - "object_name": "" - }, - "environment_before": { - "assets": [ - "
" - ], - "asset_states": { - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "can_manipulate(, )", - "": "can_manipulate(, )" - }, - "objects": [ - "", - "", - "", - "", - "", - "", - "", - "", - "" - ], - "object_states": { - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)" - } - }, - "environment_after": { - "assets": [ - "
" - ], - "asset_states": { - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "can_manipulate(, )", - "": "can_manipulate(, )" - }, - "objects": [ - "", - "", - "", - "", - "", - "", - "", - "", - "" - ], - "object_states": { - "": "screwed_into()", - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)" - } - }, - "instruction_summary": "screw leg1 to the seat", - "question": "" -} \ No newline at end of file diff --git a/kios_bt_planning/kios_agent/kios_llm_bt/out/chair_assembly/1.json b/kios_bt_planning/kios_agent/kios_llm_bt/out/chair_assembly/1.json deleted file mode 100644 index 2a19da24..00000000 --- a/kios_bt_planning/kios_agent/kios_llm_bt/out/chair_assembly/1.json +++ /dev/null @@ -1,93 +0,0 @@ -{ - "task_cohesion": { - "task_sequence": [ - "move_hand()", - "grasp_object()", - "detach_from_plane()", - "move_object()", - "attach_to_plane()", - "screw()", - "release_object()" - ], - "step_instructions": [ - "move the hand near leg2", - "grasp leg2", - "pick up leg2", - "bring leg2 near the seat", - "attach leg2 to the seat", - "screw leg2 into the seat", - "release leg2" - ], - "object_name": "" - }, - "environment_before": { - "assets": [ - "
" - ], - "asset_states": { - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "can_manipulate(, )", - "": "can_manipulate(, )" - }, - "objects": [ - "", - "", - "", - "", - "", - "", - "", - "", - "" - ], - "object_states": { - "": "screwed_into()", - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)" - } - }, - "environment_after": { - "assets": [ - "
" - ], - "asset_states": { - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "can_manipulate(, )", - "": "can_manipulate(, )" - }, - "objects": [ - "", - "", - "", - "", - "", - "", - "", - "", - "" - ], - "object_states": { - "": "screwed_into()", - "": "screwed_into()", - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)" - } - }, - "instruction_summary": "screw leg2 to the seat", - "question": "" -} \ No newline at end of file diff --git a/kios_bt_planning/kios_agent/kios_llm_bt/out/chair_assembly/2.json b/kios_bt_planning/kios_agent/kios_llm_bt/out/chair_assembly/2.json deleted file mode 100644 index c09e717f..00000000 --- a/kios_bt_planning/kios_agent/kios_llm_bt/out/chair_assembly/2.json +++ /dev/null @@ -1,93 +0,0 @@ -{ - "task_cohesion": { - "task_sequence": [ - "move_hand()", - "grasp_object()", - "detach_from_plane()", - "move_object()", - "attach_to_plane()", - "insert()", - "release_object()" - ], - "step_instructions": [ - "move the hand near the seat", - "grasp the seat", - "pick up the seat", - "bring the seat near the back", - "attach the seat to the back", - "insert the seat into the back", - "release the seat" - ], - "object_name": "" - }, - "environment_before": { - "assets": [ - "
" - ], - "asset_states": { - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "can_manipulate(, )", - "": "can_manipulate(, )" - }, - "objects": [ - "", - "", - "", - "", - "", - "", - "", - "", - "" - ], - "object_states": { - "": "screwed_into()", - "": "screwed_into()", - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)" - } - }, - "environment_after": { - "assets": [ - "
" - ], - "asset_states": { - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "inside_something()", - "": "on_something(
)", - "": "on_something(
)", - "": "can_manipulate(, )", - "": "can_manipulate(, )" - }, - "objects": [ - "", - "", - "", - "", - "", - "", - "", - "", - "" - ], - "object_states": { - "": "screwed_into()", - "": "screwed_into()", - "": "on_something(
)", - "": "inside_something()", - "": "on_something(
)", - "": "on_something(
)" - } - }, - "instruction_summary": "insert the seat to the back", - "question": "" -} \ No newline at end of file diff --git a/kios_bt_planning/kios_agent/kios_llm_bt/out/chair_assembly/3.json b/kios_bt_planning/kios_agent/kios_llm_bt/out/chair_assembly/3.json deleted file mode 100644 index 0a948eef..00000000 --- a/kios_bt_planning/kios_agent/kios_llm_bt/out/chair_assembly/3.json +++ /dev/null @@ -1,93 +0,0 @@ -{ - "task_cohesion": { - "task_sequence": [ - "move_hand()", - "grasp_object()", - "detach_from_plane()", - "move_object()", - "attach_to_plane()", - "screw()", - "release_object()" - ], - "step_instructions": [ - "move the hand near nut1", - "grasp nut1", - "pick up nut1", - "bring nut1 near the seat", - "attach nut1 to the seat", - "screw nut1 into the seat", - "release nut1" - ], - "object_name": "" - }, - "environment_before": { - "assets": [ - "
" - ], - "asset_states": { - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "inside_something()", - "": "on_something(
)", - "": "on_something(
)", - "": "can_manipulate(, )", - "": "can_manipulate(, )" - }, - "objects": [ - "", - "", - "", - "", - "", - "", - "", - "", - "" - ], - "object_states": { - "": "screwed_into()", - "": "screwed_into()", - "": "on_something(
)", - "": "inside_something()", - "": "on_something(
)", - "": "on_something(
)" - } - }, - "environment_after": { - "assets": [ - "
" - ], - "asset_states": { - "": "on_something(
)", - "": "on_something(
)", - "": "on_something(
)", - "": "inside_something()", - "": "screwed_into()", - "": "on_something(
)", - "": "can_manipulate(, )", - "": "can_manipulate(, )" - }, - "objects": [ - "", - "", - "", - "", - "", - "", - "", - "", - "" - ], - "object_states": { - "": "screwed_into()", - "": "screwed_into()", - "": "on_something(
)", - "": "inside_something()", - "": "screwed_into()", - "": "on_something(
)" - } - }, - "instruction_summary": "screw nut1 to the seat", - "question": "" -} \ No newline at end of file diff --git a/kios_bt_planning/kios_agent/kios_llm_bt/out/drawer/0.json b/kios_bt_planning/kios_agent/kios_llm_bt/out/drawer/0.json deleted file mode 100644 index 3c6551a8..00000000 --- a/kios_bt_planning/kios_agent/kios_llm_bt/out/drawer/0.json +++ /dev/null @@ -1,49 +0,0 @@ -{ - "task_cohesion": { - "task_sequence": [ - "move_hand()", - "grasp_object()", - "open_by_slide()", - "release_object()" - ], - "step_instructions": [ - "move the hand near the drawer handle", - "grasp the drawer handle", - "pull the drawer handle to open the drawer", - "release the drawer handle" - ], - "object_name": "" - }, - "environment_before": { - "assets": [ - "", - "" - ], - "asset_states": { - "": "on_something()" - }, - "objects": [ - "" - ], - "object_states": { - "": "closed()" - } - }, - "environment_after": { - "assets": [ - "", - "" - ], - "asset_states": { - "": "on_something()" - }, - "objects": [ - "" - ], - "object_states": { - "": "open()" - } - }, - "instruction_summary": "open the drawer widely", - "question": "" -} \ No newline at end of file diff --git a/kios_bt_planning/kios_agent/kios_llm_bt/out/drawer/1.json b/kios_bt_planning/kios_agent/kios_llm_bt/out/drawer/1.json deleted file mode 100644 index 3d8e80c5..00000000 --- a/kios_bt_planning/kios_agent/kios_llm_bt/out/drawer/1.json +++ /dev/null @@ -1,49 +0,0 @@ -{ - "task_cohesion": { - "task_sequence": [ - "move_hand()", - "grasp_object()", - "adjust_by_slide()", - "release_object()" - ], - "step_instructions": [ - "move the hand near the drawer handle", - "grasp the drawer handle", - "move the drawer handle to close the drawer halfway", - "release the drawer handle" - ], - "object_name": "" - }, - "environment_before": { - "assets": [ - "", - "" - ], - "asset_states": { - "": "on_something()" - }, - "objects": [ - "" - ], - "object_states": { - "": "open()" - } - }, - "environment_after": { - "assets": [ - "", - "" - ], - "asset_states": { - "": "on_something()" - }, - "objects": [ - "" - ], - "object_states": { - "": "open()" - } - }, - "instruction_summary": "close the drawer halfway", - "question": "" -} \ No newline at end of file diff --git a/kios_bt_planning/kios_agent/kios_llm_bt/out/drawer/2.json b/kios_bt_planning/kios_agent/kios_llm_bt/out/drawer/2.json deleted file mode 100644 index c18ca676..00000000 --- a/kios_bt_planning/kios_agent/kios_llm_bt/out/drawer/2.json +++ /dev/null @@ -1,49 +0,0 @@ -{ - "task_cohesion": { - "task_sequence": [ - "move_hand()", - "grasp_object()", - "close_by_slide()", - "release_object()" - ], - "step_instructions": [ - "move the hand near the drawer handle", - "grasp the drawer handle", - "move the drawer handle to close the drawer fully", - "release the drawer handle" - ], - "object_name": "" - }, - "environment_before": { - "assets": [ - "", - "" - ], - "asset_states": { - "": "on_something()" - }, - "objects": [ - "" - ], - "object_states": { - "": "open()" - } - }, - "environment_after": { - "assets": [ - "", - "" - ], - "asset_states": { - "": "on_something()" - }, - "objects": [ - "" - ], - "object_states": { - "": "closed()" - } - }, - "instruction_summary": "close the drawer fully", - "question": "" -} \ No newline at end of file diff --git a/kios_bt_planning/kios_agent/kios_llm_bt/out/fridge/0.json b/kios_bt_planning/kios_agent/kios_llm_bt/out/fridge/0.json deleted file mode 100644 index cf4c0d36..00000000 --- a/kios_bt_planning/kios_agent/kios_llm_bt/out/fridge/0.json +++ /dev/null @@ -1,53 +0,0 @@ -{ - "task_cohesion": { - "task_sequence": [ - "move_hand()", - "grasp_object()", - "open_by_rotate()", - "release_object()" - ], - "step_instructions": [ - "move the hand near the fridge handle", - "grasp the handle", - "open the fridge halfway", - "release the handle" - ], - "object_name": "" - }, - "environment_before": { - "assets": [ - "", - "" - ], - "asset_states": { - "": "on_something()" - }, - "objects": [ - "", - "" - ], - "object_states": { - "": "closed()", - "": "inside_something()" - } - }, - "environment_after": { - "assets": [ - "", - "" - ], - "asset_states": { - "": "on_something()" - }, - "objects": [ - "", - "" - ], - "object_states": { - "": "open()", - "": "inside_something()" - } - }, - "instruction_summary": "open the fridge halfway", - "question": "" -} \ No newline at end of file diff --git a/kios_bt_planning/kios_agent/kios_llm_bt/out/fridge/1.json b/kios_bt_planning/kios_agent/kios_llm_bt/out/fridge/1.json deleted file mode 100644 index cb3a4afb..00000000 --- a/kios_bt_planning/kios_agent/kios_llm_bt/out/fridge/1.json +++ /dev/null @@ -1,53 +0,0 @@ -{ - "task_cohesion": { - "task_sequence": [ - "move_hand()", - "grasp_object()", - "adjust_by_rotate()", - "release_object()" - ], - "step_instructions": [ - "move the hand near the fridge handle", - "grasp the handle", - "open the fridge wider", - "release the handle" - ], - "object_name": "" - }, - "environment_before": { - "assets": [ - "", - "" - ], - "asset_states": { - "": "on_something()" - }, - "objects": [ - "", - "" - ], - "object_states": { - "": "open()", - "": "inside_something()" - } - }, - "environment_after": { - "assets": [ - "", - "" - ], - "asset_states": { - "": "on_something()" - }, - "objects": [ - "", - "" - ], - "object_states": { - "": "open()", - "": "inside_something()" - } - }, - "instruction_summary": "open the fridge wider", - "question": "" -} \ No newline at end of file diff --git a/kios_bt_planning/kios_agent/kios_llm_bt/out/fridge/2.json b/kios_bt_planning/kios_agent/kios_llm_bt/out/fridge/2.json deleted file mode 100644 index cc7bdfa2..00000000 --- a/kios_bt_planning/kios_agent/kios_llm_bt/out/fridge/2.json +++ /dev/null @@ -1,61 +0,0 @@ -{ - "task_cohesion": { - "task_sequence": [ - "move_hand()", - "grasp_object()", - "detach_from_plane()", - "move_object()", - "move_hand()", - "move_object()", - "attach_to_plane()", - "release_object()" - ], - "step_instructions": [ - "move the hand near the juice", - "grasp the juice", - "pick up the juice from the top of the fridge", - "move the juice to the floor", - "move the hand near the juice", - "move the juice to the floor", - "place the juice on the floor", - "release the juice" - ], - "object_name": "" - }, - "environment_before": { - "assets": [ - "", - "" - ], - "asset_states": { - "": "on_something()" - }, - "objects": [ - "", - "" - ], - "object_states": { - "": "open()", - "": "inside_something()" - } - }, - "environment_after": { - "assets": [ - "", - "" - ], - "asset_states": { - "": "on_something()" - }, - "objects": [ - "", - "" - ], - "object_states": { - "": "open()", - "": "on_something()" - } - }, - "instruction_summary": "take the juice from the fridge and put it on the floor", - "question": "" -} \ No newline at end of file diff --git a/kios_bt_planning/kios_agent/kios_llm_bt/out/fridge/3.json b/kios_bt_planning/kios_agent/kios_llm_bt/out/fridge/3.json deleted file mode 100644 index 9b483592..00000000 --- a/kios_bt_planning/kios_agent/kios_llm_bt/out/fridge/3.json +++ /dev/null @@ -1,53 +0,0 @@ -{ - "task_cohesion": { - "task_sequence": [ - "move_hand()", - "grasp_object()", - "move_object()", - "release_object()" - ], - "step_instructions": [ - "move the hand near the carton box", - "grasp the carton box", - "move the carton box away from the fridge", - "release the carton box" - ], - "object_name": "" - }, - "environment_before": { - "assets": [ - "", - "" - ], - "asset_states": { - "": "on_something()" - }, - "objects": [ - "", - "" - ], - "object_states": { - "": "open()", - "": "on_something()" - } - }, - "environment_after": { - "assets": [ - "", - "" - ], - "asset_states": { - "": "on_something()" - }, - "objects": [ - "", - "" - ], - "object_states": { - "": "open()", - "": "on_something()" - } - }, - "instruction_summary": "move the carton box away from the fridge", - "question": "" -} \ No newline at end of file diff --git a/kios_bt_planning/kios_agent/kios_llm_bt/out/shelf/0.json b/kios_bt_planning/kios_agent/kios_llm_bt/out/shelf/0.json deleted file mode 100644 index 16bc759e..00000000 --- a/kios_bt_planning/kios_agent/kios_llm_bt/out/shelf/0.json +++ /dev/null @@ -1,67 +0,0 @@ -{ - "task_cohesion": { - "task_sequence": [ - "move_hand()", - "grasp_object()", - "detach_from_plane()", - "move_object()", - "attach_to_plane()", - "open_by_rotate()", - "release_object()" - ], - "step_instructions": [ - "move the hand near the spam", - "grasp the spam", - "pick up the spam", - "bring the spam above the trash bin", - "attach the spam to the trash bin", - "open the spam to check the expiration date", - "release the spam" - ], - "object_name": "" - }, - "environment_before": { - "assets": [ - "
", - "", - "", - "", - "" - ], - "asset_states": { - "": "on_something(
)", - "": "on_something()" - }, - "objects": [ - "", - "" - ], - "object_states": { - "": "on_something(
)", - "": "on_something()" - } - }, - "environment_after": { - "assets": [ - "
", - "", - "", - "", - "" - ], - "asset_states": { - "": "on_something(
)", - "": "on_something()" - }, - "objects": [ - "", - "" - ], - "object_states": { - "": "inside_something()", - "": "on_something()" - } - }, - "instruction_summary": "take the spam and throw it away if the out-of-date date is expired, otherwise put it on the shelf", - "question": "" -} \ No newline at end of file diff --git a/kios_bt_planning/kios_agent/kios_llm_bt/out/shelf/1.json b/kios_bt_planning/kios_agent/kios_llm_bt/out/shelf/1.json deleted file mode 100644 index c26e27b8..00000000 --- a/kios_bt_planning/kios_agent/kios_llm_bt/out/shelf/1.json +++ /dev/null @@ -1,63 +0,0 @@ -{ - "task_cohesion": { - "task_sequence": [ - "move_hand()", - "grasp_object()", - "detach_from_plane()", - "move_object()", - "release_object()" - ], - "step_instructions": [ - "move your hand near the spam", - "grasp the spam", - "pick up the spam", - "bring the spam above the trash bin", - "release the spam to drop it in the trash bin" - ], - "object_name": "" - }, - "environment_before": { - "assets": [ - "
", - "", - "", - "", - "" - ], - "asset_states": { - "": "on_something(
)", - "": "on_something()" - }, - "objects": [ - "", - "" - ], - "object_states": { - "": "on_something(
)", - "": "on_something()" - } - }, - "environment_after": { - "assets": [ - "
", - "", - "", - "", - "" - ], - "asset_states": { - "": "on_something(
)", - "": "on_something()" - }, - "objects": [ - "", - "" - ], - "object_states": { - "": "inside_something()", - "": "on_something()" - } - }, - "instruction_summary": "throw away the spam", - "question": "" -} \ No newline at end of file diff --git a/kios_bt_planning/kios_agent/kios_llm_bt/out/shelf/2.json b/kios_bt_planning/kios_agent/kios_llm_bt/out/shelf/2.json deleted file mode 100644 index 3b00dcb0..00000000 --- a/kios_bt_planning/kios_agent/kios_llm_bt/out/shelf/2.json +++ /dev/null @@ -1,69 +0,0 @@ -{ - "task_cohesion": { - "task_sequence": [ - "move_hand()", - "grasp_object()", - "detach_from_plane()", - "move_object()", - "move_object()", - "move_object()", - "attach_to_plane()", - "release_object()" - ], - "step_instructions": [ - "move the hand near the juice", - "grasp the juice", - "pick up the juice", - "take the juice from the top of the shelf", - "move the juice downward", - "move the juice above the table", - "place the juice", - "release the juice" - ], - "object_name": "" - }, - "environment_before": { - "assets": [ - "
", - "", - "", - "", - "" - ], - "asset_states": { - "": "on_something(
)", - "": "on_something()" - }, - "objects": [ - "", - "" - ], - "object_states": { - "": "inside_something()", - "": "on_something()" - } - }, - "environment_after": { - "assets": [ - "
", - "", - "", - "", - "" - ], - "asset_states": { - "": "on_something(
)", - "": "on_something()" - }, - "objects": [ - "", - "" - ], - "object_states": { - "": "inside_something()", - "": "on_something(
)" - } - }, - "instruction_summary": "move the juice on top of the table", - "question": "" -} \ No newline at end of file diff --git a/kios_bt_planning/kios_agent/kios_llm_bt/out/shelf/3.json b/kios_bt_planning/kios_agent/kios_llm_bt/out/shelf/3.json deleted file mode 100644 index 92a955b4..00000000 --- a/kios_bt_planning/kios_agent/kios_llm_bt/out/shelf/3.json +++ /dev/null @@ -1,63 +0,0 @@ -{ - "task_cohesion": { - "task_sequence": [ - "move_hand()", - "grasp_object()", - "detach_from_plane()", - "move_object()", - "release_object()" - ], - "step_instructions": [ - "move your hand near the juice", - "grasp the juice", - "pick up the juice", - "bring the juice above the trash bin", - "release the juice to drop it in the trash bin" - ], - "object_name": "" - }, - "environment_before": { - "assets": [ - "
", - "", - "", - "", - "" - ], - "asset_states": { - "": "on_something(
)", - "": "on_something()" - }, - "objects": [ - "", - "" - ], - "object_states": { - "": "inside_something()", - "": "on_something(
)" - } - }, - "environment_after": { - "assets": [ - "
", - "", - "", - "", - "" - ], - "asset_states": { - "": "on_something(
)", - "": "on_something()" - }, - "objects": [ - "", - "" - ], - "object_states": { - "": "inside_something()", - "": "inside_something()" - } - }, - "instruction_summary": "throw away the juice", - "question": "" -} \ No newline at end of file diff --git a/kios_bt_planning/kios_agent/kios_llm_bt/out/table/0.json b/kios_bt_planning/kios_agent/kios_llm_bt/out/table/0.json deleted file mode 100644 index ddf8b608..00000000 --- a/kios_bt_planning/kios_agent/kios_llm_bt/out/table/0.json +++ /dev/null @@ -1,61 +0,0 @@ -{ - "task_cohesion": { - "task_sequence": [ - "move_hand()", - "grasp_object()", - "detach_from_plane()", - "move_object()", - "move_object()", - "attach_to_plane()", - "release_object()" - ], - "step_instructions": [ - "move the hand near the sponge", - "grasp the sponge", - "pick up the sponge", - "take the sponge from the first table", - "move the sponge to the second table", - "place the sponge on the second table", - "release the sponge" - ], - "object_name": "" - }, - "environment_before": { - "assets": [ - "", - "", - "", - "" - ], - "asset_states": { - "": "next_to()", - "": "on_something()" - }, - "objects": [ - "" - ], - "object_states": { - "": "on_something()" - } - }, - "environment_after": { - "assets": [ - "", - "", - "", - "" - ], - "asset_states": { - "": "next_to()", - "": "on_something()" - }, - "objects": [ - "" - ], - "object_states": { - "": "on_something()" - } - }, - "instruction_summary": "put the sponge on the table2", - "question": "" -} \ No newline at end of file diff --git a/kios_bt_planning/kios_agent/kios_llm_bt/out/table/1.json b/kios_bt_planning/kios_agent/kios_llm_bt/out/table/1.json deleted file mode 100644 index 4b1f7029..00000000 --- a/kios_bt_planning/kios_agent/kios_llm_bt/out/table/1.json +++ /dev/null @@ -1,55 +0,0 @@ -{ - "task_cohesion": { - "task_sequence": [ - "move_hand()", - "grasp_object()", - "wipe_on_plane()", - "release_object()" - ], - "step_instructions": [ - "move the hand near the sponge on the second table", - "grasp the sponge", - "wipe the second table with the sponge", - "release the sponge" - ], - "object_name": "" - }, - "environment_before": { - "assets": [ - "", - "", - "", - "" - ], - "asset_states": { - "": "next_to()", - "": "on_something()" - }, - "objects": [ - "" - ], - "object_states": { - "": "on_something()" - } - }, - "environment_after": { - "assets": [ - "", - "", - "", - "" - ], - "asset_states": { - "": "next_to()", - "": "on_something()" - }, - "objects": [ - "" - ], - "object_states": { - "": "on_something()" - } - }, - "instruction_summary": "wipe the table2 with the sponge", - "question": "" -} \ No newline at end of file diff --git a/kios_bt_planning/kios_agent/kios_llm_bt/out/window/0.json b/kios_bt_planning/kios_agent/kios_llm_bt/out/window/0.json deleted file mode 100644 index e8172cb3..00000000 --- a/kios_bt_planning/kios_agent/kios_llm_bt/out/window/0.json +++ /dev/null @@ -1,67 +0,0 @@ -{ - "task_cohesion": { - "task_sequence": [ - "move_hand()", - "grasp_object()", - "detach_from_plane()", - "move_object()", - "attach_to_plane()", - "wipe_on_plane()", - "detach_from_plane()", - "move_object()", - "attach_to_plane()", - "release_object()" - ], - "step_instructions": [ - "move the hand near the sponge", - "grasp the sponge", - "pick up the sponge", - "bring the sponge above the window", - "attach the sponge to the window", - "wipe the window with the sponge", - "detach the sponge from the window", - "bring the sponge above the table", - "put down the sponge", - "release the sponge" - ], - "object_name": "" - }, - "environment_before": { - "assets": [ - "
", - "", - "", - "" - ], - "asset_states": { - "
": "next_to()", - "": "on_something()" - }, - "objects": [ - "" - ], - "object_states": { - "": "on_something(
)" - } - }, - "environment_after": { - "assets": [ - "
", - "", - "", - "" - ], - "asset_states": { - "
": "next_to()", - "": "on_something()" - }, - "objects": [ - "" - ], - "object_states": { - "": "on_something(
)" - } - }, - "instruction_summary": "get the sponge from the table, wipe the window with it, and put the sponge back on the table", - "question": "" -} \ No newline at end of file diff --git a/kios_bt_planning/kios_agent/kios_llm_bt/out/window/1.json b/kios_bt_planning/kios_agent/kios_llm_bt/out/window/1.json deleted file mode 100644 index 6b660c25..00000000 --- a/kios_bt_planning/kios_agent/kios_llm_bt/out/window/1.json +++ /dev/null @@ -1,57 +0,0 @@ -{ - "task_cohesion": { - "task_sequence": [ - "move_hand()", - "grasp_object()", - "detach_from_plane()", - "move_object()", - "release_object()" - ], - "step_instructions": [ - "move the hand near the sponge", - "grasp the sponge", - "pick up the sponge", - "bring the sponge above the trash bin", - "release the sponge and drop it in the trash bin" - ], - "object_name": "" - }, - "environment_before": { - "assets": [ - "
", - "", - "", - "" - ], - "asset_states": { - "
": "next_to()", - "": "on_something()" - }, - "objects": [ - "" - ], - "object_states": { - "": "on_something(
)" - } - }, - "environment_after": { - "assets": [ - "
", - "", - "", - "" - ], - "asset_states": { - "
": "next_to()", - "": "on_something()" - }, - "objects": [ - "" - ], - "object_states": { - "": "inside_something()" - } - }, - "instruction_summary": "throw away the sponge", - "question": "" -} \ No newline at end of file diff --git a/kios_bt_planning/kios_agent/kios_llm_bt/prompt/old_prompt_example.txt b/kios_bt_planning/kios_agent/kios_llm_bt/prompt/old_prompt_example.txt deleted file mode 100644 index 8d71b9b6..00000000 --- a/kios_bt_planning/kios_agent/kios_llm_bt/prompt/old_prompt_example.txt +++ /dev/null @@ -1,232 +0,0 @@ -[user] -I will give you some examples of the input and the output you will generate. -Example 1: -""" -- Input: -{"assets":["
", "", "", "", ""], -"asset_states": {"": "on_something(
)", - "": "on_something()"}, -"objects": ["", ""], -"object_states": {"": "on_something(
)", - "": "on_something()"}, -"instruction": "Put the juice on top of the shelf"} -- Output: -``` -{"task_cohesion": { - "task_sequence": [ - "move_hand()", - "grasp_object()", - "detach_from_plane()", - "move_object()", - "move_object()", - "move_object()", - "attach_to_plane()", - "release_object()" - ], - "step_instructions": [ - "move the hand near the juice", - "grasp the juice", - "pick up the juice", - "take the juice from the bottom of the shelf", - "move the juice upward", - "move the juice above the upper shelf", - "place the juice", - "release the juice" - ], - "object_name": ""}, -"environment_before": {"assets":["
", "", "", "", ""], - "asset_states": {"": "on_something(
)", - "": "on_something()"}, - "objects": ["", ""], - "object_states": {"": "on_something(
)", - "": "on_something()"}}, -"environment_after": {"assets":["
", "", "", "", ""], - "asset_states": {"": "on_something(
)", - "": "on_something()"}, - "objects": ["", ""], - "object_states": {"": "on_something(
)", - "": "on_something()"}}, -"instruction_summary": "put the juice on top of the shelf", -"question":""} -``` -""" -Example 2: -""" -- Input: -{"assets":["
", "", "", "", ""], -"asset_states": {"": "on_something(
)", - "": "on_something()"}, -"objects": ["", ""], -"object_states": {"": "on_something(
)", - "": "on_something()"}, -"instruction": "Throw away the spam into the trash bin."} -- Output: -``` -{"task_cohesion": { - "task_sequence": [ - "move_hand()", - "grasp_object()", - "detach_from_plane()", - "move_object()", - "release_object()" - ], - "step_instructions": [ - "move your hand near the spam", - "grasp the spam", - "pick up the spam", - "bring the spam above the trash bin", - "release the spam to drop it in the trash bin" - ], - "object_name": ""}, -"environment_before": {"assets":["
", "", "", "", ""], - "asset_states": {"": "on_something(
)", - "": "on_something()"}, - "objects": ["", ""], - "object_states": {"": "on_something(
)", - "": "on_something()"}}, -"environment_after": {"assets":["
", "", "", "", ""], - "asset_states": {"": "on_something(
)", - "": "on_something()"}, - "objects": ["", ""], - "object_states": {"": "inside_something()", - "": "on_something()"}}, -"instruction_summary": "throw away the spam", -"question":""} -``` -""" -Example 3: -""" --Input: -{"assets":["", ""], -"asset_states": {"": "on_something()"}, -"objects": [""], -"object_states": {"": "closed()"}, -"instruction": "Open the fridge."} -- Output: -``` -{"task_cohesion": { - "task_sequence": [ - "move_hand()", - "grasp_object()", - "open_by_rotate()", - "release_object()" - ], - "step_instructions": [ - "move the hand near the fridge handle", - "grasp the handle", - "open the fridge by pulling the handle", - "release the handle" - ], - "object_name": ""}, -"environment_before": {"assets":["", ""], - "asset_states": {"": "on_something()"}, - "objects": [""], - "object_states": {"": "closed()"}}, -"environment_after": {"assets":["", ""], - "asset_states": {"": "on_something()"}, - "objects": [""], - "object_states": {"": "open()"}}, -"instruction_summary": "open a fridge", -"question":""} -``` -""" -Example 4: -""" --Input: -{"assets":["
", ""], -"asset_states": {"
": "next_to()"}, -"objects": [""], -"object_states": {"": "on_something(
)"}, -"instruction": "Wipe the window with the sponge. Then, put the sponge on the table."} -- Output: -``` -{"task_cohesion": { - "task_sequence": [ - "move_hand()", - "grasp_object()", - "detach_from_plane()", - "move_object()", - "attach_to_plane()", - "wipe_on_plane()", - "detach_from_plane()", - "move_object()", - "attach_to_plane()", - "release_object()" - ], - "step_instructions": [ - "move the hand near the sponge", - "grasp the sponge", - "pick up the sponge", - "bring the sponge above the window", - "attach the sponge to the window", - "wipe the window with the sponge", - "detach the sponge from the window", - "bring the sponge above the table", - "put down the sponge", - "release the sponge" - ], - "object_name": ""}, -"environment_before": {"assets":["
", ""], - "asset_states": {"
": "next_to()"}, - "objects": [""], - "object_states": {"": "on_something(
)"},} -"environment_after": {"assets":["
", ""], - "asset_states": {"
": "next_to()"}, - "objects": [""], - "object_states": {"": "on_something(
)"}}, -"instruction_summary": "wipe the window with the sponge, then put the sponge on the table", -"question":""} -``` -""" -Example 5: -""" --Input: -{"assets":["
"], -"asset_states": {"": "can_manipulate()"}, -"objects": ["", "", ""], -"object_states": {"": "on_something(
)", "": "on_something(
), "": "can_manipulate()"}, -"instruction": "screw the nut into the bolt. Tool should be used and be put back at the end."} -- Output: -``` -{"task_cohesion": { - "task_sequence": [ - "load_tool()", - "approach()", - "grasp_object()", - "pick_up()", - "approach()", - "insert()", - "screw()", - "release_object()", - "unload_tool()" - ], - "step_instructions": [ - "load the parallel_toolbox", - "move the hand close to the nut", - "grasp the nut", - "pick up the nut", - "move the hand close to the bolt", - "insert the nut onto the bolt", - "screw the nut into the bold", - "release the nut", - "unload the parallel_toolbox" - ], - "object_name": ""}, -"environment_before": {"assets":["
"], - "asset_states": {"": "can_manipulate()"}, - "objects": ["", "", ""], - "object_states": {"": "on_something(
)", "": "on_something(
)"},} -"environment_after": {"assets":["
"], - "asset_states": {"": "can_manipulate()"}, - "objects": ["", "", ""], - "object_states": {"": "screwed_into()", "": "on_something(
)"}, -"instruction_summary": "screw the nut into the bolt. Tool should be used and be put back at the end.", -"question":""} -} -``` -""" -From these examples, learn that some robotic actions have dependencies with the actions before and after them. -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/kios_bt_planning/kios_agent/kios_llm_bt/prompt/old_prompt_output_format.txt b/kios_bt_planning/kios_agent/kios_llm_bt/prompt/old_prompt_output_format.txt deleted file mode 100644 index 7be3d38b..00000000 --- a/kios_bt_planning/kios_agent/kios_llm_bt/prompt/old_prompt_output_format.txt +++ /dev/null @@ -1,20 +0,0 @@ -[user] -You divide the actions given in the text into detailed robot actions and put them together as a python dictionary. -The dictionary has five keys. -""" -- dictionary["task_cohesion"]: A dictionary containing information about the robot's actions that have been split up. -- dictionary["environment_before"]: The state of the environment before the manipulation. -- dictionary["environment_after"]: The state of the environment after the manipulation. -- dictionary["instruction_summary"]: contains a brief summary of the given sentence. -- dictionary["question"]: If you cannot understand the given sentence, you can ask the user to rephrase the sentence. Leave this key empty if you can understand the given sentence. -""" -Three keys exist in dictionary["task_cohesion"]. -""" -- dictionary["task_cohesion"]["task_sequence"]: Contains a list of robot actions. Only the behaviors defined in the "ROBOT ACTION LIST" will be used. -- dictionary["task_cohesion"]["step_instructions"]: contains a list of instructions corresponding to dictionary["task_cohesion"]["task_sequence"]. -- dictionary["task_cohesion"]["object_name"]: The name of the manipulated object. Only objects defined in the input dictionary will be used for the object name. -""" -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/kios_bt_planning/kios_agent/kios_llm_bt/prompt/oldprompt_behaviortree.txt b/kios_bt_planning/kios_agent/kios_llm_bt/prompt/oldprompt_behaviortree.txt deleted file mode 100644 index 585ecc2f..00000000 --- a/kios_bt_planning/kios_agent/kios_llm_bt/prompt/oldprompt_behaviortree.txt +++ /dev/null @@ -1,170 +0,0 @@ -[user] -The behavior tree should be in python dictionary format and have a structure like the following example: -""" -{ - "name": "Pick Up Apple", - "identifier": 0, - "type_name": "selector", - "children": [ - { - "summary": "check if the apple is in hand", - "name": "check apple in hand", - "identifier": 1, - "type_name": "condition", - "conditions": [ - { - "object_name": "apple", - "property_name": "in", - "property_value": "hand", - "status": True, - } - ], - }, - { - "name": "Pick Up Sequence", - "identifier": 2, - "type_name": "sequence", - "children": [ - { - "summary": "check if the apple is on the ground", - "name": "check apple on the ground", - "identifier": 3, - "type_name": "condition", - "conditions": [ - { - "object_name": "apple", - "property_name": "on_the_ground", - "property_value": None, - "status": True, - } - ], - }, - { - "summary": "check if the hand is free", - "name": "check hand free", - "identifier": 4, - "type_name": "condition", - "conditions": [ - { - "object_name": "hand", - "property_name": "free", - "property_value": None, - "status": True, - } - ], - }, - { - "summary": "pick up the apple", - "name": "pick_up(apple)", - "identifier": 5, - "type_name": "action", - "effects": [ - { - "object_name": "apple", - "property_name": "on_the_ground", - "property_value": None, - "status": False, - }, - { - "object_name": "apple", - "property_name": "in", - "property_value": "hand", - "status": True, - }, - { - "object_name": "hand", - "property_name": "free", - "property_value": None, - "status": False, - }, - ], - }, - ], - }, - ], -} -""" -The dictionary that you return should be formatted as python dictionary. Follow these rules: -1. Every node should have three basic keys: - - "name": the name of the node (for action node, the name should be the same as defined in the domain knowledge) - - "identifier": the identifier of the node, which should be unique - - "type_name": the type of the node, its value can be: "selector", "sequence", "condition", "action" -2. The "selector" and "sequence" nodes can (and should) have a key "children" with a list of children nodes. -3. The "action" and "condition" nodes should have a key "summary" with a short description of the action/condition. -4. The "condition" nodes should have a key "conditions" with a list of conditions to be checked. Each condition should have the following keys: - - "object_name": the object to check - - "property_name": the property/relation of the object to be checked - - "property_value": this should be the other object if this is a relation. For property it should be None - - "status": the boolean status of the property/relation the condition expects, True if the property/relation should be True, False if the property/relation should be False -5. The "action" nodes should have a key "effects" with a list of effects to be exerted after the execution of the action. Each effect should have the following keys: - - "object_name": the object to be affected - - "property_name": the property/relation of the object to be affected - - "property_value": this should be the other object if this is a relation. For property it should be None - - "status": True if the effect is to add this property/relation. False if is to remove. -6. The behavior tree is used to represent an action in the action sequence. Its basic structure is: - - A selector as the root node. - - (Usually one) (Goal-)Condition node as the first child of the selector. It should check the goal conditions the action intends to achieve. So if the goal condition is met, the selector will directly return success. - - A sequence node as the last child of the selector node. The aim of the sequence is to achieve the goal conditions. - - (Pre-)Condition nodes as the front children of the sequence node. They should check the preconditions of the action, which is the last child node of the sequence node. So if any of the preconditions is not met, the sequence node will return failure. - - Only one action node as the last child of the sequence node. Its preconditions are checked by the condition nodes in the front of it. Its effects are exerted after the execution of the action, and should be able to satisfy the goal condition(s), which is the first child of the selector node. -7. When there is any (Pre-)condition node that is not satisfied, a behavior tree can be constructed according to the rules above, which replaces the unsatisfied (Pre-)condition, takes that (Pre-)condition as the (Goal-)condition to achieve and includes the action to achieve it. -8. The selector can have multiple (goal-)condition nodes as children, but must have only and no more than one sequence node as the last child. -9. The sequence node can have multiple (pre-)condition nodes (or selectors) as children, but must have only and no more than one action node as the last child. -10. The behavior tree should be constructed based on the action sequence you planned and the action definitions from the domain knowledge. The depth of the behavior tree is not limited, and when being expanded, it should expanded in a vertical way instead of inserting new actions into the old sequence. - -There are also some counterexamples for you to refer to: -1. One sequence should never have multiple action nodes as its children like the behavior tree below. -{ - "name": "Get the apple", - "identifier": 0, - "type_name": "selector", - "children": [ - { - "summary": "check if the apple is in hand", - "name": "check apple in hand", - "identifier": 1, - "type_name": "condition", - (condition omitted) - }, - { - "name": "Pick Up Sequence", - "identifier": 2, - "type_name": "sequence", - "children": [ - { - "summary": "check if the apple is on the table", - "name": "check apple on the table", - "identifier": 3, - "type_name": "condition", - (condition omitted) - }, - { - "summary": "go to the table", - "name": "go_to(table)", - "identifier": 4, - "type_name": "action", - (effect omitted) - }, - { - "summary": "check if the hand is free", - "name": "check hand free", - "identifier": 5, - "type_name": "condition", - (condition omitted) - }, - { - "summary": "pick up the apple", - "name": "pick_up(apple)", - "identifier": 6, - "type_name": "action", - (effect omitted) - }, - ], - }, - ], -} - -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/kios_bt_planning/kios_agent/kios_llm_bt/prompt/oldprompt_environment.txt b/kios_bt_planning/kios_agent/kios_llm_bt/prompt/oldprompt_environment.txt deleted file mode 100644 index ac8293ab..00000000 --- a/kios_bt_planning/kios_agent/kios_llm_bt/prompt/oldprompt_environment.txt +++ /dev/null @@ -1,28 +0,0 @@ -[user] -Information about world states are given as python dictionary. Example: -""" -{"environment":{"assets":["
", "", "", "", ""], - "asset_states": {"": "on_something(
)", - "": "on_something()"}, - "objects": ["", ""], - "object_states": {"": "on_something(
)", - "": "on_something()"}}} -""" -Asset states and object states are represented using those state sets: -""" -"STATE LIST" -- on_something(): Object is located on -- inside_something(): Object is located inside -- inside_hand(): Object is being grasped by a robot hand -- closed(): Object can be opened -- open(): Object can be closed or kept opened -- can_manipulate(): Object can be used to grasp and manipulate -- equipped(): Object is being equipped by the robot hand. -- screwed_into(): Object is screwed into by robot hand -- inserted_into(): Object is inserted into by robot hand -""" - should be one of the assets or objects in the environment. -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/kios_bt_planning/kios_agent/kios_llm_bt/prompt/prompt_behaviortree.txt b/kios_bt_planning/kios_agent/kios_llm_bt/prompt/prompt_behaviortree.txt deleted file mode 100644 index 5335e11e..00000000 --- a/kios_bt_planning/kios_agent/kios_llm_bt/prompt/prompt_behaviortree.txt +++ /dev/null @@ -1,189 +0,0 @@ -[user] -The behavior tree should be in python dictionary format and have a structure like the following example: -""" -{ - "name": "load_tool selector", - "identifier": 3, - "type_name": "selector", - "children": [ - { - "summary": "check if inward_claw is held by left hand", - "name": "check left_hand hold inward_claw", - "identifier": 4, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "inward_claw", - "status": True, - } - ], - }, - { - "name": "load_tool sequence", - "identifier": 5, - "type_name": "sequence", - "children": [ - { - "summary": "check if inward_claw is equippable", - "name": "check inward_claw is_equippable", - "identifier": 6, - "type_name": "condition", - "conditions": [ - { - "object_name": "inward_claw", - "property_name": "is_equippable", - "property_value": None, - "status": True, - } - ], - }, - { - "summary": "check if left hand is free", - "name": "check left_hand is_free", - "identifier": 7, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "is_free", - "property_value": None, - "status": True, - } - ], - }, - { - "summary": "equip inward_claw to left hand", - "name": "load_tool(left_hand, inward_claw)", - "identifier": 8, - "type_name": "action", - "effects": [ - { - "object_name": "left_hand", - "property_name": "is_free", - "property_value": None, - "status": False, - }, - { - "object_name": "inward_claw", - "property_name": "is_equippable", - "property_value": None, - "status": False, - }, - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "inward_claw", - "status": True, - }, - ], - }, - ], - }, - ], -} -""" -The dictionary that you return should be formatted as python dictionary. Follow these rules: -1. Every node should have three basic keys: - - "name": the name of the node (for action node, the name should be the same as defined in the domain knowledge) - - "identifier": the unique id of the node - - "type_name": node_type, can be: "selector", "sequence", "condition", "action" -2. The control flow node, "selector" and "sequence" nodes, can have "children", which is a list of children nodes. -3. The "action" and "condition" nodes should have "summary", which is a short description of the action/condition. -4. The "condition" nodes should have "conditions", which is a list of conditions to be checked. Each condition should have the following keys: - - "object_name": the object to check - - "property_name": the property/relation of the object to be checked - - "property_value": this should be the other object if this is a relation. For property it should be None - - "status": the boolean status of the property/relation the condition expects, True if the property/relation should be True, False otherwise -5. The "action" nodes should have a key "effects" with a list of effects to be exerted after the execution of the action. Each effect should have the following keys: - - "object_name": the affected object - - "property_name": the affected property/relation of the object - - "property_value": this should be the other object if this is a relation. For property it should be None - - "status": True if the effect is to add this property/relation. False if is to remove. -6. All the subtrees always have their corresponding action in ROBOT_ACTION_LIST. They are used to represent the actions in the action sequence. Its basic structure is: - - A selector as the root node. - - (Usually one) (Goal-)Condition node as the first child of the selector. It should check the goal conditions the action intends to achieve. So if the goal condition is met, the selector will directly return success. - - A sequence node as the last child of the selector node. The aim of the sequence is to achieve the goal conditions. - - (Pre-)Condition nodes as the front children of the sequence node. They should check the preconditions of the action node after them. - - Only one action node as the last child of the sequence node. Its preconditions are checked by the condition nodes in the front of it. Its effects are exerted after the execution of the action, and should be able to satisfy the goal condition(s). - A brief example: -{ - selector: load_tool selector, - "children": [ - { - (goal)condition: check if inward_claw is equipped by left hand, - }, - { - sequence: load_tool sequence, - children: [ - (pre)condition: check if inward_claw is equippable, (if not satisfied, this condition node can be expanded. A new subtree can be constructed to satisfy this) - (pre)condition: check if left hand is free, (if not satisfied, this condition node can be expanded. A new subtree can be constructed to satisfy this) - (more preconditions can be added here), - action: equip inward_claw to left hand, - ], - }, - ], -} - -7. When there is any (Pre-)condition node that is not satisfied, a behavior tree can be constructed according to the rules above, which replaces the unsatisfied (Pre-)condition, takes that (Pre-)condition as the (Goal-)condition to achieve and includes the action to achieve it. -8. The selector can have multiple (goal-)condition nodes as children, but must have only and no more than one sequence node as the last child. -9. The sequence node can have multiple (pre-)condition nodes (or selectors) as children, but must have only and no more than one action node as the last child. -10. The behavior tree should be constructed based on the action sequence you planned and the action definitions in ROBOT_ACTION_LIST. - -There are also some counterexamples for you to refer to: -1. One sequence should never have multiple action nodes as its children like the behavior tree below. -{ - "name": "Get the apple", - "identifier": 0, - "type_name": "selector", - "children": [ - { - "summary": "check if the apple is in hand", - "name": "check apple in hand", - "identifier": 1, - "type_name": "condition", - (condition omitted) - }, - { - "name": "Pick Up Sequence", - "identifier": 2, - "type_name": "sequence", - "children": [ - { - "summary": "check if the apple is on the table", - "name": "check apple on the table", - "identifier": 3, - "type_name": "condition", - (condition omitted) - }, - { - "summary": "go to the table", - "name": "go_to(table)", - "identifier": 4, - "type_name": "action", - (effect omitted) - }, - { - "summary": "check if the hand is free", - "name": "check hand free", - "identifier": 5, - "type_name": "condition", - (condition omitted) - }, - { - "summary": "pick up the apple", - "name": "pick_up(apple)", - "identifier": 6, - "type_name": "action", - (effect omitted) - }, - ], - }, - ], -} - -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/kios_bt_planning/kios_agent/kios_llm_bt/prompt/prompt_domain.txt b/kios_bt_planning/kios_agent/kios_llm_bt/prompt/prompt_domain.txt deleted file mode 100644 index a9b3acb4..00000000 --- a/kios_bt_planning/kios_agent/kios_llm_bt/prompt/prompt_domain.txt +++ /dev/null @@ -1,89 +0,0 @@ -[user] -Following is the domain knowledge you need in pddl domain format, which includes: -- the object types in the world, -- the predicates for describing the states of the world, -- the actions that the robot can apply, with their precondtions and effects defined. -""" -(define (domain robot_assembly_problem-domain) - (:requirements :strips :typing) - (:types - thing - object - tool part hand - thing - ) - (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part)) - (:action pick_up - :parameters ( ?hand - hand ?tool - tool ?part - part) - :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part)) - :effect (and (hold ?tool ?part) (not (is_free ?tool)))) - (:action put_down - :parameters ( ?hand - hand ?tool - tool ?part - part) - :precondition (and (hold ?tool ?part) (hold ?hand ?tool)) - :effect (and (not (hold ?hand ?part)) (is_free ?tool))) - (:action place - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2))) - (:action detach - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2)))) - (:action insert - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2))) - (:action pull - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2)))) - (:action screw - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2))) - (:action unscrew - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2)))) - (:action load_tool - :parameters ( ?hand - hand ?tool - tool) - :precondition (and (is_equippable ?tool) (is_free ?hand)) - :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool))) - (:action unload_tool - :parameters ( ?hand - hand ?tool - tool) - :precondition (and (hold ?hand ?tool) (is_free ?tool)) - :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool)))) -) -""" -The precidates are explained as follows: -""" -"PREDICATE_LIST" -- is_free(): is free and can hold something. (Initially defaulted to be true) (Note: can be a or a ) -- is_equippable(): is ready to be eqquipped by a robot hand. (Initially defaulted to be true) -- can_manipulate(, ): can be used to manipulate (manipulation constraint) (Initially defaulted to be false) -- can_insert_to(, ): can be inserted into (insertion assembly constraint) (Initially defaulted to be false) -- can_screw_to(, ): can be screwed into (screw assembly constraint) (Initially defaulted to be false) -- can_place_to(, ): can be placed into (placement assembly constraint) (Initially defaulted to be false) -- hold(, ): is being held by (Initially defaulted to be false) -- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (Initially defaulted to be false) -- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (Initially defaulted to be false) -- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (Initially defaulted to be false) -""" - -The actions are explained as follows: -""" -"ROBOT_ACTION_LIST" -- pick_up(, , ): use to pick up -- put_down(, , ): use to put down -- place(, , , ): use to place to (to satisfy the placement assembly constraint) -- detach(, , , ): use to detach from (to lift the existing placement assembly constraint) -- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint) -- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint) -- screw(, , , ): use to screw into (to satisfy the screw assembly constraint) -- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint) -- load_tool(, ): equips to manipulate other objects -- unload_tool(, ): unloads to manipulate other objects -""" -the items in <> indicate the type of the instances needed for the predicates and actions. -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. diff --git a/kios_bt_planning/kios_agent/kios_llm_bt/prompt/prompt_environment.txt b/kios_bt_planning/kios_agent/kios_llm_bt/prompt/prompt_environment.txt deleted file mode 100644 index 1d371a46..00000000 --- a/kios_bt_planning/kios_agent/kios_llm_bt/prompt/prompt_environment.txt +++ /dev/null @@ -1,31 +0,0 @@ -[user] -Information about world states are given as python dictionary. Example: -""" -{ - "objects": [ - {"name": "leg1", "properties": []}, - {"name": "seat", "properties": []}, - {"name": "cabinet", "properties": ["open"]}, - {"name": "parallel_box1", "properties": []}, - {"name": "table", "properties": []}, - ], - "constraints": [ - {"source": "leg1", "name": "can_screw_to", "target": "seat"}, - {"source": "parallel_box1", "name": "can_manipulate", "target": "leg1"}, - {"source": "cabinet", "name": "on_something", "target": "table"}, - ], - "relations": [ - {"source": "leg1", "name": "is_screwed_to", "target": "seat"}, - ], -} -""" -It contains three lists: - - objects are the objects and their properties (as lists of strings) in the environment. The properties can be changed by robot actions. - - constraints are the relations that are enforced (which are truth knowledge or defined by the user in the current contexts). They are given as a part of the problem and should not be changed. - - relations are the relations that are observed in the environment. They can be changed by robot actions. -The available properties, constraints and relations can be found in the [PREDICATE_LIST]. -You should only concentrate on those objects, constraints and relations that are relevant for the current task. -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/kios_bt_planning/kios_agent/kios_llm_bt/prompt/prompt_example.txt b/kios_bt_planning/kios_agent/kios_llm_bt/prompt/prompt_example.txt deleted file mode 100644 index a079e986..00000000 --- a/kios_bt_planning/kios_agent/kios_llm_bt/prompt/prompt_example.txt +++ /dev/null @@ -1,294 +0,0 @@ -[user] -I will give you some examples of the input and the output you will generate. -Example 1: -""" -- Input: -(define (problem robot_assembly_problem-problem) - (:domain robot_assembly_problem-domain) - (:objects - parallel_box1 parallel_box2 inward_claw outward_claw no_tool - tool - gear1 gear2 gear3 shaft1 shaft2 base - part - left_hand - hand - ) - (:init (can_manipulate parallel_box1 gear1) (can_manipulate outward_claw gear2) (can_manipulate inward_claw gear3) (can_manipulate parallel_box2 shaft1) (can_manipulate no_tool shaft2) (can_screw_to leg1 seat) (can_screw_to leg2 seat) (can_insert_to back seat) (can_screw_to nut1 seat) (can_screw_to nut2 seat) (can_screw_to blub base) (can_place_to lamp blub) (can_insert_to shaft1 base) (can_insert_to shaft2 base) (can_insert_to gear3 shaft2) (can_insert_to gear2 base) (can_insert_to gear1 shaft1) (is_inserted_to shaft1 base) (is_free left_hand) (is_free parallel_box1) (is_free parallel_box2) (is_free inward_claw) (is_free outward_claw) (is_free no_tool) (is_free gear1) (is_free gear2) (is_free gear3) (is_free shaft1) (is_free shaft2) (is_free base) (is_equippable parallel_box1) (is_equippable parallel_box2) (is_equippable inward_claw) (is_equippable outward_claw) (is_equippable no_tool)) - (:goal (and (is_inserted_to gear1 shaft1))) -) -- Output: -``` -{"task_cohesion": { - "action_sequence": [ - "load_tool(left_hand, parallel_box1)", - "pick_up(left_hand, parallel_box1, gear1)", - "insert(left_hand, parallel_box1, gear1, shaft1)" - ], - "step_instructions": [ - "equip the parallel_box1 on the left hand", - "pick up the gear1 with the left hand", - "insert the gear1 into the shaft1 using the left hand with the parallel_box1" - ], - "object_name": ["left_hand", "gear1", "parallel_box1", "shaft1"], - "behavior_tree": { - "name": "insert selector", - "identifier": 13, - "type_name": "selector", - "children": [ - { - "summary": "check if gear1 is inserted to shaft1", - "name": "check gear1 is_inserted_to shaft1", - "identifier": 14, - "type_name": "condition", - "conditions": [ - { - "object_name": "gear1", - "property_name": "is_inserted_to", - "property_value": "shaft1", - "status": True, - } - ], - }, - { - "name": "insert sequence", - "identifier": 15, - "type_name": "sequence", - "children": [ - { - "name": "load_tool selector", - "identifier": 0, - "type_name": "selector", - "children": [ - { - "summary": "check if parallel_box1 is hold", - "name": "check left_hand hold parallel_box1", - "identifier": 1, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "parallel_box1", - "status": True, - } - ], - }, - { - "name": "load_tool Sequence", - "identifier": 2, - "type_name": "sequence", - "children": [ - { - "summary": "check if parallel_box1 is equippable", - "name": "check parallel_box1 is_equippable", - "identifier": 3, - "type_name": "condition", - "conditions": [ - { - "object_name": "parallel_box1", - "property_name": "is_equippable", - "property_value": None, - "status": True, - } - ], - }, - { - "summary": "check if left hand is free", - "name": "check left_hand is_free", - "identifier": 4, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "is_free", - "property_value": None, - "status": True, - } - ], - }, - { - "summary": "equip parallel_box1 to left hand", - "name": "load_tool(left_hand, parallel_box1)", - "identifier": 5, - "type_name": "action", - "effects": [ - { - "object_name": "hand", - "property_name": "is_free", - "property_value": None, - "status": False, - }, - { - "object_name": "parallel_box1", - "property_name": "is_equippable", - "property_value": None, - "status": False, - }, - { - "object_name": "hand", - "property_name": "hold", - "property_value": "parallel_box1", - "status": True, - }, - ], - }, - ], - }, - ], - }, - { - "name": "pick_up selector", - "identifier": 6, - "type_name": "selector", - "children": [ - { - "summary": "check if parallel_box1 holds gear1", - "name": "check parallel_box1 hold gear1", - "identifier": 7, - "type_name": "condition", - "conditions": [ - { - "object_name": "parallel_box1", - "property_name": "hold", - "property_value": "gear1", - "status": True, - } - ], - }, - { - "name": "pick_up Sequence", - "identifier": 8, - "type_name": "sequence", - "children": [ - { - "summary": "check if parallel_box1 is free", - "name": "check parallel_box1 is_free", - "identifier": 9, - "type_name": "condition", - "conditions": [ - { - "object_name": "parallel_box1", - "property_name": "is_free", - "property_value": None, - "status": True, - } - ], - }, - { - "summary": "check if left hand holds parallel_box1", - "name": "check left_hand hold parallel_box1", - "identifier": 16, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "parallel_box1", - "status": True, - } - ], - }, - { - "summary": "pick up gear1 using parallel_box1", - "name": "pick_up(left_hand, parallel_box1, gear1)", - "identifier": 12, - "type_name": "action", - "effects": [ - { - "object_name": "parallel_box1", - "property_name": "hold", - "property_value": "gear1", - "status": True, - }, - { - "object_name": "parallel_box1", - "property_name": "is_free", - "property_value": None, - "status": False, - }, - ], - }, - ], - }, - ], - }, - { - "summary": "check if gear1 can be inserted to shaft1", - "name": "check gear1 can_insert_to shaft1", - "identifier": 18, - "type_name": "condition", - "conditions": [ - { - "object_name": "gear1", - "property_name": "can_insert_to", - "property_value": "shaft1", - "status": True, - } - ], - }, - { - "summary": "insert gear1 to shaft1", - "name": "insert(left_hand, parallel_box1, gear1, shaft1)", - "identifier": 19, - "type_name": "action", - "effects": [ - { - "object_name": "parallel_box1", - "property_name": "hold", - "property_value": "gear1", - "status": False, - }, - { - "object_name": "parallel_box1", - "property_name": "is_free", - "property_value": None, - "status": True, - }, - { - "object_name": "gear1", - "property_name": "is_inserted_to", - "property_value": "shaft1", - "status": True, - }, - ], - }, - ], - }, - ], -} -}, -"environment_before": {"objects": [ - {"name": "gear1", "properties": []}, - {"name": "shaft1", "properties": []}, - {"name": "parallel_box1", "properties": ["is_equippable", "is_free"]}, - {"name": "left_hand", "properties": ["is_free"]}, - ], - "constraints": [ - {"source": "gear1", "name": "can_insert_to", "target": "shaft1"}, - {"source": "parallel_box1", "name": "can_manipulate", "target": "gear1"}, - ], - "relations": [ - - ], -}, -"environment_after": {"objects": [ - {"name": "gear1", "properties": []}, - {"name": "shaft1", "properties": []}, - {"name": "parallel_box1", "properties": ["is_equippable", "is_free"]}, - {"name": "left_hand", "properties": []}, - ], - "constraints": [ - {"source": "gear1", "name": "can_insert_to", "target": "shaft1"}, - {"source": "parallel_box1", "name": "can_manipulate", "target": "gear1"}, - ], - "relations": [ - {"source": "gear1", "name": "is_inserted_to", "target": "shaft1"}, - {"source": "left_hand", "name": "hold", "target": "parallel_box1"}, - ], -}, -"instruction_summary": "insert gear1 into shaft1", -"question":""} -``` -""" -From these examples, learn that some robotic actions have dependencies with the actions before and after them. -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/kios_bt_planning/kios_agent/kios_llm_bt/prompt/prompt_output_format.txt b/kios_bt_planning/kios_agent/kios_llm_bt/prompt/prompt_output_format.txt deleted file mode 100644 index 0558fb35..00000000 --- a/kios_bt_planning/kios_agent/kios_llm_bt/prompt/prompt_output_format.txt +++ /dev/null @@ -1,21 +0,0 @@ -[user] -You divide the actions given in the text into detailed robot actions and contruct a behavior tree based on it in python dictionary format. -The dictionary has five keys. -""" -- dictionary["task_cohesion"]: A dictionary containing information about the robot's actions that have been split up. -- dictionary["environment_before"]: The environment state before the manipulation. -- dictionary["environment_after"]: The environment state after the manipulation. -- dictionary["instruction_summary"]: contains a brief summary of the given sentence. -- dictionary["question"]: If you cannot understand the given sentence, you can ask the user to rephrase the sentence. Leave this key empty if you can understand the given sentence. -""" -Three keys exist in dictionary["task_cohesion"]. -""" -- dictionary["task_cohesion"]["action_sequence"]: Contains a list of robot actions (with the parameters as defined in pddl domain knowledge). Only the actions defined in the "ROBOT_ACTION LIST" will be used. -- dictionary["task_cohesion"]["action_instructions"]: contains a list of instructions corresponding to dictionary["task_cohesion"]["action_sequence"] to explain the action. -- dictionary["task_cohesion"]["manipulated_objects"]: The list of the names of the manipulated object. Only objects defined in the input dictionary will be used for the object name. -- dictionary["task_cohesion"]["behavior_tree"]: contains a dictionary of the behavior tree constructed according to the dictionary ["task_cohesion"]["action_sequence"] -""" -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/kios_bt_planning/kios_agent/kios_llm_bt/prompt/prompt_problem.txt b/kios_bt_planning/kios_agent/kios_llm_bt/prompt/prompt_problem.txt deleted file mode 100644 index c22df152..00000000 --- a/kios_bt_planning/kios_agent/kios_llm_bt/prompt/prompt_problem.txt +++ /dev/null @@ -1,26 +0,0 @@ -[user] -Information about the problem is given as pddl problem file that corresponds to the domain file. -It includes the following information: -- Objects in the world -- Initial states you start with -- Goal states you need to achieve -Example: -""" -(define (problem robot_assembly_problem-problem) - (:domain robot_assembly_problem-domain) - (:objects - parallel_box1 parallel_box2 inward_claw outward_claw no_tool - tool - gear1 gear2 gear3 shaft1 shaft2 base - part - left_hand - hand - ) - (:init (can_manipulate parallel_box1 gear1) (can_manipulate outward_claw gear2) (can_manipulate inward_claw gear3) (can_manipulate parallel_box2 shaft1) (can_manipulate no_tool shaft2) (can_screw_to leg1 seat) (can_screw_to leg2 seat) (can_insert_to back seat) (can_screw_to nut1 seat) (can_screw_to nut2 seat) (can_screw_to blub base) (can_place_to lamp blub) (can_insert_to shaft1 base) (can_insert_to shaft2 base) (can_insert_to gear3 shaft2) (can_insert_to gear2 base) (can_insert_to gear1 shaft1) (is_free left_hand) (is_free parallel_box1) (is_free parallel_box2) (is_free inward_claw) (is_free outward_claw) (is_free no_tool) (is_free gear1) (is_free gear2) (is_free gear3) (is_free shaft1) (is_free shaft2) (is_free base) (is_equippable parallel_box1) (is_equippable parallel_box2) (is_equippable inward_claw) (is_equippable outward_claw) (is_equippable no_tool)) - (:goal (and (is_inserted_to shaft1 base))) -) -""" -Objects in the world are all instances of the types defined in the domain file. For example, the object "parallel_box1" is an instance of the type "tool". The object "gear1" is an instance of the type "part". -Initial states are represented using the [PREDICATE_LIST] from the domain file. For example, the initial state "can_manipulate parallel_box1 gear1" means that the object "parallel_box1" can be used to manipulate the object "gear1". -Goal states are represented using the [PREDICATE_LIST] from the domain file. For example, the goal state "is_inserted_to shaft1 base" means that the object "shaft1" is inserted to the object "base". -------------------------------------------------------- -The texts above are part of the overall instruction. Do not start working yet: -[assistant] -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/kios_bt_planning/kios_agent/kios_llm_bt/prompt/prompt_role.txt b/kios_bt_planning/kios_agent/kios_llm_bt/prompt/prompt_role.txt deleted file mode 100644 index 90b48291..00000000 --- a/kios_bt_planning/kios_agent/kios_llm_bt/prompt/prompt_role.txt +++ /dev/null @@ -1,5 +0,0 @@ -[user] -You are an excellent interpreter of human instructions for robotic assembly tasks. Given the information about the world states and some instructions, you break it down and construct robot task plans represented by behaiovr trees based on the action sequences. -Please do not begin working until I say "Start working." Instead, simply output the message "Waiting for next input." Understood? -[assistant] -Understood. Waiting for next input. \ No newline at end of file diff --git a/kios_bt_planning/kios_agent/kios_llm_bt/query/old_query.txt b/kios_bt_planning/kios_agent/kios_llm_bt/query/old_query.txt deleted file mode 100644 index f652fe76..00000000 --- a/kios_bt_planning/kios_agent/kios_llm_bt/query/old_query.txt +++ /dev/null @@ -1,23 +0,0 @@ -Start working. Resume from the environment below. -""" -{"environment":[ENVIRONMENT]} -""" -The problem is as follows: -""" -{"problem": [PROBLEM]} -""" -The instruction is as follows: -""" -{"instruction": [INSTRUCTION]} -""" -The dictionary that you return should be formatted as python dictionary. Follow these rules: -1. Always equip the proper tools before performing specific object manipulaion. The robot should not hold any tool or object at the end. -2. Make sure that each element of the ["action_instructions"] explains corresponding element of the ["action_sequence"]. Refer to the "ROBOT ACTION LIST" to understand the elements of ["action_sequence"]. -3, The length of the ["action_instructions"] list must be the same as the length of the ["action_sequence"] list. -4. Never left ',' at the end of the list. -5. Keep track of all items listed in the "objects" section of the "environment_before" field. Please ensure that you fill out both the "objects" and "object_states" sections for all listed items. -6. Use the "PREDICATE_LIST" for the "object states" field. -7. All keys of the dictionary should be double-quoted. -8. Insert ``` at the beginning and the end of the dictionary to separate it from the rest of your response. -9. Make sure that you output a consistent manipultation as a single arm robot. For example, grasping an object should not occur in successive steps. -Adhere to the output format I defined above. Follow the nine rules. Think step by step. \ No newline at end of file diff --git a/kios_bt_planning/kios_agent/kios_llm_bt/query/query.txt b/kios_bt_planning/kios_agent/kios_llm_bt/query/query.txt deleted file mode 100644 index 0a45a556..00000000 --- a/kios_bt_planning/kios_agent/kios_llm_bt/query/query.txt +++ /dev/null @@ -1,16 +0,0 @@ -Start working. Resume from problem below. -""" -{"problem": [PROBLEM]} -""" -The dictionary that you return should be formatted as python dictionary. Follow these rules: -1. Always equip the proper tools before performing specific object manipulaion. -2. Make sure that each element of the ["action_instructions"] explains corresponding element of the ["action_sequence"]. Refer to the "ROBOT ACTION LIST" to understand the elements of ["action_sequence"]. -3, The length of the ["action_instructions"] list must be the same as the length of the ["action_sequence"] list. -4. Never left ',' at the end of the list. -5. Keep track of all items listed in the "objects" section of the "environment_before" field. Please make sure you fill out the "objects" section for all listed items. -6. Use the "PREDICATE_LIST" for the states in the problem. -7. All keys of the dictionary should be double-quoted. -8. Insert ``` at the beginning and the end of the dictionary to separate it from the rest of your response. -9. Make sure that you output a consistent manipultation as a single arm robot. For example, grasping an object should not occur in successive steps. -10. When generating the behavior tree, make sure the tree has the asked structure. -Adhere to the output format I defined above. Follow the rules above. Think step by step. \ No newline at end of file diff --git a/kios_bt_planning/kios_agent/kios_llm_bt/query/second_old_query.txt b/kios_bt_planning/kios_agent/kios_llm_bt/query/second_old_query.txt deleted file mode 100644 index 530173f3..00000000 --- a/kios_bt_planning/kios_agent/kios_llm_bt/query/second_old_query.txt +++ /dev/null @@ -1,16 +0,0 @@ -Start working. Resume from problem below. -""" -{"problem": [PROBLEM]} -""" -The dictionary that you return should be formatted as python dictionary. Follow these rules: -1. Always equip the proper tools before performing specific object manipulaion. -2. Make sure that each element of the ["action_instructions"] explains corresponding element of the ["action_sequence"]. Refer to the "ROBOT ACTION LIST" to understand the elements of ["action_sequence"]. -3, The length of the ["action_instructions"] list must be the same as the length of the ["action_sequence"] list. -4. Never left ',' at the end of the list. -5. Keep track of all items listed in the "objects" section of the "environment_before" field. Please make sure you fill out the "objects" section for all listed items. -6. Use the "PREDICATE_LIST" for the "object states" field. -7. All keys of the dictionary should be double-quoted. -8. Insert ``` at the beginning and the end of the dictionary to separate it from the rest of your response. -9. Make sure that you output a consistent manipultation as a single arm robot. For example, grasping an object should not occur in successive steps. -10. For boolean values don't use capital letters. Use "true" or "false" instead of "True" or "False". -Adhere to the output format I defined above. Follow the rules above. Think step by step. \ No newline at end of file diff --git a/kios_bt_planning/kios_agent/kios_llm_bt/system/system.txt b/kios_bt_planning/kios_agent/kios_llm_bt/system/system.txt deleted file mode 100644 index 26f2afe2..00000000 --- a/kios_bt_planning/kios_agent/kios_llm_bt/system/system.txt +++ /dev/null @@ -1 +0,0 @@ -You are an excellent interpreter of instructions for robotic assembly tasks. Given the knowledge you need, the world states and some instructions, you break it down and construct robot task plans represented by behaiovr trees based on the action sequences. \ No newline at end of file diff --git a/kios_bt_planning/kios_agent/kios_routers.py b/kios_bt_planning/kios_agent/kios_routers.py deleted file mode 100644 index 9aeb854c..00000000 --- a/kios_bt_planning/kios_agent/kios_routers.py +++ /dev/null @@ -1,76 +0,0 @@ -from semantic_router import Route -from semantic_router.encoders import CohereEncoder, OpenAIEncoder -from semantic_router.layer import RouteLayer - -from getpass import getpass - -from dotenv import load_dotenv -import os -import json - -load_dotenv() - -data_dir = os.environ.get("KIOS_DATA_DIR").format(username=os.getlogin()) - - -class KiosRouterFactory: - encoder: OpenAIEncoder | CohereEncoder = None - utterance_dictionary: dict[str, list[str]] = None - - def __init__(self, encoder="openai") -> None: - # initialize the encoder - if encoder == "openai": - os.environ["OPENAI_API_KEY"] = os.getenv("OPENAI_API_KEY") or getpass( - "OpenAI API Key not found, please enter it here or stop here and check your environment variables:" - ) - self.encoder = OpenAIEncoder() - elif encoder == "cohere": - os.environ["COHERE_API_KEY"] = os.getenv("COHERE_API_KEY") or getpass( - "Cohere API Key not found, please enter it here or stop here and check your environment variables:" - ) - self.encoder = CohereEncoder() - else: - raise ValueError(f"Encoder {encoder} not supported") - - # fetch the utterances - self.utterance_dictionary = json.load( - open(os.path.join(data_dir, "router_utterances.json")) - ) - - def create_router_layer(self, route_names: list[str]) -> RouteLayer: - if len(route_names) <= 1: - raise ValueError( - f"It makes no sense to create a router with less than one route! Please provide at least two routes." - ) - return RouteLayer( - encoder=self.encoder, - routes=self._fetch_utterance(route_names), - ) - - def _fetch_utterance(self, route_names: list[str]) -> list[Route]: - # check first - for name in route_names: - if name not in self.utterance_dictionary.keys(): - raise ValueError(f"Utterance {name} not found in the data file!") - - # make the routes - return [ - Route( - name=name, - utterances=self.utterance_dictionary[name], - ) - for name in route_names - ] - - -def test(): - factory = KiosRouterFactory() - router = factory.create_router_layer(["approve", "disapprove"]) - - from pprint import pprint - - pprint(router("I guess you have to modify the plan a bit")) - - -if __name__ == "__main__": - test() diff --git a/kios_bt_planning/kios_agent/kios_tools.py b/kios_bt_planning/kios_agent/kios_tools.py deleted file mode 100644 index 49576f73..00000000 --- a/kios_bt_planning/kios_agent/kios_tools.py +++ /dev/null @@ -1,114 +0,0 @@ -from typing import Optional, Type, List - -""" -not in use yet. this part will become necessary if you want to implement an agent. -""" - -# Import things that are needed generically -from langchain.pydantic_v1 import BaseModel, Field -from langchain.tools import BaseTool, StructuredTool, tool - -from langchain.callbacks.manager import ( - AsyncCallbackManagerForToolRun, - CallbackManagerForToolRun, -) - -from kios_bt.bt_stewardship import BehaviorTreeStewardship -from kios_bt.data_types import TreeResult - - -class BehaviorTreeInput(BaseModel): - json_data: dict = Field( - description="json data for the robot task plan, including the task plan in behavior tree and the initial state of the world." - ) - - -class BehaviorTreeExecutorTool(BaseTool): - name = "behavior tree executor" - description = "useful for make the robot execute the behavior tree plan when it is verified and ready to be executed to achieve the task goal" - args_schema: Type[BaseModel] = BehaviorTreeInput - - bt_stw: BehaviorTreeStewardship - - def __init__(self, bt_stw: BehaviorTreeStewardship): - self.bt_stw = bt_stw - super().__init__() - - def _run( - self, json_data: dict, run_manager: Optional[CallbackManagerForToolRun] = None - ) -> str: - """Use the tool.""" - return "LangChain" - - async def _arun( - self, query: str, run_manager: Optional[AsyncCallbackManagerForToolRun] = None - ) -> str: - """Use the tool asynchronously.""" - raise NotImplementedError("custom_search does not support async") - - -class BehaviorTreeInput(BaseModel): - behavior_tree: dict = Field( - description="json data for the robot task plan, including the task plan in behavior tree and the initial state of the world." - ) - world_state: dict = Field(description="json data for the current world state.") - - -class BehaviorTreeSimulatorTool(BaseTool): - name = "behavior tree simulator tool" - description = "useful for simulating the behavior tree plan to see if the plan can achieve the task goal in the ideal case" - args_schema: Type[BaseModel] = BehaviorTreeInput - - def _run( - self, - behavior_tree: dict, - world_state: dict, - run_manager: Optional[CallbackManagerForToolRun] = None, - ) -> TreeResult: - """Use the tool.""" - - bt_stw: BehaviorTreeStewardship = self.metadata["bt_stw"] - tree_result = bt_stw.fake_run(bt_json=behavior_tree, world_state=world_state) - - return tree_result - - async def _arun( - self, - behavior_tree: dict, - world_state: dict, - run_manager: Optional[AsyncCallbackManagerForToolRun] = None, - ) -> TreeResult: - """Use the tool asynchronously.""" - - bt_stw: BehaviorTreeStewardship = self.metadata["bt_stw"] - tree_result = bt_stw.fake_run(bt_json=behavior_tree, world_state=world_state) - - return tree_result - - -class WorldStateQuery(BaseModel): - query: str = Field(description="a string to explain the purpose of this query.") - - -class WorldStateQueryTool(BaseTool): - name = "world state query tool" - description = "useful for querying the current world state" - args_schema: Type[BaseModel] = WorldStateQuery - - bt_stw: BehaviorTreeStewardship - - def __init__(self, bt_stw: BehaviorTreeStewardship): - self.bt_stw = bt_stw - super().__init__() - - def _run( - self, query: str, run_manager: Optional[CallbackManagerForToolRun] = None - ) -> str: - """Use the tool.""" - return - - async def _arun( - self, query: str, run_manager: Optional[AsyncCallbackManagerForToolRun] = None - ) -> str: - """Use the tool asynchronously.""" - raise NotImplementedError("custom_search does not support async") diff --git a/kios_bt_planning/kios_agent/llama_deploy.py b/kios_bt_planning/kios_agent/llama_deploy.py deleted file mode 100644 index 11f55e6d..00000000 --- a/kios_bt_planning/kios_agent/llama_deploy.py +++ /dev/null @@ -1,40 +0,0 @@ -""" -Need azure cli and azure ai sdk -Currently problematic. -last error: -Traceback (most recent call last): - File "/home/blackbird/kios_workspace/src/kios/kios_bt_planning/kios_agent/llama_deploy.py", line 23, in - client.deployments.create_or_update(deployment) - File "/home/blackbird/miniconda3/envs/kios/lib/python3.10/site-packages/azure/core/tracing/decorator.py", line 78, in wrapper_use_tracer - return func(*args, **kwargs) - File "/home/blackbird/miniconda3/envs/kios/lib/python3.10/site-packages/azure/ai/ml/_telemetry/activity.py", line 275, in wrapper - return f(*args, **kwargs) - File "/home/blackbird/miniconda3/envs/kios/lib/python3.10/site-packages/azure/ai/resources/operations/_deployment_operations.py", line 209, in create_or_update - family_usage = filtered_usage_info[family] -KeyError: 'StandardNCADSA100v4Family' -""" - -# ! NOT IN USE NOW. THE LLAMA 70B MODEL IS DEPLOYED AS PAY-AS-YOU-GO SERVICE IN AZURE. -from azure.ai.resources.client import AIClient -from azure.ai.resources.entities.deployment import Deployment -from azure.ai.resources.entities.models import PromptflowModel -from azure.identity import DefaultAzureCredential - -credential = DefaultAzureCredential() - -client = AIClient( - credential=credential, - subscription_id="74f1dc98-0762-4789-a9d1-8c145058b546", - resource_group_name="blackbird_group", - project_name="llama2-kios", -) - -model_id = "azureml://registries/azureml-meta/models/Llama-2-70b/versions/15" -deployment_name = "llam270b-deployment" - -deployment = Deployment( - name=deployment_name, - model=model_id, -) - -client.deployments.create_or_update(deployment) diff --git a/kios_bt_planning/kios_agent/llm_agents.py b/kios_bt_planning/kios_agent/llm_agents.py deleted file mode 100644 index 5da9da63..00000000 --- a/kios_bt_planning/kios_agent/llm_agents.py +++ /dev/null @@ -1,145 +0,0 @@ -import json - -""" -trash -""" - -from langchain_core.messages import ( - AIMessage, - BaseMessage, - ChatMessage, - FunctionMessage, - HumanMessage, -) - -from langchain.tools.render import format_tool_to_openai_function -from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder -from langgraph.graph import END, StateGraph -from langgraph.prebuilt.tool_executor import ToolExecutor, ToolInvocation -from langchain_openai import ChatOpenAI - -from kios_agent.llm_supporter import KiosLLMSupporter - - -def create_agent(llm, tools, system_message: str): - """Create an agent.""" - functions = [format_tool_to_openai_function(t) for t in tools] - - prompt = ChatPromptTemplate.from_messages( - [ - ( - "system", - "You are a helpful AI assistant, collaborating with other assistants." - " Use the provided tools to progress towards answering the question." - " If you are unable to fully answer, that's OK, another assistant with different tools " - " will help where you left off. Execute what you can to make progress." - " If you or any of the other assistants have the final answer or deliverable," - " prefix your response with FINAL ANSWER so the team knows to stop." - " You have access to the following tools: {tool_names}.\n{system_message}", - ), - MessagesPlaceholder(variable_name="messages"), - ] - ) - prompt = prompt.partial(system_message=system_message) - prompt = prompt.partial(tool_names=", ".join([tool.name for tool in tools])) - return prompt | llm.bind_functions(functions) - - -def create_agent_with_supporter(tools, llm_spt: KiosLLMSupporter): - """Create an agent from the spt.""" - functions = [format_tool_to_openai_function(t) for t in tools] - - prompt = ChatPromptTemplate.from_messages( - [ - ( - "system", - "You are a helpful AI assistant, collaborating with other assistants." - " Use the provided tools to progress towards answering the question." - " If you are unable to fully answer, that's OK, another assistant with different tools " - " will help where you left off. Execute what you can to make progress." - " If you or any of the other assistants have the final answer or deliverable," - " prefix your response with FINAL ANSWER so the team knows to stop." - " You have access to the following tools: {tool_names}.\n{system_message}", - ), - MessagesPlaceholder(variable_name="messages"), - ] - ) - - llm = ChatOpenAI( - model=llm_spt.model_name, - temperature=0, - ) - - prompt = prompt.partial(system_message=system_message) - prompt = prompt.partial(tool_names=", ".join([tool.name for tool in tools])) - return prompt | llm.bind_functions(functions) - - -from langchain_core.agents import AgentActionMessageLog, AgentFinish - - -def parse(output): - # If no function was invoked, return to user - if "function_call" not in output.additional_kwargs: - return AgentFinish(return_values={"output": output.content}, log=output.content) - - # Parse out the function call - function_call = output.additional_kwargs["function_call"] - name = function_call["name"] - inputs = json.loads(function_call["arguments"]) - - # If the Response function was invoked, return to the user with the function inputs - if name == "Response": - return AgentFinish(return_values=inputs, log=str(function_call)) - # Otherwise, return an agent action - else: - return AgentActionMessageLog( - tool=name, tool_input=inputs, log="", message_log=[output] - ) - - -from typing import List - -from langchain_core.pydantic_v1 import BaseModel, Field - - -class Response(BaseModel): - """Final response to the question being asked""" - - answer: str = Field(description="The final answer to respond to the user") - sources: List[int] = Field( - description="List of page chunks that contain answer to the question. Only include a page chunk if it contains relevant information" - ) - - -from langchain.agents import AgentExecutor -from langchain.agents.format_scratchpad import format_to_openai_function_messages -from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder -from langchain_openai import ChatOpenAI - -prompt = ChatPromptTemplate.from_messages( - [ - ("system", "You are a helpful assistant"), - ("user", "{input}"), - MessagesPlaceholder(variable_name="agent_scratchpad"), - ] -) - -llm = ChatOpenAI(temperature=0) - -llm_with_tools = llm.bind_functions([retriever_tool, Response]) - -agent = ( - { - "input": lambda x: x["input"], - # Format agent scratchpad from intermediate steps - "agent_scratchpad": lambda x: format_to_openai_function_messages( - x["intermediate_steps"] - ), - } - | prompt - | llm_with_tools - | parse -) - -agent_executor = AgentExecutor(tools=[retriever_tool], agent=agent, verbose=True) diff --git a/kios_bt_planning/kios_agent/llm_supporter.py b/kios_bt_planning/kios_agent/llm_supporter.py deleted file mode 100644 index 1930ce59..00000000 --- a/kios_bt_planning/kios_agent/llm_supporter.py +++ /dev/null @@ -1,264 +0,0 @@ -from typing import Any, List, Dict # ! use embedded typing in python from 3.10. - -from dotenv import load_dotenv - -""" -supporter for generating the llm prompt. -this is a historical file and should be removed later. -""" - -import openai -import tiktoken -import json -import os -import re -import ast -import argparse -import sys -import textwrap - -from langsmith import traceable -from langsmith.wrappers import wrap_openai - -from langchain_core.prompts import ( - ChatPromptTemplate, - FewShotChatMessagePromptTemplate, - BaseChatPromptTemplate, -) -from langchain_core.output_parsers import JsonOutputParser -from langchain_openai import ChatOpenAI - -from kios_agent.data_types import AgentResponse, KiosPromptSkeleton - - -""" -the langchain version of the kios llm -""" - -load_dotenv() - - -class KiosLLMSupporter: - prompt_directories: Dict[str, str] = None - - ######################################## * - max_token_length: int - max_completion_length: int - last_response: str = None - - model_name: str = None - - ######################################## * prompts - prompt_skeleton: KiosPromptSkeleton = None - - prompt_dir: str = None - prompt_load_order: List[str] = None - - # the problem description - problem: str = None - - # the query template - query: str = None - - # variable to keep the latest instruction - instruction: str = None - - # the "system" prompt part - system_message: str = None - - # The list of messages for prompting, will be sent to api - # including: the required format (in prompt folder), the last response as assistant prompt - messages: List[Dict[str, str]] = None - - ######################################## * dir - history_dir: str = None - - def __init__(self): - pass - - def initialize_from_prompt_skeleton(self, prompt_skeleton: KiosPromptSkeleton): - self.model_name = prompt_skeleton.model_name - self.prompt_skeleton = prompt_skeleton - self.initialize( - prompt_dir=prompt_skeleton.prompt_dir, - prompt_load_order=prompt_skeleton.prompt_load_order, - ) - - # ! extend this method later. - def initialize(self, prompt_dir: str = None, prompt_load_order: List[str] = None): - - data_dir = os.environ.get("KIOS_DATA_DIR").format(username=os.getlogin()) - - if prompt_dir is not None: - self.prompt_dir = os.path.join(data_dir, prompt_dir) - else: - # self.prompt_dir = os.path.join(script_dir, "prompts") - raise Exception("prompt_dir is not given!") - - self.prompt_directories = { - "system": os.path.join(self.prompt_dir, "system"), - "query": os.path.join(self.prompt_dir, "query"), - "prompt": os.path.join(self.prompt_dir, "prompt"), - } - - # * check the prompt directories - print("prompt directories:") - for key, value in self.prompt_directories.items(): - print(f'["{key}"] = {value}') - - # default prompt load order - if prompt_load_order is not None: - self.prompt_load_order = prompt_load_order - else: - # self.prompt_load_order = [ - # "prompt_role", # your are a good interpreter ... Done - # "prompt_domain", # domain knowledge ... Done - # "prompt_problem", # how the problem is provided ... Done - # "prompt_environment", # how to express the environment ... Done - # # "prompt_behaviortree", # how to construct the behavior tree ... Done - # "prompt_bt_skeleton", # how to construct the skeleton behavior tree ... Done - # "prompt_bt_skeleton_example", # some skeleton examples ... Done - # "prompt_output_format", # the output format ... Done - # # "prompt_example", # some examples ... Done - # ] - raise Exception("prompt_load_order is not given!") - - # history directory - if not os.path.exists(os.path.join(data_dir, "query_history")): - os.makedirs(os.path.join(data_dir, "query_history")) - self.history_dir = os.path.join(data_dir, "query_history") - - # default - self.max_token_length: int = 16000 - self.max_completion_length: int = 4000 - self.last_response = None - - # prompt initialization - self.messages = [] - self.query = "" - self.instruction = "" - - # *load prompt file - # system - fp_system = os.path.join(self.prompt_directories["system"], "system.txt") - with open(fp_system) as f: - self.system_message = f.read() - - # prompt for domain knowledge, user: brabrabra, assistant: do nothing and wait for the next prompt - for prompt_name in self.prompt_load_order: - fp_prompt = os.path.join( - self.prompt_directories["prompt"], prompt_name + ".txt" - ) - with open(fp_prompt) as f: - data = f.read() - data_spilit = re.split(r"\[user\]\n|\[assistant\]\n", data) - data_spilit = [item for item in data_spilit if len(item) != 0] - assert len(data_spilit) % 2 == 0 - # load sparately to user and assistant - message = {} - for i, item in enumerate(data_spilit): - if i % 2 == 0: - message["input"] = item - else: - message["output"] = item - self.messages.append(message) - message = {} - - # load the query template - fp_query = os.path.join(self.prompt_directories["query"], "query.txt") - with open(fp_query) as f: - self.query = f.read() - - def create_prompt(self): - """ - create the prompt from messages for gpt api. - if too long, truncate the prompt and call this recursively - """ - - example_template = ChatPromptTemplate.from_messages( - [ - ("human", "{input}"), - ("ai", "{output}"), - ] - ) - - few_shot_prompt = FewShotChatMessagePromptTemplate( - example_prompt=example_template, - examples=self.messages, - ) - - system_prompt = ChatPromptTemplate.from_messages( - [ - ("system", self.system_message), - ] - ) - - query_prompt = ChatPromptTemplate.from_messages( - [ - ("human", self.query), - ] - ) - - final_prompt = ChatPromptTemplate.from_messages( - [ - system_prompt, - few_shot_prompt, - query_prompt, - ] - ) - - return final_prompt - - def extract_json_part(self, text): - """ - extract the markdown code block part from the text - """ - if text.find("```") == -1: - return text - text_json = text[text.find("```") + 3 : text.find("```", text.find("```") + 3)] - return text_json - - def dump_json(self, dump_name=None): - """ - dump the json dictionary to a file, used to save into a json file - """ - if dump_name is not None: - # dump the dictionary to json file dump 1, 2, ... - fp = os.path.join(dump_name + ".json") - with open(fp, "w") as f: - json.dump(self.json_dict, f, indent=4) - - def record_history(self, query: str, response: str, problem_name: str): - - # * setup the problem directory - while not os.path.exists(os.path.join(self.history_dir, problem_name)): - os.makedirs(os.path.join(self.history_dir, problem_name)) - - problem_dir = os.path.join(self.history_dir, problem_name) - - # * the try number under it - i = 0 - while os.path.exists(os.path.join(problem_dir, str(i))): - i = i + 1 - os.makedirs(os.path.join(problem_dir, str(i))) - this_problem_dir = os.path.join(problem_dir, str(i)) - - # get the json response - json_response = response - - # dump to a text file - with open(os.path.join(this_problem_dir, "response.txt"), "w") as f: - f.write(json_response) - - # write the query problem - with open(os.path.join(this_problem_dir, "query.txt"), "w") as f: - f.write(query) - - # copy the prompts - with open(os.path.join(this_problem_dir, "message.txt"), "w") as f: - f.write(str(self.messages)) - - -if __name__ == "__main__": - # test_llm() - pass diff --git a/kios_bt_planning/kios_agent/old_result.txt b/kios_bt_planning/kios_agent/old_result.txt deleted file mode 100644 index fbfbc706..00000000 --- a/kios_bt_planning/kios_agent/old_result.txt +++ /dev/null @@ -1,260 +0,0 @@ -``` -{ - "task_cohesion": { - "action_sequence": [ - "load_tool(left_hand, parallel_box1)", - "pick_up(left_hand, parallel_box1, gear1)", - "insert(left_hand, parallel_box1, gear1, shaft1)" - ], - "action_instructions": [ - "Equip the parallel_box1 on the left hand.", - "Pick up the gear1 with the left hand using the parallel_box1.", - "Insert the gear1 into the shaft1 using the left hand equipped with the parallel_box1." - ], - "manipulated_objects": ["left_hand", "parallel_box1", "gear1", "shaft1"], - "behavior_tree": { - "name": "insert gear1 into shaft1 selector", - "identifier": 1, - "type_name": "selector", - "children": [ - { - "summary": "Check if gear1 is already inserted into shaft1", - "name": "check gear1 is_inserted_to shaft1", - "identifier": 2, - "type_name": "condition", - "conditions": [ - { - "object_name": "gear1", - "property_name": "is_inserted_to", - "property_value": "shaft1", - "status": True - } - ] - }, - { - "name": "insert gear1 into shaft1 sequence", - "identifier": 3, - "type_name": "sequence", - "children": [ - { - "name": "load_tool selector", - "identifier": 4, - "type_name": "selector", - "children": [ - { - "summary": "Check if parallel_box1 is held by left hand", - "name": "check left_hand hold parallel_box1", - "identifier": 5, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "parallel_box1", - "status": True - } - ] - }, - { - "name": "load_tool sequence", - "identifier": 6, - "type_name": "sequence", - "children": [ - { - "summary": "Check if parallel_box1 is equippable", - "name": "check parallel_box1 is_equippable", - "identifier": 7, - "type_name": "condition", - "conditions": [ - { - "object_name": "parallel_box1", - "property_name": "is_equippable", - "property_value": None, - "status": True - } - ] - }, - { - "summary": "Check if left hand is free", - "name": "check left_hand is_free", - "identifier": 8, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "is_free", - "property_value": None, - "status": True - } - ] - }, - { - "summary": "Equip parallel_box1 to left hand", - "name": "load_tool(left_hand, parallel_box1)", - "identifier": 9, - "type_name": "action", - "effects": [ - { - "object_name": "left_hand", - "property_name": "is_free", - "property_value": None, - "status": False - }, - { - "object_name": "parallel_box1", - "property_name": "is_equippable", - "property_value": None, - "status": False - }, - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "parallel_box1", - "status": True - } - ] - } - ] - } - ] - }, - { - "name": "pick_up selector", - "identifier": 10, - "type_name": "selector", - "children": [ - { - "summary": "Check if parallel_box1 holds gear1", - "name": "check parallel_box1 hold gear1", - "identifier": 11, - "type_name": "condition", - "conditions": [ - { - "object_name": "parallel_box1", - "property_name": "hold", - "property_value": "gear1", - "status": True - } - ] - }, - { - "name": "pick_up sequence", - "identifier": 12, - "type_name": "sequence", - "children": [ - { - "summary": "Check if gear1 is free", - "name": "check gear1 is_free", - "identifier": 13, - "type_name": "condition", - "conditions": [ - { - "object_name": "gear1", - "property_name": "is_free", - "property_value": None, - "status": True - } - ] - }, - { - "summary": "Pick up gear1 using parallel_box1", - "name": "pick_up(left_hand, parallel_box1, gear1)", - "identifier": 14, - "type_name": "action", - "effects": [ - { - "object_name": "gear1", - "property_name": "is_free", - "property_value": None, - "status": False - }, - { - "object_name": "parallel_box1", - "property_name": "hold", - "property_value": "gear1", - "status": True - } - ] - } - ] - } - ] - }, - { - "summary": "Check if gear1 can be inserted into shaft1", - "name": "check gear1 can_insert_to shaft1", - "identifier": 15, - "type_name": "condition", - "conditions": [ - { - "object_name": "gear1", - "property_name": "can_insert_to", - "property_value": "shaft1", - "status": True - } - ] - }, - { - "summary": "Insert gear1 into shaft1", - "name": "insert(left_hand, parallel_box1, gear1, shaft1)", - "identifier": 16, - "type_name": "action", - "effects": [ - { - "object_name": "gear1", - "property_name": "is_inserted_to", - "property_value": "shaft1", - "status": True - }, - { - "object_name": "parallel_box1", - "property_name": "hold", - "property_value": "gear1", - "status": False - }, - { - "object_name": "parallel_box1", - "property_name": "is_free", - "property_value": None, - "status": True - } - ] - } - ] - } - ] - } - }, - "environment_before": { - "objects": [ - {"name": "left_hand", "properties": ["is_free"]}, - {"name": "parallel_box1", "properties": ["is_free", "is_equippable"]}, - {"name": "gear1", "properties": ["is_free"]}, - {"name": "shaft1", "properties": []} - ], - "constraints": [ - {"source": "parallel_box1", "name": "can_manipulate", "target": "gear1"}, - {"source": "gear1", "name": "can_insert_to", "target": "shaft1"} - ], - "relations": [] - }, - "environment_after": { - "objects": [ - {"name": "left_hand", "properties": []}, - {"name": "parallel_box1", "properties": ["is_free", "is_equippable"]}, - {"name": "gear1", "properties": []}, - {"name": "shaft1", "properties": []} - ], - "constraints": [ - {"source": "parallel_box1", "name": "can_manipulate", "target": "gear1"}, - {"source": "gear1", "name": "can_insert_to", "target": "shaft1"} - ], - "relations": [ - {"source": "gear1", "name": "is_inserted_to", "target": "shaft1"}, - {"source": "left_hand", "name": "hold", "target": "parallel_box1"} - ] - }, - "instruction_summary": "Equip parallel_box1 to left hand, pick up gear1, and insert it into shaft1.", - "question": "" -} -``` \ No newline at end of file diff --git a/kios_bt_planning/kios_agent/query_history/gearset/0/assistant.txt b/kios_bt_planning/kios_agent/query_history/gearset/0/assistant.txt deleted file mode 100644 index 4995e0ba..00000000 --- a/kios_bt_planning/kios_agent/query_history/gearset/0/assistant.txt +++ /dev/null @@ -1 +0,0 @@ -Understood. I will wait for further instructions before starting to work. \ No newline at end of file diff --git a/kios_bt_planning/kios_agent/query_history/gearset/0/query.txt b/kios_bt_planning/kios_agent/query_history/gearset/0/query.txt deleted file mode 100644 index 8adc1b5e..00000000 --- a/kios_bt_planning/kios_agent/query_history/gearset/0/query.txt +++ /dev/null @@ -1,16 +0,0 @@ -Start working. Resume from problem below. -""" -{"problem": "(define (problem robot_assembly_problem-problem) (:domain robot_assembly_problem-domain) (:objects parallel_box1 parallel_box2 inward_claw outward_claw no_tool - tool gear1 gear2 gear3 shaft1 shaft2 base - part left_hand - hand ) (:init (can_manipulate parallel_box1 gear1) (can_manipulate outward_claw gear2) (can_manipulate inward_claw gear3) (can_manipulate parallel_box2 shaft1) (can_manipulate no_tool shaft2) (can_screw_to leg1 seat) (can_screw_to leg2 seat) (can_insert_to back seat) (can_screw_to nut1 seat) (can_screw_to nut2 seat) (can_screw_to blub base) (can_place_to lamp blub) (can_insert_to shaft1 base) (can_insert_to shaft2 base) (can_insert_to gear3 shaft2) (can_insert_to gear2 base) (can_insert_to gear1 shaft1) (is_inserted_to shaft1 base) (is_free left_hand) (is_free parallel_box1) (is_free parallel_box2) (is_free inward_claw) (is_free outward_claw) (is_free no_tool) (is_free gear1) (is_free gear2) (is_free gear3) (is_free shaft1) (is_free shaft2) (is_free base) (is_equippable parallel_box1) (is_equippable parallel_box2) (is_equippable inward_claw) (is_equippable outward_claw) (is_equippable no_tool)) (:goal (and (is_inserted_to gear1 shaft1))) )"} -""" -The dictionary that you return should be formatted as json file. Follow these rules: -1. Always equip the proper tools before performing specific object manipulaion. -2. Make sure that each element of the ["action_instructions"] explains corresponding element of the ["action_sequence"]. Refer to the "ROBOT ACTION LIST" to understand the elements of ["action_sequence"]. -3, The length of the ["action_instructions"] list must be the same as the length of the ["action_sequence"] list. -4. Never left ',' at the end of the list. Use json boolean values, "true" or "false", instead of "True" or "False". -5. Keep track of all items listed in the "objects" section of the "environment_before" field. Please make sure you fill out the "objects" section for all listed items. -6. Use the "PREDICATE_LIST" for the states in the problem. -7. All keys of the dictionary should be double-quoted. -8. Insert ``` at the beginning and the end of the dictionary to separate it from the rest of your response. -9. Make sure that you output a consistent manipultation as a single arm robot. For example, grasping an object should not occur in successive steps. -10. When generating the behavior tree, make sure the tree has the asked structure. -Adhere to the output format I defined above. Follow the rules above. Think step by step. \ No newline at end of file diff --git a/kios_bt_planning/kios_agent/query_history/gearset/0/response.txt b/kios_bt_planning/kios_agent/query_history/gearset/0/response.txt deleted file mode 100644 index 1bd5e7f5..00000000 --- a/kios_bt_planning/kios_agent/query_history/gearset/0/response.txt +++ /dev/null @@ -1,245 +0,0 @@ - -{ - "task_cohesion": { - "action_sequence": [ - "load_tool(left_hand, parallel_box1)", - "pick_up(left_hand, parallel_box1, gear1)", - "insert(left_hand, parallel_box1, gear1, shaft1)" - ], - "action_instructions": [ - "Equip the parallel_box1 on the left hand.", - "Pick up the gear1 with the left hand using the parallel_box1.", - "Insert the gear1 into the shaft1 using the left hand equipped with the parallel_box1." - ], - "manipulated_objects": ["left_hand", "parallel_box1", "gear1", "shaft1"], - "behavior_tree": { - "name": "insert selector", - "identifier": 1, - "type_name": "selector", - "children": [ - { - "summary": "check if gear1 is inserted to shaft1", - "name": "check gear1 is_inserted_to shaft1", - "identifier": 2, - "type_name": "condition", - "conditions": [ - { - "object_name": "gear1", - "property_name": "is_inserted_to", - "property_value": "shaft1", - "status": true - } - ] - }, - { - "name": "insert sequence", - "identifier": 3, - "type_name": "sequence", - "children": [ - { - "name": "load_tool selector", - "identifier": 4, - "type_name": "selector", - "children": [ - { - "summary": "check if parallel_box1 is held by left hand", - "name": "check left_hand hold parallel_box1", - "identifier": 5, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "parallel_box1", - "status": true - } - ] - }, - { - "name": "load_tool sequence", - "identifier": 6, - "type_name": "sequence", - "children": [ - { - "summary": "check if parallel_box1 is equippable", - "name": "check parallel_box1 is_equippable", - "identifier": 7, - "type_name": "condition", - "conditions": [ - { - "object_name": "parallel_box1", - "property_name": "is_equippable", - "property_value": null, - "status": true - } - ] - }, - { - "summary": "check if left hand is free", - "name": "check left_hand is_free", - "identifier": 8, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "is_free", - "property_value": null, - "status": true - } - ] - }, - { - "summary": "equip parallel_box1 to left hand", - "name": "load_tool(left_hand, parallel_box1)", - "identifier": 9, - "type_name": "action", - "effects": [ - { - "object_name": "left_hand", - "property_name": "is_free", - "property_value": null, - "status": false - }, - { - "object_name": "parallel_box1", - "property_name": "is_equippable", - "property_value": null, - "status": false - }, - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "parallel_box1", - "status": true - } - ] - } - ] - } - ] - }, - { - "name": "pick_up selector", - "identifier": 10, - "type_name": "selector", - "children": [ - { - "summary": "check if parallel_box1 holds gear1", - "name": "check parallel_box1 hold gear1", - "identifier": 11, - "type_name": "condition", - "conditions": [ - { - "object_name": "parallel_box1", - "property_name": "hold", - "property_value": "gear1", - "status": true - } - ] - }, - { - "name": "pick_up sequence", - "identifier": 12, - "type_name": "sequence", - "children": [ - { - "summary": "check if gear1 is free", - "name": "check gear1 is_free", - "identifier": 13, - "type_name": "condition", - "conditions": [ - { - "object_name": "gear1", - "property_name": "is_free", - "property_value": null, - "status": true - } - ] - }, - { - "summary": "pick up gear1 using parallel_box1", - "name": "pick_up(left_hand, parallel_box1, gear1)", - "identifier": 14, - "type_name": "action", - "effects": [ - { - "object_name": "parallel_box1", - "property_name": "hold", - "property_value": "gear1", - "status": true - }, - { - "object_name": "gear1", - "property_name": "is_free", - "property_value": null, - "status": false - } - ] - } - ] - } - ] - }, - { - "summary": "insert gear1 into shaft1", - "name": "insert(left_hand, parallel_box1, gear1, shaft1)", - "identifier": 15, - "type_name": "action", - "effects": [ - { - "object_name": "gear1", - "property_name": "is_inserted_to", - "property_value": "shaft1", - "status": true - }, - { - "object_name": "parallel_box1", - "property_name": "hold", - "property_value": "gear1", - "status": false - }, - { - "object_name": "parallel_box1", - "property_name": "is_free", - "property_value": null, - "status": true - } - ] - } - ] - } - ] - } - }, - "environment_before": { - "objects": [ - {"name": "gear1", "properties": ["is_free"]}, - {"name": "shaft1", "properties": ["is_free"]}, - {"name": "parallel_box1", "properties": ["is_free", "is_equippable"]}, - {"name": "left_hand", "properties": ["is_free"]} - ], - "constraints": [ - {"source": "parallel_box1", "name": "can_manipulate", "target": "gear1"}, - {"source": "gear1", "name": "can_insert_to", "target": "shaft1"} - ], - "relations": [] - }, - "environment_after": { - "objects": [ - {"name": "gear1", "properties": []}, - {"name": "shaft1", "properties": []}, - {"name": "parallel_box1", "properties": ["is_free", "is_equippable"]}, - {"name": "left_hand", "properties": []} - ], - "constraints": [ - {"source": "parallel_box1", "name": "can_manipulate", "target": "gear1"}, - {"source": "gear1", "name": "can_insert_to", "target": "shaft1"} - ], - "relations": [ - {"source": "gear1", "name": "is_inserted_to", "target": "shaft1"}, - {"source": "left_hand", "name": "hold", "target": "parallel_box1"} - ] - }, - "instruction_summary": "Equip parallel_box1 to left hand, pick up gear1, and insert it into shaft1.", - "question": "" -} diff --git a/kios_bt_planning/kios_agent/query_history/gearset/0/user.txt b/kios_bt_planning/kios_agent/query_history/gearset/0/user.txt deleted file mode 100644 index 8adc1b5e..00000000 --- a/kios_bt_planning/kios_agent/query_history/gearset/0/user.txt +++ /dev/null @@ -1,16 +0,0 @@ -Start working. Resume from problem below. -""" -{"problem": "(define (problem robot_assembly_problem-problem) (:domain robot_assembly_problem-domain) (:objects parallel_box1 parallel_box2 inward_claw outward_claw no_tool - tool gear1 gear2 gear3 shaft1 shaft2 base - part left_hand - hand ) (:init (can_manipulate parallel_box1 gear1) (can_manipulate outward_claw gear2) (can_manipulate inward_claw gear3) (can_manipulate parallel_box2 shaft1) (can_manipulate no_tool shaft2) (can_screw_to leg1 seat) (can_screw_to leg2 seat) (can_insert_to back seat) (can_screw_to nut1 seat) (can_screw_to nut2 seat) (can_screw_to blub base) (can_place_to lamp blub) (can_insert_to shaft1 base) (can_insert_to shaft2 base) (can_insert_to gear3 shaft2) (can_insert_to gear2 base) (can_insert_to gear1 shaft1) (is_inserted_to shaft1 base) (is_free left_hand) (is_free parallel_box1) (is_free parallel_box2) (is_free inward_claw) (is_free outward_claw) (is_free no_tool) (is_free gear1) (is_free gear2) (is_free gear3) (is_free shaft1) (is_free shaft2) (is_free base) (is_equippable parallel_box1) (is_equippable parallel_box2) (is_equippable inward_claw) (is_equippable outward_claw) (is_equippable no_tool)) (:goal (and (is_inserted_to gear1 shaft1))) )"} -""" -The dictionary that you return should be formatted as json file. Follow these rules: -1. Always equip the proper tools before performing specific object manipulaion. -2. Make sure that each element of the ["action_instructions"] explains corresponding element of the ["action_sequence"]. Refer to the "ROBOT ACTION LIST" to understand the elements of ["action_sequence"]. -3, The length of the ["action_instructions"] list must be the same as the length of the ["action_sequence"] list. -4. Never left ',' at the end of the list. Use json boolean values, "true" or "false", instead of "True" or "False". -5. Keep track of all items listed in the "objects" section of the "environment_before" field. Please make sure you fill out the "objects" section for all listed items. -6. Use the "PREDICATE_LIST" for the states in the problem. -7. All keys of the dictionary should be double-quoted. -8. Insert ``` at the beginning and the end of the dictionary to separate it from the rest of your response. -9. Make sure that you output a consistent manipultation as a single arm robot. For example, grasping an object should not occur in successive steps. -10. When generating the behavior tree, make sure the tree has the asked structure. -Adhere to the output format I defined above. Follow the rules above. Think step by step. \ No newline at end of file diff --git a/kios_bt_planning/kios_agent/tools_tutorial.py b/kios_bt_planning/kios_agent/tools_tutorial.py deleted file mode 100644 index 3fc1c7ed..00000000 --- a/kios_bt_planning/kios_agent/tools_tutorial.py +++ /dev/null @@ -1,165 +0,0 @@ -# Import things that are needed generically -from langchain.pydantic_v1 import BaseModel, Field -from langchain.tools import BaseTool, StructuredTool, tool - -from typing import Optional, Type - -from langchain.callbacks.manager import ( - AsyncCallbackManagerForToolRun, - CallbackManagerForToolRun, -) - - -# * tool decorator -@tool -def search(query: str) -> str: - """Look up things online.""" - return "LangChain" - - -# print(search.name) -# print(search.description) -# print(search.args) - - -@tool -def multiply(a: int, b: int) -> int: - """Multiply two numbers.""" - return a * b - - -# print(multiply.name) -# print(multiply.description) -# print(multiply.args) - - -class SearchInput(BaseModel): - query: str = Field(description="should be a search query") - - -@tool("search-tool", args_schema=SearchInput, return_direct=True) -def search(query: str) -> str: - """Look up things online.""" - return "LangChain" - - -# print(search.name) -# print(search.description) -# print(search.args) -# print(search.return_direct) - - -# * BaseTool -class SearchInput(BaseModel): - query: str = Field(description="should be a search query") - - -class CalculatorInput(BaseModel): - a: int = Field(description="first number") - b: int = Field(description="second number") - - -class CustomSearchTool(BaseTool): - name = "custom_search" - description = "useful for when you need to answer questions about current events" - args_schema: Type[BaseModel] = SearchInput - - def _run( - self, query: str, run_manager: Optional[CallbackManagerForToolRun] = None - ) -> str: - """Use the tool.""" - return "LangChain" - - async def _arun( - self, query: str, run_manager: Optional[AsyncCallbackManagerForToolRun] = None - ) -> str: - """Use the tool asynchronously.""" - raise NotImplementedError("custom_search does not support async") - - -class CustomCalculatorTool(BaseTool): - name = "Calculator" - description = "useful for when you need to answer questions about math" - args_schema: Type[BaseModel] = CalculatorInput - return_direct: bool = True - - def _run( - self, a: int, b: int, run_manager: Optional[CallbackManagerForToolRun] = None - ) -> str: - """Use the tool.""" - return a * b - - async def _arun( - self, - a: int, - b: int, - run_manager: Optional[AsyncCallbackManagerForToolRun] = None, - ) -> str: - """Use the tool asynchronously.""" - raise NotImplementedError("Calculator does not support async") - - -# search = CustomSearchTool() -# print(search.name) -# print(search.description) -# print(search.args) - -# multiply = CustomCalculatorTool() -# print(multiply.name) -# print(multiply.description) -# print(multiply.args) -# print(multiply.return_direct) - - -# * StructuredTool -def search_function(query: str): - return "LangChain" - - -search = StructuredTool.from_function( - func=search_function, - name="Search", - args_schema=SearchInput, - description="useful for when you need to answer questions about current events", - # * coroutine= ... <- you can specify an async method if desired as well -) - -# print(search.name) -# print(search.description) -# print(search.args) - -# * handling errors -from langchain_core.tools import ToolException - - -def search_tool1(s: str): - raise ToolException("The search tool1 is not available.") - - -search = StructuredTool.from_function( - func=search_tool1, - name="Search_tool1", - description="A bad tool", - handle_tool_error=True, -) - -# search.run("test") - - -def _handle_error(error: ToolException) -> str: - return ( - "The following errors occurred during tool execution:" - + error.args[0] - + "Please try another tool." - ) - - -search = StructuredTool.from_function( - func=search_tool1, - name="Search_tool1", - description="A bad tool", - handle_tool_error=_handle_error, -) - -# print(search.run("test")) -# search.run("test") diff --git a/kios_bt_planning/kios_bt/__init__.py b/kios_bt_planning/kios_bt/__init__.py deleted file mode 100644 index e69de29b..00000000 diff --git a/kios_bt_planning/kios_bt/behavior_nodes.py b/kios_bt_planning/kios_bt/behavior_nodes.py deleted file mode 100644 index 2e353c60..00000000 --- a/kios_bt_planning/kios_bt/behavior_nodes.py +++ /dev/null @@ -1,438 +0,0 @@ -""" -action nodes and condition nodes that used for generating the nodes in subtrees -""" - -############################################################################## -# Imports -############################################################################## - -# for multiprocessing -import atexit -import multiprocessing -import multiprocessing.connection -from multiprocessing import Manager - -# for abstract class -from abc import ABC, abstractmethod -from typing import Any - -# for testing -import time - -# pytrees -import py_trees.common -import py_trees.console as console - -# kios -from kios_utils.task import * -from kios_bt.data_types import ( - Action, - Condition, -) -from kios_world.world_interface import WorldInterface -from kios_robot.robot_interface import RobotInterface -from kios_robot.robot_command import RobotCommand -from kios_robot.mios_async import fake_robot_command_monitor, robot_command_monitor - -############################################################################## -# Classes -############################################################################## - - -class BehaviorNode( - py_trees.behaviour.Behaviour, - # ABC, -): - """kios_bt template node.""" - - def __init__( - self, - behavior_name: str, - world_interface: WorldInterface, - robot_interface: RobotInterface, - ): - """Configure the name of the behaviour.""" - self.behavior_name = behavior_name - super(BehaviorNode, self).__init__(self.behavior_name) - self.monitor = None - self.world_interface = world_interface - self.robot_interface = robot_interface - - def terminate(self, new_status: py_trees.common.Status) -> None: - """called after execution or when interrupted.""" - # * stop the monitor process, regardless of the result - if self.monitor is None: - pass - # self.debug(f'Node "{self.behavior_name}" has no monitor to terminate') - else: - self.monitor.terminate() - - self.logger.debug( - "%s.terminate()[%s->%s]" - % (self.__class__.__name__, self.status, new_status) - ) - - -class ActionNode(BehaviorNode): - """Demonstrates the at-a-distance style action behaviour.""" - - action: Action - - world_interface: WorldInterface - robot_interface: RobotInterface - - monitor: ... - shared_data: Any - robot_command: RobotCommand - multiprocessing_manager: Any - parent_connection: multiprocessing.connection.Connection - child_connection: multiprocessing.connection.Connection - - def __init__( - self, - action: Action, - world_interface: WorldInterface, - robot_interface: RobotInterface, - ): - self.action = action - """Configure the name of the behaviour.""" - self.identifier = action.identifier - self.behavior_name = self.action.name - super().__init__(self.behavior_name, world_interface, robot_interface) - - self.monitor = None - - # * setup the task - - self.multiprocessing_manager = Manager() - self.shared_data = self.multiprocessing_manager.dict() - self.robot_command = self.robot_interface.generate_robot_command( - action, self.shared_data - ) - self.robot_command.initialize() - self.logger.debug("%s.__init__()" % (self.__class__.__name__)) - - def take_effect(self): - """ - interact with the world interface to exert the effects - """ - self.logger.info(f"Try to exert the effects of action {self.behavior_name}.") - self.world_interface.take_effect(self.action) - - def setup(self, **kwargs: int) -> None: - # setup the task - # skipped. the task is setup in the __init__ function - self.logger.debug("%s.setup()" % (self.__class__.__name__)) - - def initialise(self) -> None: - # else, reset the task and start the external process - self.logger.debug("%s.initialise()" % (self.__class__.__name__)) - self.robot_command.interrupt() - # * launch the subprocess, start the mios skill execution - self.parent_connection, self.child_connection = multiprocessing.Pipe() - self.monitor = multiprocessing.Process( - target=robot_command_monitor, - args=( - self.robot_command, - self.child_connection, - ), - ) - atexit.register(self.monitor.terminate) - self.monitor.start() - self.logger.info(f"Action node {self.behavior_name} started.") - - def update(self) -> py_trees.common.Status: - """Increment the counter, monitor and decide on a new status.""" - self.logger.debug("%s.update()" % (self.__class__.__name__)) - new_status = py_trees.common.Status.RUNNING - - # ! BBREMOVE this part. now startup won't be checked anymore. - # # * check the result of the startup of the task - # task_start_response = self.task.shared_data["task_start_response"] - # print(task_start_response) - - # if task_start_response is not None: - # if bool(task_start_response["result"]["result"]) == False: - # self.logger.debug("Task startup failed") - # new_status = py_trees.common.Status.FAILURE - # return new_status - - # if bool(task_start_response["result"]["result"]) == True: - # print("Task startup succeeded") - - # else: - # # ! this should never happen - # self.logger.debug("Task startup in progress") - # self.logger.debug("ERRRRRRRRRRRRRRRRRRRRRRRORRR") - # new_status = py_trees.common.Status.RUNNING - # return new_status - - # * check if the task is finished - if self.parent_connection.poll(): - self.result = self.parent_connection.recv().pop() # ! here only bool - if self.result == True: - self.logger.info(f'Action "{self.behavior_name}" finished successfully') - new_status = py_trees.common.Status.SUCCESS - # * exert the effects - self.take_effect() - # ! I think here the action node should not return a success. it should always return running. - # ! and it should "task effect". - # ! the target condition node in the selector will be fulfilled by the "task effect" - # ! the action node will be interrupted by the selector, and the tick goes on... - # ! YOU WILL UNDERSTAND ME SOONER OR LATER... - else: - self.logger.info(f'Action "{self.behavior_name}" failed with error') - new_status = py_trees.common.Status.FAILURE - - return new_status - - -class ConditionNode(BehaviorNode): - """abstract condition node.""" - - def __init__( - self, - condition: Condition, - world_interface: WorldInterface, - robot_interface: RobotInterface, - ): - self.condition = condition - """Configure the name of the behaviour.""" - self.identifier = condition.identifier - self.behavior_name = condition.name - super().__init__(self.behavior_name, world_interface, robot_interface) - - self.logger.debug("%s.__init__()" % (self.__class__.__name__)) - - def register_predicates(self) -> None: - # don't need this for now - pass - - def setup(self, **kwargs: int) -> None: - # register the predicates on the blackboard here - # self.register_predicates() - # self.logger.debug( - # "%s.setup()->register the predicates" % (self.__class__.__name__) - # ) - pass - - def initialise(self) -> None: - self.logger.debug("%s.initialise()" % (self.__class__.__name__)) - # may implement some observing actions here - - def update(self) -> py_trees.common.Status: - """Increment the counter, monitor and decide on a new status.""" - self.logger.debug("%s.update()" % (self.__class__.__name__)) - new_status = py_trees.common.Status.FAILURE - - result = self.world_interface.check_condition(self.condition) - - if result == True: - self.logger.info(f'Condition "{self.behavior_name}" is satisfied') - new_status = py_trees.common.Status.SUCCESS - - else: - self.logger.info(f'Condition "{self.behavior_name}" is not satisfied') - new_status = py_trees.common.Status.FAILURE - - return new_status - - -class ActionNodeTest(ActionNode): - def __init__( - self, - action: Action, - world_interface: WorldInterface, - robot_interface: RobotInterface, - ): - self.action = action - """Configure the name of the behaviour.""" - self.identifier = action.identifier - self.behavior_name = self.action.name - super(ActionNode, self).__init__( - self.behavior_name, world_interface, robot_interface - ) - - # * setup the task - self.multiprocessing_manager = Manager() - self.shared_data = self.multiprocessing_manager.dict() - self.robot_command = None - self.logger.debug("%s.__init__()" % (self.__class__.__name__)) - - def register_predicates(self) -> None: - pass - - def setup(self, **kwargs: int) -> None: - self.logger.debug("%s.setup()" % (self.__class__.__name__)) - - def initialise(self) -> None: - # else, reset the task and start the external process - self.logger.debug("%s.initialise()" % (self.__class__.__name__)) - self.robot_command.interrupt() - # * launch the subprocess, start the mios skill execution - self.parent_connection, self.child_connection = multiprocessing.Pipe() - self.monitor = multiprocessing.Process( - target=fake_robot_command_monitor, - args=( - self.robot_command, - self.child_connection, - ), - ) - atexit.register(self.monitor.terminate) - self.monitor.start() - self.logger.info(f"Action node {self.behavior_name} started.") - - def update(self) -> py_trees.common.Status: - """Increment the counter, monitor and decide on a new status.""" - self.logger.debug("%s.update()" % (self.__class__.__name__)) - new_status = py_trees.common.Status.RUNNING - - # * check if the task is finished - if self.parent_connection.poll(): - self.result = self.parent_connection.recv().pop() # ! here only bool - if self.result == True: - self.logger.info(f'Action "{self.behavior_name}" finished successfully') - new_status = py_trees.common.Status.SUCCESS - # * exert the effects - self.take_effect() - else: - self.logger.info(f'Action "{self.behavior_name}" failed with error') - new_status = py_trees.common.Status.FAILURE - - return new_status - - -class ActionNodeSim(ActionNode): - success_flag: bool - - @staticmethod - def from_action_node(action_node: ActionNode) -> "ActionNodeSim": - return ActionNodeSim(action_node.action, action_node.world_interface) - - def __init__( - self, - action: Action, - world_interface: WorldInterface, - robot_interface: RobotInterface = None, # not needed for simulation - ): - self.success_flag = False - self.tick_times = 0 - self.tick_times_target = 3 - self.hasTakenEffect = False - self.action = action - """Configure the name of the behaviour.""" - self.identifier = action.identifier - self.behavior_name = self.action.name - super(ActionNode, self).__init__( - self.behavior_name, world_interface, robot_interface - ) - self.logger.debug("%s.__init__()" % (self.__class__.__name__)) - - def setup(self, **kwargs: int) -> None: - self.logger.debug("%s.setup()" % (self.__class__.__name__)) - - def initialise(self) -> None: - self.logger.debug("%s.initialise()" % (self.__class__.__name__)) - self.tick_times = 0 - self.hasTakenEffect = False - self.logger.info(f"Sim action node {self.behavior_name} started.") - - def take_effect(self): - """ - interact with the world interface to exert the effects - """ - if self.hasTakenEffect: - return - self.logger.info(f"Try to exert the effects of action {self.behavior_name}.") - self.world_interface.take_effect(self.action) - self.hasTakenEffect = True # ! warning - - def update(self) -> py_trees.common.Status: - """ - running ---> success - """ - self.logger.debug("%s.update()" % (self.__class__.__name__)) - - if self.tick_times >= self.tick_times_target: - self.logger.info(f'Action "{self.behavior_name}" finished successfully') - # new_status = py_trees.common.Status.SUCCESS - new_status = py_trees.common.Status.RUNNING - # ! As I said, the action node should always return running. - # ! this will help to catch the mistakes in action effects. - self.take_effect() - - else: - self.tick_times += 1 - new_status = py_trees.common.Status.RUNNING - - return new_status - - -class ActionNodeOnlySuccess(ActionNode): - - @staticmethod - def from_action_node(action_node: ActionNode) -> "ActionNodeSim": - return ActionNodeSim(action_node.action, action_node.world_interface) - - def __init__( - self, - action: Action, - world_interface: WorldInterface, - robot_interface: RobotInterface = None, # not needed for simulation - ): - self.success_flag = False - self.tick_times = 0 - self.tick_times_target = 3 - self.hasTakenEffect = False - self.action = action - """Configure the name of the behaviour.""" - self.identifier = action.identifier - self.behavior_name = self.action.name - super(ActionNode, self).__init__( - self.behavior_name, world_interface, robot_interface - ) - self.logger.debug("%s.__init__()" % (self.__class__.__name__)) - - def setup(self, **kwargs: int) -> None: - self.logger.debug("%s.setup()" % (self.__class__.__name__)) - - def initialise(self) -> None: - self.logger.debug("%s.initialise()" % (self.__class__.__name__)) - self.tick_times = 0 - self.hasTakenEffect = False - self.logger.info(f"Sim action node {self.behavior_name} started.") - - def take_effect(self): - """ - interact with the world interface to exert the effects - """ - if self.hasTakenEffect: - return - self.logger.info(f"Try to exert the effects of action {self.behavior_name}.") - self.world_interface.take_effect(self.action) - self.hasTakenEffect = True - - def update(self) -> py_trees.common.Status: - new_status = py_trees.common.Status.SUCCESS - return new_status - - -############################################################################## -# Main -############################################################################## - - -# def main() -> None: -# """Entry point for the demo script.""" - -# py_trees.logging.level = py_trees.logging.Level.DEBUG - -# action = ActionNode(name="Action") -# action.setup() -# try: -# for _unused_i in range(0, 12): -# action.tick_once() -# time.sleep(0.5) -# print("\n") -# except KeyboardInterrupt: -# pass diff --git a/kios_bt_planning/kios_bt/bt_factory.py b/kios_bt_planning/kios_bt/bt_factory.py deleted file mode 100644 index cc8a6e90..00000000 --- a/kios_bt_planning/kios_bt/bt_factory.py +++ /dev/null @@ -1,404 +0,0 @@ -from kios_bt.data_types import ( - # ActionInstance, - # GroundedAction, - Action, - Condition, - ObjectProperty, -) -import copy -import py_trees -import json -import re -from pprint import pprint -from typing import List, Dict, Any, Tuple, Optional -from kios_bt.behavior_nodes import ActionNode, ConditionNode, ActionNodeTest - -from kios_world.world_interface import WorldInterface -from kios_robot.robot_interface import RobotInterface - -from kios_utils.skeleton_parser import ( - parse_node_name, - ParsedNode, - ground_action, - parse_node_type, -) - - -""" -bt_stewardship: the stewardship of the behavior tree, kios class. -behavior_tree: py_trees.trees.BehaviourTree, the wrapper of the tree root -tree_root: py_trees.behaviour.Behaviour, the root of the tree -""" - - -def id_generator(limit=1000): - """Generate unique IDs from 0 up to limit-1.""" - for id in range(limit): - yield id - - -generator = id_generator() # Create a generator - - -class BehaviorTreeFactory: - visualization_only: bool - roster: Dict[int, Any] = {} # * roster hasn't be tested yet. - node_skeleton_dict: dict[any, str] = {} - - world_interface: WorldInterface = None - robot_interface: RobotInterface = None - - id_generator = id_generator() - - def __init__( - self, - world_interface: WorldInterface = None, - robot_interface: RobotInterface = None, - visualization_only: bool = False, - ): - """ - initialize the subtree factory with lists of preconditions, - actions, and effects that are available to the factory - """ - - if world_interface is None: - self.world_interface = WorldInterface() - self.world_interface.initialize() - else: - self.world_interface = world_interface - - if robot_interface is None: - self.robot_interface = RobotInterface() - self.robot_interface.initialize() - else: - self.robot_interface = robot_interface - - self.visualization_only = visualization_only - - def initialize(self): - pass - - ########################################################## - # ! visualization only, dirty imp. - def parse_name(self, name: str) -> str: - """ - get the type od the node from the name - """ - pattern = r"(selector|sequence|target|precondition|condition|action)" - match = re.search(pattern, name) - if match: - return match.group(0) - else: - raise ValueError(f"unable to parse the type of the node from {name}") - - def from_json_to_simple_bt(self, json_data: dict): - """ - generate a behavior tree from a json file (but you need to parse it first) - can be a skeleton or a complete bt - """ - if json_data.get("type_name") is None: - # parse the type from "name" - if json_data.get("name") is None: - raise ValueError( - f'both "type_name" and "name" are missing for node {json_data}!!!' - ) - - json_data["type_name"] = self.parse_name(json_data["name"]) - - if json_data["type_name"] == "selector": - control_flow_node = py_trees.composites.Selector( - name=json_data["name"], memory=False - ) - for child in json_data["children"]: - child_node = self.from_json_to_simple_bt(child) - control_flow_node.add_child(child_node) - - return control_flow_node - - elif json_data["type_name"] == "sequence": - control_flow_node = py_trees.composites.Sequence( - name=json_data["name"], memory=False - ) - for child in json_data["children"]: - child_node = self.from_json_to_simple_bt(child) - control_flow_node.add_child(child_node) - - return control_flow_node - - elif json_data["type_name"] in ["precondition", "condition", "target"]: - return self.__from_json_to_simple_condition_node(json_data) - elif json_data["type_name"] == "action": - return self.__from_json_to_simple_action_node(json_data) - else: - raise ValueError(f"unknown type name {json_data['type_name']}") - - def __from_json_to_simple_condition_node(self, json_data: dict) -> ConditionNode: - """ - from json to condition node, and add it to the roster - """ - condition = Condition( - summary=json_data["summary"], - identifier=0, - name=json_data["name"], - conditions=[], - ) - condition_node = ConditionNode( - condition, - self.world_interface, - self.robot_interface, - ) - return condition_node - - def __from_json_to_simple_action_node( - self, json_data: Dict[str, Any] - ) -> ActionNodeTest: - """ - from json to action node, and add it to the roster - """ - effects = [] - action = Action( - summary=json_data["summary"], - identifier=0, - name=json_data["name"], - effects=[], - ) - # ! BBHACK - action_node = ActionNodeTest( - action, - self.world_interface, - self.robot_interface, - ) - return action_node - - ############################################################# - # * this is for a complete json behavior tree - # * json to bt_stw - def from_json_to_behavior_tree( - self, json_data: dict - ) -> Tuple[Dict[str, Any], py_trees.trees.BehaviourTree]: - - tree_root = self.from_json_to_tree_root(json_data) - behavior_tree = py_trees.trees.BehaviourTree(tree_root) - return [self.roster, behavior_tree] - - # * json to bt - def from_json_to_tree_root(self, json_data: dict) -> py_trees.behaviour.Behaviour: - """ - generate a behavior tree from a json file (but you need to parse it first) - """ - if json_data["type_name"] == "selector": - control_flow_node = py_trees.composites.Selector( - name=json_data["name"], memory=False - ) - for child in json_data["children"]: - child_node = self.from_json_to_tree_root(child) - control_flow_node.add_child(child_node) - - self.roster[json_data["identifier"]] = control_flow_node - return control_flow_node - - elif json_data["type_name"] == "sequence": - control_flow_node = py_trees.composites.Sequence( - name=json_data["name"], memory=False - ) - for child in json_data["children"]: - child_node = self.from_json_to_tree_root(child) - control_flow_node.add_child(child_node) - - self.roster[json_data["identifier"]] = control_flow_node - return control_flow_node - - elif json_data["type_name"] == "condition": - return self.from_json_to_condition_node(json_data) - elif json_data["type_name"] == "action": - return self.from_json_to_action_node(json_data) - - def from_json_to_action_node(self, json_data: Dict[str, Any]) -> ActionNode: - """ - from json to action node, and add it to the roster - """ - effects = [] - for effect in json_data["effects"]: - _effect = ObjectProperty( - object_name=effect["object_name"], - property_name=effect["property_name"], - property_value=effect["property_value"], - status=effect["status"], - ) - effects.append(_effect) - action = Action( - summary=json_data["summary"], - identifier=json_data["identifier"], - name=json_data["name"], - effects=effects, - ) - # ! BBHACK - action_node = ActionNode( - action, - self.world_interface, - self.robot_interface, - ) - self.roster[json_data["identifier"]] = action_node - return action_node - - def from_json_to_condition_node(self, json_data: dict) -> ConditionNode: - """ - from json to condition node, and add it to the roster - """ - conditions = [] - for condition in json_data["conditions"]: - _condition = ObjectProperty( - object_name=condition["object_name"], - property_name=condition["property_name"], - property_value=condition["property_value"], - status=condition["status"], - ) - conditions.append(_condition) - condition = Condition( - summary=json_data["summary"], - identifier=json_data["identifier"], - name=json_data["name"], - conditions=conditions, - ) - condition_node = ConditionNode( - condition, - self.world_interface, - self.robot_interface, - ) - self.roster[json_data["identifier"]] = condition_node - return condition_node - - ################################* new parse method ################################ - # * json skeleton to bt_stw - def from_skeleton_to_behavior_tree( - self, json_data: dict - ) -> Tuple[Dict[str, Any], py_trees.trees.BehaviourTree, Optional[Dict[int, str]]]: - - # ! clear roster here - self.roster = {} - self.node_skeleton_dict = {} - - tree_root = self.from_skeleton_to_tree_root(json_data) - behavior_tree = py_trees.trees.BehaviourTree(tree_root) - return [self.roster, behavior_tree, self.node_skeleton_dict] - - def from_skeleton_to_tree_root(self, skeleton: dict): - """ - generate a bt based on a skeleton - """ - if "name" not in skeleton.keys(): - raise ValueError("name is missing in the skeleton.") - parsed_type = parse_node_type(skeleton["name"]) - if parsed_type == "selector": - control_flow_node = py_trees.composites.Selector( - name=skeleton["name"], memory=False - ) - - # * skip identifier since it is not a must for a skeleton - if "identifier" not in skeleton.keys(): - skeleton["identifier"] = next(self.id_generator) - - self.roster[skeleton["identifier"]] = control_flow_node - # * summary - self.node_skeleton_dict[control_flow_node.id] = { - "summary": skeleton["summary"], - "name": skeleton["name"], - } - - for child in skeleton["children"]: - child_node = self.from_skeleton_to_tree_root(child) - control_flow_node.add_child(child_node) - return control_flow_node - elif parsed_type == "sequence": - control_flow_node = py_trees.composites.Sequence( - name=skeleton["name"], memory=False - ) - if "identifier" not in skeleton.keys(): - skeleton["identifier"] = next(self.id_generator) - self.roster[skeleton["identifier"]] = control_flow_node - # * summary - self.node_skeleton_dict[control_flow_node.id] = { - "summary": skeleton["summary"], - "name": skeleton["name"], - } - for child in skeleton["children"]: - child_node = self.from_skeleton_to_tree_root(child) - control_flow_node.add_child(child_node) - return control_flow_node - elif parsed_type in ["precondition", "condition", "target"]: - parsed_node = parse_node_name(skeleton["name"]) - return self.from_skeleton_to_condition_node(skeleton, parsed_node) - elif parsed_type == "action": - parsed_node = parse_node_name(skeleton["name"]) - return self.from_skeleton_to_action_node(skeleton, parsed_node) - else: - raise ValueError(f"unknown type name {parsed_node.typename}") - - def from_skeleton_to_condition_node( - self, skeleton: dict, parsed_node: ParsedNode - ) -> ConditionNode: - """ - from skeleton to condition node - """ - pprint(skeleton) - # ! here we cheat. all the condition check are now only true. - op = ObjectProperty( - object_name=parsed_node.params[0], - property_name=parsed_node.itemname, - property_value=( - parsed_node.params[1] if len(parsed_node.params) == 2 else None - ), - status=True, - ) - if "identifier" not in skeleton.keys(): - skeleton["identifier"] = next(self.id_generator) - - condition = Condition( - summary=skeleton["summary"], - identifier=skeleton["identifier"], - name=skeleton["name"], - conditions=[op], - ) - condition_node = ConditionNode( - condition, - self.world_interface, - self.robot_interface, - ) - self.roster[skeleton["identifier"]] = condition_node - # * summary - self.node_skeleton_dict[condition_node.id] = { - "summary": skeleton["summary"], - "name": skeleton["name"], - } - - return condition_node - - def from_skeleton_to_action_node( - self, skeleton: dict, parsed_node: ParsedNode - ) -> ActionNode: - """ - from skeleton to action node - """ - _, effects = ground_action(parsed_node.itemname, parsed_node.params) - if "identifier" not in skeleton.keys(): - skeleton["identifier"] = next(self.id_generator) - - action = Action( - summary=skeleton["summary"], - identifier=skeleton["identifier"], - name=skeleton["name"], - effects=effects, - ) - action_node = ActionNode( - action, - self.world_interface, - self.robot_interface, - ) - self.roster[skeleton["identifier"]] = action_node - # * summary - self.node_skeleton_dict[action_node.id] = { - "summary": skeleton["summary"], - "name": skeleton["name"], - } - - return action_node diff --git a/kios_bt_planning/kios_bt/bt_simulator.py b/kios_bt_planning/kios_bt/bt_simulator.py deleted file mode 100644 index f9f21166..00000000 --- a/kios_bt_planning/kios_bt/bt_simulator.py +++ /dev/null @@ -1,20 +0,0 @@ -# from kios_bt.behavior_nodes import ActionNode, ConditionNode, ActionNodeTest -# from kios_robot.robot_interface import RobotInterface -# from kios_world.world_interface import WorldInterface -# import py_trees - -# import copy -# TODO: abstract the sim methods here later. - -# class BTSimulator: -# def __init__( -# self, -# bt: py_trees.trees.BehaviourTree, -# ): - -# self.bt = copy.deepcopy(bt) -# self.world_interface = copy.deepcopy(world_interface) -# self.robot_interface = copy.deepcopy(robot_interface) - -# def run(self): -# self.bt.run() diff --git a/kios_bt_planning/kios_bt/bt_stewardship.py b/kios_bt_planning/kios_bt/bt_stewardship.py deleted file mode 100644 index 93df21fc..00000000 --- a/kios_bt_planning/kios_bt/bt_stewardship.py +++ /dev/null @@ -1,621 +0,0 @@ -""" -a class that manages the behavior tree, help to localize the behavior nodes in -the behavior tree and help to modify the tree. (interfaces opened for planner/LLM) - -should provide: - -- simulator fake run -- tree mod -- tree visualization -- test run -""" - -from typing import List, Set, Dict, Any, Tuple, Optional - -from kios_bt.behavior_nodes import ( - ActionNode, - ConditionNode, - ActionNodeTest, - ActionNodeSim, - ActionNodeOnlySuccess, -) -from kios_bt.bt_factory import BehaviorTreeFactory -from kios_world.world_interface import WorldInterface -from kios_robot.robot_interface import RobotInterface -from kios_bt.data_types import TreeResult, Action, Condition - -from kios_utils.pddl_problem_parser import ( - parse_problem_objects, - parse_problem_init, - parse_problem, -) - -import py_trees - -import functools -import copy -import sched -import time - - -class BehaviorTreeStewardship: - behavior_tree: py_trees.trees.BehaviourTree = None # for pytree bt functionality - behaviortree_factory: BehaviorTreeFactory = None # for the roster - bt_json: dict = None # for tree regeneration. - bt_skeleton: dict = None # for tree regeneration. - world_interface: WorldInterface = None # for world states - robot_interface: RobotInterface = None # for robot actions - roster: Dict[int, Any] = {} # for tree mod - skeleton_dict = {} # for tree to dict - - def __init__( - self, - behaviortree_factory: BehaviorTreeFactory = None, - world_interface: WorldInterface = None, - robot_interface: RobotInterface = None, # ! set up the scene before passing in! - ) -> None: - if robot_interface is None: - raise Exception("robot_interface is not set") - else: - self.robot_interface = robot_interface - - if world_interface is None: - # ! now the world interface is a required parameter - raise Exception("world_interface is not set") - else: - self.world_interface = world_interface - - if behaviortree_factory is None: - self.behaviortree_factory = BehaviorTreeFactory( - None, - world_interface=self.world_interface, - robot_interface=self.robot_interface, - ) - self.behaviortree_factory.initialize() - else: - self.behaviortree_factory = behaviortree_factory - - def initialize(self): - py_trees.logging.level = py_trees.logging.Level.DEBUG - - pass - - def load_bt_json(self, bt_json: dict) -> None: - self.bt_json = bt_json - - def generate_behavior_tree(self) -> None: - self.roster, self.behavior_tree = ( - self.behaviortree_factory.from_json_to_behavior_tree(self.bt_json) - ) - - def generate_behavior_tree_from_json(self, json_data: dict) -> None: - self.roster, self.behavior_tree = ( - self.behaviortree_factory.from_json_to_behavior_tree(json_data) - ) - - def generate_behavior_tree_from_skeleton(self, skeleton: dict) -> None: - self.roster, self.behavior_tree = ( - self.behaviortree_factory.from_skeleton_to_behavior_tree(skeleton) - ) - - def load_world_state(self, world_state: dict) -> None: - self.world_interface.load_world_from_json(world_state) - - def set_world_state(self, world_state: dict) -> None: - """ - set the world state to exactly the given state - """ - self.world_interface.clear_world() - self.world_interface.load_world_from_json(world_state) - - # * tick handlers - def post_tick_handler( - self, - snapshot_visitor: py_trees.visitors.SnapshotVisitor, - behaviour_tree: py_trees.trees.BehaviourTree, - ) -> None: - """ - Print data about the part of the tree visited. - - Args: - snapshot_handler: gather data about the part of the tree visited - behaviour_tree: tree to gather data from - """ - print( - "\n" - + py_trees.display.unicode_tree( - root=behaviour_tree.root, - visited=snapshot_visitor.visited, - previously_visited=snapshot_visitor.previously_visited, - ) - ) - print(py_trees.display.unicode_blackboard()) - - def pre_tick_handler(self, behaviour_tree: py_trees.trees.BehaviourTree) -> None: - """Print a banner with current tick count prior to ticking the tree. - - Args: - behaviour_tree: the tree to tick (used to fetch the count number) - """ - print("\n--------- Run %s ---------\n" % behaviour_tree.count) - - def setup_behavior_tree(self, behavior_tree=None) -> None: - """ - setup vor/nachhandlers. setup the tree. - """ - if behavior_tree == None: - behavior_tree = self.behavior_tree - behavior_tree.add_pre_tick_handler(self.pre_tick_handler) - behavior_tree.visitors.append(py_trees.visitors.DebugVisitor()) - snapshot_visitor = py_trees.visitors.SnapshotVisitor() - behavior_tree.add_post_tick_handler( - functools.partial(self.post_tick_handler, snapshot_visitor) - ) - behavior_tree.visitors.append(snapshot_visitor) - behavior_tree.setup(timeout=15) - - def render_dot_tree(self): - py_trees.display.render_dot_tree( - self.behavior_tree.root, with_blackboard_variables=False - ) - - def tick_loop_test(self): - while True: - try: - self.behavior_tree.tick() - py_trees.console.read_single_keypress() - except KeyboardInterrupt: - break - - print("\n") - - def tick_1000HZ_test(self): - py_trees.logging.level = py_trees.logging.Level.DEBUG - - tick_count = 0 - - def tick(): - nonlocal tick_count - self.behavior_tree.tick() - tick_count += 1 - if self.behavior_tree.root.status == py_trees.common.Status.SUCCESS: - print("\033[94mTree finished with success\033[0m") - elif self.behavior_tree.root.status == py_trees.common.Status.FAILURE: - print("\033[91mTree finished with failure\033[0m") - elif self.behavior_tree.root.status == py_trees.common.Status.RUNNING: - scheduler.enter(1 / 1000, 1, tick) - else: - raise Exception("Unknown status!") - - scheduler = sched.scheduler(time.time, time.sleep) - scheduler.enter(0, 1, tick) - scheduler.run() - - def tick_frequency_test(self): - py_trees.logging.level = py_trees.logging.Level.DEBUG - - tick_count = 0 - start_time = time.time() - - def tick(): - nonlocal tick_count - self.behavior_tree.tick() - tick_count += 1 - if self.behavior_tree.root.status == py_trees.common.Status.SUCCESS: - print("\033[94mTree finished with success\033[0m") - elif self.behavior_tree.root.status == py_trees.common.Status.FAILURE: - print("\033[91mTree finished with failure\033[0m") - elif self.behavior_tree.root.status == py_trees.common.Status.RUNNING: - scheduler.enter(1 / 1000, 1, tick) - else: - raise Exception("Unknown status!") - - scheduler = sched.scheduler(time.time, time.sleep) - scheduler.enter(0, 1, tick) - scheduler.run() - - end_time = time.time() - run_time = end_time - start_time - frequency = tick_count / run_time - - print(f"Tick count: {tick_count}") - print(f"Run time: {run_time} seconds") - print(f"Running frequency: {frequency} Hz") - - def tick_until_success( - self, - bt: py_trees.trees.BehaviourTree, - timeout_ms: float = 5000, - ) -> TreeResult: - time_ms = 0 - - period_ms = 1 - iter_per_round = 10 - - while time_ms < timeout_ms: - bt.tick_tock(period_ms=period_ms, number_of_iterations=iter_per_round) - time_ms += period_ms * iter_per_round - if bt.root.status == py_trees.common.Status.SUCCESS: - return TreeResult( - result="success", - summary="Behavior tree tick returns success", - defect_node=None, - ) - elif bt.root.status == py_trees.common.Status.FAILURE: - tip_node = bt.root.tip() - defect_node = None - if tip_node is None: - defect_node = None - return TreeResult( - result="failure", - summary="Behavior tree tick returns failure with the defect node unknown", - defect_node=defect_node, - ) - if isinstance(tip_node, ActionNode): - defect_node = tip_node.action - elif isinstance(tip_node, ConditionNode): - defect_node = tip_node.condition - - return TreeResult( - result="failure", - summary="Behavior tree tick returns failure", # ! the exceptions should be caught and handled! - defect_node=defect_node, - ) - elif bt.root.status == py_trees.common.Status.RUNNING: - continue - elif bt.root.status == py_trees.common.Status.INVALID: - return TreeResult( - result="invalid", - summary="Behavior tree tick returns invalid status!", - defect_node=None, - ) - - # default timeout - return TreeResult( - result="timeout", - summary="Behavior tree tick returns timeout!", - defect_node=None, - ) - - def tick_loop_until_success( - self, - bt: py_trees.trees.BehaviourTree, - loop_max: int = 20, - ) -> TreeResult: - i = 0 - - while i < loop_max: - i += 1 - bt.tick() - if bt.root.status == py_trees.common.Status.SUCCESS: - return TreeResult( - result="success", - summary="Behavior tree tick returns success", - final_node=None, - world_state=self.world_interface.get_world_to_json(), - ) - elif bt.root.status == py_trees.common.Status.FAILURE: - tip_node = bt.root.tip() - if tip_node is None: - return TreeResult( - result="failure", - summary="Behavior tree tick returns failure with the defect node unknown", - final_node=None, - world_state=self.world_interface.get_world_to_json(), - ) - - return TreeResult( - result="failure", - summary="Behavior tree tick returns failure. The reason could be: A condition failed and an action to satisfy it is not present; An action node failed and no alternative is available; the node is against the definition in domain knowledge; The tree is not well-structured.", - final_node=self.extract_node_metadata(tip_node).to_json(), - world_state=self.world_interface.get_world_to_json(), - ) - elif bt.root.status == py_trees.common.Status.RUNNING: - continue - elif bt.root.status == py_trees.common.Status.INVALID: - return TreeResult( - result="invalid", - summary="Behavior tree tick returns invalid status!", - final_node=None, - world_state=self.world_interface.get_world_to_json(), - ) - - # default timeout - tip_node = bt.root.tip() - return TreeResult( - result="timeout", - summary="Behavior tree tick returns timeout! The reason could be: an action node is being ticked repeatedly while the condition for the next step can not be satisfied; two actions are in one sequence and the tree got stuck here. the tree is not well-structured.", - final_node=self.extract_node_metadata(tip_node).to_json(), - world_state=self.world_interface.get_world_to_json(), - ) - - def extract_node_metadata( - self, node: py_trees.behaviour.Behaviour - ) -> Action | Condition | None: - """ - extract the metadata from the node: name, id, effect/condition, summary - """ - if node is None: - return None - - if isinstance(node, ActionNode): - node_meta = node.action - elif isinstance(node, ConditionNode): - node_meta = node.condition - - return node_meta - - def tick_once_test(self): - self.behavior_tree.tick() - py_trees.console.read_single_keypress() - - print("\n") - - def tick_once(self): - self.behavior_tree.tick() - - # * methods to modify the tree - ########################################################## - def insert_subtree( - self, - subtree: py_trees.behaviour.Behaviour, # * insert this subtree - parent_identifer: int, # * insert as the child of the parent node - index: int = 0, # * insert as the (first) child in the parent node - ): - """ - insert a subtree into the tree as a child of a specific node at a specific index - """ - parent_id = self.roster[parent_identifer].id - self.behavior_tree.insert_subtree(subtree, parent_id, index) - - def replace_subtree( - self, - subtree: py_trees.behaviour.Behaviour, - identifier: int, - ): - """ - replace a specific subtree in the tree with another subtree - """ - to_replace_node_id = self.roster[identifier].id - self.behavior_tree.replace_subtree(to_replace_node_id, subtree) - - def prune_subtree(self, identifier: int): - """ - remove a specific subtree from the tree - """ - to_remove_node_id = self.roster[identifier].id - self.behavior_tree.prune_subtree(to_remove_node_id) - - ########################################################## - # * the behavior tree simulator tool - def simulate_behavior_tree(self) -> TreeResult: - # ! maybe it is better to imp a new class for the simulation - """ - simulate the tree run with action nodes returning only success. - - this should be the following step of the full tree generation. - """ - # * record world check point - self.world_interface.record_check_point() - - roster, sim_bt = self.behaviortree_factory.from_json_to_behavior_tree( - self.bt_json - ) - root = sim_bt.root - - self.replace_action_node_with_sim(sim_bt, root) - - self.setup_behavior_tree(sim_bt) - - tree_result = self.tick_loop_until_success(sim_bt, loop_max=20) - - # * restore world check point - # ! dirty but works well. - self.world_interface.restore_check_point() - - return tree_result - - # ! please generate two trees instead. - def replace_action_node_with_sim(self, stw, root): - for child in root.children: - # replace the action node with sim node - if isinstance(child, ActionNode): - sim_node = ActionNodeSim.from_action_node(child) - stw.replace_subtree(child.id, sim_node) - # recursively replace the children - self.replace_action_node_with_sim(stw, child) - - def replace_action_node_with_only_success(self, stw, root): - for child in root.children: - # replace the action node with sim node - if isinstance(child, ActionNode): - sim_node = ActionNodeOnlySuccess.from_action_node(child) - stw.replace_subtree(child.id, sim_node) - # * update the skeleton dict, add the new replacement node - self.skeleton_dict[sim_node.id] = self.skeleton_dict[child.id] - # recursively replace the children - self.replace_action_node_with_only_success(stw, child) - - def fake_run(self, world_state: dict, bt_json: dict): - """ - fake run for graph testing - """ - self.set_world_state(world_state) - - roster, sim_bt = self.behaviortree_factory.from_json_to_behavior_tree(bt_json) - root = sim_bt.root - - self.replace_action_node_with_sim(sim_bt, root) - - self.setup_behavior_tree(sim_bt) - - tree_result = self.tick_loop_until_success(sim_bt, loop_max=20) - - # # * restore world check point - # self.world_interface.restore_check_point() - - return tree_result - - def sk_fake_run(self, world_state: dict, skeleton_json: dict): - """ - fake run for testing - """ - self.set_world_state(world_state) - self.bt_skeleton = skeleton_json - - roster, bt = self.behaviortree_factory.from_skeleton_to_behavior_tree( - skeleton_json - ) - root = bt.root - - self.replace_action_node_with_sim(bt, root) - - self.setup_behavior_tree(bt) - - tree_result = self.tick_loop_until_success(bt, loop_max=100) - - return tree_result, skeleton_json - - def sk_sim_run(self, world_state: dict, skeleton_json: dict): - """ - fake run for graph testing - """ - self.set_world_state(world_state) - - self.world_interface.record_check_point() - - _, sim_bt, _ = self.behaviortree_factory.from_skeleton_to_behavior_tree( - skeleton_json - ) - root = sim_bt.root - - self.replace_action_node_with_sim(sim_bt, root) - - self.setup_behavior_tree(sim_bt) - - tree_result = self.tick_loop_until_success(sim_bt, loop_max=100) - - # * restore world check point - self.world_interface.restore_check_point() - - return tree_result, skeleton_json - - def sk_baseline( - self, world_state: dict, skeleton_json: dict, ut_dict: dict - ) -> dict: - """baseline expanding, return a solution bt skeleton to start from this world state to achieve the skeleton json condition node. - - - Args: - world_state (dict): ws to start with - skeleton_json (dict): the condition sk node to achieve. should be a condition node in sk form with summary and a legal name. - - Raises: - Exception: see the code - - Returns: - dict: the solution bt sk json. - """ - - from pprint import pprint - - self.set_world_state(world_state) - self.world_interface.record_check_point("baseline_start") - - _, sim_bt, self.skeleton_dict = ( - self.behaviortree_factory.from_skeleton_to_behavior_tree(skeleton_json) - ) - self.replace_action_node_with_only_success(sim_bt, sim_bt.root) - self.setup_behavior_tree(sim_bt) - - from kios_utils.pybt_test import generate_bt_stewardship, render_dot_tree - - while True: - sim_bt.tick() - if sim_bt.root.status == py_trees.common.Status.SUCCESS: - pprint("Behavior tree tick returns success!") - break - elif sim_bt.root.status == py_trees.common.Status.FAILURE: - tip_node = sim_bt.root.tip() - if tip_node is None: - raise Exception( - "Behavior tree tick returns failure with the defect node unknown" - ) - assert isinstance(tip_node, ConditionNode) - new_target = tip_node.condition.to_string() - new_ut = ut_dict.get(new_target, None) - if new_ut is None: - pprint(f"cannot find a solution ut for the condtiion {new_target}!") - pprint("the available targets are:") - pprint(ut_dict.keys()) - raise Exception( - f"cannot find a solution ut for the condtiion {new_target}!" - ) - break - _, new_subtree, new_sk_dict = ( - self.behaviortree_factory.from_skeleton_to_behavior_tree(new_ut) - ) - self.skeleton_dict.update(new_sk_dict) - self.replace_action_node_with_only_success( - new_subtree, new_subtree.root - ) - # render_dot_tree(new_subtree) - # pause = input("paused here! check the tree.") - if sim_bt.root.id == tip_node.id: - sim_bt = new_subtree - else: - sim_bt.replace_subtree(tip_node.id, new_subtree.root) - # render_dot_tree(sim_bt) - # pause = input("paused here! check the tree.") - - sim_bt.setup(timeout=15) - - self.world_interface.restore_check_point("baseline_start") - elif sim_bt.root.status == py_trees.common.Status.RUNNING: - print("Behavior tree is running") - else: - raise Exception("Unknown status!") - - # * restore world check point - self.world_interface.restore_check_point("baseline_start") - render_dot_tree(sim_bt) - return self.from_tree_root_to_sk_json(sim_bt.root) - - def from_tree_root_to_sk_json( - self, tree_root: py_trees.behaviour.Behaviour - ) -> dict: - """ - convert the tree root to sk_json - """ - if isinstance(tree_root, py_trees.composites.Selector): - compound_sk = self.skeleton_dict[tree_root.id] - compound_sk["children"] = [ - self.from_tree_root_to_sk_json(child) for child in tree_root.children - ] - return compound_sk - elif isinstance(tree_root, py_trees.composites.Sequence): - compound_sk = self.skeleton_dict[tree_root.id] - compound_sk["children"] = [ - self.from_tree_root_to_sk_json(child) for child in tree_root.children - ] - return compound_sk - elif isinstance(tree_root, ActionNode): - return self.skeleton_dict[tree_root.id] - elif isinstance(tree_root, ConditionNode): - return self.skeleton_dict[tree_root.id] - else: - raise Exception("Unknown node type!") - - ########################################################## - - def query_world_state(self): - return self.world_interface.get_world_to_json() - - def update_world_state_with_pddl_problem(self, pddl_problem: str): - """ - bb problem parser - """ - - world_state = parse_problem(pddl_problem) - - self.world_interface.update_world(world_state) diff --git a/kios_bt_planning/kios_bt/control_nodes.py b/kios_bt_planning/kios_bt/control_nodes.py deleted file mode 100644 index f5ce95a2..00000000 --- a/kios_bt_planning/kios_bt/control_nodes.py +++ /dev/null @@ -1,73 +0,0 @@ -""" -selector without memory stolen from dynamic bt package -""" - -# ! this feature is now a part of py_trees. -# ! this file is preserved for overwriting the compound control nodes if needed. -# import py_trees as pt -# from typing import Any, List, Tuple - - -# class RSequence(pt.composites.Selector): -# """ -# Rsequence for py_trees. - -# Reactive sequence overriding sequence with memory, py_trees' only available sequence. -# Author: Christopher Iliffe Sprague, sprague@kth.se -# """ - -# def __init__(self, name: str = "Sequence", children: List[Any] = None): -# super().__init__(name=name, children=children) - -# def tick(self): -# """ -# Run the tick behaviour for this selector. - -# Note that the status of the tick is always determined by its children, -# not by the user customized update function. - -# Yields -# ------ -# class:`~py_trees.behaviour.Behaviour`: a reference to itself or one of its children. - -# """ -# self.logger.debug( -# "%s.tick()" % self.__class__.__name__ -# ) # pylint: disable=consider-using-f-string -# # Required behaviour for *all* behaviours and composites is -# # for tick() to check if it isn't running and initialise -# if self.status != pt.common.Status.RUNNING: -# # selectors dont do anything specific on initialization -# # - the current child is managed by the update, never needs to be 'initialized' -# # run subclass (user) handles -# self.initialise() -# # run any work designated by a customized instance of this class -# self.update() -# previous = self.current_child -# for child in self.children: -# for node in child.tick(): -# yield node -# if node is child and ( -# node.status == pt.common.Status.RUNNING -# or node.status == pt.common.Status.FAILURE -# ): -# self.current_child = child -# self.status = node.status -# if previous is None or previous != self.current_child: -# # we interrupted, invalidate everything at a lower priority -# passed = False -# for sibling in self.children: -# if passed and sibling.status != pt.common.Status.INVALID: -# sibling.stop(pt.common.Status.INVALID) -# if sibling == self.current_child: -# passed = True -# yield self -# return -# # all children succeeded, -# # set succeed ourselves and current child to the last bugger who failed us -# self.status = pt.common.Status.SUCCESS -# try: -# self.current_child = self.children[-1] -# except IndexError: -# self.current_child = None -# yield self diff --git a/kios_bt_planning/kios_bt/data_types.py b/kios_bt_planning/kios_bt/data_types.py deleted file mode 100644 index e6a069f4..00000000 --- a/kios_bt_planning/kios_bt/data_types.py +++ /dev/null @@ -1,103 +0,0 @@ -""" -typing dataclasses for the kios_bt_planning package -""" - -from dataclasses import dataclass -from typing import Dict, List, Any, Optional - -import py_trees -import json - - -# * in use -@dataclass -class TreeResult: - """ - a result from the simulation - """ - - result: str - summary: str - final_node: Optional[dict] # ! alaerm - world_state: Dict[str, List[Dict[str, str]]] - # raw_node: Optional[py_trees.behaviour.Behaviour] - - def to_json(self): - return { - "result": self.result, - "summary": self.summary, - "final_node": self.final_node, - "world_state": self.world_state, - } - - -@dataclass -class ObjectProperty: # for indicating the conditions and the effects in an object-centric way - """ - a property of an object - if it is a property, then the property_value should be None - if it is a relation, then the property_value should be the name of the target object - status: True if the property is true, False if the property is false - """ - - object_name: str - property_name: str - property_value: str - status: bool - - def to_json(self): - return { - "object_name": self.object_name, - "property_name": self.property_name, - "property_value": self.property_value, - "status": self.status, - } - - def to_string(self): - if self.property_value is None: - return f"{self.property_name}({self.object_name})" - else: - return f"{self.property_name}({self.object_name}, {self.property_value})" - - -@dataclass -class Condition: # for generating a condition node - summary: str - identifier: int - name: str # the name of the condition you want this node to check - conditions: List[ObjectProperty] - - def to_json(self): - return { - "summary": self.summary, - "identifier": self.identifier, - "name": self.name, - "conditions": [condition.to_json() for condition in self.conditions], - } - - def to_string(self): - # ! this is a temporary solution - return self.conditions[0].to_string() - - -@dataclass -class Action: # for generating an action node - summary: str - identifier: int - name: str # the name of the action you want this node to conduct - effects: List[ObjectProperty] # the effects of the action - - def to_json(self): - return { - "summary": self.summary, - "identifier": self.identifier, - "name": self.name, - "effects": [effect.to_json() for effect in self.effects], - } - - -@dataclass -class ControlFlow: - identifier: int - name: str - children: List[Any] diff --git a/kios_bt_planning/kios_bt/mios_async.py b/kios_bt_planning/kios_bt/mios_async.py deleted file mode 100644 index c54b8286..00000000 --- a/kios_bt_planning/kios_bt/mios_async.py +++ /dev/null @@ -1,97 +0,0 @@ -# #!/usr/bin/env python - -# ############################################################################## -# # Imports -# ############################################################################## - -# # ! DISCARDED - -# from kios_utils.task import * - -# import atexit -# import multiprocessing -# import multiprocessing.connection -# import time - -# # ! BBDEV: this should be discarded later. -# # * new func: in robot.skill_engine. - - -# def mios_monitor( -# task: Task, pipe_connection: multiprocessing.connection.Connection -# ) -> None: -# """Emulate a (potentially) long running external process. - -# Args: -# pipe_connection: connection to the mios_monitor process -# """ -# try: -# task.interrupt() -# task.start() -# # hanlde startup failure -# # ! check the response here -# print(str(task.task_start_response)) - -# task.shared_data["task_start_response"] = task.task_start_response - -# if bool(task.task_start_response["result"]["result"]) == False: -# pipe_connection.send([False]) -# return - -# _ = task.wait() - -# if bool(task.task_wait_response["result"]["result"]) == True: -# pipe_connection.send([True]) -# else: -# pipe_connection.send([False]) -# except KeyboardInterrupt: -# pass - - -# def fake_monitor( -# task: Task, pipe_connection: multiprocessing.connection.Connection -# ) -> None: -# """Emulate a (potentially) long running external process. - -# Args: -# pipe_connection: connection to the mios_monitor process -# """ -# try: -# # * skip -# # task.interrupt() -# # task.start() -# # hanlde startup failure -# # ! check the response here -# # print(str(task.task_start_response)) - -# time.sleep(2) -# # * fake a start response -# # print("start fake monitor") -# task.shared_data["task_start_response"] = {"result": {"result": True}} - -# # * skip the start -# # if bool(task.task_start_response["result"]["result"]) == False: -# # pipe_connection.send([False]) -# # return -# # * skip the wait -# # _ = task.wait() - -# # * sleep for 5 seconds -# time.sleep(5) - -# # * skip the wait response -# # if bool(task.task_wait_response["result"]["result"]) == True: -# # pipe_connection.send([True]) -# # else: -# # pipe_connection.send([False]) - -# # * send fake response -# print("send fake response") -# pipe_connection.send([True]) - -# except KeyboardInterrupt: -# pass - - -# def kios_skill_async(): -# raise NotImplementedError diff --git a/kios_bt_planning/kios_bt_planning.egg-info/PKG-INFO b/kios_bt_planning/kios_bt_planning.egg-info/PKG-INFO deleted file mode 100644 index 579cc198..00000000 --- a/kios_bt_planning/kios_bt_planning.egg-info/PKG-INFO +++ /dev/null @@ -1,6 +0,0 @@ -Metadata-Version: 2.1 -Name: kios-bt-planning -Version: 0.1.0 -Summary: Behavior Tree Planning for KIOS -Author: blackbird -Requires-Dist: py-trees diff --git a/kios_bt_planning/kios_bt_planning.egg-info/SOURCES.txt b/kios_bt_planning/kios_bt_planning.egg-info/SOURCES.txt deleted file mode 100644 index 3b2fac6e..00000000 --- a/kios_bt_planning/kios_bt_planning.egg-info/SOURCES.txt +++ /dev/null @@ -1,85 +0,0 @@ -setup.py -kios_agent/RAG_chain_tutorial.py -kios_agent/__init__.py -kios_agent/data_types.py -kios_agent/kios_llm.py -kios_agent/kios_tools.py -kios_agent/llama_deploy.py -kios_agent/llm_agents.py -kios_agent/llm_supporter.py -kios_agent/tools_tutorial.py -kios_bt/__init__.py -kios_bt/behavior_nodes.py -kios_bt/bt_factory.py -kios_bt/bt_simulator.py -kios_bt/bt_stewardship.py -kios_bt/control_nodes.py -kios_bt/data_types.py -kios_bt/mios_async.py -kios_bt_planning.egg-info/PKG-INFO -kios_bt_planning.egg-info/SOURCES.txt -kios_bt_planning.egg-info/dependency_links.txt -kios_bt_planning.egg-info/requires.txt -kios_bt_planning.egg-info/top_level.txt -kios_domain/__init__.py -kios_domain/domain copy.py -kios_domain/domain_v0.py -kios_domain/domain_v1.py -kios_domain/knowledge_v0.py -kios_domain/knowledge_v1.py -kios_domain/pddl_interface.py -kios_domain/problem1.py -kios_domain/problem2.py -kios_domain/problem3.py -kios_domain/problem4.py -kios_future/__init__.py -kios_future/embeddings.py -kios_plan/__init__.py -kios_plan/dynamic_planning.py -kios_robot/__init__.py -kios_robot/async_robot_command.py -kios_robot/data_types.py -kios_robot/mios_async.py -kios_robot/mios_task_factory.py -kios_robot/robot_command.py -kios_robot/robot_interface.py -kios_robot/robot_proprioceptor.py -kios_robot/kios_vision/__init__.py -kios_scene/__init__.py -kios_scene/mios_ltm_manipulator.py -kios_scene/mongodb_interface.py -kios_scene/scene_factory.py -kios_utils/__init__.py -kios_utils/bblab_utils.py -kios_utils/math_utils.py -kios_utils/parsers.py -kios_utils/pddl_problem_parser.py -kios_utils/pybt_test.py -kios_utils/skeleton_parser.py -kios_utils/task.py -kios_utils/test_skeleton_parser.py -kios_utils/ws_client.py -kios_world/__init__.py -kios_world/data_types.py -kios_world/graph_interface.py -kios_world/neo4j_interface.py -kios_world/world_interface.py -tests/__init__.py -tests/bt_generation_test.py -tests/bt_json_io_test.py -tests/chain_test.py -tests/chain_test2.py -tests/feature_test.py -tests/ft_test.py -tests/graph_basebone_test.py -tests/graph_interface_test.py -tests/ltm_test.py -tests/mios_test.py -tests/multiprocess_test.py -tests/neo4j_test.py -tests/networkx_test.py -tests/pddl_interface_test.py -tests/pytrees_blackboard_test.py -tests/robot_interface_test.py -tests/scene_test.py -tests/visualization_debug.py \ No newline at end of file diff --git a/kios_bt_planning/kios_bt_planning.egg-info/dependency_links.txt b/kios_bt_planning/kios_bt_planning.egg-info/dependency_links.txt deleted file mode 100644 index 8b137891..00000000 --- a/kios_bt_planning/kios_bt_planning.egg-info/dependency_links.txt +++ /dev/null @@ -1 +0,0 @@ - diff --git a/kios_bt_planning/kios_bt_planning.egg-info/requires.txt b/kios_bt_planning/kios_bt_planning.egg-info/requires.txt deleted file mode 100644 index 741bfe49..00000000 --- a/kios_bt_planning/kios_bt_planning.egg-info/requires.txt +++ /dev/null @@ -1 +0,0 @@ -py-trees diff --git a/kios_bt_planning/kios_bt_planning.egg-info/top_level.txt b/kios_bt_planning/kios_bt_planning.egg-info/top_level.txt deleted file mode 100644 index 3593d5fb..00000000 --- a/kios_bt_planning/kios_bt_planning.egg-info/top_level.txt +++ /dev/null @@ -1,10 +0,0 @@ -kios_agent -kios_bt -kios_domain -kios_future -kios_plan -kios_robot -kios_scene -kios_utils -kios_world -tests diff --git a/kios_bt_planning/kios_domain/__init__.py b/kios_bt_planning/kios_domain/__init__.py deleted file mode 100644 index e69de29b..00000000 diff --git a/kios_bt_planning/kios_domain/domain copy.py b/kios_bt_planning/kios_domain/domain copy.py deleted file mode 100644 index 8ff09668..00000000 --- a/kios_bt_planning/kios_domain/domain copy.py +++ /dev/null @@ -1,376 +0,0 @@ -from unified_planning.shortcuts import * - -######################################## Type ######################################## - -Thing = UserType("Thing") - -Tool = UserType("Tool", father=Thing) -Part = UserType("Part", father=Thing) -# Feature = UserType("Feature", father=Part) - -Hand = UserType("Hand", father=Thing) - -######################################## Fluent ######################################## - -# * property -is_free = Fluent("is_free", BoolType(), p=Thing) -is_equippable = Fluent("is_equippable", BoolType(), tool=Tool) - -# * relation -can_manipulate = Fluent("can_manipulate", BoolType(), tool=Tool, part=Part) -can_insert_to = Fluent("can_insert_to", BoolType(), part1=Part, part2=Part) -can_screw_to = Fluent("can_screw_to", BoolType(), part1=Part, part2=Part) -can_place_to = Fluent("can_place_to", BoolType(), part1=Part, part2=Part) -hold = Fluent("hold", BoolType(), thing1=Thing, thing2=Thing) -is_inserted_to = Fluent("is_inserted_to", BoolType(), part1=Part, part2=Part) -is_screwed_to = Fluent("is_screwed_to", BoolType(), part1=Part, part2=Part) -is_placed_to = Fluent("is_placed_to", BoolType(), part1=Part, part2=Part) - -######################################## Action ######################################## -# * load_tool -load_tool = unified_planning.model.InstantaneousAction( - "load_tool", - hand=Hand, - tool=Tool, -) - -tool = load_tool.parameter("tool") -hand = load_tool.parameter("hand") - -load_tool.add_precondition(is_equippable(tool)) -load_tool.add_precondition(is_free(hand)) - -load_tool.add_effect(is_free(hand), False) -load_tool.add_effect(is_equippable(tool), False) -load_tool.add_effect(hold(hand, tool), True) - -# * unload_tool -unload_tool = unified_planning.model.InstantaneousAction( - "unload_tool", - hand=Hand, - tool=Tool, -) - -tool = unload_tool.parameter("tool") -hand = unload_tool.parameter("hand") - -unload_tool.add_precondition(hold(hand, tool)) -unload_tool.add_precondition(is_free(tool)) - -unload_tool.add_effect(is_free(hand), True) -unload_tool.add_effect(is_equippable(tool), True) -unload_tool.add_effect(hold(hand, tool), False) - -# * pick_up -pick_up = unified_planning.model.InstantaneousAction( - "pick_up", - hand=Hand, - tool=Tool, - part=Part, -) - -part = pick_up.parameter("part") -tool = pick_up.parameter("tool") -hand = pick_up.parameter("hand") - -pick_up.add_precondition(is_free(tool)) -pick_up.add_precondition(hold(hand, tool)) -pick_up.add_precondition(can_manipulate(tool, part)) - -pick_up.add_effect(hold(tool, part), True) -pick_up.add_effect(is_free(tool), False) - -# * put_down -put_down = unified_planning.model.InstantaneousAction( - "put_down", - hand=Hand, - tool=Tool, - part=Part, -) - -part = put_down.parameter("part") -tool = put_down.parameter("tool") -hand = put_down.parameter("hand") - -put_down.add_precondition(hold(tool, part)) -put_down.add_precondition(hold(hand, tool)) - -put_down.add_effect(hold(hand, part), False) -put_down.add_effect(is_free(tool), True) - - -# * place -place = unified_planning.model.InstantaneousAction( - "place", - hand=Hand, - tool=Tool, - part1=Part, - part2=Part, -) - -part1 = place.parameter("part1") -part2 = place.parameter("part2") -tool = place.parameter("tool") -hand = place.parameter("hand") - -place.add_precondition(hold(hand, tool)) -place.add_precondition(hold(tool, part1)) -place.add_precondition(can_place_to(part1, part2)) - -place.add_effect(hold(tool, part1), False) -place.add_effect(is_free(tool), True) -place.add_effect(is_placed_to(part1, part2), True) - -# * detach -detach = unified_planning.model.InstantaneousAction( - "detach", - hand=Hand, - tool=Tool, - part1=Part, - part2=Part, -) - -part1 = detach.parameter("part1") -part2 = detach.parameter("part2") -tool = detach.parameter("tool") -hand = detach.parameter("hand") - -detach.add_precondition(hold(hand, tool)) -detach.add_precondition(is_free(tool)) -detach.add_precondition(can_manipulate(tool, part1)) -detach.add_precondition(is_placed_to(part1, part2)) - -detach.add_effect(hold(tool, part1), True) -detach.add_effect(is_free(tool), False) -detach.add_effect(is_placed_to(part1, part2), False) - -# * push, can be move -# ! implement later - -# * insert -insert = unified_planning.model.InstantaneousAction( - "insert", - hand=Hand, - tool=Tool, - part1=Part, - part2=Part, -) - -hand = insert.parameter("hand") -part1 = insert.parameter("part1") -part2 = insert.parameter("part2") -tool = insert.parameter("tool") - -insert.add_precondition(hold(hand, tool)) -insert.add_precondition(hold(tool, part1)) -insert.add_precondition(can_insert_to(part1, part2)) - -insert.add_effect(hold(tool, part1), False) -insert.add_effect(is_free(tool), True) -insert.add_effect(is_inserted_to(part1, part2), True) - -# * pull-out -pullout = unified_planning.model.InstantaneousAction( - "pull", - hand=Hand, - tool=Tool, - part1=Part, - part2=Part, -) - -part1 = pullout.parameter("part1") -part2 = pullout.parameter("part2") -tool = pullout.parameter("tool") -hand = pullout.parameter("hand") - -pullout.add_precondition(hold(hand, tool)) -pullout.add_precondition(is_free(tool)) -pullout.add_precondition(is_inserted_to(part1, part2)) -pullout.add_precondition(can_manipulate(tool, part1)) - -pullout.add_effect(hold(tool, part1), True) -pullout.add_effect(is_free(tool), False) -pullout.add_effect(is_inserted_to(part1, part2), False) - -# * screw -screw = unified_planning.model.InstantaneousAction( - "screw", - hand=Hand, - tool=Tool, - part1=Part, - part2=Part, -) - -part1 = screw.parameter("part1") -part2 = screw.parameter("part2") -tool = screw.parameter("tool") -hand = screw.parameter("hand") - -screw.add_precondition(hold(hand, tool)) -screw.add_precondition(hold(tool, part1)) -screw.add_precondition(can_screw_to(part1, part2)) - -screw.add_effect(hold(tool, part1), False) -screw.add_effect(is_free(tool), True) -screw.add_effect(is_screwed_to(part1, part2), True) - -# * unscrew -unscrew = unified_planning.model.InstantaneousAction( - "unscrew", - hand=Hand, - tool=Tool, - part1=Part, - part2=Part, -) - -part1 = unscrew.parameter("part1") -part2 = unscrew.parameter("part2") -tool = unscrew.parameter("tool") -hand = unscrew.parameter("hand") - -unscrew.add_precondition(hold(hand, tool)) -unscrew.add_precondition(is_free(tool)) -unscrew.add_precondition(is_screwed_to(part1, part2)) -unscrew.add_precondition(can_manipulate(tool, part1)) - -unscrew.add_effect(hold(tool, part1), True) -unscrew.add_effect(is_free(tool), False) -unscrew.add_effect(is_screwed_to(part1, part2), False) - -######################################## Problem ######################################## - -# problem = Problem("kios_tamp_problem") - -# ########################## * knowledge - -# # * property -# problem.add_fluent(is_free, default_initial_value=True) -# problem.add_fluent(is_equippable, default_initial_value=True) - -# # * relation -# problem.add_fluent(can_manipulate, default_initial_value=False) -# problem.add_fluent(can_insert_to, default_initial_value=False) -# problem.add_fluent(can_screw_to, default_initial_value=False) -# problem.add_fluent(can_place_to, default_initial_value=False) -# problem.add_fluent(hold, default_initial_value=False) -# problem.add_fluent(is_inserted_to, default_initial_value=False) -# problem.add_fluent(is_screwed_to, default_initial_value=False) -# problem.add_fluent(is_placed_to, default_initial_value=False) - -# # * action - -# problem.add_action(pick_up) -# problem.add_action(put_down) -# problem.add_action(place) -# problem.add_action(detach) -# problem.add_action(insert) -# problem.add_action(pullout) -# problem.add_action(screw) -# problem.add_action(unscrew) -# problem.add_action(load_tool) -# problem.add_action(unload_tool) - -# ########################* instances ######################################## -# # hand instances -# left_hand = unified_planning.model.Object("left_hand", Hand) - -# # tool instances -# parallel_box1 = unified_planning.model.Object("parallel_box1", Tool) -# parallel_box2 = unified_planning.model.Object("parallel_box2", Tool) -# inward_claw = unified_planning.model.Object("inward_claw", Tool) -# outward_claw = unified_planning.model.Object("outward_claw", Tool) -# no_tool = unified_planning.model.Object("no_tool", Tool) - -# # part instances -# # * chair -# leg1 = unified_planning.model.Object("leg1", Part) -# leg2 = unified_planning.model.Object("leg2", Part) -# nut1 = unified_planning.model.Object("nut1", Part) -# nut2 = unified_planning.model.Object("nut2", Part) -# seat = unified_planning.model.Object("seat", Part) -# back = unified_planning.model.Object("back", Part) - -# # * lamp -# base = unified_planning.model.Object("base", Part) -# blub = unified_planning.model.Object("blub", Part) -# lamp = unified_planning.model.Object("lamp", Part) - -# # * gearset -# gear1 = unified_planning.model.Object("gear1", Part) -# gear2 = unified_planning.model.Object("gear2", Part) -# gear3 = unified_planning.model.Object("gear3", Part) -# shaft1 = unified_planning.model.Object("shaft1", Part) -# shaft2 = unified_planning.model.Object("shaft2", Part) -# base = unified_planning.model.Object("base", Part) - -# ########################## * current problem context ###################### -# # add all the related instances to the problem here - -# # * hand instances -# problem.add_object(left_hand) - -# # * tool instances -# problem.add_object(parallel_box1) -# problem.add_object(parallel_box2) -# problem.add_object(inward_claw) -# problem.add_object(outward_claw) -# problem.add_object(no_tool) - -# # * part instances -# # * chair -# # not this time -# # * lamp -# # not this time -# # * gearset -# problem.add_object(gear1) -# problem.add_object(gear2) -# problem.add_object(gear3) -# problem.add_object(shaft1) -# problem.add_object(shaft2) -# problem.add_object(base) - -# ########################## * problem constraints ########################## - -# ############ * tools' manipulability -# problem.set_initial_value(can_manipulate(parallel_box1, gear1), True) -# problem.set_initial_value(can_manipulate(outward_claw, gear2), True) -# problem.set_initial_value(can_manipulate(inward_claw, gear3), True) -# problem.set_initial_value(can_manipulate(parallel_box2, shaft1), True) -# problem.set_initial_value(can_manipulate(no_tool, shaft2), True) - -# ############ * assembly constraints -# # * chair -# # not implemented yet -# # * lamp -# # not implemented yet -# # * gearset -# problem.set_initial_value(can_insert_to(shaft1, base), True) -# problem.set_initial_value(can_insert_to(shaft2, base), True) -# problem.set_initial_value(can_insert_to(gear3, shaft2), True) -# problem.set_initial_value(can_insert_to(gear2, base), True) -# problem.set_initial_value(can_insert_to(gear1, shaft1), True) - -# ########################## * initial world state ########################## -# # ! BBNOTE: LLM CONSENTRATION FOR STATE KEEPING LOAD SAVING -# # ! Default initial value: see fluent setup part above - - -# # problem.set_initial_value(hold(left_hand, parallel_box2), True) -# # problem.set_initial_value(hold(parallel_box2, shaft1), True) - -# ########################## * goal ########################################## - -# problem.add_goal(is_inserted_to(shaft1, base)) -# print(problem) - - -# # with OneshotPlanner(name="pyperplan") as planner: -# # result = planner.solve(problem) -# # if result.status == up.engines.PlanGenerationResultStatus.SOLVED_SATISFICING: -# # print("Pyperplan returned: %s" % result.plan) -# # else: -# # print("No plan found.") - -# with OneshotPlanner(problem_kind=problem.kind) as planner: -# result = planner.solve(problem) -# print("%s returned: %s" % (planner.name, result.plan)) diff --git a/kios_bt_planning/kios_domain/domain_v0.py b/kios_bt_planning/kios_domain/domain_v0.py deleted file mode 100644 index 8ff09668..00000000 --- a/kios_bt_planning/kios_domain/domain_v0.py +++ /dev/null @@ -1,376 +0,0 @@ -from unified_planning.shortcuts import * - -######################################## Type ######################################## - -Thing = UserType("Thing") - -Tool = UserType("Tool", father=Thing) -Part = UserType("Part", father=Thing) -# Feature = UserType("Feature", father=Part) - -Hand = UserType("Hand", father=Thing) - -######################################## Fluent ######################################## - -# * property -is_free = Fluent("is_free", BoolType(), p=Thing) -is_equippable = Fluent("is_equippable", BoolType(), tool=Tool) - -# * relation -can_manipulate = Fluent("can_manipulate", BoolType(), tool=Tool, part=Part) -can_insert_to = Fluent("can_insert_to", BoolType(), part1=Part, part2=Part) -can_screw_to = Fluent("can_screw_to", BoolType(), part1=Part, part2=Part) -can_place_to = Fluent("can_place_to", BoolType(), part1=Part, part2=Part) -hold = Fluent("hold", BoolType(), thing1=Thing, thing2=Thing) -is_inserted_to = Fluent("is_inserted_to", BoolType(), part1=Part, part2=Part) -is_screwed_to = Fluent("is_screwed_to", BoolType(), part1=Part, part2=Part) -is_placed_to = Fluent("is_placed_to", BoolType(), part1=Part, part2=Part) - -######################################## Action ######################################## -# * load_tool -load_tool = unified_planning.model.InstantaneousAction( - "load_tool", - hand=Hand, - tool=Tool, -) - -tool = load_tool.parameter("tool") -hand = load_tool.parameter("hand") - -load_tool.add_precondition(is_equippable(tool)) -load_tool.add_precondition(is_free(hand)) - -load_tool.add_effect(is_free(hand), False) -load_tool.add_effect(is_equippable(tool), False) -load_tool.add_effect(hold(hand, tool), True) - -# * unload_tool -unload_tool = unified_planning.model.InstantaneousAction( - "unload_tool", - hand=Hand, - tool=Tool, -) - -tool = unload_tool.parameter("tool") -hand = unload_tool.parameter("hand") - -unload_tool.add_precondition(hold(hand, tool)) -unload_tool.add_precondition(is_free(tool)) - -unload_tool.add_effect(is_free(hand), True) -unload_tool.add_effect(is_equippable(tool), True) -unload_tool.add_effect(hold(hand, tool), False) - -# * pick_up -pick_up = unified_planning.model.InstantaneousAction( - "pick_up", - hand=Hand, - tool=Tool, - part=Part, -) - -part = pick_up.parameter("part") -tool = pick_up.parameter("tool") -hand = pick_up.parameter("hand") - -pick_up.add_precondition(is_free(tool)) -pick_up.add_precondition(hold(hand, tool)) -pick_up.add_precondition(can_manipulate(tool, part)) - -pick_up.add_effect(hold(tool, part), True) -pick_up.add_effect(is_free(tool), False) - -# * put_down -put_down = unified_planning.model.InstantaneousAction( - "put_down", - hand=Hand, - tool=Tool, - part=Part, -) - -part = put_down.parameter("part") -tool = put_down.parameter("tool") -hand = put_down.parameter("hand") - -put_down.add_precondition(hold(tool, part)) -put_down.add_precondition(hold(hand, tool)) - -put_down.add_effect(hold(hand, part), False) -put_down.add_effect(is_free(tool), True) - - -# * place -place = unified_planning.model.InstantaneousAction( - "place", - hand=Hand, - tool=Tool, - part1=Part, - part2=Part, -) - -part1 = place.parameter("part1") -part2 = place.parameter("part2") -tool = place.parameter("tool") -hand = place.parameter("hand") - -place.add_precondition(hold(hand, tool)) -place.add_precondition(hold(tool, part1)) -place.add_precondition(can_place_to(part1, part2)) - -place.add_effect(hold(tool, part1), False) -place.add_effect(is_free(tool), True) -place.add_effect(is_placed_to(part1, part2), True) - -# * detach -detach = unified_planning.model.InstantaneousAction( - "detach", - hand=Hand, - tool=Tool, - part1=Part, - part2=Part, -) - -part1 = detach.parameter("part1") -part2 = detach.parameter("part2") -tool = detach.parameter("tool") -hand = detach.parameter("hand") - -detach.add_precondition(hold(hand, tool)) -detach.add_precondition(is_free(tool)) -detach.add_precondition(can_manipulate(tool, part1)) -detach.add_precondition(is_placed_to(part1, part2)) - -detach.add_effect(hold(tool, part1), True) -detach.add_effect(is_free(tool), False) -detach.add_effect(is_placed_to(part1, part2), False) - -# * push, can be move -# ! implement later - -# * insert -insert = unified_planning.model.InstantaneousAction( - "insert", - hand=Hand, - tool=Tool, - part1=Part, - part2=Part, -) - -hand = insert.parameter("hand") -part1 = insert.parameter("part1") -part2 = insert.parameter("part2") -tool = insert.parameter("tool") - -insert.add_precondition(hold(hand, tool)) -insert.add_precondition(hold(tool, part1)) -insert.add_precondition(can_insert_to(part1, part2)) - -insert.add_effect(hold(tool, part1), False) -insert.add_effect(is_free(tool), True) -insert.add_effect(is_inserted_to(part1, part2), True) - -# * pull-out -pullout = unified_planning.model.InstantaneousAction( - "pull", - hand=Hand, - tool=Tool, - part1=Part, - part2=Part, -) - -part1 = pullout.parameter("part1") -part2 = pullout.parameter("part2") -tool = pullout.parameter("tool") -hand = pullout.parameter("hand") - -pullout.add_precondition(hold(hand, tool)) -pullout.add_precondition(is_free(tool)) -pullout.add_precondition(is_inserted_to(part1, part2)) -pullout.add_precondition(can_manipulate(tool, part1)) - -pullout.add_effect(hold(tool, part1), True) -pullout.add_effect(is_free(tool), False) -pullout.add_effect(is_inserted_to(part1, part2), False) - -# * screw -screw = unified_planning.model.InstantaneousAction( - "screw", - hand=Hand, - tool=Tool, - part1=Part, - part2=Part, -) - -part1 = screw.parameter("part1") -part2 = screw.parameter("part2") -tool = screw.parameter("tool") -hand = screw.parameter("hand") - -screw.add_precondition(hold(hand, tool)) -screw.add_precondition(hold(tool, part1)) -screw.add_precondition(can_screw_to(part1, part2)) - -screw.add_effect(hold(tool, part1), False) -screw.add_effect(is_free(tool), True) -screw.add_effect(is_screwed_to(part1, part2), True) - -# * unscrew -unscrew = unified_planning.model.InstantaneousAction( - "unscrew", - hand=Hand, - tool=Tool, - part1=Part, - part2=Part, -) - -part1 = unscrew.parameter("part1") -part2 = unscrew.parameter("part2") -tool = unscrew.parameter("tool") -hand = unscrew.parameter("hand") - -unscrew.add_precondition(hold(hand, tool)) -unscrew.add_precondition(is_free(tool)) -unscrew.add_precondition(is_screwed_to(part1, part2)) -unscrew.add_precondition(can_manipulate(tool, part1)) - -unscrew.add_effect(hold(tool, part1), True) -unscrew.add_effect(is_free(tool), False) -unscrew.add_effect(is_screwed_to(part1, part2), False) - -######################################## Problem ######################################## - -# problem = Problem("kios_tamp_problem") - -# ########################## * knowledge - -# # * property -# problem.add_fluent(is_free, default_initial_value=True) -# problem.add_fluent(is_equippable, default_initial_value=True) - -# # * relation -# problem.add_fluent(can_manipulate, default_initial_value=False) -# problem.add_fluent(can_insert_to, default_initial_value=False) -# problem.add_fluent(can_screw_to, default_initial_value=False) -# problem.add_fluent(can_place_to, default_initial_value=False) -# problem.add_fluent(hold, default_initial_value=False) -# problem.add_fluent(is_inserted_to, default_initial_value=False) -# problem.add_fluent(is_screwed_to, default_initial_value=False) -# problem.add_fluent(is_placed_to, default_initial_value=False) - -# # * action - -# problem.add_action(pick_up) -# problem.add_action(put_down) -# problem.add_action(place) -# problem.add_action(detach) -# problem.add_action(insert) -# problem.add_action(pullout) -# problem.add_action(screw) -# problem.add_action(unscrew) -# problem.add_action(load_tool) -# problem.add_action(unload_tool) - -# ########################* instances ######################################## -# # hand instances -# left_hand = unified_planning.model.Object("left_hand", Hand) - -# # tool instances -# parallel_box1 = unified_planning.model.Object("parallel_box1", Tool) -# parallel_box2 = unified_planning.model.Object("parallel_box2", Tool) -# inward_claw = unified_planning.model.Object("inward_claw", Tool) -# outward_claw = unified_planning.model.Object("outward_claw", Tool) -# no_tool = unified_planning.model.Object("no_tool", Tool) - -# # part instances -# # * chair -# leg1 = unified_planning.model.Object("leg1", Part) -# leg2 = unified_planning.model.Object("leg2", Part) -# nut1 = unified_planning.model.Object("nut1", Part) -# nut2 = unified_planning.model.Object("nut2", Part) -# seat = unified_planning.model.Object("seat", Part) -# back = unified_planning.model.Object("back", Part) - -# # * lamp -# base = unified_planning.model.Object("base", Part) -# blub = unified_planning.model.Object("blub", Part) -# lamp = unified_planning.model.Object("lamp", Part) - -# # * gearset -# gear1 = unified_planning.model.Object("gear1", Part) -# gear2 = unified_planning.model.Object("gear2", Part) -# gear3 = unified_planning.model.Object("gear3", Part) -# shaft1 = unified_planning.model.Object("shaft1", Part) -# shaft2 = unified_planning.model.Object("shaft2", Part) -# base = unified_planning.model.Object("base", Part) - -# ########################## * current problem context ###################### -# # add all the related instances to the problem here - -# # * hand instances -# problem.add_object(left_hand) - -# # * tool instances -# problem.add_object(parallel_box1) -# problem.add_object(parallel_box2) -# problem.add_object(inward_claw) -# problem.add_object(outward_claw) -# problem.add_object(no_tool) - -# # * part instances -# # * chair -# # not this time -# # * lamp -# # not this time -# # * gearset -# problem.add_object(gear1) -# problem.add_object(gear2) -# problem.add_object(gear3) -# problem.add_object(shaft1) -# problem.add_object(shaft2) -# problem.add_object(base) - -# ########################## * problem constraints ########################## - -# ############ * tools' manipulability -# problem.set_initial_value(can_manipulate(parallel_box1, gear1), True) -# problem.set_initial_value(can_manipulate(outward_claw, gear2), True) -# problem.set_initial_value(can_manipulate(inward_claw, gear3), True) -# problem.set_initial_value(can_manipulate(parallel_box2, shaft1), True) -# problem.set_initial_value(can_manipulate(no_tool, shaft2), True) - -# ############ * assembly constraints -# # * chair -# # not implemented yet -# # * lamp -# # not implemented yet -# # * gearset -# problem.set_initial_value(can_insert_to(shaft1, base), True) -# problem.set_initial_value(can_insert_to(shaft2, base), True) -# problem.set_initial_value(can_insert_to(gear3, shaft2), True) -# problem.set_initial_value(can_insert_to(gear2, base), True) -# problem.set_initial_value(can_insert_to(gear1, shaft1), True) - -# ########################## * initial world state ########################## -# # ! BBNOTE: LLM CONSENTRATION FOR STATE KEEPING LOAD SAVING -# # ! Default initial value: see fluent setup part above - - -# # problem.set_initial_value(hold(left_hand, parallel_box2), True) -# # problem.set_initial_value(hold(parallel_box2, shaft1), True) - -# ########################## * goal ########################################## - -# problem.add_goal(is_inserted_to(shaft1, base)) -# print(problem) - - -# # with OneshotPlanner(name="pyperplan") as planner: -# # result = planner.solve(problem) -# # if result.status == up.engines.PlanGenerationResultStatus.SOLVED_SATISFICING: -# # print("Pyperplan returned: %s" % result.plan) -# # else: -# # print("No plan found.") - -# with OneshotPlanner(problem_kind=problem.kind) as planner: -# result = planner.solve(problem) -# print("%s returned: %s" % (planner.name, result.plan)) diff --git a/kios_bt_planning/kios_domain/domain_v1.py b/kios_bt_planning/kios_domain/domain_v1.py deleted file mode 100644 index bf5d4324..00000000 --- a/kios_bt_planning/kios_domain/domain_v1.py +++ /dev/null @@ -1,364 +0,0 @@ -from unified_planning.shortcuts import * - -""" -is_free to is_empty, -is_equippable removed, -load/unload_tool to change_tool -""" -######################################## Type ######################################## - -Thing = UserType("Thing") - -Tool = UserType("Tool", father=Thing) -Part = UserType("Part", father=Thing) -# Feature = UserType("Feature", father=Part) - -Hand = UserType("Hand", father=Thing) - -######################################## Fluent ######################################## - -# * property -is_empty = Fluent("is_empty", BoolType(), p=Tool) - -# * relation -can_manipulate = Fluent("can_manipulate", BoolType(), tool=Tool, part=Part) -can_insert_to = Fluent("can_insert_to", BoolType(), part1=Part, part2=Part) -can_screw_to = Fluent("can_screw_to", BoolType(), part1=Part, part2=Part) -can_place_to = Fluent("can_place_to", BoolType(), part1=Part, part2=Part) -hold = Fluent("hold", BoolType(), thing1=Thing, thing2=Thing) -is_inserted_to = Fluent("is_inserted_to", BoolType(), part1=Part, part2=Part) -is_screwed_to = Fluent("is_screwed_to", BoolType(), part1=Part, part2=Part) -is_placed_to = Fluent("is_placed_to", BoolType(), part1=Part, part2=Part) - -######################################## Action ######################################## -# * change_tool -change_tool = unified_planning.model.InstantaneousAction( - "change_tool", - hand=Hand, - tool1=Tool, - tool2=Tool, -) -hand = change_tool.parameter("hand") -tool1 = change_tool.parameter("tool1") -tool2 = change_tool.parameter("tool2") - -change_tool.add_precondition(hold(hand, tool1)) -change_tool.add_precondition(is_empty(tool1)) - -change_tool.add_effect(hold(hand, tool1), False) -change_tool.add_effect(hold(hand, tool2), True) - - -# * pick_up -pick_up = unified_planning.model.InstantaneousAction( - "pick_up", - hand=Hand, - tool=Tool, - part=Part, -) - -part = pick_up.parameter("part") -tool = pick_up.parameter("tool") -hand = pick_up.parameter("hand") - -pick_up.add_precondition(is_empty(tool)) -pick_up.add_precondition(hold(hand, tool)) -pick_up.add_precondition(can_manipulate(tool, part)) - -pick_up.add_effect(hold(tool, part), True) -pick_up.add_effect(is_empty(tool), False) - -# * put_down -put_down = unified_planning.model.InstantaneousAction( - "put_down", - hand=Hand, - tool=Tool, - part=Part, -) - -part = put_down.parameter("part") -tool = put_down.parameter("tool") -hand = put_down.parameter("hand") - -put_down.add_precondition(hold(tool, part)) -put_down.add_precondition(hold(hand, tool)) - -put_down.add_effect(hold(hand, part), False) -put_down.add_effect(is_empty(tool), True) - - -# * place -place = unified_planning.model.InstantaneousAction( - "place", - hand=Hand, - tool=Tool, - part1=Part, - part2=Part, -) - -part1 = place.parameter("part1") -part2 = place.parameter("part2") -tool = place.parameter("tool") -hand = place.parameter("hand") - -place.add_precondition(hold(hand, tool)) -place.add_precondition(hold(tool, part1)) -place.add_precondition(can_place_to(part1, part2)) - -place.add_effect(hold(tool, part1), False) -place.add_effect(is_empty(tool), True) -place.add_effect(is_placed_to(part1, part2), True) - -# * detach -detach = unified_planning.model.InstantaneousAction( - "detach", - hand=Hand, - tool=Tool, - part1=Part, - part2=Part, -) - -part1 = detach.parameter("part1") -part2 = detach.parameter("part2") -tool = detach.parameter("tool") -hand = detach.parameter("hand") - -detach.add_precondition(hold(hand, tool)) -detach.add_precondition(is_empty(tool)) -detach.add_precondition(can_manipulate(tool, part1)) -detach.add_precondition(is_placed_to(part1, part2)) - -detach.add_effect(hold(tool, part1), True) -detach.add_effect(is_empty(tool), False) -detach.add_effect(is_placed_to(part1, part2), False) - -# * push, can be move -# ! implement later - -# * insert -insert = unified_planning.model.InstantaneousAction( - "insert", - hand=Hand, - tool=Tool, - part1=Part, - part2=Part, -) - -hand = insert.parameter("hand") -part1 = insert.parameter("part1") -part2 = insert.parameter("part2") -tool = insert.parameter("tool") - -insert.add_precondition(hold(hand, tool)) -insert.add_precondition(hold(tool, part1)) -insert.add_precondition(can_insert_to(part1, part2)) - -insert.add_effect(hold(tool, part1), False) -insert.add_effect(is_empty(tool), True) -insert.add_effect(is_inserted_to(part1, part2), True) - -# * pull-out -pullout = unified_planning.model.InstantaneousAction( - "pull", - hand=Hand, - tool=Tool, - part1=Part, - part2=Part, -) - -part1 = pullout.parameter("part1") -part2 = pullout.parameter("part2") -tool = pullout.parameter("tool") -hand = pullout.parameter("hand") - -pullout.add_precondition(hold(hand, tool)) -pullout.add_precondition(is_empty(tool)) -pullout.add_precondition(is_inserted_to(part1, part2)) -pullout.add_precondition(can_manipulate(tool, part1)) - -pullout.add_effect(hold(tool, part1), True) -pullout.add_effect(is_empty(tool), False) -pullout.add_effect(is_inserted_to(part1, part2), False) - -# * screw -screw = unified_planning.model.InstantaneousAction( - "screw", - hand=Hand, - tool=Tool, - part1=Part, - part2=Part, -) - -part1 = screw.parameter("part1") -part2 = screw.parameter("part2") -tool = screw.parameter("tool") -hand = screw.parameter("hand") - -screw.add_precondition(hold(hand, tool)) -screw.add_precondition(hold(tool, part1)) -screw.add_precondition(can_screw_to(part1, part2)) - -screw.add_effect(hold(tool, part1), False) -screw.add_effect(is_empty(tool), True) -screw.add_effect(is_screwed_to(part1, part2), True) - -# * unscrew -unscrew = unified_planning.model.InstantaneousAction( - "unscrew", - hand=Hand, - tool=Tool, - part1=Part, - part2=Part, -) - -part1 = unscrew.parameter("part1") -part2 = unscrew.parameter("part2") -tool = unscrew.parameter("tool") -hand = unscrew.parameter("hand") - -unscrew.add_precondition(hold(hand, tool)) -unscrew.add_precondition(is_empty(tool)) -unscrew.add_precondition(is_screwed_to(part1, part2)) -unscrew.add_precondition(can_manipulate(tool, part1)) - -unscrew.add_effect(hold(tool, part1), True) -unscrew.add_effect(is_empty(tool), False) -unscrew.add_effect(is_screwed_to(part1, part2), False) - -######################################## Problem ######################################## - -# problem = Problem("kios_tamp_problem") - -# ########################## * knowledge - -# # * property -# problem.add_fluent(is_empty, default_initial_value=True) -# problem.add_fluent(is_equippable, default_initial_value=True) - -# # * relation -# problem.add_fluent(can_manipulate, default_initial_value=False) -# problem.add_fluent(can_insert_to, default_initial_value=False) -# problem.add_fluent(can_screw_to, default_initial_value=False) -# problem.add_fluent(can_place_to, default_initial_value=False) -# problem.add_fluent(hold, default_initial_value=False) -# problem.add_fluent(is_inserted_to, default_initial_value=False) -# problem.add_fluent(is_screwed_to, default_initial_value=False) -# problem.add_fluent(is_placed_to, default_initial_value=False) - -# # * action - -# problem.add_action(pick_up) -# problem.add_action(put_down) -# problem.add_action(place) -# problem.add_action(detach) -# problem.add_action(insert) -# problem.add_action(pullout) -# problem.add_action(screw) -# problem.add_action(unscrew) -# problem.add_action(load_tool) -# problem.add_action(unload_tool) - -# ########################* instances ######################################## -# # hand instances -# left_hand = unified_planning.model.Object("left_hand", Hand) - -# # tool instances -# parallel_box1 = unified_planning.model.Object("parallel_box1", Tool) -# parallel_box2 = unified_planning.model.Object("parallel_box2", Tool) -# inward_claw = unified_planning.model.Object("inward_claw", Tool) -# outward_claw = unified_planning.model.Object("outward_claw", Tool) -# no_tool = unified_planning.model.Object("no_tool", Tool) - -# # part instances -# # * chair -# leg1 = unified_planning.model.Object("leg1", Part) -# leg2 = unified_planning.model.Object("leg2", Part) -# nut1 = unified_planning.model.Object("nut1", Part) -# nut2 = unified_planning.model.Object("nut2", Part) -# seat = unified_planning.model.Object("seat", Part) -# back = unified_planning.model.Object("back", Part) - -# # * lamp -# base = unified_planning.model.Object("base", Part) -# blub = unified_planning.model.Object("blub", Part) -# lamp = unified_planning.model.Object("lamp", Part) - -# # * gearset -# gear1 = unified_planning.model.Object("gear1", Part) -# gear2 = unified_planning.model.Object("gear2", Part) -# gear3 = unified_planning.model.Object("gear3", Part) -# shaft1 = unified_planning.model.Object("shaft1", Part) -# shaft2 = unified_planning.model.Object("shaft2", Part) -# base = unified_planning.model.Object("base", Part) - -# ########################## * current problem context ###################### -# # add all the related instances to the problem here - -# # * hand instances -# problem.add_object(left_hand) - -# # * tool instances -# problem.add_object(parallel_box1) -# problem.add_object(parallel_box2) -# problem.add_object(inward_claw) -# problem.add_object(outward_claw) -# problem.add_object(no_tool) - -# # * part instances -# # * chair -# # not this time -# # * lamp -# # not this time -# # * gearset -# problem.add_object(gear1) -# problem.add_object(gear2) -# problem.add_object(gear3) -# problem.add_object(shaft1) -# problem.add_object(shaft2) -# problem.add_object(base) - -# ########################## * problem constraints ########################## - -# ############ * tools' manipulability -# problem.set_initial_value(can_manipulate(parallel_box1, gear1), True) -# problem.set_initial_value(can_manipulate(outward_claw, gear2), True) -# problem.set_initial_value(can_manipulate(inward_claw, gear3), True) -# problem.set_initial_value(can_manipulate(parallel_box2, shaft1), True) -# problem.set_initial_value(can_manipulate(no_tool, shaft2), True) - -# ############ * assembly constraints -# # * chair -# # not implemented yet -# # * lamp -# # not implemented yet -# # * gearset -# problem.set_initial_value(can_insert_to(shaft1, base), True) -# problem.set_initial_value(can_insert_to(shaft2, base), True) -# problem.set_initial_value(can_insert_to(gear3, shaft2), True) -# problem.set_initial_value(can_insert_to(gear2, base), True) -# problem.set_initial_value(can_insert_to(gear1, shaft1), True) - -# ########################## * initial world state ########################## -# # ! BBNOTE: LLM CONSENTRATION FOR STATE KEEPING LOAD SAVING -# # ! Default initial value: see fluent setup part above - - -# # problem.set_initial_value(hold(left_hand, parallel_box2), True) -# # problem.set_initial_value(hold(parallel_box2, shaft1), True) - -# ########################## * goal ########################################## - -# problem.add_goal(is_inserted_to(shaft1, base)) -# print(problem) - - -# # with OneshotPlanner(name="pyperplan") as planner: -# # result = planner.solve(problem) -# # if result.status == up.engines.PlanGenerationResultStatus.SOLVED_SATISFICING: -# # print("Pyperplan returned: %s" % result.plan) -# # else: -# # print("No plan found.") - -# with OneshotPlanner(problem_kind=problem.kind) as planner: -# result = planner.solve(problem) -# print("%s returned: %s" % (planner.name, result.plan)) diff --git a/kios_bt_planning/kios_domain/domains/domain_001.pddl b/kios_bt_planning/kios_domain/domains/domain_001.pddl deleted file mode 100644 index 060e98ae..00000000 --- a/kios_bt_planning/kios_domain/domains/domain_001.pddl +++ /dev/null @@ -1,48 +0,0 @@ -(define (domain robot_assembly_problem-domain) - (:requirements :strips :typing) - (:types - thing - object - tool part hand - thing - ) - (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part)) - (:action pick_up - :parameters ( ?hand - hand ?tool - tool ?part - part) - :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part)) - :effect (and (hold ?tool ?part) (not (is_free ?tool)))) - (:action put_down - :parameters ( ?hand - hand ?tool - tool ?part - part) - :precondition (and (hold ?tool ?part) (hold ?hand ?tool)) - :effect (and (not (hold ?hand ?part)) (is_free ?tool))) - (:action place - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2))) - (:action detach - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2)))) - (:action insert - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2))) - (:action pull - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2)))) - (:action screw - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2)) - :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2))) - (:action unscrew - :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part) - :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1)) - :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2)))) - (:action load_tool - :parameters ( ?hand - hand ?tool - tool) - :precondition (and (is_equippable ?tool) (is_free ?hand)) - :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool))) - (:action unload_tool - :parameters ( ?hand - hand ?tool - tool) - :precondition (and (hold ?hand ?tool) (is_free ?tool)) - :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool)))) -) diff --git a/kios_bt_planning/kios_domain/knowledge_v0.py b/kios_bt_planning/kios_domain/knowledge_v0.py deleted file mode 100644 index 785d127b..00000000 --- a/kios_bt_planning/kios_domain/knowledge_v0.py +++ /dev/null @@ -1,102 +0,0 @@ -from kios_domain.domain_v0 import * - -problem = Problem("robot_assembly_problem") - -########################## * knowledge - -# * property -# ! BBMOD -problem.add_fluent(is_free, default_initial_value=False) -problem.add_fluent(is_equippable, default_initial_value=False) - -# * relation -problem.add_fluent(can_manipulate, default_initial_value=False) -problem.add_fluent(can_insert_to, default_initial_value=False) -problem.add_fluent(can_screw_to, default_initial_value=False) -problem.add_fluent(can_place_to, default_initial_value=False) -problem.add_fluent(hold, default_initial_value=False) -problem.add_fluent(is_inserted_to, default_initial_value=False) -problem.add_fluent(is_screwed_to, default_initial_value=False) -problem.add_fluent(is_placed_to, default_initial_value=False) - -# * action - -problem.add_action(pick_up) -problem.add_action(put_down) -problem.add_action(place) -problem.add_action(detach) -problem.add_action(insert) -problem.add_action(pullout) -problem.add_action(screw) -problem.add_action(unscrew) -problem.add_action(load_tool) -problem.add_action(unload_tool) - -########################* instances ######################################## -# hand instances -left_hand = unified_planning.model.Object("left_hand", Hand) - -# tool instances -parallel_box1 = unified_planning.model.Object("parallel_box1", Tool) -parallel_box2 = unified_planning.model.Object("parallel_box2", Tool) -inward_claw = unified_planning.model.Object("inward_claw", Tool) -outward_claw = unified_planning.model.Object("outward_claw", Tool) -no_tool = unified_planning.model.Object("no_tool", Tool) - -# part instances -# * chair -leg1 = unified_planning.model.Object("leg1", Part) -leg2 = unified_planning.model.Object("leg2", Part) -nut1 = unified_planning.model.Object("nut1", Part) -nut2 = unified_planning.model.Object("nut2", Part) -seat = unified_planning.model.Object("seat", Part) -back = unified_planning.model.Object("back", Part) - -# * lamp -lampbase = unified_planning.model.Object("lampbase", Part) -blub = unified_planning.model.Object("blub", Part) -lamp = unified_planning.model.Object("lamp", Part) - -# * gearset -gear1 = unified_planning.model.Object("gear1", Part) -gear2 = unified_planning.model.Object("gear2", Part) -gear3 = unified_planning.model.Object("gear3", Part) - -shaft1 = unified_planning.model.Object("shaft1", Part) -shaft2 = unified_planning.model.Object("shaft2", Part) -shaft3 = unified_planning.model.Object("shaft3", Part) - -gearbase = unified_planning.model.Object("gearbase", Part) - -gearbase_hole1 = unified_planning.model.Object("gearbase_hole1", Part) -gearbase_hole3 = unified_planning.model.Object("gearbase_hole3", Part) - - - -########################## * problem constraints ########################## - -############ * tools' manipulability -problem.set_initial_value(can_manipulate(parallel_box2, gear1), True) -problem.set_initial_value(can_manipulate(outward_claw, gear2), True) -problem.set_initial_value(can_manipulate(outward_claw, gear3), True) -problem.set_initial_value(can_manipulate(no_tool, shaft3), True) -problem.set_initial_value(can_manipulate(parallel_box1, shaft1), True) - -############ * assembly constraints -# * chair -problem.set_initial_value(can_screw_to(leg1, seat), True) -problem.set_initial_value(can_screw_to(leg2, seat), True) -problem.set_initial_value(can_insert_to(back, seat), True) -problem.set_initial_value(can_screw_to(nut1, seat), True) -problem.set_initial_value(can_screw_to(nut2, seat), True) - -# * lamp -problem.set_initial_value(can_screw_to(blub, lampbase), True) -problem.set_initial_value(can_place_to(lamp, blub), True) - -# * gearset -problem.set_initial_value(can_insert_to(shaft1, gearbase_hole1), True) -problem.set_initial_value(can_insert_to(shaft3, gearbase_hole3), True) -problem.set_initial_value(can_insert_to(gear3, shaft3), True) -problem.set_initial_value(can_insert_to(gear2, shaft2), True) -problem.set_initial_value(can_insert_to(gear1, shaft1), True) diff --git a/kios_bt_planning/kios_domain/knowledge_v1.py b/kios_bt_planning/kios_domain/knowledge_v1.py deleted file mode 100644 index 484a8d3c..00000000 --- a/kios_bt_planning/kios_domain/knowledge_v1.py +++ /dev/null @@ -1,105 +0,0 @@ -from kios_domain.domain_v1 import * - -problem = Problem("robot_assembly_problem") - -########################## * knowledge - -# * property -# ! BBMOD -problem.add_fluent(is_empty, default_initial_value=False) - -# * relation -problem.add_fluent(can_manipulate, default_initial_value=False) -problem.add_fluent(can_insert_to, default_initial_value=False) -problem.add_fluent(can_screw_to, default_initial_value=False) -problem.add_fluent(can_place_to, default_initial_value=False) -problem.add_fluent(hold, default_initial_value=False) -problem.add_fluent(is_inserted_to, default_initial_value=False) -problem.add_fluent(is_screwed_to, default_initial_value=False) -problem.add_fluent(is_placed_to, default_initial_value=False) - -# * action - -problem.add_action(pick_up) -problem.add_action(put_down) -problem.add_action(place) -problem.add_action(detach) -problem.add_action(insert) -problem.add_action(pullout) -problem.add_action(screw) -problem.add_action(unscrew) -problem.add_action(change_tool) - -########################* instances ######################################## -# hand instances -left_hand = unified_planning.model.Object("left_hand", Hand) - -# tool instances -clampgripper = unified_planning.model.Object("clampgripper", Tool) -parallelgripper = unified_planning.model.Object("parallelgripper", Tool) -inwardgripper = unified_planning.model.Object("inwardgripper", Tool) -outwardgripper = unified_planning.model.Object("outwardgripper", Tool) -defaultgripper = unified_planning.model.Object("defaultgripper", Tool) - -# part instances -# * chair -leg1 = unified_planning.model.Object("leg1", Part) -leg2 = unified_planning.model.Object("leg2", Part) -nut1 = unified_planning.model.Object("nut1", Part) -nut2 = unified_planning.model.Object("nut2", Part) -seat = unified_planning.model.Object("seat", Part) -chairback = unified_planning.model.Object("chairback", Part) - -# * lamp -lampbase = unified_planning.model.Object("lampbase", Part) -lampblub = unified_planning.model.Object("lampblub", Part) -lampshade = unified_planning.model.Object("lampshade", Part) - -# * cabinet - -# * square table - -# * round table - -# * gearset -gear1 = unified_planning.model.Object("gear1", Part) -gear2 = unified_planning.model.Object("gear2", Part) -gear3 = unified_planning.model.Object("gear3", Part) - -shaft1 = unified_planning.model.Object("shaft1", Part) -shaft2 = unified_planning.model.Object("shaft2", Part) -shaft3 = unified_planning.model.Object("shaft3", Part) - -gearbase = unified_planning.model.Object("gearbase", Part) - -gearbase_hole1 = unified_planning.model.Object("gearbase_hole1", Part) -gearbase_hole3 = unified_planning.model.Object("gearbase_hole3", Part) - - -########################## * problem constraints ########################## - -############ * tools' manipulability -problem.set_initial_value(can_manipulate(parallelgripper, gear1), True) -problem.set_initial_value(can_manipulate(outwardgripper, gear2), True) -problem.set_initial_value(can_manipulate(outwardgripper, gear3), True) -problem.set_initial_value(can_manipulate(defaultgripper, shaft3), True) -problem.set_initial_value(can_manipulate(clampgripper, shaft1), True) - -############ * assembly constraints -# * chair -problem.set_initial_value(can_screw_to(leg1, seat), True) -problem.set_initial_value(can_screw_to(leg2, seat), True) -problem.set_initial_value(can_insert_to(chairback, seat), True) -problem.set_initial_value(can_screw_to(nut1, seat), True) -problem.set_initial_value(can_screw_to(nut2, seat), True) - -# * lamp -problem.set_initial_value(can_screw_to(lampblub, lampbase), True) -problem.set_initial_value(can_place_to(lampshade, lampblub), True) - -# * gearset -problem.set_initial_value(can_insert_to(shaft1, gearbase_hole1), True) -problem.set_initial_value(can_insert_to(shaft3, gearbase_hole3), True) -problem.set_initial_value(can_insert_to(gear3, shaft3), True) -problem.set_initial_value(can_insert_to(gear2, shaft2), True) -problem.set_initial_value(can_insert_to(gear1, shaft1), True) diff --git a/kios_bt_planning/kios_domain/pddl_interface.py b/kios_bt_planning/kios_domain/pddl_interface.py deleted file mode 100644 index 45ea1318..00000000 --- a/kios_bt_planning/kios_domain/pddl_interface.py +++ /dev/null @@ -1,175 +0,0 @@ -# Import the PDDLReader and PDDLWriter classes -from unified_planning.io import PDDLReader, PDDLWriter - -from unified_planning.shortcuts import * - -import os -import os.path -import re - -from typing import List, Dict, Any, Tuple - - -class PddlInterface: - pddl_writer: PDDLWriter = None - pddl_reader: PDDLReader = None - problem: Problem = None - default_domain_path: str = None - default_problem_path: str = None - - pddl_domain_file_name: str = "domain_001.pddl" - pddl_problem_file_name: str = "problem_001.pddl" - - def __init__( - self, - default_domain_path: str = None, - default_problem_path: str = None, - ): - if default_domain_path is None: - self.default_domain_path = os.path.join( - os.path.dirname(os.path.realpath(__file__)), - "domains", - ) - if not os.path.exists(self.default_domain_path): - os.makedirs(self.default_domain_path) - if default_problem_path is None: - self.default_problem_path = os.path.join( - os.path.dirname(os.path.realpath(__file__)), - "problems", - ) - if not os.path.exists(self.default_problem_path): - os.makedirs(self.default_problem_path) - - print( - "\033[93m################################## BB WARNING ###################################" - ) - print("default_domain_path: ", self.default_domain_path) - print("default_problem_path: ", self.default_problem_path) - print("Please be aware of problem consistency:") - print("Known error: syntax error can happen if the initial state is") - print("using some instance that is not added into the problem. This") - print("can always happen when the pddl file you want to parse is ") - print("generated from unified planning problem by the pddl writer.") - print("This won't be fixed currently. Please be aware of this.") - print( - "################################################################################" - ) - print("\033[0m") # Reset color to default - - def set_pddl_names(self, domain_name: str, problem_name: str): - self.pddl_domain_file_name = domain_name - self.pddl_problem_file_name = problem_name - - def update_problem(self): - pass - - def setup_from_up(self, problem: Problem): - self.problem = problem - self.pddl_writer = PDDLWriter(problem) - self.pddl_reader = PDDLReader() - - def setup_from_pddl( - self, - domain_file_name: str, - problem_file_name: str, - domain_path: str = None, - problem_path: str = None, - ): - if domain_path is None: - domain_path = self.default_domain_path - if problem_path is None: - problem_path = self.default_problem_path - - if domain_file_name is None or problem_file_name is None: - raise Exception("YOU MUST SPECIFY THE DOMAIN AND PROBLEM FILE NAME") - - self.pddl_reader = PDDLReader() - - self.problem = self.pddl_reader.parse_problem( - os.path.join(domain_path, self.pddl_domain_file_name), - os.path.join(problem_path, self.pddl_problem_file_name), - ) - self.pddl_writer = PDDLWriter(self.problem) - - def write_pddl( - self, - domain_path: str = None, - problem_path: str = None, - ): - if domain_path is None: - domain_path = self.default_domain_path - if problem_path is None: - problem_path = self.default_problem_path - - if self.problem is None: - raise Exception("Problem is not set") - - self.update_problem() - self.pddl_writer = PDDLWriter(self.problem) - - domain_file_path = os.path.join(domain_path, self.pddl_domain_file_name) - problem_file_path = os.path.join(problem_path, self.pddl_problem_file_name) - - # Check if the domain and problem files already exist, if so, - # modify the file name using regular expressions - if os.path.exists(domain_file_path): - # Modify the domain file name using regular expressions - domain_file_name = self.pddl_domain_file_name - match = re.match(r"domain_(\d+)\.pddl", domain_file_name) - if match: - file_number = int(match.group(1)) - while os.path.exists( - os.path.join(domain_path, f"domain_{file_number:03}.pddl") - ): - file_number += 1 - domain_file_name = f"domain_{file_number:03}.pddl" - else: - domain_file_name = "domain_001.pddl" - - domain_file_path = os.path.join(domain_path, domain_file_name) - - if os.path.exists(problem_file_path): - # Modify the problem file name using regular expressions - problem_file_name = self.pddl_problem_file_name - match = re.match(r"problem_(\d+)\.pddl", problem_file_name) - if match: - file_number = int(match.group(1)) - while os.path.exists( - os.path.join(problem_path, f"problem_{file_number:03}.pddl") - ): - file_number += 1 - problem_file_name = f"problem_{file_number:03}.pddl" - else: - problem_file_name = "problem_001.pddl" - - problem_file_path = os.path.join(problem_path, problem_file_name) - - # Create the directories if they don't exist - if not os.path.exists(domain_path): - os.makedirs(domain_path) - if not os.path.exists(problem_path): - os.makedirs(problem_path) - - self.pddl_writer.write_domain(domain_file_path) - self.pddl_writer.write_problem(problem_file_path) - - def get_pddl_string(self) -> Tuple[str, str]: - if self.problem is None: - raise Exception("Problem is not set") - self.update_problem() - writer = PDDLWriter(self.problem) - return writer.get_domain(), writer.get_problem() - - def print_domain(self): - if self.problem is None: - raise Exception("Problem is not set") - self.update_problem() - writer = PDDLWriter(self.problem) - writer.print_domain() - - def print_problem(self): - if self.problem is None: - raise Exception("Problem is not set") - self.update_problem() - writer = PDDLWriter(self.problem) - writer.print_problem() diff --git a/kios_bt_planning/kios_domain/problem1.py b/kios_bt_planning/kios_domain/problem1.py deleted file mode 100644 index 15beeef7..00000000 --- a/kios_bt_planning/kios_domain/problem1.py +++ /dev/null @@ -1,59 +0,0 @@ -from kios_domain.knowledge_v0 import * - -########################## * current problem context ###################### -# add all the related instances to the problem here - -# * hand instances -problem.add_object(left_hand) - -# * tool instances -problem.add_object(parallel_box1) -problem.add_object(parallel_box2) -problem.add_object(inward_claw) -problem.add_object(outward_claw) -problem.add_object(no_tool) - -# * part instances -# * chair -# not this time -# * lamp -# not this time -# * gearset -problem.add_object(gear1) -problem.add_object(gear2) -problem.add_object(gear3) -problem.add_object(shaft1) -problem.add_object(shaft2) -problem.add_object(gearset_base) - - -########################## * initial world state ########################## -# ! BBNOTE: LLM CONSENTRATION FOR STATE KEEPING LOAD SAVING -# ! Default initial value: hand is free, tools are equippable and free, parts are not assembled -problem.set_initial_value(is_free(left_hand), True) -problem.set_initial_value(is_free(parallel_box1), True) -problem.set_initial_value(is_free(parallel_box2), True) -problem.set_initial_value(is_free(inward_claw), True) -problem.set_initial_value(is_free(outward_claw), True) - -problem.set_initial_value(is_equippable(parallel_box1), True) -problem.set_initial_value(is_equippable(parallel_box2), True) -problem.set_initial_value(is_equippable(inward_claw), True) -problem.set_initial_value(is_equippable(outward_claw), True) - - -########################## * goal ########################################## -problem.add_goal(is_inserted_to(shaft1, gearset_base)) -# print(problem) - -########################## * planning ###################################### -# with OneshotPlanner(name="pyperplan") as planner: -# result = planner.solve(problem) -# if result.status == up.engines.PlanGenerationResultStatus.SOLVED_SATISFICING: -# print("Pyperplan returned: %s" % result.plan) -# else: -# print("No plan found.") - -# with OneshotPlanner(problem_kind=problem.kind) as planner: -# result = planner.solve(problem) -# print("%s returned: %s" % (planner.name, result.plan)) diff --git a/kios_bt_planning/kios_domain/problem2.py b/kios_bt_planning/kios_domain/problem2.py deleted file mode 100644 index 1f1d3522..00000000 --- a/kios_bt_planning/kios_domain/problem2.py +++ /dev/null @@ -1,60 +0,0 @@ -from kios_domain.knowledge_v0 import * - -########################## * current problem context ###################### -# add all the related instances to the problem here - -# * hand instances -problem.add_object(left_hand) - -# * tool instances -problem.add_object(parallel_box1) -problem.add_object(parallel_box2) -problem.add_object(inward_claw) -problem.add_object(outward_claw) -problem.add_object(no_tool) - -# * part instances -# * chair -# not this time -# * lamp -# not this time -# * gearset -problem.add_object(gear1) -problem.add_object(gear2) -problem.add_object(gear3) -problem.add_object(shaft1) -problem.add_object(shaft2) -problem.add_object(gearset_base) - - -########################## * initial world state ########################## -# ! BBNOTE: LLM CONSENTRATION FOR STATE KEEPING LOAD SAVING -# ! Default initial value: hand is free, tools are equippable and free, parts are not assembled -problem.set_initial_value(is_free(left_hand), True) -problem.set_initial_value(is_free(parallel_box1), True) -problem.set_initial_value(is_free(parallel_box2), True) -problem.set_initial_value(is_free(inward_claw), True) -problem.set_initial_value(is_free(outward_claw), True) - -problem.set_initial_value(is_equippable(parallel_box1), True) -problem.set_initial_value(is_equippable(parallel_box2), True) -problem.set_initial_value(is_equippable(inward_claw), True) -problem.set_initial_value(is_equippable(outward_claw), True) - -problem.set_initial_value(is_inserted_to(shaft1, gearset_base), True) - -########################## * goal ########################################## -problem.add_goal(is_inserted_to(gear1, shaft1)) -# print(problem) - -########################## * planning ###################################### -# with OneshotPlanner(name="pyperplan") as planner: -# result = planner.solve(problem) -# if result.status == up.engines.PlanGenerationResultStatus.SOLVED_SATISFICING: -# print("Pyperplan returned: %s" % result.plan) -# else: -# print("No plan found.") - -with OneshotPlanner(problem_kind=problem.kind) as planner: - result = planner.solve(problem) - print("%s returned: %s" % (planner.name, result.plan)) diff --git a/kios_bt_planning/kios_domain/problem3.py b/kios_bt_planning/kios_domain/problem3.py deleted file mode 100644 index 8b48b2c8..00000000 --- a/kios_bt_planning/kios_domain/problem3.py +++ /dev/null @@ -1,60 +0,0 @@ -from kios_domain.knowledge_v0 import * - -########################## * current problem context ###################### -# add all the related instances to the problem here - -# * hand instances -problem.add_object(left_hand) - -# * tool instances -problem.add_object(parallel_box1) -problem.add_object(parallel_box2) -problem.add_object(inward_claw) -problem.add_object(outward_claw) -problem.add_object(no_tool) - -# * part instances -# * chair -# not this time -# * lamp -# not this time -# * gearset -problem.add_object(gear1) -problem.add_object(gear2) -problem.add_object(gear3) -problem.add_object(shaft1) -problem.add_object(shaft2) -problem.add_object(gearset_base) - - -########################## * initial world state ########################## -# ! BBNOTE: LLM CONSENTRATION FOR STATE KEEPING LOAD SAVING -# ! Default initial value: hand is free, tools are equippable and free, parts are not assembled -problem.set_initial_value(is_free(left_hand), True) -problem.set_initial_value(is_free(parallel_box1), True) -problem.set_initial_value(is_free(parallel_box2), True) -problem.set_initial_value(is_free(inward_claw), True) -problem.set_initial_value(is_free(outward_claw), True) - -problem.set_initial_value(is_equippable(parallel_box1), True) -problem.set_initial_value(is_equippable(parallel_box2), True) -problem.set_initial_value(is_equippable(inward_claw), True) -problem.set_initial_value(is_equippable(outward_claw), True) - -problem.set_initial_value(is_inserted_to(shaft2, gearset_base), True) - -########################## * goal ########################################## -problem.add_goal(is_inserted_to(gear3, shaft2)) -# print(problem) - -########################## * planning ###################################### -# with OneshotPlanner(name="pyperplan") as planner: -# result = planner.solve(problem) -# if result.status == up.engines.PlanGenerationResultStatus.SOLVED_SATISFICING: -# print("Pyperplan returned: %s" % result.plan) -# else: -# print("No plan found.") - -with OneshotPlanner(problem_kind=problem.kind) as planner: - result = planner.solve(problem) - print("%s returned: %s" % (planner.name, result.plan)) diff --git a/kios_bt_planning/kios_domain/problem4.py b/kios_bt_planning/kios_domain/problem4.py deleted file mode 100644 index 4856a680..00000000 --- a/kios_bt_planning/kios_domain/problem4.py +++ /dev/null @@ -1,75 +0,0 @@ -from kios_domain.knowledge_v0 import * - -########################## * current problem context ###################### -# add all the related instances to the problem here - -# * hand instances -problem.add_object(left_hand) - -# * tool instances -problem.add_object(parallel_box1) -problem.add_object(parallel_box2) -problem.add_object(inward_claw) -problem.add_object(outward_claw) -problem.add_object(no_tool) - -# * part instances -# * chair -# not this time -# * lamp -# not this time -# * gearset -problem.add_object(gear1) -problem.add_object(gear2) -problem.add_object(gear3) - -problem.add_object(shaft1) -problem.add_object(shaft2) -problem.add_object(shaft3) - -problem.add_object(gearbase) -problem.add_object(gearbase_hole1) -problem.add_object(gearbase_hole3) - - -########################## * initial world state ########################## -# ! BBNOTE: LLM CONSENTRATION FOR STATE KEEPING LOAD SAVING -# ! Default initial value: hand is free, tools are equippable and free, parts are not assembled -problem.set_initial_value(is_free(left_hand), True) -problem.set_initial_value(is_free(parallel_box1), True) -problem.set_initial_value(is_free(parallel_box2), True) -problem.set_initial_value(is_free(inward_claw), True) -problem.set_initial_value(is_free(outward_claw), True) -problem.set_initial_value(is_free(no_tool), True) - -problem.set_initial_value(is_equippable(parallel_box1), True) -problem.set_initial_value(is_equippable(parallel_box2), True) -problem.set_initial_value(is_equippable(inward_claw), True) -problem.set_initial_value(is_equippable(outward_claw), True) -problem.set_initial_value(is_equippable(no_tool), True) - -########################## * goal ########################################## -# problem.add_goal(is_inserted_to(shaft1, gearbase_hole1)) -# print(problem) - -########################## * check ######################################### -from pprint import pprint - -with SequentialSimulator(problem=problem) as simulator: - initial_state = simulator.get_initial_state() - if simulator.is_applicable(initial_state, load_tool, (left_hand, parallel_box1)): - print(f"load_tool is applicable!") - -# pprint(problem) - -########################## * planning ###################################### -# with OneshotPlanner(name="pyperplan") as planner: -# result = planner.solve(problem) -# if result.status == up.engines.PlanGenerationResultStatus.SOLVED_SATISFICING: -# print("Pyperplan returned: %s" % result.plan) -# else: -# print("No plan found.") - -with OneshotPlanner(problem_kind=problem.kind) as planner: - result = planner.solve(problem) - print("%s returned: %s" % (planner.name, result.plan)) diff --git a/kios_bt_planning/kios_domain/problems/problem_001.pddl b/kios_bt_planning/kios_domain/problems/problem_001.pddl deleted file mode 100644 index 022efa3a..00000000 --- a/kios_bt_planning/kios_domain/problems/problem_001.pddl +++ /dev/null @@ -1,10 +0,0 @@ -(define (problem robot_assembly_problem-problem) - (:domain robot_assembly_problem-domain) - (:objects - parallel_box1 parallel_box2 inward_claw outward_claw no_tool - tool - gear1 gear2 gear3 shaft1 shaft2 shaft3 gearbase gearbase_hole1 gearbase_hole3 - part - left_hand - hand - ) - (:init (can_manipulate parallel_box2 gear1) (can_manipulate outward_claw gear2) (can_manipulate outward_claw gear3) (can_manipulate no_tool shaft3) (can_manipulate parallel_box1 shaft1) (can_screw_to leg1 seat) (can_screw_to leg2 seat) (can_insert_to back seat) (can_screw_to nut1 seat) (can_screw_to nut2 seat) (can_screw_to blub lampbase) (can_place_to lamp blub) (can_insert_to shaft1 gearbase_hole1) (can_insert_to shaft3 gearbase_hole3) (can_insert_to gear3 shaft3) (can_insert_to gear2 shaft2) (can_insert_to gear1 shaft1)) - (:goal (and )) -) diff --git a/kios_bt_planning/kios_future/__init__.py b/kios_bt_planning/kios_future/__init__.py deleted file mode 100644 index e69de29b..00000000 diff --git a/kios_bt_planning/kios_future/embeddings.py b/kios_bt_planning/kios_future/embeddings.py deleted file mode 100644 index c8078bcc..00000000 --- a/kios_bt_planning/kios_future/embeddings.py +++ /dev/null @@ -1,33 +0,0 @@ -from kios_utils.parsers import match_type - - -def embed_ut_nl(unit_subtree: dict) -> str: - """ - embed the unit subtree into the overall tree - """ - selector_children = unit_subtree["children"] - target = "" - # * target - for node in selector_children: - if match_type(node)[0] == "target": - target += node["summary"] - - sequence_children = selector_children[1]["children"] - preconditions = [] - action = "" - for node in sequence_children: - if match_type(node)[0] == "precondition": - preconditions.append(node["summary"]) - if match_type(node)[0] == "action": - action = node["summary"] - - embedding = f"if {' and '.join(preconditions)} then {action}, {target}" - - return embedding - - -def embed_ft_nl(fulltree: dict) -> str: - pass - - - diff --git a/kios_bt_planning/kios_plan/__init__.py b/kios_bt_planning/kios_plan/__init__.py deleted file mode 100644 index e69de29b..00000000 diff --git a/kios_bt_planning/kios_plan/dynamic_planning.py b/kios_bt_planning/kios_plan/dynamic_planning.py deleted file mode 100644 index b7a1d537..00000000 --- a/kios_bt_planning/kios_plan/dynamic_planning.py +++ /dev/null @@ -1,503 +0,0 @@ -hand_list = [ - "left_hand", -] -tool_list = [ - "inwardgripper", - "outwardgripper", - "parallelgripper", - "clampgripper", - "defaultgripper", -] -part_list = [ - "gear1", - "shaft1", - "gear2", - "shaft2", - "gear3", - "shaft3", - "gearbase_hole1", - "gearbase_hole3", - "gearbase", -] - - -def generate_unit_tree_entry(action_create_func: object, kwargs: dict): - """ - create action - """ - return action_create_func(**kwargs) - - -def create_pick_up(hand: str, tool: str, part1: str): - """ - create pick up action - """ - target = f"hold({tool}, {part1})" - ut = { - "summary": f"selector to pick up the {part1} with the {tool} in the {hand}", - "name": f"selector: pick_up({hand}, {tool}, {part1})", - "children": [ - { - "summary": f"the target is to make the {tool} hold the {part1}", - "name": f"target: hold({tool}, {part1})", - }, - { - "summary": f"sequence to pick up the {part1} with the {tool} in the {hand}", - "name": f"sequence: pick_up({hand}, {tool}, {part1})", - "children": [ - { - "summary": f"a precondition is the {tool} is empty", - "name": f"precondition: is_empty({tool})", - }, - { - "summary": f"a precondition is the {hand} is holding {tool}", - "name": f"precondition: hold({hand}, {tool})", - }, - { - "summary": f"the action to pick up the {part1} with the {tool} in the {hand}", - "name": f"action: pick_up({hand}, {tool}, {part1})", - }, - ], - }, - ], - } - return target, ut - - -def create_change_tool(hand: str, tool1: str, tool2: str): - """ - create change tool action - """ - target = f"hold({hand}, {tool2})" - ut = { - "summary": f"selector to change tool in {hand} from {tool1} to {tool2}", - "name": f"selector: change_tool({hand}, {tool1}, {tool2})", - "children": [ - { - "summary": f"the target is to make {hand} hold {tool2}", - "name": f"target: hold({hand}, {tool2})", - }, - { - "summary": f"sequence to change tool in {hand} from {tool1} to {tool2}", - "name": f"sequence: change_tool({hand}, {tool1}, {tool2})", - "children": [ - { - "summary": f"a precondition is {hand} is holding {tool1}", - "name": f"precondition: hold({hand}, {tool1})", - }, - { - "summary": f"a precondition is {tool1} is empty", - "name": f"precondition: is_empty({tool1})", - }, - { - "summary": f"the action to change tool in {hand} from {tool1} to {tool2}", - "name": f"action: change_tool({hand}, {tool1}, {tool2})", - }, - ], - }, - ], - } - return target, ut - - -def create_put_down(hand: str, tool: str, part: str): - """ - create put down action - """ - target = f"is_empty({tool})" - ut = { - "summary": f"selector to put down the {part} held by the {tool} in the {hand}", - "name": f"selector: put_down({hand}, {tool}, {part})", - "children": [ - { - "summary": f"the target is to make the {tool} empty", - "name": f"target: is_empty({tool})", - }, - { - "summary": f"sequence to put down the {part} held by the {tool} in the {hand}", - "name": f"sequence: put_down({hand}, {tool}, {part})", - "children": [ - { - "summary": f"a precondition is the {hand} is holding {tool}", - "name": f"precondition: hold({hand}, {tool})", - }, - { - "summary": f"a precondition is the {tool} is holding {part}", - "name": f"precondition: hold({tool}, {part})", - }, - { - "summary": f"the action to put down the {part} held by the {tool} in the {hand}", - "name": f"action: put_down({hand}, {tool}, {part})", - }, - ], - }, - ], - } - return target, ut - - -def create_insert(hand: str, tool: str, part1: str, part2: str): - """ - create insert action - """ - - target = f"is_inserted_to({part1}, {part2})" - ut = { - "summary": f"selector to insert the {part1} into the {part2} with the {tool} in the {hand}", - "name": f"selector: insert({hand}, {tool}, {part1}, {part2})", - "children": [ - { - "summary": f"the target is to make the {part1} be inserted into the {part2}", - "name": f"target: is_inserted_to({part1}, {part2})", - }, - { - "summary": f"sequence to insert the {part1} into the {part2} with the {tool} in the {hand}", - "name": f"sequence: insert({hand}, {tool}, {part1}, {part2})", - "children": [ - { - "summary": f"a precondition is the {hand} is holding {tool}", - "name": f"precondition: hold({hand}, {tool})", - }, - { - "summary": f"a precondition is the {tool} is holding {part1}", - "name": f"precondition: hold({tool}, {part1})", - }, - { - "summary": f"the action to insert the {part1} into the {part2} with the {tool} in the {hand}", - "name": f"action: insert({hand}, {tool}, {part1}, {part2})", - }, - ], - }, - ], - } - return target, ut - - -def create_screw(hand: str, tool: str, part1: str, part2: str): - """ - create screw action - """ - target = f"is_screwed_to({part1}, {part2})" - unit_tree = { - "summary": f"selector to screw the {part1} into the {part2} with the {tool} in the {hand}", - "name": f"selector: screw({hand}, {tool}, {part1}, {part2})", - "children": [ - { - "summary": f"the target is to make the {part1} be screwed into the {part2}", - "name": f"target: is_screwed_to({part1}, {part2})", - }, - { - "summary": f"sequence to screw the {part1} into the {part2} with the {tool} in the {hand}", - "name": f"sequence: screw({hand}, {tool}, {part1}, {part2})", - "children": [ - { - "summary": f"a precondition is the {hand} is holding {tool}", - "name": f"precondition: hold({hand}, {tool})", - }, - { - "summary": f"a precondition is the {tool} is holding {part1}", - "name": f"precondition: hold({tool}, {part1})", - }, - { - "summary": f"the action to screw the {part1} into the {part2} with the {tool} in the {hand}", - "name": f"action: screw({hand}, {tool}, {part1}, {part2})", - }, - ], - }, - ], - } - return target, unit_tree - - -def create_place(hand: str, tool: str, part1: str, part2: str): - """ - create place action - """ - target = f"is_placed_to({part1}, {part2})" - unit_tree = { - "summary": f"selector to place the {part1} into the {part2} with the {tool} in the {hand}", - "name": f"selector: place({hand}, {tool}, {part1}, {part2})", - "children": [ - { - "summary": f"the target is to make the {part1} be placed into the {part2}", - "name": f"target: is_placed_to({part1}, {part2})", - }, - { - "summary": f"sequence to place the {part1} into the {part2} with the {tool} in the {hand}", - "name": f"sequence: place({hand}, {tool}, {part1}, {part2})", - "children": [ - { - "summary": f"a precondition is the {hand} is holding {tool}", - "name": f"precondition: hold({hand}, {tool})", - }, - { - "summary": f"a precondition is the {tool} is holding {part1}", - "name": f"precondition: hold({tool}, {part1})", - }, - { - "summary": f"the action to place the {part1} into the {part2} with the {tool} in the {hand}", - "name": f"action: place({hand}, {tool}, {part1}, {part2})", - }, - ], - }, - ], - } - return target, unit_tree - - -gearset_ut_dict: dict[str, dict] = {} -lamp_ut_dict: dict[str, dict] = {} -chair_ut_dict: dict[str, dict] = {} -# * change tool -# for tool1 in tool_list: -# for tool2 in tool_list: -# if tool1 != tool2: -# target, unit_tree = create_change_tool("left_hand", tool1, tool2) -# gearset_ut_dict[target] = unit_tree -# target, unit_tree = create_change_tool("left_hand", "defaultgripper", "parallelgripper") -# gearset_ut_dict[target] = unit_tree -# target, unit_tree = create_change_tool("left_hand", "clampgripper", "parallelgripper") -# gearset_ut_dict[target] = unit_tree -# target, unit_tree = create_change_tool("left_hand", "outwardgripper", "parallelgripper") -# gearset_ut_dict[target] = unit_tree -target, unit_tree = create_change_tool("left_hand", "inwardgripper", "parallelgripper") -gearset_ut_dict[target] = unit_tree - -# target, unit_tree = create_change_tool("left_hand", "outwardgripper", "clampgripper") -# gearset_ut_dict[target] = unit_tree -target, unit_tree = create_change_tool("left_hand", "defaultgripper", "clampgripper") -gearset_ut_dict[target] = unit_tree -# target, unit_tree = create_change_tool("left_hand", "parallelgripper", "clampgripper") -# gearset_ut_dict[target] = unit_tree -# target, unit_tree = create_change_tool("left_hand", "inwardgripper", "clampgripper") -# gearset_ut_dict[target] = unit_tree - -# target, unit_tree = create_change_tool("left_hand", "clampgripper", "outwardgripper") -# gearset_ut_dict[target] = unit_tree -target, unit_tree = create_change_tool("left_hand", "defaultgripper", "outwardgripper") -gearset_ut_dict[target] = unit_tree -# target, unit_tree = create_change_tool("left_hand", "parallelgripper", "outwardgripper") -# gearset_ut_dict[target] = unit_tree -# target, unit_tree = create_change_tool("left_hand", "inwardgripper", "outwardgripper") -# gearset_ut_dict[target] = unit_tree - -# target, unit_tree = create_change_tool("left_hand", "clampgripper", "defaultgripper") -# gearset_ut_dict[target] = unit_tree -# target, unit_tree = create_change_tool("left_hand", "outwardgripper", "defaultgripper") -# gearset_ut_dict[target] = unit_tree -target, unit_tree = create_change_tool("left_hand", "parallelgripper", "defaultgripper") -gearset_ut_dict[target] = unit_tree -# target, unit_tree = create_change_tool("left_hand", "inwardgripper", "defaultgripper") -# gearset_ut_dict[target] = unit_tree - -# * pick up -target, unit_tree = create_pick_up("left_hand", "clampgripper", "shaft1") -gearset_ut_dict[target] = unit_tree -target, unit_tree = create_pick_up("left_hand", "defaultgripper", "shaft3") -gearset_ut_dict[target] = unit_tree -target, unit_tree = create_pick_up("left_hand", "parallelgripper", "gear1") -gearset_ut_dict[target] = unit_tree -target, unit_tree = create_pick_up("left_hand", "outwardgripper", "gear2") -gearset_ut_dict[target] = unit_tree -target, unit_tree = create_pick_up("left_hand", "outwardgripper", "gear3") -gearset_ut_dict[target] = unit_tree - -# * put down -# target, unit_tree = create_put_down("left_hand", "clampgripper", "shaft1") -# gearset_ut_dict[target] = unit_tree -target, unit_tree = create_put_down("left_hand", "parallelgripper", "gear1") -gearset_ut_dict[target] = unit_tree -# target, unit_tree = create_put_down("left_hand", "outwardgripper", "gear2") -# gearset_ut_dict[target] = unit_tree -# target, unit_tree = create_put_down("left_hand", "outwardgripper", "gear3") -# gearset_ut_dict[target] = unit_tree -# target, unit_tree = create_put_down("left_hand", "defaultgripper", "shaft3") -# gearset_ut_dict[target] = unit_tree -# * insert -target, unit_tree = create_insert( - "left_hand", "clampgripper", "shaft1", "gearbase_hole1" -) -gearset_ut_dict[target] = unit_tree -target, unit_tree = create_insert("left_hand", "parallelgripper", "gear1", "shaft1") -gearset_ut_dict[target] = unit_tree -target, unit_tree = create_insert("left_hand", "outwardgripper", "gear2", "shaft2") -gearset_ut_dict[target] = unit_tree -target, unit_tree = create_insert("left_hand", "outwardgripper", "gear3", "shaft3") -gearset_ut_dict[target] = unit_tree -target, unit_tree = create_insert( - "left_hand", "defaultgripper", "shaft3", "gearbase_hole3" -) -gearset_ut_dict[target] = unit_tree -# * screw -# ! no action -# * place -# ! no action - -#############################################################################* lamp -# * change tool -# for tool1 in tool_list: -# for tool2 in tool_list: -# if tool1 != tool2: -# target, unit_tree = create_change_tool("left_hand", tool1, tool2) -# lamp_ut_dict[target] = unit_tree -# target, unit_tree = create_change_tool("left_hand", "defaultgripper", "parallelgripper") -# lamp_ut_dict[target] = unit_tree -# target, unit_tree = create_change_tool("left_hand", "clampgripper", "parallelgripper") -# lamp_ut_dict[target] = unit_tree -target, unit_tree = create_change_tool("left_hand", "outwardgripper", "parallelgripper") -lamp_ut_dict[target] = unit_tree -# target, unit_tree = create_change_tool("left_hand", "inwardgripper", "parallelgripper") -# lamp_ut_dict[target] = unit_tree - -# target, unit_tree = create_change_tool("left_hand", "outwardgripper", "clampgripper") -# lamp_ut_dict[target] = unit_tree -target, unit_tree = create_change_tool("left_hand", "defaultgripper", "clampgripper") -lamp_ut_dict[target] = unit_tree -# target, unit_tree = create_change_tool("left_hand", "parallelgripper", "clampgripper") -# lamp_ut_dict[target] = unit_tree -# target, unit_tree = create_change_tool("left_hand", "inwardgripper", "clampgripper") -# lamp_ut_dict[target] = unit_tree - -# target, unit_tree = create_change_tool("left_hand", "clampgripper", "outwardgripper") -# lamp_ut_dict[target] = unit_tree -# target, unit_tree = create_change_tool("left_hand", "defaultgripper", "outwardgripper") -# lamp_ut_dict[target] = unit_tree -# target, unit_tree = create_change_tool("left_hand", "parallelgripper", "outwardgripper") -# lamp_ut_dict[target] = unit_tree -target, unit_tree = create_change_tool("left_hand", "inwardgripper", "outwardgripper") -lamp_ut_dict[target] = unit_tree - -# target, unit_tree = create_change_tool("left_hand", "clampgripper", "defaultgripper") -# lamp_ut_dict[target] = unit_tree -# target, unit_tree = create_change_tool("left_hand", "outwardgripper", "defaultgripper") -# lamp_ut_dict[target] = unit_tree -target, unit_tree = create_change_tool("left_hand", "parallelgripper", "defaultgripper") -lamp_ut_dict[target] = unit_tree -# target, unit_tree = create_change_tool("left_hand", "inwardgripper", "defaultgripper") -# lamp_ut_dict[target] = unit_tree - -# * pick up -target, unit_tree = create_pick_up("left_hand", "clampgripper", "lampbulb") -lamp_ut_dict[target] = unit_tree -target, unit_tree = create_pick_up("left_hand", "outwardgripper", "lampshade") -lamp_ut_dict[target] = unit_tree - - -# * put down -# target, unit_tree = create_put_down("left_hand", "clampgripper", "lampbulb") -# lamp_ut_dict[target] = unit_tree -# target, unit_tree = create_put_down("left_hand", "outwardgripper", "lampshade") -# lamp_ut_dict[target] = unit_tree -target, unit_tree = create_put_down("left_hand", "defaultgripper", "cube") -lamp_ut_dict[target] = unit_tree - - -# * actions -target, unit_tree = create_screw("left_hand", "clampgripper", "lampbulb", "lampbase") -lamp_ut_dict[target] = unit_tree -target, unit_tree = create_place("left_hand", "outwardgripper", "lampshade", "lampbulb") -lamp_ut_dict[target] = unit_tree - -#############################################################################* chair - -# * change tool -# for tool1 in tool_list: -# for tool2 in tool_list: -# if tool1 != tool2: -# target, unit_tree = create_change_tool("left_hand", tool1, tool2) -# chair_ut_dict[target] = unit_tree -# target, unit_tree = create_change_tool("left_hand", "defaultgripper", "parallelgripper") -# chair_ut_dict[target] = unit_tree -# target, unit_tree = create_change_tool("left_hand", "clampgripper", "parallelgripper") -# chair_ut_dict[target] = unit_tree -target, unit_tree = create_change_tool("left_hand", "outwardgripper", "parallelgripper") -chair_ut_dict[target] = unit_tree -# target, unit_tree = create_change_tool("left_hand", "inwardgripper", "parallelgripper") -# chair_ut_dict[target] = unit_tree - -# target, unit_tree = create_change_tool("left_hand", "outwardgripper", "clampgripper") -# chair_ut_dict[target] = unit_tree -target, unit_tree = create_change_tool("left_hand", "defaultgripper", "clampgripper") -chair_ut_dict[target] = unit_tree -# target, unit_tree = create_change_tool("left_hand", "parallelgripper", "clampgripper") -# chair_ut_dict[target] = unit_tree -# target, unit_tree = create_change_tool("left_hand", "inwardgripper", "clampgripper") -# chair_ut_dict[target] = unit_tree - -# target, unit_tree = create_change_tool("left_hand", "clampgripper", "outwardgripper") -# chair_ut_dict[target] = unit_tree -# target, unit_tree = create_change_tool("left_hand", "defaultgripper", "outwardgripper") -# chair_ut_dict[target] = unit_tree -# target, unit_tree = create_change_tool("left_hand", "parallelgripper", "outwardgripper") -# chair_ut_dict[target] = unit_tree -target, unit_tree = create_change_tool("left_hand", "inwardgripper", "outwardgripper") -chair_ut_dict[target] = unit_tree - -# target, unit_tree = create_change_tool("left_hand", "clampgripper", "defaultgripper") -# chair_ut_dict[target] = unit_tree -# target, unit_tree = create_change_tool("left_hand", "outwardgripper", "defaultgripper") -# chair_ut_dict[target] = unit_tree -target, unit_tree = create_change_tool("left_hand", "parallelgripper", "defaultgripper") -chair_ut_dict[target] = unit_tree -# target, unit_tree = create_change_tool("left_hand", "inwardgripper", "defaultgripper") -# chair_ut_dict[target] = unit_tree - -target, unit_tree = create_change_tool("left_hand", "clampgripper", "inwardgripper") -chair_ut_dict[target] = unit_tree -# target, unit_tree = create_change_tool("left_hand", "outwardgripper", "inwardgripper") -# chair_ut_dict[target] = unit_tree -# target, unit_tree = create_change_tool("left_hand", "defaultgripper", "inwardgripper") -# chair_ut_dict[target] = unit_tree -# target, unit_tree = create_change_tool("left_hand", "parallelgripper", "inwardgripper") -# chair_ut_dict[target] = unit_tree - -# * pick up -target, unit_tree = create_pick_up("left_hand", "clampgripper", "chairback") -chair_ut_dict[target] = unit_tree -target, unit_tree = create_pick_up("left_hand", "inwardgripper", "chairnut1") -chair_ut_dict[target] = unit_tree -target, unit_tree = create_pick_up("left_hand", "inwardgripper", "chairnut2") -chair_ut_dict[target] = unit_tree -target, unit_tree = create_pick_up("left_hand", "defaultgripper", "chairleg1") -chair_ut_dict[target] = unit_tree -target, unit_tree = create_pick_up("left_hand", "defaultgripper", "chairleg2") -chair_ut_dict[target] = unit_tree -target, unit_tree = create_pick_up("left_hand", "defaultgripper", "chairseat") -chair_ut_dict[target] = unit_tree - - -# * put down -# target, unit_tree = create_put_down("left_hand", "clampgripper", "chairback") -# chair_ut_dict[target] = unit_tree -# target, unit_tree = create_put_down("left_hand", "inwardgripper", "chairnut1") -# chair_ut_dict[target] = unit_tree -# target, unit_tree = create_put_down("left_hand", "inwardgripper", "chairnut2") -# chair_ut_dict[target] = unit_tree -target, unit_tree = create_put_down("left_hand", "defaultgripper", "chairleg1") -chair_ut_dict[target] = unit_tree -# target, unit_tree = create_put_down("left_hand", "defaultgripper", "chairleg2") -# chair_ut_dict[target] = unit_tree -# target, unit_tree = create_put_down("left_hand", "defaultgripper", "chairseat") -# chair_ut_dict[target] = unit_tree - - -# * actions -target, unit_tree = create_screw( - "left_hand", "inwardgripper", "chairnut1", "chairseatbolt1" -) -chair_ut_dict[target] = unit_tree -target, unit_tree = create_screw( - "left_hand", "inwardgripper", "chairnut2", "chairseatbolt2" -) -chair_ut_dict[target] = unit_tree -target, unit_tree = create_insert( - "left_hand", "clampgripper", "chairback", "chairseatconnector" -) -chair_ut_dict[target] = unit_tree -target, unit_tree = create_screw( - "left_hand", "defaultgripper", "chairleg1", "chairseatthread1" -) -chair_ut_dict[target] = unit_tree -target, unit_tree = create_screw( - "left_hand", "defaultgripper", "chairnut2", "chairseatthread2" -) diff --git a/kios_bt_planning/kios_robot/__init__.py b/kios_bt_planning/kios_robot/__init__.py deleted file mode 100644 index e69de29b..00000000 diff --git a/kios_bt_planning/kios_robot/async_robot_command.py b/kios_bt_planning/kios_robot/async_robot_command.py deleted file mode 100644 index fd661536..00000000 --- a/kios_bt_planning/kios_robot/async_robot_command.py +++ /dev/null @@ -1,232 +0,0 @@ -from typing import List, Any -import time -from tabulate import tabulate -from collections import deque -import asyncio -import logging - -from kios_utils.task import * -from kios_robot.robot_interface import RobotInterface -from kios_robot.data_types import MiosInterfaceResponse, MiosTaskResult -from kios_robot.data_types import MiosSkill, MiosCall, KiosCall -from kios_robot.data_types import TaskScene - - -class AsyncRobotCommand: - robot_address: str = None - robot_port: int = None - shared_data: Any = None - - robot_interface: RobotInterface = None - - task_scene: TaskScene = None - - tasks: deque[MiosSkill | MiosCall | KiosCall] = [] - - emergency_flag: bool - error_flag: bool - current_task: MiosSkill | MiosCall | KiosCall - - def __init__( - self, - robot_address: str, - robot_port: int, - shared_data: Any, - task_scene: TaskScene, - robot_interface: RobotInterface, # ! not used yet - ): - if robot_address is not None: - self.robot_address = robot_address - else: - raise Exception("robot_address is not set") - - if robot_port is not None: - self.robot_port = robot_port - else: - raise Exception("robot_port is not set") - - if shared_data is not None: - self.shared_data = shared_data - else: - print("warning: robot command shared_data is not set.") - - if task_scene is not None: - self.task_scene = task_scene - else: - # raise Exception("robot_scene is not set") - print("warning: robot command task_scene is not set.") - - if robot_interface is not None: - self.robot_interface = robot_interface - else: - raise Exception("robot_interface is not set") - - self.tasks = deque() - self.emergency_flag = False - self.error_flag = False - self.current_task = None - - def initialize(self): - pass - # self.task_list = [] ! dont do this. the task list should be kept. - # * currently the command response is just a boolean. - # * can be extended to a more complex data type. - - def interrupt(self): - # dirty. - payload = { - "raise_exception": False, - "recover": False, - "empty_queue": False, - } - call_method(self.robot_address, self.robot_port, "stop_task", payload=payload) - - async def add_task( - self, task: MiosSkill | MiosCall | KiosCall, is_emergent: bool = False - ): - """Add a task to the task list. If it's emergent, clear current tasks and set the emergency flag.""" - if is_emergent: - self.emergency_flag = True - # self.tasks.clear() # discard or postpone existing tasks - self.tasks.appendleft(task) - else: - self.tasks.append(task) - - # If the robot is idle or an emergency task is added, start task processing - if not self.current_task or is_emergent: - await self.process_tasks() - - async def process_tasks(self): - """Process tasks one by one, giving priority to emergent tasks.""" - while self.tasks: - if self.emergency_flag: - self.emergency_flag = ( - False # Reset the flag after accepting the emergent task - ) - self.current_task = self.tasks.popleft() - await self.execute_task(self.current_task) - - self.current_task = None - - async def execute_task(self, task) -> None: - """Simulate task execution.""" - print(f"Executing task: {str(task)}") - - try: - if isinstance(task, MiosSkill): # * use general task - mios_task = Task(self.robot_address) - mios_task.add_skill( - task.skill_name, - task.skill_type, - task.skill_parameters, - ) - start_result = mios_task.start() - - logging.info(f"Starting mios skill: {str(task)}") - mios_start_response = MiosInterfaceResponse.from_json( - start_result["result"] - ) - logging.info("\033[92mMios replied: ") - logging.info(mios_start_response) - logging.info("\033[0m") - - # * check if the task is started successfully - if mios_start_response.has_finished == False: - logging.error(f"Mios skill {str(task)} failed to start!") - raise Exception("mios skill failed to start!") - - result = mios_task.wait() - mios_response = MiosInterfaceResponse.from_json(result["result"]) - logging.info("\033[92mMios replied: ") - logging.info(mios_response) - logging.info("\033[0m") # Reset color to default - - # * if the task is not finished successfully, return a false - if mios_response.has_finished == False: - logging.error(f"Mios skill {str(task)} failed to finish!") - raise Exception("mios skill failed to finish!") - - if mios_response.task_result.has_succeeded == False: - logging.error(f"Mios skill {str(task)} failed to succeed!") - raise Exception("mios skill failed to succeed!") - - elif isinstance(task, MiosCall): # * use general call - result = call_method( - self.robot_address, - self.robot_port, - task.method_name, - task.method_payload, - ) - if result is None: - logging.error( - f"Mios call {str(task)} failed! Please check mios for more debug info." - ) - raise Exception("Mios call failed!") - - mios_response = MiosInterfaceResponse.from_json(result["result"]) - # print("Result: " + str(result)) - logging.info("\033[92mMios replied: ") - logging.info(mios_response) - logging.info("\033[0m") # Reset color to default - if mios_response.has_finished == False: - logging.error(f"Mios call {str(task)} failed!") - raise Exception("Mios call failed!") - - elif isinstance(task, KiosCall): # * use general call - result_bool = task.method(*task.args) - if result_bool != True: - logging.error(f"Kios call {str(task)} failed!") - raise Exception("Kios call failed!") - - else: - raise Exception("Unknown task type: {}".format(task)) - except Exception as e: - logging.warning(f"recovering robot from error: {str(e)}") - await self.recover_robot() - logging.warning(f"redo task: {str(task)}") - await self.execute_task(task) - - logging.info(f"Task finished: {str(task)}") - - async def recover_robot(self): - """Recover the robot to a safe state before executing an emergent task.""" - print("Recovering robot...") - self.interrupt() - # more methods to recover the robot - - async def add_tasks( - self, - new_tasks: List[MiosSkill | MiosCall | KiosCall], - is_emergent: bool = False, - ): - """Add a task to the task list. If it's emergent, clear current tasks and set the emergency flag.""" - if is_emergent: - self.emergency_flag = True - # self.tasks.clear() # discard or postpone existing tasks - self.tasks.extendleft(new_tasks) - else: - self.tasks.extend(new_tasks) - - # If the robot is idle or an emergency task is added, start task processing - if not self.current_task or is_emergent: - await self.process_tasks() - - def prepend_task(self, task: MiosSkill | MiosCall | KiosCall): - self.tasks.appendleft(task) - - def prepend_tasks(self, tasks: List[MiosSkill | MiosCall | KiosCall]): - self.tasks.extendleft(tasks) - - def clear_tasks(self): - self.tasks.clear() - - def show_tasks(self): - task_table = [] - for task in self.task_list: - task_type = type(task).__name__ # mios_skill, mios_call, kios_call - task_name = str(task) - task_table.append([task_type, task_name]) - - print(tabulate(task_table, headers=["Task Type", "Task Name"], tablefmt="grid")) - - # pass diff --git a/kios_bt_planning/kios_robot/data_types.py b/kios_bt_planning/kios_robot/data_types.py deleted file mode 100644 index 464ebd50..00000000 --- a/kios_bt_planning/kios_robot/data_types.py +++ /dev/null @@ -1,482 +0,0 @@ -from dataclasses import dataclass, field -from typing import List, Optional, Dict, Any -import numpy as np -import json - -from kios_utils.math_utils import * -from tabulate import tabulate - - -@dataclass -class MiosSkill: - skill_name: str - skill_type: str - skill_parameters: Dict[str, Any] - - def __str__(self) -> str: - return self.skill_name - - -@dataclass -class MiosCall: - method_name: str - method_payload: Dict[str, Any] - - def __str__(self) -> str: - return self.method_name - - -@dataclass -class KiosCall: - method: object - args: List[Any] - call_name: str = field(default="default name") # ! alarm: test! - - def __str__(self) -> str: - return self.call_name - - -# @dataclass - - -@dataclass -class MiosObject: - name: str - O_T_OB: np.ndarray - O_T_TCP: np.ndarray - OB_T_gp: np.ndarray - OB_T_TCP: np.ndarray - OB_I: np.ndarray - q: List[float] - grasp_width: float - grasp_force: float - mass: float - geometry: json = field(default=None) - - @staticmethod - def from_json(json: Dict[str, Any]) -> "MiosObject": - O_T_TCP = ( - np.reshape(np.array(json["O_T_TCP"]), (4, 4)).T - if "O_T_TCP" in json - else None - ) - return MiosObject( - name=json["name"], - O_T_OB=np.reshape(np.array(json["O_T_OB"]), (4, 4)).T, - O_T_TCP=O_T_TCP, - OB_T_gp=np.reshape(np.array(json["OB_T_gp"]), (4, 4)).T, - OB_T_TCP=np.reshape(np.array(json["OB_T_TCP"]), (4, 4)).T, - OB_I=np.reshape(np.array(json["OB_I"]), (3, 3)).T, - q=json["q"], - grasp_width=json["grasp_width"], - grasp_force=json["grasp_force"], - mass=json["mass"], - geometry=json["geometry"], - ) - - @staticmethod - def to_json(mios_object: "MiosObject") -> Dict[str, Any]: - return { - "name": mios_object.name, - "O_T_OB": mios_object.O_T_OB.T.flatten().tolist(), - "O_T_TCP": mios_object.O_T_TCP.T.flatten().tolist(), - "OB_T_gp": mios_object.OB_T_gp.T.flatten().tolist(), - "OB_T_TCP": mios_object.OB_T_TCP.T.flatten().tolist(), - "OB_I": mios_object.OB_I.T.flatten().tolist(), - "q": mios_object.q, - "grasp_width": mios_object.grasp_width, - "grasp_force": mios_object.grasp_force, - "mass": mios_object.mass, - "geometry": mios_object.geometry, - } - - def __str__(self) -> str: - table = [ - ["name", str(self.name)], - ["O_T_OB", str(self.O_T_OB)], - ["O_T_TCP", str(self.O_T_TCP)], - ["OB_T_gp", str(self.OB_T_gp)], - ["OB_T_TCP", str(self.OB_T_TCP)], - ["OB_I", str(self.OB_I)], - ["q", str(self.q)], - ["grasp_width", str(self.grasp_width)], - ["grasp_force", str(self.grasp_force)], - ["mass", str(self.mass)], - ["geometry", str(self.geometry)], - ] - return tabulate(table, headers=["Attribute", "Value"], tablefmt="plain") - - @staticmethod - def generate_dummy(object_name: str) -> "MiosObject": - return MiosObject( - name=object_name, - O_T_OB=np.eye(4), - O_T_TCP=np.eye(4), - OB_T_gp=np.eye(4), - OB_T_TCP=np.eye(4), - OB_I=np.eye(3), - q=[0, 0, 0, 0, 0, 0, 0], - grasp_width=0.0, - grasp_force=0.0, - mass=0.0, - geometry=None, - ) - - -@dataclass -class MiosInterfaceResponse: - has_finished: ( - bool # * whether the task is fully conducted or not (has exception or not) - ) - error_message: Optional[str] - task_result: "MiosTaskResult" = field(default=None) - - @staticmethod - def from_json(json_response: Dict[str, Any]) -> "MiosInterfaceResponse": - """ - BB: the json should be response["result"] !!! - """ - # instantiate the task result first - task_result = None - if isinstance(json_response, str): - print(f"mios response: {json_response}") - return MiosInterfaceResponse( - has_finished="???", - error_message="mios response is not a dictionary!", - task_result=None, - ) - if json_response.get("task_result") is not None: - task_result = MiosTaskResult.from_json(json_response["task_result"]) - - # instantiate the response - return MiosInterfaceResponse( - has_finished=json_response.get("result"), - error_message=json_response.get("error"), - task_result=task_result, - ) - - def __str__(self) -> str: - table = [ - ["has_finished", self.has_finished], - ["error_message", self.error_message], - ["task_result", self.task_result], - ] - return tabulate(table, headers=["Attribute", "Value"], tablefmt="plain") - - -@dataclass -class MiosTaskResult: - has_exception: bool - has_external_stop: bool - has_succeeded: bool - error_reasons: List[str] = field( - default_factory=[] - ) # ? what is this? usually user stop can invoke this - custom_results: Dict[str, Any] = field(default=None) - skill_results: Dict[str, Any] = field(default=None) - - @staticmethod - def from_json(json_task_result: Dict[str, Any]) -> "MiosTaskResult": - return MiosTaskResult( - error_reasons=json_task_result["error"], - has_exception=json_task_result["exception"], - has_external_stop=json_task_result["external_stop"], - has_succeeded=json_task_result["success"], - custom_results=json_task_result["results"], - # skill_results=json_task_result["skill_results"], - ) - - def __str__(self) -> str: - table = [ - ["has_exception", self.has_exception], - ["has_external_stop", self.has_external_stop], - ["has_succeeded", self.has_succeeded], - ["error_reasons", self.error_reasons], - ["custom_results", self.custom_results], - ["skill_results", self.skill_results], - ] - return tabulate(table, headers=["Attribute", "Value"], tablefmt="plain") - - -##################################################################################### - - -@dataclass -class KiosObject: - name: str - source: str - O_T_TCP: np.ndarray - O_T_EE: np.ndarray - joint_pose: List[float] = field(default=None) - reference_object: None = field(default=None) - - def __str__(self) -> str: - table = [ - ["name", self.name], - ["source", self.source], - ["joint_pose", self.joint_pose], - ["O_T_TCP", self.O_T_TCP.tolist()], - ["O_T_EE", self.O_T_EE.tolist()], - ["reference_object", self.reference_object], - ] - return tabulate(table, headers=["Attribute", "Value"], tablefmt="plain") - - @staticmethod - def from_mios_object(mios_object: MiosObject) -> "KiosObject": - return KiosObject( - name=mios_object.name, - source="mios", - joint_pose=mios_object.q, - O_T_TCP=mios_object.O_T_TCP, - O_T_EE=mios_object.O_T_OB, - reference_object=None, - ) - - @staticmethod - def from_json(json: Dict[str, Any]) -> "KiosObject": - return KiosObject( - name=json["object_name"], - source=json["source"], - joint_pose=json["joint_pose"] if "joint_pose" in json else None, - O_T_TCP=np.array(json["O_T_TCP"]) if "O_T_TCP" in json else None, - O_T_EE=np.array(json["O_T_EE"]) if "O_T_EE" in json else None, - reference_object=( - json["reference_object"] if "reference_object" in json else None - ), - ) - - @staticmethod - def from_relation(name: str, relation: "ReferenceRelation") -> "KiosObject": - if relation.relative_joint_pose is None: - joint_pose = None - else: - joint_pose = ( - relation.reference_object.joint_pose + relation.relative_joint_pose - ) - - O_T_TCP = relation.reference_object.O_T_TCP.dot(relation.relative_HT) - - return KiosObject( - name=name, - source="relation", - joint_pose=joint_pose, - O_T_TCP=O_T_TCP, - reference_object=relation.reference_object, - ) - - -# ! BBWORK suspend here. need to figure out if using MiosObject is a good idea. -@dataclass -class ReferenceRelation: - name: str # ! this is not unique! - reference_object: KiosObject - relative_HT: np.ndarray # from reference object to this object - relative_joint_pose: List[float] = field( - default=None - ) # from reference object to this object - - def __init__( - self, - reference_object: KiosObject, - relative_joint_pose: List[float] = None, - relative_cartesian_pose: np.ndarray = None, - relative_HT: np.ndarray = None, - ): - if reference_object.isinstance(KiosObject): - self.reference_object = reference_object - else: - raise Exception("reference_object is not a MiosObject!") - - self.relative_joint_pose = relative_joint_pose - - if relative_HT is not None: - self.relative_HT = relative_HT - elif relative_cartesian_pose is not None: - self.relative_HT = HT_from_xyzrpy(relative_cartesian_pose) - else: - raise Exception( - "relative_HT and relative_cartesian_pose are both None! At least one of them should be set!" - ) - - @staticmethod # * not sure necessary - def from_json(json: Dict[str, Any]) -> "ReferenceRelation": - return ReferenceRelation( - reference_object=json[ - "reference_object" - ], # ! BUG, should be an object instead of a string. - relative_joint_pose=json["relative_joint_pose"], - relative_HT=json["relative_HT"], - relative_cartesian_pose=json["relative_cartesian_pose"], - ) - - -@dataclass -class Toolbox: - name: str - # * for future you should consider using these parameters to invoke the gripper-related skills - EE_finger_width_max: float = field(default=0.08) - EE_finger_width_min: float = field(default=0.0) - load_width: float = field( - default=0.042 - ) # the width the hand to reach in order to load this tool - unload_width: float = field( - default=0.08 - ) # the width the hand to reach in order to unload this tool - grasp_force: float = field(default=70.0) # the force the hand to grasp this tool - grasp_speed: float = field(default=0.1) # the speed the hand to grasp this tool - grasp_eps_in: float = field(default=0.005) # not used yet - grasp_eps_out: float = field(default=0.005) - # kinematics parameters - EE_T_TCP: np.ndarray = field(default=np.eye(4)) - - def __str__(self): - table = [ - ["name", self.name], - ["EE_finger_width_max", self.EE_finger_width_max], - ["EE_finger_width_min", self.EE_finger_width_min], - ["load_width", self.load_width], - ["unload_width", self.unload_width], - ["grasp_force", self.grasp_force], - ["grasp_speed", self.grasp_speed], - ["grasp_eps_in", self.grasp_eps_in], - ["grasp_eps_out", self.grasp_eps_out], - ["EE_T_TCP", self.EE_T_TCP.tolist()], - ] - return tabulate(table, headers=["Attribute", "Value"], tablefmt="plain") - - @staticmethod - def from_json(json: Dict[str, Any]) -> "Toolbox": - return Toolbox( - name=json["name"], - EE_finger_width_max=json["EE_finger_width_max"], - EE_finger_width_min=json["EE_finger_width_min"], - load_width=json["load_width"], - unload_width=json["unload_width"], - grasp_force=json["grasp_force"], - grasp_speed=json["grasp_speed"], - grasp_eps_in=json["grasp_eps_in"], - grasp_eps_out=json["grasp_eps_out"], - EE_T_TCP=np.reshape(np.array(json["EE_T_TCP"]), (4, 4)).T, - ) - - @staticmethod - def to_json(toolbox: "Toolbox") -> Dict[str, Any]: - return { - "name": toolbox.name, - "EE_finger_width_max": toolbox.EE_finger_width_max, - "EE_finger_width_min": toolbox.EE_finger_width_min, - "load_width": toolbox.load_width, - "unload_width": toolbox.unload_width, - "grasp_force": toolbox.grasp_force, - "grasp_speed": toolbox.grasp_speed, - "grasp_eps_in": toolbox.grasp_eps_in, - "grasp_eps_out": toolbox.grasp_eps_out, - "EE_T_TCP": toolbox.EE_T_TCP.T.flatten().tolist(), - } - - -@dataclass -class TaskScene: - tool_map: Dict[str, Toolbox] = field(default_factory=dict) - object_map: Dict[str, KiosObject] = field(default_factory=dict) - reference_map: Dict[str, ReferenceRelation] = field(default_factory=dict) - - def __str__(self) -> str: - table = [ - ["tools", self.tool_map], - ["objects", self.object_map], - ["references", self.reference_map], - ] - return tabulate(table, headers=["Attribute", "Value"], tablefmt="plain") - - def get_object(self, object_name: str) -> Optional[KiosObject]: - return self.object_map.get(object_name, None) - - def get_tool(self, tool_name: str = None) -> Toolbox: - if tool_name is None: - return self.tool_map.get("no_tool") - tool = self.tool_map.get(tool_name) - if tool is None: - from pprint import pprint - - pprint("current available tools are:") - pprint(self.tool_map.keys()) - raise Exception(f"Tool {tool_name} is not in the scene!") - return tool - - -@dataclass -class RobotState: - joint_pose: List[float] - O_T_EE: np.ndarray - EE_T_TCP: np.ndarray - O_T_TCP: np.ndarray - status: str - current_task: str - gripper_width: float - TF_T_EE_d: np.ndarray = field(default=None) - q_d: List[float] = field(default=None) - - def __str__(self) -> str: - table = [ - ["joint_pose", self.joint_pose], - [ - "O_T_EE", - "\n".join(["\t".join(map(str, row)) for row in self.O_T_EE.tolist()]), - ], - [ - "EE_T_TCP", - "\n".join(["\t".join(map(str, row)) for row in self.EE_T_TCP.tolist()]), - ], - [ - "O_T_TCP", - "\n".join(["\t".join(map(str, row)) for row in self.O_T_TCP.tolist()]), - ], - ["status", self.status], - ["current_task", self.current_task], - ["gripper_width", self.gripper_width], - # ? what is this? - # ["q_d", self.q_d], - # ["TF_T_EE_d", "\n".join(["\t".join(map(str, row)) for row in self.TF_T_EE_d.tolist()])], - ] - return tabulate(table, headers=["Attribute", "Value"], tablefmt="plain") - - @staticmethod - def from_json(json: Dict[str, Any]) -> "RobotState": - return RobotState( - joint_pose=json["q"], - O_T_EE=np.reshape(np.array(json["O_T_EE"]), (4, 4)).T, - EE_T_TCP=np.reshape(np.array(json["EE_T_TCP"]), (4, 4)).T, - O_T_TCP=np.reshape(np.array(json["O_T_TCP"]), (4, 4)).T, - status=json["status"], - current_task=json["current_task"], - gripper_width=json["gripper_width"], - # q_d=json["q_d"], - # TF_T_EE_d=np.reshape(np.array(json["TF_T_EE_d"]), (4, 4)).T, - ) - - @staticmethod - def to_json(robot_state: "RobotState") -> Dict[str, Any]: - return { - "q": robot_state.joint_pose, - "O_T_EE": robot_state.O_T_EE.T.flatten().tolist(), - "EE_T_TCP": robot_state.EE_T_TCP.T.flatten().tolist(), - "O_T_TCP": robot_state.O_T_TCP.T.flatten().tolist(), - "status": robot_state.status, - "current_task": robot_state.current_task, - "gripper_width": robot_state.gripper_width, - # "q_d": robot_state.q_d, - # "TF_T_EE_d": robot_state.TF_T_EE_d.T.flatten().tolist(), - } - - -@dataclass -class ParsedAction: - action_name: str - args: List[str] - - @staticmethod - def from_string(action_string: str) -> "ParsedAction": - # ! haven't tested yet - action_name, *args = action_string.split() - return ParsedAction(action_name, args) diff --git a/kios_bt_planning/kios_robot/kios_vision/__init__.py b/kios_bt_planning/kios_robot/kios_vision/__init__.py deleted file mode 100644 index e69de29b..00000000 diff --git a/kios_bt_planning/kios_robot/mios_async.py b/kios_bt_planning/kios_robot/mios_async.py deleted file mode 100644 index e961c85d..00000000 --- a/kios_bt_planning/kios_robot/mios_async.py +++ /dev/null @@ -1,143 +0,0 @@ -#!/usr/bin/env python - -############################################################################## -# Imports -############################################################################## - -from kios_utils.task import * - -import atexit -import multiprocessing -import multiprocessing.connection -import time - -from kios_robot.robot_command import RobotCommand - - -def mios_monitor( - task: Task, pipe_connection: multiprocessing.connection.Connection -) -> None: - """Emulate a (potentially) long running external process. - - Args: - pipe_connection: connection to the mios_monitor process - """ - try: - task.interrupt() - task.start() - # hanlde startup failure - # ! check the response here - print(str(task.task_start_response)) - - task.shared_data["task_start_response"] = task.task_start_response - - if bool(task.task_start_response["result"]["result"]) == False: - pipe_connection.send([False]) - return - - _ = task.wait() - - if bool(task.task_wait_response["result"]["result"]) == True: - pipe_connection.send([True]) - else: - pipe_connection.send([False]) - except KeyboardInterrupt: - pass - - -def fake_monitor( - task: Task, pipe_connection: multiprocessing.connection.Connection -) -> None: - """Emulate a (potentially) long running external process. - - Args: - pipe_connection: connection to the mios_monitor process - """ - try: - # * skip - # task.interrupt() - # task.start() - # hanlde startup failure - # ! check the response here - # print(str(task.task_start_response)) - - time.sleep(2) - # * fake a start response - # print("start fake monitor") - task.shared_data["task_start_response"] = {"result": {"result": True}} - - # * skip the start - # if bool(task.task_start_response["result"]["result"]) == False: - # pipe_connection.send([False]) - # return - # * skip the wait - # _ = task.wait() - - # * sleep for 5 seconds - time.sleep(5) - - # * skip the wait response - # if bool(task.task_wait_response["result"]["result"]) == True: - # pipe_connection.send([True]) - # else: - # pipe_connection.send([False]) - - # * send fake response - print("send fake response") - pipe_connection.send([True]) - - except KeyboardInterrupt: - pass - - -def robot_command_monitor( - robot_command: RobotCommand, pipe_connection: multiprocessing.connection.Connection -) -> None: - """Emulate a (potentially) long running external process. - - Args: - pipe_connection: connection to the mios_monitor process - """ - try: - robot_command.interrupt() - response = robot_command.execute_task_list_sync() - - if response == False: - pipe_connection.send([False]) - return - - if response == True: - pipe_connection.send([True]) - return - - except KeyboardInterrupt: - pass - - -def fake_robot_command_monitor( - robot_command: RobotCommand, pipe_connection: multiprocessing.connection.Connection -) -> None: - """Emulate a (potentially) long running external process. - - Args: - pipe_connection: connection to the mios_monitor process - """ - try: - time.sleep(2) - # * fake a start response - - # * skip the start - - # * skip the wait - - # * sleep for 5 seconds - time.sleep(5) - - # * skip the wait response - - # * send fake response - print("send fake response") - pipe_connection.send([True]) - - except KeyboardInterrupt: - pass diff --git a/kios_bt_planning/kios_robot/mios_response_examples.txt b/kios_bt_planning/kios_robot/mios_response_examples.txt deleted file mode 100644 index 0e2cb545..00000000 --- a/kios_bt_planning/kios_robot/mios_response_examples.txt +++ /dev/null @@ -1,66 +0,0 @@ -context = { - "result": { - "error": "Could not move gripper.", - "result": False, - } -} - - -context = { - "result": { - "error": "", - "result": True, - "task_result": { - "error": [], - "exception": False, - "external_stop": False, - "results": None, - # "skill_results": { - # "move": { - # "cost": { - # "contact_forces": 4.031948767079044, - # "custom": 0.0, - # "desired_force": 6.572478413516299, - # "desired_pose": 0.00033891758423726154, - # "distance": 7.348467835327242e-06, - # "effort_avg": 38.444886800768884, - # "effort_total": 34.108555016576695, - # "time": 4.45, - # }, - # "heuristic": 0.010562131317092108, - # } - # }, - "success": True, - }, - } -} - - -failure_response = { - "result": { - "error": "", - "result": True, - "task_result": { - "error": ["UserStopped"], - "exception": False, - "external_stop": False, - "results": None, - # "skill_results": { - # "move": { - # "cost": { - # "contact_forces": 3.6188234620319477, - # "custom": 0.0, - # "desired_force": 6.858632516376034, - # "desired_pose": 0.0005703592006608017, - # "distance": 0.0006923226797301379, - # "effort_avg": 39.31453474415038, - # "effort_total": 33.02963405771327, - # "time": 1.889, - # }, - # "heuristic": 0.41518457485917976, - # } - # }, - "success": False, - }, - } -} diff --git a/kios_bt_planning/kios_robot/mios_task_factory.py b/kios_bt_planning/kios_robot/mios_task_factory.py deleted file mode 100644 index bd211764..00000000 --- a/kios_bt_planning/kios_robot/mios_task_factory.py +++ /dev/null @@ -1,1470 +0,0 @@ -from typing import List, Any, Dict -import re -import numpy as np -from pprint import pprint - -from kios_scene.mongodb_interface import MongoDBInterface - -# from kios_robot.robot_interface import RobotInterface -# ! circular import - -# from kios_robot.robot_proprioceptor import RobotProprioceptor -from kios_robot.data_types import ( - MiosCall, - MiosSkill, - KiosCall, - Toolbox, - TaskScene, - MiosObject, -) - -from kios_bt.data_types import Action - -""" -BB knows this is not a good design to create generate method for each task/call. -A better implementation can be a common interface for generating mios tasks, -which takes formatted node data as input. -Anyway, the current implementation is straightforward and easy to understand. -""" - - -class MiosTaskFactory: - task_scene: TaskScene - robot_interface: Any - # robot_proprioceptor: RobotProprioceptor - - def __init__( - self, - task_scene: TaskScene = None, - robot_interface: Any = None, - ): - self.task_scene = task_scene - self.robot_interface = robot_interface - - def initialize(self): - pass - - def setup_scene(self, task_scene: TaskScene): - self.task_scene = task_scene - - def parse_action(self, action: Action) -> Dict[str, Any]: - """parse the action string to get the action name and arguments. - - Args: - action (Action): _description_ - - Raises: - Exception: _description_ - - Returns: - Dict[str, Any]: action_name: str, args: List[str] - """ - # use re to parse the action - pattern = r"(?:action:\s)?(\w+):?\s*\(([\w\s,]+)\)" # ! alarm - action_string = action.name - match = re.match(pattern, action_string) - if match: - action_name = match.group(1) # Extract the action name from the match - args = [ - arg.strip() for arg in match.group(2).split(",") - ] # Extract and clean up the arguments - return { - "name": action_name, - "args": args, - } # Return a dictionary with the action name and arguments - else: - raise Exception( - f'Action "{action_string}" is not in the correct format.' - ) # Raise an exception if the action format is incorrect - # return {"action_name": None, "args": []} - - # * BBCORE - def generate_tasks( - self, action: Action, shared_data: Any = None - ) -> List[MiosCall | MiosSkill]: - """core function. - generate mios tasks from a kios action and return them in a list. - - Args: - action (Action): _description_ - - Returns: - List[MiosCall or MiosSkill]: _description_ - """ - parsed_action = self.parse_action(action) - - if parsed_action["name"] == "load_tool": - return self.generate_load_tool_skill(parsed_action) - elif parsed_action["name"] == "unload_tool": - return self.generate_unload_tool_skill(parsed_action) - elif parsed_action["name"] == "change_tool": - return self.generate_change_tool_skill(parsed_action) - elif parsed_action["name"] == "pick_up": - return self.generate_pick_up_skill(parsed_action) - elif parsed_action["name"] == "screw": # ! mod later - return self.generate_screw_skill(parsed_action) - elif parsed_action["name"] == "put_down": - return self.generate_put_down_skill(parsed_action) - elif parsed_action["name"] == "place": - return self.generate_place_skill(parsed_action) - elif parsed_action["name"] == "insert": - return self.generate_insert_skill(parsed_action) - else: - raise Exception(f"action {parsed_action['name']} is not supported yet.") - - def generate_fake_mios_tasks(self, action: Action) -> List[MiosCall | MiosSkill]: - """ - test function. - """ - print("!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!") - print("generate_fake_mios_tasks, skipped for test.") - # print(action) - print(self.parse_action(action)) - return [ - MiosCall( - method_name="test_method", - method_payload={"test": "test"}, - ) - ] - - ################################################################### - # * kios call methods - def generate_kios_dummy_call(self) -> KiosCall: - return KiosCall( - method=print, - args=["this is a dummy call from kios!"], - ) - - def generate_update_object_from_mios_call(self, object_name: str) -> KiosCall: - """update the object in kios from mios. - - Args: - object_name (str): _description_ - - Returns: - KiosCall: _description_ - """ - return KiosCall( - method=self.robot_interface.proprioceptor.update_scene_object_from_mios, - args=[self.task_scene, object_name], - ) - - ################################################################### - # * methods to generate one mios task - def generate_teach_O_T_TCP_call(self, object_name: str) -> MiosCall: - payload = { - "object_name": object_name, - } - return MiosCall(method_name="teach_O_T_TCP", method_payload=payload) - - def generate_teach_object_call(self, object_name: str) -> MiosCall: - payload = { - "object": object_name, - } - return MiosCall(method_name="teach_object", method_payload=payload) - - # ! this doesn't make sense if you use the object in the scene. turn to the scene method. - def generate_update_mios_memory_environment_call(self) -> MiosCall: - payload = {} - return MiosCall(method_name="update_memory_environment", method_payload=payload) - - def generate_move_above_mp(self, object_name: str) -> MiosSkill: - """ - move to 15cm above the object in the z axis of the object coordinate system. - """ - # default: 15cm above the object - # get the object from the scene - kios_object = self.task_scene.get_object(object_name) - if kios_object is None: - raise Exception(f'object "{object_name}" is not found in the scene!') - # get the O_T_EE and modify it to be 15cm above the object - object_O_T_TCP = kios_object.O_T_TCP - above_O_T_TCP = object_O_T_TCP @ np.array( - [ - [1, 0, 0, 0], - [0, 1, 0, 0], - [0, 0, 1, -0.15], - [0, 0, 0, 1], - ] - ) - - # generate the cartesian move - return self.generate_cartesian_move_mp(O_T_TCP=above_O_T_TCP) - - def generate_reach_mp(self, object_name: str) -> MiosSkill: - """ - move to the object. - """ - # get the object from the scene - kios_object = self.task_scene.get_object(object_name) - if kios_object is None: - raise Exception(f'object "{object_name}" is not found in the scene!') - # get the O_T_EE and modify it to be 5cm above the object - object_O_T_TCP = kios_object.O_T_TCP - - # generate the cartesian move - return self.generate_cartesian_move_mp(O_T_TCP=object_O_T_TCP) - - def generate_cartesian_move_mp( - self, object_name: str = None, O_T_TCP: np.ndarray = None, dx_d=None, ddx_d=None - ) -> MiosSkill: - """make the TCP of the robot move to the pose defined by the mios object or the O_T_TCP. - IMPORTANT: Here the robot is asked to move to the TCP pose. - the O_T_EE for cartesian motion will be calculated by mios. - - Args: - object_name (str, optional): _description_. Defaults to None. - O_T_TCP (np.ndarray, optional): _description_. Defaults to None. - - Raises: - Exception: _description_ - - Returns: - MiosSkill: _description_ - """ - if object_name is None and O_T_TCP is None: - raise Exception("cartesian target is not set!") - if object_name: - context = { - "skill": { - "p0": { - "dX_d": dx_d if dx_d is not None else [0.1, 1], - "ddX_d": ddx_d if ddx_d is not None else [0.5, 1], - "K_x": [1500, 1500, 1000, 150, 150, 150], - }, - "objects": {"GoalPose": object_name}, - }, - "control": {"control_mode": 0}, - } - else: - context = { - "skill": { - "p0": { - "dX_d": dx_d if dx_d is not None else [0.5, 1], - "ddX_d": ddx_d if ddx_d is not None else [0.3, 1], - "K_x": [1500, 1500, 1500, 150, 150, 150], - "T_T_EE_g": O_T_TCP.T.flatten().tolist(), # ! TTEE IS ACTUALLY O_T_TCP! - }, - # "objects": {"GoalPose": "NullObject"}, - }, - "control": {"control_mode": 0}, - } - - return MiosSkill( - skill_name="cartesian_move", - skill_type="KiosCartesianMove", - skill_parameters=context, - ) - - def generate_joint_move_mp(self, joint_location: str) -> MiosSkill: - """ - * available. debug: task context error, "skill", "control" and "user" are on the same level. - move to a joint position. - Args: - joint_location (str): the stored Taskframe_HomogeneousT_EE_goal in mongo DB - """ - context = { - "skill": { - "speed": 1, - "acc": 0.8, - "q_g": [0, 0, 0, 0, 0, 0, 0], - "q_g_offset": [0, 0, 0, 0, 0, 0, 0], - "objects": {"goal_pose": joint_location}, - }, - "control": {"control_mode": 3}, - "user": {"env_X": [0.005, 0.005, 0.005, 0.0175, 0.0175, 0.0175]}, - } - - return MiosSkill( - skill_name="joint_move", - skill_type="MoveToPoseJoint", - skill_parameters=context, - ) - - def generate_gripper_grasp_mp( - self, width=0.01, speed=0.2, force=50, epsilon_inner=0.05, epsilon_outer=0.05 - ) -> MiosCall: - payload = { - "width": width, - "speed": speed, - "force": force, - "epsilon_inner": epsilon_inner, - "epsilon_outer": epsilon_outer, - } - # return call_method(self.robot_address, self.robot_port, "grasp", payload) - return MiosCall(method_name="grasp", method_payload=payload) - - def generate_gripper_release_mp(self, width=0.08, speed=0.2) -> MiosCall: - payload = { - "width": width, - "speed": speed, - } - # return call_method(self.robot_address, self.robot_port, "release", payload) - return self.generate_gripper_move_mp(width=width, speed=speed) - - def generate_gripper_move_mp( - self, - width: float, - speed: float = 0.05, - ) -> MiosCall: - payload = { - "width": width, - "speed": speed, - } - pprint(payload) - return MiosCall(method_name="move_gripper", method_payload=payload) - - def generate_gripper_home_mp(self) -> MiosCall: - payload = {} - return MiosCall(method_name="home_gripper", method_payload=payload) - - # preserved for future use - def generate_equip_tool(self) -> MiosCall: - # move finger to the right position - return self.generate_gripper_move_mp(0.042) - - # preserved for future use - def generate_lift_tool(self) -> MiosCall: - # move finger to the right position - return self.generate_gripper_move_mp(0.08) - - def generate_screw_in_mp(self, object_name: str = None, O_T_OB=None) -> MiosSkill: - """ - # ! DO NOT USE THIS METHOD - """ - if object_name is None and O_T_OB is None: - raise Exception("Object target is not set!") - if object_name: - payload = { - "skill": { - "p0": { - "dX_d": [0.1, 0.5], - "ddX_d": [0.5, 1], - # "O_T_OB": - "K_x": [1500, 1500, 1500, 150, 150, 150], - "F_ff": [0, 0, 0, 0, 0, 2], - }, - "objects": {"Container": object_name}, - }, - "control": {"control_mode": 0}, - } - else: - payload = { - "skill": { - "p0": { - "dX_d": [0.1, 0.3], - "ddX_d": [0.5, 0.5], - "K_x": [ - 1000, - 1000, - 1000, - 500, - 500, - 100, - ], # ! is EE_k_x if you don't set frame!!! - "O_T_OB": O_T_OB.T.flatten().tolist(), - "F_ff": [0, 0, 3, 0, 0, -10], - }, - # "objects": {"GoalPose": "NullObject"}, - }, - "control": {"control_mode": 0}, - } - - return MiosSkill( - skill_name="screw_in", - skill_type="KiosScrewIn", - skill_parameters=payload, - ) - - def generate_insert_mp_mod( - self, insertable: str, container: str, param: Dict[str, Any] = None - ) -> MiosCall: - if container is None: - raise Exception("container is not set!") - - # get the container from the scene - kios_object = self.task_scene.get_object(container) - - if param is None: - param = { - "search_a": [2, 2, 1, 1, 1, 0], - "search_f": [1, 1, 1, 1.5, 1.5, 0], - "F_ext_contact": [13.0, 2.0], - "f_push": [0, 0, 5, 0, 0, 0], - "K_x": [300, 300, 500, 500, 500, 800], - "D_x": [0.7, 0.7, 0.7, 0.7, 0.7, 0.7], - } - - search_a = ( - param["search_a"] if "search_a" in param.keys() else [2, 2, 1, 1, 1, 0] - ) - search_f = ( - param["search_f"] if "search_f" in param.keys() else [1, 1, 1, 1.5, 1.5, 0] - ) - F_ext_contact = ( - param["F_ext_contact"] if "F_ext_contact" in param.keys() else [13.0, 2.0] - ) - f_push = param["f_push"] if "f_push" in param.keys() else [0, 0, 5, 0, 0, 0] - K_x = param["K_x"] if "K_x" in param.keys() else [300, 300, 500, 500, 500, 800] - D_x = param["D_x"] if "D_x" in param.keys() else [0.7, 0.7, 0.7, 0.7, 0.7, 0.7] - - if kios_object is not None: - print(f'object "{container}" is found in the scene!') - O_T_TCP = kios_object.O_T_TCP - payload = { - "skill": { - "objects": { - # "Container": container, - }, - # "time_max": 35, - "p0": { - "O_T_TCP": O_T_TCP.T.flatten().tolist(), - "dX_d": [0.05, 0.3], - "ddX_d": [0.5, 1], - "DeltaX": [0, 0, 0, 0, 0, 0], - "K_x": [1500, 1500, 1500, 600, 600, 600], - }, - "p1": { - "dX_d": [0.05, 0.1], - "ddX_d": [0.1, 0.05], - "K_x": [1500, 1500, 500, 800, 800, 800], - }, - "p2": { - # "search_a": [10, 10, 0, 2, 2, 0], - # "search_f": [1, 1, 0, 1.2, 1.2, 0], - "search_a": search_a, - "search_f": search_f, - "search_phi": ( - param["search_phi"] - if "search_phi" in param.keys() - else [ - 0, - 3.14159265358979323846 / 2, - 0, - 3.14159265358979323846 / 2, - 0, - 0, - ] - ), - "K_x": K_x, - "D_x": D_x, - "f_push": f_push, - # "dX_d": [0.1, 0.5], - # "ddX_d": [0.5, 1], - "dX_d": [0.03, 0.3], - "ddX_d": [0.3, 1], - }, - "p3": { - "dX_d": [0.1, 0.5], - "ddX_d": [0.5, 1], - "f_push": 7, - "K_x": [500, 500, 0, 800, 800, 800], - }, - }, - "control": {"control_mode": 0}, - "user": { - "env_X": ( - param["env_X"] - if "env_X" in param.keys() - else [0.01, 0.01, 0.002, 0.05, 0.05, 0.05] - ), - "env_dX": [0.001, 0.001, 0.001, 0.005, 0.005, 0.005], - "F_ext_contact": F_ext_contact, - "tau_ext_max": [100, 100, 100, 100, 40, 40, 40], - }, - } - else: - raise Exception(f'object "{container}" is not found in the scene!') - # else: - # payload = { - # "skill": { - # "objects": { - # "Container": container, - # }, - # "time_max": 20, - # "p0": { # ! approach should be move above in mios skill. mod it - # "dX_d": [0.1, 0.3], - # "ddX_d": [0.5, 4], - # "DeltaX": [0, 0, 0, 0, 0, 0], - # "K_x": [1500, 1500, 1500, 600, 600, 600], - # }, - # "p1": { - # "dX_d": [0.03, 0.1], - # "ddX_d": [0.05, 0.01], - # "K_x": [500, 500, 500, 600, 600, 600], - # }, - # "p2": { - # # "search_a": [10, 10, 0, 2, 2, 0], - # # "search_f": [1, 1, 0, 1.2, 1.2, 0], - # "search_a": [5, 5, 0, 2, 2, 0], - # "search_f": [1, 1, 0, 1.2, 1.2, 0], - # "search_phi": [ - # 0, - # 3.14159265358979323846 / 2, - # 0, - # 3.14159265358979323846 / 2, - # 0, - # 0, - # ], - # "K_x": [500, 500, 500, 800, 800, 800], - # "f_push": [0, 0, 3, 0, 0, 0], - # # "dX_d": [0.1, 0.5], - # # "ddX_d": [0.5, 1], - # "dX_d": [0.08, 0.5], - # "ddX_d": [0.3, 1], - # }, - # "p3": { - # "dX_d": [0.1, 0.5], - # "ddX_d": [0.5, 1], - # "f_push": 7, - # "K_x": [500, 500, 0, 800, 800, 800], - # }, - # }, - # "control": {"control_mode": 0}, - # "user": { - # "env_X": [0.01, 0.01, 0.002, 0.05, 0.05, 0.05], - # "env_dX": [0.001, 0.001, 0.001, 0.005, 0.005, 0.005], - # "F_ext_contact": [10.0, 2.0], # ! this doesn't work. change the param in the mongodb! - # }, - # } - insert = MiosSkill( - skill_name="insert", - skill_type="KiosInsertion", - skill_parameters=payload, - ) - - return insert - - def generate_update_tool_call(self, tool_name: str = None) -> MiosCall: - """let mios know the tool is loaded and it need to change EE_T_TCP. - - Args: - tool_name (str, optional): _description_. Defaults to None. - - Returns: - MiosCall: _description_ - """ - # get the tool from the scene - if tool_name is None: - tool = self.task_scene.get_tool("no_tool") # ! BBCHANGE - else: - tool = self.task_scene.get_tool(tool_name) - # get the EE_T_TCP - EE_T_TCP = tool.EE_T_TCP - EE_finger_width_max = tool.EE_finger_width_max - EE_finger_width_min = tool.EE_finger_width_min - assert isinstance(EE_T_TCP, np.ndarray) - payload = { - "EE_T_TCP": EE_T_TCP.T.flatten().tolist(), - "EE_finger_width_max": EE_finger_width_max, - "EE_finger_width_min": EE_finger_width_min, - } - pprint(payload) - return MiosCall(method_name="change_tool", method_payload=payload) - - ################################################################### - - ################################################################### - # * methods to generate a sequence of mios tasks - - def generate_change_tool_skill( - self, parsed_action: Dict[str, Any] - ) -> list[MiosCall | MiosSkill]: - tool1 = parsed_action["args"][1] - tool2 = parsed_action["args"][2] - if tool1 is None or tool2 is None: - raise Exception("tool_name is not set!") - - action1 = {"args": ["xxxx", tool1]} - unload_skill = self.generate_unload_tool_skill(action1) - action2 = {"args": ["xxxx", tool2]} - load_skill = self.generate_load_tool_skill(action2) - return unload_skill + load_skill - - def generate_load_tool_skill( - self, parsed_action: Dict[str, Any] - ) -> List[MiosSkill | MiosCall]: - tool_name = parsed_action["args"][1] - if tool_name is None: - raise Exception("tool_name is not set!") - - payload = { - "skill": { - "objects": { - "Tool": tool_name, - }, - "time_max": 30, - "MoveAbove": { - "dX_d": [0.5, 1.2], - "ddX_d": [0.3, 1], - "DeltaX": [0, 0, 0, 0, 0, 0], - "K_x": [1500, 1500, 1500, 600, 600, 600], - }, - "MoveIn": { - "dX_d": [0.2, 0.4], - "ddX_d": [0.1, 0.1], - "DeltaX": [0, 0, 0, 0, 0, 0], - "K_x": [1500, 1500, 1500, 600, 600, 600], - }, - "GripperMove": { - "width": 0.04, - "speed": 0.1, - "force": 70, - "K_x": [1500, 1500, 1500, 100, 100, 100], - "eps_in": 0.05, - "eps_out": 0.05, - }, - "Retreat": { - "dX_d": [0.4, 0.4], - "ddX_d": [0.1, 0.1], - "DeltaX": [0, 0, 0, 0, 0, 0], - "K_x": [1500, 1500, 1500, 600, 600, 600], - }, - }, - "control": {"control_mode": 0}, - "user": { - "env_X": [0.01, 0.01, 0.002, 0.05, 0.05, 0.05], - "env_dX": [0.001, 0.001, 0.001, 0.005, 0.005, 0.005], - }, - } - - load_tool = MiosSkill( - skill_name="load_tool", - skill_type="KiosLoadTool", - skill_parameters=payload, - ) - - update_tool = self.generate_update_tool_call(tool_name) - - return [ - load_tool, - update_tool, - ] - - def generate_unload_tool_skill( - self, parsed_action: Dict[str, Any] - ) -> List[MiosSkill | MiosCall]: - tool_name = parsed_action["args"][1] - if tool_name is None: - raise Exception("tool_name is not set!") - - payload = { - "skill": { - "objects": { - "Tool": tool_name, - }, - "time_max": 30, - "MoveAbove": { - "dX_d": [0.5, 1.2], - "ddX_d": [0.3, 1], - "DeltaX": [0, 0, 0, 0, 0, 0], - "K_x": [1500, 1500, 1500, 600, 600, 600], - }, - "MoveIn": { - "dX_d": [0.4, 0.4], - "ddX_d": [0.1, 0.1], - "DeltaX": [0, 0, 0, 0, 0, 0], - "K_x": [1500, 1500, 1500, 600, 600, 600], - }, - "GripperMove": { - "width": 0.08, - "speed": 0.1, - "force": 70, - "K_x": [1500, 1500, 1500, 100, 100, 100], - "eps_in": 0.05, - "eps_out": 0.05, - }, - "Retreat": { - "dX_d": [0.4, 0.4], - "ddX_d": [0.1, 0.1], - "DeltaX": [0, 0, 0, 0, 0, 0], - "K_x": [1500, 1500, 1500, 600, 600, 600], - }, - }, - "control": {"control_mode": 0}, - "user": { - "env_X": [0.01, 0.01, 0.002, 0.05, 0.05, 0.05], - "env_dX": [0.001, 0.001, 0.001, 0.005, 0.005, 0.005], - }, - } - - unload_tool = MiosSkill( - skill_name="unload_tool", - skill_type="KiosLoadTool", - skill_parameters=payload, - ) - - # ! name changed - update_tool = self.generate_update_tool_call(tool_name="defaultgripper") - - return [ - unload_tool, - update_tool, - ] - - # ! BUG - def generate_pick_up_skill( - self, parsed_action: Dict[str, Any] - ) -> List[MiosCall | MiosSkill]: - # tool_name = parsed_action["args"][1] - # if tool_name is None: - # raise Exception("tool_name is not set!") - object_name = parsed_action["args"][2] - if object_name is None: - raise Exception("object_name is not set!") - - task_list = [] - # move above - move_above = self.generate_move_above_mp(object_name) - task_list.append(move_above) - # move in - # ! BUG - # reach = self.generate_cartesian_move_mp(object_name=object_name) - reach = self.generate_reach_mp(object_name) - task_list.append(reach) - # grasp - grasp = self.generate_gripper_grasp_mp() - task_list.append(grasp) - # retreat - retreat = self.generate_move_above_mp(object_name) - task_list.append(retreat) - # update object in mios - update_object_in_mios = self.generate_teach_object_call(object_name) - task_list.append(update_object_in_mios) - # update object in kios - update_object_in_kios = self.generate_update_object_from_mios_call(object_name) - task_list.append(update_object_in_kios) - - return task_list - - # ! BUG - def generate_put_down_skill( - self, parsed_action: Dict[str, Any] - ) -> List[MiosCall | MiosSkill]: - return [self.generate_kios_dummy_call()] - - def generate_place_skill( - self, parsed_action: Dict[str, Any] - ) -> List[MiosCall | MiosSkill]: - return [self.generate_kios_dummy_call()] - - def generate_screw_skill( - self, parsed_action: Dict[str, Any] - ) -> List[MiosCall | MiosSkill]: - return [self.generate_kios_dummy_call()] - - # def generate_drive_skill( - # self, parsed_action: Dict[str, Any] - # ) -> List[MiosCall | MiosSkill]: - # drivable = parsed_action["args"][2] - # container = parsed_action["args"][3] - - # if drivable is None or container is None: - # raise Exception( - # f'drivable "{drivable}" or container "{container}" is not set!' - # ) - - # kios_object = self.task_scene.get_object(container) - # if kios_object is None: - # raise Exception(f'object "{container}" is not found in the scene!') - def generate_loose_gripper_call(self) -> MiosCall: - payload = {} - return MiosCall(method_name="loose_gripper", method_payload=payload) - - def generate_drive_in_mp(self, drivable, container) -> MiosSkill: - - if container is None: - raise Exception("container is not set!") - - # get the container from the scene - kios_object = self.task_scene.get_object(container) - if kios_object is not None: - payload = { - "skill": { - "objects": { - # "Container": container, - }, - # "time_max": 60, - "p0": { - "K_x": [1500, 1500, 0, 600, 600, 600], - "dq_max": 1, - "ddq_max": 0.8, - "tighten_torque": 1.5, - "f_push": [0, 0, 8, 0, 0, 0], - }, - "p1": { - "K_x": [500, 500, 500, 600, 600, 600], - }, - "p2": { - "K_x": [500, 500, 500, 800, 800, 800], - "dq_max": 1, - "ddq_max": 1.0, - }, - "p3": { - "grasp_force": 30, - "K_x": [500, 500, 500, 800, 800, 800], - }, - }, - "control": {"control_mode": 1}, - "user": { - "env_X": [0.01, 0.01, 0.002, 0.05, 0.05, 0.05], - "env_dX": [0.001, 0.001, 0.001, 0.005, 0.005, 0.005], - }, - } - - return MiosSkill( - skill_name="drivein", - skill_type="KiosDriveIn", - skill_parameters=payload, - ) - - # def generate_drive_out_mp(self, drivable, container) -> MiosSkill: - - # if container is None: - # raise Exception("container is not set!") - - # # get the container from the scene - # kios_object = self.task_scene.get_object(container) - # if kios_object is not None: - # payload = { - # "skill": { - # "objects": { - # # "Container": container, - # }, - # # "time_max": 60, - # "p0": { - # "K_x": [1500, 1500, 0, 600, 600, 600], - # "dq_max": 1, - # "ddq_max": 0.8, - # "limit_tighten_torque": 60, - # "f_pull": [0, 0, 8, 0, 0, 0], - # "z_offset": 0.05, - # }, - # "p1": { - # "K_x": [500, 500, 500, 600, 600, 600], - # }, - # "p2": { - # "K_x": [500, 500, 500, 800, 800, 800], - # "dq_max": 1, - # "ddq_max": 1.0, - # }, - # "p3": { - # "grasp_force": 30, - # "K_x": [500, 500, 500, 800, 800, 800], - # }, - # }, - # "control": {"control_mode": 1}, - # "user": { - # "env_X": [0.01, 0.01, 0.002, 0.05, 0.05, 0.05], - # "env_dX": [0.001, 0.001, 0.001, 0.005, 0.005, 0.005], - # }, - # } - - # return MiosSkill( - # skill_name="driveout", - # skill_type="KiosDriveOut", - # skill_parameters=payload, - # ) - - def generate_drive_out_mp(self, drivable, container) -> MiosSkill: - - if container is None: - raise Exception("container is not set!") - - # get the container from the scene - kios_object = self.task_scene.get_object(container) - if kios_object is not None: - payload = { - "skill": { - "objects": { - # "Container": container, - }, - # "time_max": 60, - "p0": { - "K_x": [1500, 1500, 3, 600, 600, 0.5], - "D_x": [0.7, 0.7, 0.7, 0.7, 0.7, 1000], - # "dX_max": 0.3, - # "ddX_max": 0.3, - "f_drive": [0, 0, 0, 0, 0, 1], - "limit_tighten_torque": 60, - "f_pull": [0, 0, 8, 0, 0, 0], - "z_offset": 0.05, - }, - "p1": { - "K_x": [500, 500, 500, 600, 600, 600], - }, - "p2": { - "K_x": [500, 500, 500, 800, 800, 800], - "dX_max": [0.1, 0.3], - "ddX_max": [0.5, 1], - }, - "p3": { - "grasp_force": 30, - "K_x": [500, 500, 500, 800, 800, 800], - }, - }, - "control": {"control_mode": 0}, - "user": { - "env_X": [0.01, 0.01, 0.002, 0.05, 0.05, 0.05], - "env_dX": [0.001, 0.001, 0.001, 0.005, 0.005, 0.005], - }, - } - - return MiosSkill( - skill_name="driveout", - skill_type="KiosDriveOut", - skill_parameters=payload, - ) - - # def generate_drive_in_mp( - # self, - # drivable, - # container, - # clockwise: bool = True, - # ) -> MiosSkill: - - # if container is None: - # raise Exception("container is not set!") - - # # get the container from the scene - # kios_object = self.task_scene.get_object(container) - # if kios_object is not None: - # payload = { - # "skill": { - # "objects": { - # # "Container": container, - # }, - # "time_max": 60, - # "p0": { - # "K_x": [1500, 1500, 1500, 600, 600, 600], - # "dq_max": 0.5, - # "ddq_max": 0.6, - # "clockwise": clockwise, - # "tighten_torque": 3, - # }, - # }, - # "control": {"control_mode": 1}, - # "user": { - # "env_X": [0.01, 0.01, 0.002, 0.05, 0.05, 0.05], - # "env_dX": [0.001, 0.001, 0.001, 0.005, 0.005, 0.005], - # }, - # } - - # return MiosSkill( - # skill_name="drivein", - # skill_type="KiosDriveIn", - # skill_parameters=payload, - # ) - - def generate_insert_skill( - self, parsed_action: Dict[str, Any] - ) -> List[MiosCall | MiosSkill]: - insertable = parsed_action["args"][2] - container = parsed_action["args"][3] - if container is None: - raise Exception("container is not set!") - - # get the container from the scene - kios_object = self.task_scene.get_object(container) - if kios_object is not None: - print(f'object "{container}" is found in the scene!') - O_T_TCP = kios_object.O_T_TCP - payload = { - "skill": { - "objects": { - # "Container": container, - }, - "time_max": 25, - "p0": { - "O_T_TCP": O_T_TCP.T.flatten().tolist(), - "dX_d": [0.05, 0.3], - "ddX_d": [0.5, 1], - "DeltaX": [0, 0, 0, 0, 0, 0], - "K_x": [1500, 1500, 1500, 600, 600, 600], - }, - "p1": { - "dX_d": [0.05, 0.1], - "ddX_d": [0.1, 0.05], - "K_x": [1500, 1500, 500, 800, 800, 800], - }, - "p2": { - # "search_a": [10, 10, 0, 2, 2, 0], - # "search_f": [1, 1, 0, 1.2, 1.2, 0], - "search_a": [4, 4, 0, 1, 1, 0], - "search_f": [1, 1, 0, 1.5, 1.5, 0], - "search_phi": [ - 0, - 3.14159265358979323846 / 2, - 0, - 3.14159265358979323846 / 2, - 0, - 0, - ], - "K_x": [300, 300, 500, 500, 500, 800], - "f_push": [0, 0, 5, 0, 0, 0], - # "dX_d": [0.1, 0.5], - # "ddX_d": [0.5, 1], - "dX_d": [0.03, 0.3], - "ddX_d": [0.3, 1], - }, - "p3": { - "dX_d": [0.1, 0.5], - "ddX_d": [0.5, 1], - "f_push": 7, - "K_x": [500, 500, 0, 800, 800, 800], - }, - }, - "control": {"control_mode": 0}, - "user": { - "env_X": [0.01, 0.01, 0.002, 0.05, 0.05, 0.05], - "env_dX": [0.001, 0.001, 0.001, 0.005, 0.005, 0.005], - "F_ext_contact": [10.0, 2.0], - }, - } - # else: - # payload = { - # "skill": { - # "objects": { - # "Container": container, - # }, - # "time_max": 20, - # "p0": { # ! approach should be move above in mios skill. mod it - # "dX_d": [0.1, 0.3], - # "ddX_d": [0.5, 4], - # "DeltaX": [0, 0, 0, 0, 0, 0], - # "K_x": [1500, 1500, 1500, 600, 600, 600], - # }, - # "p1": { - # "dX_d": [0.03, 0.1], - # "ddX_d": [0.05, 0.01], - # "K_x": [500, 500, 500, 600, 600, 600], - # }, - # "p2": { - # # "search_a": [10, 10, 0, 2, 2, 0], - # # "search_f": [1, 1, 0, 1.2, 1.2, 0], - # "search_a": [5, 5, 0, 2, 2, 0], - # "search_f": [1, 1, 0, 1.2, 1.2, 0], - # "search_phi": [ - # 0, - # 3.14159265358979323846 / 2, - # 0, - # 3.14159265358979323846 / 2, - # 0, - # 0, - # ], - # "K_x": [500, 500, 500, 800, 800, 800], - # "f_push": [0, 0, 3, 0, 0, 0], - # # "dX_d": [0.1, 0.5], - # # "ddX_d": [0.5, 1], - # "dX_d": [0.08, 0.5], - # "ddX_d": [0.3, 1], - # }, - # "p3": { - # "dX_d": [0.1, 0.5], - # "ddX_d": [0.5, 1], - # "f_push": 7, - # "K_x": [500, 500, 0, 800, 800, 800], - # }, - # }, - # "control": {"control_mode": 0}, - # "user": { - # "env_X": [0.01, 0.01, 0.002, 0.05, 0.05, 0.05], - # "env_dX": [0.001, 0.001, 0.001, 0.005, 0.005, 0.005], - # "F_ext_contact": [10.0, 2.0], # ! this doesn't work. change the param in the mongodb! - # }, - # } - - # ! - from pprint import pprint - - pprint(payload) - insert = MiosSkill( - skill_name="insert", - skill_type="KiosInsertion", - skill_parameters=payload, - ) - update_object_in_mios = self.generate_teach_object_call(insertable) - update_object_in_kios = self.generate_update_object_from_mios_call(insertable) - - release = self.generate_gripper_release_mp(width=0.042) - - retreat = self.generate_move_above_mp(container) - - return [ - insert, - update_object_in_mios, - update_object_in_kios, - release, - retreat, - ] - - def generate_insert_skill_mod( - self, parsed_action: Dict[str, Any], param: Dict[str, Any] = None - ) -> List[MiosCall | MiosSkill]: - insertable = parsed_action["args"][2] - container = parsed_action["args"][3] - if container is None: - raise Exception("container is not set!") - - # get the container from the scene - kios_object = self.task_scene.get_object(container) - - if param is None: - param = { - "search_a": [2, 2, 1, 1, 1, 0], - "search_f": [1, 1, 1, 1.5, 1.5, 0], - "F_ext_contact": [13.0, 2.0], - "f_push": [0, 0, 5, 0, 0, 0], - "K_x": [300, 300, 500, 500, 500, 800], - "D_x": [0.7, 0.7, 0.7, 0.7, 0.7, 0.7], - } - - search_a = ( - param["search_a"] if "search_a" in param.keys() else [2, 2, 1, 1, 1, 0] - ) - search_f = ( - param["search_f"] if "search_f" in param.keys() else [1, 1, 1, 1.5, 1.5, 0] - ) - F_ext_contact = ( - param["F_ext_contact"] if "F_ext_contact" in param.keys() else [13.0, 2.0] - ) - f_push = param["f_push"] if "f_push" in param.keys() else [0, 0, 5, 0, 0, 0] - K_x = param["K_x"] if "K_x" in param.keys() else [300, 300, 500, 500, 500, 800] - D_x = param["D_x"] if "D_x" in param.keys() else [0.7, 0.7, 0.7, 0.7, 0.7, 0.7] - - if kios_object is not None: - print(f'object "{container}" is found in the scene!') - O_T_TCP = kios_object.O_T_TCP - payload = { - "skill": { - "objects": { - # "Container": container, - }, - # "time_max": 35, - "p0": { - "O_T_TCP": O_T_TCP.T.flatten().tolist(), - "dX_d": [0.05, 0.3], - "ddX_d": [0.5, 1], - "DeltaX": [0, 0, 0, 0, 0, 0], - "K_x": [1500, 1500, 1500, 600, 600, 600], - }, - "p1": { - "dX_d": [0.05, 0.1], - "ddX_d": [0.1, 0.05], - "K_x": [1500, 1500, 500, 800, 800, 800], - }, - "p2": { - # "search_a": [10, 10, 0, 2, 2, 0], - # "search_f": [1, 1, 0, 1.2, 1.2, 0], - "search_a": search_a, - "search_f": search_f, - "search_phi": [ - 0, - 3.14159265358979323846 / 2, - 0, - 3.14159265358979323846 / 2, - 0, - 0, - ], - "K_x": K_x, - "D_x": D_x, - "f_push": f_push, - # "dX_d": [0.1, 0.5], - # "ddX_d": [0.5, 1], - "dX_d": [0.03, 0.3], - "ddX_d": [0.3, 1], - }, - "p3": { - "dX_d": [0.1, 0.5], - "ddX_d": [0.5, 1], - "f_push": 7, - "K_x": [500, 500, 0, 800, 800, 800], - }, - }, - "control": {"control_mode": 0}, - "user": { - "env_X": ( - param["env_X"] - if "env_X" in param.keys() - else [0.01, 0.01, 0.002, 0.05, 0.05, 0.05] - ), - "env_dX": [0.001, 0.001, 0.001, 0.005, 0.005, 0.005], - "F_ext_contact": F_ext_contact, - }, - } - # else: - # payload = { - # "skill": { - # "objects": { - # "Container": container, - # }, - # "time_max": 20, - # "p0": { # ! approach should be move above in mios skill. mod it - # "dX_d": [0.1, 0.3], - # "ddX_d": [0.5, 4], - # "DeltaX": [0, 0, 0, 0, 0, 0], - # "K_x": [1500, 1500, 1500, 600, 600, 600], - # }, - # "p1": { - # "dX_d": [0.03, 0.1], - # "ddX_d": [0.05, 0.01], - # "K_x": [500, 500, 500, 600, 600, 600], - # }, - # "p2": { - # # "search_a": [10, 10, 0, 2, 2, 0], - # # "search_f": [1, 1, 0, 1.2, 1.2, 0], - # "search_a": [5, 5, 0, 2, 2, 0], - # "search_f": [1, 1, 0, 1.2, 1.2, 0], - # "search_phi": [ - # 0, - # 3.14159265358979323846 / 2, - # 0, - # 3.14159265358979323846 / 2, - # 0, - # 0, - # ], - # "K_x": [500, 500, 500, 800, 800, 800], - # "f_push": [0, 0, 3, 0, 0, 0], - # # "dX_d": [0.1, 0.5], - # # "ddX_d": [0.5, 1], - # "dX_d": [0.08, 0.5], - # "ddX_d": [0.3, 1], - # }, - # "p3": { - # "dX_d": [0.1, 0.5], - # "ddX_d": [0.5, 1], - # "f_push": 7, - # "K_x": [500, 500, 0, 800, 800, 800], - # }, - # }, - # "control": {"control_mode": 0}, - # "user": { - # "env_X": [0.01, 0.01, 0.002, 0.05, 0.05, 0.05], - # "env_dX": [0.001, 0.001, 0.001, 0.005, 0.005, 0.005], - # "F_ext_contact": [10.0, 2.0], # ! this doesn't work. change the param in the mongodb! - # }, - # } - insert = MiosSkill( - skill_name="insert", - skill_type="KiosInsertion", - skill_parameters=payload, - ) - update_object_in_mios = self.generate_teach_object_call(insertable) - update_object_in_kios = self.generate_update_object_from_mios_call(insertable) - - release = self.generate_gripper_release_mp( - width=param["width"] if "width" in param.keys() else 0.042 - ) - - retreat = self.generate_move_above_mp(container) - - return [ - insert, - update_object_in_mios, - update_object_in_kios, - release, - retreat, - ] - - def generate_insert_mp( - self, - insertable: str, - container: str, - param: Dict[str, Any] = None, - ) -> MiosSkill: - if container is None: - raise Exception("container is not set!") - - # get the container from the scene - kios_object = self.task_scene.get_object(container) - - if param is None: - param = { - "search_a": [2, 2, 1, 1, 1, 0], - "search_f": [1, 1, 1, 1.5, 1.5, 0], - "F_ext_contact": [13.0, 2.0], - "f_push": [0, 0, 5, 0, 0, 0], - "K_x": [300, 300, 500, 500, 500, 800], - "D_x": [0.7, 0.7, 0.7, 0.7, 0.7, 0.7], - } - - search_a = ( - param["search_a"] if "search_a" in param.keys() else [2, 2, 1, 1, 1, 0] - ) - search_f = ( - param["search_f"] if "search_f" in param.keys() else [1, 1, 1, 1.5, 1.5, 0] - ) - F_ext_contact = ( - param["F_ext_contact"] if "F_ext_contact" in param.keys() else [13.0, 2.0] - ) - f_push = param["f_push"] if "f_push" in param.keys() else [0, 0, 5, 0, 0, 0] - K_x = param["K_x"] if "K_x" in param.keys() else [300, 300, 500, 500, 500, 800] - D_x = param["D_x"] if "D_x" in param.keys() else [0.7, 0.7, 0.7, 1.4, 1.4, 1.4] - - if kios_object is not None: - print(f'object "{container}" is found in the scene!') - O_T_TCP = kios_object.O_T_TCP - payload = { - "skill": { - "objects": { - # "Container": container, - }, - "time_max": 60, - "p0": { - "O_T_TCP": O_T_TCP.T.flatten().tolist(), - "dX_d": [0.15, 0.5], - "ddX_d": [0.5, 1], - "DeltaX": [0, 0, 0, 0, 0, 0], - "K_x": [1500, 1500, 1500, 600, 600, 600], - }, - "p1": { - "dX_d": [0.05, 0.5], - "ddX_d": [0.1, 0.05], - "K_x": [1500, 1500, 500, 800, 800, 800], - }, - "p2": { - "search_a": search_a, - "search_f": search_f, - "search_phi": [ - 0, - 3.14159265358979323846 / 2, - 0, - 3.14159265358979323846 / 2, - 0, - 0, - ], - "K_x": K_x, - "D_x": D_x, - "f_push": f_push, - "dX_d": [0.03, 0.3], - "ddX_d": [0.3, 1], - }, - "p3": { - "dX_d": [0.1, 0.5], - "ddX_d": [0.5, 1], - "f_push": 7, - "K_x": [500, 500, 0, 800, 800, 800], - }, - }, - "control": {"control_mode": 0}, - "user": { - "env_X": ( - param["env_X"] - if "env_X" in param.keys() - else [0.01, 0.01, 0.002, 0.05, 0.05, 0.05] - ), - "env_dX": [0.001, 0.001, 0.001, 0.005, 0.005, 0.005], - "F_ext_contact": F_ext_contact, - }, - } - - return MiosSkill( - skill_name="insert", - skill_type="KiosInsertion", - skill_parameters=payload, - ) - - def generate_insert_skill_standalone_mod( - self, parsed_action: Dict[str, Any], param: Dict[str, Any] = None - ) -> List[MiosCall | MiosSkill]: - insertable = parsed_action["args"][2] - container = parsed_action["args"][3] - if container is None: - raise Exception("container is not set!") - - # get the container from the scene - kios_object = self.task_scene.get_object(container) - - if param is None: - param = { - "search_a": [2, 2, 1, 1, 1, 0], - "search_f": [1, 1, 1, 1.5, 1.5, 0], - "F_ext_contact": [13.0, 2.0], - "f_push": [0, 0, 5, 0, 0, 0], - "K_x": [300, 300, 500, 500, 500, 800], - "D_x": [0.7, 0.7, 0.7, 0.7, 0.7, 0.7], - } - - search_a = ( - param["search_a"] if "search_a" in param.keys() else [2, 2, 1, 1, 1, 0] - ) - search_f = ( - param["search_f"] if "search_f" in param.keys() else [1, 1, 1, 1.5, 1.5, 0] - ) - F_ext_contact = ( - param["F_ext_contact"] if "F_ext_contact" in param.keys() else [13.0, 2.0] - ) - f_push = param["f_push"] if "f_push" in param.keys() else [0, 0, 5, 0, 0, 0] - K_x = param["K_x"] if "K_x" in param.keys() else [300, 300, 500, 500, 500, 800] - D_x = param["D_x"] if "D_x" in param.keys() else [0.7, 0.7, 0.7, 0.7, 0.7, 0.7] - - if kios_object is not None: - print(f'object "{container}" is found in the scene!') - O_T_TCP = kios_object.O_T_TCP - payload = { - "skill": { - "objects": { - # "Container": container, - }, - "time_max": 25, - "p0": { - "O_T_TCP": O_T_TCP.T.flatten().tolist(), - "dX_d": [0.05, 0.3], - "ddX_d": [0.5, 1], - "DeltaX": [0, 0, 0, 0, 0, 0], - "K_x": [1500, 1500, 1500, 600, 600, 600], - }, - "p1": { - "dX_d": [0.05, 0.1], - "ddX_d": [0.1, 0.05], - "K_x": [1500, 1500, 500, 800, 800, 800], - }, - "p2": { - "search_a": search_a, - "search_f": search_f, - "search_phi": [ - 0, - 3.14159265358979323846 / 2, - 0, - 3.14159265358979323846 / 2, - 0, - 0, - ], - "K_x": K_x, - "D_x": D_x, - "f_push": f_push, - "dX_d": [0.03, 0.3], - "ddX_d": [0.3, 1], - }, - "p3": { - "dX_d": [0.1, 0.5], - "ddX_d": [0.5, 1], - "f_push": 7, - "K_x": [500, 500, 0, 800, 800, 800], - }, - }, - "control": {"control_mode": 0}, - "user": { - "env_X": ( - param["env_X"] - if "env_X" in param.keys() - else [0.01, 0.01, 0.002, 0.05, 0.05, 0.05] - ), - "env_dX": [0.001, 0.001, 0.001, 0.005, 0.005, 0.005], - "F_ext_contact": F_ext_contact, - }, - } - - insert = MiosSkill( - skill_name="insert", - skill_type="KiosInsertion", - skill_parameters=payload, - ) - update_object_in_mios = self.generate_teach_object_call(insertable) - update_object_in_kios = self.generate_update_object_from_mios_call(insertable) - - return [ - insert, - # update_object_in_mios, - # update_object_in_kios, - ] diff --git a/kios_bt_planning/kios_robot/robot_command.py b/kios_bt_planning/kios_robot/robot_command.py deleted file mode 100644 index ea1349c0..00000000 --- a/kios_bt_planning/kios_robot/robot_command.py +++ /dev/null @@ -1,169 +0,0 @@ -from kios_utils.task import * -from typing import List, Any -import time -from tabulate import tabulate - - -from kios_robot.robot_interface import RobotInterface -from kios_robot.data_types import MiosInterfaceResponse, MiosTaskResult -from kios_robot.data_types import MiosSkill, MiosCall, KiosCall -from kios_robot.data_types import TaskScene - -from kios_utils.bblab_utils import bb_result_test - - -class RobotCommand: - robot_address: str = None - robot_port: int = None - shared_data: Any = None - - robot_interface: RobotInterface = None - - task_scene: TaskScene = None - - task_list: List[MiosSkill | MiosCall | KiosCall] = [] - - def __init__( - self, - robot_address: str, - robot_port: int, - shared_data: Any, - task_scene: TaskScene, - robot_interface: RobotInterface, # ! not used yet - ): - if robot_address is not None: - self.robot_address = robot_address - else: - raise Exception("robot_address is not set") - - if robot_port is not None: - self.robot_port = robot_port - else: - raise Exception("robot_port is not set") - - if shared_data is not None: - self.shared_data = shared_data - else: - print("warning: robot command shared_data is not set.") - - if task_scene is not None: - self.task_scene = task_scene - else: - # raise Exception("robot_scene is not set") - print("warning: robot command task_scene is not set.") - - if robot_interface is not None: - self.robot_interface = robot_interface - else: - raise Exception("robot_interface is not set") - self.task_list = [] - - def initialize(self): - pass - # self.task_list = [] ! dont do this. the task list should be kept. - # * currently the command response is just a boolean. - # * can be extended to a more complex data type. - - def interrupt(self): - # dirty. - payload = { - "raise_exception": False, - "recover": False, - "empty_queue": False, - } - call_method(self.robot_address, self.robot_port, "stop_task", payload=payload) - - @bb_result_test - def execute_task_list_sync(self) -> bool: - self.show_tasks() - for task_item in self.task_list: - if isinstance(task_item, MiosSkill): # * use general task - mios_task = Task(self.robot_address) - mios_task.add_skill( - task_item.skill_name, - task_item.skill_type, - task_item.skill_parameters, - ) - start_result = mios_task.start() - mios_start_response = MiosInterfaceResponse.from_json( - start_result["result"] - ) - # print("Result: " + str(result)) - print("\033[92mMios replied: ") - print(mios_start_response) - print("\033[0m") - - # * check if the task is started successfully - if mios_start_response.has_finished == False: - raise Exception("task failed to start") - - result = mios_task.wait() - mios_response = MiosInterfaceResponse.from_json(result["result"]) - # print("Result: " + str(result)) - print("\033[92mMios replied: ") - print(mios_response) - print("\033[0m") # Reset color to default - - # * if the task is not finished successfully, return a false - if mios_response.has_finished == False: - return False - if mios_response.task_result.has_succeeded == False: - return False - - elif isinstance(task_item, MiosCall): # * use general call - print(f"\033[92mStart the call: {task_item.method_name}\033[0m") - result = call_method( - self.robot_address, - self.robot_port, - task_item.method_name, - task_item.method_payload, - ) - if result is None: - raise Exception( - "Mios call failed. Please check mios for more debug info." - ) - mios_response = MiosInterfaceResponse.from_json(result["result"]) - # print("Result: " + str(result)) - print("\033[92mMios replied: ") - print(mios_response) - print("\033[0m") # Reset color to default - if mios_response.has_finished == False: - return False - - elif isinstance(task_item, KiosCall): # * use general call - result_bool = task_item.method(*task_item.args) - if result_bool != True: - return False - - else: - raise Exception("Unknown task type: {}".format(task_item)) - # raise Exception("Unknown task type: {}".format(task_item)) - - print("\033[94mRobot command has successfully finished\033[0m") # Print in blue - return True - - def add_task(self, task: MiosSkill | MiosCall | KiosCall): - self.task_list.append(task) - - def add_tasks(self, tasks: List[MiosSkill | MiosCall | KiosCall]): - self.task_list.extend(tasks) - - def prepend_task(self, task: MiosSkill | MiosCall | KiosCall): - self.task_list.insert(0, task) - - def prepend_tasks(self, tasks: List[MiosSkill | MiosCall | KiosCall]): - self.task_list = tasks + self.task_list - - def clear_tasks(self): - self.task_list = [] - - def show_tasks(self): - task_table = [] - for task in self.task_list: - task_type = type(task).__name__ # mios_skill, mios_call, kios_call - task_name = str(task) - task_table.append([task_type, task_name]) - - print(tabulate(task_table, headers=["Task Type", "Task Name"], tablefmt="grid")) - - # pass diff --git a/kios_bt_planning/kios_robot/robot_interface.py b/kios_bt_planning/kios_robot/robot_interface.py deleted file mode 100644 index de5c8e2c..00000000 --- a/kios_bt_planning/kios_robot/robot_interface.py +++ /dev/null @@ -1,91 +0,0 @@ -""" -a robot interface for robot behavior control. -some functionaltiy is tailored for mios. -""" - -from kios_utils.task import * -import numpy as np -from typing import Any, List, Dict - -from kios_bt.data_types import Action - -from kios_robot.robot_proprioceptor import RobotProprioceptor -from kios_robot.mios_task_factory import MiosTaskFactory -from kios_robot.data_types import TaskScene, MiosInterfaceResponse - - -class RobotInterface: - robot_address: str = None - robot_port: int = None - - proprioceptor: RobotProprioceptor = None - mios_task_factory: MiosTaskFactory = None - - task_scene: TaskScene = None - - def __init__(self, robot_address: str = None, robot_port: int = None): - if robot_address is not None: - self.robot_address = robot_address - else: - self.robot_address = "127.0.0.1" - - if robot_port is not None: - self.robot_port = robot_port - else: - self.robot_port = 12000 - - # assert self.test_connection() == True - - self.initialize() - - def initialize(self): - self.proprioceptor = RobotProprioceptor(self.robot_address, self.robot_port) - self.mios_task_factory = MiosTaskFactory(self.task_scene, robot_interface=self) - - def mios_setup(self): - pass - # dummy_object = self.proprioceptor.get_dummy_object() - - def setup_scene(self, task_scene: TaskScene): - self.task_scene = task_scene - self.mios_task_factory.setup_scene(task_scene) - - def test_connection(self) -> bool: - response = call_method(self.robot_address, self.robot_port, "test_connection") - mios_response = MiosInterfaceResponse.from_json(response["result"]) - print(mios_response) - return mios_response.has_finished - - # * BBCORE - def generate_robot_command(self, action: Action, shared_data: Any): - """ - shard data is shared between the action node and the robot command thread. - """ - from kios_robot.robot_command import RobotCommand - - robot_command = RobotCommand( - robot_address=self.robot_address, - robot_port=self.robot_port, - shared_data=shared_data, - task_scene=self.task_scene, - robot_interface=self, # ! LET'S HACK! - ) - """core method. - generate a robot command from an action. load the shared data into the command for possible use. - - Raises: - Exception: ... - - Returns: - RobotCommand: the robot command for the action node to execute. - """ - # ! hack - tasks = self.mios_task_factory.generate_tasks( - action=action, shared_data=shared_data - ) - if tasks is not None and len(tasks) > 0: - robot_command.add_tasks(tasks) - else: - raise Exception("Action to robot command: None task is generated!") - - return robot_command diff --git a/kios_bt_planning/kios_robot/robot_proprioceptor.py b/kios_bt_planning/kios_robot/robot_proprioceptor.py deleted file mode 100644 index ed81cec8..00000000 --- a/kios_bt_planning/kios_robot/robot_proprioceptor.py +++ /dev/null @@ -1,230 +0,0 @@ -from kios_utils.task import * -import numpy as np -from typing import Any, List, Dict -from termcolor import colored -from pprint import pprint - -from kios_robot.data_types import ( - MiosObject, - KiosObject, - MiosInterfaceResponse, - RobotState, - TaskScene, -) - -from kios_scene.mongodb_interface import MongoDBInterface - - -class RobotProprioceptor: - robot_address: str = None - robot_port: int = None - - mongodb_interface: MongoDBInterface = None - - def __init__(self, robot_address: str, robot_port: int): - if robot_address is not None: - self.robot_address = robot_address - else: - raise Exception("robot_address is not set") - - if robot_port is not None: - self.robot_port = robot_port - else: - raise Exception("robot_port is not set") - - self.initialize() - - def initialize(self): - pass - self.mongodb_interface = MongoDBInterface() - - def test_connection(self) -> bool: - response = call_method(self.robot_address, self.robot_port, "test_connection") - mios_response = MiosInterfaceResponse.from_json(response["result"]) - print(mios_response) - return mios_response.has_finished - - def get_robot_state(self) -> RobotState: - response = call_method(self.robot_address, self.robot_port, "get_state") - mios_response = MiosInterfaceResponse.from_json(response["result"]) - if mios_response.has_finished: - robot_state = RobotState.from_json(response["result"]) - print(robot_state) - return robot_state - else: - raise Exception("Robot state is not ready yet.") - - def update_scene_object_from_mios(self, scene: TaskScene, object_name: str) -> bool: - try: - mios_object = self.mongodb_interface.query_mios_object(object_name) - - if scene.object_map.get(object_name) is None: - print( - "\033[93m" - + f"object {object_name} is not in the scene. Add it now." - ) - # scene.object_map[object_name] = KiosObject.from_mios_object(mios_object) - - scene.object_map[object_name] = KiosObject.from_mios_object(mios_object) - except Exception as e: - print(e) - raise e - - return True - - def update_scene_from_vision(self, scene: TaskScene): - raise NotImplementedError - - def update_scene(self, scene: TaskScene): - # * tool map does not need to be updated - # * check the obejct map first - for object_name, kios_object in scene.object_map.items(): - if kios_object.source == "mios": - mios_object = self.mongodb_interface.query_mios_object(object_name) - kios_object = kios_object.from_mios_object(mios_object) - elif kios_object.source == "pre-defined": - pass - elif kios_object.source == "tag_detection": - raise NotImplementedError - elif kios_object.source == "segmentation": - raise NotImplementedError - else: - raise Exception("Undefined source!") - - # * then the relation object map. the strategy is to generate them again - pass - - def teach_object(self, object_name: str): - return call_method( - self.robot_address, self.robot_port, "teach_object", {"object": object_name} - ) - - def teach_object_TCP(self, object_name: str): - """ - teach an mios object to mios mongodb, which has a O_T_TCP that is calcalated w.r.t. the equipped tool. - """ - # if you teach the robot directly, the O_T_EE (base to hand) will be recorded under this object name. - # this method tell mios to take the equipped tool into account - # the O_T_TCP (base to tool) will be calculated and recorded under this object name. - # ! after that, don't forget to reset the tool to the default tool. - response = call_method( - self.robot_address, - self.robot_port, - "teach_object_TCP", - {"object": object_name}, - ) - mios_response = MiosInterfaceResponse.from_json(response["result"]) - print(mios_response) - - # ! this will lead to a jerk in the robot. dont use it. - def change_EE_T_TCP(self, new_EE_T_TCP: np.ndarray): - payload = { - "EE_T_TCP": new_EE_T_TCP.T.flatten().tolist(), - } - return call_method( - self.robot_address, - self.robot_port, - "change_EE_T_TCP", - payload, - ) - - def get_object(self, object: str) -> MiosObject | None: - response = call_method( - self.robot_address, - self.robot_port, - "get_object", - {"object_name": object}, - ) - mios_response = MiosInterfaceResponse.from_json(response["result"]) - print(mios_response) - if mios_response.has_finished: - mios_object = MiosObject.from_json(response["result"]) - print(mios_object) - return mios_object - - def align_object(self, object_name: str, **kwargs: Dict[str, Any]): - """ - cheat method for BB usecase - """ - self.modify_object_position( - object_name=object_name, - x=kwargs["x"] if "x" in kwargs.keys() else 0, - y=kwargs["y"] if "y" in kwargs.keys() else 0, - z=kwargs["z"] if "z" in kwargs.keys() else 0, - R=[1, 0, 0, 0, -1, 0, 0, 0, -1], - ) - - def modify_object_position(self, object_name: str, **kwargs: Dict[str, Any]): - this_object = self.get_object(object_name) - O_T_EE = this_object.O_T_OB - O_T_TCP = this_object.O_T_TCP - if "x" in kwargs.keys(): - O_T_EE[0, 3] += kwargs["x"] - O_T_TCP[0, 3] += kwargs["x"] - if "y" in kwargs.keys(): - O_T_EE[1, 3] += kwargs["y"] - O_T_TCP[1, 3] += kwargs["y"] - if "z" in kwargs.keys(): - O_T_EE[2, 3] += kwargs["z"] - O_T_TCP[2, 3] += kwargs["z"] - if "R" in kwargs.keys(): - R = np.reshape(kwargs["R"], (3, 3)).T - O_T_EE[0:3, 0:3] = R - O_T_TCP[0:3, 0:3] = R - - this_object_json = this_object.to_json(this_object) - this_object_json["object"] = object_name - - response = call_method( - self.robot_address, - self.robot_port, - "set_object", - this_object_json, - ) - - mios_response = MiosInterfaceResponse.from_json(response["result"]) - print(colored(f"mios replied:\n {mios_response}", "green")) - - def get_object_O_T_OB(self, object_name: str): - """ - based on the investigation, O_T_OB = O_T_EE, EE_T_OB = I - """ - response = call_method( - self.robot_address, - self.robot_port, - "get_object", - {"object": object_name}, - ) - - return np.reshape(np.array(response["result"]["O_T_OB"]), (4, 4)).T - - def get_dummy_object(self) -> MiosObject | None: - response = call_method( - self.robot_address, - self.robot_port, - "get_object", - {"object_name": "NoneObject"}, - ) - - mios_response = MiosInterfaceResponse.from_json(response["result"]) - print(mios_response) - if mios_response.has_finished: - dummy_object = MiosObject.from_json(response["result"]) - print(dummy_object) - return dummy_object - else: - return None - - def set_object(self, name: str, HT: np.ndarray): - payload = { - "object": name, - "data": { - "x": HT[0, 3], - "y": HT[1, 3], - "z": HT[2, 3], - "R": HT[0:3, 0:3].T.flatten().tolist(), - }, - } - return call_method( - self.robot_address, self.robot_port, "set_partial_object_data", payload - ) diff --git a/kios_bt_planning/kios_scene/__init__.py b/kios_bt_planning/kios_scene/__init__.py deleted file mode 100644 index e69de29b..00000000 diff --git a/kios_bt_planning/kios_scene/mios_ltm_manipulator.py b/kios_bt_planning/kios_scene/mios_ltm_manipulator.py deleted file mode 100644 index cbf71e0b..00000000 --- a/kios_bt_planning/kios_scene/mios_ltm_manipulator.py +++ /dev/null @@ -1,135 +0,0 @@ -import os -import json -from kios_scene.mongodb_interface import MongoDBInterface -from kios_robot.data_types import MiosObject -from bson import ObjectId - -""" -this helps with fast deployment of the task scene -""" - - -# json serializer for ObjectId -def json_serial(obj): - """JSON serializer for objects not serializable by default json code""" - if isinstance(obj, ObjectId): - return str(obj) - raise TypeError("Type %s not serializable" % type(obj)) - - -class LangTermMemoryManipulator: - mongodb_interface: MongoDBInterface = None - backup_dir: str = None - - def __init__(self, backup_dir: str = None) -> None: - self.mongodb_interface = MongoDBInterface() - if backup_dir is not None: - self.backup_dir = backup_dir - else: - if not os.path.exists(os.path.join(os.getcwd(), "mios_memory_backups")): - os.makedirs(os.path.join(os.getcwd(), "mios_memory_backups")) - self.backup_dir = os.path.join(os.getcwd(), "mios_memory_backups") - - def backup_mios_environment(self, backup_name: str) -> None: - documents = self.mongodb_interface.fetch_all_mios_objects() - backup_file_path = os.path.join(self.backup_dir, f"{backup_name}.json") - if os.path.exists(backup_file_path): - print( - "\033[91mThe backup file with name '{}' already exists. Type yes to overwrite. Type c to cancel the operation.".format( - backup_name - ) - ) - user_input = input() - if user_input == "yes": - print( - "\033[92mOverwriting the backup file '{}'".format(backup_file_path) - ) - with open(backup_file_path, "w") as backup_file: - json.dump(documents, backup_file, default=json_serial, indent=4) - elif user_input == "c": - print("\033[92mOperation canceled.") - return - else: - print("\033[93mInvalid input. Operation canceled.") - return - else: - print( - "\033[92mWrite the mios objects to a new backup file '{}'".format( - backup_file_path - ) - ) - with open(backup_file_path, "w") as backup_file: - json.dump(documents, backup_file, default=json_serial, indent=4) - - def clear_mios_environment(self) -> None: - self.mongodb_interface.erase_all_mios_objects() - - def show_backups(self) -> None: - print("The existing backups are:") - for file in os.listdir(self.backup_dir): - backup_name = file.split(".")[0] - print("- {}".format(backup_name)) - - def update_to_mios_environment(self, backup_name: str) -> None: - """ - update objects. - if not exist, insert, if exist, update - """ - backup_file_path = os.path.join(self.backup_dir, f"{backup_name}.json") - if not os.path.exists(backup_file_path): - print( - "\033[93mThe backup file '{}' does not exist.".format(backup_file_path) - ) - return - - print( - "\033[91mPlease make sure you want to execute this operation. Type yes to confirm. Type c to cancel the operation." - ) - user_input = input() - if user_input == "yes": - with open(backup_file_path, "r") as backup_file: - documents = json.load(backup_file) - self.mongodb_interface.update_mios_objects(documents) - print( - "\033[92mUpdated the mios objects from '{}'".format( - backup_file_path - ) - ) - elif user_input == "c": - print("\033[92mOperation canceled.") - return - else: - print("\033[93mInvalid input. Operation canceled.") - return - - def restore_to_mios_environment(self, backup_name: str) -> None: - """ - erase all and inject the backup - """ - backup_file_path = os.path.join(self.backup_dir, f"{backup_name}.json") - if not os.path.exists(backup_file_path): - print( - "\033[93mThe backup file '{}' does not exist.".format(backup_file_path) - ) - return - - print( - "\033[91mPlease make sure you want to execute this operation. Type yes to confirm. Type c to cancel the operation." - ) - user_input = input() - if user_input == "yes": - with open(backup_file_path, "r") as backup_file: - documents = json.load(backup_file) - self.mongodb_interface.erase_all_mios_objects() - self.mongodb_interface.inject_mios_objects(documents) - print( - "\033[92mRestored the mios objects from '{}'".format( - backup_file_path - ) - ) - elif user_input == "c": - print("\033[92mOperation canceled.") - return - else: - print("\033[93mInvalid input. Operation canceled.") - return diff --git a/kios_bt_planning/kios_scene/mongodb_interface.py b/kios_bt_planning/kios_scene/mongodb_interface.py deleted file mode 100644 index d6919d04..00000000 --- a/kios_bt_planning/kios_scene/mongodb_interface.py +++ /dev/null @@ -1,153 +0,0 @@ -from pymongo import MongoClient, database, collection -from typing import Dict, List -import logging - -from kios_robot.data_types import MiosObject - - -class MongoDBInterface: - client: MongoClient = None - mios_database: database.Database = None - mios_environment_collection: collection.Collection = None - kios_database: database.Database = None - mios_collections: Dict[str, collection.Collection] = {} - kios_collections: Dict[str, collection.Collection] = {} - - def __init__(self, address: str = "127.0.0.1", port: int = 27017): - self.client = MongoClient(address, port) - - self.mios_database = self.client["miosL"] - self.mios_environment_collection = self.mios_database["environment"] - self.mios_collections["environment"] = self.mios_environment_collection - self.kios_database = self.client["kios"] - # ! BBTEST - # self.setup_defaults() - - # ! DISCARDED - def setup_defaults(self): - # if tools collection does not exist, create it. add the default tools. - if "tools" not in self.kios_database.list_collection_names(): - self.kios_database.create_collection("tools") - self.kios_database["tools"].insert_one( - { - "name": "parallel_box1", - "EE_T_TCP": [ - [1, 0, 0, 0.0], - [0, 1, 0, 0.0], - [0, 0, 1, 0.1], - [0, 0, 0, 1], - ], - } - ) - self.kios_database["tools"].insert_one( - { - "name": "parallel_box2", - "EE_T_TCP": [ - [1, 0, 0, 0.0], - [0, 1, 0, 0.0], - [0, 0, 1, 0.1], - [0, 0, 0, 1], - ], - } - ) - self.kios_database["tools"].insert_one( - { - "name": "inward_claw", - "EE_T_TCP": [ - [1, 0, 0, 0.0], - [0, 1, 0, 0.0], - [0, 0, 1, 0.1], - [0, 0, 0, 1], - ], - } - ) - self.kios_database["tools"].insert_one( - { - "name": "outward_claw", - "EE_T_TCP": [ - [1, 0, 0, 0.0], - [0, 1, 0, 0.0], - [0, 0, 1, 0.1], - [0, 0, 0, 1], - ], - } - ) - - def query_mios_object(self, object_name: str) -> MiosObject: - """ - be careful. the name can be duplicated. - """ - query = {"name": object_name} - result = self.mios_environment_collection.find(query) - count = self.mios_environment_collection.count_documents(query) - if count > 1: - raise Exception("Duplicated object name!") - elif count == 0: - # raise Exception(f'object "{object_name}" not found in the database') - logging.warning(f'object "{object_name}" not found in the database') - # ! HACK - return MiosObject.generate_dummy(object_name=object_name) - for document in result: - return MiosObject.from_json(document) - - def fetch_all_mios_objects(self) -> List[Dict]: - target_collection = self.mios_collections.get("environment") - documents = list(target_collection.find()) - return documents - - def erase_all_mios_objects(self) -> None: - print( - "\033[91mDangerous operation. Type 'yes' to continue. Type 'c' to cancel.\033[0m" - ) - user_input = input() - if user_input == "yes": - # Erase all the objects in the collection - target_collection = self.mios_collections.get("environment") - target_collection.delete_many({}) - print("\033[92mAll objects erased.\033[0m") - elif user_input == "c": - # Quit the program - print("\033[92mOperation canceled.\033[0m") - else: - # Invalid input, handle accordingly - print("\033[93mInvalid input. Operation canceled.\033[0m") - - def inject_mios_objects(self, documents: List[Dict]): - target_collection = self.mios_collections.get("environment") - target_collection.insert_many(documents) - - def update_mios_objects(self, documents: List[Dict]): - target_collection = self.mios_collections.get("environment") - for document in documents: - query = {"name": document["name"]} - target_collection.replace_one(query, document) - - def query(self, query): - return self.collection.find(query) - - # def query_from_collection(self, key: str, collection_name: str): - # target_collection = self.mios_collections.get(collection_name) - # if target_collection is None: - # raise Exception(f"Collection {collection_name} not found") - # query = {key: collection_name} - # return target_collection.find(query) - - def save(self, document): - return self.collection.insert_one(document) - - def fetch_all_from_mios_collection(self, collection_name: str) -> List[Dict]: - target_collection = self.mios_collections.get(collection_name) - if target_collection is None: - raise Exception(f"Collection {collection_name} not found") - documents = list(target_collection.find()) - return documents - - -def test_query(): - interface = MongoDBInterface() - result = interface.query_mios_object("EndEffector") - print(result) - - -if __name__ == "__main__": - test_query() diff --git a/kios_bt_planning/kios_scene/scene_factory.py b/kios_bt_planning/kios_scene/scene_factory.py deleted file mode 100644 index 6650b170..00000000 --- a/kios_bt_planning/kios_scene/scene_factory.py +++ /dev/null @@ -1,106 +0,0 @@ -from typing import Dict, List, Any -import json -import numpy as np - -from kios_robot.data_types import ( - MiosObject, - TaskScene, - ReferenceRelation, - Toolbox, - KiosObject, -) -from kios_scene.mongodb_interface import MongoDBInterface -import json -import os - - -class SceneFactory: - task_scene: TaskScene = None - - def __init__(self): - pass - - def create_scene_from_json(self, scene_json: json) -> TaskScene: - # create the reference objects first - mongodb_interface = MongoDBInterface() - self.task_scene = TaskScene() - - for reference_object_json in scene_json["reference_objects"]: - if reference_object_json["source"] == "pre-defined": - self.task_scene.object_map[reference_object_json["object_name"]] = ( - KiosObject.from_json(reference_object_json) - ) - elif reference_object_json["source"] == "mios": - mios_object = mongodb_interface.query_mios_object( - reference_object_json["object_name"] - ) - kios_object = KiosObject.from_mios_object(mios_object) - self.task_scene.object_map[kios_object.name] = kios_object - elif reference_object_json["source"] == "tag_detection": - raise NotImplementedError - elif reference_object_json["source"] == "segmentation": - raise NotImplementedError - else: - raise Exception("Undefined source!") - - # # create the relative objects - # for object_name, reference_relation in scene_json["relative_objects"].items(): - # # construct the reference relation - # raise NotImplementedError - # # ! BBWORK suspend here. need to figure out if using MiosObject is a good idea. - - # create the tools - for tool_json in scene_json["tools"]: - tool = Toolbox( - name=tool_json["tool_name"], - EE_T_TCP=np.array(tool_json["EE_T_TCP"]), - EE_finger_width_max=( - tool_json["EE_finger_width_max"] - if "EE_finger_width_max" in tool_json.keys() - else 0.08 - ), - EE_finger_width_min=( - tool_json["EE_finger_width_min"] - if "EE_finger_width_min" in tool_json.keys() - else 0.01 - ), - # ! BBCHANGE - ) - # print(tool) - self.task_scene.tool_map[tool.name] = tool - - return self.task_scene - - def create_test_scene(self) -> TaskScene: - file_dir = os.path.dirname(os.path.abspath(__file__)) - - # Define the path to the JSON file - json_file_path = os.path.join(file_dir, "test_scene.json") - - # Open the JSON file and load it as a JSON object - with open(json_file_path, "r") as file: - json_object = json.load(file) - - scene = self.create_scene_from_json(json_object) - # print(scene) - return scene - - -def test_sf(): - # Get the current directory - current_dir = os.path.dirname(os.path.abspath(__file__)) - - # Define the path to the JSON file - json_file_path = os.path.join(current_dir, "test_scene.json") - - # Open the JSON file and load it as a JSON object - with open(json_file_path, "r") as file: - json_object = json.load(file) - - sf = SceneFactory() - scene = sf.create_scene_from_json(json_object) - print(scene) - - -if __name__ == "__main__": - test_sf() diff --git a/kios_bt_planning/kios_scene/test_scene copy.json b/kios_bt_planning/kios_scene/test_scene copy.json deleted file mode 100644 index 368e4c4f..00000000 --- a/kios_bt_planning/kios_scene/test_scene copy.json +++ /dev/null @@ -1,116 +0,0 @@ -{ - "reference_objects": - [ - { - "object_name": "test_box1", - "source": "pre-defined", - "O_T_TCP": [ - [1, 0, 0, 0.0], - [0, 1, 0, 0.0], - [0, 0, 1, -0.1], - [0, 0, 0, 1] - ] - }, - { - "object_name": "TestObject1", - "source": "mios" - }, - { - "object_name": "test_object1", - "source": "mios" - }, - { - "object_name": "gear3", - "source": "mios" - }, - { - "object_name": "shaft2", - "source": "mios" - }, - { - "object_name": "inward_claw", - "source": "mios" - }, - { - "object_name": "test", - "source": "mios" - } - ], - "tools": - [ - { - "tool_name": "parallel_box1", - "EE_T_TCP": [ - [1, 0, 0, 0.0], - [0, 1, 0, 0.0], - [0, 0, 1, 0.085], - [0, 0, 0, 1] - ], - "EE_finger_width_max": 0.042, - "EE_finger_width_min": 0.01, - "load_width": 0.042, - "unload_width": 0.08 - }, - { - "tool_name": "parallel_box2", - "EE_T_TCP": [ - [1, 0, 0, 0.0], - [0, 1, 0, 0.0], - [0, 0, 1, 0.07], - [0, 0, 0, 1] - ], - "EE_finger_width_max": 0.042, - "EE_finger_width_min": 0.01, - "load_width": 0.042, - "unload_width": 0.08 - }, - { - "tool_name": "inward_claw", - "EE_T_TCP": [ - [1, 0, 0, 0.0], - [0, 1, 0, 0.0], - [0, 0, 1, 0.095], - [0, 0, 0, 1] - ], - "EE_finger_width_max": 0.042, - "EE_finger_width_min": 0.01, - "load_width": 0.042, - "unload_width": 0.08 - }, - { - "tool_name": "outward_claw", - "EE_T_TCP": [ - [1, 0, 0, 0.0], - [0, 1, 0, 0.0], - [0, 0, 1, 0.095], - [0, 0, 0, 1] - ], - "EE_finger_width_max": 0.042, - "EE_finger_width_min": 0.01, - "load_width": 0.042, - "unload_width": 0.08 - }, - { - "tool_name": "no_tool", - "EE_T_TCP": [ - [1, 0, 0, 0.0], - [0, 1, 0, 0.0], - [0, 0, 1, 0.025], - [0, 0, 0, 1] - ] - }, - { - "tool_name": "default_tool", - "EE_T_TCP": [ - [1, 0, 0, 0.0], - [0, 1, 0, 0.0], - [0, 0, 1, 0.0], - [0, 0, 0, 1] - ], - "EE_finger_width_max": 0.08, - "EE_finger_width_min": 0.01, - "load_width": 0.08, - "unload_width": 0.08 - } - ] -} diff --git a/kios_bt_planning/kios_scene/test_scene.json b/kios_bt_planning/kios_scene/test_scene.json deleted file mode 100644 index 62f2cdad..00000000 --- a/kios_bt_planning/kios_scene/test_scene.json +++ /dev/null @@ -1,143 +0,0 @@ -{ - "reference_objects": - [ - { - "object_name": "test_box1", - "source": "pre-defined", - "O_T_TCP": [ - [1, 0, 0, 0.0], - [0, 1, 0, 0.0], - [0, 0, 1, -0.1], - [0, 0, 0, 1] - ] - }, - { - "object_name": "TestObject1", - "source": "mios" - }, - { - "object_name": "inward_claw", - "source": "mios" - }, - { - "object_name": "outward_claw", - "source": "mios" - }, - { - "object_name": "parallel_box1", - "source": "mios" - }, - { - "object_name": "parallel_box2", - "source": "mios" - }, - { - "object_name": "no_tool", - "source": "mios" - }, - { - "object_name": "gear1", - "source": "mios" - }, - { - "object_name": "gear2", - "source": "mios" - }, - { - "object_name": "gear3", - "source": "mios" - }, - { - "object_name": "shaft1", - "source": "mios" - }, - { - "object_name": "shaft2", - "source": "mios" - }, - { - "object_name": "shaft3", - "source": "mios" - }, - { - "object_name": "gearbase", - "source": "mios" - }, - { - "object_name": "gearbase_hole1", - "source": "mios" - }, - { - "object_name": "gearbase_hole3", - "source": "mios" - } - ], - "tools": - [ - { - "tool_name": "parallel_box1", - "EE_T_TCP": [ - [1, 0, 0, 0.0], - [0, 1, 0, 0.0], - [0, 0, 1, 0.085], - [0, 0, 0, 1] - ], - "EE_finger_width_max": 0.042, - "EE_finger_width_min": 0.01, - "load_width": 0.042, - "unload_width": 0.08 - }, - { - "tool_name": "parallel_box2", - "EE_T_TCP": [ - [1, 0, 0, 0.0], - [0, 1, 0, 0.0], - [0, 0, 1, 0.07], - [0, 0, 0, 1] - ], - "EE_finger_width_max": 0.042, - "EE_finger_width_min": 0.01, - "load_width": 0.042, - "unload_width": 0.08 - }, - { - "tool_name": "inward_claw", - "EE_T_TCP": [ - [1, 0, 0, 0.0], - [0, 1, 0, 0.0], - [0, 0, 1, 0.095], - [0, 0, 0, 1] - ], - "EE_finger_width_max": 0.042, - "EE_finger_width_min": 0.01, - "load_width": 0.042, - "unload_width": 0.08 - }, - { - "tool_name": "outward_claw", - "EE_T_TCP": [ - [1, 0, 0, 0.0], - [0, 1, 0, 0.0], - [0, 0, 1, 0.125], - [0, 0, 0, 1] - ], - "EE_finger_width_max": 0.042, - "EE_finger_width_min": 0.01, - "load_width": 0.042, - "unload_width": 0.08 - }, - { - "tool_name": "no_tool", - "EE_T_TCP": [ - [1, 0, 0, 0.0], - [0, 1, 0, 0.0], - [0, 0, 1, 0.025], - [0, 0, 0, 1] - ], - "EE_finger_width_max": 0.08, - "EE_finger_width_min": 0.01, - "load_width": 0.08, - "unload_width": 0.08 - } - ] -} diff --git a/kios_bt_planning/kios_utils/__init__.py b/kios_bt_planning/kios_utils/__init__.py deleted file mode 100644 index e69de29b..00000000 diff --git a/kios_bt_planning/kios_utils/bblab_utils.py b/kios_bt_planning/kios_utils/bblab_utils.py deleted file mode 100644 index 53f8a0e4..00000000 --- a/kios_bt_planning/kios_utils/bblab_utils.py +++ /dev/null @@ -1,41 +0,0 @@ -import time - - -def bb_result_test(func): - - def wrapper(*args, **kwargs): - result = func(*args, **kwargs) - if result == False: - pause = input("Failed. Press enter to continue") - - return result - - return wrapper - - -def execution_timer(func): - """ - be careful with this since the return value is suppressed - """ - - def wrapper(*args, **kwargs): - start = time.perf_counter() - func(*args, **kwargs) - end = time.perf_counter() - print(f"Execution time: {end - start} seconds") - execution_duration = end - start - return execution_duration - - return wrapper - - -def test_timer(): - @execution_timer - def test_function(): - time.sleep(2) - - print(test_function()) # 2.0000000000000004 - - -if __name__ == "__main__": - test_timer() diff --git a/kios_bt_planning/kios_utils/math_utils.py b/kios_bt_planning/kios_utils/math_utils.py deleted file mode 100644 index 4d86e531..00000000 --- a/kios_bt_planning/kios_utils/math_utils.py +++ /dev/null @@ -1,42 +0,0 @@ -import numpy as np -from spatialmath import * - -# ! NOT IN USE I GUESS - - -def HT_inverse(HT: np.ndarray) -> np.ndarray: - """ - Inverse of a homogeneous transformation matrix - """ - R = HT[0:3, 0:3] - t = HT[0:3, 3] - HT_inv = np.zeros((4, 4)) - HT_inv[0:3, 0:3] = R.transpose() - HT_inv[0:3, 3] = -R.transpose().dot(t) - HT_inv[3, 3] = 1 - return HT_inv - - -def R_from_rpy(vec: np.ndarray) -> np.ndarray: - """ - Rotation matrix from a 3D rpy vector - """ - R = SE3.RPY(vec, unit="deg", order="xyz") - return R - - -def HT_from_xyzrpy(vec: np.ndarray) -> np.ndarray: - """ - Homogeneous transformation matrix from a 6D xyzrpy vector - """ - HT = SE3(vec[0:3]) * SE3.RPY(vec[3:6], unit="deg", order="xyz") - return HT - - -# not tested -def generate_HT(HT: np.ndarray, xyz: np.ndarray, rpy: np.ndarray) -> np.ndarray: - """ - Generate a homogeneous transformation matrix from a 6D vector - """ - HT_new = HT * SE3(xyz) * SE3.RPY(rpy, order="xyz") - return HT_new diff --git a/kios_bt_planning/kios_utils/parsers.py b/kios_bt_planning/kios_utils/parsers.py deleted file mode 100644 index 8c46e15a..00000000 --- a/kios_bt_planning/kios_utils/parsers.py +++ /dev/null @@ -1,13 +0,0 @@ -import re - -def match_type(node: dict) -> tuple[str, str]: - node_name = node["name"] - match = re.search( - r"(selector|sequence|action|precondition|condition|target):\s*(.+)", node_name - ) - if match: - node_type = match.group(1) - node_body = match.group(2) - return node_type, node_body - else: - raise ValueError(f"the node name {node_name} does not match any type.") diff --git a/kios_bt_planning/kios_utils/pddl_problem_parser.py b/kios_bt_planning/kios_utils/pddl_problem_parser.py deleted file mode 100644 index 84ea8b1e..00000000 --- a/kios_bt_planning/kios_utils/pddl_problem_parser.py +++ /dev/null @@ -1,190 +0,0 @@ -import re -import os -from dotenv import load_dotenv -import json - -load_dotenv() - -# * kios data dir -data_dir = os.environ.get("KIOS_DATA_DIR").format(username=os.getlogin()) -print(data_dir) -world_definition_json = os.path.join(data_dir, "world_definition.json") - -with open(world_definition_json, "r") as f: - world_definition = json.load(f) - -properties = world_definition["properties"] - -constraints = world_definition["constraints"] - -relations = world_definition["relations"] - -# ! now in json file. -# properties = [ -# "is_free", -# "is_equippable", -# ] - -# constraints = [ -# "can_manipulate", -# "can_screw_to", -# "can_insert_to", -# "can_place_to", -# ] - -# relations = [ -# "hold", -# "is_inserted_to", -# "is_screwed_to", -# "is_placed_to", -# ] - - -def parse_problem_objects(problem: str) -> dict[str, list[dict[str, any]]]: - pass - - -def parse_problem_init( - problem: str, object_list: list[str] = None -) -> dict[str, list[dict[str, any]]]: - """ - parse to get the world state from the problem. - pass the object list if you want to add all the objects to the world state. - """ - init_pattern = r"\(:init\s+(.*?)\s+\)" - element_pattern = r"\((.*?)\)" - - init_match = re.search(init_pattern, problem, re.DOTALL) - if init_match: - init_content = init_match.group(1) - elements = re.findall(element_pattern, init_content) - - parsed_init = [] - for element in elements: - parts = element.split() - name = parts[0] - args = parts[1:] - parsed_init.append({"name": name, "args": args}) - else: - raise ValueError("No init found in the problem") - - if object_list is None: - object_list = [] - - world_state: dict[str, list[dict[str, any]]] = { - "objects": [], - "constraints": [], - "relations": [], - } - - # add all the objects - for item in object_list: - world_state["objects"].append({"name": item, "properties": []}) - - for item in parsed_init: - item_name = item.get("name") - item_type = classify_item(item_name) - - if item_type == "property": - world_state["objects"].append( - {"name": item["args"][0], "properties": [item_name]} # ! list or item? - ) - - elif item_type == "constraint": - world_state["constraints"].append( - { - "source": item["args"][0], - "name": item_name, - "target": item["args"][1], - } - ) - elif item_type == "relation": - world_state["relations"].append( - { - "source": item["args"][0], - "name": item_name, - "target": item["args"][1], - } - ) - else: - pass - - return world_state - - -def parse_problem(problem: str) -> dict[str, list[dict[str, any]]]: - # get objects - object_dict = parse_problem_objects(problem=problem) - - # merge - object_list: list[str] = [] - for key, value in object_dict.items(): - # type is not used - object_list.extend(value) - - # get world_state - world_state = parse_problem_init(problem=problem, object_list=object_list) - - return world_state - - -def classify_item(item_name: str) -> str: - if item_name in properties: - return "property" - elif item_name in constraints: - return "constraint" - elif item_name in relations: - return "relation" - else: - raise ValueError(f"Unknown item name: {item_name}") - - -# Usage example: -example_problem = """ -(define (problem robot_assembly_problem-problem) - (:domain robot_assembly_problem-domain) - (:objects - parallel_box1 parallel_box2 inward_claw outward_claw no_tool - tool - gear1 gear2 gear3 shaft1 shaft2 shaft3 gearbase gearbase_hole1 gearbase_hole3 - part - left_hand - hand - ) - (:init (can_manipulate parallel_box2 gear1) (is_free left_hand) (is_free no_tool) (is_equippable no_tool) (can_manipulate outward_claw gear2) (can_manipulate outward_claw gear3) (can_manipulate no_tool shaft3) (can_manipulate parallel_box1 shaft1) (can_screw_to leg1 seat) (can_screw_to leg2 seat) (can_insert_to back seat) (can_screw_to nut1 seat) (can_screw_to nut2 seat) (can_screw_to blub lampbase) (can_place_to lamp blub) (can_insert_to shaft1 gearbase_hole1) (can_insert_to shaft3 gearbase_hole3) (can_insert_to gear3 shaft3) (can_insert_to gear2 shaft2) (can_insert_to gear1 shaft1)) - (:goal (and )) - )""" - - -def parse_problem_objects(problem: str) -> dict[str, list[str]]: - objects_start = problem.find("(:objects") - objects_end = problem.find(")", objects_start) - objects_section = problem[objects_start:objects_end] - - items = {} - lines = objects_section.split("\n") - for line in lines: - if "-" in line: - item, *other_items = line.split("-") - item = item.strip() - other_items = [item.strip() for item in other_items] - items[other_items[0]] = item.split() - - return items - - -def test(problem: str): - # parsed_init = parse_problem_init(problem) - # print(parsed_init) - world_state = parse_problem(problem) - # for key, value in world_state.items(): - # print(f"{key}:") - # for item in value: - # print(item) - from pprint import pprint - - pprint(world_state) - - pprint(parse_problem_objects(problem)) - - -if __name__ == "__main__": - test(example_problem) - pass diff --git a/kios_bt_planning/kios_utils/pybt_test.py b/kios_bt_planning/kios_utils/pybt_test.py deleted file mode 100644 index 7a1fd62c..00000000 --- a/kios_bt_planning/kios_utils/pybt_test.py +++ /dev/null @@ -1,138 +0,0 @@ -import py_trees -import functools -import os -import time -import sched - - -def post_tick_handler( - snapshot_visitor: py_trees.visitors.SnapshotVisitor, - behaviour_tree: py_trees.trees.BehaviourTree, -) -> None: - """ - Print data about the part of the tree visited. - - Args: - snapshot_handler: gather data about the part of the tree visited - behaviour_tree: tree to gather data from - """ - print( - "\n" - + py_trees.display.unicode_tree( - root=behaviour_tree.root, - visited=snapshot_visitor.visited, - previously_visited=snapshot_visitor.previously_visited, - ) - ) - print(py_trees.display.unicode_blackboard()) - - -def pre_tick_handler(behaviour_tree: py_trees.trees.BehaviourTree) -> None: - """Print a banner with current tick count prior to ticking the tree. - - Args: - behaviour_tree: the tree to tick (used to fetch the count number) - """ - print("\n--------- Run %s ---------\n" % behaviour_tree.count) - - -def generate_bt_stewardship(bt) -> py_trees.trees.BehaviourTree: - #################### - # Tree Stewardship - #################### - behaviour_tree = py_trees.trees.BehaviourTree(bt) - behaviour_tree.add_pre_tick_handler(pre_tick_handler) - behaviour_tree.visitors.append(py_trees.visitors.DebugVisitor()) - snapshot_visitor = py_trees.visitors.SnapshotVisitor() - behaviour_tree.add_post_tick_handler( - functools.partial(post_tick_handler, snapshot_visitor) - ) - behaviour_tree.visitors.append(snapshot_visitor) - behaviour_tree.setup(timeout=15) - - return behaviour_tree - - -def render_dot_tree(bt: py_trees.trees.BehaviourTree): - py_trees.display.render_dot_tree( - bt.root, - # visibility_level=py_trees.common.VisibilityLevel.ALL, - with_blackboard_variables=False, - # with_qualified_names=True, - # target_directory=os.path(__file__), - ) - - -def tick_loop_test(bt: py_trees.trees.BehaviourTree): - py_trees.logging.level = py_trees.logging.Level.DEBUG - while True: - try: - bt.tick() - py_trees.console.read_single_keypress() - except KeyboardInterrupt: - break - - print("\n") - - -def tick_1000HZ_test(bt: py_trees.trees.BehaviourTree): - py_trees.logging.level = py_trees.logging.Level.DEBUG - - tick_count = 0 - - def tick(): - nonlocal tick_count - bt.tick() - tick_count += 1 - if bt.root.status == py_trees.common.Status.SUCCESS: - print("\033[94mTree finished with success\033[0m") - elif bt.root.status == py_trees.common.Status.FAILURE: - print("\033[91mTree finished with failure\033[0m") - elif bt.root.status == py_trees.common.Status.RUNNING: - scheduler.enter(1 / 1000, 1, tick) - else: - raise Exception("Unknown status!") - - scheduler = sched.scheduler(time.time, time.sleep) - scheduler.enter(0, 1, tick) - scheduler.run() - - -def tick_frequency_test(bt: py_trees.trees.BehaviourTree): - py_trees.logging.level = py_trees.logging.Level.DEBUG - - tick_count = 0 - start_time = time.time() - - def tick(): - nonlocal tick_count - bt.tick() - tick_count += 1 - if bt.root.status == py_trees.common.Status.SUCCESS: - print("\033[94mTree finished with success\033[0m") - elif bt.root.status == py_trees.common.Status.FAILURE: - print("\033[91mTree finished with failure\033[0m") - elif bt.root.status == py_trees.common.Status.RUNNING: - scheduler.enter(1 / 1000, 1, tick) - else: - raise Exception("Unknown status!") - - scheduler = sched.scheduler(time.time, time.sleep) - scheduler.enter(0, 1, tick) - scheduler.run() - - end_time = time.time() - run_time = end_time - start_time - frequency = tick_count / run_time - - print(f"Tick count: {tick_count}") - print(f"Run time: {run_time} seconds") - print(f"Running frequency: {frequency} Hz") - - -def tick_once_test(bt: py_trees.trees.BehaviourTree): - py_trees.logging.level = py_trees.logging.Level.DEBUG - bt.tick() - py_trees.console.read_single_keypress() - - print("\n") diff --git a/kios_bt_planning/kios_utils/skeleton_parser.py b/kios_bt_planning/kios_utils/skeleton_parser.py deleted file mode 100644 index 14a6c0af..00000000 --- a/kios_bt_planning/kios_utils/skeleton_parser.py +++ /dev/null @@ -1,157 +0,0 @@ -import os -import re -from dotenv import load_dotenv -import json -from dataclasses import dataclass - -from pprint import pprint - -from kios_bt.data_types import ObjectProperty - -""" -only skeleton node parser here -""" - - -load_dotenv() - -data_dir = os.environ.get("KIOS_DATA_DIR").format(username=os.getlogin()) -print(data_dir) -world_definition_json = os.path.join(data_dir, "world_definition.json") - -with open(world_definition_json, "r") as f: - world_definition = json.load(f) - - -@dataclass -class ParsedNode: - """ - class to store the parsed node (condition or action) - """ - - typename: str - itemname: str - params: list[str] - - -def parse_node_name(name: str) -> ParsedNode: - """ - parse a node name to extract the type, name and parameters - typename: selector, sequence, condition, precondition, target, action - itemname: the name of the condition/action - params: the parameters of the condition/action - - Example: - a condition: hold(hand1, tool1) - an action: pick_up(hand1, tool1, target1) - """ - pattern = r"(\w+): (\w+)\((.*?)\)" - match = re.match(pattern, name) - if match: - typename = match.group(1) - itemname = match.group(2) - params = match.group(3).split(", ") if match.group(3) else [] - return ParsedNode(typename, itemname, params) - else: - raise ValueError(f'invalid node name "{name}"!!!') - - -def parse_node_type(name: str) -> str: - - pattern = r"(selector|sequence|condition|precondition|target|action)" - match = re.search(pattern, name) - if match: - return match.group(1) - else: - return None - - -def ground_action( - action_name: str, params: list[str] -) -> tuple[list[ObjectProperty], list[ObjectProperty]]: - """ - furthre ground the parsed action node, extract its precondition and effect according to the definition. - - The definitions are in the world_definition.json file in the "data" directory. See dotenv import sentence. - - return: - - a list of precondition ObjectProperty - - a list of effect ObjectProperty - - """ - if world_definition["actions"].get(action_name) is None: - raise ValueError(f"Action {action_name} is not defined in the world definition") - action_data = world_definition["actions"][action_name] - template = action_data["template"] - precondition = action_data["precondition"] - effect = action_data["effect"] - - # Extract the placeholders from the template (pattern: ?) - placeholders = re.findall(r"\?\w+", template) - - # Replace the placeholders one by one with the corresponding parameter value - for i in range(len(placeholders)): - escaped_placeholder = re.escape(placeholders[i]) - precondition = [ - re.sub(escaped_placeholder, params[i], pre) for pre in precondition - ] - effect = [re.sub(escaped_placeholder, params[i], eff) for eff in effect] - - precondition_list: list[ObjectProperty] = [] - effect_list: list[ObjectProperty] = [] - - for item in precondition: - match = re.match(r"\(?(not )?\(?(\w+) (\w+)? ?(\w+)?\)?\)?", item) - - if match: - # Extract the property name, object name, and property value from the match object - status, property_name, object_name, property_value = match.groups() - - # Determine the status based on whether the string starts with "(not" - status = False if status else True - - # Return a dictionary with the extracted values - precondition_list.append( - ObjectProperty( - object_name=object_name, - property_name=property_name, - property_value=property_value if property_value else None, - status=status, - ) - ) - - for item in effect: - match = re.match(r"\(?(not )?\(?(\w+) (\w+)? ?(\w+)?\)?\)?", item) - - if match: - # Extract the property name, object name, and property value from the match object - status, property_name, object_name, property_value = match.groups() - - # Determine the status based on whether the string starts with "(not" - status = False if status else True - - # Return a dictionary with the extracted values - effect_list.append( - ObjectProperty( - object_name=object_name, - property_name=property_name, - property_value=property_value if property_value else None, - status=status, - ) - ) - - return precondition_list, effect_list - - -# Test the function with the 'unscrew' action and some parameters -# pprint(ground_action("unscrew", ["a1", "b2", "c3", "d4"])) -# print(ground_action("load_tool", ["a1", "b2"])) -# print(ground_action("put_down", ["a1", "b2", "c3"])) -# print(ground_action("place", ["a1", "b2", "c3", "d4"])) -# pprint(ground_action("detach", ["a1", "b2", "c3", "d4"])) -# pprint(ground_action("insert", ["a1", "b2", "c3", "d4"])) -# print(ground_action("pull", ["a1", "b2", "c3", "d4"])) -# print(ground_action("screw", ["a1", "b2", "c3", "d4"])) -# pprint(ground_action("unload_tool", ["a1", "b2"])) -# print(ground_action("pick_up", ["a1", "b2", "c3"])) -# print(ground_action("unload_tool", ["a1", "b2"])) diff --git a/kios_bt_planning/kios_utils/task.py b/kios_bt_planning/kios_utils/task.py deleted file mode 100644 index 600e17fa..00000000 --- a/kios_bt_planning/kios_utils/task.py +++ /dev/null @@ -1,93 +0,0 @@ -#!/usr/bin/python3 -u -from .ws_client import * -import time - - -class Skill: - def __init__(self, skill_name=None, skill_type=None, skill_context=None): - self.skill_name = skill_name - self.skill_type = skill_type - self.skill_context = skill_context - - def initialize(self, skill_name, skill_type, skill_context): - self.skill_name = skill_name - self.skill_type = skill_type - self.skill_context = skill_context - - -class Task: - def __init__(self, robot, shared_data=None): - self.skill_names = [] - self.skill_types = [] - self.skill_context = dict() - - self.robot = robot - self.task_uuid = "INVALID" - self.t_0 = 0 - - self.task_start_response = None - self.task_wait_response = None - self.shared_data = shared_data - - def add_skill(self, name, skill_type, context): - self.skill_names.append(name) - self.skill_types.append(skill_type) - self.skill_context[name] = context - - # def add_skill(self, skill: Skill): - # self.skill_names.append(skill.skill_name) - # self.skill_types.append(skill.skill_type) - # self.skill_context[skill.skill_name] = skill.skill_context - - def start(self, queue: bool = False): - self.t_0 = time.time() - parameters = { - "parameters": { - "skill_names": self.skill_names, - "skill_types": self.skill_types, - "as_queue": queue, - }, - "skills": self.skill_context, - } - # ceased. 13022024 - # print(self.skill_context) - print(f"\033[92mStart the skill: {self.skill_names[0]}\033[0m") - response = start_task(self.robot, "GenericTask", parameters) - self.task_start_response = response - - # ! bbtodo YOU SHOULD CHECK THE RESPONSE HERE - - self.task_uuid = response["result"]["task_uuid"] - return response - - def wait(self): - print(f"\033[92mWait for skill {self.skill_names[0]} to finish\033[0m") - result = wait_for_task(self.robot, self.task_uuid) - print("Task execution took " + str(time.time() - self.t_0) + " s.") - self.task_wait_response = result - return result - - def stop(self): - result = stop_task(self.robot) - print("Task execution took " + str(time.time() - self.t_0) + " s.") - return result - - def interrupt(self): - result = stop_task( - self.robot, raise_exception=False, recover=False, empty_queue=False - ) - print("Current task is interrupted.") - print(str(result)) - return result - - def initialize(self): - self.shared_data["task_start_response"] = None - self.task_start_response = None - self.task_wait_response = None - self.task_uuid = "INVALID" - self.t_0 = 0 - - def clear_skills(self): - self.skill_names = [] - self.skill_types = [] - self.skill_context = dict() diff --git a/kios_bt_planning/kios_utils/test_skeleton_parser.py b/kios_bt_planning/kios_utils/test_skeleton_parser.py deleted file mode 100644 index 74b1dec7..00000000 --- a/kios_bt_planning/kios_utils/test_skeleton_parser.py +++ /dev/null @@ -1,84 +0,0 @@ -# import unittest -# from kios_utils.skeleton_parser import parse_node_name - - -# class TestSkeletonParser(unittest.TestCase): -# def test_parse_node_name(self): -# # Test case 1: Valid input -# name = "BehaviorNode: ActionNode(param1, param2)" -# expected_output = { -# "typename": "BehaviorNode", -# "itemname": "ActionNode", -# "params": ["param1", "param2"], -# } -# self.assertEqual(parse_node_name(name), expected_output) - -# # Test case 2: Empty input -# name = "" -# expected_output = {} -# self.assertEqual(parse_node_name(name), expected_output) - -# # Test case 3: Invalid input -# name = "InvalidName" -# expected_output = {} -# self.assertEqual(parse_node_name(name), expected_output) - -# # Test case 4: Input with missing parameters -# name = "BehaviorNode: ActionNode()" -# expected_output = { -# "typename": "BehaviorNode", -# "itemname": "ActionNode", -# "params": [], -# } -# self.assertEqual(parse_node_name(name), expected_output) - -# # Test case 5: Input with single parameter -# name = "BehaviorNode: ActionNode(param1)" -# expected_output = { -# "typename": "BehaviorNode", -# "itemname": "ActionNode", -# "params": ["param1"], -# } -# self.assertEqual(parse_node_name(name), expected_output) - - -# if __name__ == "__main__": -# unittest.main() - -# def test_parse_node_name(self): -# # Test case 1: Valid input -# name = "BehaviorNode: ActionNode(param1, param2)" -# expected_output = { -# "typename": "BehaviorNode", -# "itemname": "ActionNode", -# "params": ["param1", "param2"], -# } -# self.assertEqual(parse_node_name(name), expected_output) - -# # Test case 2: Empty input -# name = "" -# expected_output = {} -# self.assertEqual(parse_node_name(name), expected_output) - -# # Test case 3: Invalid input -# name = "InvalidName" -# expected_output = {} -# self.assertEqual(parse_node_name(name), expected_output) - -# # Test case 4: Input with missing parameters -# name = "BehaviorNode: ActionNode()" -# expected_output = { -# "typename": "BehaviorNode", -# "itemname": "ActionNode", -# "params": [], -# } -# self.assertEqual(parse_node_name(name), expected_output) - -# # Test case 5: Input with single parameter -# name = "BehaviorNode: ActionNode(param1)" -# expected_output = { -# "typename": "BehaviorNode", -# "itemname": "ActionNode", -# "params": ["param1"], -# } -# self.assertEqual(parse_node_name(name), expected_output) diff --git a/kios_bt_planning/kios_utils/ws_client.py b/kios_bt_planning/kios_utils/ws_client.py deleted file mode 100644 index 784fe0dd..00000000 --- a/kios_bt_planning/kios_utils/ws_client.py +++ /dev/null @@ -1,224 +0,0 @@ -import json -import websockets -import asyncio -import socket -from concurrent.futures import TimeoutError as ConnectionTimeoutError -import websockets.exceptions -import sys -import os - - -mios_communication_root = os.path.dirname(__file__) -if mios_communication_root not in sys.path: - sys.path.insert(0, os.path.abspath(mios_communication_root)) - # print("add path mios_communication_root in the file", __file__) - - -async def send( - hostname, port=12000, endpoint="mios/core", request=None, timeout=100, silent=False -): - """sending msg to the wbesoecket server - - Args: - hostname (_type_): ip of the mios PC - port (int, optional): mios port Defaults to 12000 (left: 12000, right: 13000) - endpoint (str, optional): endpoint. Defaults to "mios/core". - request (_type_, optional): _description_. Defaults to None. - timeout (int, optional): _description_. Defaults to 100. - silent (bool, optional): . Defaults to False. - - Returns: - _type_: reponse or None - """ - uri = "ws://" + hostname + ":" + str(port) + "/" + endpoint - try: - async with websockets.connect(uri, close_timeout=1000) as websocket: - message = json.dumps(request) - await websocket.send(message) - response = await asyncio.wait_for(websocket.recv(), timeout=timeout) - return json.loads(response) - except ConnectionRefusedError as e: - if silent is False: - print("ConnectionRefusedError: ") - print(e) - print( - "Hostname: " - + hostname - + ", port: " - + str(port) - + ", endpoint: " - + endpoint - ) - return None - except ConnectionResetError as e: - if silent is False: - print("ConnectionResetError: ") - print(e) - print( - "Hostname: " - + hostname - + ", port: " - + str(port) - + ", endpoint: " - + endpoint - ) - return None - except ConnectionAbortedError as e: - if silent is False: - print("ConnectionAbortedError: ") - print(e) - print( - "Hostname: " - + hostname - + ", port: " - + str(port) - + ", endpoint: " - + endpoint - ) - return None - except websockets.ConnectionClosedError as e: - if silent is False: - print("ConnectionClosedError: ") - print(e) - print( - "Hostname: " - + hostname - + ", port: " - + str(port) - + ", endpoint: " - + endpoint - ) - return None - except ConnectionTimeoutError as e: - if silent is False: - print("ConnectionTimeoutError: ") - print(e) - print( - "Hostname: " - + hostname - + ", port: " - + str(port) - + ", endpoint: " - + endpoint - ) - return None - except websockets.exceptions.InvalidMessage as e: - if silent is False: - print("InvalidMessage: ") - print(e) - print( - "Hostname: " - + hostname - + ", port: " - + str(port) - + ", endpoint: " - + endpoint - ) - return None - - -def call_server(hostname, port, endpoint, request, timeout): - asyncio.set_event_loop(asyncio.new_event_loop()) - return asyncio.get_event_loop().run_until_complete( - send(hostname, request=request, port=port, endpoint=endpoint, timeout=timeout) - ) - - -def call_method( - hostname: str, - port: int, - method, - payload=None, - endpoint="mios/core", - timeout=100, - silent=False, -): - """sending request to websocket server - - Args: - hostname (str): ip of the mios PC - port (int): mios port Defaults to 12000 (left: 12000, right: 13000) - method (_type_): name of the registed functions in mios - payload (_type_, optional): payload, which contains the details of the tasks - endpoint (str, optional): _description_. Defaults to "mios/core". - timeout (int, optional): _description_. Defaults to 100. - silent (bool, optional): _description_. Defaults to False. - - Returns: - _type_: the return of the registed functions - """ - try: - request = {"method": method, "request": payload} - asyncio.set_event_loop(asyncio.new_event_loop()) - return asyncio.get_event_loop().run_until_complete( - send( - hostname, - request=request, - port=port, - endpoint=endpoint, - timeout=timeout, - silent=silent, - ) - ) - except socket.gaierror as e: - print(e) - print( - "Hostname: " + hostname + ", port:" + str(port) + ", endpoint: " + endpoint - ) - return None - - -def start_task(hostname: str, task: str, parameters={}, queue=True): - """start a task in mios - - Args: - hostname (str): ip of the mios PC - task (str): task name - parameters (dict, optional): _description_. Defaults to {}. - queue (bool, optional): _description_. Defaults to False. - - Returns: - _type_: _description_ - """ - payload = {"task": task, "parameters": parameters, "queue": queue} - return call_method(hostname, 12000, "start_task", payload) - - -def stop_task(hostname: str, raise_exception=False, recover=False, empty_queue=False): - """start the current executed task in mios - - Args: - hostname (str): ip of the mios PC - raise_exception (bool, optional): _description_. Defaults to False. - recover (bool, optional): _description_. Defaults to False. - empty_queue (bool, optional): _description_. Defaults to False. - - Returns: - _type_: _description_ - """ - payload = { - "raise_exception": raise_exception, - "recover": recover, - "empty_queue": empty_queue, - } - return call_method(hostname, 12000, "stop_task", payload) - - -def wait_for_task(hostname: str, task_uuid: str): - """blocking the current proocess until the task finishes - - Args: - hostname (str): ip of the mios PC - task_uuid (str): uuid of the task to be stopped - - Returns: - _type_: _description_ - """ - payload = {"task_uuid": task_uuid} - return call_method(hostname, 12000, "wait_for_task", payload) - - -def start_task_and_wait(hostname, task, parameters, queue=False): - response = start_task(hostname, task, parameters, queue) - response = wait_for_task(hostname, response["result"]["task_uuid"]) - return response diff --git a/kios_bt_planning/kios_world/__init__.py b/kios_bt_planning/kios_world/__init__.py deleted file mode 100644 index e69de29b..00000000 diff --git a/kios_bt_planning/kios_world/data_types.py b/kios_bt_planning/kios_world/data_types.py deleted file mode 100644 index 90f43ba7..00000000 --- a/kios_bt_planning/kios_world/data_types.py +++ /dev/null @@ -1,57 +0,0 @@ -from typing import Set - - -class WorldNode: - def __init__(self, name: str) -> None: - self.name = name - self.properties = set() - - def update_properties(self, prop_set: Set[str]): - self.properties.update(prop_set) - - def add_property(self, prop: str): - self.properties.add(prop) - - def difference_update_properties(self, prop_set: Set[str]): - self.properties.difference_update(prop_set) - - def discard_property(self, prop: str): - self.properties.discard(prop) - - def check_property(self, prop: str) -> bool: - # ! BBDEBUG 11022024 - if prop in self.properties: - return True - else: - return False - - def __repr__(self): - # return f"{self.name}: {', '.join(self.properties)}" - return f"{', '.join(self.properties)}" - - -class Relationship: - """ - directed relationship from source to target - """ - - def __init__(self, source, relation_name, target, isConstraint=False): - self.relation_name = relation_name - self.objects = [source, target] - self.isConstraint = isConstraint - - def __repr__(self): - return f"{self.objects[0]} {self.relation_name} {self.objects[1]}" - - def __eq__(self, other): - if not isinstance(other, Relationship): - print("invalid comparison! not a relationship!") - return False - return ( # * check if the two relationships are the same - self.relation_name == other.relation_name - and self.objects[0] == other.objects[0] - and self.objects[1] == other.objects[1] - ) - - def __hash__(self): - return hash((self.relation_name, self.objects[0], self.objects[1])) diff --git a/kios_bt_planning/kios_world/graph_interface.py b/kios_bt_planning/kios_world/graph_interface.py deleted file mode 100644 index 0f3b80bc..00000000 --- a/kios_bt_planning/kios_world/graph_interface.py +++ /dev/null @@ -1,197 +0,0 @@ -""" -very simple interface to model the objects in the world and the states of them. -""" - -import networkx as nx -from typing import Set, Dict, List, Any - -from kios_world.neo4j_interface import Neo4jInterface -from kios_world.data_types import WorldNode, Relationship - -""" -this is the current state holder of the world -""" - - -class GraphInterface: - def __init__(self): - self.nodes: Dict[str, WorldNode] = {} - self.relations: Set[Relationship] = set() - self.graph = nx.DiGraph() - - self.neo4j = Neo4jInterface() - self.neo4j.close_driver() - - def add_node(self, node_name: str, raise_error=False): - """ - add a node to the graph. - if the node already exists, do nothing. - """ - if node_name in self.nodes: - return # do nothing if node already exists - self.nodes[node_name] = WorldNode(node_name) - - def add_properties(self, node_name: str, props: List[str]): - """ - add a list of properties to a node. - if the node does not exist, create it. - if the property already exists, do nothing. - """ - self.add_node(node_name) # create node if it does not exist - # * list -> set - prop_set = set(props) - self.nodes[node_name].update_properties(prop_set) - - def add_property(self, node_name: str, prop: str): - self.add_node(node_name) - self.nodes[node_name].add_property(prop) - - # * update the property status - def update_property(self, node_name: str, prop: str, status: bool): - self.add_node(node_name) - if status: - self.nodes[node_name].add_property(prop) - else: - self.nodes[node_name].discard_property(prop) - - def remove_property(self, node_name: str, prop: str): - if node_name not in self.nodes: - self.add_node(node_name) - return # do nothing if node does not exist - self.nodes[node_name].discard_property(prop) - - def remove_properties(self, node_name: str, props: List[str]): - if node_name not in self.nodes: - self.add_node(node_name) # create node if it does not exist - return # do nothing if node does not exist - # * list -> set - prop_set = set(props) - self.nodes[node_name].difference_update_properties(prop_set) - - def get_node(self, node_name: str) -> WorldNode: - if node_name not in self.nodes: - return None # do nothing if node does not exist - return self.nodes[node_name] - - def update_relation(self, source: str, name: str, target: str, status: bool): - if status: - self.add_relation(source, name, target) - else: - self.remove_relation(source, name, target) - - def add_relation(self, source: str, name: str, target: str, isConstraint=False): - """ - add a relation between two objects. - if the objects do not exist, create them. - if the relation already exists, do nothing. - """ - source_node = self.get_node(source) - target_node = self.get_node(target) - # ! think twice. should the inconsistency be handled here? - if source_node is None: - # raise ValueError(f"Object {source} does not exist in the database!") - self.add_node(source) - if target_node is None: - # raise ValueError(f"Object {target} does not exist in the database!") - self.add_node(target) - rel = Relationship(source, name, target, isConstraint=isConstraint) - self.relations.add(rel) - - def remove_relation(self, source: str, name: str, target: str): - rel = Relationship(source, name, target) - self.relations.discard(rel) - - # for condition node checking - def check_property(self, node_name: str, prop: str, status: bool = True) -> bool: - if node_name not in self.nodes: - return not status - return self.nodes[node_name].check_property(prop) == status - - def check_relation( - self, source: str, name: str, target: str, status: bool = True - ) -> bool: - rel = Relationship(source, name, target) - return (rel in self.relations) == status - - def discard_relation(self, source: str, name: str, target: str): - rel = Relationship(source, name, target) - self.relations.discard(rel) - - def clear(self): - self.nodes = {} - self.relations = set() - # self.graph = nx.DiGraph() - - def from_json(self, json_data: Dict[str, Any]): - # * add nodes - for node in json_data["objects"]: - self.add_node(node["name"]) - self.add_properties(node["name"], node["properties"]) - - if json_data.get("constraints") is None: - print("\033[93mWarning: No constraint found in the JSON data.\033[0m") - else: - # * add constraints (unchangeable relations) - for constraint in json_data["constraints"]: - self.add_relation( - constraint["source"], - constraint["name"], - constraint["target"], - isConstraint=True, - ) - - # * add relations - if json_data.get("relations") is None: - print("\033[93mWarning: No relations found in the JSON data.\033[0m") - else: - for relation in json_data["relations"]: - self.add_relation( - relation["source"], relation["name"], relation["target"] - ) - - def to_json(self) -> Dict[str, Any]: - json_data = {"objects": [], "constraints": [], "relations": []} - - # * add nodes - for node_name, node in self.nodes.items(): - node_data = { - "name": node_name, - "properties": list(node.properties), - } # ! BBFIX 20022024 - json_data["objects"].append(node_data) - - # * add constraints and relations (unchangeable relations) - for rel in self.relations: - if rel.isConstraint: - constraint_data = { - "source": rel.objects[0], - "name": rel.relation_name, - "target": rel.objects[1], - } - json_data["constraints"].append(constraint_data) - else: - relation_data = { - "source": rel.objects[0], - "name": rel.relation_name, - "target": rel.objects[1], - } - json_data["relations"].append(relation_data) - - return json_data - - def refresh_neo4j(self): - self.neo4j.open_driver() - self.neo4j.clear_database() - self.neo4j.create_objects(list(self.nodes.keys())) - self.neo4j.set_properties(self.nodes) - self.neo4j.create_relations(self.relations) - self.neo4j.close_driver() - - def refresh_networkx(self): - self.graph.clear() - for node_name, node in self.nodes.items(): - self.graph.add_node(node_name, properties=node.properties) - for rel in self.relations: - self.graph.add_edge( - rel.objects[0], rel.objects[1], relation=rel.relation_name - ) diff --git a/kios_bt_planning/kios_world/neo4j_interface.py b/kios_bt_planning/kios_world/neo4j_interface.py deleted file mode 100644 index 6471bf2b..00000000 --- a/kios_bt_planning/kios_world/neo4j_interface.py +++ /dev/null @@ -1,174 +0,0 @@ -from neo4j import GraphDatabase - -from typing import List, Dict, Any, Set -from kios_world.data_types import WorldNode, Relationship - -""" -currently this module is just for visulization. -""" - - -class Neo4jInterface: - """ - Interface for the Neo4j database - """ - - def __init__( - self, - uri: str = "neo4j://localhost:7687", - user: str = "neo4j", - password: str = "14637982", - ): - self.uri = uri - self.user = user - self.password = password - self.driver = GraphDatabase.driver(uri, auth=(user, password)) - - def _clear_database(self, tx): - tx.run("MATCH (n) DETACH DELETE n") - - def clear_database(self): - with self.driver.session() as session: - session.write_transaction(self._clear_database) - - def open_driver(self): - self.driver = GraphDatabase.driver(self.uri, auth=(self.user, self.password)) - - def close_driver(self): - self.driver.close() - - def _create_objects(self, tx, object_names: list): - for obj in object_names: - tx.run(f"CREATE (o:Object {{name: '{obj}'}})") - - def create_objects(self, object_names: list): - with self.driver.session() as session: - session.write_transaction(self._create_objects, object_names) - - def _set_properties(self, tx, objects_and_properties: Dict[str, Set[str]]): - for obj, properties in objects_and_properties.items(): - for prop in properties: - tx.run(f"MATCH (o:Object {{name: '{obj}'}}) SET o.{prop} = ''") - # tx.run(f"MATCH (o:Object {{name: '{obj}'}}) SET o.{prop} = ''") - - def set_properties(self, world_node_dict: Dict[str, WorldNode]): - with self.driver.session() as session: - objects_and_properties = {} - for obj, world_node in world_node_dict.items(): - objects_and_properties[obj] = world_node.properties - session.write_transaction(self._set_properties, objects_and_properties) - - def _create_relations(self, tx, relations: Dict[str, List[str]]): - """ - create relations between objects, if objects do not exist, create them. - """ - for relation, objects in relations.items(): - for obj in objects: - result_from = tx.run(f"MATCH (o:Object {{name: '{obj[0]}'}}) RETURN o") - result_to = tx.run(f"MATCH (o:Object {{name: '{obj[1]}'}}) RETURN o") - if result_from.single() and result_to.single(): - tx.run( - f"MATCH (from:Object {{name: '{obj[0]}'}}), (to:Object {{name: '{obj[1]}'}}) " - f"CREATE (from)-[:{relation.upper()}]->(to)" - ) - else: - # raise ValueError( - # f"Object {obj} does not exist in the database!" - # ) - if not result_from.single(): - self.create_objects([obj[0]]) - if not result_to.single(): - self.create_objects([obj[1]]) - tx.run( - f"MATCH (from:Object {{name: '{obj[0]}'}}), (to:Object {{name: '{obj[1]}'}}) " - f"CREATE (from)-[:{relation.upper()}]->(to)" - ) - - def create_relations(self, relation_set: Set[Relationship]): - with self.driver.session() as session: - relations = {} - for relation in relation_set: - if relation.relation_name not in relations: - relations[relation.relation_name] = [] - relations[relation.relation_name].append(relation.objects) - session.write_transaction(self._create_relations, relations) - - def fetch_all(self): - with self.driver.session() as session: - result = session.run("MATCH (from)-[r]->(to) RETURN from.name, r, to.name") - return [ - (record["from.name"], record["r"].type, record["to.name"]) - for record in result - ] - - def fetch_nodes(self): - with self.driver.session() as session: - result = session.run("MATCH (o:Object) RETURN o.name") - return [record["o.name"] for record in result] - - def fetch_relations(self): - with self.driver.session() as session: - result = session.run("MATCH (from)-[r]->(to) RETURN from.name, r, to.name") - return [ - (record["from.name"], record["r"].type, record["to.name"]) - for record in result - ] - - def fetch_specific(self, node_names: List[str]): - """ - fetch specific nodes and relations from the database - """ - with self.driver.session() as session: - result = session.run( - f"MATCH (from)-[r]->(to) WHERE from.name IN {node_names} AND to.name IN {node_names} RETURN from.name, r, to.name" - ) - return [ - (record["from.name"], record["r"].type, record["to.name"]) - for record in result - ] - - def remove_nodes(self, node_names: List[str]): - """ - remove specific nodes from the database - """ - with self.driver.session() as session: - for node_name in node_names: - session.run(f"MATCH (o:Object {{name: '{node_name}'}}) DETACH DELETE o") - - def remove_relations(self, relations: Dict[str, List[str]]): - """ - remove specific relations from the database - """ - with self.driver.session() as session: - for relation, objects in relations.items(): - for obj in objects: - session.run( - f"MATCH (from:Object {{name: '{obj[0]}'}})-[r:{relation.upper()}]->(to:Object {{name: '{obj[1]}'}}) DELETE r" - ) - - def print_all(self): - with self.driver.session() as session: - result = session.run("MATCH (from)-[r]->(to) RETURN from, r, to") - for record in result: - print( - f"{record['from']['name']} - {record['r'].type} - {record['to']['name']}" - ) - - -def test_neo4j(): - neo4j_interface = Neo4jInterface( - uri="neo4j://localhost:7687", user="neo4j", password="14637982" - ) - neo4j_interface.clear_database() - neo4j_interface.create_objects(["a", "b", "c", "d", "e", "f"]) - relations = { - "on": [("a", "b"), ("b", "c"), ("c", "d"), ("d", "e"), ("e", "f")], - "next_to": [("a", "c"), ("c", "e"), ("b", "d"), ("d", "f")], - } - neo4j_interface.create_relations(relations) - neo4j_interface.print_all() - neo4j_interface.close_driver() - - -if __name__ == "__main__": - test_neo4j() diff --git a/kios_bt_planning/kios_world/world_interface.py b/kios_bt_planning/kios_world/world_interface.py deleted file mode 100644 index 6667fd34..00000000 --- a/kios_bt_planning/kios_world/world_interface.py +++ /dev/null @@ -1,176 +0,0 @@ -from typing import Any, Dict -import py_trees -from kios_bt.data_types import ( - # GroundedAction, - # GroundedCondition, - Action, - Condition, - ObjectProperty, -) - -from kios_world.neo4j_interface import Neo4jInterface -from kios_world.graph_interface import GraphInterface - - -class WorldInterface: - neo4j: Neo4jInterface = None - blackboard: py_trees.blackboard.Client = None - graph_interface: GraphInterface = None - - check_point: Dict[str, dict] = {} - - def __init__(self, graph_interface=True, blackboard=False, neo4j=False) -> None: - if graph_interface: - self.graph_interface = GraphInterface() - - if neo4j: - self.neo4j = Neo4jInterface() - - if blackboard: - self.blackboard = py_trees.blackboard.Client(name="WorldInterface") - - def initialize(self): - pass - - def record_check_point(self, check_point_name: str = None): - """ - record the current world state - """ - if check_point_name is None: - check_point_name = "default" - self.check_point[check_point_name] = self.graph_interface.to_json() - - def restore_check_point(self, check_point_name: str = None): - """ - restore the world state to the check point - """ - if check_point_name is None: - check_point_name = "default" - self.graph_interface.from_json(self.check_point[check_point_name]) - - def load_world_from_json(self, json_data: dict): - """ - load the world from a json file, but you need to parse it first - """ - self.graph_interface.from_json(json_data) - - def clear_world(self): - """ - clear the world - """ - self.graph_interface.clear() - - def update_world(self, world: dict[str, list[dict[str, Any]]]): - """ - update the world state, add new, do not remove any. - """ - object_list = world.get("objects") - constraint_list = world.get("constraints") - relation_list = world.get("relations") - for item in object_list: - # self.graph_interface.add_node(item.get("name")) - self.graph_interface.add_properties( - item.get("name"), item.get("properties") - ) - - # ! BBFIX 20022024 - for item in constraint_list: - self.graph_interface.add_relation( - source=item.get("source"), - target=item.get("target"), - name=item.get("name"), - isConstraint=True, - ) - - for item in relation_list: - self.graph_interface.add_relation( - source=item.get("source"), - target=item.get("target"), - name=item.get("name"), - isConstraint=False, - ) - - def get_world_to_json(self) -> dict: - """ - return the world state in json dict - """ - return self.graph_interface.to_json() - - # for action node - def take_effect(self, action: Action): - # to add/update - to_update_list = action.effects - for item in to_update_list: - if item.property_value is None: - # this is a property - self.graph_interface.update_property( - item.object_name, item.property_name, item.status - ) - else: - # this is a relation - self.graph_interface.update_relation( - item.object_name, - item.property_name, - item.property_value, - item.status, - ) - - # def register_predicates(self, predicates: dict) -> None: # ! discard - # """ - # add this predicate to the world. do nothing if the predicate already exists - # """ - # if predicates is not None: - # for key, _ in predicates.items(): - # self.blackboard.register_key( - # key=key, access=py_trees.common.Access.WRITE - # ) - - def get_object(self, object_name: str) -> Any: - raise NotImplementedError - - def get_objects(self, object_name: str) -> Any: - raise NotImplementedError - - def set_predicates(self, predicates: dict) -> None: - """ - "take effects" - """ - if predicates is not None: - try: - for key, value in predicates.items(): - self.blackboard.set(name=key, value=value) - except KeyError: - print("KeyError: %s triggered this problem!" % (key)) - - def get_predicates(self) -> dict: - raise NotImplementedError - - def query_state(self, query: dict) -> bool: - raise NotImplementedError - - # for condition node - def check_condition(self, condition: Condition) -> bool: - condition_list = condition.conditions - for item in condition_list: - if not self.check_property(item): - return False - - return True - - def check_property(self, prop: ObjectProperty) -> bool: - # ! BBDEBUG 11022024 - """ - check if the property is in the same status as specified in the property object - """ - if prop.property_value is None: - # this is a property - return self.graph_interface.check_property( - prop.object_name, prop.property_name, prop.status - ) - else: - # this is a relation - return self.graph_interface.check_relation( - prop.object_name, prop.property_name, prop.property_value, prop.status - ) - - return False diff --git a/kios_bt_planning/setup.py b/kios_bt_planning/setup.py deleted file mode 100644 index 1d56135f..00000000 --- a/kios_bt_planning/setup.py +++ /dev/null @@ -1,16 +0,0 @@ -from setuptools import setup, find_packages - -core_requirements = [ - "py-trees", -] - -setup( - name="kios_bt_planning", - author="blackbird", - description="Behavior Tree Planning for KIOS", - version="0.1.0", - packages=find_packages(), - # include_package_data=True, - # python_requires=">3.7,<3.9", - install_requires=core_requirements, -) diff --git a/kios_bt_planning/tests/__init__.py b/kios_bt_planning/tests/__init__.py deleted file mode 100644 index e69de29b..00000000 diff --git a/kios_bt_planning/tests/bt_generation_test.py b/kios_bt_planning/tests/bt_generation_test.py deleted file mode 100644 index 6cb5bdb0..00000000 --- a/kios_bt_planning/tests/bt_generation_test.py +++ /dev/null @@ -1,270 +0,0 @@ -from kios_bt.data_types import Action, Condition, ObjectProperty, ControlFlow - -from kios_bt.behavior_nodes import ActionNode, ConditionNode, ActionNodeTest - -from kios_world.world_interface import WorldInterface - -from kios_bt.bt_factory import BehaviorTreeFactory - -from kios_bt.bt_stewardship import BehaviorTreeStewardship - -from kios_utils.pybt_test import ( - generate_bt_stewardship, - tick_once_test, - render_dot_tree, - tick_loop_test, -) - -from kios_agent.kios_llm import KiosLLM - -# from kios_agent.llm_supporter import KiosLLM # ! TESTING NOW - -from typing import List, Dict, Any - -import json -import os - -import py_trees - - -def test_bt(bt_json: json): - test_class = BehaviorTreeFactory() - bt = test_class.from_json_to_simple_bt(bt_json) - # bt = test_class.from_json_to_tree_root(bt_json) - bt_stewardship = generate_bt_stewardship(bt) - # bt_stewardship.setup(timeout=15) - render_dot_tree(bt_stewardship) - - # tick_loop_test(bt_stewardship) - - -def test_skeleton_generation(bt_json: json): - test_class = BehaviorTreeFactory() - bt = test_class.from_skeleton_to_tree_root(bt_json) - bt_stewardship = generate_bt_stewardship(bt) - # bt_stewardship.setup(timeout=15) - render_dot_tree(bt_stewardship) - - # tick_loop_test(bt_stewardship) - - -def visualize_bt(bt_json: json): - test_class = BehaviorTreeFactory() - bt = test_class.from_json_to_simple_bt(bt_json) - bt_stewardship = generate_bt_stewardship(bt) - # bt_stewardship.setup(timeout=15) - render_dot_tree(bt_stewardship) - - # tick_loop_test(bt_stewardship) - - -def main(): - py_trees.logging.level = py_trees.logging.Level.DEBUG - - # problem_name = "gearset_problem1_skeleton_v1" - # problem = "(define (problem robot_assembly_problem-problem)\ - # (:domain robot_assembly_problem-domain)\ - # (:objects\ - # parallel_box1 parallel_box2 inward_claw outward_claw no_tool - tool\ - # gear1 gear2 gear3 shaft1 shaft2 base - part\ - # left_hand - hand\ - # )\ - # (:init (can_manipulate parallel_box1 gear1) (can_manipulate outward_claw gear2) (can_manipulate inward_claw gear3) (can_manipulate parallel_box2 shaft1) (can_manipulate no_tool shaft2) (can_screw_to leg1 seat) (can_screw_to leg2 seat) (can_insert_to back seat) (can_screw_to nut1 seat) (can_screw_to nut2 seat) (can_screw_to blub base) (can_place_to lamp blub) (can_insert_to shaft1 base) (can_insert_to shaft2 base) (can_insert_to gear3 shaft2) (can_insert_to gear2 base) (can_insert_to gear1 shaft1) (is_inserted_to shaft1 base) (is_free left_hand) (is_free parallel_box1) (is_free parallel_box2) (is_free inward_claw) (is_free outward_claw) (is_free no_tool) (is_free gear1) (is_free gear2) (is_free gear3) (is_free shaft1) (is_free shaft2) (is_free base) (is_equippable parallel_box1) (is_equippable parallel_box2) (is_equippable inward_claw) (is_equippable outward_claw) (is_equippable no_tool))\ - # (:goal (and (is_inserted_to gear1 shaft1)))\ - # )" # ! CHEAT - - # problem_name = "gearset2" # ! updated default initial states - # problem = "(define (problem robot_assembly_problem-problem)\ - # (:domain robot_assembly_problem-domain)\ - # (:objects\ - # parallel_box1 parallel_box2 inward_claw outward_claw no_tool - tool\ - # gear1 gear2 gear3 shaft1 shaft2 base - part\ - # left_hand - hand\ - # )\ - # (:init (can_manipulate parallel_box1 gear1) (can_manipulate outward_claw gear2) (can_manipulate inward_claw gear3) (can_manipulate parallel_box2 shaft1) (can_manipulate no_tool shaft2) (can_screw_to leg1 seat) (can_screw_to leg2 seat) (can_insert_to back seat) (can_screw_to nut1 seat) (can_screw_to nut2 seat) (can_screw_to blub base) (can_place_to lamp blub) (can_insert_to shaft1 base) (can_insert_to shaft2 base) (can_insert_to gear3 shaft2) (can_insert_to gear2 base) (can_insert_to gear1 shaft1) (is_inserted_to shaft2 base) (is_free left_hand) (is_free parallel_box1) (is_free parallel_box2) (is_free inward_claw) (is_free outward_claw) (is_free no_tool) (is_equippable parallel_box1) (is_equippable parallel_box2) (is_equippable inward_claw) (is_equippable outward_claw) (is_equippable no_tool))\ - # (:goal (and (is_inserted_to gear3 shaft2)))\ - # )\ - # " - - # problem_name = "gearset3" # ! updated default initial states - # problem = "(define (problem robot_assembly_problem-problem)\ - # (:domain robot_assembly_problem-domain)\ - # (:objects\ - # parallel_box1 parallel_box2 inward_claw outward_claw no_tool - tool\ - # gear1 gear2 gear3 shaft1 shaft2 base - part\ - # left_hand - hand\ - # )\ - # (:init (can_manipulate parallel_box1 gear1) (can_manipulate outward_claw gear2) (can_manipulate inward_claw gear3) (can_manipulate parallel_box2 shaft1) (can_manipulate no_tool shaft2) (can_screw_to leg1 seat) (can_screw_to leg2 seat) (can_insert_to back seat) (can_screw_to nut1 seat) (can_screw_to nut2 seat) (can_screw_to blub base) (can_place_to lamp blub) (can_insert_to shaft1 base) (can_insert_to shaft2 base) (can_insert_to gear3 shaft2) (can_insert_to gear2 base) (can_insert_to gear1 shaft1) (is_free left_hand) (is_free parallel_box1) (is_free parallel_box2) (is_free inward_claw) (is_free outward_claw) (is_equippable parallel_box1) (is_equippable parallel_box2) (is_equippable inward_claw) (is_equippable outward_claw) (is_inserted_to shaft2 base))\ - # (:goal (and (is_inserted_to gear3 shaft2)))\ - # )" - - # problem_name = "gearset1" # ! updated domain - # problem = '(define (problem robot_assembly_problem-problem)\ - # (:domain robot_assembly_problem-domain)\ - # (:objects\ - # parallel_box1 parallel_box2 inward_claw outward_claw no_tool - tool\ - # gear1 gear2 gear3 shaft1 shaft2 shaft3 gearbase gearbase_hole1 gearbase_hole3 - part\ - # left_hand - hand\ - # )\ - # (:init (can_manipulate parallel_box2 gear1) (can_manipulate outward_claw gear2) (can_manipulate outward_claw gear3) (can_manipulate no_tool shaft3) (can_manipulate parallel_box1 shaft1) (can_screw_to leg1 seat) (can_screw_to leg2 seat) (can_insert_to back seat) (can_screw_to nut1 seat) (can_screw_to nut2 seat) (can_screw_to blub lampbase) (can_place_to lamp blub) (can_insert_to shaft1 gearbase_hole1) (can_insert_to shaft3 gearbase_hole3) (can_insert_to gear3 shaft3) (can_insert_to gear2 shaft2) (can_insert_to gear1 shaft1) (is_free left_hand) (is_free parallel_box1) (is_free parallel_box2) (is_free inward_claw) (is_free outward_claw) (is_free no_tool) (is_equippable parallel_box1) (is_equippable parallel_box2) (is_equippable inward_claw) (is_equippable outward_claw) (is_equippable no_tool))\ - # (:goal (and (is_inserted_to shaft1 gearbase_hole1)))\ - # )\ - # ' - - # * refine problem - problem_name = "gearset1_sk" # super skeleton - file_dir = os.path.dirname(os.path.abspath(__file__)) - problem_dir = os.path.join(file_dir, "cot_sk.txt") - with open(problem_dir, "r") as f: - problem = f.read() - - ### * end_to_end - # feature = "e2e" - # model = "gpt-4-1106-preview" - # ver = "v2" - - # prompt_dir = "prompts/end_to_end_v2" - # prompt_load_order = [ - # "e2e_role", # your are a good interpreter - # "e2e_output_format", # how to generate the output - # "e2e_domain", # domain knowledge - # "e2e_problem", # the problem format - # "e2e_state", # hot to describe the state - # "e2e_bt", # how to build tree - # "e2e_chain", # COT - # "e2e_example", # some skeleton examples ... Done - # ] - ### * - - ### * refine_sk - feature = "re_sk" - model = "gpt-4-1106-preview" - # model = "gpt-3.5-turbo-16k-0613" - ver = "v0" - - prompt_dir = "prompts/sk_refine" - prompt_load_order = [ - "refine_role", # your are a good interpreter - "refine_input_format", # about the input - "refine_state", # about the state - # "refine_domain", # domain knowledge - "refine_help", # tips about how to refine the nodes - "refine_controlflow", # how to refine controlflow nodes - "refine_condition", # how to refine condition nodes - "refine_action", # how to refine action nodes - "refine_output_format", # the output format - # "refine_chain", # COT - ] - ### * - - ### *skeleton - # feature = "skeleton" - # model = "gpt-4-1106-preview" - # ver = "v2" - # prompt_dir = "prompts/skeleton" - # prompt_load_order = [ - # "prompt_role", # your are a good interpreter ... Done - # "prompt_domain", # domain knowledge ... Done - # "prompt_problem", # how the problem is provided ... Done - # "prompt_environment", # how to express the environment ... Done - # # "prompt_behaviortree", # how to construct the behavior tree ... Done - # "prompt_bt_skeleton", # how to construct the skeleton behavior tree ... Done - # "prompt_bt_skeleton_example", # some skeleton examples ... Done - # "prompt_output_format", # the output format ... Done - # # "prompt_example", # some examples ... Done - # ] - ### * - - ### *COT spsk - # feature = "cot_ap_skeleton" - # model = "gpt-4-1106-preview" - # ver = "v1" - # """ - # add chain, try to solve state inconsistency by applying COT - # """ - # prompt_dir = "prompts/cot_sp_skeleton" - # prompt_load_order = [ - # "cot_role", # your are a good interpreter - # "cot_output_format", # how to generate the output - # "cot_domain", # domain knowledge - # "cot_problem", # the problem format - # "cot_state", # hot to describe the state - # "cot_bt", # how to build tree - # "cot_chain", # COT - # "cot_example", # some skeleton examples ... Done - # ] - ### * - - ## *COT skeleton - # feature = "cot_skeleton" - # model = "gpt-4-1106-preview" - # ver = "v0" - # """ - # add chain, try to solve state inconsistency by applying COT - # use skeleton - # """ - # prompt_dir = "prompts/cot_skeleton" - # prompt_load_order = [ - # "cot_sk_role", # your are a good interpreter - # "cot_sk_output_format", # how to generate the output - # "cot_sk_domain", # domain knowledge - # "cot_sk_problem", # the problem format - # "cot_sk_state", # hot to describe the state - # "cot_sk_bt", # how to build tree - # "cot_sk_chain", # COT - # "cot_sk_example", # some skeleton examples ... Done - # ] - ## * - - llm_model = KiosLLM() - - llm_model.initialize( - prompt_dir=prompt_dir, - prompt_load_order=prompt_load_order, - ) - - # * model = "gpt-3.5-turbo-16k-0613" - # * model="gpt-4-0613", # not this - # * model="gpt-4-1106-preview", - - full_problem_name = f"{problem_name}_{feature}_{ver}_{model}" - - response = llm_model.query_llm( - problem=problem, - problem_name=full_problem_name, - model=model, - ) - - bt = response["task_plan"]["behavior_tree"] - - test_bt(bt) - - -current_dir = os.path.dirname(os.path.abspath(__file__)) - -with open(os.path.join(current_dir, "gearset3_cot_sk.txt"), "r") as f: - sk_bt = json.load(f) - -bt_skeleton = sk_bt["task_plan"]["behavior_tree"] - - -if __name__ == "__main__": - # main() - - # test_bt(result_bt_json) - # test_bt(bt_skeleton) - # visualize_bt(bt_skeleton) - - test_skeleton_generation(bt_skeleton) - - -# ! should the is_free has a default true? or false? -# ! can I explain the knowledge differently from the pddl domain? - -# ! BUG in example: is_free is still default to be true!!! -# ! BUG in example: env after ist falsch! -# ! BUG in example: the boolean value is T F instead of t f. also should be null not None -# ! The action name "action: xxx(xxx, xxx)" is not consistent with the format requirement in action parser! -# ! BUG in output_format: four instead of three! - -# ! SYSTEMATIC BUG in problem: the example problem still applies the is free to the parts!!!!!!!! diff --git a/kios_bt_planning/tests/bt_json_io_test.py b/kios_bt_planning/tests/bt_json_io_test.py deleted file mode 100644 index 221c990d..00000000 --- a/kios_bt_planning/tests/bt_json_io_test.py +++ /dev/null @@ -1,1447 +0,0 @@ -from kios_bt.data_types import Action, Condition, ObjectProperty, ControlFlow - -from kios_bt.behavior_nodes import ActionNode, ConditionNode, ActionNodeTest - -from kios_world.world_interface import WorldInterface - -from kios_bt.bt_factory import BehaviorTreeFactory - -from kios_bt.bt_stewardship import BehaviorTreeStewardship - -from kios_utils.pybt_test import ( - generate_bt_stewardship, - tick_once_test, - render_dot_tree, - tick_loop_test, -) - -from typing import List, Dict, Any - -import json - -import py_trees - - -def test_simple_bt(): - bt_json = { - "name": "Pick Up Apple", - "identifier": 0, - "type_name": "selector", - "children": [ - { - "summary": "check if the apple is in hand", - "name": "check apple in hand", - "identifier": 1, - "type_name": "condition", - "conditions": [ - { - "object_name": "apple", - "property_name": "in", - "property_value": "hand", - "status": True, - } - ], - }, - { - "name": "Pick Up Sequence", - "identifier": 2, - "type_name": "sequence", - "children": [ - { - "summary": "check if the apple is on the ground", - "name": "check apple on the ground", - "identifier": 3, - "type_name": "condition", - "conditions": [ - { - "object_name": "apple", - "property_name": "on_the_ground", - "property_value": None, - "status": True, - } - ], - }, - { - "summary": "check if the hand is free", - "name": "check hand free", - "identifier": 4, - "type_name": "condition", - "conditions": [ - { - "object_name": "hand", - "property_name": "free", - "property_value": None, - "status": True, - } - ], - }, - { - "summary": "pick up the apple", - "name": "pick_up", - "identifier": 5, - "type_name": "action", - "effects": [ - { - "object_name": "apple", - "property_name": "on_the_ground", - "property_value": None, - "status": False, - }, - { - "object_name": "apple", - "property_name": "in", - "property_value": "hand", - "status": True, - }, - { - "object_name": "hand", - "property_name": "free", - "property_value": None, - "status": False, - }, - ], - }, - ], - }, - ], - } - - # be_stewardship = BehaviorTreeStewardship() - - test_class = BehaviorTreeFactory() - bt = test_class.from_json_to_bt(bt_json) - bt_stewardship = generate_bt_stewardship(bt) - bt_stewardship.setup(timeout=15) - render_dot_tree(bt_stewardship) - - tick_loop_test(bt_stewardship) - - -example_json_data = { - "name": "load_tool selector", - "identifier": 0, - "type_name": "selector", - "children": [ - { - "summary": "check if parallel_box1 is hold", - "name": "check left_hand hold parallel_box1", - "identifier": 1, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "parallel_box1", - "status": True, - } - ], - }, - { - "name": "load_tool Sequence", - "identifier": 2, - "type_name": "sequence", - "children": [ - { - "summary": "check if parallel_box1 is equippable", - "name": "check parallel_box1 is_equippable", - "identifier": 3, - "type_name": "condition", - "conditions": [ - { - "object_name": "parallel_box1", - "property_name": "is_equippable", - "property_value": None, - "status": True, - } - ], - }, - { - "summary": "check if left hand is free", - "name": "check left_hand is_free", - "identifier": 4, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "is_free", - "property_value": None, - "status": True, - } - ], - }, - { - "summary": "equip parallel_box1 to left hand", - "name": "load_tool(left_hand, parallel_box1)", - "identifier": 5, - "type_name": "action", - "effects": [ - { - "object_name": "hand", - "property_name": "is_free", - "property_value": None, - "status": False, - }, - { - "object_name": "parallel_box1", - "property_name": "is_equippable", - "property_value": None, - "status": False, - }, - { - "object_name": "hand", - "property_name": "hold", - "property_value": "parallel_box1", - "status": True, - }, - ], - }, - ], - }, - ], -} - -example_json_data = { - "name": "pick_up selector", - "identifier": 6, - "type_name": "selector", - "children": [ - { - "summary": "check if parallel_box1 holds gear1", - "name": "check parallel_box1 hold gear1", - "identifier": 7, - "type_name": "condition", - "conditions": [ - { - "object_name": "parallel_box1", - "property_name": "hold", - "property_value": "gear1", - "status": True, - } - ], - }, - { - "name": "pick_up Sequence", - "identifier": 8, - "type_name": "sequence", - "children": [ - { - "summary": "check if parallel_box1 is free", - "name": "check parallel_box1 is_free", - "identifier": 9, - "type_name": "condition", - "conditions": [ - { - "object_name": "parallel_box1", - "property_name": "is_free", - "property_value": None, - "status": True, - } - ], - }, - { - "summary": "check if left hand holds parallel_box1", - "name": "check left_hand hold parallel_box1", - "identifier": 10, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "parallel_box1", - "status": True, - } - ], - }, - { - "summary": "check if parallel_box1 can manipulate gear1", - "name": "check left_hand hold parallel_box1", - "identifier": 11, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "parallel_box1", - "status": True, - } - ], - }, - { - "summary": "pick up gear1 using parallel_box1", - "name": "pick_up(left_hand, parallel_box1, gear1)", - "identifier": 12, - "type_name": "action", - "effects": [ - { - "object_name": "parallel_box1", - "property_name": "hold", - "property_value": "gear1", - "status": True, - }, - { - "object_name": "parallel_box1", - "property_name": "is_free", - "property_value": None, - "status": False, - }, - ], - }, - ], - }, - ], -} - -this_example_json_data = { - "name": "insert selector", - "identifier": 13, - "type_name": "selector", - "children": [ - { - "summary": "check if gear1 is inserted to shaft1", - "name": "check gear1 is_inserted_to shaft1", - "identifier": 14, - "type_name": "condition", - "conditions": [ - { - "object_name": "gear1", - "property_name": "is_inserted_to", - "property_value": "shaft1", - "status": True, - } - ], - }, - { - "name": "insert sequence", - "identifier": 15, - "type_name": "sequence", - "children": [ - { - "summary": "check if left hand holds parallel_box1", - "name": "check left_hand hold parallel_box1", - "identifier": 16, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "parallel_box1", - "status": True, - } - ], - }, - ################ - { - "name": "pick_up selector", - "identifier": 6, - "type_name": "selector", - "children": [ - { - "summary": "check if parallel_box1 holds gear1", - "name": "check parallel_box1 hold gear1", - "identifier": 7, - "type_name": "condition", - "conditions": [ - { - "object_name": "parallel_box1", - "property_name": "hold", - "property_value": "gear1", - "status": True, - } - ], - }, - { - "name": "pick_up Sequence", - "identifier": 8, - "type_name": "sequence", - "children": [ - { - "summary": "check if parallel_box1 is free", - "name": "check parallel_box1 is_free", - "identifier": 9, - "type_name": "condition", - "conditions": [ - { - "object_name": "parallel_box1", - "property_name": "is_free", - "property_value": None, - "status": True, - } - ], - }, - ################ - { - "name": "load_tool selector", - "identifier": 0, - "type_name": "selector", - "children": [ - { - "summary": "check if parallel_box1 is hold", - "name": "check left_hand hold parallel_box1", - "identifier": 1, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "parallel_box1", - "status": True, - } - ], - }, - { - "name": "load_tool Sequence", - "identifier": 2, - "type_name": "sequence", - "children": [ - { - "summary": "check if parallel_box1 is equippable", - "name": "check parallel_box1 is_equippable", - "identifier": 3, - "type_name": "condition", - "conditions": [ - { - "object_name": "parallel_box1", - "property_name": "is_equippable", - "property_value": None, - "status": True, - } - ], - }, - { - "summary": "check if left hand is free", - "name": "check left_hand is_free", - "identifier": 4, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "is_free", - "property_value": None, - "status": True, - } - ], - }, - { - "summary": "equip parallel_box1 to left hand", - "name": "load_tool(left_hand, parallel_box1)", - "identifier": 5, - "type_name": "action", - "effects": [ - { - "object_name": "left_hand", - "property_name": "is_free", - "property_value": None, - "status": False, - }, - { - "object_name": "parallel_box1", - "property_name": "is_equippable", - "property_value": None, - "status": False, - }, - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "parallel_box1", - "status": True, - }, - ], - }, - ], - }, - ], - }, - ################ - { - "summary": "check if parallel_box1 can manipulate gear1", - "name": "check left_hand hold parallel_box1", - "identifier": 11, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "parallel_box1", - "status": True, - } - ], - }, - { - "summary": "pick up gear1 using parallel_box1", - "name": "pick_up(left_hand, parallel_box1, gear1)", - "identifier": 12, - "type_name": "action", - "effects": [ - { - "object_name": "parallel_box1", - "property_name": "hold", - "property_value": "gear1", - "status": True, - }, - { - "object_name": "parallel_box1", - "property_name": "is_free", - "property_value": None, - "status": False, - }, - ], - }, - ], - }, - ], - }, - ################ - { - "summary": "check if gear1 can be inserted to shaft1", - "name": "check gear1 can_insert_to shaft1", - "identifier": 18, - "type_name": "condition", - "conditions": [ - { - "object_name": "gear1", - "property_name": "can_insert_to", - "property_value": "shaft1", - "status": True, - } - ], - }, - { - "summary": "insert gear1 to shaft1", - "name": "insert(left_hand, parallel_box1, gear1, shaft1)", - "identifier": 19, - "type_name": "action", - "effects": [ - { - "object_name": "parallel_box1", - "property_name": "hold", - "property_value": "gear1", - "status": False, - }, - { - "object_name": "parallel_box1", - "property_name": "is_free", - "property_value": None, - "status": True, - }, - { - "object_name": "gear1", - "property_name": "is_inserted_to", - "property_value": "shaft1", - "status": True, - }, - ], - }, - ], - }, - ], -} - -example_json_data = { - "name": "insert selector", - "identifier": 13, - "type_name": "selector", - "children": [ - { - "summary": "check if gear1 is inserted to shaft1", - "name": "check gear1 is_inserted_to shaft1", - "identifier": 14, - "type_name": "condition", - "conditions": [ - { - "object_name": "gear1", - "property_name": "is_inserted_to", - "property_value": "shaft1", - "status": True, - } - ], - }, - { - "name": "insert sequence", - "identifier": 15, - "type_name": "sequence", - "children": [ - { - "summary": "check if left hand holds parallel_box1", - "name": "check left_hand hold parallel_box1", - "identifier": 16, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "parallel_box1", - "status": True, - } - ], - }, - { - "summary": "check if parallel_box1 holds gear1", - "name": "check parallel_box1 hold gear1", - "identifier": 17, - "type_name": "condition", - "conditions": [ - { - "object_name": "parallel_box1", - "property_name": "hold", - "property_value": "gear1", - "status": True, - } - ], - }, - { - "summary": "check if gear1 can be inserted to shaft1", - "name": "check gear1 can_insert_to shaft1", - "identifier": 18, - "type_name": "condition", - "conditions": [ - { - "object_name": "gear1", - "property_name": "can_insert_to", - "property_value": "shaft1", - "status": True, - } - ], - }, - { - "summary": "insert gear1 to shaft1", - "name": "insert(left_hand, parallel_box1, gear1, shaft1)", - "identifier": 19, - "type_name": "action", - "effects": [ - { - "object_name": "parallel_box1", - "property_name": "hold", - "property_value": "gear1", - "status": False, - }, - { - "object_name": "parallel_box1", - "property_name": "is_free", - "property_value": None, - "status": True, - }, - { - "object_name": "gear1", - "property_name": "is_inserted_to", - "property_value": "shaft1", - "status": True, - }, - ], - }, - ], - }, - ], -} - -result_bt_json = { - "name": "insert gear3 into shaft2 selector", - "identifier": 0, - "type_name": "selector", - "children": [ - { - "summary": "check if gear3 is inserted to shaft2", - "name": "check gear3 is_inserted_to shaft2", - "identifier": 1, - "type_name": "condition", - "conditions": [ - { - "object_name": "gear3", - "property_name": "is_inserted_to", - "property_value": "shaft2", - "status": True, - } - ], - }, - { - "name": "insert gear3 into shaft2 sequence", - "identifier": 2, - "type_name": "sequence", - "children": [ - { - "name": "load_tool selector", - "identifier": 3, - "type_name": "selector", - "children": [ - { - "summary": "check if inward_claw is held by left hand", - "name": "check left_hand hold inward_claw", - "identifier": 4, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "inward_claw", - "status": True, - } - ], - }, - { - "name": "load_tool sequence", - "identifier": 5, - "type_name": "sequence", - "children": [ - { - "summary": "check if inward_claw is equippable", - "name": "check inward_claw is_equippable", - "identifier": 6, - "type_name": "condition", - "conditions": [ - { - "object_name": "inward_claw", - "property_name": "is_equippable", - "property_value": None, - "status": True, - } - ], - }, - { - "summary": "check if left hand is free", - "name": "check left_hand is_free", - "identifier": 7, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "is_free", - "property_value": None, - "status": True, - } - ], - }, - { - "summary": "equip inward_claw to left hand", - "name": "load_tool(left_hand, inward_claw)", - "identifier": 8, - "type_name": "action", - "effects": [ - { - "object_name": "left_hand", - "property_name": "is_free", - "property_value": None, - "status": False, - }, - { - "object_name": "inward_claw", - "property_name": "is_equippable", - "property_value": None, - "status": False, - }, - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "inward_claw", - "status": True, - }, - ], - }, - ], - }, - ], - }, - { - "name": "pickup selector", - "identifier": 9, - "type_name": "selector", - "children": [ - { - "summary": "check if gear3 is held by inward_claw", - "name": "check inward_claw hold gear3", - "identifier": 10, - "type_name": "condition", - "conditions": [ - { - "object_name": "inward_claw", - "property_name": "hold", - "property_value": "gear3", - "status": True, - } - ], - }, - { - "name": "pickup sequence", - "identifier": 11, - "type_name": "sequence", - "children": [ - { - "summary": "check if gear3 is free", - "name": "check gear3 is_free", - "identifier": 12, - "type_name": "condition", - "conditions": [ - { - "object_name": "gear3", - "property_name": "is_free", - "property_value": None, - "status": True, - } - ], - }, - { - "summary": "pick up gear3 using inward_claw", - "name": "pickup(left_hand, gear3, inward_claw)", - "identifier": 13, - "type_name": "action", - "effects": [ - { - "object_name": "inward_claw", - "property_name": "hold", - "property_value": "gear3", - "status": True, - }, - { - "object_name": "inward_claw", - "property_name": "is_free", - "property_value": None, - "status": False, - }, - ], - }, - ], - }, - ], - }, - { - "summary": "check if gear3 can be inserted to shaft2", - "name": "check gear3 can_insert_to shaft2", - "identifier": 14, - "type_name": "condition", - "conditions": [ - { - "object_name": "gear3", - "property_name": "can_insert_to", - "property_value": "shaft2", - "status": True, - } - ], - }, - { - "summary": "insert gear3 into shaft2", - "name": "insert(left_hand, inward_claw, gear3, shaft2)", - "identifier": 15, - "type_name": "action", - "effects": [ - { - "object_name": "inward_claw", - "property_name": "hold", - "property_value": "gear3", - "status": False, - }, - { - "object_name": "inward_claw", - "property_name": "is_free", - "property_value": None, - "status": True, - }, - { - "object_name": "gear3", - "property_name": "is_inserted_to", - "property_value": "shaft2", - "status": True, - }, - ], - }, - ], - }, - ], -} - -response_ed1 = { - "name": "insert selector", - "identifier": 0, - "type_name": "selector", - "children": [ - { - "summary": "check if gear1 is inserted to shaft1", - "name": "check gear1 is_inserted_to shaft1", - "identifier": 1, - "type_name": "condition", - "conditions": [ - { - "object_name": "gear1", - "property_name": "is_inserted_to", - "property_value": "shaft1", - "status": True, - } - ], - }, - { - "name": "insert sequence", - "identifier": 2, - "type_name": "sequence", - "children": [ - { - "name": "load_tool selector", - "identifier": 3, - "type_name": "selector", - "children": [ - { - "summary": "check if parallel_box1 is equipped", - "name": "check left_hand hold parallel_box1", - "identifier": 4, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "parallel_box1", - "status": True, - } - ], - }, - { - "name": "load_tool sequence", - "identifier": 5, - "type_name": "sequence", - "children": [ - { - "summary": "check if parallel_box1 is equippable", - "name": "check parallel_box1 is_equippable", - "identifier": 6, - "type_name": "condition", - "conditions": [ - { - "object_name": "parallel_box1", - "property_name": "is_equippable", - "property_value": None, - "status": True, - } - ], - }, - { - "summary": "check if left hand is free", - "name": "check left_hand is_free", - "identifier": 7, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "is_free", - "property_value": None, - "status": True, - } - ], - }, - { - "summary": "equip parallel_box1 to left hand", - "name": "load_tool(left_hand, parallel_box1)", - "identifier": 8, - "type_name": "action", - "effects": [ - { - "object_name": "left_hand", - "property_name": "is_free", - "property_value": None, - "status": False, - }, - { - "object_name": "parallel_box1", - "property_name": "is_equippable", - "property_value": None, - "status": False, - }, - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "parallel_box1", - "status": True, - }, - ], - }, - ], - }, - ], - }, - { - "name": "pick_up selector", - "identifier": 9, - "type_name": "selector", - "children": [ - { - "summary": "check if gear1 is held by parallel_box1", - "name": "check parallel_box1 hold gear1", - "identifier": 10, - "type_name": "condition", - "conditions": [ - { - "object_name": "parallel_box1", - "property_name": "hold", - "property_value": "gear1", - "status": True, - } - ], - }, - { - "name": "pick_up sequence", - "identifier": 11, - "type_name": "sequence", - "children": [ - { - "summary": "check if gear1 is free", - "name": "check gear1 is_free", - "identifier": 12, - "type_name": "condition", - "conditions": [ - { - "object_name": "gear1", - "property_name": "is_free", - "property_value": None, - "status": True, - } - ], - }, - { - "summary": "pick up gear1 using parallel_box1", - "name": "pick_up(left_hand, parallel_box1, gear1)", - "identifier": 13, - "type_name": "action", - "effects": [ - { - "object_name": "gear1", - "property_name": "is_free", - "property_value": None, - "status": False, - }, - { - "object_name": "parallel_box1", - "property_name": "hold", - "property_value": "gear1", - "status": True, - }, - ], - }, - ], - }, - ], - }, - { - "summary": "check if gear1 can be inserted to shaft1", - "name": "check gear1 can_insert_to shaft1", - "identifier": 14, - "type_name": "condition", - "conditions": [ - { - "object_name": "gear1", - "property_name": "can_insert_to", - "property_value": "shaft1", - "status": True, - } - ], - }, - { - "summary": "insert gear1 to shaft1", - "name": "insert(left_hand, parallel_box1, gear1, shaft1)", - "identifier": 15, - "type_name": "action", - "effects": [ - { - "object_name": "gear1", - "property_name": "is_inserted_to", - "property_value": "shaft1", - "status": True, - }, - { - "object_name": "parallel_box1", - "property_name": "hold", - "property_value": "gear1", - "status": False, - }, - { - "object_name": "parallel_box1", - "property_name": "is_free", - "property_value": None, - "status": True, - }, - ], - }, - ], - }, - ], -} - -prompt_example = { - "name": "insert selector", - "identifier": 13, - "type_name": "selector", - "children": [ - { - "summary": "check if gear1 is inserted to shaft1", - "name": "check gear1 is_inserted_to shaft1", - "identifier": 14, - "type_name": "condition", - "conditions": [ - { - "object_name": "gear1", - "property_name": "is_inserted_to", - "property_value": "shaft1", - "status": True, - } - ], - }, - { - "name": "insert sequence", - "identifier": 15, - "type_name": "sequence", - "children": [ - { - "name": "load_tool selector", - "identifier": 0, - "type_name": "selector", - "children": [ - { - "summary": "check if parallel_box1 is hold", - "name": "check left_hand hold parallel_box1", - "identifier": 1, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "parallel_box1", - "status": True, - } - ], - }, - { - "name": "load_tool Sequence", - "identifier": 2, - "type_name": "sequence", - "children": [ - { - "summary": "check if parallel_box1 is equippable", - "name": "check parallel_box1 is_equippable", - "identifier": 3, - "type_name": "condition", - "conditions": [ - { - "object_name": "parallel_box1", - "property_name": "is_equippable", - "property_value": None, - "status": True, - } - ], - }, - { - "summary": "check if left hand is free", - "name": "check left_hand is_free", - "identifier": 4, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "is_free", - "property_value": None, - "status": True, - } - ], - }, - { - "summary": "equip parallel_box1 to left hand", - "name": "load_tool(left_hand, parallel_box1)", - "identifier": 5, - "type_name": "action", - "effects": [ - { - "object_name": "hand", - "property_name": "is_free", - "property_value": None, - "status": False, - }, - { - "object_name": "parallel_box1", - "property_name": "is_equippable", - "property_value": None, - "status": False, - }, - { - "object_name": "hand", - "property_name": "hold", - "property_value": "parallel_box1", - "status": True, - }, - ], - }, - ], - }, - ], - }, - ################ - { - "name": "pick_up selector", - "identifier": 6, - "type_name": "selector", - "children": [ - { - "summary": "check if parallel_box1 holds gear1", - "name": "check parallel_box1 hold gear1", - "identifier": 7, - "type_name": "condition", - "conditions": [ - { - "object_name": "parallel_box1", - "property_name": "hold", - "property_value": "gear1", - "status": True, - } - ], - }, - { - "name": "pick_up Sequence", - "identifier": 8, - "type_name": "sequence", - "children": [ - { - "summary": "check if parallel_box1 is free", - "name": "check parallel_box1 is_free", - "identifier": 9, - "type_name": "condition", - "conditions": [ - { - "object_name": "parallel_box1", - "property_name": "is_free", - "property_value": None, - "status": True, - } - ], - }, - ################ - { - "summary": "check if left hand holds parallel_box1", - "name": "check left_hand hold parallel_box1", - "identifier": 16, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "parallel_box1", - "status": True, - } - ], - }, - ################ - { - "summary": "pick up gear1 using parallel_box1", - "name": "pick_up(left_hand, parallel_box1, gear1)", - "identifier": 12, - "type_name": "action", - "effects": [ - { - "object_name": "parallel_box1", - "property_name": "hold", - "property_value": "gear1", - "status": True, - }, - { - "object_name": "parallel_box1", - "property_name": "is_free", - "property_value": None, - "status": False, - }, - ], - }, - ], - }, - ], - }, - ################ - { - "summary": "check if gear1 can be inserted to shaft1", - "name": "check gear1 can_insert_to shaft1", - "identifier": 18, - "type_name": "condition", - "conditions": [ - { - "object_name": "gear1", - "property_name": "can_insert_to", - "property_value": "shaft1", - "status": True, - } - ], - }, - { - "summary": "insert gear1 to shaft1", - "name": "insert(left_hand, parallel_box1, gear1, shaft1)", - "identifier": 19, - "type_name": "action", - "effects": [ - { - "object_name": "parallel_box1", - "property_name": "hold", - "property_value": "gear1", - "status": False, - }, - { - "object_name": "parallel_box1", - "property_name": "is_free", - "property_value": None, - "status": True, - }, - { - "object_name": "gear1", - "property_name": "is_inserted_to", - "property_value": "shaft1", - "status": True, - }, - ], - }, - ], - }, - ], -} - -bt_skeleton = { - "summary": "selector to insert gear1 into shaft1", - "name": "selector: insert gear1 into shaft1", - "type_name": "selector", - "children": [ - { - "summary": "check if gear1 is inserted to shaft1", - "name": "target: is_inserted_to(gear1, shaft1)", - "type_name": "condition", - }, - { - "summary": "sequence to insert gear1 into shaft1", - "name": "sequence: insert gear1 into shaft1", - "type_name": "sequence", - "children": [ - { - "summary": "selector to load parallel_box1", - "name": "selector: load parallel_box1", - "type_name": "selector", - "children": [ - { - "summary": "check if parallel_box1 is held by left hand", - "name": "target: hold(left_hand, parallel_box1)", - "type_name": "condition", - }, - { - "summary": "sequence to load parallel_box1", - "name": "sequence: load parallel_box1", - "type_name": "sequence", - "children": [ - { - "summary": "check if parallel_box1 is equippable", - "name": "precondition: is_equippable(parallel_box1)", - "type_name": "condition", - }, - { - "summary": "check if left hand is free", - "name": "precondition: is_free(left_hand)", - "type_name": "condition", - }, - { - "summary": "equip parallel_box1 to left hand", - "name": "action: load_tool(left_hand, parallel_box1)", - "type_name": "action", - "effects": [ - {"summary": "left hand will be not free"}, - { - "summary": "parallel_box1 will be not equippable" - }, - { - "summary": "left hand will hold parallel_box1" - }, - ], - }, - ], - }, - ], - }, - { - "summary": "selector to pick up gear1", - "name": "selector: pick up gear1", - "type_name": "selector", - "children": [ - { - "summary": "check if parallel_box1 holds gear1", - "name": "target: hold(parallel_box1, gear1)", - "type_name": "condition", - }, - { - "summary": "sequence to pick up gear1", - "name": "sequence: pick up gear1", - "type_name": "sequence", - "children": [ - { - "summary": "check if parallel_box1 can manipulate gear1", - "name": "precondition: can_manipulate(parallel_box1, gear1)", - "type_name": "condition", - }, - { - "summary": "check if gear1 is free", - "name": "precondition: is_free(gear1)", - "type_name": "condition", - }, - { - "summary": "pick up gear1 using parallel_box1", - "name": "action: pick_up(left_hand, parallel_box1, gear1)", - "type_name": "action", - "effects": [ - {"summary": "parallel_box1 will hold gear1"}, - {"summary": "gear1 will be not free"}, - ], - }, - ], - }, - ], - }, - { - "summary": "check if gear1 can be inserted to shaft1", - "name": "precondition: can_insert_to(gear1, shaft1)", - "type_name": "condition", - }, - { - "summary": "insert gear1 to shaft1", - "name": "action: insert(left_hand, parallel_box1, gear1, shaft1)", - "type_name": "action", - "effects": [ - {"summary": "parallel_box1 will be free"}, - {"summary": "gear1 will be inserted to shaft1"}, - ], - }, - ], - }, - ], -} - - -def test_result_bt(result_bt): - test_class = BehaviorTreeFactory() - bt = test_class.from_json_to_bt(result_bt) - bt_stewardship = generate_bt_stewardship(bt) - bt_stewardship.setup(timeout=15) - render_dot_tree(bt_stewardship) - - tick_loop_test(bt_stewardship) - - -def test_bt(bt_json: json): - test_class = BehaviorTreeFactory() - bt = test_class.from_json_to_simple_bt(bt_json) - bt_stewardship = generate_bt_stewardship(bt) - bt_stewardship.setup(timeout=15) - render_dot_tree(bt_stewardship) - - tick_loop_test(bt_stewardship) - - -def visualize_bt(bt_json: json): - test_class = BehaviorTreeFactory() - bt = test_class.from_json_to_simple_bt(bt_json) - bt_stewardship = generate_bt_stewardship(bt) - # bt_stewardship.setup(timeout=15) - render_dot_tree(bt_stewardship) - - # tick_loop_test(bt_stewardship) - - -if __name__ == "__main__": - # test_simple_bt() - # test_result_bt(result_bt_json) - # test_result_bt(result_bt_json) - # test_bt(prompt_example) - py_trees.logging.level = py_trees.logging.Level.DEBUG - # test_bt(result_bt_json) - test_bt(bt_skeleton) - # visualize_bt(bt_skeleton) diff --git a/kios_bt_planning/tests/chain_test.py b/kios_bt_planning/tests/chain_test.py deleted file mode 100644 index 07b489bc..00000000 --- a/kios_bt_planning/tests/chain_test.py +++ /dev/null @@ -1,240 +0,0 @@ -from kios_bt.data_types import Action, Condition, ObjectProperty, ControlFlow -from kios_bt.behavior_nodes import ActionNode, ConditionNode, ActionNodeTest -from kios_world.world_interface import WorldInterface -from kios_bt.bt_factory import BehaviorTreeFactory -from kios_bt.bt_stewardship import BehaviorTreeStewardship -from kios_utils.pybt_test import ( - generate_bt_stewardship, - tick_once_test, - render_dot_tree, - tick_loop_test, -) -from kios_agent.llm_supporter import KiosLLMSupporter - - -from typing import List, Dict, Any -import json -import os -import logging - -import py_trees - -from langchain_openai import ChatOpenAI -from langchain_core.output_parsers import StrOutputParser, JsonOutputParser - - -def test_bt(bt_json: json): - test_class = BehaviorTreeFactory() - bt = test_class.from_json_to_simple_bt(bt_json) - # bt = test_class.from_json_to_tree_root(bt_json) - bt_stewardship = generate_bt_stewardship(bt) - bt_stewardship.setup(timeout=15) - render_dot_tree(bt_stewardship) - - tick_loop_test(bt_stewardship) - - -def visualize_bt(bt_json: json): - test_class = BehaviorTreeFactory() - bt = test_class.from_json_to_simple_bt(bt_json) - bt_stewardship = generate_bt_stewardship(bt) - # bt_stewardship.setup(timeout=15) - render_dot_tree(bt_stewardship) - - # tick_loop_test(bt_stewardship) - - -def main(): - py_trees.logging.level = py_trees.logging.Level.DEBUG - - # problem_name = "gearset_problem1_skeleton_v1" - # problem = "(define (problem robot_assembly_problem-problem)\ - # (:domain robot_assembly_problem-domain)\ - # (:objects\ - # parallel_box1 parallel_box2 inward_claw outward_claw no_tool - tool\ - # gear1 gear2 gear3 shaft1 shaft2 base - part\ - # left_hand - hand\ - # )\ - # (:init (can_manipulate parallel_box1 gear1) (can_manipulate outward_claw gear2) (can_manipulate inward_claw gear3) (can_manipulate parallel_box2 shaft1) (can_manipulate no_tool shaft2) (can_screw_to leg1 seat) (can_screw_to leg2 seat) (can_insert_to back seat) (can_screw_to nut1 seat) (can_screw_to nut2 seat) (can_screw_to blub base) (can_place_to lamp blub) (can_insert_to shaft1 base) (can_insert_to shaft2 base) (can_insert_to gear3 shaft2) (can_insert_to gear2 base) (can_insert_to gear1 shaft1) (is_inserted_to shaft1 base) (is_free left_hand) (is_free parallel_box1) (is_free parallel_box2) (is_free inward_claw) (is_free outward_claw) (is_free no_tool) (is_free gear1) (is_free gear2) (is_free gear3) (is_free shaft1) (is_free shaft2) (is_free base) (is_equippable parallel_box1) (is_equippable parallel_box2) (is_equippable inward_claw) (is_equippable outward_claw) (is_equippable no_tool))\ - # (:goal (and (is_inserted_to gear1 shaft1)))\ - # )" # ! CHEAT - - # problem_name = "gearset2" # ! updated default initial states - # problem = "(define (problem robot_assembly_problem-problem)\ - # (:domain robot_assembly_problem-domain)\ - # (:objects\ - # parallel_box1 parallel_box2 inward_claw outward_claw no_tool - tool\ - # gear1 gear2 gear3 shaft1 shaft2 base - part\ - # left_hand - hand\ - # )\ - # (:init (can_manipulate parallel_box1 gear1) (can_manipulate outward_claw gear2) (can_manipulate inward_claw gear3) (can_manipulate parallel_box2 shaft1) (can_manipulate no_tool shaft2) (can_screw_to leg1 seat) (can_screw_to leg2 seat) (can_insert_to back seat) (can_screw_to nut1 seat) (can_screw_to nut2 seat) (can_screw_to blub base) (can_place_to lamp blub) (can_insert_to shaft1 base) (can_insert_to shaft2 base) (can_insert_to gear3 shaft2) (can_insert_to gear2 base) (can_insert_to gear1 shaft1) (is_inserted_to shaft2 base) (is_free left_hand) (is_free parallel_box1) (is_free parallel_box2) (is_free inward_claw) (is_free outward_claw) (is_free no_tool) (is_equippable parallel_box1) (is_equippable parallel_box2) (is_equippable inward_claw) (is_equippable outward_claw) (is_equippable no_tool))\ - # (:goal (and (is_inserted_to gear3 shaft2)))\ - # )\ - # " - - problem_name = "gearset3" # ! updated default initial states - problem = "(define (problem robot_assembly_problem-problem)\ - (:domain robot_assembly_problem-domain)\ - (:objects\ - parallel_box1 parallel_box2 inward_claw outward_claw no_tool - tool\ - gear1 gear2 gear3 shaft1 shaft2 base - part\ - left_hand - hand\ - )\ - (:init (can_manipulate parallel_box1 gear1) (can_manipulate outward_claw gear2) (can_manipulate inward_claw gear3) (can_manipulate parallel_box2 shaft1) (can_manipulate no_tool shaft2) (can_screw_to leg1 seat) (can_screw_to leg2 seat) (can_insert_to back seat) (can_screw_to nut1 seat) (can_screw_to nut2 seat) (can_screw_to blub base) (can_place_to lamp blub) (can_insert_to shaft1 base) (can_insert_to shaft2 base) (can_insert_to gear3 shaft2) (can_insert_to gear2 base) (can_insert_to gear1 shaft1) (is_free left_hand) (is_free parallel_box1) (is_free parallel_box2) (is_free inward_claw) (is_free outward_claw) (is_equippable parallel_box1) (is_equippable parallel_box2) (is_equippable inward_claw) (is_equippable outward_claw) (is_inserted_to shaft2 base))\ - (:goal (and (is_inserted_to gear3 shaft2)))\ - )" - - # * refine problem - # problem_name = "gearset3_sk" # super skeleton - # file_dir = os.path.dirname(os.path.abspath(__file__)) - # problem_dir = os.path.join(file_dir, "gearset3_cot_sk.txt") - # with open(problem_dir, "r") as f: - # problem = f.read() - - ### * end_to_end - # feature = "e2e" - # model = "gpt-4-1106-preview" - # ver = "v2" - - # prompt_dir = "prompts/end_to_end_v2" - # prompt_load_order = [ - # "e2e_role", # your are a good interpreter - # "e2e_output_format", # how to generate the output - # "e2e_domain", # domain knowledge - # "e2e_problem", # the problem format - # "e2e_state", # hot to describe the state - # "e2e_bt", # how to build tree - # "e2e_chain", # COT - # "e2e_example", # some skeleton examples ... Done - # ] - ### * - - ### * refine_sk - # feature = "re_sk" - # model = "gpt-4-1106-preview" - # # model = "gpt-3.5-turbo-16k-0613" - # ver = "v1" - - # prompt_dir = "prompts/sk_refine" - # prompt_load_order = [ - # "refine_role", # your are a good interpreter - # "refine_input_format", # about the input - # "refine_state", # about the state - # # "refine_domain", # domain knowledge - # "refine_help", # tips about how to refine the nodes - # "refine_controlflow", # how to refine controlflow nodes - # "refine_condition", # how to refine condition nodes - # "refine_action", # how to refine action nodes - # "refine_output_format", # the output format - # # "refine_chain", # COT - # ] - ### * - - ### *skeleton - # feature = "skeleton" - # model = "gpt-4-1106-preview" - # ver = "v2" - # prompt_dir = "prompts/skeleton" - # prompt_load_order = [ - # "prompt_role", # your are a good interpreter ... Done - # "prompt_domain", # domain knowledge ... Done - # "prompt_problem", # how the problem is provided ... Done - # "prompt_environment", # how to express the environment ... Done - # # "prompt_behaviortree", # how to construct the behavior tree ... Done - # "prompt_bt_skeleton", # how to construct the skeleton behavior tree ... Done - # "prompt_bt_skeleton_example", # some skeleton examples ... Done - # "prompt_output_format", # the output format ... Done - # # "prompt_example", # some examples ... Done - # ] - ### * - - ### *COT spsk - # feature = "cot_ap_skeleton" - # model = "gpt-4-1106-preview" - # ver = "v1" - # """ - # add chain, try to solve state inconsistency by applying COT - # """ - # prompt_dir = "prompts/cot_sp_skeleton" - # prompt_load_order = [ - # "cot_role", # your are a good interpreter - # "cot_output_format", # how to generate the output - # "cot_domain", # domain knowledge - # "cot_problem", # the problem format - # "cot_state", # hot to describe the state - # "cot_bt", # how to build tree - # "cot_chain", # COT - # "cot_example", # some skeleton examples ... Done - # ] - ### * - - ## *COT skeleton - feature = "cot_skeleton" - model = "gpt-4-1106-preview" - ver = "v1" - """ - add chain, try to solve state inconsistency by applying COT - use skeleton - """ - prompt_dir = "prompts/cot_skeleton_chain" - prompt_load_order = [ - "cot_sk_role", # your are a good interpreter - "cot_sk_output_format", # how to generate the output - "cot_sk_domain", # domain knowledge - "cot_sk_problem", # the problem format - "cot_sk_state", # hot to describe the state - "cot_sk_bt", # how to build tree - "cot_sk_chain", # COT - "cot_sk_example", # some skeleton examples ... Done - ] - ## * - - llm_supporter = KiosLLMSupporter() - - llm_supporter.initialize( - prompt_dir=prompt_dir, - prompt_load_order=prompt_load_order, - ) - - # * model = "gpt-3.5-turbo-16k-0613" - # * model="gpt-4-0613", # not this - # * model="gpt-4-1106-preview", - - full_problem_name = f"{problem_name}_{feature}_{ver}_{model}" - - prompt = llm_supporter.create_prompt() - - llm = ChatOpenAI(model_name=model, temperature=0) - - chain = prompt | llm | JsonOutputParser() - - response = chain.invoke( - { - "problem": problem, - } - ) - - bt = response.get("task_plan").get("behavior_tree") - - test_bt(bt) - - # print(response) - - -if __name__ == "__main__": - main() - - # test_bt(result_bt_json) - # test_bt(bt_skeleton) - # visualize_bt(bt_skeleton) - -# ! should the is_free has a default true? or false? -# ! can I explain the knowledge differently from the pddl domain? - -# ! BUG in example: is_free is still default to be true!!! -# ! BUG in example: env after ist falsch! -# ! BUG in example: the boolean value is T F instead of t f. also should be null not None -# ! The action name "action: xxx(xxx, xxx)" is not consistent with the format requirement in action parser! -# ! BUG in output_format: four instead of three! - -# ! SYSTEMATIC BUG in problem: the example problem still applies the is free to the parts!!!!!!!! diff --git a/kios_bt_planning/tests/chain_test2.py b/kios_bt_planning/tests/chain_test2.py deleted file mode 100644 index 582994d9..00000000 --- a/kios_bt_planning/tests/chain_test2.py +++ /dev/null @@ -1,208 +0,0 @@ -from kios_bt.data_types import Action, Condition, ObjectProperty, ControlFlow -from kios_bt.behavior_nodes import ActionNode, ConditionNode, ActionNodeTest -from kios_world.world_interface import WorldInterface -from kios_bt.bt_factory import BehaviorTreeFactory -from kios_bt.bt_stewardship import BehaviorTreeStewardship -from kios_utils.pybt_test import ( - generate_bt_stewardship, - tick_once_test, - render_dot_tree, - tick_loop_test, -) - -from kios_agent.llm_supporter import KiosLLMSupporter -from kios_agent.data_types import AgentResponse, KiosPromptSkeleton - - -from typing import List, Dict, Any -import json -import os -import logging - -import py_trees - -from dotenv import load_dotenv - -from langchain_openai import ChatOpenAI -from langchain_core.output_parsers import StrOutputParser, JsonOutputParser - -# * model = "gpt-3.5-turbo-16k-0613" -# * model="gpt-4-0613", # not this -# * model="gpt-4-1106-preview", - - -def test_bt(bt_json: json): - test_class = BehaviorTreeFactory() - bt = test_class.from_json_to_simple_bt(bt_json) - # bt = test_class.from_json_to_tree_root(bt_json) - bt_stewardship = generate_bt_stewardship(bt) - bt_stewardship.setup(timeout=15) - render_dot_tree(bt_stewardship) - - tick_loop_test(bt_stewardship) - - -def visualize_bt(bt_json: json): - test_class = BehaviorTreeFactory() - bt = test_class.from_json_to_simple_bt(bt_json) - bt_stewardship = generate_bt_stewardship(bt) - # bt_stewardship.setup(timeout=15) - render_dot_tree(bt_stewardship) - - -def main(): - py_trees.logging.level = py_trees.logging.Level.DEBUG - - problem_name = "gearset3" # ! updated default initial states - problem = "(define (problem robot_assembly_problem-problem)\ - (:domain robot_assembly_problem-domain)\ - (:objects\ - parallel_box1 parallel_box2 inward_claw outward_claw no_tool - tool\ - gear1 gear2 gear3 shaft1 shaft2 base - part\ - left_hand - hand\ - )\ - (:init (can_manipulate parallel_box1 gear1) (can_manipulate outward_claw gear2) (can_manipulate inward_claw gear3) (can_manipulate parallel_box2 shaft1) (can_manipulate no_tool shaft2) (can_screw_to leg1 seat) (can_screw_to leg2 seat) (can_insert_to back seat) (can_screw_to nut1 seat) (can_screw_to nut2 seat) (can_screw_to blub base) (can_place_to lamp blub) (can_insert_to shaft1 base) (can_insert_to shaft2 base) (can_insert_to gear3 shaft2) (can_insert_to gear2 base) (can_insert_to gear1 shaft1) (is_free left_hand) (is_free parallel_box1) (is_free parallel_box2) (is_free inward_claw) (is_free outward_claw) (is_equippable parallel_box1) (is_equippable parallel_box2) (is_equippable inward_claw) (is_equippable outward_claw) (is_inserted_to shaft2 base))\ - (:goal (and (is_inserted_to gear3 shaft2)))\ - )" - - # * refine problem - # problem_name = "gearset3_sk" # super skeleton - # file_dir = os.path.dirname(os.path.abspath(__file__)) - # problem_dir = os.path.join(file_dir, "gearset3_cot_sk.txt") - # with open(problem_dir, "r") as f: - # problem = f.read() - - load_dotenv() - - data_dir = os.environ.get("KIOS_DATA_DIR").format(username=os.getlogin()) - print(data_dir) - prompt_sk_dir = os.path.join(data_dir, "prompt_skeletons") - - llm_spt_cot_sk = KiosLLMSupporter() - - with open(os.path.join(prompt_sk_dir, "cot_skeleton.json"), "r") as f: - cot_sk_pptsk_json = json.load(f) - cot_sk_pptsk = KiosPromptSkeleton.from_json(cot_sk_pptsk_json) - llm_spt_cot_sk.initialize_from_prompt_skeleton(cot_sk_pptsk) - - cot_sk_ppt = llm_spt_cot_sk.create_prompt() - - cot_sk_llm = ChatOpenAI( - model_name=llm_spt_cot_sk.prompt_skeleton.model_name, temperature=0 - ) - - cot_sk_chain = cot_sk_ppt | cot_sk_llm | JsonOutputParser() - - llm_spt_re_sk = KiosLLMSupporter() - - with open(os.path.join(prompt_sk_dir, "refine_skeleton.json"), "r") as f: - re_sk_pptsk_json = json.load(f) - re_sk_pptsk = KiosPromptSkeleton.from_json(re_sk_pptsk_json) - llm_spt_re_sk.initialize_from_prompt_skeleton(re_sk_pptsk) - - re_sk_ppt = llm_spt_re_sk.create_prompt() - - re_sk_llm = ChatOpenAI( - model_name=llm_spt_re_sk.prompt_skeleton.model_name, temperature=0 - ) - - re_sk_chain = re_sk_ppt | re_sk_llm | JsonOutputParser() - - # response = re_sk_chain.invoke( - # { - # "problem": problem, - # } - # ) - - # bt = response.get("task_plan").get("behavior_tree") - - # test_bt(bt) - - # print(response) - - llm_spt_skeleton_generator = KiosLLMSupporter() - - with open(os.path.join(prompt_sk_dir, "skeleton_generator.json"), "r") as f: - skeleton_generator_pptsk_json = json.load(f) - skeleton_generator_pptsk = KiosPromptSkeleton.from_json( - skeleton_generator_pptsk_json - ) - llm_spt_skeleton_generator.initialize_from_prompt_skeleton( - skeleton_generator_pptsk - ) - - skeleton_generator_ppt = llm_spt_skeleton_generator.create_prompt() - - skeleton_generator_llm = ChatOpenAI( - model_name=llm_spt_skeleton_generator.prompt_skeleton.model_name, temperature=0 - ) - - skeleton_generator_chain = ( - skeleton_generator_ppt | skeleton_generator_llm | JsonOutputParser() - ) - - problem = """ - (define (problem robot_assembly_problem-problem) - (:domain robot_assembly_problem-domain) - (:objects - parallel_box1 parallel_box2 inward_claw outward_claw no_tool - tool - gear1 gear2 gear3 shaft1 shaft2 shaft3 gearbase gearbase_hole1 gearbase_hole3 - part - left_hand - hand - ) - (:init (can_manipulate parallel_box2 gear1) (can_manipulate outward_claw gear2) (can_manipulate outward_claw gear3) (can_manipulate no_tool shaft3) (can_manipulate parallel_box1 shaft1) (can_screw_to leg1 seat) (can_screw_to leg2 seat) (can_insert_to back seat) (can_screw_to nut1 seat) (can_screw_to nut2 seat) (can_screw_to blub lampbase) (can_place_to lamp blub) (can_insert_to shaft1 gearbase_hole1) (can_insert_to shaft3 gearbase_hole3) (can_insert_to gear3 shaft3) (can_insert_to gear2 shaft2) (can_insert_to gear1 shaft1)) - (:goal (and )) - ) - """ - - world_state = { - "objects": [ - {"name": "parallel_box1", "properties": ["is_free", "is_equippable"]}, - {"name": "parallel_box2", "properties": ["is_free", "is_equippable"]}, - {"name": "inward_claw", "properties": ["is_free", "is_equippable"]}, - {"name": "outward_claw", "properties": ["is_free", "is_equippable"]}, - {"name": "no_tool", "properties": ["is_free", "is_equippable"]}, - {"name": "gear1", "properties": []}, - {"name": "gear2", "properties": []}, - {"name": "gear3", "properties": []}, - {"name": "shaft1", "properties": []}, - {"name": "shaft2", "properties": []}, - {"name": "shaft3", "properties": []}, - {"name": "gearbase", "properties": []}, - {"name": "gearbase_hole1", "properties": []}, - {"name": "gearbase_hole3", "properties": []}, - {"name": "left_hand", "properties": ["is_free"]}, - ], - "constraints": [], - "relations": [], - } - - response = skeleton_generator_chain.invoke( - { - "problem": problem, - "world_state": world_state, - "instructions": "insert shaft1 into gearbase_hole1", - } - ) - - bt_skeleton = response.get("task_plan").get("behavior_tree") - - test_bt(bt_skeleton) - - -if __name__ == "__main__": - main() - - # test_bt(result_bt_json) - # test_bt(bt_skeleton) - # visualize_bt(bt_skeleton) - -# ! should the is_free has a default true? or false? -# ! can I explain the knowledge differently from the pddl domain? - -# ! BUG in example: is_free is still default to be true!!! -# ! BUG in example: env after ist falsch! -# ! BUG in example: the boolean value is T F instead of t f. also should be null not None -# ! The action name "action: xxx(xxx, xxx)" is not consistent with the format requirement in action parser! -# ! BUG in output_format: four instead of three! - -# ! SYSTEMATIC BUG in problem: the example problem still applies the is free to the parts!!!!!!!! diff --git a/kios_bt_planning/tests/cot_sk.txt b/kios_bt_planning/tests/cot_sk.txt deleted file mode 100644 index db3fe36e..00000000 --- a/kios_bt_planning/tests/cot_sk.txt +++ /dev/null @@ -1,239 +0,0 @@ -{ - "task_plan": { - "action_sequence": [ - "load_tool(left_hand, parallel_box1)", - "pick_up(left_hand, parallel_box1, shaft1)", - "insert(left_hand, parallel_box1, shaft1, gearbase_hole1)" - ], - "action_instructions": [ - "Equip the parallel_box1 on the left hand to manipulate shaft1.", - "Pick up shaft1 using the parallel_box1 equipped on the left hand.", - "Insert shaft1 into gearbase_hole1 using the parallel_box1 equipped on the left hand." - ], - "manipulated_objects": [ - "left_hand", - "parallel_box1", - "shaft1", - "gearbase_hole1" - ], - "behavior_tree": { - "summary": "selector to insert shaft1 into gearbase_hole1", - "name": "selector: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)", - "children": [ - { - "summary": "condition node to check if shaft1 is inserted to gearbase_hole1", - "name": "target: is_inserted_to(shaft1, gearbase_hole1)" - }, - { - "summary": "sequence to insert shaft1 into gearbase_hole1", - "name": "sequence: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)", - "children": [ - { - "summary": "selector to load parallel_box1", - "name": "selector: load_tool(left_hand, parallel_box1)", - "children": [ - { - "summary": "condition node to check if left_hand holds parallel_box1", - "name": "target: hold(left_hand, parallel_box1)" - }, - { - "summary": "sequence to load parallel_box1", - "name": "sequence: load_tool(left_hand, parallel_box1)", - "children": [ - { - "summary": "condition node to check if parallel_box1 is equippable", - "name": "precondition: is_equippable(parallel_box1)" - }, - { - "summary": "condition node to check if left hand is free", - "name": "precondition: is_free(left_hand)" - }, - { - "summary": "action node to equip parallel_box1 to left hand", - "name": "action: load_tool(left_hand, parallel_box1)", - "effects": [ - { - "summary": "left_hand will be not free" - }, - { - "summary": "parallel_box1 will be not equippable" - }, - { - "summary": "left_hand will hold parallel_box1" - } - ] - } - ] - } - ] - }, - { - "summary": "selector to use left_hand with parallel_box1 to pick up shaft1", - "name": "selector: pick_up(left_hand, parallel_box1, shaft1)", - "children": [ - { - "summary": "condition node to check if parallel_box1 holds shaft1", - "name": "target: hold(parallel_box1, shaft1)" - }, - { - "summary": "sequence to use left hand with parallel_box1 to pick up shaft1", - "name": "sequence: pick_up(left_hand, parallel_box1, shaft1)", - "children": [ - { - "summary": "condition node to check if parallel_box1 is free", - "name": "precondition: is_free(parallel_box1)" - }, - { - "summary": "condition node to check if left hand holds parallel_box1", - "name": "precondition: hold(left_hand, parallel_box1)" - }, - { - "summary": "action node to use left hand with parallel_box1 to pick up shaft1", - "name": "action: pick_up(left_hand, parallel_box1, shaft1)", - "effects": [ - { - "summary": "parallel_box1 will hold shaft1" - }, - { - "summary": "parallel_box1 will be not free" - } - ] - } - ] - } - ] - }, - { - "summary": "condition node to check if shaft1 can be inserted to gearbase_hole1", - "name": "precondition: can_insert_to(shaft1, gearbase_hole1)" - }, - { - "summary": "action node to use left_hand with parallel_box1 to insert shaft1 to gearbase_hole1", - "name": "action: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)", - "effects": [ - { - "summary": "parallel_box1 will be not holding shaft1" - }, - { - "summary": "parallel_box1 will be free" - }, - { - "summary": "shaft1 will be inserted to gearbase_hole1" - } - ] - } - ] - } - ] - } - }, - "initial_state": { - "objects": [ - { - "name": "parallel_box1", - "properties": ["is_free", "is_equippable"] - }, - { - "name": "parallel_box2", - "properties": ["is_free", "is_equippable"] - }, - { - "name": "inward_claw", - "properties": ["is_free", "is_equippable"] - }, - { - "name": "outward_claw", - "properties": ["is_free", "is_equippable"] - }, - { - "name": "no_tool", - "properties": ["is_free", "is_equippable"] - }, - { - "name": "gear1", - "properties": [] - }, - { - "name": "gear2", - "properties": [] - }, - { - "name": "gear3", - "properties": [] - }, - { - "name": "shaft1", - "properties": [] - }, - { - "name": "shaft2", - "properties": [] - }, - { - "name": "shaft3", - "properties": [] - }, - { - "name": "gearbase", - "properties": [] - }, - { - "name": "gearbase_hole1", - "properties": [] - }, - { - "name": "gearbase_hole3", - "properties": [] - }, - { - "name": "left_hand", - "properties": ["is_free"] - } - ], - "constraints": [ - { - "source": "parallel_box1", - "name": "can_manipulate", - "target": "shaft1" - }, - { - "source": "parallel_box2", - "name": "can_manipulate", - "target": "gear1" - }, - { - "source": "outward_claw", - "name": "can_manipulate", - "target": "gear2" - }, - { - "source": "outward_claw", - "name": "can_manipulate", - "target": "gear3" - }, - { - "source": "no_tool", - "name": "can_manipulate", - "target": "shaft3" - }, - { - "source": "shaft1", - "name": "can_insert_to", - "target": "gearbase_hole1" - }, - { - "source": "shaft3", - "name": "can_insert_to", - "target": "gearbase_hole3" - }, - { - "source": "gear3", - "name": "can_insert_to", - "target": "shaft3" - } - ], - "relations": [] - }, - "instruction_summary": "Insert shaft1 into gearbase_hole1.", - "question": "" -} diff --git a/kios_bt_planning/tests/feature_test.py b/kios_bt_planning/tests/feature_test.py deleted file mode 100644 index f3486f71..00000000 --- a/kios_bt_planning/tests/feature_test.py +++ /dev/null @@ -1,26 +0,0 @@ -class ClassA: - def __init__(self, value): - self.value = value - - -class ClassB: - def __init__(self, value): - self.value = value - - -def map_instances(a, b): - # Perform mapping logic between instances of ClassA and ClassB - # For example, you can access and modify attributes of the instances - a.value += b.value - - -# Create instances of ClassA and ClassB -instance_a = ClassA(10) -instance_b = ClassB(5) - -# Map the instances using the map_instances function -mapped_instances = map(map_instances, [instance_a], [instance_b]) - -# Access the mapped instances -for mapped_instance in mapped_instances: - print(mapped_instance.value) diff --git a/kios_bt_planning/tests/ft_test.py b/kios_bt_planning/tests/ft_test.py deleted file mode 100644 index 3a22f5c9..00000000 --- a/kios_bt_planning/tests/ft_test.py +++ /dev/null @@ -1,3 +0,0 @@ -# * llama-factory -# * llama-factory -# * llama-factory \ No newline at end of file diff --git a/kios_bt_planning/tests/gearset3_cot_sk.txt b/kios_bt_planning/tests/gearset3_cot_sk.txt deleted file mode 100644 index 4bba59f9..00000000 --- a/kios_bt_planning/tests/gearset3_cot_sk.txt +++ /dev/null @@ -1,143 +0,0 @@ - -{ - "task_plan": { - "action_sequence": [ - "load_tool(left_hand, inward_claw)", - "pick_up(left_hand, inward_claw, gear3)", - "insert(left_hand, inward_claw, gear3, shaft2)" - ], - "action_instructions": [ - "Equip the inward_claw on the left hand.", - "Pick up the gear3 with the inward_claw in the left hand.", - "Insert the gear3 into the shaft2 using the inward_claw in the left hand." - ], - "manipulated_objects": ["left_hand", "inward_claw", "gear3", "shaft2"], - "behavior_tree": { - "summary": "selector to insert gear3 into shaft2", - "name": "selector: insert(left_hand, inward_claw, gear3, shaft2)", - "children": [ - { - "summary": "condition node to check if gear3 is inserted to shaft2", - "name": "target: is_inserted_to(gear3, shaft2)" - }, - { - "summary": "sequence to insert gear3 into shaft2", - "name": "sequence: insert(left_hand, inward_claw, gear3, shaft2)", - "children": [ - { - "summary": "selector to load inward_claw", - "name": "selector: load_tool(left_hand, inward_claw)", - "children": [ - { - "summary": "condition node to check if left_hand holds inward_claw", - "name": "target: hold(left_hand, inward_claw)" - }, - { - "summary": "sequence to load inward_claw", - "name": "sequence: load_tool(left_hand, inward_claw)", - "children": [ - { - "summary": "condition node to check if inward_claw is equippable", - "name": "precondition: is_equippable(inward_claw)" - }, - { - "summary": "condition node to check if left hand is free", - "name": "precondition: is_free(left_hand)" - }, - { - "summary": "action node to equip inward_claw to left hand", - "name": "action: load_tool(left_hand, inward_claw)", - "effects": [ - { - "summary": "left_hand will be not free" - }, - { - "summary": "inward_claw will be not equippable" - }, - { - "summary": "left_hand will hold inward_claw" - } - ] - } - ] - } - ] - }, - { - "summary": "selector to use left_hand with inward_claw to pick up gear3", - "name": "selector: pick_up(left_hand, inward_claw, gear3)", - "children": [ - { - "summary": "condition node to check if inward_claw holds gear3", - "name": "target: hold(inward_claw, gear3)" - }, - { - "summary": "sequence to use left hand with inward_claw to pick up gear3", - "name": "sequence: pick_up(left_hand, inward_claw, gear3)", - "children": [ - { - "summary": "condition node to check if inward_claw is free", - "name": "precondition: is_free(inward_claw)" - }, - { - "summary": "condition node to check if left hand holds inward_claw", - "name": "precondition: hold(left_hand, inward_claw)" - }, - { - "summary": "action node to use left hand with inward_claw to pick up gear3", - "name": "action: pick_up(left_hand, inward_claw, gear3)", - "effects": [ - { - "summary": "inward_claw will hold gear3" - }, - { - "summary": "inward_claw will be not free" - } - ] - } - ] - } - ] - }, - { - "summary": "condition node to check if gear3 can be inserted to shaft2", - "name": "precondition: can_insert_to(gear3, shaft2)" - }, - { - "summary": "action node to use left_hand with inward_claw to insert gear3 to shaft2", - "name": "action: insert(left_hand, inward_claw, gear3, shaft2)", - "effects": [ - { - "summary": "inward_claw will be not holding gear3" - }, - { - "summary": "inward_claw will be free" - }, - { - "summary": "gear3 will be inserted to shaft2" - } - ] - } - ] - } - ] - } - }, - "initial_state": { - "objects": [ - {"name": "gear3", "properties": []}, - {"name": "shaft2", "properties": []}, - {"name": "inward_claw", "properties": ["is_equippable", "is_free"]}, - {"name": "left_hand", "properties": ["is_free"]} - ], - "constraints": [ - {"source": "gear3", "name": "can_insert_to", "target": "shaft2"}, - {"source": "inward_claw", "name": "can_manipulate", "target": "gear3"} - ], - "relations": [ - {"source": "shaft2", "name": "is_inserted_to", "target": "base"} - ] - }, - "instruction_summary": "Insert gear3 into shaft2", - "question": "" -} diff --git a/kios_bt_planning/tests/graph_basebone_test.py b/kios_bt_planning/tests/graph_basebone_test.py deleted file mode 100644 index 6f52a3b2..00000000 --- a/kios_bt_planning/tests/graph_basebone_test.py +++ /dev/null @@ -1,110 +0,0 @@ -""" -very simple interface to model the objects in the world and the states of them. -""" -import networkx as nx -from typing import Set, Dict - - -class WorldNode: - def __init__(self, name: str) -> None: - self.name = name - self.states = set() - - def update_states(self, states: Set[str]): - self.states.update(states) - - def difference_update_states(self, states: Set[str]): - self.states.difference_update(states) - - def __repr__(self): - return f"{self.name}: {', '.join(self.states)}" - - -class Relationship: - """ - directed relationship from source to target - """ - - def __init__(self, source, name, target): - self.name = name - self.objects = [source, target] - - def __repr__(self): - return f"{self.objects[0]} {self.name} {self.objects[1]}" - - -class GraphInterface: - def __init__(self): - self.objects: Dict[str, WorldNode] = {} - self.graph = nx.DiGraph() - - def __repr__(self): - print_str = "" - for obj in self.objects.values(): - print_str += f"{obj}\n" - - return print_str - - def add_node(self, object_name: str, states: Set[str] = None): - if object_name not in self.objects: - self.objects[object_name] = WorldNode(object_name) - if states is not None: - self.objects[object_name].update_states(states) - else: - raise ValueError(f"Object {object_name} already exists!") - - def add_states(self, object_name: str, states: Set[str]): - if object_name not in self.objects: - raise ValueError(f"Object {object_name} does not exist!") - self.objects[object_name].update_states(states) - - def remove_states(self, object_name: str, states: Set[str]): - if object_name not in self.objects: - raise ValueError(f"Object {object_name} does not exist!") - self.objects[object_name].difference_update_states(states) - - def get_object(self, object_name: str) -> WorldNode: - if object_name not in self.objects: - raise ValueError(f"Object {object_name} does not exist!") - return self.objects[object_name] - - def update_neo4j(self): - raise NotImplementedError - - -def test_world_object(): - print("test_world_object ----------------------------------") - obj = WorldNode("obj") - obj.update_states({"state1", "state2"}) - print(obj) - obj.difference_update_states( - {"state1"} - ) # Remove individual state instead of a set of states - print(obj) - - -def test_relationship(): - print("test_relationship ----------------------------------") - rel = Relationship("ball", "on", "desk") - print(rel) - - -def test_object_depot(): - print("test_object_depot ----------------------------------") - depot = GraphInterface() - depot.add_node("ball", {"state1", "state2"}) - depot.add_node("desk", {"state1", "state2"}) - depot.add_node("cup", {"state1", "state2"}) - depot.add_node("table", {"state1", "state2"}) - print(depot) - depot.add_states("ball", {"state3"}) - print(depot) - depot.remove_states("ball", {"state3"}) - print(depot) - print(depot.get_object("ball")) - - -if __name__ == "__main__": - test_world_object() - test_relationship() - test_object_depot() diff --git a/kios_bt_planning/tests/graph_interface_test.py b/kios_bt_planning/tests/graph_interface_test.py deleted file mode 100644 index 9db26847..00000000 --- a/kios_bt_planning/tests/graph_interface_test.py +++ /dev/null @@ -1,102 +0,0 @@ -from kios_world.graph_interface import GraphInterface -from kios_world.data_types import Relationship, WorldNode -import networkx as nx - - -def test_world_object(): - print("test_world_object ----------------------------------") - obj = WorldNode("ball") - print(obj) - obj.update_properties({"state1", "state2"}) - print(obj) - obj.add_property("state3") - print(obj) - obj.difference_update_properties({"state3"}) - print(obj) - obj.discard_property("state2") - print(obj) - - -def test_relationship(): - print("test_relationship ----------------------------------") - rel = Relationship("ball", "on", "table") - print(rel) - rel2 = Relationship("ball", "on", "table") - print(rel == rel2) - rel3 = Relationship("ball", "on", "cup") - print(rel == rel3) - print(rel == "ball") - print(rel == 1) - print(rel == [1, 2, 3]) - print(rel == {"a": 1, "b": 2}) - - -def test_graph_interface(): - print("test_graph_interface ----------------------------------") - graph = GraphInterface() - graph.add_node("ball") - graph.add_node("table") - graph.add_node("cup") - graph.add_properties("ball", ["state1", "state2"]) - graph.add_property("table", "state3") - graph.add_relation("ball", "on", "table") - graph.add_relation("ball", "on", "cup") - graph.add_relation("cup", "on", "table") - graph.add_relation("cup", "on", "ball") - print(graph.nodes) - print(graph.relations) - graph.remove_property("ball", "state1") - graph.remove_properties("ball", ["state2"]) - print(graph.nodes) - graph.discard_relation("ball", "on", "cup") - print(graph.relations) - - -def test_networkx_and_neo4j(): - print("test_networkx ----------------------------------") - # Create a new directed graph - gi = GraphInterface() - gi.add_node("table") - gi.add_node("ball") - gi.add_node("cabinet") - gi.add_properties("cabinet", ["open"]) - gi.add_relation("ball", "on_something", "table") - gi.refresh_networkx() - print(gi.graph.nodes) - # visualize the graph, invluding the properties and relation names - import matplotlib.pyplot as plt - - nx.draw(gi.graph, with_labels=True) - # plt.show() - - print("test_neo4j ----------------------------------") - gi.refresh_neo4j() - - -def test_from_json(): - json_data = { - "objects": [ - {"name": "table", "properties": ["state1", "state2"]}, - {"name": "ball", "properties": ["state1", "state2"]}, - {"name": "cabinet", "properties": ["open"]}, - ], - "relations": [ - {"source": "ball", "name": "on_something", "target": "table"}, - {"source": "ball", "name": "on_something", "target": "cabinet"}, - {"source": "cabinet", "name": "on_something", "target": "table"}, - {"source": "cabinet", "name": "on_something", "target": "ball"}, - ], - } - gi = GraphInterface() - gi.from_json(json_data) - print(gi.nodes) - print(gi.relations) - gi.refresh_neo4j() - - -if __name__ == "__main__": - # test_world_object() - # test_relationship() - # test_graph_interface() - # test_networkx_and_neo4j() - test_from_json() diff --git a/kios_bt_planning/tests/ltm_test.py b/kios_bt_planning/tests/ltm_test.py deleted file mode 100644 index b8028aae..00000000 --- a/kios_bt_planning/tests/ltm_test.py +++ /dev/null @@ -1,66 +0,0 @@ -import numpy as np -from spatialmath import * -from spatialmath.base import trnorm - - -from kios_robot.mios_task_factory import MiosTaskFactory -from kios_robot.robot_command import RobotCommand -from kios_robot.robot_interface import RobotInterface -from kios_robot.data_types import Toolbox -from kios_scene.scene_factory import SceneFactory - -from kios_scene.mios_ltm_manipulator import LangTermMemoryManipulator - -# ri = RobotInterface() -# sf = SceneFactory() -# scene = sf.create_test_scene() -# ri.setup_scene(scene) - -ltm_manipulator = LangTermMemoryManipulator() - - -# def teach_object(object: str): -# ri.proprioceptor.teach_object(object) -def backup_mios_environment(backup_name: str): - ltm_manipulator.backup_mios_environment(backup_name) - - -def clear_mios_environment(): - ltm_manipulator.clear_mios_environment() - - -def show_backups(): - ltm_manipulator.show_backups() - - -def restore_to_mios_environment(backup_name: str): - ltm_manipulator.restore_to_mios_environment(backup_name) - - -def teach_object_TCP(object_name: str): - ri.proprioceptor.teach_object_TCP(object_name) - ri.proprioceptor.update_scene_object_from_mios(scene=scene, object_name=object_name) - - -def get_object(object: str): - ri.proprioceptor.get_object(object) - - -def set_tool(toolbox_name: str): - if "robot_command" in locals(): - print("robot command is set") - print(f"the robot command's tasks are {robot_command.task_list}") - else: - print("robot command is not set") - - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - shared_data=None, - task_scene=scene, - robot_interface=ri, - ) - print(f"the number of tasks in the task list is {len(robot_command.task_list)}") - robot_command.add_task(ri.mios_task_factory.generate_update_tool_call(toolbox_name)) - robot_command.execute_task_list_sync() - robot_command.clear_tasks() diff --git a/kios_bt_planning/tests/mios_memory_backups/gearset.json b/kios_bt_planning/tests/mios_memory_backups/gearset.json deleted file mode 100644 index ff0d1f09..00000000 --- a/kios_bt_planning/tests/mios_memory_backups/gearset.json +++ /dev/null @@ -1,2502 +0,0 @@ -[ - { - "_id": "643d40f9dd8c607f760e5af8", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "NullObject", - "q": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ] - }, - { - "_id": "643d40f9dd8c607f760e5af9", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "NoneObject", - "q": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ] - }, - { - "_id": "643d40f9dd8c607f760e5afa", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "EndEffector", - "q": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ] - }, - { - "_id": "6491ad548829c1097f0151a9", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "ring", - "q": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ] - }, - { - "_id": "64f8a17c8d1b98ac5f0c6b91", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "geometry": null, - "grasp_force": 1.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "TestObject1", - "q": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ] - }, - { - "_id": "6526b75ece654b001f026d61", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "gripper_connected", - "q": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ] - }, - { - "_id": "6526da7d86c098f7fe057f61", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "tool_load", - "q": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ] - }, - { - "_id": "6530168b631a09bd8e08d211", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "tool_grasp", - "q": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ] - }, - { - "_id": "65a8008494d3e83f7e035b11", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "kios_guidance", - "q": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ] - }, - { - "_id": "65b2664c677d37a9900fd7b1", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.020523632927385155, - 0.999779614758663, - 0.0004993052070315127, - 0.0, - 0.9997626802269471, - -0.020520362759945312, - -0.00585190899106819, - 0.0, - -0.0058404858401676345, - 0.000619301067554896, - -0.99998275244693, - 0.0, - 0.49407319098460234, - -0.3343870569274952, - 0.3249152265674699, - 1.0 - ], - "O_T_TCP": [ - 0.020523632927385155, - 0.999779614758663, - 0.0004993052070315127, - 0.0, - 0.9997626802269471, - -0.020520362759945312, - -0.00585190899106819, - 0.0, - -0.0058404858401676345, - 0.000619301067554896, - -0.99998275244693, - 0.0, - 0.49407319098460234, - -0.3343870569274952, - 0.3249152265674699, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "parallel_box1", - "q": [ - -0.4889897713452055, - 0.1520400121422783, - -0.11095685046074681, - -1.8813407051080338, - 0.015587054766497403, - 2.027237653652827, - -1.3705014213786446 - ] - }, - { - "_id": "65bb9aa7666d3392110788a1", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - -0.0055492434676263965, - 0.9999687095518451, - 0.003540157533654855, - 0.0, - 0.9994642785591136, - 0.005433245086445076, - 0.0319747186594265, - 0.0, - 0.03195509884729998, - 0.003715768034059008, - -0.9994823984070837, - 0.0, - 0.5073771436733974, - 0.2821340073022507, - 0.3273610300796691, - 1.0 - ], - "O_T_TCP": [ - -0.0055492434676263965, - 0.9999687095518451, - 0.003540157533654855, - 0.0, - 0.9994642785591136, - 0.005433245086445076, - 0.0319747186594265, - 0.0, - 0.03195509884729998, - 0.003715768034059008, - -0.9994823984070837, - 0.0, - 0.5073771436733974, - 0.2821340073022507, - 0.3273610300796691, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "inward_claw", - "q": [ - -0.3558367933193034, - 0.12986394129696988, - 0.8984244877664667, - -1.9720981870183556, - -0.1313234915672495, - 2.079114055554075, - -0.1914156752901866 - ] - }, - { - "_id": "65bbb462b8aab7a7d90d58c1", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.9949942224611444, - -0.05041086343049174, - 0.08617417837347448, - 0.0, - -0.04503393357509275, - -0.9969720659778388, - -0.06324074151875426, - 0.0, - 0.08910298453945457, - 0.05904454700105643, - -0.9942707878719971, - 0.0, - 0.6906053200978064, - -0.01614832324583123, - 0.2887581517923185, - 1.0 - ], - "O_T_TCP": [ - 0.9949942224611444, - -0.05041086343049174, - 0.08617417837347448, - 0.0, - -0.04503393357509275, - -0.9969720659778388, - -0.06324074151875426, - 0.0, - 0.08910298453945457, - 0.05904454700105643, - -0.9942707878719971, - 0.0, - 0.6990701036290545, - -0.01053909128073087, - 0.1943024269444788, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "test_object1", - "q": [ - -0.5258051569263356, - 0.4877585826129244, - 0.5891767306537179, - -1.6077540129764138, - -0.23719313260094485, - 2.1022328375975285, - 0.9414903357566682 - ] - }, - { - "_id": "65bbba7c5a0f2e8e04072891", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.005516821389947922, - -0.9949443596914403, - -0.10018000150304039, - 0.0, - -0.999821715401967, - -0.003733290448562814, - -0.017981847911249182, - 0.0, - 0.017517274375192557, - 0.10026327397364804, - -0.9948067254449727, - 0.0, - 0.643320166007396, - 0.11088726408849338, - 0.3035784369329164, - 1.0 - ], - "O_T_TCP": [ - 0.005516821389947922, - -0.9949443596914403, - -0.10018000150304039, - 0.0, - -0.999821715401967, - -0.003733290448562814, - -0.017981847911249182, - 0.0, - 0.017517274375192557, - 0.10026327397364804, - -0.9948067254449727, - 0.0, - 0.6455098253042951, - 0.12342017333519939, - 0.17922759625229484, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "gear3", - "q": [ - -0.4348329623676183, - 0.39139715203497827, - 0.6694900968644402, - -1.711178953283276, - -0.16270597490334937, - 2.05581780107816, - 2.617633598559432 - ] - }, - { - "_id": "65bbbab25a0f2e8e04072892", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.9980187907769038, - -0.05571650543342081, - -0.02889482963615323, - 0.0, - -0.05609811498518191, - -0.9983365128919355, - -0.012568031790528969, - 0.0, - -0.028147058566118602, - 0.014164350073066843, - -0.9995034338515715, - 0.0, - 0.507308310840537, - -0.04026469732735698, - 0.3761685279429274, - 1.0 - ], - "O_T_TCP": [ - 0.9980187907769038, - -0.05571650543342081, - -0.02889482963615323, - 0.0, - -0.05609811498518191, - -0.9983365128919355, - -0.012568031790528969, - 0.0, - -0.028147058566118602, - 0.014164350073066843, - -0.9995034338515715, - 0.0, - 0.5046343402767557, - -0.038919084070415624, - 0.2812157017270281, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "shaft2", - "q": [ - -0.04717889234179201, - -0.13185073856974885, - -0.034486469687599886, - -2.0883627749325773, - 0.0077889278013025046, - 1.9273695102532702, - 0.7572235297293566 - ] - }, - { - "_id": "65bfcf0f5a0e8797bb01e281", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.9626021505255208, - 0.09747130791239009, - -0.2527393733447546, - 0.0, - 0.10100018601888176, - -0.9948762346088859, - 0.0009935470823088165, - 0.0, - -0.2513523930695091, - -0.026483624172027178, - -0.9675332511851704, - 0.0, - 0.4078663752332101, - 0.03825942293592056, - 0.3978052092701189, - 1.0 - ], - "O_T_TCP": [ - 0.9626021505255208, - 0.09747130791239009, - -0.2527393733447546, - 0.0, - 0.10100018601888176, - -0.9948762346088859, - 0.0009935470823088165, - 0.0, - -0.2513523930695091, - -0.026483624172027178, - -0.9675332511851704, - 0.0, - 0.4078663752332101, - 0.03825942293592056, - 0.3978052092701189, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "test", - "q": [ - 0.07570727255051596, - -0.2966134769039597, - 0.017011426452203837, - -2.1700104833456524, - 0.0015263474812373624, - 1.6179441309364524, - 0.7754216439119388 - ] - }, - { - "_id": "65ce303af045be1f60050f41", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - -0.00034130509165598697, - 0.9999088204921296, - 0.012766405683707687, - 0.0, - 0.9999006778257512, - 0.00017031581705472163, - 0.013392250644843597, - 0.0, - 0.01338911300712114, - 0.012769954401065334, - -0.9998288153066389, - 0.0, - 0.34586891787172797, - 0.28478778305517327, - 0.325633698814514, - 1.0 - ], - "O_T_TCP": [ - -0.00034130509165598697, - 0.9999088204921296, - 0.012766405683707687, - 0.0, - 0.9999006778257512, - 0.00017031581705472163, - 0.013392250644843597, - 0.0, - 0.01338911300712114, - 0.012769954401065334, - -0.9998288153066389, - 0.0, - 0.34586891787172797, - 0.28478778305517327, - 0.325633698814514, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "outward_claw", - "q": [ - -0.26322421671319607, - -0.48738894555267787, - 0.8759835133827272, - -2.392889458566437, - 0.4210987845394346, - 2.0469462890460046, - -0.42977081686632607 - ] - }, - { - "_id": "65ce3098f045be1f60050f42", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.016519502886665666, - 0.9997483636116822, - -0.014527986142256165, - 0.0, - 0.999314451509194, - -0.01603160985419312, - 0.033081133468638016, - 0.0, - 0.0328405343279425, - -0.015064800427202131, - -0.999347062382806, - 0.0, - 0.3334408867064976, - -0.33165373934959796, - 0.32422469388972297, - 1.0 - ], - "O_T_TCP": [ - 0.016519502886665666, - 0.9997483636116822, - -0.014527986142256165, - 0.0, - 0.999314451509194, - -0.01603160985419312, - 0.033081133468638016, - 0.0, - 0.0328405343279425, - -0.015064800427202131, - -0.999347062382806, - 0.0, - 0.3334408867064976, - -0.33165373934959796, - 0.32422469388972297, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "parallel_box2", - "q": [ - -0.49195081603365076, - -0.2536298767587599, - -0.28172348515657036, - -2.357436443128084, - -0.06903641326254598, - 2.1465019415994413, - -1.4987806172708702 - ] - }, - { - "_id": "65ce3113f045be1f60050f43", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.04974608360593623, - 0.9984294213293633, - -0.025392217134241925, - 0.0, - 0.9983309602704036, - -0.048970656646075854, - 0.030297119573227983, - 0.0, - 0.029006620484010686, - -0.026857516648752487, - -0.9992183393871225, - 0.0, - 0.5024405445686172, - -0.18230572536646317, - 0.357430939789341, - 1.0 - ], - "O_T_TCP": [ - 0.04974608360593623, - 0.9984294213293633, - -0.025392217134241925, - 0.0, - 0.9983309602704036, - -0.048970656646075854, - 0.030297119573227983, - 0.0, - 0.029006620484010686, - -0.026857516648752487, - -0.9992183393871225, - 0.0, - 0.5049061073097582, - -0.18458861428160714, - 0.27249738094143555, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "shaft1", - "q": [ - -0.47460197166807755, - -0.08180334212190152, - 0.12801081447643145, - -2.0952010168109023, - -0.005599479275858759, - 2.050477765321731, - -1.0803308445021476 - ] - }, - { - "_id": "65ce315df045be1f60050f44", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.06323640929784359, - 0.9978521827830034, - 0.016520434483373037, - 0.0, - 0.9977655821240671, - -0.06356378921021756, - 0.020105589547900215, - 0.0, - 0.021112914320698742, - 0.015212408525423958, - -0.9996613563981266, - 0.0, - 0.3844454340627629, - -0.034519055467682386, - 0.3511666416459713, - 1.0 - ], - "O_T_TCP": [ - 0.06323640929784359, - 0.9978521827830034, - 0.016520434483373037, - 0.0, - 0.9977655821240671, - -0.06356378921021756, - 0.020105589547900215, - 0.0, - 0.021112914320698742, - 0.015212408525423958, - -0.9996613563981266, - 0.0, - 0.38624003178002225, - -0.03322600074302135, - 0.2661954263521305, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "gearbase_hole1", - "q": [ - -0.4779728950460752, - -0.5647525487445973, - 0.34118822538716465, - -2.533109467228605, - 0.2194305768719429, - 2.0038177083303617, - -0.9983835912605101 - ] - }, - { - "_id": "65ce31d5f045be1f60050f45", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.016773866766090845, - 0.998555668652971, - 0.050852344055508855, - 0.0, - 0.999651562629023, - -0.017761257817640888, - 0.019027295007331226, - 0.0, - 0.019903398084193856, - 0.05051643648500831, - -0.9985248842114842, - 0.0, - 0.6166632788352968, - -0.06722110520041134, - 0.29656367644075515, - 1.0 - ], - "O_T_TCP": [ - 0.016773866766090845, - 0.998555668652971, - 0.050852344055508855, - 0.0, - 0.999651562629023, - -0.017761257817640888, - 0.019027295007331226, - 0.0, - 0.019903398084193856, - 0.05051643648500831, - -0.9985248842114842, - 0.0, - 0.6171608637874016, - -0.06595819428828614, - 0.27160055433546804, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "shaft3", - "q": [ - -0.6255808761370809, - 0.2721908670313347, - 0.5482196540661509, - -1.844289931665387, - -0.10189347790347204, - 2.091038903571352, - -0.8129646023995416 - ] - }, - { - "_id": "65ce3207f045be1f60050f46", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - -0.15915497945225549, - 0.9860926026332786, - -0.04766359668224283, - 0.0, - 0.9850428396108329, - 0.16183700693102337, - 0.058992662427146104, - 0.0, - 0.06588723038902841, - -0.03756243183872672, - -0.9971198205762556, - 0.0, - 0.5043739432190215, - 0.08072733253032562, - 0.2875533813662191, - 1.0 - ], - "O_T_TCP": [ - -0.15915497945225549, - 0.9860926026332786, - -0.04766359668224283, - 0.0, - 0.9850428396108329, - 0.16183700693102337, - 0.058992662427146104, - 0.0, - 0.06588723038902841, - -0.03756243183872672, - -0.9971198205762556, - 0.0, - 0.5060211239787472, - 0.07978827173435744, - 0.2626253858518127, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "gearbase_hole3", - "q": [ - -0.6320993901689848, - -0.13105960240977546, - 0.7911574711811249, - -2.301617453056469, - 0.05951137099465102, - 2.2677469761391547, - -0.8261827120045797 - ] - }, - { - "_id": "65ce326bf045be1f60050f47", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.9982800059810064, - 0.0028567036015946868, - 0.058391915573158704, - 0.0, - 0.002665300689743662, - -0.9999811917035863, - 0.003355489907203343, - 0.0, - 0.05840152737118158, - -0.003194147969025999, - -0.9982880641475511, - 0.0, - 0.38329798339116805, - -0.1792311431159989, - 0.2652977941807128, - 1.0 - ], - "O_T_TCP": [ - 0.9982800059810064, - 0.0028567036015946868, - 0.058391915573158704, - 0.0, - 0.002665300689743662, - -0.9999811917035863, - 0.003355489907203343, - 0.0, - 0.05840152737118158, - -0.003194147969025999, - -0.9982880641475511, - 0.0, - 0.38738609030715077, - -0.1794547334738307, - 0.19541762969038423, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "gear1", - "q": [ - -0.743737426383209, - -0.3330958715231419, - 0.30138807209333335, - -2.598988366910377, - 0.16539736091956653, - 2.3272367282354502, - 0.21644120803806513 - ] - }, - { - "_id": "65ce3349f045be1f60050f48", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - -0.43529958087664555, - -0.8981800970094251, - 0.06138025029245873, - 0.0, - -0.9001402451487228, - 0.43304221718444524, - -0.04693318762651431, - 0.0, - 0.01557451519504121, - -0.07568228756712969, - -0.9970103438906971, - 0.0, - 0.32523152397741284, - 0.13156707585417038, - 0.31425307836568933, - 1.0 - ], - "O_T_TCP": [ - -0.43529958087664555, - -0.8981800970094251, - 0.06138025029245873, - 0.0, - -0.9001402451487228, - 0.43304221718444524, - -0.04693318762651431, - 0.0, - 0.01557451519504121, - -0.07568228756712969, - -0.9970103438906971, - 0.0, - 0.327178338376793, - 0.12210678990827917, - 0.18962678537935218, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "gear2", - "q": [ - -0.44017739813892465, - -0.8408044241896847, - 0.7812836869963553, - -2.6489709372327193, - 0.5117495865954292, - 1.9353651503056144, - 2.7550759355656957 - ] - }, - { - "_id": "65ce3392f045be1f60050f49", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - -0.19207017045850883, - -0.9745158408444792, - -0.11579582230989756, - 0.0, - -0.9619084037954584, - 0.21032850120116167, - -0.1745705908546879, - 0.0, - 0.19448071224163638, - 0.07785667044396119, - -0.9778116339220793, - 0.0, - 0.6668309353879216, - 0.1002329166290898, - 0.3839163655785476, - 1.0 - ], - "O_T_TCP": [ - -0.19207017045850883, - -0.9745158408444792, - -0.11579582230989756, - 0.0, - -0.9619084037954584, - 0.21032850120116167, - -0.1745705908546879, - 0.0, - 0.19448071224163638, - 0.07785667044396119, - -0.9778116339220793, - 0.0, - 0.6911410244181262, - 0.10996500043458494, - 0.2616899113382877, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "no_tool", - "q": [ - -0.449013091942916, - 0.2972115466353616, - 0.6671699467458255, - -1.6625072732841806, - -0.16246066116120655, - 2.102316742226921, - 2.8067302440068085 - ] - }, - { - "_id": "65ce3399f045be1f60050f4a", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.37121811244401426, - -0.911720063290249, - 0.1759101648249346, - 0.0, - -0.9285316648543275, - -0.3639617925463895, - 0.07308561986107809, - 0.0, - -0.00260909728174289, - -0.1904725312477871, - -0.9816890584352234, - 0.0, - 0.5482232455292271, - -0.09299434620256447, - 0.4961290551977721, - 1.0 - ], - "O_T_TCP": [ - 0.37121811244401426, - -0.911720063290249, - 0.1759101648249346, - 0.0, - -0.9285316648543275, - -0.3639617925463895, - 0.07308561986107809, - 0.0, - -0.00260909728174289, - -0.1904725312477871, - -0.9816890584352234, - 0.0, - 0.5478971083690092, - -0.11680341260853785, - 0.37341792289336917, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "gearbase", - "q": [ - -0.5710980864097402, - -0.030782228559778448, - 0.4631181385266316, - -1.681660839952254, - -0.1779492470121923, - 1.6707865634759267, - 1.8844083975966868 - ] - } -] \ No newline at end of file diff --git a/kios_bt_planning/tests/mios_memory_backups/test1.json b/kios_bt_planning/tests/mios_memory_backups/test1.json deleted file mode 100644 index 5ab8177d..00000000 --- a/kios_bt_planning/tests/mios_memory_backups/test1.json +++ /dev/null @@ -1,1402 +0,0 @@ -[ - { - "_id": "643d40f9dd8c607f760e5af8", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "NullObject", - "q": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ] - }, - { - "_id": "643d40f9dd8c607f760e5af9", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "NoneObject", - "q": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ] - }, - { - "_id": "643d40f9dd8c607f760e5afa", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.9955438553058651, - 0.09228610240983252, - 0.01887999909811827, - 0.0, - 0.09234404194668681, - -0.9957150210327226, - -0.0022184922721129674, - 0.0, - 0.018594720700055672, - 0.003952137869116485, - -0.9998192921565128, - 0.0, - 0.3994408666264498, - 0.0944824439109824, - 0.49698720379356415, - 1.0 - ], - "O_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "EndEffector", - "q": [ - -0.03314445700875499, - -0.48551726122113803, - 0.20318817959603191, - -2.1170156195853562, - 0.09584772052776036, - 1.6586850308577217, - 0.8334085900995163 - ] - }, - { - "_id": "6491ad548829c1097f0151a9", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "ring", - "q": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ] - }, - { - "_id": "64f8a17c8d1b98ac5f0c6b91", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "geometry": null, - "grasp_force": 1.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "TestObject1", - "q": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ] - }, - { - "_id": "6526b75ece654b001f026d61", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "gripper_connected", - "q": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ] - }, - { - "_id": "6526da7d86c098f7fe057f61", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "tool_load", - "q": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ] - }, - { - "_id": "6530168b631a09bd8e08d211", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "tool_grasp", - "q": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ] - }, - { - "_id": "65a8008494d3e83f7e035b11", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "kios_guidance", - "q": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ] - }, - { - "_id": "65b2664c677d37a9900fd7b1", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "parallel_box1", - "q": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ] - }, - { - "_id": "65bb9aa7666d3392110788a1", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.018250590497878832, - 0.9986564285385214, - -0.04830114477289645, - 0.0, - 0.9990673900248209, - -0.016336674543731198, - 0.03972669323078785, - 0.0, - 0.03888498615373948, - -0.04898207731023214, - -0.9980424409583983, - 0.0, - 0.42411099559896687, - 0.2836677083348409, - 0.32915021716719534, - 1.0 - ], - "O_T_TCP": [ - 0.018250590497878832, - 0.9986564285385214, - -0.04830114477289645, - 0.0, - 0.9990673900248209, - -0.016336674543731198, - 0.03972669323078785, - 0.0, - 0.03888498615373948, - -0.04898207731023214, - -0.9980424409583983, - 0.0, - 0.42411099559896687, - 0.2836677083348409, - 0.32915021716719534, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "inward_claw", - "q": [ - 0.03235261634924404, - -0.13342255676201226, - 0.5591054797882165, - -2.1985537768245114, - 0.00951215328855647, - 2.090457687298454, - -0.1851527091728316 - ] - }, - { - "_id": "65bbb462b8aab7a7d90d58c1", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.9949942224611444, - -0.05041086343049174, - 0.08617417837347448, - 0.0, - -0.04503393357509275, - -0.9969720659778388, - -0.06324074151875426, - 0.0, - 0.08910298453945457, - 0.05904454700105643, - -0.9942707878719971, - 0.0, - 0.6906053200978064, - -0.01614832324583123, - 0.2887581517923185, - 1.0 - ], - "O_T_TCP": [ - 0.9949942224611444, - -0.05041086343049174, - 0.08617417837347448, - 0.0, - -0.04503393357509275, - -0.9969720659778388, - -0.06324074151875426, - 0.0, - 0.08910298453945457, - 0.05904454700105643, - -0.9942707878719971, - 0.0, - 0.6990701036290545, - -0.01053909128073087, - 0.1943024269444788, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "test_object1", - "q": [ - -0.5258051569263356, - 0.4877585826129244, - 0.5891767306537179, - -1.6077540129764138, - -0.23719313260094485, - 2.1022328375975285, - 0.9414903357566682 - ] - }, - { - "_id": "65bbba7c5a0f2e8e04072891", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.9073993187547345, - 0.41985945309725825, - -0.018035045592499493, - 0.0, - 0.42021897763524346, - -0.9069959355128604, - 0.02747964154697724, - 0.0, - -0.004820218582108226, - -0.03251430244091363, - -0.9994596468240234, - 0.0, - 0.6788377911570959, - 0.04934718725774592, - 0.38332340671214893, - 1.0 - ], - "O_T_TCP": [ - 0.9073993187547345, - 0.41985945309725825, - -0.018035045592499493, - 0.0, - 0.42021897763524346, - -0.9069959355128604, - 0.02747964154697724, - 0.0, - -0.004820218582108226, - -0.03251430244091363, - -0.9994596468240234, - 0.0, - 0.6783798703917956, - 0.04625832852585913, - 0.28837474026386667, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "gear3", - "q": [ - -0.0042451985391477745, - 0.4112671838536627, - 0.11089969751477464, - -1.3619879786173503, - -0.07757619036237953, - 1.76245303940773, - 0.46454000569612647 - ] - }, - { - "_id": "65bbbab25a0f2e8e04072892", - "OB_I": [ - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0, - 0.0 - ], - "OB_T_TCP": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "OB_T_gp": [ - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0, - 0.0, - 0.0, - 0.0, - 0.0, - 1.0 - ], - "O_T_OB": [ - 0.9906790719863731, - 0.13202813538348201, - -0.03323092981828252, - 0.0, - 0.1334570281545639, - -0.9900112536755228, - 0.04525136609186127, - 0.0, - -0.026925059392365894, - -0.04926543101457196, - -0.9984227353599607, - 0.0, - 0.5092485546122931, - -0.03161534677996447, - 0.37666630153596137, - 1.0 - ], - "O_T_TCP": [ - 0.9906790719863731, - 0.13202813538348201, - -0.03323092981828252, - 0.0, - 0.1334570281545639, - -0.9900112536755228, - 0.04525136609186127, - 0.0, - -0.026925059392365894, - -0.04926543101457196, - -0.9984227353599607, - 0.0, - 0.5066906739700183, - -0.036295562726348804, - 0.2818161416767651, - 1.0 - ], - "geometry": null, - "grasp_force": 0.0, - "grasp_width": 0.0, - "mass": 0.0, - "name": "shaft2", - "q": [ - -0.004074829769631227, - -0.13090544894912237, - -0.03858465880018257, - -2.087356446081556, - -0.05922271298037634, - 1.9311111306349433, - 0.6314789032074318 - ] - } -] \ No newline at end of file diff --git a/kios_bt_planning/tests/mios_test.py b/kios_bt_planning/tests/mios_test.py deleted file mode 100644 index 591aff28..00000000 --- a/kios_bt_planning/tests/mios_test.py +++ /dev/null @@ -1,276 +0,0 @@ -""" -test the functionality of the tree and the communication with the robot -""" - -import kios_bt.action_nodes as action_nodes -import kios_bt.condition_nodes as condition_nodes -import py_trees -import functools - - -class TreeRoot(py_trees.composites.Sequence): - "create the tree root." - - def __init__(self, name: str = "root", memory: bool = False): - super().__init__(name, memory) - - def set_initial_state(self, initial_state: dict): - try: - for key, value in initial_state.items(): - py_trees.blackboard.Blackboard().set(variable_name=key, value=value) - except KeyError: - print("KeyError: %s" % (key)) - - -def create_condition_test_tree() -> py_trees.behaviour.Behaviour: - """create condition test tree.""" - selector = py_trees.composites.Selector(name="load the tool", memory=False) - - is_tool_load = condition_nodes.IsToolLoaded(name="is_tool_loaded") - - sub_sequence = py_trees.composites.Sequence(name="Task Sequence", memory=False) - - hand_have = condition_nodes.isInHand(name="does_hand_have") - - load_tool = action_nodes.ToolLoad(name="load_tool") - - sub_sequence.add_children([hand_have, load_tool]) - - selector.add_children([is_tool_load, sub_sequence]) - - return selector - - -def post_tick_handler( - snapshot_visitor: py_trees.visitors.SnapshotVisitor, - behaviour_tree: py_trees.trees.BehaviourTree, -) -> None: - """ - Print data about the part of the tree visited. - - Args: - snapshot_handler: gather data about the part of the tree visited - behaviour_tree: tree to gather data from - """ - print( - "\n" - + py_trees.display.unicode_tree( - root=behaviour_tree.root, - visited=snapshot_visitor.visited, - previously_visited=snapshot_visitor.previously_visited, - ) - ) - print(py_trees.display.unicode_blackboard()) - - -def pre_tick_handler(behaviour_tree: py_trees.trees.BehaviourTree) -> None: - """Print a banner with current tick count prior to ticking the tree. - - Args: - behaviour_tree: the tree to tick (used to fetch the count number) - """ - print("\n--------- Run %s ---------\n" % behaviour_tree.count) - - -################################################################################################### -def test_blackboard(): - root = TreeRoot() - initial_state = { - "inHand": "Tool1", - } - root.set_initial_state(initial_state) - - is_in_hand = condition_nodes.isInHand(name="is_in_hand") - root.add_children([is_in_hand]) - root.setup() - print("inHand: %s" % (py_trees.blackboard.Blackboard().get(variable_name="inHand"))) - py_trees.blackboard.Blackboard().set(variable_name="inHand", value="Tool2") - print("inHand: %s" % (py_trees.blackboard.Blackboard().get(variable_name="inHand"))) - - root_2 = TreeRoot() - initial_state_2 = { - "inHand": "Tool1", - } - root_2.set_initial_state(initial_state_2) - is_in_hand_2 = condition_nodes.isInHand(name="is_in_hand_2") - root_2.add_children([is_in_hand_2]) - root_2.setup() - print("inHand: %s" % (py_trees.blackboard.Blackboard().get(variable_name="inHand"))) - py_trees.blackboard.Blackboard().set(variable_name="inHand", value="Tool2") - print("inHand: %s" % (py_trees.blackboard.Blackboard().get(variable_name="inHand"))) - # * the blackboard is a global variable - # * so different trees are using the same blackboard - - -################################################################################################### -def test_tree(): - root = TreeRoot() - initial_state = { - "inHand": "nothing", - } - root.set_initial_state(initial_state) - selector = py_trees.composites.Selector(name="load the tool", memory=False) - is_tool_load = condition_nodes.IsToolLoaded(name="is_tool_loaded") - sub_sequence = py_trees.composites.Sequence(name="Task Sequence", memory=False) - in_hand = condition_nodes.isInHand(name="in_hand") - load_tool = action_nodes.ToolLoadTest(name="load_tool") - sub_sequence.add_children([in_hand, load_tool]) - selector.add_children([is_tool_load, sub_sequence]) - root.add_children([selector]) - root.setup() - - #################### - # Tree Stewardship - #################### - behaviour_tree = py_trees.trees.BehaviourTree(root) - behaviour_tree.add_pre_tick_handler(pre_tick_handler) - behaviour_tree.visitors.append(py_trees.visitors.DebugVisitor()) - snapshot_visitor = py_trees.visitors.SnapshotVisitor() - behaviour_tree.add_post_tick_handler( - functools.partial(post_tick_handler, snapshot_visitor) - ) - behaviour_tree.visitors.append(snapshot_visitor) - behaviour_tree.setup(timeout=15) - - #################### - # Tick Tock - #################### - - while True: - try: - behaviour_tree.tick() - py_trees.console.read_single_keypress() - except KeyboardInterrupt: - break - - print("\n") - - -def create_load_tool_tree() -> py_trees.composites.Selector: - selector = py_trees.composites.Selector(name="load the tool", memory=False) - - is_tool_load = condition_nodes.IsToolLoaded(["tool1"]) - - sequence = py_trees.composites.Sequence(name="load tool sequence", memory=False) - - hand_free = condition_nodes.isInHand(["nothing"]) - load_tool = action_nodes.ToolLoad(["tool1"]) - sequence.add_children([hand_free, load_tool]) - - selector.add_children([is_tool_load, sequence]) - - return selector - -def test_tree(): - """ - tree with real actions to test the tree functionality and the communication with the robot - """ - root = TreeRoot() - initial_state = { - "inHand": "nothing", - "inTool": "nothing", - "cube1-at": "cube1", - } - root.set_initial_state(initial_state) - - selector = py_trees.composites.Selector(name="pick up cube selector", memory=False) - - is_cube_in_hand = condition_nodes.isInTool(["cube1"]) - - sequence = py_trees.composites.Sequence(name="pick up cube sequence", memory=False) - is_tool_load = create_load_tool_tree() - tool_free = condition_nodes.isInTool(["nothing"]) - pick_up_cube = action_nodes.ToolPick(["cube1"]) - sequence.add_children([is_tool_load, tool_free, pick_up_cube]) - - selector.add_children([is_cube_in_hand, sequence]) - - root.add_children([selector]) - - root.setup() - - #################### - # Tree Stewardship - #################### - behaviour_tree = py_trees.trees.BehaviourTree(root) - behaviour_tree.add_pre_tick_handler(pre_tick_handler) - behaviour_tree.visitors.append(py_trees.visitors.DebugVisitor()) - snapshot_visitor = py_trees.visitors.SnapshotVisitor() - behaviour_tree.add_post_tick_handler( - functools.partial(post_tick_handler, snapshot_visitor) - ) - behaviour_tree.visitors.append(snapshot_visitor) - behaviour_tree.setup(timeout=15) - - #################### - # Tick Tock - #################### - # py_trees.display.render_dot_tree(root, with_blackboard_variables=True) - - while True: - try: - behaviour_tree.tick() - py_trees.console.read_single_keypress() - except KeyboardInterrupt: - break - - print("\n") - - -def test_fake_action(): - """ - tree with fake actions to test the tree functionality - """ - - root = TreeRoot("root", memory=False) - initial_state = { - "inHand": "nothing", - "inTool": "nothing", - "cube1-at": "cube1", - } - root.set_initial_state(initial_state) - - sequence = py_trees.composites.Sequence(name="pick up cube sequence", memory=False) - - load_tool = action_nodes.ToolLoadTest(["tool1"]) - - load_tool2 = action_nodes.ToolLoadTest(["tool2"]) - - sequence.add_children([load_tool, load_tool2]) - - root.add_children([sequence]) - - #################### - # Tree Stewardship - #################### - behaviour_tree = py_trees.trees.BehaviourTree(root) - behaviour_tree.add_pre_tick_handler(pre_tick_handler) - behaviour_tree.visitors.append(py_trees.visitors.DebugVisitor()) - snapshot_visitor = py_trees.visitors.SnapshotVisitor() - behaviour_tree.add_post_tick_handler( - functools.partial(post_tick_handler, snapshot_visitor) - ) - behaviour_tree.visitors.append(snapshot_visitor) - behaviour_tree.setup(timeout=15) - - #################### - # Tick Tock - #################### - # py_trees.display.render_dot_tree(root, with_blackboard_variables=True) - - while True: - try: - behaviour_tree.tick() - py_trees.console.read_single_keypress() - except KeyboardInterrupt: - break - - print("\n") - - -if __name__ == "__main__": - # test_blackboard() - py_trees.logging.level = py_trees.logging.Level.DEBUG - test_tree() - - # print("\n") diff --git a/kios_bt_planning/tests/multiprocess_test.py b/kios_bt_planning/tests/multiprocess_test.py deleted file mode 100644 index f4885257..00000000 --- a/kios_bt_planning/tests/multiprocess_test.py +++ /dev/null @@ -1,42 +0,0 @@ -from multiprocessing import Process, Manager -from time import sleep - - -class Task: - def __init__(self, shared_data): - self.shared_data = shared_data - - def update(self, value): - self.shared_data["data"] = value - - def get_data(self): - return self.shared_data["data"] - - -def worker(shared_task): - for value in range(5, 21, 5): - sleep(5) - shared_task.update(value) - print(f"Subprocess: {shared_task.get_data()}") - - -if __name__ == "__main__": - with Manager() as manager: - # Create a shared dictionary - shared_data = manager.dict({"data": 0}) - - # Create a shared version of Task - shared_task = Task(shared_data) - - # Create and start the subprocess - p = Process(target=worker, args=(shared_task,)) - p.start() - - while shared_task.get_data() < 20: - print(f"Main process: {shared_task.get_data()}") - sleep(1) - - p.join() - - # Final value in the main process - print(f"Main process final value: {shared_task.get_data()}") diff --git a/kios_bt_planning/tests/neo4j_test.py b/kios_bt_planning/tests/neo4j_test.py deleted file mode 100644 index c5b55d0a..00000000 --- a/kios_bt_planning/tests/neo4j_test.py +++ /dev/null @@ -1,140 +0,0 @@ -from neo4j import GraphDatabase - -from typing import List, Dict, Any - - -class Neo4jInterface: - """ - Interface for the Neo4j database - """ - - def __init__(self, uri: str, user: str, password: str): - self.driver = GraphDatabase.driver(uri, auth=(user, password)) - - def _clear_database(self, tx): - tx.run("MATCH (n) DETACH DELETE n") - - def clear_database(self): - with self.driver.session() as session: - session.write_transaction(self._clear_database) - - def close_driver(self): - self.driver.close() - - def _create_objects(self, tx, object_names: list): - for obj in object_names: - tx.run(f"CREATE (o:Object {{name: '{obj}'}})") - - def create_objects(self, object_names: list): - with self.driver.session() as session: - session.write_transaction(self._create_objects, object_names) - - def _create_relations(self, tx, relations: Dict[str, List[str]]): - """ - create relations between objects, if objects do not exist, create them. - """ - for relation, objects in relations.items(): - for obj in objects: - result_from = tx.run(f"MATCH (o:Object {{name: '{obj[0]}'}}) RETURN o") - result_to = tx.run(f"MATCH (o:Object {{name: '{obj[1]}'}}) RETURN o") - if result_from.single() and result_to.single(): - tx.run( - f"MATCH (from:Object {{name: '{obj[0]}'}}), (to:Object {{name: '{obj[1]}'}}) " - f"CREATE (from)-[:{relation.upper()}]->(to)" - ) - else: - # raise ValueError( - # f"Object {obj} does not exist in the database!" - # ) - if not result_from.single(): - self.create_objects([obj[0]]) - if not result_to.single(): - self.create_objects([obj[1]]) - tx.run( - f"MATCH (from:Object {{name: '{obj[0]}'}}), (to:Object {{name: '{obj[1]}'}}) " - f"CREATE (from)-[:{relation.upper()}]->(to)" - ) - - def create_relations(self, relations: Dict[str, List[str]]): - with self.driver.session() as session: - session.write_transaction(self._create_relations, relations) - - def fetch_all(self): - with self.driver.session() as session: - result = session.run("MATCH (from)-[r]->(to) RETURN from.name, r, to.name") - return [ - (record["from.name"], record["r"].type, record["to.name"]) - for record in result - ] - - def fetch_nodes(self): - with self.driver.session() as session: - result = session.run("MATCH (o:Object) RETURN o.name") - return [record["o.name"] for record in result] - - def fetch_relations(self): - with self.driver.session() as session: - result = session.run("MATCH (from)-[r]->(to) RETURN from.name, r, to.name") - return [ - (record["from.name"], record["r"].type, record["to.name"]) - for record in result - ] - - def fetch_specific(self, node_names: List[str]): - """ - fetch specific nodes and relations from the database - """ - with self.driver.session() as session: - result = session.run( - f"MATCH (from)-[r]->(to) WHERE from.name IN {node_names} AND to.name IN {node_names} RETURN from.name, r, to.name" - ) - return [ - (record["from.name"], record["r"].type, record["to.name"]) - for record in result - ] - - def remove_nodes(self, node_names: List[str]): - """ - remove specific nodes from the database - """ - with self.driver.session() as session: - for node_name in node_names: - session.run(f"MATCH (o:Object {{name: '{node_name}'}}) DETACH DELETE o") - - def remove_relations(self, relations: Dict[str, List[str]]): - """ - remove specific relations from the database - """ - with self.driver.session() as session: - for relation, objects in relations.items(): - for obj in objects: - session.run( - f"MATCH (from:Object {{name: '{obj[0]}'}})-[r:{relation.upper()}]->(to:Object {{name: '{obj[1]}'}}) DELETE r" - ) - - def print_all(self): - with self.driver.session() as session: - result = session.run("MATCH (from)-[r]->(to) RETURN from, r, to") - for record in result: - print( - f"{record['from']['name']} - {record['r'].type} - {record['to']['name']}" - ) - - -def test_neo4j(): - neo4j_interface = Neo4jInterface( - uri="neo4j://localhost:7687", user="neo4j", password="14637982" - ) - neo4j_interface.clear_database() - neo4j_interface.create_objects(["a", "b", "c", "d", "e", "f"]) - relations = { - "on": [("a", "b"), ("b", "c"), ("c", "d"), ("d", "e"), ("e", "f")], - "next_to": [("a", "c"), ("c", "e"), ("b", "d"), ("d", "f")], - } - neo4j_interface.create_relations(relations) - neo4j_interface.print_all() - neo4j_interface.close_driver() - - -if __name__ == "__main__": - test_neo4j() diff --git a/kios_bt_planning/tests/networkx_test.py b/kios_bt_planning/tests/networkx_test.py deleted file mode 100644 index 12e97347..00000000 --- a/kios_bt_planning/tests/networkx_test.py +++ /dev/null @@ -1,37 +0,0 @@ -import networkx as nx - -# Create a new directed graph -G = nx.DiGraph() - -# Add nodes -G.add_node("table") -G.add_node("ball") -G.add_node("cabinet") - -# Add property to cabinet -G.nodes["cabinet"]["open"] = True - -# Add relation (edge) from ball to table -G.add_edge("ball", "table", relation="on_something") - - -# Function to check if specific relation exists -def check_relation(graph, source, target, relation): - if graph.has_edge(source, target): - return graph[source][target].get("relation") == relation - return False - - -# Check if 'ball' is 'on_something' 'table' -relation_exists = check_relation(G, "ball", "table", "on_something") -print("Relation exists:", relation_exists) # Output: True - - -# Function to check if a node has a specific property -def check_property(graph, node, property_key): - return graph.nodes[node].get(property_key) is not None - - -# Check if 'cabinet' has property 'open' -property_exists = check_property(G, "cabinet", "open") -print("Cabinet has 'open' property:", property_exists) # Output: True diff --git a/kios_bt_planning/tests/pddl_interface_test.py b/kios_bt_planning/tests/pddl_interface_test.py deleted file mode 100644 index 274453f4..00000000 --- a/kios_bt_planning/tests/pddl_interface_test.py +++ /dev/null @@ -1,48 +0,0 @@ -from kios_domain.pddl_interface import PddlInterface - -# from kios_domain.problem1 import problem as problem1 - -# from kios_domain.problem2 import problem as problem2 - -# from kios_domain.problem3 import problem as problem3 -from kios_domain.problem4 import problem as problem4 - -import os - -""" -use this to generate the domain and problem files. -""" - - -def test_from_up(): - print("test_from_up ----------------------------------") - pddl_interface = PddlInterface() - - pddl_interface.setup_from_up(problem4) - - pddl_interface.write_pddl() - - pddl_interface.print_domain() - - pddl_interface.print_problem() - - -def test_from_pddl(): # ! this suffers from the domain-problem inconsistency problem now - print("test_from_pddl ----------------------------------") - - pddl_interface = PddlInterface() - - pddl_interface.setup_from_pddl( - domain_file_name="domain.pddl", problem_file_name="problem.pddl" - ) - - pddl_interface.print_domain() - - pddl_interface.print_problem() - - print(pddl_interface.get_pddl_string()) - - -if __name__ == "__main__": - test_from_up() - # test_from_pddl() diff --git a/kios_bt_planning/tests/pytrees_blackboard_test.py b/kios_bt_planning/tests/pytrees_blackboard_test.py deleted file mode 100644 index 0366c475..00000000 --- a/kios_bt_planning/tests/pytrees_blackboard_test.py +++ /dev/null @@ -1,10 +0,0 @@ -""" -test the functionality of the blackboard in pytrees -""" -import py_trees - -blackboard = py_trees.blackboard.Client(name="Client") -blackboard.register_key(key="foo", access=py_trees.common.Access.WRITE) -blackboard.register_key(key="bar", access=py_trees.common.Access.READ) -blackboard.foo = [["entity1", "entity2"], ["entity3", "entity4"]] -print(blackboard) diff --git a/kios_bt_planning/tests/robot_interface_test.py b/kios_bt_planning/tests/robot_interface_test.py deleted file mode 100644 index 7fdeeae7..00000000 --- a/kios_bt_planning/tests/robot_interface_test.py +++ /dev/null @@ -1,277 +0,0 @@ -import numpy as np -from spatialmath import * -from spatialmath.base import trnorm - - -from kios_robot.mios_task_factory import MiosTaskFactory -from kios_robot.robot_command import RobotCommand -from kios_robot.robot_interface import RobotInterface -from kios_robot.data_types import Toolbox -from kios_scene.scene_factory import SceneFactory - -from kios_scene.mios_ltm_manipulator import LangTermMemoryManipulator - -ri = RobotInterface() -sf = SceneFactory() -scene = sf.create_test_scene() -ri.setup_scene(scene) - -ltm_manipulator = LangTermMemoryManipulator() - - -# def teach_object(object: str): -# ri.proprioceptor.teach_object(object) -def backup_mios_environment(backup_name: str): - ltm_manipulator.backup_mios_environment(backup_name) - - -def clear_mios_environment(): - ltm_manipulator.clear_mios_environment() - - -def show_backups(): - ltm_manipulator.show_backups() - - -def restore_to_mios_environment(backup_name: str): - ltm_manipulator.restore_to_mios_environment(backup_name) - - -def teach_object_TCP(object_name: str): - ri.proprioceptor.teach_object_TCP(object_name) - ri.proprioceptor.update_scene_object_from_mios(scene=scene, object_name=object_name) - - -def get_object(object: str): - ri.proprioceptor.get_object(object) - - -def set_tool(toolbox_name: str): - if "robot_command" in locals(): - print("robot command is set") - print(f"the robot command's tasks are {robot_command.task_list}") - else: - print("robot command is not set") - - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - shared_data=None, - task_scene=scene, - robot_interface=ri, - ) - print(f"the number of tasks in the task list is {len(robot_command.task_list)}") - robot_command.add_task(ri.mios_task_factory.generate_update_tool_call(toolbox_name)) - robot_command.execute_task_list_sync() - robot_command.clear_tasks() - - -def test_cartesian_joint_move(): - # need object "test" - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - shared_data=None, - task_scene=scene, - robot_interface=ri, - ) - - O_T_TCP = scene.get_object("test").O_T_TCP - print(O_T_TCP) - print(type(O_T_TCP)) - print(O_T_TCP.shape) - - robot_command.add_task( - ri.mios_task_factory.generate_cartesian_move_mp(O_T_TCP=O_T_TCP) - ) - - # rotate in z axis by 90 degrees - # rotation_90 = SE3.Rz(np.deg2rad(90)) - - # rotate in x axis by 90 degrees - rotation_90 = SE3.Rx(np.deg2rad(90)) - - se_O_T_TCP = SE3(trnorm(O_T_TCP)) - result = se_O_T_TCP * rotation_90 - O_T_TCP = result.A - - robot_command.add_task( - ri.mios_task_factory.generate_cartesian_move_mp(O_T_TCP=O_T_TCP) - ) - - robot_command.execute_task_list_sync() - - -def test_screw_in(object_name: str): - # need object "test" - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - shared_data=None, - task_scene=scene, - robot_interface=ri, - ) - - O_T_OB = scene.get_object(object_name).O_T_TCP - - print(O_T_OB) - - robot_command.add_task(ri.mios_task_factory.generate_screw_in_mp(O_T_OB=O_T_OB)) - - robot_command.execute_task_list_sync() - - -def tool_test(): - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - shared_data=None, - task_scene=scene, - robot_interface=ri, - ) - - robot_command.add_mios_task( - ri.mios_task_factory.generate_cartesian_move_mp("test_location") - ) - robot_command.add_mios_task( - ri.mios_task_factory.generate_update_tool_call("parallel_box1") - ) - robot_command.add_mios_task( - ri.mios_task_factory.generate_cartesian_move_mp("test_location") - ) - - robot_command.execute_task_list_sync() - -def load_tool_test(tool_name: str): - parsed_action = { - "action_name": "load_tool", - "args": ["left_hand", tool_name], - } - - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - task_scene=scene, - shared_data=None, - robot_interface=ri, - ) - - robot_command.add_tasks(ri.mios_task_factory.generate_load_tool_skill(parsed_action)) - - robot_command.execute_task_list_sync() - -def pick_test(): - sf = SceneFactory() - scene = sf.create_test_scene() - ri.setup_scene(scene) - - parsed_action = { - "action_name": "pick", - "args": [None, None, "test_object"], - } - - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - shared_data=None, - task_scene=scene, - robot_interface=ri, - ) - - robot_command.add_mios_task(ri.mios_task_factory.generate_gripper_home_mp()) - - pick_up_tasks = ri.mios_task_factory.generate_pick_up_skill( - parsed_action=parsed_action - ) - for task in pick_up_tasks: - robot_command.add_task(task) - - robot_command.task_list.append(ri.mios_task_factory.generate_gripper_release_mp()) - - robot_command.execute_task_list_sync() - - -def move_gripper(width: float): - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - task_scene=scene, - shared_data=None, - robot_interface=ri, - ) - - robot_command.add_task(ri.mios_task_factory.generate_gripper_move_mp(width=width)) - - robot_command.execute_task_list_sync() - - -def home_gripper(): - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - task_scene=scene, - shared_data=None, - robot_interface=ri, - ) - - robot_command.add_task(ri.mios_task_factory.generate_gripper_home_mp()) - - robot_command.execute_task_list_sync() - - -def cartesian_move(object: str): - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - task_scene=scene, - shared_data=None, - robot_interface=ri, - ) - - robot_command.add_task(ri.mios_task_factory.generate_cartesian_move_mp(object)) - - robot_command.execute_task_list_sync() - - -def insert_test(object_name: str): - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - task_scene=scene, - shared_data=None, - robot_interface=ri, - ) - - insert_action = { - "action_name": "insert", - "args": [None, None, "gear3", object_name], - } - - robot_command.add_tasks(ri.mios_task_factory.generate_insert_skill(insert_action)) - - robot_command.execute_task_list_sync() - - -def drive_test(): - robot_command = RobotCommand( - robot_address="127.0.0.1", - robot_port=12000, - task_scene=scene, - shared_data=None, - robot_interface=ri, - ) - - drive_action = { - "action_name": "drive", - "args": [None, None, None, None], - } - - robot_command.add_tasks(ri.mios_task_factory.generate_drive_skill(drive_action)) - - robot_command.execute_task_list_sync() - - -if __name__ == "__main__": - # tool_test() - # pick_test() - pass diff --git a/kios_bt_planning/tests/scene_test.py b/kios_bt_planning/tests/scene_test.py deleted file mode 100644 index 5391657a..00000000 --- a/kios_bt_planning/tests/scene_test.py +++ /dev/null @@ -1,102 +0,0 @@ -from kios_scene.scene_factory import SceneFactory - - -def test_scene_factory(): - sf = SceneFactory() - - # scene_json = { - # "reference_objects": [ - # { - # "object_name": "base", - # "source": "pre-defined", # can be mios, vision, or pre-defined - # "joint_pose": [0, 0, 0, 0, 0, 0, 0], - # "O_T_EE": [ - # [1, 0, 0, 0.5], - # [0, 1, 0, 0.5], - # [0, 0, 1, 0.5], - # [0, 0, 0, 1], - # ], - # "reference_object": None, - # }, - # { - # "object_name": "another_base", - # "source": "mios", # means the object is from mios mongodb, usually a location that is taught. - # "key": "another_base", # the key to find the object in the mongodb - # }, - # { - # "object_name": "a_third_base", - # "source": "vision", # means the object is from vision detection - # "key": "a_third_base", # the key to find the object in the detection list. the location is preprocessed before being accessed. - # }, - # ], - # "relative_objects": { - # "shaft1_hole": { - # "reference_object": "base", - # "relative_joint_pose": None, - # "relative_HT": [ - # [1, 0, 0, 0], - # [0, 1, 0, 0], - # [0, 0, 1, -0.2], - # [0, 0, 0, 1], - # ], - # "relative_cartesian_pose": None, - # }, - # "shaft2": { - # "reference_object": "base", - # "relative_joint_pose": [0, 0, 0, 0, 0, 0, 0], - # "relative_HT": None, - # "relative_cartesian_pose": [0, 0, 0, 0, 0, 0.2], - # }, - # }, - # } - - scene_json = { - "reference_objects": [ - { - "object_name": "base", - "source": "pre-defined", # can be mios, vision, or pre-defined - "joint_pose": [0, 0, 0, 0, 0, 0, 0], - "O_T_EE": [ - [1, 0, 0, 0.5], - [0, 1, 0, 0.5], - [0, 0, 1, 0.5], - [0, 0, 0, 1], - ], - "reference_object": None, - }, - { - "object_name": "parallel_box1", - "source": "mios", # means the object is from mios mongodb, usually a location that is taught. - "key": "parallel_box1", # the key to find the object in the mongodb - }, - { - "object_name": "inward_craw", - "source": "mios", - "key": "inward_craw", - }, - { - "object_name": "outward_craw", - "source": "mios", - "key": "outward_craw", - }, - ], - "relative_objects": { - # "shaft1_hole": { - # "reference_object": "base", - # "relative_joint_pose": None, - # "relative_HT": [ - # [1, 0, 0, 0], - # [0, 1, 0, 0], - # [0, 0, 1, -0.2], - # [0, 0, 0, 1], - # ], - # "relative_cartesian_pose": None, - # }, - # "shaft2": { - # "reference_object": "base", - # "relative_joint_pose": [0, 0, 0, 0, 0, 0, 0], - # "relative_HT": None, - # "relative_cartesian_pose": [0, 0, 0, 0, 0, 0.2], - # }, - }, - } diff --git a/kios_bt_planning/tests/visualization_debug.py b/kios_bt_planning/tests/visualization_debug.py deleted file mode 100644 index 2634485f..00000000 --- a/kios_bt_planning/tests/visualization_debug.py +++ /dev/null @@ -1,195 +0,0 @@ -# -# License: BSD -# https://raw.githubusercontent.com/splintered-reality/py_trees/devel/LICENSE -# -############################################################################## -# Documentation -############################################################################## - -""" -A py_trees demo. - -.. argparse:: - :module: py_trees.demos.dot_graphs - :func: command_line_argument_parser - :prog: py-trees-demo-dot-graphs - -.. graphviz:: dot/demo-dot-graphs.dot - -""" - -############################################################################## -# Imports -############################################################################## - -import argparse -import subprocess -import typing - -import py_trees -import py_trees.console as console - -############################################################################## -# Classes -############################################################################## - - -def description() -> str: - """ - Print description and usage information about the program. - - Returns: - the program description string - """ - name = "py-trees-demo-dot-graphs" - content = "Renders a dot graph for a simple tree, with blackboxes.\n" - if py_trees.console.has_colours: - banner_line = console.green + "*" * 79 + "\n" + console.reset - s = banner_line - s += console.bold_white + "Dot Graphs".center(79) + "\n" + console.reset - s += banner_line - s += "\n" - s += content - s += "\n" - s += console.white - s += console.bold + " Generate Full Dot Graph" + console.reset + "\n" - s += "\n" - s += console.cyan + " {0}".format(name) + console.reset + "\n" - s += "\n" - s += console.bold + " With Varying Visibility Levels" + console.reset + "\n" - s += "\n" - s += ( - console.cyan - + " {0}".format(name) - + console.yellow - + " --level=all" - + console.reset - + "\n" - ) - s += ( - console.cyan - + " {0}".format(name) - + console.yellow - + " --level=detail" - + console.reset - + "\n" - ) - s += ( - console.cyan - + " {0}".format(name) - + console.yellow - + " --level=component" - + console.reset - + "\n" - ) - s += ( - console.cyan - + " {0}".format(name) - + console.yellow - + " --level=big_picture" - + console.reset - + "\n" - ) - s += "\n" - s += banner_line - else: - s = content - return s - - -def epilog() -> typing.Optional[str]: - """ - Print a noodly epilog for --help. - - Returns: - the noodly message - """ - if py_trees.console.has_colours: - return ( - console.cyan - + "And his noodly appendage reached forth to tickle the blessed...\n" - + console.reset - ) - else: - return None - - -def command_line_argument_parser() -> argparse.ArgumentParser: - """ - Process command line arguments. - - Returns: - the argument parser - """ - parser = argparse.ArgumentParser( - description=description(), - epilog=epilog(), - formatter_class=argparse.RawDescriptionHelpFormatter, - ) - parser.add_argument( - "-l", - "--level", - action="store", - default="fine_detail", - choices=["all", "fine_detail", "detail", "component", "big_picture"], - help="visibility level", - ) - return parser - - -def create_tree(level: str) -> py_trees.behaviour.Behaviour: - """ - Create the root behaviour and it's subtree. - - Returns: - the root behaviour - """ - root = py_trees.composites.Selector(name="Demo Dot Graphs %s" % level, memory=False) - first_blackbox = py_trees.composites.Sequence(name="BlackBox 1", memory=True) - first_blackbox.add_child(py_trees.behaviours.Running("Worker")) - first_blackbox.add_child(py_trees.behaviours.Running("Worker")) - first_blackbox.add_child(py_trees.behaviours.Running("Worker")) - # first_blackbox.blackbox_level = py_trees.common.BlackBoxLevel.BIG_PICTURE - second_blackbox = py_trees.composites.Sequence(name="Blackbox 2", memory=True) - second_blackbox.add_child(py_trees.behaviours.Running("Worker")) - second_blackbox.add_child(py_trees.behaviours.Running("Worker")) - second_blackbox.add_child(py_trees.behaviours.Running("Worker")) - # second_blackbox.blackbox_level = py_trees.common.BlackBoxLevel.COMPONENT - third_blackbox = py_trees.composites.Sequence(name="Blackbox 3", memory=True) - third_blackbox.add_child(py_trees.behaviours.Running("Worker")) - third_blackbox.add_child(py_trees.behaviours.Running("Worker")) - third_blackbox.add_child(py_trees.behaviours.Running("Worker")) - third_blackbox.blackbox_level = py_trees.common.BlackBoxLevel.DETAIL - root.add_child(first_blackbox) - root.add_child(second_blackbox) - first_blackbox.add_child(third_blackbox) - return root - - -############################################################################## -# Main -############################################################################## - - -def main() -> None: - """Entry point for the demo script.""" - py_trees.logging.level = py_trees.logging.Level.DEBUG - - root = create_tree("TEST") - py_trees.display.render_dot_tree(root, py_trees.common.VisibilityLevel.DETAIL) - - if py_trees.utilities.which("xdot"): - try: - subprocess.call(["xdot", "demo_dot_graphs_%s.dot" % "DETAIL"]) - except KeyboardInterrupt: - pass - else: - print("") - console.logerror( - "No xdot viewer found, skipping display [hint: sudo apt install xdot]" - ) - print("") - - -if __name__ == "__main__": - main() diff --git a/kios_bt_planning/tree_root.png b/kios_bt_planning/tree_root.png deleted file mode 100644 index ae23ec3d..00000000 Binary files a/kios_bt_planning/tree_root.png and /dev/null differ diff --git a/kios_bt_planning/tree_root.py b/kios_bt_planning/tree_root.py deleted file mode 100644 index dfedc491..00000000 --- a/kios_bt_planning/tree_root.py +++ /dev/null @@ -1,382 +0,0 @@ -import kios_bt.action_nodes as action_nodes -import kios_bt.condition_nodes as condition_nodes -import py_trees -import functools - - -class TreeRoot(py_trees.composites.Sequence): - "create the tree root." - - def __init__(self): - super().__init__(name="tree_root", memory=False) - - def set_initial_state(self, initial_state: dict): - try: - for key, value in initial_state.items(): - py_trees.blackboard.Blackboard().set(variable_name=key, value=value) - except KeyError: - print("KeyError: %s" % (key)) - - -def create_condition_test_tree() -> py_trees.behaviour.Behaviour: - """create condition test tree.""" - selector = py_trees.composites.Selector(name="load the tool", memory=False) - - is_tool_load = condition_nodes.IsToolLoaded(name="is_tool_loaded") - - sub_sequence = py_trees.composites.Sequence(name="Task Sequence", memory=False) - - hand_have = condition_nodes.isInHand(name="does_hand_have") - - load_tool = action_nodes.ToolLoad(name="load_tool") - - sub_sequence.add_children([hand_have, load_tool]) - - selector.add_children([is_tool_load, sub_sequence]) - - return selector - - -def create_root() -> py_trees.behaviour.Behaviour: - """ - Create the root behaviour and it's subtree. - - Returns: - the root behaviour - """ - root = py_trees.composites.Sequence(name="test_sequence", memory=False) - - tool_load = action_nodes.ToolLoad(name="ToolLoad") - - root.add_children([tool_load]) - - # condition_one = py_trees.behaviours.StatusQueue( - # name="Condition 1", - # queue=[ - # py_trees.common.Status.SUCCESS, - # py_trees.common.Status.FAILURE, - # py_trees.common.Status.SUCCESS, - # ], - # eventually=py_trees.common.Status.SUCCESS, - # ) - # condition_two = py_trees.behaviours.StatusQueue( - # name="Condition 2", - # queue=[ - # py_trees.common.Status.SUCCESS, - # py_trees.common.Status.SUCCESS, - # py_trees.common.Status.FAILURE, - # ], - # eventually=py_trees.common.Status.SUCCESS, - # ) - # selector = py_trees.composites.Selector(name="Selector", memory=False) - # subtask = Action(name="Subtask") - - # selector.add_children([condition_one, subtask]) - - # task_sequence = py_trees.composites.Sequence(name="Task Sequence", memory=True) - # # task_one = py_trees.behaviours.Success(name="Worker 1") - # # task_two = py_trees.behaviours.Running(name="Worker 2") - # task_one = Action(name="Worker 1") - # task_two = Action(name="Worker 2") - - # eternal_guard.add_children([selector, task_sequence]) - # task_sequence.add_children([task_one, task_two]) - return root - - -def post_tick_handler( - snapshot_visitor: py_trees.visitors.SnapshotVisitor, - behaviour_tree: py_trees.trees.BehaviourTree, -) -> None: - """ - Print data about the part of the tree visited. - - Args: - snapshot_handler: gather data about the part of the tree visited - behaviour_tree: tree to gather data from - """ - print( - "\n" - + py_trees.display.unicode_tree( - root=behaviour_tree.root, - visited=snapshot_visitor.visited, - previously_visited=snapshot_visitor.previously_visited, - ) - ) - print(py_trees.display.unicode_blackboard()) - - -def pre_tick_handler(behaviour_tree: py_trees.trees.BehaviourTree) -> None: - """Print a banner with current tick count prior to ticking the tree. - - Args: - behaviour_tree: the tree to tick (used to fetch the count number) - """ - print("\n--------- Run %s ---------\n" % behaviour_tree.count) - - -def main() -> None: - """Entry point for the demo script.""" - py_trees.logging.level = py_trees.logging.Level.DEBUG - root = create_root() - - #################### - # Tree Stewardship - #################### - behaviour_tree = py_trees.trees.BehaviourTree(root) - behaviour_tree.add_pre_tick_handler(pre_tick_handler) - behaviour_tree.visitors.append(py_trees.visitors.DebugVisitor()) - snapshot_visitor = py_trees.visitors.SnapshotVisitor() - behaviour_tree.add_post_tick_handler( - functools.partial(post_tick_handler, snapshot_visitor) - ) - behaviour_tree.visitors.append(snapshot_visitor) - behaviour_tree.setup(timeout=15) - - #################### - # Tick Tock - #################### - - while True: - try: - behaviour_tree.tick() - py_trees.console.read_single_keypress() - except KeyboardInterrupt: - break - - print("\n") - - -def test_root(): - initial_state = { - "inHand": "Tool1", - } - root = TreeRoot() - root.set_initial_state(initial_state) - - -def test_blackboard(): - root = TreeRoot() - initial_state = { - "inHand": "Tool1", - } - root.set_initial_state(initial_state) - - is_in_hand = condition_nodes.isInHand(name="is_in_hand") - root.add_children([is_in_hand]) - root.setup() - print("inHand: %s" % (py_trees.blackboard.Blackboard().get(variable_name="inHand"))) - py_trees.blackboard.Blackboard().set(variable_name="inHand", value="Tool2") - print("inHand: %s" % (py_trees.blackboard.Blackboard().get(variable_name="inHand"))) - - root_2 = TreeRoot() - initial_state_2 = { - "inHand": "Tool1", - } - root_2.set_initial_state(initial_state_2) - is_in_hand_2 = condition_nodes.isInHand(name="is_in_hand_2") - root_2.add_children([is_in_hand_2]) - root_2.setup() - print("inHand: %s" % (py_trees.blackboard.Blackboard().get(variable_name="inHand"))) - py_trees.blackboard.Blackboard().set(variable_name="inHand", value="Tool2") - print("inHand: %s" % (py_trees.blackboard.Blackboard().get(variable_name="inHand"))) - # * the blackboard is a global variable - # * so different trees are using the same blackboard - - -def test_tree(): - root = TreeRoot() - initial_state = { - "inHand": "nothing", - } - root.set_initial_state(initial_state) - selector = py_trees.composites.Selector(name="load the tool", memory=False) - is_tool_load = condition_nodes.IsToolLoaded(name="is_tool_loaded") - sub_sequence = py_trees.composites.Sequence(name="Task Sequence", memory=False) - in_hand = condition_nodes.isInHand(name="in_hand") - load_tool = action_nodes.ToolLoadTest(name="load_tool") - sub_sequence.add_children([in_hand, load_tool]) - selector.add_children([is_tool_load, sub_sequence]) - root.add_children([selector]) - root.setup() - - #################### - # Tree Stewardship - #################### - behaviour_tree = py_trees.trees.BehaviourTree(root) - behaviour_tree.add_pre_tick_handler(pre_tick_handler) - behaviour_tree.visitors.append(py_trees.visitors.DebugVisitor()) - snapshot_visitor = py_trees.visitors.SnapshotVisitor() - behaviour_tree.add_post_tick_handler( - functools.partial(post_tick_handler, snapshot_visitor) - ) - behaviour_tree.visitors.append(snapshot_visitor) - behaviour_tree.setup(timeout=15) - - #################### - # Tick Tock - #################### - - while True: - try: - behaviour_tree.tick() - py_trees.console.read_single_keypress() - except KeyboardInterrupt: - break - - print("\n") - - -def create_load_tool_tree() -> py_trees.composites.Selector: - selector = py_trees.composites.Selector(name="load the tool", memory=False) - - is_tool_load = condition_nodes.IsToolLoaded(["tool1"]) - - sequence = py_trees.composites.Sequence(name="load tool sequence", memory=False) - - hand_free = condition_nodes.isInHand(["nothing"]) - load_tool = action_nodes.ToolLoadTest(["tool1"]) - sequence.add_children([hand_free, load_tool]) - - selector.add_children([is_tool_load, sequence]) - - return selector - - -def test_tree(): - root = TreeRoot() - initial_state = { - "inHand": "nothing", - "inTool": "nothing", - "cube1-at": "cube1", - } - root.set_initial_state(initial_state) - - selector = py_trees.composites.Selector(name="pick up cube selector", memory=False) - - is_cube_in_hand = condition_nodes.isInHand(["cube1"]) - - sequence = py_trees.composites.Sequence(name="pick up cube sequence", memory=False) - is_tool_load = create_load_tool_tree() - tool_free = condition_nodes.isInTool(["nothing"]) - pick_up_cube = action_nodes.ToolPick(["cube1"]) - sequence.add_children([is_tool_load, tool_free, pick_up_cube]) - - selector.add_children([is_cube_in_hand, sequence]) - - root.add_children([selector]) - - root.setup() - - #################### - # Tree Stewardship - #################### - behaviour_tree = py_trees.trees.BehaviourTree(root) - behaviour_tree.add_pre_tick_handler(pre_tick_handler) - behaviour_tree.visitors.append(py_trees.visitors.DebugVisitor()) - snapshot_visitor = py_trees.visitors.SnapshotVisitor() - behaviour_tree.add_post_tick_handler( - functools.partial(post_tick_handler, snapshot_visitor) - ) - behaviour_tree.visitors.append(snapshot_visitor) - behaviour_tree.setup(timeout=15) - - #################### - # Tick Tock - #################### - # py_trees.display.render_dot_tree(root, with_blackboard_variables=True) - - while True: - try: - behaviour_tree.tick() - py_trees.console.read_single_keypress() - except KeyboardInterrupt: - break - - print("\n") - - -def test_fake_action(): - root = TreeRoot() - initial_state = { - "inHand": "nothing", - "inTool": "nothing", - "cube1-at": "cube1", - } - root.set_initial_state(initial_state) - - sequence = py_trees.composites.Sequence(name="pick up cube sequence", memory=False) - - load_tool = action_nodes.ToolLoadTest(["tool1"]) - - load_tool2 = action_nodes.ToolLoadTest(["tool2"]) - - sequence.add_children([load_tool, load_tool2]) - - root.add_children([sequence]) - - #################### - # Tree Stewardship - #################### - behaviour_tree = py_trees.trees.BehaviourTree(root) - behaviour_tree.add_pre_tick_handler(pre_tick_handler) - behaviour_tree.visitors.append(py_trees.visitors.DebugVisitor()) - snapshot_visitor = py_trees.visitors.SnapshotVisitor() - behaviour_tree.add_post_tick_handler( - functools.partial(post_tick_handler, snapshot_visitor) - ) - behaviour_tree.visitors.append(snapshot_visitor) - behaviour_tree.setup(timeout=15) - - #################### - # Tick Tock - #################### - # py_trees.display.render_dot_tree(root, with_blackboard_variables=True) - - while True: - try: - behaviour_tree.tick() - py_trees.console.read_single_keypress() - except KeyboardInterrupt: - break - - print("\n") - - -if __name__ == "__main__": - # main() - # test_root() - # test_blackboard() - py_trees.logging.level = py_trees.logging.Level.DEBUG - test_fake_action() - - # py_trees.logging.level = py_trees.logging.Level.DEBUG - # root = py_trees.composites.Sequence(name="test_sequence", memory=False) - # action_node = action_nodes.ToolLoad(["tool1"]) - # root.add_children([action_node]) - - # #################### - # # Tree Stewardship - # #################### - # behaviour_tree = py_trees.trees.BehaviourTree(root) - # behaviour_tree.add_pre_tick_handler(pre_tick_handler) - # behaviour_tree.visitors.append(py_trees.visitors.DebugVisitor()) - # snapshot_visitor = py_trees.visitors.SnapshotVisitor() - # behaviour_tree.add_post_tick_handler( - # functools.partial(post_tick_handler, snapshot_visitor) - # ) - # behaviour_tree.visitors.append(snapshot_visitor) - # behaviour_tree.setup(timeout=15) - - # #################### - # # Tick Tock - # #################### - - # while True: - # try: - # behaviour_tree.tick() - # py_trees.console.read_single_keypress() - # except KeyboardInterrupt: - # break - - # print("\n") diff --git a/kios_logo.png b/kios_logo.png deleted file mode 100644 index d78ca4c7..00000000 Binary files a/kios_logo.png and /dev/null differ diff --git a/last_response.txt b/last_response.txt deleted file mode 100644 index fe7bcf18..00000000 --- a/last_response.txt +++ /dev/null @@ -1,245 +0,0 @@ - -{ - "task_cohesion": { - "action_sequence": [ - "load_tool(left_hand, parallel_box1)", - "pick_up(left_hand, parallel_box1, gear1)", - "insert(left_hand, parallel_box1, gear1, shaft1)" - ], - "action_instructions": [ - "Equip the parallel_box1 on the left hand.", - "Pick up the gear1 with the left hand using the parallel_box1.", - "Insert the gear1 into the shaft1 using the left hand equipped with the parallel_box1." - ], - "manipulated_objects": ["left_hand", "parallel_box1", "gear1", "shaft1"], - "behavior_tree": { - "name": "insert selector", - "identifier": 1, - "type_name": "selector", - "children": [ - { - "summary": "check if gear1 is inserted to shaft1", - "name": "check gear1 is_inserted_to shaft1", - "identifier": 2, - "type_name": "condition", - "conditions": [ - { - "object_name": "gear1", - "property_name": "is_inserted_to", - "property_value": "shaft1", - "status": true - } - ] - }, - { - "name": "insert sequence", - "identifier": 3, - "type_name": "sequence", - "children": [ - { - "name": "load_tool selector", - "identifier": 4, - "type_name": "selector", - "children": [ - { - "summary": "check if parallel_box1 is held by left hand", - "name": "check left_hand hold parallel_box1", - "identifier": 5, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "parallel_box1", - "status": true - } - ] - }, - { - "name": "load_tool sequence", - "identifier": 6, - "type_name": "sequence", - "children": [ - { - "summary": "check if parallel_box1 is equippable", - "name": "check parallel_box1 is_equippable", - "identifier": 7, - "type_name": "condition", - "conditions": [ - { - "object_name": "parallel_box1", - "property_name": "is_equippable", - "property_value": null, - "status": true - } - ] - }, - { - "summary": "check if left hand is free", - "name": "check left_hand is_free", - "identifier": 8, - "type_name": "condition", - "conditions": [ - { - "object_name": "left_hand", - "property_name": "is_free", - "property_value": null, - "status": true - } - ] - }, - { - "summary": "equip parallel_box1 to left hand", - "name": "load_tool(left_hand, parallel_box1)", - "identifier": 9, - "type_name": "action", - "effects": [ - { - "object_name": "left_hand", - "property_name": "is_free", - "property_value": null, - "status": false - }, - { - "object_name": "parallel_box1", - "property_name": "is_equippable", - "property_value": null, - "status": false - }, - { - "object_name": "left_hand", - "property_name": "hold", - "property_value": "parallel_box1", - "status": true - } - ] - } - ] - } - ] - }, - { - "name": "pick_up selector", - "identifier": 10, - "type_name": "selector", - "children": [ - { - "summary": "check if parallel_box1 holds gear1", - "name": "check parallel_box1 hold gear1", - "identifier": 11, - "type_name": "condition", - "conditions": [ - { - "object_name": "parallel_box1", - "property_name": "hold", - "property_value": "gear1", - "status": true - } - ] - }, - { - "name": "pick_up sequence", - "identifier": 12, - "type_name": "sequence", - "children": [ - { - "summary": "check if parallel_box1 is free", - "name": "check parallel_box1 is_free", - "identifier": 13, - "type_name": "condition", - "conditions": [ - { - "object_name": "parallel_box1", - "property_name": "is_free", - "property_value": null, - "status": true - } - ] - }, - { - "summary": "pick up gear1 using parallel_box1", - "name": "pick_up(left_hand, parallel_box1, gear1)", - "identifier": 14, - "type_name": "action", - "effects": [ - { - "object_name": "parallel_box1", - "property_name": "hold", - "property_value": "gear1", - "status": true - }, - { - "object_name": "parallel_box1", - "property_name": "is_free", - "property_value": null, - "status": false - } - ] - } - ] - } - ] - }, - { - "summary": "insert gear1 into shaft1", - "name": "insert(left_hand, parallel_box1, gear1, shaft1)", - "identifier": 15, - "type_name": "action", - "effects": [ - { - "object_name": "gear1", - "property_name": "is_inserted_to", - "property_value": "shaft1", - "status": true - }, - { - "object_name": "parallel_box1", - "property_name": "hold", - "property_value": "gear1", - "status": false - }, - { - "object_name": "parallel_box1", - "property_name": "is_free", - "property_value": null, - "status": true - } - ] - } - ] - } - ] - } - }, - "environment_before": { - "objects": [ - {"name": "gear1", "properties": ["is_free"]}, - {"name": "shaft1", "properties": ["is_free"]}, - {"name": "parallel_box1", "properties": ["is_free", "is_equippable"]}, - {"name": "left_hand", "properties": ["is_free"]} - ], - "constraints": [ - {"source": "parallel_box1", "name": "can_manipulate", "target": "gear1"}, - {"source": "gear1", "name": "can_insert_to", "target": "shaft1"} - ], - "relations": [] - }, - "environment_after": { - "objects": [ - {"name": "gear1", "properties": []}, - {"name": "shaft1", "properties": []}, - {"name": "parallel_box1", "properties": ["is_free", "is_equippable"]}, - {"name": "left_hand", "properties": []} - ], - "constraints": [ - {"source": "parallel_box1", "name": "can_manipulate", "target": "gear1"}, - {"source": "gear1", "name": "can_insert_to", "target": "shaft1"} - ], - "relations": [ - {"source": "gear1", "name": "is_inserted_to", "target": "shaft1"}, - {"source": "left_hand", "name": "hold", "target": "parallel_box1"} - ] - }, - "instruction_summary": "Insert gear1 into shaft1 using parallel_box1 with the left hand.", - "question": "" -} diff --git a/my_file.txt b/my_file.txt deleted file mode 100644 index e69de29b..00000000 diff --git a/old_tutorial.md b/old_tutorial.md deleted file mode 100644 index e69de29b..00000000 diff --git a/pseudo_code b/pseudo_code deleted file mode 100644 index b4d45f4a..00000000 --- a/pseudo_code +++ /dev/null @@ -1,61 +0,0 @@ -given: initial state , goal state set , - behavior tree set , tree utility function , - max_plan_failure , max_execution_failure - -def: state , plan

- ----------------------------------------------------------- -START: - -feasible_plan_list = [] - -while difference_in_success_rate(feasible_plan_list) < threshold: # convergence condition - - robot_recover() - - # initialize in planning - plan_count = 0, exe_count = 0 - state = s0 - - plan = [], result[] = [], last_plan = [] - task_success = False - - while task_success == False: - - update_utility_function(result[]) # actually the success rate function of each subtree based on state transition - - # dynamic planning. should always expand, ground and execute until the task is finished or the quit condition is met - - if failure_in_condition(plan): # failure in condition: dynamic expanding - if plan_count > plan_max: - break # too many planning tries are forbidden - else - last_plan = plan # record the last plan - plan = expand(last_plan, state) # expand the plan - plan_count += 1 # try the new plan - - else if failure_in_action(plan): # failure in action: parameter issue or plan issue - robot_recover_last_action() # (a problem here: how to recover the last action in real experiment?) - - new_plan = expand(last_plan) # expand the last plan - if isBetter(new_plan, last_plan): # find a better plan than the last one - exe_count = 0 # reset the execution count - plan = new_plan # and last_plan is still the same - else - if exe_count > exe_max: - break # quit and start again when having tried too many times - else - exe_count += 1 # try one more time - - grounded_plan = ground_plan(plan) # ground the plan - task_success, result[] = execute(grounded_p) # execute the plan and get the result - - update_utility_function(task_success, result[]) # update the utility function based on the result of the task - - if task_success == True: - feasible_plan_list.append([plan, utility(plan)]) - -plan* = feasible_plan_list.best_plan() - -END ---------------------------------------------------------- \ No newline at end of file diff --git a/old_readme.md b/readme.md similarity index 100% rename from old_readme.md rename to readme.md diff --git a/requirements.txt b/requirements.txt deleted file mode 100644 index 3f97a5d4..00000000 --- a/requirements.txt +++ /dev/null @@ -1,39 +0,0 @@ -# basics -numpy -websockets -tabulate -python-dotenv - -# behavior tree -py_trees==2.2.3 - -# pddl classical planning -unified-planning -unified-planning[engines] # for solving the pddl problems - -# graph modules -networkx -neo4j -graphviz - -# openai -openai # you may need to upgrade this after installing it -# * see openai api change log for more info -tiktoken -cohere -langchain -langchain-openai -langgraph - - -# robotics -roboticstoolbox-python - -# LLAMA -torch -fairscale -fire -sentencepiece - -# scene -pymongo diff --git a/selector_insert_shaft1_into_gearbase_hole1.dot b/selector_insert_shaft1_into_gearbase_hole1.dot deleted file mode 100644 index 1aff0351..00000000 --- a/selector_insert_shaft1_into_gearbase_hole1.dot +++ /dev/null @@ -1,59 +0,0 @@ -digraph pastafarianism { -ordering=out; -graph [fontname="times-roman"]; -node [fontname="times-roman"]; -edge [fontname="times-roman"]; -selector [fillcolor=cyan, fontcolor=black, fontsize=9, label="selector: insert_shaft1_into_gearbase_hole1", shape=octagon, style=filled]; -target [fillcolor=gray, fontcolor=black, fontsize=9, label="target: is_inserted_to(shaft1, gearbase_hole1)", shape=ellipse, style=filled]; -"selector: insert_shaft1_into_gearbase_hole1" -> "target: is_inserted_to(shaft1, gearbase_hole1)"; -sequence [fillcolor=orange, fontcolor=black, fontsize=9, label="sequence: insert_shaft1_into_gearbase_hole1", shape=box, style=filled]; -"selector: insert_shaft1_into_gearbase_hole1" -> "sequence: insert_shaft1_into_gearbase_hole1"; -selector [fillcolor=cyan, fontcolor=black, fontsize=9, label="selector: put_down_gear3_with_outwardgripper", shape=octagon, style=filled]; -"sequence: insert_shaft1_into_gearbase_hole1" -> "selector: put_down_gear3_with_outwardgripper"; -target [fillcolor=gray, fontcolor=black, fontsize=9, label="target: is_empty(outwardgripper)", shape=ellipse, style=filled]; -"selector: put_down_gear3_with_outwardgripper" -> "target: is_empty(outwardgripper)"; -sequence [fillcolor=orange, fontcolor=black, fontsize=9, label="sequence: put_down_gear3_with_outwardgripper", shape=box, style=filled]; -"selector: put_down_gear3_with_outwardgripper" -> "sequence: put_down_gear3_with_outwardgripper"; -precondition [fillcolor=gray, fontcolor=black, fontsize=9, label="precondition: hold(left_hand, outwardgripper)", shape=ellipse, style=filled]; -"sequence: put_down_gear3_with_outwardgripper" -> "precondition: hold(left_hand, outwardgripper)"; -precondition [fillcolor=gray, fontcolor=black, fontsize=9, label="precondition: hold(outwardgripper, gear3)", shape=ellipse, style=filled]; -"sequence: put_down_gear3_with_outwardgripper" -> "precondition: hold(outwardgripper, gear3)"; -action [fillcolor=gray, fontcolor=black, fontsize=9, label="action: put_down(left_hand, outwardgripper, gear3)", shape=ellipse, style=filled]; -"sequence: put_down_gear3_with_outwardgripper" -> "action: put_down(left_hand, outwardgripper, gear3)"; -selector [fillcolor=cyan, fontcolor=black, fontsize=9, label="selector: change_tool(left_hand, outwardgripper, clampgripper)", shape=octagon, style=filled]; -"sequence: insert_shaft1_into_gearbase_hole1" -> "selector: change_tool(left_hand, outwardgripper, clampgripper)"; -target [fillcolor=gray, fontcolor=black, fontsize=9, label="target: hold(left_hand, clampgripper)", shape=ellipse, style=filled]; -"selector: change_tool(left_hand, outwardgripper, clampgripper)" -> "target: hold(left_hand, clampgripper)"; -sequence [fillcolor=orange, fontcolor=black, fontsize=9, label="sequence: change_tool(left_hand, outwardgripper, clampgripper)", shape=box, style=filled]; -"selector: change_tool(left_hand, outwardgripper, clampgripper)" -> "sequence: change_tool(left_hand, outwardgripper, clampgripper)"; -precondition [fillcolor=gray, fontcolor=black, fontsize=9, label="precondition: hold(left_hand, outwardgripper)*", shape=ellipse, style=filled]; -"sequence: change_tool(left_hand, outwardgripper, clampgripper)" -> "precondition: hold(left_hand, outwardgripper)*"; -precondition [fillcolor=gray, fontcolor=black, fontsize=9, label="precondition: is_empty(outwardgripper)", shape=ellipse, style=filled]; -"sequence: change_tool(left_hand, outwardgripper, clampgripper)" -> "precondition: is_empty(outwardgripper)"; -action [fillcolor=gray, fontcolor=black, fontsize=9, label="action: change_tool(left_hand, outwardgripper, clampgripper)", shape=ellipse, style=filled]; -"sequence: change_tool(left_hand, outwardgripper, clampgripper)" -> "action: change_tool(left_hand, outwardgripper, clampgripper)"; -selector [fillcolor=cyan, fontcolor=black, fontsize=9, label="selector: pick_up_shaft1_with_clampgripper", shape=octagon, style=filled]; -"sequence: insert_shaft1_into_gearbase_hole1" -> "selector: pick_up_shaft1_with_clampgripper"; -target [fillcolor=gray, fontcolor=black, fontsize=9, label="target: hold(clampgripper, shaft1)", shape=ellipse, style=filled]; -"selector: pick_up_shaft1_with_clampgripper" -> "target: hold(clampgripper, shaft1)"; -sequence [fillcolor=orange, fontcolor=black, fontsize=9, label="sequence: pick_up_shaft1_with_clampgripper", shape=box, style=filled]; -"selector: pick_up_shaft1_with_clampgripper" -> "sequence: pick_up_shaft1_with_clampgripper"; -precondition [fillcolor=gray, fontcolor=black, fontsize=9, label="precondition: hold(left_hand, clampgripper)", shape=ellipse, style=filled]; -"sequence: pick_up_shaft1_with_clampgripper" -> "precondition: hold(left_hand, clampgripper)"; -precondition [fillcolor=gray, fontcolor=black, fontsize=9, label="precondition: is_empty(clampgripper)", shape=ellipse, style=filled]; -"sequence: pick_up_shaft1_with_clampgripper" -> "precondition: is_empty(clampgripper)"; -action [fillcolor=gray, fontcolor=black, fontsize=9, label="action: pick_up(left_hand, clampgripper, shaft1)", shape=ellipse, style=filled]; -"sequence: pick_up_shaft1_with_clampgripper" -> "action: pick_up(left_hand, clampgripper, shaft1)"; -selector [fillcolor=cyan, fontcolor=black, fontsize=9, label="selector: insert_shaft1_into_gearbase_hole1_with_clampgripper", shape=octagon, style=filled]; -"sequence: insert_shaft1_into_gearbase_hole1" -> "selector: insert_shaft1_into_gearbase_hole1_with_clampgripper"; -target [fillcolor=gray, fontcolor=black, fontsize=9, label="target: is_inserted_to(shaft1, gearbase_hole1)*", shape=ellipse, style=filled]; -"selector: insert_shaft1_into_gearbase_hole1_with_clampgripper" -> "target: is_inserted_to(shaft1, gearbase_hole1)*"; -sequence [fillcolor=orange, fontcolor=black, fontsize=9, label="sequence: insert_shaft1_into_gearbase_hole1_with_clampgripper", shape=box, style=filled]; -"selector: insert_shaft1_into_gearbase_hole1_with_clampgripper" -> "sequence: insert_shaft1_into_gearbase_hole1_with_clampgripper"; -precondition [fillcolor=gray, fontcolor=black, fontsize=9, label="precondition: hold(left_hand, clampgripper)*", shape=ellipse, style=filled]; -"sequence: insert_shaft1_into_gearbase_hole1_with_clampgripper" -> "precondition: hold(left_hand, clampgripper)*"; -precondition [fillcolor=gray, fontcolor=black, fontsize=9, label="precondition: hold(clampgripper, shaft1)", shape=ellipse, style=filled]; -"sequence: insert_shaft1_into_gearbase_hole1_with_clampgripper" -> "precondition: hold(clampgripper, shaft1)"; -action [fillcolor=gray, fontcolor=black, fontsize=9, label="action: insert(left_hand, clampgripper, shaft1, gearbase_hole1)", shape=ellipse, style=filled]; -"sequence: insert_shaft1_into_gearbase_hole1_with_clampgripper" -> "action: insert(left_hand, clampgripper, shaft1, gearbase_hole1)"; -} diff --git a/selector_insert_shaft1_into_gearbase_hole1.png b/selector_insert_shaft1_into_gearbase_hole1.png deleted file mode 100644 index bd5446b7..00000000 Binary files a/selector_insert_shaft1_into_gearbase_hole1.png and /dev/null differ diff --git a/selector_insert_shaft1_into_gearbase_hole1.svg b/selector_insert_shaft1_into_gearbase_hole1.svg deleted file mode 100644 index 8522d0be..00000000 --- a/selector_insert_shaft1_into_gearbase_hole1.svg +++ /dev/null @@ -1,361 +0,0 @@ - - - - - - -pastafarianism - - - -selector - -selector: insert_shaft1_into_gearbase_hole1_with_clampgripper - - - -target - -target: is_inserted_to(shaft1, gearbase_hole1)* - - - -selector: insert_shaft1_into_gearbase_hole1 - -selector: insert_shaft1_into_gearbase_hole1 - - - -target: is_inserted_to(shaft1, gearbase_hole1) - -target: is_inserted_to(shaft1, gearbase_hole1) - - - -selector: insert_shaft1_into_gearbase_hole1->target: is_inserted_to(shaft1, gearbase_hole1) - - - - - -sequence: insert_shaft1_into_gearbase_hole1 - -sequence: insert_shaft1_into_gearbase_hole1 - - - -selector: insert_shaft1_into_gearbase_hole1->sequence: insert_shaft1_into_gearbase_hole1 - - - - - -sequence - -sequence: insert_shaft1_into_gearbase_hole1_with_clampgripper - - - -selector: put_down_gear3_with_outwardgripper - -selector: put_down_gear3_with_outwardgripper - - - -sequence: insert_shaft1_into_gearbase_hole1->selector: put_down_gear3_with_outwardgripper - - - - - -selector: change_tool(left_hand, outwardgripper, clampgripper) - -selector: change_tool(left_hand, outwardgripper, clampgripper) - - - -sequence: insert_shaft1_into_gearbase_hole1->selector: change_tool(left_hand, outwardgripper, clampgripper) - - - - - -selector: pick_up_shaft1_with_clampgripper - -selector: pick_up_shaft1_with_clampgripper - - - -sequence: insert_shaft1_into_gearbase_hole1->selector: pick_up_shaft1_with_clampgripper - - - - - -selector: insert_shaft1_into_gearbase_hole1_with_clampgripper - -selector: insert_shaft1_into_gearbase_hole1_with_clampgripper - - - -sequence: insert_shaft1_into_gearbase_hole1->selector: insert_shaft1_into_gearbase_hole1_with_clampgripper - - - - - -target: is_empty(outwardgripper) - -target: is_empty(outwardgripper) - - - -selector: put_down_gear3_with_outwardgripper->target: is_empty(outwardgripper) - - - - - -sequence: put_down_gear3_with_outwardgripper - -sequence: put_down_gear3_with_outwardgripper - - - -selector: put_down_gear3_with_outwardgripper->sequence: put_down_gear3_with_outwardgripper - - - - - -precondition: hold(left_hand, outwardgripper) - -precondition: hold(left_hand, outwardgripper) - - - -sequence: put_down_gear3_with_outwardgripper->precondition: hold(left_hand, outwardgripper) - - - - - -precondition: hold(outwardgripper, gear3) - -precondition: hold(outwardgripper, gear3) - - - -sequence: put_down_gear3_with_outwardgripper->precondition: hold(outwardgripper, gear3) - - - - - -action: put_down(left_hand, outwardgripper, gear3) - -action: put_down(left_hand, outwardgripper, gear3) - - - -sequence: put_down_gear3_with_outwardgripper->action: put_down(left_hand, outwardgripper, gear3) - - - - - -precondition - -precondition: hold(clampgripper, shaft1) - - - -action - -action: insert(left_hand, clampgripper, shaft1, gearbase_hole1) - - - -target: hold(left_hand, clampgripper) - -target: hold(left_hand, clampgripper) - - - -selector: change_tool(left_hand, outwardgripper, clampgripper)->target: hold(left_hand, clampgripper) - - - - - -sequence: change_tool(left_hand, outwardgripper, clampgripper) - -sequence: change_tool(left_hand, outwardgripper, clampgripper) - - - -selector: change_tool(left_hand, outwardgripper, clampgripper)->sequence: change_tool(left_hand, outwardgripper, clampgripper) - - - - - -precondition: hold(left_hand, outwardgripper)* - -precondition: hold(left_hand, outwardgripper)* - - - -sequence: change_tool(left_hand, outwardgripper, clampgripper)->precondition: hold(left_hand, outwardgripper)* - - - - - -precondition: is_empty(outwardgripper) - -precondition: is_empty(outwardgripper) - - - -sequence: change_tool(left_hand, outwardgripper, clampgripper)->precondition: is_empty(outwardgripper) - - - - - -action: change_tool(left_hand, outwardgripper, clampgripper) - -action: change_tool(left_hand, outwardgripper, clampgripper) - - - -sequence: change_tool(left_hand, outwardgripper, clampgripper)->action: change_tool(left_hand, outwardgripper, clampgripper) - - - - - -target: hold(clampgripper, shaft1) - -target: hold(clampgripper, shaft1) - - - -selector: pick_up_shaft1_with_clampgripper->target: hold(clampgripper, shaft1) - - - - - -sequence: pick_up_shaft1_with_clampgripper - -sequence: pick_up_shaft1_with_clampgripper - - - -selector: pick_up_shaft1_with_clampgripper->sequence: pick_up_shaft1_with_clampgripper - - - - - -precondition: hold(left_hand, clampgripper) - -precondition: hold(left_hand, clampgripper) - - - -sequence: pick_up_shaft1_with_clampgripper->precondition: hold(left_hand, clampgripper) - - - - - -precondition: is_empty(clampgripper) - -precondition: is_empty(clampgripper) - - - -sequence: pick_up_shaft1_with_clampgripper->precondition: is_empty(clampgripper) - - - - - -action: pick_up(left_hand, clampgripper, shaft1) - -action: pick_up(left_hand, clampgripper, shaft1) - - - -sequence: pick_up_shaft1_with_clampgripper->action: pick_up(left_hand, clampgripper, shaft1) - - - - - -target: is_inserted_to(shaft1, gearbase_hole1)* - -target: is_inserted_to(shaft1, gearbase_hole1)* - - - -selector: insert_shaft1_into_gearbase_hole1_with_clampgripper->target: is_inserted_to(shaft1, gearbase_hole1)* - - - - - -sequence: insert_shaft1_into_gearbase_hole1_with_clampgripper - -sequence: insert_shaft1_into_gearbase_hole1_with_clampgripper - - - -selector: insert_shaft1_into_gearbase_hole1_with_clampgripper->sequence: insert_shaft1_into_gearbase_hole1_with_clampgripper - - - - - -precondition: hold(left_hand, clampgripper)* - -precondition: hold(left_hand, clampgripper)* - - - -sequence: insert_shaft1_into_gearbase_hole1_with_clampgripper->precondition: hold(left_hand, clampgripper)* - - - - - -precondition: hold(clampgripper, shaft1) - -precondition: hold(clampgripper, shaft1) - - - -sequence: insert_shaft1_into_gearbase_hole1_with_clampgripper->precondition: hold(clampgripper, shaft1) - - - - - -action: insert(left_hand, clampgripper, shaft1, gearbase_hole1) - -action: insert(left_hand, clampgripper, shaft1, gearbase_hole1) - - - -sequence: insert_shaft1_into_gearbase_hole1_with_clampgripper->action: insert(left_hand, clampgripper, shaft1, gearbase_hole1) - - - - - diff --git a/selector_insertshaft1_gearbase_hole1.dot b/selector_insertshaft1_gearbase_hole1.dot deleted file mode 100644 index 0c73bb32..00000000 --- a/selector_insertshaft1_gearbase_hole1.dot +++ /dev/null @@ -1,37 +0,0 @@ -digraph pastafarianism { -ordering=out; -graph [fontname="times-roman"]; -node [fontname="times-roman"]; -edge [fontname="times-roman"]; -selector [fillcolor=cyan, fontcolor=black, fontsize=9, label="selector: insert(shaft1, gearbase_hole1)", shape=octagon, style=filled]; -target [fillcolor=gray, fontcolor=black, fontsize=9, label="target: is_inserted_to(shaft1, gearbase_hole1)", shape=ellipse, style=filled]; -"selector: insert(shaft1, gearbase_hole1)" -> "target: is_inserted_to(shaft1, gearbase_hole1)"; -sequence [fillcolor=orange, fontcolor=black, fontsize=9, label="sequence: insert(shaft1, gearbase_hole1)", shape=box, style=filled]; -"selector: insert(shaft1, gearbase_hole1)" -> "sequence: insert(shaft1, gearbase_hole1)"; -selector [fillcolor=cyan, fontcolor=black, fontsize=9, label="selector: change_tool(left_hand, outwardgripper, clampgripper)", shape=octagon, style=filled]; -"sequence: insert(shaft1, gearbase_hole1)" -> "selector: change_tool(left_hand, outwardgripper, clampgripper)"; -target [fillcolor=gray, fontcolor=black, fontsize=9, label="target: hold(left_hand, clampgripper)", shape=ellipse, style=filled]; -"selector: change_tool(left_hand, outwardgripper, clampgripper)" -> "target: hold(left_hand, clampgripper)"; -sequence [fillcolor=orange, fontcolor=black, fontsize=9, label="sequence: change_tool(left_hand, outwardgripper, clampgripper)", shape=box, style=filled]; -"selector: change_tool(left_hand, outwardgripper, clampgripper)" -> "sequence: change_tool(left_hand, outwardgripper, clampgripper)"; -precondition [fillcolor=gray, fontcolor=black, fontsize=9, label="precondition: hold(left_hand, outwardgripper)", shape=ellipse, style=filled]; -"sequence: change_tool(left_hand, outwardgripper, clampgripper)" -> "precondition: hold(left_hand, outwardgripper)"; -precondition [fillcolor=gray, fontcolor=black, fontsize=9, label="precondition: is_empty(outwardgripper)", shape=ellipse, style=filled]; -"sequence: change_tool(left_hand, outwardgripper, clampgripper)" -> "precondition: is_empty(outwardgripper)"; -action [fillcolor=gray, fontcolor=black, fontsize=9, label="action: change_tool(left_hand, outwardgripper, clampgripper)", shape=ellipse, style=filled]; -"sequence: change_tool(left_hand, outwardgripper, clampgripper)" -> "action: change_tool(left_hand, outwardgripper, clampgripper)"; -selector [fillcolor=cyan, fontcolor=black, fontsize=9, label="selector: pick_up(left_hand, clampgripper, shaft1)", shape=octagon, style=filled]; -"sequence: insert(shaft1, gearbase_hole1)" -> "selector: pick_up(left_hand, clampgripper, shaft1)"; -target [fillcolor=gray, fontcolor=black, fontsize=9, label="target: hold(clampgripper, shaft1)", shape=ellipse, style=filled]; -"selector: pick_up(left_hand, clampgripper, shaft1)" -> "target: hold(clampgripper, shaft1)"; -sequence [fillcolor=orange, fontcolor=black, fontsize=9, label="sequence: pick_up(left_hand, clampgripper, shaft1)", shape=box, style=filled]; -"selector: pick_up(left_hand, clampgripper, shaft1)" -> "sequence: pick_up(left_hand, clampgripper, shaft1)"; -precondition [fillcolor=gray, fontcolor=black, fontsize=9, label="precondition: is_empty(clampgripper)", shape=ellipse, style=filled]; -"sequence: pick_up(left_hand, clampgripper, shaft1)" -> "precondition: is_empty(clampgripper)"; -precondition [fillcolor=gray, fontcolor=black, fontsize=9, label="precondition: hold(left_hand, clampgripper)", shape=ellipse, style=filled]; -"sequence: pick_up(left_hand, clampgripper, shaft1)" -> "precondition: hold(left_hand, clampgripper)"; -action [fillcolor=gray, fontcolor=black, fontsize=9, label="action: pick_up(left_hand, clampgripper, shaft1)", shape=ellipse, style=filled]; -"sequence: pick_up(left_hand, clampgripper, shaft1)" -> "action: pick_up(left_hand, clampgripper, shaft1)"; -action [fillcolor=gray, fontcolor=black, fontsize=9, label="action: insert(left_hand, clampgripper, shaft1, gearbase_hole1)", shape=ellipse, style=filled]; -"sequence: insert(shaft1, gearbase_hole1)" -> "action: insert(left_hand, clampgripper, shaft1, gearbase_hole1)"; -} diff --git a/selector_insertshaft1_gearbase_hole1.png b/selector_insertshaft1_gearbase_hole1.png deleted file mode 100644 index ddf7bfa2..00000000 Binary files a/selector_insertshaft1_gearbase_hole1.png and /dev/null differ diff --git a/selector_insertshaft1_gearbase_hole1.svg b/selector_insertshaft1_gearbase_hole1.svg deleted file mode 100644 index aba3cacb..00000000 --- a/selector_insertshaft1_gearbase_hole1.svg +++ /dev/null @@ -1,229 +0,0 @@ - - - - - - -pastafarianism - - - -selector - -selector: pick_up(left_hand, clampgripper, shaft1) - - - -target - -target: hold(clampgripper, shaft1) - - - -selector: insert(shaft1, gearbase_hole1) - -selector: insert(shaft1, gearbase_hole1) - - - -target: is_inserted_to(shaft1, gearbase_hole1) - -target: is_inserted_to(shaft1, gearbase_hole1) - - - -selector: insert(shaft1, gearbase_hole1)->target: is_inserted_to(shaft1, gearbase_hole1) - - - - - -sequence: insert(shaft1, gearbase_hole1) - -sequence: insert(shaft1, gearbase_hole1) - - - -selector: insert(shaft1, gearbase_hole1)->sequence: insert(shaft1, gearbase_hole1) - - - - - -sequence - -sequence: pick_up(left_hand, clampgripper, shaft1) - - - -selector: change_tool(left_hand, outwardgripper, clampgripper) - -selector: change_tool(left_hand, outwardgripper, clampgripper) - - - -sequence: insert(shaft1, gearbase_hole1)->selector: change_tool(left_hand, outwardgripper, clampgripper) - - - - - -selector: pick_up(left_hand, clampgripper, shaft1) - -selector: pick_up(left_hand, clampgripper, shaft1) - - - -sequence: insert(shaft1, gearbase_hole1)->selector: pick_up(left_hand, clampgripper, shaft1) - - - - - -action: insert(left_hand, clampgripper, shaft1, gearbase_hole1) - -action: insert(left_hand, clampgripper, shaft1, gearbase_hole1) - - - -sequence: insert(shaft1, gearbase_hole1)->action: insert(left_hand, clampgripper, shaft1, gearbase_hole1) - - - - - -target: hold(left_hand, clampgripper) - -target: hold(left_hand, clampgripper) - - - -selector: change_tool(left_hand, outwardgripper, clampgripper)->target: hold(left_hand, clampgripper) - - - - - -sequence: change_tool(left_hand, outwardgripper, clampgripper) - -sequence: change_tool(left_hand, outwardgripper, clampgripper) - - - -selector: change_tool(left_hand, outwardgripper, clampgripper)->sequence: change_tool(left_hand, outwardgripper, clampgripper) - - - - - -precondition: hold(left_hand, outwardgripper) - -precondition: hold(left_hand, outwardgripper) - - - -sequence: change_tool(left_hand, outwardgripper, clampgripper)->precondition: hold(left_hand, outwardgripper) - - - - - -precondition: is_empty(outwardgripper) - -precondition: is_empty(outwardgripper) - - - -sequence: change_tool(left_hand, outwardgripper, clampgripper)->precondition: is_empty(outwardgripper) - - - - - -action: change_tool(left_hand, outwardgripper, clampgripper) - -action: change_tool(left_hand, outwardgripper, clampgripper) - - - -sequence: change_tool(left_hand, outwardgripper, clampgripper)->action: change_tool(left_hand, outwardgripper, clampgripper) - - - - - -precondition - -precondition: hold(left_hand, clampgripper) - - - -action - -action: insert(left_hand, clampgripper, shaft1, gearbase_hole1) - - - -target: hold(clampgripper, shaft1) - -target: hold(clampgripper, shaft1) - - - -selector: pick_up(left_hand, clampgripper, shaft1)->target: hold(clampgripper, shaft1) - - - - - -sequence: pick_up(left_hand, clampgripper, shaft1) - -sequence: pick_up(left_hand, clampgripper, shaft1) - - - -selector: pick_up(left_hand, clampgripper, shaft1)->sequence: pick_up(left_hand, clampgripper, shaft1) - - - - - -precondition: is_empty(clampgripper) - -precondition: is_empty(clampgripper) - - - -sequence: pick_up(left_hand, clampgripper, shaft1)->precondition: is_empty(clampgripper) - - - - - -precondition: hold(left_hand, clampgripper) - -precondition: hold(left_hand, clampgripper) - - - -sequence: pick_up(left_hand, clampgripper, shaft1)->precondition: hold(left_hand, clampgripper) - - - - - -action: pick_up(left_hand, clampgripper, shaft1) - -action: pick_up(left_hand, clampgripper, shaft1) - - - -sequence: pick_up(left_hand, clampgripper, shaft1)->action: pick_up(left_hand, clampgripper, shaft1) - - - - - diff --git a/tree.png b/tree.png deleted file mode 100644 index 19799cb0..00000000 Binary files a/tree.png and /dev/null differ diff --git a/variables.json b/variables.json deleted file mode 100644 index 46ddbb53..00000000 --- a/variables.json +++ /dev/null @@ -1,111 +0,0 @@ -{ - "move": { - "name": "move", - "variables": { - "l_from": null, - "l_to": null - }, - "mios_parameters": [ - "something here" - ], - "preconditions": { - "true": { - "robot_at": "l_from", - "reachable": [ - "l_from", - "l_to" - ] - }, - "false": {} - }, - "effects": { - "true": { - "robot_at": "l_to" - }, - "false": { - "robot_at": "l_from" - } - } - }, - "load_tool": { - "name": "load_tool", - "variables": { - "tool": null - }, - "mios_parameters": [ - "something here" - ], - "preconditions": { - "true": { - "hand_free": false - }, - "false": {} - }, - "effects": { - "true": { - "in_hand": "tool" - }, - "false": { - "hand_free": false - } - } - }, - "unload_tool": { - "name": "unload_tool", - "variables": { - "tool": null - }, - "mios_parameters": [ - "something here" - ], - "preconditions": { - "true": { - "in_hand": "tool", - "tool_free": "tool" - }, - "false": {} - }, - "effects": { - "true": { - "hand_free": false - }, - "false": { - "in_hand": "tool" - } - } - }, - "pick": { - "name": "pick", - "variables": { - "part": null, - "tool": null, - "l_from": null, - "l_to": null - }, - "mios_parameters": [ - "something here" - ], - "preconditions": { - "true": { - "robot_at": "l_from", - "reachable": [ - "l_from", - "l_to" - ], - "in_hand": "tool", - "tool_free": "tool" - }, - "false": {} - }, - "effects": { - "true": { - "robot_at": "l_from", - "in_tool": "part" - }, - "false": { - "robot_at": "l_to", - "tool_free": "tool" - } - } - } -} \ No newline at end of file diff --git a/wandb/debug-cli.blackbird.log b/wandb/debug-cli.blackbird.log deleted file mode 100644 index e69de29b..00000000 diff --git a/wandb/debug-internal.log b/wandb/debug-internal.log deleted file mode 120000 index f3ecc8f4..00000000 --- a/wandb/debug-internal.log +++ /dev/null @@ -1 +0,0 @@ -run-20240304_142731-ftjob-L36vT4dDnkWzL9GckJ0SssVm/logs/debug-internal.log \ No newline at end of file diff --git a/wandb/debug.log b/wandb/debug.log deleted file mode 120000 index 57841d3f..00000000 --- a/wandb/debug.log +++ /dev/null @@ -1 +0,0 @@ -run-20240304_142731-ftjob-L36vT4dDnkWzL9GckJ0SssVm/logs/debug.log \ No newline at end of file diff --git a/wandb/latest-run b/wandb/latest-run deleted file mode 120000 index b6ee5382..00000000 --- a/wandb/latest-run +++ /dev/null @@ -1 +0,0 @@ -run-20240304_142731-ftjob-L36vT4dDnkWzL9GckJ0SssVm \ No newline at end of file diff --git a/wandb/run-20240303_155426-ftjob-UInKOefMZ40cQzVNlZq43njb/files/conda-environment.yaml b/wandb/run-20240303_155426-ftjob-UInKOefMZ40cQzVNlZq43njb/files/conda-environment.yaml deleted file mode 100644 index 5b7d2077..00000000 --- a/wandb/run-20240303_155426-ftjob-UInKOefMZ40cQzVNlZq43njb/files/conda-environment.yaml +++ /dev/null @@ -1,400 +0,0 @@ -name: kios -channels: - - defaults -dependencies: - - _libgcc_mutex=0.1=main - - _openmp_mutex=5.1=1_gnu - - asttokens=2.0.5=pyhd3eb1b0_0 - - bzip2=1.0.8=h7b6447c_0 - - ca-certificates=2023.12.12=h06a4308_0 - - decorator=5.1.1=pyhd3eb1b0_0 - - exceptiongroup=1.2.0=py310h06a4308_0 - - executing=0.8.3=pyhd3eb1b0_0 - - ipython=8.20.0=py310h06a4308_0 - - jedi=0.18.1=py310h06a4308_1 - - ld_impl_linux-64=2.38=h1181459_1 - - libffi=3.3=he6710b0_2 - - libgcc-ng=11.2.0=h1234567_1 - - libgomp=11.2.0=h1234567_1 - - libstdcxx-ng=11.2.0=h1234567_1 - - libuuid=1.41.5=h5eee18b_0 - - matplotlib-inline=0.1.6=py310h06a4308_0 - - ncurses=6.4=h6a678d5_0 - - openssl=1.1.1w=h7f8727e_0 - - parso=0.8.3=pyhd3eb1b0_0 - - pexpect=4.8.0=pyhd3eb1b0_3 - - pip=23.3.1=py310h06a4308_0 - - prompt-toolkit=3.0.43=py310h06a4308_0 - - prompt_toolkit=3.0.43=hd3eb1b0_0 - - ptyprocess=0.7.0=pyhd3eb1b0_2 - - pure_eval=0.2.2=pyhd3eb1b0_0 - - python=3.10.0=h12debd9_5 - - readline=8.2=h5eee18b_0 - - setuptools=68.2.2=py310h06a4308_0 - - six=1.16.0=pyhd3eb1b0_1 - - sqlite=3.41.2=h5eee18b_0 - - stack_data=0.2.0=pyhd3eb1b0_0 - - tk=8.6.12=h1ccaba5_0 - - traitlets=5.7.1=py310h06a4308_0 - - wcwidth=0.2.5=pyhd3eb1b0_0 - - wheel=0.41.2=py310h06a4308_0 - - xz=5.4.5=h5eee18b_0 - - zlib=1.2.13=h5eee18b_0 - - pip: - - adal==1.2.7 - - aenum==3.1.15 - - aiohttp==3.9.1 - - aiosignal==1.3.1 - - alembic==1.13.1 - - annotated-types==0.6.0 - - ansitable==0.9.10 - - anthropic==0.8.1 - - antlr4-python3-runtime==4.7.2 - - anyio==4.2.0 - - appdirs==1.4.4 - - applicationinsights==0.11.10 - - argcomplete==3.2.2 - - asgiref==3.7.2 - - async-timeout==4.0.3 - - attrs==23.2.0 - - azure-ai-generative==1.0.0b2 - - azure-ai-ml==1.12.1 - - azure-ai-resources==1.0.0b2 - - azure-common==1.1.28 - - azure-core==1.29.7 - - azure-graphrbac==0.61.1 - - azure-identity==1.15.0 - - azure-keyvault==4.2.0 - - azure-keyvault-certificates==4.7.0 - - azure-keyvault-keys==4.8.0 - - azure-keyvault-secrets==4.7.0 - - azure-mgmt-authorization==4.0.0 - - azure-mgmt-containerregistry==10.3.0 - - azure-mgmt-core==1.4.0 - - azure-mgmt-keyvault==10.3.0 - - azure-mgmt-network==25.1.0 - - azure-mgmt-resource==23.0.1 - - azure-mgmt-storage==21.1.0 - - azure-search-documents==11.4.0b11 - - azure-storage-blob==12.19.0 - - azure-storage-file-datalake==12.14.0 - - azure-storage-file-share==12.15.0 - - azureml-core==1.54.0.post1 - - azureml-dataprep==5.1.2 - - azureml-dataprep-native==41.0.0 - - azureml-dataprep-rslex==2.22.2 - - azureml-fsspec==1.3.0 - - azureml-metrics==0.0.43 - - azureml-mlflow==1.54.0.post1 - - azureml-telemetry==1.54.0 - - backoff==2.2.1 - - backports-tempfile==1.0 - - backports-weakref==1.0.post1 - - banal==1.0.6 - - bcrypt==4.1.2 - - beautifulsoup4==4.12.3 - - blinker==1.7.0 - - bs4==0.0.2 - - build==1.0.3 - - cachetools==5.3.2 - - cattrs==23.2.3 - - certifi==2023.11.17 - - cffi==1.16.0 - - cfgv==3.4.0 - - chardet==5.2.0 - - charset-normalizer==3.3.2 - - chroma-hnswlib==0.7.3 - - chromadb==0.4.22 - - click==8.1.7 - - cloudpickle==2.2.1 - - cohere==4.46 - - colorama==0.4.6 - - colored==1.4.4 - - coloredlogs==15.0.1 - - comm==0.2.1 - - contextlib2==21.6.0 - - contourpy==1.2.0 - - cryptography==41.0.7 - - cycler==0.12.1 - - databricks-cli==0.18.0 - - dataclasses-json==0.6.3 - - dataset==1.6.2 - - datasets==2.16.1 - - debugpy==1.8.1 - - deprecated==1.2.14 - - dill==0.3.7 - - dirtyjson==1.0.8 - - distlib==0.3.8 - - distro==1.9.0 - - dnspython==2.5.0 - - docker==6.1.3 - - docker-pycreds==0.4.0 - - emoji==2.10.0 - - entrypoints==0.4 - - evaluate==0.4.1 - - fairscale==0.4.13 - - faiss-cpu==1.7.4 - - fastapi==0.109.1 - - fastavro==1.9.3 - - filelock==3.13.1 - - filetype==1.2.0 - - fire==0.5.0 - - flask==2.3.3 - - flatbuffers==23.5.26 - - fonttools==4.47.2 - - frozenlist==1.4.1 - - fsspec==2023.10.0 - - gitdb==4.0.11 - - gitpython==3.1.41 - - google-api-core==2.15.0 - - google-auth==2.27.0 - - google-search-results==2.4.1 - - googleapis-common-protos==1.62.0 - - gradientai==1.7.0 - - greenlet==3.0.3 - - grpcio==1.60.0 - - grpcio-tools==1.60.0 - - h11==0.14.0 - - httpcore==1.0.2 - - httptools==0.6.1 - - httpx==0.26.0 - - huggingface-hub==0.20.3 - - humanfriendly==10.0 - - identify==2.5.33 - - idna==3.6 - - importlib-metadata==6.11.0 - - importlib-resources==6.1.1 - - iniconfig==2.0.0 - - ipykernel==6.29.2 - - isodate==0.6.1 - - itsdangerous==2.1.2 - - jaraco-classes==3.3.0 - - jeepney==0.8.0 - - jinja2==3.1.3 - - jmespath==1.0.1 - - joblib==1.3.2 - - jsonpatch==1.33 - - jsonpath-python==1.0.6 - - jsonpickle==3.0.2 - - jsonpointer==2.4 - - jsonschema==4.21.1 - - jsonschema-specifications==2023.12.1 - - jupyter-client==8.6.0 - - jupyter-core==5.7.1 - - keyring==24.3.0 - - kiwisolver==1.4.5 - - knack==0.11.0 - - kubernetes==29.0.0 - - langchain==0.1.7 - - langchain-anthropic==0.0.1.post1 - - langchain-community==0.0.20 - - langchain-core==0.1.23 - - langchain-experimental==0.0.51 - - langchain-openai==0.0.6 - - langchainhub==0.1.14 - - langdetect==1.0.9 - - langgraph==0.0.24 - - langsmith==0.0.87 - - llama-index==0.10.4 - - llama-index-agent-openai==0.1.1 - - llama-index-core==0.10.3 - - llama-index-embeddings-adapter==0.1.0 - - llama-index-embeddings-openai==0.1.1 - - llama-index-finetuning==0.1.0 - - llama-index-legacy==0.9.48 - - llama-index-llms-gradient==0.1.0 - - llama-index-llms-openai==0.1.1 - - llama-index-multi-modal-llms-openai==0.1.1 - - llama-index-postprocessor-cohere-rerank==0.1.0 - - llama-index-program-openai==0.1.1 - - llama-index-question-gen-openai==0.1.1 - - llama-index-readers-file==0.1.3 - - lxml==5.1.0 - - mako==1.3.1 - - markdown==3.5.2 - - markupsafe==2.1.4 - - marshmallow==3.20.2 - - matplotlib==3.8.3 - - mlflow-skinny==2.9.2 - - mmh3==4.1.0 - - monotonic==1.6 - - more-itertools==10.2.0 - - mpmath==1.3.0 - - msal==1.26.0 - - msal-extensions==1.0.0 - - msrest==0.7.1 - - msrestazure==0.6.4 - - multidict==6.0.4 - - multipledispatch==1.0.0 - - multiprocess==0.70.15 - - mypy-extensions==1.0.0 - - ndg-httpsclient==0.5.1 - - neo4j==5.16.0 - - nest-asyncio==1.6.0 - - networkx==3.2.1 - - nltk==3.8.1 - - nodeenv==1.8.0 - - numpy==1.26.3 - - nvidia-cublas-cu12==12.1.3.1 - - nvidia-cuda-cupti-cu12==12.1.105 - - nvidia-cuda-nvrtc-cu12==12.1.105 - - nvidia-cuda-runtime-cu12==12.1.105 - - nvidia-cudnn-cu12==8.9.2.26 - - nvidia-cufft-cu12==11.0.2.54 - - nvidia-curand-cu12==10.3.2.106 - - nvidia-cusolver-cu12==11.4.5.107 - - nvidia-cusparse-cu12==12.1.0.106 - - nvidia-nccl-cu12==2.18.1 - - nvidia-nvjitlink-cu12==12.3.101 - - nvidia-nvtx-cu12==12.1.105 - - oauthlib==3.2.2 - - onnxruntime==1.17.0 - - openai==1.13.3 - - opencensus==0.11.4 - - opencensus-context==0.1.3 - - opencensus-ext-azure==1.1.13 - - opencensus-ext-logging==0.1.1 - - opentelemetry-api==1.22.0 - - opentelemetry-exporter-otlp-proto-common==1.22.0 - - opentelemetry-exporter-otlp-proto-grpc==1.22.0 - - opentelemetry-instrumentation==0.43b0 - - opentelemetry-instrumentation-asgi==0.43b0 - - opentelemetry-instrumentation-fastapi==0.43b0 - - opentelemetry-proto==1.22.0 - - opentelemetry-sdk==1.22.0 - - opentelemetry-semantic-conventions==0.43b0 - - opentelemetry-util-http==0.43b0 - - overrides==7.7.0 - - packaging==23.2 - - pandas==2.2.0 - - paramiko==3.4.0 - - pathspec==0.12.1 - - pgraph-python==0.6.2 - - pillow==10.2.0 - - pkginfo==1.9.6 - - platformdirs==4.1.0 - - pluggy==1.4.0 - - portalocker==2.8.2 - - posthog==3.3.4 - - pre-commit==3.6.0 - - progress==1.6 - - promptflow==0.1.0b8 - - promptflow-tools==0.1.0b15 - - promptflow-vectordb==0.2.3 - - protobuf==4.25.2 - - psutil==5.9.8 - - pulsar-client==3.4.0 - - py-trees==2.2.3 - - pyarrow==13.0.0 - - pyarrow-hotfix==0.6 - - pyasn1==0.5.1 - - pyasn1-modules==0.3.0 - - pycparser==2.21 - - pydantic==1.10.14 - - pydantic-core==2.14.6 - - pydash==6.0.2 - - pydot==2.0.0 - - pygments==2.17.2 - - pyjwt==2.8.0 - - pymongo==4.6.1 - - pymupdf==1.23.22 - - pymupdfb==1.23.22 - - pynacl==1.5.0 - - pyopenssl==23.3.0 - - pyparsing==3.1.1 - - pypdf==4.0.1 - - pyperplan==2.1 - - pypika==0.48.9 - - pyproject-hooks==1.0.0 - - pysocks==1.7.1 - - pytamer==0.1.15 - - pytest==8.0.0 - - python-dateutil==2.8.2 - - python-dotenv==1.0.1 - - python-graphviz==0.20.1 - - python-iso639==2024.1.2 - - python-magic==0.4.27 - - pytz==2023.3.post1 - - pyyaml==6.0.1 - - pyzmq==25.1.2 - - rapidfuzz==3.6.1 - - referencing==0.32.1 - - regex==2023.12.25 - - requests==2.31.0 - - requests-cache==1.1.1 - - requests-oauthlib==1.3.1 - - responses==0.18.0 - - roboticstoolbox-python==1.1.0 - - rpds-py==0.17.1 - - rsa==4.9 - - rtb-data==1.0.1 - - rtoml==0.9.0 - - ruamel-yaml==0.17.40 - - ruamel-yaml-clib==0.2.8 - - safetensors==0.4.2 - - scikit-learn==1.4.0 - - scipy==1.12.0 - - secretstorage==3.3.3 - - sentence-transformers==2.4.0 - - sentencepiece==0.1.99 - - sentry-sdk==1.40.6 - - setproctitle==1.3.3 - - smmap==5.0.1 - - sniffio==1.3.0 - - soupsieve==2.5 - - spatialgeometry==1.1.0 - - spatialmath-python==1.1.9 - - sqlalchemy==2.0.27 - - sqlparse==0.4.4 - - starlette==0.35.1 - - strictyaml==1.7.3 - - swift-sim==1.1.0 - - sympy==1.12 - - tabulate==0.9.0 - - tarski==0.8.2 - - tavily-python==0.3.1 - - tenacity==8.2.3 - - termcolor==2.4.0 - - threadpoolctl==3.2.0 - - tika==2.6.0 - - tiktoken==0.5.2 - - tokenizers==0.15.1 - - tomli==2.0.1 - - torch==2.1.2 - - tornado==6.4 - - tqdm==4.66.1 - - transformers==4.37.2 - - triton==2.1.0 - - typer==0.9.0 - - types-requests==2.31.0.20240125 - - typing-extensions==4.9.0 - - typing-inspect==0.9.0 - - tzdata==2023.4 - - unified-planning==1.0.0 - - unstructured==0.12.2 - - unstructured-client==0.15.5 - - up-aries==0.3.3 - - up-enhsp==0.0.15 - - up-fast-downward==0.2.3 - - up-fmap==0.0.7 - - up-lpg==0.0.7 - - up-pyperplan==1.0.0 - - up-symk==1.1.0 - - up-tamer==1.0.0 - - url-normalize==1.4.3 - - urllib3==2.1.0 - - uvicorn==0.27.0.post1 - - uvloop==0.19.0 - - virtualenv==20.25.0 - - waitress==2.1.2 - - wandb==0.16.3 - - watchfiles==0.21.0 - - websocket-client==1.7.0 - - websockets==12.0 - - werkzeug==3.0.1 - - wikipedia==1.4.0 - - wrapt==1.16.0 - - xxhash==3.4.1 - - yarl==1.9.4 - - zipp==3.17.0 -prefix: /home/blackbird/miniconda3/envs/kios diff --git a/wandb/run-20240303_155426-ftjob-UInKOefMZ40cQzVNlZq43njb/files/config.yaml b/wandb/run-20240303_155426-ftjob-UInKOefMZ40cQzVNlZq43njb/files/config.yaml deleted file mode 100644 index 48076fc0..00000000 --- a/wandb/run-20240303_155426-ftjob-UInKOefMZ40cQzVNlZq43njb/files/config.yaml +++ /dev/null @@ -1,78 +0,0 @@ -wandb_version: 1 - -_wandb: - desc: null - value: - python_version: 3.10.0 - cli_version: 0.16.3 - is_jupyter_run: false - is_kaggle_kernel: false - start_time: 1709477666.67656 - t: - 1: - - 55 - - 95 - 2: - - 55 - - 95 - 3: - - 13 - - 14 - - 23 - - 58 - 4: 3.10.0 - 5: 0.16.3 - 8: - - 5 - 13: linux-x86_64 -id: - desc: null - value: ftjob-UInKOefMZ40cQzVNlZq43njb -created_at: - desc: null - value: '2024-03-01 19:03:57' -error: - desc: null - value: Error(code=None, message=None, param=None, error=None) -fine_tuned_model: - desc: null - value: ft:gpt-3.5-turbo-0125:kifabrik-mirmi::8y1cXwVw -finished_at: - desc: null - value: '2024-03-01 19:10:00' -model: - desc: null - value: gpt-3.5-turbo-0125 -object: - desc: null - value: fine_tuning.job -organization_id: - desc: null - value: org-ygWwisor3PBcF2dPNRDTzzgd -result_files: - desc: null - value: file-rSPSnF6tj1Kft651iTNNnXfU -status: - desc: null - value: succeeded -trained_tokens: - desc: null - value: 311060 -training_file: - desc: null - value: file-EhbEPGHSCMj1bD6TGi6WQMxT -validation_file: - desc: null - value: null -user_provided_suffix: - desc: null - value: null -hyperparameters: - desc: null - value: - n_epochs: 10 - batch_size: 1 - learning_rate_multiplier: 2 -n_train: - desc: null - value: 10 diff --git a/wandb/run-20240303_155426-ftjob-UInKOefMZ40cQzVNlZq43njb/files/media/table/train_data_100_8264d7298a2b7cbb081e.table.json b/wandb/run-20240303_155426-ftjob-UInKOefMZ40cQzVNlZq43njb/files/media/table/train_data_100_8264d7298a2b7cbb081e.table.json deleted file mode 100644 index 3384f5a7..00000000 --- a/wandb/run-20240303_155426-ftjob-UInKOefMZ40cQzVNlZq43njb/files/media/table/train_data_100_8264d7298a2b7cbb081e.table.json +++ /dev/null @@ -1 +0,0 @@ -{"columns": ["role: system", "role: user", "role: assistant"], "data": [["You are an excellent behavior tree builder for robotic assembly tasks", "You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you construct robot task plans represented by behaiovr trees. \n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object that is ready to be parsed.\n\n Following is the domain knowledge you need in pddl domain format, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\"\"\"\n(define (domain robot_assembly_problem-domain)\n (:requirements :strips :typing)\n (:types\n thing - object\n tool part hand - thing\n )\n (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part))\n (:action pick_up\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part))\n :effect (and (hold ?tool ?part) (not (is_free ?tool))))\n (:action put_down\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (hold ?tool ?part) (hold ?hand ?tool))\n :effect (and (not (hold ?hand ?part)) (is_free ?tool)))\n (:action place\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2)))\n (:action detach\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2))))\n (:action insert\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2)))\n (:action pull\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2))))\n (:action screw\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2)))\n (:action unscrew\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2))))\n (:action load_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (is_equippable ?tool) (is_free ?hand))\n :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool)))\n (:action unload_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (hold ?hand ?tool) (is_free ?tool))\n :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool))))\n)\n\"\"\"\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_free( or ): or is free and can hold something. (default: true)\n- is_equippable(): is ready to be eqquipped by a robot hand. (default: true)\n- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false)\n- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false)\n- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false)\n- can_place_to(, ): can be placed into (placement assembly constraint) (default: false)\n- hold(, ): is holding (default: false)\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding .\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into .\n- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free.\n- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: unload_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to unload the outwardgripper in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\",\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding a outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\",\n },\n {\n \"summary\": \"Unload the outwardgripper in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outwardgripper)\",\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: insert(left_hand, defaultgripper, gear1, shaft1)\n---------------\n\n ", "{\n \"summary\": \"Selector to insert gear1 into shaft1 using defaultgripper in the left hand\",\n \"name\": \"selector: insert(left_hand, defaultgripper, gear1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"Check the target that gear1 is inserted into shaft1\",\n \"name\": \"target: is_inserted_to(gear1, shaft1)\",\n },\n {\n \"summary\": \"Sequence to insert gear1 into shaft1 using defaultgripper in the left hand\",\n \"name\": \"sequence: insert(left_hand, defaultgripper, gear1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"Check the precondition that the left hand is holding defaultgripper\",\n \"name\": \"precondition: hold(left_hand, defaultgripper)\",\n },\n {\n \"summary\": \"Check the precondition that defaultgripper is holding gear1\",\n \"name\": \"precondition: hold(defaultgripper, gear1)\",\n },\n {\n \"summary\": \"Insert gear1 into shaft1 using defaultgripper\",\n \"name\": \"action: insert(left_hand, defaultgripper, gear1, shaft1)\",\n }\n ]\n }\n ]\n}"], ["You are an excellent behavior tree builder for robotic assembly tasks", "You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you construct robot task plans represented by behaiovr trees. \n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need in pddl domain format, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\"\"\"\n(define (domain robot_assembly_problem-domain)\n (:requirements :strips :typing)\n (:types\n thing - object\n tool part hand - thing\n )\n (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part))\n (:action pick_up\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part))\n :effect (and (hold ?tool ?part) (not (is_free ?tool))))\n (:action put_down\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (hold ?tool ?part) (hold ?hand ?tool))\n :effect (and (not (hold ?hand ?part)) (is_free ?tool)))\n (:action place\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2)))\n (:action detach\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2))))\n (:action insert\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2)))\n (:action pull\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2))))\n (:action screw\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2)))\n (:action unscrew\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2))))\n (:action load_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (is_equippable ?tool) (is_free ?hand))\n :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool)))\n (:action unload_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (hold ?hand ?tool) (is_free ?tool))\n :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool))))\n)\n\"\"\"\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_free( or ): or is free and can hold something. (default: true)\n- is_equippable(): is ready to be eqquipped by a robot hand. (default: true)\n- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false)\n- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false)\n- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false)\n- can_place_to(, ): can be placed into (placement assembly constraint) (default: false)\n- hold(, ): is holding (default: false)\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding .\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into .\n- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free.\n- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: unload_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to unload the outwardgripper in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\",\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding a outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\",\n },\n {\n \"summary\": \"Unload the outwardgripper in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outwardgripper)\",\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: insert(left_hand, defaultgripper, gear1, shaft1)\n---------------\n\n ", "{\n \"summary\": \"Selector to insert gear1 into shaft1 using defaultgripper in the left hand\",\n \"name\": \"selector: insert(left_hand, defaultgripper, gear1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"Check the target that gear1 is inserted into shaft1\",\n \"name\": \"target: is_inserted_to(gear1, shaft1)\",\n },\n {\n \"summary\": \"Sequence to insert gear1 into shaft1 using defaultgripper in the left hand\",\n \"name\": \"sequence: insert(left_hand, defaultgripper, gear1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"Check the precondition that the left hand is holding defaultgripper\",\n \"name\": \"precondition: hold(left_hand, defaultgripper)\",\n },\n {\n \"summary\": \"Check the precondition that defaultgripper is holding gear1\",\n \"name\": \"precondition: hold(defaultgripper, gear1)\",\n },\n {\n \"summary\": \"Insert gear1 into shaft1 using defaultgripper\",\n \"name\": \"action: insert(left_hand, defaultgripper, gear1, shaft1)\",\n }\n ]\n }\n ]\n}"], ["You are an excellent behavior tree builder for robotic assembly tasks", "You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you construct robot task plans represented by behaiovr trees. \n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need in pddl domain format, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\"\"\"\n(define (domain robot_assembly_problem-domain)\n (:requirements :strips :typing)\n (:types\n thing - object\n tool part hand - thing\n )\n (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part))\n (:action pick_up\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part))\n :effect (and (hold ?tool ?part) (not (is_free ?tool))))\n (:action put_down\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (hold ?tool ?part) (hold ?hand ?tool))\n :effect (and (not (hold ?hand ?part)) (is_free ?tool)))\n (:action place\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2)))\n (:action detach\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2))))\n (:action insert\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2)))\n (:action pull\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2))))\n (:action screw\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2)))\n (:action unscrew\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2))))\n (:action load_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (is_equippable ?tool) (is_free ?hand))\n :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool)))\n (:action unload_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (hold ?hand ?tool) (is_free ?tool))\n :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool))))\n)\n\"\"\"\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_free( or ): or is free and can hold something. (default: true)\n- is_equippable(): is ready to be eqquipped by a robot hand. (default: true)\n- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false)\n- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false)\n- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false)\n- can_place_to(, ): can be placed into (placement assembly constraint) (default: false)\n- hold(, ): is holding (default: false)\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding .\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into .\n- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free.\n- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: unload_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to unload the outwardgripper in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\",\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding a outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\",\n },\n {\n \"summary\": \"Unload the outwardgripper in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outwardgripper)\",\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: insert(left_hand, defaultgripper, gear1, shaft1)\n---------------\n\n\n Return a JSON object.\n ", "{\n \"summary\": \"Selector to insert gear1 into shaft1 using defaultgripper in the left hand\",\n \"name\": \"selector: insert(left_hand, defaultgripper, gear1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"Check the target that gear1 is inserted into shaft1\",\n \"name\": \"target: is_inserted_to(gear1, shaft1)\",\n },\n {\n \"summary\": \"Sequence to insert gear1 into shaft1 using defaultgripper in the left hand\",\n \"name\": \"sequence: insert(left_hand, defaultgripper, gear1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"Check the precondition that the left hand is holding defaultgripper\",\n \"name\": \"precondition: hold(left_hand, defaultgripper)\",\n },\n {\n \"summary\": \"Check the precondition that defaultgripper is holding gear1\",\n \"name\": \"precondition: hold(defaultgripper, gear1)\",\n },\n {\n \"summary\": \"Insert gear1 into shaft1 using defaultgripper\",\n \"name\": \"action: insert(left_hand, defaultgripper, gear1, shaft1)\",\n }\n ]\n }\n ]\n}"], ["You are an excellent behavior tree builder for robotic assembly tasks", "You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you construct robot task plans represented by behaiovr trees. \n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need in pddl domain format, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\"\"\"\n(define (domain robot_assembly_problem-domain)\n (:requirements :strips :typing)\n (:types\n thing - object\n tool part hand - thing\n )\n (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part))\n (:action pick_up\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part))\n :effect (and (hold ?tool ?part) (not (is_free ?tool))))\n (:action put_down\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (hold ?tool ?part) (hold ?hand ?tool))\n :effect (and (not (hold ?hand ?part)) (is_free ?tool)))\n (:action place\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2)))\n (:action detach\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2))))\n (:action insert\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2)))\n (:action pull\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2))))\n (:action screw\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2)))\n (:action unscrew\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2))))\n (:action load_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (is_equippable ?tool) (is_free ?hand))\n :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool)))\n (:action unload_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (hold ?hand ?tool) (is_free ?tool))\n :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool))))\n)\n\"\"\"\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_free( or ): or is free and can hold something. (default: true)\n- is_equippable(): is ready to be eqquipped by a robot hand. (default: true)\n- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false)\n- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false)\n- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false)\n- can_place_to(, ): can be placed into (placement assembly constraint) (default: false)\n- hold(, ): is holding (default: false)\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding .\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into .\n- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free.\n- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: unload_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to unload the outwardgripper in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outwardgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\"\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding a outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"Unload the outwardgripper in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outwardgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: insert(left_hand, defaultgripper, gear1, shaft1)\n---------------\n\n\n Return a JSON object.\n ", "{\n \"summary\": \"Selector to insert gear1 into shaft1 using defaultgripper in the left hand\",\n \"name\": \"selector: insert(left_hand, defaultgripper, gear1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"Check the target that gear1 is inserted into shaft1\",\n \"name\": \"target: is_inserted_to(gear1, shaft1)\"\n },\n {\n \"summary\": \"Sequence to insert gear1 into shaft1 using defaultgripper in the left hand\",\n \"name\": \"sequence: insert(left_hand, defaultgripper, gear1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"Check the precondition that the left hand is holding defaultgripper\",\n \"name\": \"precondition: hold(left_hand, defaultgripper)\"\n },\n {\n \"summary\": \"Check the precondition that defaultgripper is holding gear1\",\n \"name\": \"precondition: hold(defaultgripper, gear1)\"\n },\n {\n \"summary\": \"Insert gear1 into shaft1 using defaultgripper\",\n \"name\": \"action: insert(left_hand, defaultgripper, gear1, shaft1)\"\n }\n ]\n }\n ]\n}"], ["You are an excellent behavior tree builder for robotic assembly tasks", "You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you construct robot task plans represented by behaiovr trees. \n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need in pddl domain format, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\"\"\"\n(define (domain robot_assembly_problem-domain)\n (:requirements :strips :typing)\n (:types\n thing - object\n tool part hand - thing\n )\n (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part))\n (:action pick_up\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part))\n :effect (and (hold ?tool ?part) (not (is_free ?tool))))\n (:action put_down\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (hold ?tool ?part) (hold ?hand ?tool))\n :effect (and (not (hold ?hand ?part)) (is_free ?tool)))\n (:action place\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2)))\n (:action detach\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2))))\n (:action insert\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2)))\n (:action pull\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2))))\n (:action screw\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2)))\n (:action unscrew\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2))))\n (:action load_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (is_equippable ?tool) (is_free ?hand))\n :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool)))\n (:action unload_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (hold ?hand ?tool) (is_free ?tool))\n :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool))))\n)\n\"\"\"\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_free( or ): or is free and can hold something. (default: true)\n- is_equippable(): is ready to be eqquipped by a robot hand. (default: true)\n- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false)\n- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false)\n- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false)\n- can_place_to(, ): can be placed into (placement assembly constraint) (default: false)\n- hold(, ): is holding (default: false)\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding .\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into .\n- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free.\n- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: unload_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to unload the outwardgripper in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outwardgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\"\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding a outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"Unload the outwardgripper in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outwardgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)\n---------------\n\n\n Return a JSON object.\n ", "{\n \"summary\": \"Selector to insert shaft1 into gearbase_hole1 using parallel_box1 in the left hand\",\n \"name\": \"selector: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)\",\n \"children\": [\n {\n \"summary\": \"Check the target that shaft1 is inserted into gearbase_hole1\",\n \"name\": \"target: is_inserted_to(shaft1, gearbase_hole1)\"\n },\n {\n \"summary\": \"Sequence to insert shaft1 into gearbase_hole1 using parallel_box1 in the left hand\",\n \"name\": \"sequence: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)\",\n \"children\": [\n {\n \"summary\": \"Check the precondition that the left hand is holding parallel_box1\",\n \"name\": \"precondition: hold(left_hand, parallel_box1)\"\n },\n {\n \"summary\": \"Check the precondition that parallel_box1 is holding shaft1\",\n \"name\": \"precondition: hold(parallel_box1, shaft1)\"\n },\n {\n \"summary\": \"Insert shaft1 into gearbase_hole1 using parallel_box1\",\n \"name\": \"action: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)\"\n }\n ]\n }\n ]\n}"], ["You are an excellent behavior tree builder for robotic assembly tasks", "You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you construct robot task plans represented by behaiovr trees. \n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need in pddl domain format, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\"\"\"\n(define (domain robot_assembly_problem-domain)\n (:requirements :strips :typing)\n (:types\n thing - object\n tool part hand - thing\n )\n (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part))\n (:action pick_up\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part))\n :effect (and (hold ?tool ?part) (not (is_free ?tool))))\n (:action put_down\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (hold ?tool ?part) (hold ?hand ?tool))\n :effect (and (not (hold ?hand ?part)) (is_free ?tool)))\n (:action place\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2)))\n (:action detach\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2))))\n (:action insert\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2)))\n (:action pull\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2))))\n (:action screw\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2)))\n (:action unscrew\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2))))\n (:action load_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (is_equippable ?tool) (is_free ?hand))\n :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool)))\n (:action unload_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (hold ?hand ?tool) (is_free ?tool))\n :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool))))\n)\n\"\"\"\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_free( or ): or is free and can hold something. (default: true)\n- is_equippable(): is ready to be eqquipped by a robot hand. (default: true)\n- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false)\n- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false)\n- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false)\n- can_place_to(, ): can be placed into (placement assembly constraint) (default: false)\n- hold(, ): is holding (default: false)\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding .\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into .\n- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free.\n- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: unload_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to unload the outwardgripper in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outwardgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\"\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding a outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"Unload the outwardgripper in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outwardgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)\n---------------\n\n\n Return a JSON object.\n ", "{\n \"summary\": \"Selector to insert shaft1 into gearbase_hole1 using parallel_box1 in the left hand\",\n \"name\": \"selector: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)\",\n \"children\": [\n {\n \"summary\": \"Check the target that shaft1 is inserted into gearbase_hole1\",\n \"name\": \"target: is_inserted_to(shaft1, gearbase_hole1)\"\n },\n {\n \"summary\": \"Sequence to insert shaft1 into gearbase_hole1 using parallel_box1 in the left hand\",\n \"name\": \"sequence: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)\",\n \"children\": [\n {\n \"summary\": \"Check the precondition that the left hand is holding parallel_box1\",\n \"name\": \"precondition: hold(left_hand, parallel_box1)\"\n },\n {\n \"summary\": \"Check the precondition that parallel_box1 is holding shaft1\",\n \"name\": \"precondition: hold(parallel_box1, shaft1)\"\n },\n {\n \"summary\": \"Insert shaft1 into gearbase_hole1 using parallel_box1\",\n \"name\": \"action: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)\"\n }\n ]\n }\n ]\n}"], ["You are an excellent behavior tree builder for robotic assembly tasks", "You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you construct robot task plans represented by behaiovr trees. \n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need in pddl domain format, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\"\"\"\n(define (domain robot_assembly_problem-domain)\n (:requirements :strips :typing)\n (:types\n thing - object\n tool part hand - thing\n )\n (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part))\n (:action pick_up\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part))\n :effect (and (hold ?tool ?part) (not (is_free ?tool))))\n (:action put_down\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (hold ?tool ?part) (hold ?hand ?tool))\n :effect (and (not (hold ?hand ?part)) (is_free ?tool)))\n (:action place\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2)))\n (:action detach\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2))))\n (:action insert\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2)))\n (:action pull\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2))))\n (:action screw\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2)))\n (:action unscrew\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2))))\n (:action load_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (is_equippable ?tool) (is_free ?hand))\n :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool)))\n (:action unload_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (hold ?hand ?tool) (is_free ?tool))\n :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool))))\n)\n\"\"\"\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_free( or ): or is free and can hold something. (default: true)\n- is_equippable(): is ready to be eqquipped by a robot hand. (default: true)\n- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false)\n- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false)\n- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false)\n- can_place_to(, ): can be placed into (placement assembly constraint) (default: false)\n- hold(, ): is holding (default: false)\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding .\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into .\n- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free.\n- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: unload_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to unload the outwardgripper in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outwardgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\"\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding a outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"Unload the outwardgripper in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outwardgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)\n---------------\n\n\n Return a JSON object.\n ", "{\n \"summary\": \"Selector to insert shaft1 into gearbase_hole1 using parallel_box1 in the left hand\",\n \"name\": \"selector: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)\",\n \"children\": [\n {\n \"summary\": \"Check the target that shaft1 is inserted into gearbase_hole1\",\n \"name\": \"target: is_inserted_to(shaft1, gearbase_hole1)\"\n },\n {\n \"summary\": \"Sequence to insert shaft1 into gearbase_hole1 using parallel_box1 in the left hand\",\n \"name\": \"sequence: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)\",\n \"children\": [\n {\n \"summary\": \"Check the precondition that the left hand is holding parallel_box1\",\n \"name\": \"precondition: hold(left_hand, parallel_box1)\"\n },\n {\n \"summary\": \"Check the precondition that parallel_box1 is holding shaft1\",\n \"name\": \"precondition: hold(parallel_box1, shaft1)\"\n },\n {\n \"summary\": \"Insert shaft1 into gearbase_hole1 using parallel_box1\",\n \"name\": \"action: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)\"\n }\n ]\n }\n ]\n}"], ["You are an excellent behavior tree builder for robotic assembly tasks", "You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you construct robot task plans represented by behaiovr trees. \n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need in pddl domain format, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\"\"\"\n(define (domain robot_assembly_problem-domain)\n (:requirements :strips :typing)\n (:types\n thing - object\n tool part hand - thing\n )\n (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part))\n (:action pick_up\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part))\n :effect (and (hold ?tool ?part) (not (is_free ?tool))))\n (:action put_down\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (hold ?tool ?part) (hold ?hand ?tool))\n :effect (and (not (hold ?hand ?part)) (is_free ?tool)))\n (:action place\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2)))\n (:action detach\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2))))\n (:action insert\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2)))\n (:action pull\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2))))\n (:action screw\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2)))\n (:action unscrew\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2))))\n (:action load_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (is_equippable ?tool) (is_free ?hand))\n :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool)))\n (:action unload_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (hold ?hand ?tool) (is_free ?tool))\n :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool))))\n)\n\"\"\"\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_free( or ): or is free and can hold something. (default: true)\n- is_equippable(): is ready to be eqquipped by a robot hand. (default: true)\n- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false)\n- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false)\n- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false)\n- can_place_to(, ): can be placed into (placement assembly constraint) (default: false)\n- hold(, ): is holding (default: false)\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding .\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into .\n- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free.\n- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: unload_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to unload the outwardgripper in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outwardgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\"\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding a outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"Unload the outwardgripper in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outwardgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: load_tool(left_hand, parallel_box1)\n---------------\n\n\n Return a JSON object.\n ", "{\n \"summary\": \"Selector to load the parallel_box1 in the left hand\",\n \"name\": \"selector: load_tool(left_hand, parallel_box1)\",\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\"\n },\n {\n \"summary\": \"Sequence to load the tool in the left hand\",\n \"name\": \"sequence: load_tool(left_hand, parallel_box1)\",\n \"children\": [\n {\n \"summary\": \"Check the precondition that the parallel_box1 is equippable\",\n \"name\": \"precondition: is_equippable(parallel_box1)\"\n },\n {\n \"summary\": \"Check the precondition that the left hand is free\",\n \"name\": \"precondition: is_free(left_hand)\"\n },\n {\n \"summary\": \"Load the parallel_box1 in the left hand\",\n \"name\": \"action: load_tool(left_hand, parallel_box1)\"\n }\n ]\n }\n ]\n}"], ["You are an excellent behavior tree builder for robotic assembly tasks", "You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you construct robot task plans represented by behaiovr trees. \n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need in pddl domain format, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\"\"\"\n(define (domain robot_assembly_problem-domain)\n (:requirements :strips :typing)\n (:types\n thing - object\n tool part hand - thing\n )\n (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part))\n (:action pick_up\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part))\n :effect (and (hold ?tool ?part) (not (is_free ?tool))))\n (:action put_down\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (hold ?tool ?part) (hold ?hand ?tool))\n :effect (and (not (hold ?hand ?part)) (is_free ?tool)))\n (:action place\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2)))\n (:action detach\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2))))\n (:action insert\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2)))\n (:action pull\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2))))\n (:action screw\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2)))\n (:action unscrew\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2))))\n (:action load_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (is_equippable ?tool) (is_free ?hand))\n :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool)))\n (:action unload_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (hold ?hand ?tool) (is_free ?tool))\n :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool))))\n)\n\"\"\"\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_free( or ): or is free and can hold something. (default: true)\n- is_equippable(): is ready to be eqquipped by a robot hand. (default: true)\n- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false)\n- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false)\n- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false)\n- can_place_to(, ): can be placed into (placement assembly constraint) (default: false)\n- hold(, ): is holding (default: false)\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding .\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into .\n- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free.\n- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: unload_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to unload the outwardgripper in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outwardgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\"\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding a outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"Unload the outwardgripper in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outwardgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: unload_tool(left_hand, outward_claw)\n---------------\n\n\n Return a JSON object.\n ", "{\n \"summary\": \"Selector to unload the outward_claw in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outward_claw)\",\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\"\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outward_claw)\",\n \"children\": [\n {\n \"summary\": \"Check the precondition that the left hand is holding a outward_claw\",\n \"name\": \"precondition: hold(left_hand, outward_claw)\"\n },\n {\n \"summary\": \"Unload the outward_claw in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outward_claw)\"\n }\n ]\n }\n ]\n}"], ["You are an excellent behavior tree builder for robotic assembly tasks", "You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you construct robot task plans represented by behaiovr trees. \n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need in pddl domain format, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\"\"\"\n(define (domain robot_assembly_problem-domain)\n (:requirements :strips :typing)\n (:types\n thing - object\n tool part hand - thing\n )\n (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part))\n (:action pick_up\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part))\n :effect (and (hold ?tool ?part) (not (is_free ?tool))))\n (:action put_down\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (hold ?tool ?part) (hold ?hand ?tool))\n :effect (and (not (hold ?hand ?part)) (is_free ?tool)))\n (:action place\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2)))\n (:action detach\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2))))\n (:action insert\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2)))\n (:action pull\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2))))\n (:action screw\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2)))\n (:action unscrew\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2))))\n (:action load_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (is_equippable ?tool) (is_free ?hand))\n :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool)))\n (:action unload_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (hold ?hand ?tool) (is_free ?tool))\n :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool))))\n)\n\"\"\"\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_free( or ): or is free and can hold something. (default: true)\n- is_equippable(): is ready to be eqquipped by a robot hand. (default: true)\n- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false)\n- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false)\n- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false)\n- can_place_to(, ): can be placed into (placement assembly constraint) (default: false)\n- hold(, ): is holding (default: false)\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding .\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into .\n- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free.\n- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: unload_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to unload the outwardgripper in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outwardgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\"\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding a outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"Unload the outwardgripper in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outwardgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: pick_up(left_hand, parallel_box1, shaft1)\n---------------\n\n\n Return a JSON object.\n ", "{\n \"summary\": \"Selector to pick up the shaft1 with the parallel_box1 in the left hand\",\n \"name\": \"selector: pick_up(left_hand, parallel_box1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"Check the target that the parallel_box1 is holding the shaft1\",\n \"name\": \"target: hold(parallel_box1, shaft1)\"\n },\n {\n \"summary\": \"Sequence to pick up the shaft1 with the parallel_box1 in the left hand\",\n \"name\": \"sequence: pick_up(left_hand, parallel_box1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"Check the precondition that the left hand is holding the parallel_box1\",\n \"name\": \"precondition: hold(left_hand, parallel_box1)\"\n },\n {\n \"summary\": \"Pick up the shaft1 with the parallel_box1 in the left hand\",\n \"name\": \"action: pick_up(left_hand, parallel_box1, shaft1)\"\n }\n ]\n }\n ]\n}"]]} \ No newline at end of file diff --git a/wandb/run-20240303_155426-ftjob-UInKOefMZ40cQzVNlZq43njb/files/output.log b/wandb/run-20240303_155426-ftjob-UInKOefMZ40cQzVNlZq43njb/files/output.log deleted file mode 100644 index 87412b3b..00000000 --- a/wandb/run-20240303_155426-ftjob-UInKOefMZ40cQzVNlZq43njb/files/output.log +++ /dev/null @@ -1,45 +0,0 @@ -wandb: Waiting for the OpenAI fine-tuning job to be finished... -wandb: Fine-tuning finished, logging metrics, model metadata, and more to W&B -wandb: Waiting for artifact train-file-EhbEPGHSCMj1bD6TGi6WQMxT to be committed... -wandb: | 0.456 MB of 0.456 MB uploaded -wandb: Committed artifact proneverfake/OpenAI-Fine-Tune/train-file-EhbEPGHSCMj1bD6TGi6WQMxT:v0 -Traceback (most recent call last): - File "/home/blackbird/kios/data/finetuning/visualize.py", line 5, in - WandbLogger.sync(fine_tune_job_id="ftjob-UInKOefMZ40cQzVNlZq43njb") - File "/home/blackbird/miniconda3/envs/kios/lib/python3.10/site-packages/wandb/integration/openai/fine_tuning.py", line 148, in sync - cls._log_fine_tune( - File "/home/blackbird/miniconda3/envs/kios/lib/python3.10/site-packages/wandb/integration/openai/fine_tuning.py", line 237, in _log_fine_tune - cls._log_artifacts(fine_tune, project, entity) - File "/home/blackbird/miniconda3/envs/kios/lib/python3.10/site-packages/wandb/integration/openai/fine_tuning.py", line 344, in _log_artifacts - json.dump(dict_fine_tune, f, indent=2) - File "/home/blackbird/miniconda3/envs/kios/lib/python3.10/json/__init__.py", line 179, in dump - for chunk in iterable: - File "/home/blackbird/miniconda3/envs/kios/lib/python3.10/json/encoder.py", line 431, in _iterencode - yield from _iterencode_dict(o, _current_indent_level) - File "/home/blackbird/miniconda3/envs/kios/lib/python3.10/json/encoder.py", line 405, in _iterencode_dict - yield from chunks - File "/home/blackbird/miniconda3/envs/kios/lib/python3.10/json/encoder.py", line 438, in _iterencode - o = _default(o) - File "/home/blackbird/miniconda3/envs/kios/lib/python3.10/json/encoder.py", line 179, in default - raise TypeError(f'Object of type {o.__class__.__name__} ' -TypeError: Object of type Error is not JSON serializable -Traceback (most recent call last): - File "/home/blackbird/kios/data/finetuning/visualize.py", line 5, in - WandbLogger.sync(fine_tune_job_id="ftjob-UInKOefMZ40cQzVNlZq43njb") - File "/home/blackbird/miniconda3/envs/kios/lib/python3.10/site-packages/wandb/integration/openai/fine_tuning.py", line 148, in sync - cls._log_fine_tune( - File "/home/blackbird/miniconda3/envs/kios/lib/python3.10/site-packages/wandb/integration/openai/fine_tuning.py", line 237, in _log_fine_tune - cls._log_artifacts(fine_tune, project, entity) - File "/home/blackbird/miniconda3/envs/kios/lib/python3.10/site-packages/wandb/integration/openai/fine_tuning.py", line 344, in _log_artifacts - json.dump(dict_fine_tune, f, indent=2) - File "/home/blackbird/miniconda3/envs/kios/lib/python3.10/json/__init__.py", line 179, in dump - for chunk in iterable: - File "/home/blackbird/miniconda3/envs/kios/lib/python3.10/json/encoder.py", line 431, in _iterencode - yield from _iterencode_dict(o, _current_indent_level) - File "/home/blackbird/miniconda3/envs/kios/lib/python3.10/json/encoder.py", line 405, in _iterencode_dict - yield from chunks - File "/home/blackbird/miniconda3/envs/kios/lib/python3.10/json/encoder.py", line 438, in _iterencode - o = _default(o) - File "/home/blackbird/miniconda3/envs/kios/lib/python3.10/json/encoder.py", line 179, in default - raise TypeError(f'Object of type {o.__class__.__name__} ' -TypeError: Object of type Error is not JSON serializable \ No newline at end of file diff --git a/wandb/run-20240303_155426-ftjob-UInKOefMZ40cQzVNlZq43njb/files/requirements.txt b/wandb/run-20240303_155426-ftjob-UInKOefMZ40cQzVNlZq43njb/files/requirements.txt deleted file mode 100644 index 63c80899..00000000 --- a/wandb/run-20240303_155426-ftjob-UInKOefMZ40cQzVNlZq43njb/files/requirements.txt +++ /dev/null @@ -1,376 +0,0 @@ -adal==1.2.7 -aenum==3.1.15 -aiohttp==3.9.1 -aiosignal==1.3.1 -alembic==1.13.1 -annotated-types==0.6.0 -ansitable==0.9.10 -anthropic==0.8.1 -antlr4-python3-runtime==4.7.2 -anyio==4.2.0 -appdirs==1.4.4 -applicationinsights==0.11.10 -argcomplete==3.2.2 -asgiref==3.7.2 -asttokens==2.0.5 -async-timeout==4.0.3 -attrs==23.2.0 -azure-ai-generative==1.0.0b2 -azure-ai-ml==1.12.1 -azure-ai-resources==1.0.0b2 -azure-common==1.1.28 -azure-core==1.29.7 -azure-graphrbac==0.61.1 -azure-identity==1.15.0 -azure-keyvault-certificates==4.7.0 -azure-keyvault-keys==4.8.0 -azure-keyvault-secrets==4.7.0 -azure-keyvault==4.2.0 -azure-mgmt-authorization==4.0.0 -azure-mgmt-containerregistry==10.3.0 -azure-mgmt-core==1.4.0 -azure-mgmt-keyvault==10.3.0 -azure-mgmt-network==25.1.0 -azure-mgmt-resource==23.0.1 -azure-mgmt-storage==21.1.0 -azure-search-documents==11.4.0b11 -azure-storage-blob==12.19.0 -azure-storage-file-datalake==12.14.0 -azure-storage-file-share==12.15.0 -azureml-core==1.54.0.post1 -azureml-dataprep-native==41.0.0 -azureml-dataprep-rslex==2.22.2 -azureml-dataprep==5.1.2 -azureml-fsspec==1.3.0 -azureml-metrics==0.0.43 -azureml-mlflow==1.54.0.post1 -azureml-telemetry==1.54.0 -backoff==2.2.1 -backports.tempfile==1.0 -backports.weakref==1.0.post1 -banal==1.0.6 -bcrypt==4.1.2 -beautifulsoup4==4.12.3 -blinker==1.7.0 -bs4==0.0.2 -build==1.0.3 -cachetools==5.3.2 -cattrs==23.2.3 -certifi==2023.11.17 -cffi==1.16.0 -cfgv==3.4.0 -chardet==5.2.0 -charset-normalizer==3.3.2 -chroma-hnswlib==0.7.3 -chromadb==0.4.22 -click==8.1.7 -cloudpickle==2.2.1 -cohere==4.46 -colorama==0.4.6 -colored==1.4.4 -coloredlogs==15.0.1 -comm==0.2.1 -contextlib2==21.6.0 -contourpy==1.2.0 -cryptography==41.0.7 -cycler==0.12.1 -databricks-cli==0.18.0 -dataclasses-json==0.6.3 -dataset==1.6.2 -datasets==2.16.1 -debugpy==1.8.1 -decorator==5.1.1 -deprecated==1.2.14 -dill==0.3.7 -dirtyjson==1.0.8 -distlib==0.3.8 -distro==1.9.0 -dnspython==2.5.0 -docker-pycreds==0.4.0 -docker==6.1.3 -emoji==2.10.0 -entrypoints==0.4 -evaluate==0.4.1 -exceptiongroup==1.2.0 -executing==0.8.3 -fairscale==0.4.13 -faiss-cpu==1.7.4 -fastapi==0.109.1 -fastavro==1.9.3 -filelock==3.13.1 -filetype==1.2.0 -fire==0.5.0 -flask==2.3.3 -flatbuffers==23.5.26 -fonttools==4.47.2 -frozenlist==1.4.1 -fsspec==2023.10.0 -gitdb==4.0.11 -gitpython==3.1.41 -google-api-core==2.15.0 -google-auth==2.27.0 -google-search-results==2.4.1 -googleapis-common-protos==1.62.0 -gradientai==1.7.0 -graphviz==0.20.1 -greenlet==3.0.3 -grpcio-tools==1.60.0 -grpcio==1.60.0 -h11==0.14.0 -httpcore==1.0.2 -httptools==0.6.1 -httpx==0.26.0 -huggingface-hub==0.20.3 -humanfriendly==10.0 -identify==2.5.33 -idna==3.6 -importlib-metadata==6.11.0 -importlib-resources==6.1.1 -iniconfig==2.0.0 -ipykernel==6.29.2 -ipython==8.20.0 -isodate==0.6.1 -itsdangerous==2.1.2 -jaraco.classes==3.3.0 -jedi==0.18.1 -jeepney==0.8.0 -jinja2==3.1.3 -jmespath==1.0.1 -joblib==1.3.2 -jsonpatch==1.33 -jsonpath-python==1.0.6 -jsonpickle==3.0.2 -jsonpointer==2.4 -jsonschema-specifications==2023.12.1 -jsonschema==4.21.1 -jupyter-client==8.6.0 -jupyter-core==5.7.1 -keyring==24.3.0 -kios-bt-planning==0.1.0 -kiwisolver==1.4.5 -knack==0.11.0 -kubernetes==29.0.0 -langchain-anthropic==0.0.1.post1 -langchain-community==0.0.20 -langchain-core==0.1.23 -langchain-experimental==0.0.51 -langchain-openai==0.0.6 -langchain==0.1.7 -langchainhub==0.1.14 -langdetect==1.0.9 -langgraph==0.0.24 -langsmith==0.0.87 -llama-index-agent-openai==0.1.1 -llama-index-core==0.10.3 -llama-index-embeddings-adapter==0.1.0 -llama-index-embeddings-openai==0.1.1 -llama-index-finetuning==0.1.0 -llama-index-legacy==0.9.48 -llama-index-llms-gradient==0.1.0 -llama-index-llms-openai==0.1.1 -llama-index-multi-modal-llms-openai==0.1.1 -llama-index-postprocessor-cohere-rerank==0.1.0 -llama-index-program-openai==0.1.1 -llama-index-question-gen-openai==0.1.1 -llama-index-readers-file==0.1.3 -llama-index==0.10.4 -lxml==5.1.0 -mako==1.3.1 -markdown==3.5.2 -markupsafe==2.1.4 -marshmallow==3.20.2 -matplotlib-inline==0.1.6 -matplotlib==3.8.3 -mlflow-skinny==2.9.2 -mmh3==4.1.0 -monotonic==1.6 -more-itertools==10.2.0 -mpmath==1.3.0 -msal-extensions==1.0.0 -msal==1.26.0 -msrest==0.7.1 -msrestazure==0.6.4 -multidict==6.0.4 -multipledispatch==1.0.0 -multiprocess==0.70.15 -mypy-extensions==1.0.0 -ndg-httpsclient==0.5.1 -neo4j==5.16.0 -nest-asyncio==1.6.0 -networkx==3.2.1 -nltk==3.8.1 -nodeenv==1.8.0 -numpy==1.26.3 -nvidia-cublas-cu12==12.1.3.1 -nvidia-cuda-cupti-cu12==12.1.105 -nvidia-cuda-nvrtc-cu12==12.1.105 -nvidia-cuda-runtime-cu12==12.1.105 -nvidia-cudnn-cu12==8.9.2.26 -nvidia-cufft-cu12==11.0.2.54 -nvidia-curand-cu12==10.3.2.106 -nvidia-cusolver-cu12==11.4.5.107 -nvidia-cusparse-cu12==12.1.0.106 -nvidia-nccl-cu12==2.18.1 -nvidia-nvjitlink-cu12==12.3.101 -nvidia-nvtx-cu12==12.1.105 -oauthlib==3.2.2 -onnxruntime==1.17.0 -openai==1.13.3 -opencensus-context==0.1.3 -opencensus-ext-azure==1.1.13 -opencensus-ext-logging==0.1.1 -opencensus==0.11.4 -opentelemetry-api==1.22.0 -opentelemetry-exporter-otlp-proto-common==1.22.0 -opentelemetry-exporter-otlp-proto-grpc==1.22.0 -opentelemetry-instrumentation-asgi==0.43b0 -opentelemetry-instrumentation-fastapi==0.43b0 -opentelemetry-instrumentation==0.43b0 -opentelemetry-proto==1.22.0 -opentelemetry-sdk==1.22.0 -opentelemetry-semantic-conventions==0.43b0 -opentelemetry-util-http==0.43b0 -overrides==7.7.0 -packaging==23.2 -pandas==2.2.0 -paramiko==3.4.0 -parso==0.8.3 -pathspec==0.12.1 -pexpect==4.8.0 -pgraph-python==0.6.2 -pillow==10.2.0 -pip==23.3.1 -pkginfo==1.9.6 -platformdirs==4.1.0 -pluggy==1.4.0 -portalocker==2.8.2 -posthog==3.3.4 -pre-commit==3.6.0 -progress==1.6 -prompt-toolkit==3.0.43 -promptflow-tools==0.1.0b15 -promptflow-vectordb==0.2.3 -promptflow==0.1.0b8 -protobuf==4.25.2 -psutil==5.9.8 -ptyprocess==0.7.0 -pulsar-client==3.4.0 -pure-eval==0.2.2 -py-trees==2.2.3 -pyarrow-hotfix==0.6 -pyarrow==13.0.0 -pyasn1-modules==0.3.0 -pyasn1==0.5.1 -pycparser==2.21 -pydantic-core==2.14.6 -pydantic==1.10.14 -pydash==6.0.2 -pydot==2.0.0 -pygments==2.15.1 -pyjwt==2.8.0 -pymongo==4.6.1 -pymupdf==1.23.22 -pymupdfb==1.23.22 -pynacl==1.5.0 -pyopenssl==23.3.0 -pyparsing==3.1.1 -pypdf==4.0.1 -pyperplan==2.1 -pypika==0.48.9 -pyproject-hooks==1.0.0 -pysocks==1.7.1 -pytamer==0.1.15 -pytest==8.0.0 -python-dateutil==2.8.2 -python-dotenv==1.0.1 -python-iso639==2024.1.2 -python-magic==0.4.27 -pytz==2023.3.post1 -pyyaml==6.0.1 -pyzmq==25.1.2 -rapidfuzz==3.6.1 -referencing==0.32.1 -regex==2023.12.25 -requests-cache==1.1.1 -requests-oauthlib==1.3.1 -requests==2.31.0 -responses==0.18.0 -roboticstoolbox-python==1.1.0 -rpds-py==0.17.1 -rsa==4.9 -rtb-data==1.0.1 -rtoml==0.9.0 -ruamel.yaml.clib==0.2.8 -ruamel.yaml==0.17.40 -safetensors==0.4.2 -scikit-learn==1.4.0 -scipy==1.12.0 -secretstorage==3.3.3 -sentence-transformers==2.4.0 -sentencepiece==0.1.99 -sentry-sdk==1.40.6 -setproctitle==1.3.3 -setuptools==68.2.2 -six==1.16.0 -smmap==5.0.1 -sniffio==1.3.0 -soupsieve==2.5 -spatialgeometry==1.1.0 -spatialmath-python==1.1.9 -sqlalchemy==2.0.27 -sqlparse==0.4.4 -stack-data==0.2.0 -starlette==0.35.1 -strictyaml==1.7.3 -swift-sim==1.1.0 -sympy==1.12 -tabulate==0.9.0 -tarski==0.8.2 -tavily-python==0.3.1 -tenacity==8.2.3 -termcolor==2.4.0 -threadpoolctl==3.2.0 -tika==2.6.0 -tiktoken==0.5.2 -tokenizers==0.15.1 -tomli==2.0.1 -torch==2.1.2 -tornado==6.4 -tqdm==4.66.1 -traitlets==5.7.1 -transformers==4.37.2 -triton==2.1.0 -typer==0.9.0 -types-requests==2.31.0.20240125 -typing-extensions==4.9.0 -typing-inspect==0.9.0 -tzdata==2023.4 -unified-planning==1.0.0 -unstructured-client==0.15.5 -unstructured==0.12.2 -up-aries==0.3.3 -up-enhsp==0.0.15 -up-fast-downward==0.2.3 -up-fmap==0.0.7 -up-lpg==0.0.7 -up-pyperplan==1.0.0 -up-symk==1.1.0 -up-tamer==1.0.0 -url-normalize==1.4.3 -urllib3==2.1.0 -uvicorn==0.27.0.post1 -uvloop==0.19.0 -virtualenv==20.25.0 -waitress==2.1.2 -wandb==0.16.3 -watchfiles==0.21.0 -wcwidth==0.2.5 -websocket-client==1.7.0 -websockets==12.0 -werkzeug==3.0.1 -wheel==0.41.2 -wikipedia==1.4.0 -wrapt==1.16.0 -xxhash==3.4.1 -yarl==1.9.4 -zipp==3.17.0 \ No newline at end of file diff --git a/wandb/run-20240303_155426-ftjob-UInKOefMZ40cQzVNlZq43njb/files/wandb-metadata.json b/wandb/run-20240303_155426-ftjob-UInKOefMZ40cQzVNlZq43njb/files/wandb-metadata.json deleted file mode 100644 index 30d3cbc5..00000000 --- a/wandb/run-20240303_155426-ftjob-UInKOefMZ40cQzVNlZq43njb/files/wandb-metadata.json +++ /dev/null @@ -1,80 +0,0 @@ -{ - "os": "Linux-5.15.0-92-generic-x86_64-with-glibc2.31", - "python": "3.10.0", - "heartbeatAt": "2024-03-03T14:54:27.331849", - "startedAt": "2024-03-03T14:54:26.673729", - "docker": null, - "cuda": null, - "args": [], - "state": "running", - "program": "/home/blackbird/kios/data/finetuning/visualize.py", - "codePathLocal": "data/finetuning/visualize.py", - "codePath": "data/finetuning/visualize.py", - "git": { - "remote": "git@gitlab.lrz.de:bblab/ma/kios.git", - "commit": "bd7f44195e793e386f8713bf2d213783c079685a" - }, - "email": "jicong.ao@tum.de", - "root": "/home/blackbird/kios", - "host": "bblab-laptop", - "username": "blackbird", - "executable": "/home/blackbird/miniconda3/envs/kios/bin/python", - "cpu_count": 4, - "cpu_count_logical": 8, - "cpu_freq": { - "current": 2421.682625, - "min": 400.0, - "max": 4200.0 - }, - "cpu_freq_per_core": [ - { - "current": 2400.0, - "min": 400.0, - "max": 4200.0 - }, - { - "current": 2913.464, - "min": 400.0, - "max": 4200.0 - }, - { - "current": 1352.593, - "min": 400.0, - "max": 4200.0 - }, - { - "current": 2400.0, - "min": 400.0, - "max": 4200.0 - }, - { - "current": 2400.0, - "min": 400.0, - "max": 4200.0 - }, - { - "current": 2400.0, - "min": 400.0, - "max": 4200.0 - }, - { - "current": 2400.0, - "min": 400.0, - "max": 4200.0 - }, - { - "current": 2400.0, - "min": 400.0, - "max": 4200.0 - } - ], - "disk": { - "/": { - "total": 467.8895797729492, - "used": 173.48290252685547 - } - }, - "memory": { - "total": 15.418136596679688 - } -} diff --git a/wandb/run-20240303_155426-ftjob-UInKOefMZ40cQzVNlZq43njb/files/wandb-summary.json b/wandb/run-20240303_155426-ftjob-UInKOefMZ40cQzVNlZq43njb/files/wandb-summary.json deleted file mode 100644 index 5edb287f..00000000 --- a/wandb/run-20240303_155426-ftjob-UInKOefMZ40cQzVNlZq43njb/files/wandb-summary.json +++ /dev/null @@ -1 +0,0 @@ -{"train_loss": 0.0, "train_accuracy": 1.0, "_timestamp": 1709477673.639342, "_runtime": 6.962782144546509, "_step": 100, "fine_tuned_model": "ft:gpt-3.5-turbo-0125:kifabrik-mirmi::8y1cXwVw", "train_data": {"_type": "table-file", "sha256": "8264d7298a2b7cbb081e336c2dac00a1e281a5e9036e648e629b929dc967597f", "size": 118530, "path": "media/table/train_data_100_8264d7298a2b7cbb081e.table.json", "ncols": 3, "nrows": 10}, "_wandb": {"runtime": 10}} \ No newline at end of file diff --git a/wandb/run-20240303_155426-ftjob-UInKOefMZ40cQzVNlZq43njb/run-ftjob-UInKOefMZ40cQzVNlZq43njb.wandb b/wandb/run-20240303_155426-ftjob-UInKOefMZ40cQzVNlZq43njb/run-ftjob-UInKOefMZ40cQzVNlZq43njb.wandb deleted file mode 100644 index f1dacf16..00000000 Binary files a/wandb/run-20240303_155426-ftjob-UInKOefMZ40cQzVNlZq43njb/run-ftjob-UInKOefMZ40cQzVNlZq43njb.wandb and /dev/null differ diff --git a/wandb/run-20240303_162941-ftjob-AncbMuC4XGCe9OOk4zjjAz5g/files/conda-environment.yaml b/wandb/run-20240303_162941-ftjob-AncbMuC4XGCe9OOk4zjjAz5g/files/conda-environment.yaml deleted file mode 100644 index 5b7d2077..00000000 --- a/wandb/run-20240303_162941-ftjob-AncbMuC4XGCe9OOk4zjjAz5g/files/conda-environment.yaml +++ /dev/null @@ -1,400 +0,0 @@ -name: kios -channels: - - defaults -dependencies: - - _libgcc_mutex=0.1=main - - _openmp_mutex=5.1=1_gnu - - asttokens=2.0.5=pyhd3eb1b0_0 - - bzip2=1.0.8=h7b6447c_0 - - ca-certificates=2023.12.12=h06a4308_0 - - decorator=5.1.1=pyhd3eb1b0_0 - - exceptiongroup=1.2.0=py310h06a4308_0 - - executing=0.8.3=pyhd3eb1b0_0 - - ipython=8.20.0=py310h06a4308_0 - - jedi=0.18.1=py310h06a4308_1 - - ld_impl_linux-64=2.38=h1181459_1 - - libffi=3.3=he6710b0_2 - - libgcc-ng=11.2.0=h1234567_1 - - libgomp=11.2.0=h1234567_1 - - libstdcxx-ng=11.2.0=h1234567_1 - - libuuid=1.41.5=h5eee18b_0 - - matplotlib-inline=0.1.6=py310h06a4308_0 - - ncurses=6.4=h6a678d5_0 - - openssl=1.1.1w=h7f8727e_0 - - parso=0.8.3=pyhd3eb1b0_0 - - pexpect=4.8.0=pyhd3eb1b0_3 - - pip=23.3.1=py310h06a4308_0 - - prompt-toolkit=3.0.43=py310h06a4308_0 - - prompt_toolkit=3.0.43=hd3eb1b0_0 - - ptyprocess=0.7.0=pyhd3eb1b0_2 - - pure_eval=0.2.2=pyhd3eb1b0_0 - - python=3.10.0=h12debd9_5 - - readline=8.2=h5eee18b_0 - - setuptools=68.2.2=py310h06a4308_0 - - six=1.16.0=pyhd3eb1b0_1 - - sqlite=3.41.2=h5eee18b_0 - - stack_data=0.2.0=pyhd3eb1b0_0 - - tk=8.6.12=h1ccaba5_0 - - traitlets=5.7.1=py310h06a4308_0 - - wcwidth=0.2.5=pyhd3eb1b0_0 - - wheel=0.41.2=py310h06a4308_0 - - xz=5.4.5=h5eee18b_0 - - zlib=1.2.13=h5eee18b_0 - - pip: - - adal==1.2.7 - - aenum==3.1.15 - - aiohttp==3.9.1 - - aiosignal==1.3.1 - - alembic==1.13.1 - - annotated-types==0.6.0 - - ansitable==0.9.10 - - anthropic==0.8.1 - - antlr4-python3-runtime==4.7.2 - - anyio==4.2.0 - - appdirs==1.4.4 - - applicationinsights==0.11.10 - - argcomplete==3.2.2 - - asgiref==3.7.2 - - async-timeout==4.0.3 - - attrs==23.2.0 - - azure-ai-generative==1.0.0b2 - - azure-ai-ml==1.12.1 - - azure-ai-resources==1.0.0b2 - - azure-common==1.1.28 - - azure-core==1.29.7 - - azure-graphrbac==0.61.1 - - azure-identity==1.15.0 - - azure-keyvault==4.2.0 - - azure-keyvault-certificates==4.7.0 - - azure-keyvault-keys==4.8.0 - - azure-keyvault-secrets==4.7.0 - - azure-mgmt-authorization==4.0.0 - - azure-mgmt-containerregistry==10.3.0 - - azure-mgmt-core==1.4.0 - - azure-mgmt-keyvault==10.3.0 - - azure-mgmt-network==25.1.0 - - azure-mgmt-resource==23.0.1 - - azure-mgmt-storage==21.1.0 - - azure-search-documents==11.4.0b11 - - azure-storage-blob==12.19.0 - - azure-storage-file-datalake==12.14.0 - - azure-storage-file-share==12.15.0 - - azureml-core==1.54.0.post1 - - azureml-dataprep==5.1.2 - - azureml-dataprep-native==41.0.0 - - azureml-dataprep-rslex==2.22.2 - - azureml-fsspec==1.3.0 - - azureml-metrics==0.0.43 - - azureml-mlflow==1.54.0.post1 - - azureml-telemetry==1.54.0 - - backoff==2.2.1 - - backports-tempfile==1.0 - - backports-weakref==1.0.post1 - - banal==1.0.6 - - bcrypt==4.1.2 - - beautifulsoup4==4.12.3 - - blinker==1.7.0 - - bs4==0.0.2 - - build==1.0.3 - - cachetools==5.3.2 - - cattrs==23.2.3 - - certifi==2023.11.17 - - cffi==1.16.0 - - cfgv==3.4.0 - - chardet==5.2.0 - - charset-normalizer==3.3.2 - - chroma-hnswlib==0.7.3 - - chromadb==0.4.22 - - click==8.1.7 - - cloudpickle==2.2.1 - - cohere==4.46 - - colorama==0.4.6 - - colored==1.4.4 - - coloredlogs==15.0.1 - - comm==0.2.1 - - contextlib2==21.6.0 - - contourpy==1.2.0 - - cryptography==41.0.7 - - cycler==0.12.1 - - databricks-cli==0.18.0 - - dataclasses-json==0.6.3 - - dataset==1.6.2 - - datasets==2.16.1 - - debugpy==1.8.1 - - deprecated==1.2.14 - - dill==0.3.7 - - dirtyjson==1.0.8 - - distlib==0.3.8 - - distro==1.9.0 - - dnspython==2.5.0 - - docker==6.1.3 - - docker-pycreds==0.4.0 - - emoji==2.10.0 - - entrypoints==0.4 - - evaluate==0.4.1 - - fairscale==0.4.13 - - faiss-cpu==1.7.4 - - fastapi==0.109.1 - - fastavro==1.9.3 - - filelock==3.13.1 - - filetype==1.2.0 - - fire==0.5.0 - - flask==2.3.3 - - flatbuffers==23.5.26 - - fonttools==4.47.2 - - frozenlist==1.4.1 - - fsspec==2023.10.0 - - gitdb==4.0.11 - - gitpython==3.1.41 - - google-api-core==2.15.0 - - google-auth==2.27.0 - - google-search-results==2.4.1 - - googleapis-common-protos==1.62.0 - - gradientai==1.7.0 - - greenlet==3.0.3 - - grpcio==1.60.0 - - grpcio-tools==1.60.0 - - h11==0.14.0 - - httpcore==1.0.2 - - httptools==0.6.1 - - httpx==0.26.0 - - huggingface-hub==0.20.3 - - humanfriendly==10.0 - - identify==2.5.33 - - idna==3.6 - - importlib-metadata==6.11.0 - - importlib-resources==6.1.1 - - iniconfig==2.0.0 - - ipykernel==6.29.2 - - isodate==0.6.1 - - itsdangerous==2.1.2 - - jaraco-classes==3.3.0 - - jeepney==0.8.0 - - jinja2==3.1.3 - - jmespath==1.0.1 - - joblib==1.3.2 - - jsonpatch==1.33 - - jsonpath-python==1.0.6 - - jsonpickle==3.0.2 - - jsonpointer==2.4 - - jsonschema==4.21.1 - - jsonschema-specifications==2023.12.1 - - jupyter-client==8.6.0 - - jupyter-core==5.7.1 - - keyring==24.3.0 - - kiwisolver==1.4.5 - - knack==0.11.0 - - kubernetes==29.0.0 - - langchain==0.1.7 - - langchain-anthropic==0.0.1.post1 - - langchain-community==0.0.20 - - langchain-core==0.1.23 - - langchain-experimental==0.0.51 - - langchain-openai==0.0.6 - - langchainhub==0.1.14 - - langdetect==1.0.9 - - langgraph==0.0.24 - - langsmith==0.0.87 - - llama-index==0.10.4 - - llama-index-agent-openai==0.1.1 - - llama-index-core==0.10.3 - - llama-index-embeddings-adapter==0.1.0 - - llama-index-embeddings-openai==0.1.1 - - llama-index-finetuning==0.1.0 - - llama-index-legacy==0.9.48 - - llama-index-llms-gradient==0.1.0 - - llama-index-llms-openai==0.1.1 - - llama-index-multi-modal-llms-openai==0.1.1 - - llama-index-postprocessor-cohere-rerank==0.1.0 - - llama-index-program-openai==0.1.1 - - llama-index-question-gen-openai==0.1.1 - - llama-index-readers-file==0.1.3 - - lxml==5.1.0 - - mako==1.3.1 - - markdown==3.5.2 - - markupsafe==2.1.4 - - marshmallow==3.20.2 - - matplotlib==3.8.3 - - mlflow-skinny==2.9.2 - - mmh3==4.1.0 - - monotonic==1.6 - - more-itertools==10.2.0 - - mpmath==1.3.0 - - msal==1.26.0 - - msal-extensions==1.0.0 - - msrest==0.7.1 - - msrestazure==0.6.4 - - multidict==6.0.4 - - multipledispatch==1.0.0 - - multiprocess==0.70.15 - - mypy-extensions==1.0.0 - - ndg-httpsclient==0.5.1 - - neo4j==5.16.0 - - nest-asyncio==1.6.0 - - networkx==3.2.1 - - nltk==3.8.1 - - nodeenv==1.8.0 - - numpy==1.26.3 - - nvidia-cublas-cu12==12.1.3.1 - - nvidia-cuda-cupti-cu12==12.1.105 - - nvidia-cuda-nvrtc-cu12==12.1.105 - - nvidia-cuda-runtime-cu12==12.1.105 - - nvidia-cudnn-cu12==8.9.2.26 - - nvidia-cufft-cu12==11.0.2.54 - - nvidia-curand-cu12==10.3.2.106 - - nvidia-cusolver-cu12==11.4.5.107 - - nvidia-cusparse-cu12==12.1.0.106 - - nvidia-nccl-cu12==2.18.1 - - nvidia-nvjitlink-cu12==12.3.101 - - nvidia-nvtx-cu12==12.1.105 - - oauthlib==3.2.2 - - onnxruntime==1.17.0 - - openai==1.13.3 - - opencensus==0.11.4 - - opencensus-context==0.1.3 - - opencensus-ext-azure==1.1.13 - - opencensus-ext-logging==0.1.1 - - opentelemetry-api==1.22.0 - - opentelemetry-exporter-otlp-proto-common==1.22.0 - - opentelemetry-exporter-otlp-proto-grpc==1.22.0 - - opentelemetry-instrumentation==0.43b0 - - opentelemetry-instrumentation-asgi==0.43b0 - - opentelemetry-instrumentation-fastapi==0.43b0 - - opentelemetry-proto==1.22.0 - - opentelemetry-sdk==1.22.0 - - opentelemetry-semantic-conventions==0.43b0 - - opentelemetry-util-http==0.43b0 - - overrides==7.7.0 - - packaging==23.2 - - pandas==2.2.0 - - paramiko==3.4.0 - - pathspec==0.12.1 - - pgraph-python==0.6.2 - - pillow==10.2.0 - - pkginfo==1.9.6 - - platformdirs==4.1.0 - - pluggy==1.4.0 - - portalocker==2.8.2 - - posthog==3.3.4 - - pre-commit==3.6.0 - - progress==1.6 - - promptflow==0.1.0b8 - - promptflow-tools==0.1.0b15 - - promptflow-vectordb==0.2.3 - - protobuf==4.25.2 - - psutil==5.9.8 - - pulsar-client==3.4.0 - - py-trees==2.2.3 - - pyarrow==13.0.0 - - pyarrow-hotfix==0.6 - - pyasn1==0.5.1 - - pyasn1-modules==0.3.0 - - pycparser==2.21 - - pydantic==1.10.14 - - pydantic-core==2.14.6 - - pydash==6.0.2 - - pydot==2.0.0 - - pygments==2.17.2 - - pyjwt==2.8.0 - - pymongo==4.6.1 - - pymupdf==1.23.22 - - pymupdfb==1.23.22 - - pynacl==1.5.0 - - pyopenssl==23.3.0 - - pyparsing==3.1.1 - - pypdf==4.0.1 - - pyperplan==2.1 - - pypika==0.48.9 - - pyproject-hooks==1.0.0 - - pysocks==1.7.1 - - pytamer==0.1.15 - - pytest==8.0.0 - - python-dateutil==2.8.2 - - python-dotenv==1.0.1 - - python-graphviz==0.20.1 - - python-iso639==2024.1.2 - - python-magic==0.4.27 - - pytz==2023.3.post1 - - pyyaml==6.0.1 - - pyzmq==25.1.2 - - rapidfuzz==3.6.1 - - referencing==0.32.1 - - regex==2023.12.25 - - requests==2.31.0 - - requests-cache==1.1.1 - - requests-oauthlib==1.3.1 - - responses==0.18.0 - - roboticstoolbox-python==1.1.0 - - rpds-py==0.17.1 - - rsa==4.9 - - rtb-data==1.0.1 - - rtoml==0.9.0 - - ruamel-yaml==0.17.40 - - ruamel-yaml-clib==0.2.8 - - safetensors==0.4.2 - - scikit-learn==1.4.0 - - scipy==1.12.0 - - secretstorage==3.3.3 - - sentence-transformers==2.4.0 - - sentencepiece==0.1.99 - - sentry-sdk==1.40.6 - - setproctitle==1.3.3 - - smmap==5.0.1 - - sniffio==1.3.0 - - soupsieve==2.5 - - spatialgeometry==1.1.0 - - spatialmath-python==1.1.9 - - sqlalchemy==2.0.27 - - sqlparse==0.4.4 - - starlette==0.35.1 - - strictyaml==1.7.3 - - swift-sim==1.1.0 - - sympy==1.12 - - tabulate==0.9.0 - - tarski==0.8.2 - - tavily-python==0.3.1 - - tenacity==8.2.3 - - termcolor==2.4.0 - - threadpoolctl==3.2.0 - - tika==2.6.0 - - tiktoken==0.5.2 - - tokenizers==0.15.1 - - tomli==2.0.1 - - torch==2.1.2 - - tornado==6.4 - - tqdm==4.66.1 - - transformers==4.37.2 - - triton==2.1.0 - - typer==0.9.0 - - types-requests==2.31.0.20240125 - - typing-extensions==4.9.0 - - typing-inspect==0.9.0 - - tzdata==2023.4 - - unified-planning==1.0.0 - - unstructured==0.12.2 - - unstructured-client==0.15.5 - - up-aries==0.3.3 - - up-enhsp==0.0.15 - - up-fast-downward==0.2.3 - - up-fmap==0.0.7 - - up-lpg==0.0.7 - - up-pyperplan==1.0.0 - - up-symk==1.1.0 - - up-tamer==1.0.0 - - url-normalize==1.4.3 - - urllib3==2.1.0 - - uvicorn==0.27.0.post1 - - uvloop==0.19.0 - - virtualenv==20.25.0 - - waitress==2.1.2 - - wandb==0.16.3 - - watchfiles==0.21.0 - - websocket-client==1.7.0 - - websockets==12.0 - - werkzeug==3.0.1 - - wikipedia==1.4.0 - - wrapt==1.16.0 - - xxhash==3.4.1 - - yarl==1.9.4 - - zipp==3.17.0 -prefix: /home/blackbird/miniconda3/envs/kios diff --git a/wandb/run-20240303_162941-ftjob-AncbMuC4XGCe9OOk4zjjAz5g/files/config.yaml b/wandb/run-20240303_162941-ftjob-AncbMuC4XGCe9OOk4zjjAz5g/files/config.yaml deleted file mode 100644 index 81ae069f..00000000 --- a/wandb/run-20240303_162941-ftjob-AncbMuC4XGCe9OOk4zjjAz5g/files/config.yaml +++ /dev/null @@ -1,78 +0,0 @@ -wandb_version: 1 - -_wandb: - desc: null - value: - python_version: 3.10.0 - cli_version: 0.16.3 - is_jupyter_run: false - is_kaggle_kernel: false - start_time: 1709479781.143355 - t: - 1: - - 55 - - 95 - 2: - - 55 - - 95 - 3: - - 13 - - 14 - - 23 - - 58 - 4: 3.10.0 - 5: 0.16.3 - 8: - - 5 - 13: linux-x86_64 -id: - desc: null - value: ftjob-AncbMuC4XGCe9OOk4zjjAz5g -created_at: - desc: null - value: '2024-03-03 16:12:46' -error: - desc: null - value: Error(code=None, message=None, param=None, error=None) -fine_tuned_model: - desc: null - value: ft:gpt-3.5-turbo-0125:kifabrik-mirmi:kios-ut-gen-v1:8yhtwL2A -finished_at: - desc: null - value: '2024-03-03 16:18:48' -model: - desc: null - value: gpt-3.5-turbo-0125 -object: - desc: null - value: fine_tuning.job -organization_id: - desc: null - value: org-ygWwisor3PBcF2dPNRDTzzgd -result_files: - desc: null - value: file-5b2b35XQYTgpFUQglt1jjtIY -status: - desc: null - value: succeeded -trained_tokens: - desc: null - value: 500793 -training_file: - desc: null - value: file-D8aBgTaXbp3g42RcV2bTiKDK -validation_file: - desc: null - value: null -user_provided_suffix: - desc: null - value: kios_ut_gen_v1 -hyperparameters: - desc: null - value: - n_epochs: 3 - batch_size: 6 - learning_rate_multiplier: 0.1 -n_train: - desc: null - value: 60 diff --git a/wandb/run-20240303_162941-ftjob-AncbMuC4XGCe9OOk4zjjAz5g/files/media/table/train_data_30_80f7369497bd53a29f90.table.json b/wandb/run-20240303_162941-ftjob-AncbMuC4XGCe9OOk4zjjAz5g/files/media/table/train_data_30_80f7369497bd53a29f90.table.json deleted file mode 100644 index 5167299e..00000000 --- a/wandb/run-20240303_162941-ftjob-AncbMuC4XGCe9OOk4zjjAz5g/files/media/table/train_data_30_80f7369497bd53a29f90.table.json +++ /dev/null @@ -1 +0,0 @@ -{"columns": ["role: system", "role: user", "role: assistant"], "data": [["You are an excellent behavior tree builder for robotic assembly tasks", "You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you construct robot task plans represented by behaiovr trees. \n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object that is ready to be parsed.\n\n Following is the domain knowledge you need in pddl domain format, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\"\"\"\n(define (domain robot_assembly_problem-domain)\n (:requirements :strips :typing)\n (:types\n thing - object\n tool part hand - thing\n )\n (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part))\n (:action pick_up\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part))\n :effect (and (hold ?tool ?part) (not (is_free ?tool))))\n (:action put_down\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (hold ?tool ?part) (hold ?hand ?tool))\n :effect (and (not (hold ?hand ?part)) (is_free ?tool)))\n (:action place\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2)))\n (:action detach\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2))))\n (:action insert\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2)))\n (:action pull\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2))))\n (:action screw\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2)))\n (:action unscrew\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2))))\n (:action load_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (is_equippable ?tool) (is_free ?hand))\n :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool)))\n (:action unload_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (hold ?hand ?tool) (is_free ?tool))\n :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool))))\n)\n\"\"\"\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_free( or ): or is free and can hold something. (default: true)\n- is_equippable(): is ready to be eqquipped by a robot hand. (default: true)\n- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false)\n- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false)\n- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false)\n- can_place_to(, ): can be placed into (placement assembly constraint) (default: false)\n- hold(, ): is holding (default: false)\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding .\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into .\n- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free.\n- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: unload_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to unload the outwardgripper in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\",\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding a outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\",\n },\n {\n \"summary\": \"Unload the outwardgripper in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outwardgripper)\",\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: insert(left_hand, defaultgripper, gear1, shaft1)\n---------------\n\n ", "{\n \"summary\": \"Selector to insert gear1 into shaft1 using defaultgripper in the left hand\",\n \"name\": \"selector: insert(left_hand, defaultgripper, gear1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"Check the target that gear1 is inserted into shaft1\",\n \"name\": \"target: is_inserted_to(gear1, shaft1)\",\n },\n {\n \"summary\": \"Sequence to insert gear1 into shaft1 using defaultgripper in the left hand\",\n \"name\": \"sequence: insert(left_hand, defaultgripper, gear1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"Check the precondition that the left hand is holding defaultgripper\",\n \"name\": \"precondition: hold(left_hand, defaultgripper)\",\n },\n {\n \"summary\": \"Check the precondition that defaultgripper is holding gear1\",\n \"name\": \"precondition: hold(defaultgripper, gear1)\",\n },\n {\n \"summary\": \"Insert gear1 into shaft1 using defaultgripper\",\n \"name\": \"action: insert(left_hand, defaultgripper, gear1, shaft1)\",\n }\n ]\n }\n ]\n}"], ["You are an excellent behavior tree builder for robotic assembly tasks", "You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you construct robot task plans represented by behaiovr trees. \n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need in pddl domain format, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\"\"\"\n(define (domain robot_assembly_problem-domain)\n (:requirements :strips :typing)\n (:types\n thing - object\n tool part hand - thing\n )\n (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part))\n (:action pick_up\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part))\n :effect (and (hold ?tool ?part) (not (is_free ?tool))))\n (:action put_down\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (hold ?tool ?part) (hold ?hand ?tool))\n :effect (and (not (hold ?hand ?part)) (is_free ?tool)))\n (:action place\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2)))\n (:action detach\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2))))\n (:action insert\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2)))\n (:action pull\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2))))\n (:action screw\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2)))\n (:action unscrew\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2))))\n (:action load_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (is_equippable ?tool) (is_free ?hand))\n :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool)))\n (:action unload_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (hold ?hand ?tool) (is_free ?tool))\n :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool))))\n)\n\"\"\"\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_free( or ): or is free and can hold something. (default: true)\n- is_equippable(): is ready to be eqquipped by a robot hand. (default: true)\n- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false)\n- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false)\n- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false)\n- can_place_to(, ): can be placed into (placement assembly constraint) (default: false)\n- hold(, ): is holding (default: false)\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding .\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into .\n- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free.\n- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: unload_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to unload the outwardgripper in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\",\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding a outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\",\n },\n {\n \"summary\": \"Unload the outwardgripper in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outwardgripper)\",\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: insert(left_hand, defaultgripper, gear1, shaft1)\n---------------\n\n ", "{\n \"summary\": \"Selector to insert gear1 into shaft1 using defaultgripper in the left hand\",\n \"name\": \"selector: insert(left_hand, defaultgripper, gear1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"Check the target that gear1 is inserted into shaft1\",\n \"name\": \"target: is_inserted_to(gear1, shaft1)\",\n },\n {\n \"summary\": \"Sequence to insert gear1 into shaft1 using defaultgripper in the left hand\",\n \"name\": \"sequence: insert(left_hand, defaultgripper, gear1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"Check the precondition that the left hand is holding defaultgripper\",\n \"name\": \"precondition: hold(left_hand, defaultgripper)\",\n },\n {\n \"summary\": \"Check the precondition that defaultgripper is holding gear1\",\n \"name\": \"precondition: hold(defaultgripper, gear1)\",\n },\n {\n \"summary\": \"Insert gear1 into shaft1 using defaultgripper\",\n \"name\": \"action: insert(left_hand, defaultgripper, gear1, shaft1)\",\n }\n ]\n }\n ]\n}"], ["You are an excellent behavior tree builder for robotic assembly tasks", "You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you construct robot task plans represented by behaiovr trees. \n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need in pddl domain format, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\"\"\"\n(define (domain robot_assembly_problem-domain)\n (:requirements :strips :typing)\n (:types\n thing - object\n tool part hand - thing\n )\n (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part))\n (:action pick_up\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part))\n :effect (and (hold ?tool ?part) (not (is_free ?tool))))\n (:action put_down\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (hold ?tool ?part) (hold ?hand ?tool))\n :effect (and (not (hold ?hand ?part)) (is_free ?tool)))\n (:action place\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2)))\n (:action detach\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2))))\n (:action insert\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2)))\n (:action pull\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2))))\n (:action screw\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2)))\n (:action unscrew\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2))))\n (:action load_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (is_equippable ?tool) (is_free ?hand))\n :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool)))\n (:action unload_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (hold ?hand ?tool) (is_free ?tool))\n :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool))))\n)\n\"\"\"\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_free( or ): or is free and can hold something. (default: true)\n- is_equippable(): is ready to be eqquipped by a robot hand. (default: true)\n- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false)\n- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false)\n- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false)\n- can_place_to(, ): can be placed into (placement assembly constraint) (default: false)\n- hold(, ): is holding (default: false)\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding .\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into .\n- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free.\n- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: unload_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to unload the outwardgripper in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\",\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding a outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\",\n },\n {\n \"summary\": \"Unload the outwardgripper in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outwardgripper)\",\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: insert(left_hand, defaultgripper, gear1, shaft1)\n---------------\n\n\n Return a JSON object.\n ", "{\n \"summary\": \"Selector to insert gear1 into shaft1 using defaultgripper in the left hand\",\n \"name\": \"selector: insert(left_hand, defaultgripper, gear1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"Check the target that gear1 is inserted into shaft1\",\n \"name\": \"target: is_inserted_to(gear1, shaft1)\",\n },\n {\n \"summary\": \"Sequence to insert gear1 into shaft1 using defaultgripper in the left hand\",\n \"name\": \"sequence: insert(left_hand, defaultgripper, gear1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"Check the precondition that the left hand is holding defaultgripper\",\n \"name\": \"precondition: hold(left_hand, defaultgripper)\",\n },\n {\n \"summary\": \"Check the precondition that defaultgripper is holding gear1\",\n \"name\": \"precondition: hold(defaultgripper, gear1)\",\n },\n {\n \"summary\": \"Insert gear1 into shaft1 using defaultgripper\",\n \"name\": \"action: insert(left_hand, defaultgripper, gear1, shaft1)\",\n }\n ]\n }\n ]\n}"], ["You are an excellent behavior tree builder for robotic assembly tasks", "You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you construct robot task plans represented by behaiovr trees. \n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need in pddl domain format, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\"\"\"\n(define (domain robot_assembly_problem-domain)\n (:requirements :strips :typing)\n (:types\n thing - object\n tool part hand - thing\n )\n (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part))\n (:action pick_up\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part))\n :effect (and (hold ?tool ?part) (not (is_free ?tool))))\n (:action put_down\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (hold ?tool ?part) (hold ?hand ?tool))\n :effect (and (not (hold ?hand ?part)) (is_free ?tool)))\n (:action place\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2)))\n (:action detach\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2))))\n (:action insert\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2)))\n (:action pull\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2))))\n (:action screw\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2)))\n (:action unscrew\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2))))\n (:action load_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (is_equippable ?tool) (is_free ?hand))\n :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool)))\n (:action unload_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (hold ?hand ?tool) (is_free ?tool))\n :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool))))\n)\n\"\"\"\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_free( or ): or is free and can hold something. (default: true)\n- is_equippable(): is ready to be eqquipped by a robot hand. (default: true)\n- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false)\n- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false)\n- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false)\n- can_place_to(, ): can be placed into (placement assembly constraint) (default: false)\n- hold(, ): is holding (default: false)\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding .\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into .\n- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free.\n- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: unload_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to unload the outwardgripper in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outwardgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\"\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding a outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"Unload the outwardgripper in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outwardgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: insert(left_hand, defaultgripper, gear1, shaft1)\n---------------\n\n\n Return a JSON object.\n ", "{\n \"summary\": \"Selector to insert gear1 into shaft1 using defaultgripper in the left hand\",\n \"name\": \"selector: insert(left_hand, defaultgripper, gear1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"Check the target that gear1 is inserted into shaft1\",\n \"name\": \"target: is_inserted_to(gear1, shaft1)\"\n },\n {\n \"summary\": \"Sequence to insert gear1 into shaft1 using defaultgripper in the left hand\",\n \"name\": \"sequence: insert(left_hand, defaultgripper, gear1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"Check the precondition that the left hand is holding defaultgripper\",\n \"name\": \"precondition: hold(left_hand, defaultgripper)\"\n },\n {\n \"summary\": \"Check the precondition that defaultgripper is holding gear1\",\n \"name\": \"precondition: hold(defaultgripper, gear1)\"\n },\n {\n \"summary\": \"Insert gear1 into shaft1 using defaultgripper\",\n \"name\": \"action: insert(left_hand, defaultgripper, gear1, shaft1)\"\n }\n ]\n }\n ]\n}"], ["You are an excellent behavior tree builder for robotic assembly tasks", "You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you construct robot task plans represented by behaiovr trees. \n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need in pddl domain format, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\"\"\"\n(define (domain robot_assembly_problem-domain)\n (:requirements :strips :typing)\n (:types\n thing - object\n tool part hand - thing\n )\n (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part))\n (:action pick_up\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part))\n :effect (and (hold ?tool ?part) (not (is_free ?tool))))\n (:action put_down\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (hold ?tool ?part) (hold ?hand ?tool))\n :effect (and (not (hold ?hand ?part)) (is_free ?tool)))\n (:action place\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2)))\n (:action detach\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2))))\n (:action insert\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2)))\n (:action pull\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2))))\n (:action screw\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2)))\n (:action unscrew\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2))))\n (:action load_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (is_equippable ?tool) (is_free ?hand))\n :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool)))\n (:action unload_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (hold ?hand ?tool) (is_free ?tool))\n :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool))))\n)\n\"\"\"\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_free( or ): or is free and can hold something. (default: true)\n- is_equippable(): is ready to be eqquipped by a robot hand. (default: true)\n- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false)\n- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false)\n- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false)\n- can_place_to(, ): can be placed into (placement assembly constraint) (default: false)\n- hold(, ): is holding (default: false)\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding .\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into .\n- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free.\n- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: unload_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to unload the outwardgripper in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outwardgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\"\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding a outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"Unload the outwardgripper in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outwardgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)\n---------------\n\n\n Return a JSON object.\n ", "{\n \"summary\": \"Selector to insert shaft1 into gearbase_hole1 using parallel_box1 in the left hand\",\n \"name\": \"selector: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)\",\n \"children\": [\n {\n \"summary\": \"Check the target that shaft1 is inserted into gearbase_hole1\",\n \"name\": \"target: is_inserted_to(shaft1, gearbase_hole1)\"\n },\n {\n \"summary\": \"Sequence to insert shaft1 into gearbase_hole1 using parallel_box1 in the left hand\",\n \"name\": \"sequence: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)\",\n \"children\": [\n {\n \"summary\": \"Check the precondition that the left hand is holding parallel_box1\",\n \"name\": \"precondition: hold(left_hand, parallel_box1)\"\n },\n {\n \"summary\": \"Check the precondition that parallel_box1 is holding shaft1\",\n \"name\": \"precondition: hold(parallel_box1, shaft1)\"\n },\n {\n \"summary\": \"Insert shaft1 into gearbase_hole1 using parallel_box1\",\n \"name\": \"action: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)\"\n }\n ]\n }\n ]\n}"], ["You are an excellent behavior tree builder for robotic assembly tasks", "You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you construct robot task plans represented by behaiovr trees. \n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need in pddl domain format, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\"\"\"\n(define (domain robot_assembly_problem-domain)\n (:requirements :strips :typing)\n (:types\n thing - object\n tool part hand - thing\n )\n (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part))\n (:action pick_up\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part))\n :effect (and (hold ?tool ?part) (not (is_free ?tool))))\n (:action put_down\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (hold ?tool ?part) (hold ?hand ?tool))\n :effect (and (not (hold ?hand ?part)) (is_free ?tool)))\n (:action place\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2)))\n (:action detach\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2))))\n (:action insert\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2)))\n (:action pull\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2))))\n (:action screw\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2)))\n (:action unscrew\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2))))\n (:action load_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (is_equippable ?tool) (is_free ?hand))\n :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool)))\n (:action unload_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (hold ?hand ?tool) (is_free ?tool))\n :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool))))\n)\n\"\"\"\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_free( or ): or is free and can hold something. (default: true)\n- is_equippable(): is ready to be eqquipped by a robot hand. (default: true)\n- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false)\n- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false)\n- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false)\n- can_place_to(, ): can be placed into (placement assembly constraint) (default: false)\n- hold(, ): is holding (default: false)\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding .\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into .\n- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free.\n- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: unload_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to unload the outwardgripper in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outwardgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\"\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding a outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"Unload the outwardgripper in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outwardgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)\n---------------\n\n\n Return a JSON object.\n ", "{\n \"summary\": \"Selector to insert shaft1 into gearbase_hole1 using parallel_box1 in the left hand\",\n \"name\": \"selector: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)\",\n \"children\": [\n {\n \"summary\": \"Check the target that shaft1 is inserted into gearbase_hole1\",\n \"name\": \"target: is_inserted_to(shaft1, gearbase_hole1)\"\n },\n {\n \"summary\": \"Sequence to insert shaft1 into gearbase_hole1 using parallel_box1 in the left hand\",\n \"name\": \"sequence: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)\",\n \"children\": [\n {\n \"summary\": \"Check the precondition that the left hand is holding parallel_box1\",\n \"name\": \"precondition: hold(left_hand, parallel_box1)\"\n },\n {\n \"summary\": \"Check the precondition that parallel_box1 is holding shaft1\",\n \"name\": \"precondition: hold(parallel_box1, shaft1)\"\n },\n {\n \"summary\": \"Insert shaft1 into gearbase_hole1 using parallel_box1\",\n \"name\": \"action: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)\"\n }\n ]\n }\n ]\n}"], ["You are an excellent behavior tree builder for robotic assembly tasks", "You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you construct robot task plans represented by behaiovr trees. \n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need in pddl domain format, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\"\"\"\n(define (domain robot_assembly_problem-domain)\n (:requirements :strips :typing)\n (:types\n thing - object\n tool part hand - thing\n )\n (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part))\n (:action pick_up\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part))\n :effect (and (hold ?tool ?part) (not (is_free ?tool))))\n (:action put_down\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (hold ?tool ?part) (hold ?hand ?tool))\n :effect (and (not (hold ?hand ?part)) (is_free ?tool)))\n (:action place\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2)))\n (:action detach\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2))))\n (:action insert\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2)))\n (:action pull\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2))))\n (:action screw\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2)))\n (:action unscrew\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2))))\n (:action load_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (is_equippable ?tool) (is_free ?hand))\n :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool)))\n (:action unload_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (hold ?hand ?tool) (is_free ?tool))\n :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool))))\n)\n\"\"\"\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_free( or ): or is free and can hold something. (default: true)\n- is_equippable(): is ready to be eqquipped by a robot hand. (default: true)\n- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false)\n- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false)\n- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false)\n- can_place_to(, ): can be placed into (placement assembly constraint) (default: false)\n- hold(, ): is holding (default: false)\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding .\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into .\n- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free.\n- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: unload_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to unload the outwardgripper in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outwardgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\"\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding a outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"Unload the outwardgripper in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outwardgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)\n---------------\n\n\n Return a JSON object.\n ", "{\n \"summary\": \"Selector to insert shaft1 into gearbase_hole1 using parallel_box1 in the left hand\",\n \"name\": \"selector: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)\",\n \"children\": [\n {\n \"summary\": \"Check the target that shaft1 is inserted into gearbase_hole1\",\n \"name\": \"target: is_inserted_to(shaft1, gearbase_hole1)\"\n },\n {\n \"summary\": \"Sequence to insert shaft1 into gearbase_hole1 using parallel_box1 in the left hand\",\n \"name\": \"sequence: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)\",\n \"children\": [\n {\n \"summary\": \"Check the precondition that the left hand is holding parallel_box1\",\n \"name\": \"precondition: hold(left_hand, parallel_box1)\"\n },\n {\n \"summary\": \"Check the precondition that parallel_box1 is holding shaft1\",\n \"name\": \"precondition: hold(parallel_box1, shaft1)\"\n },\n {\n \"summary\": \"Insert shaft1 into gearbase_hole1 using parallel_box1\",\n \"name\": \"action: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)\"\n }\n ]\n }\n ]\n}"], ["You are an excellent behavior tree builder for robotic assembly tasks", "You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you construct robot task plans represented by behaiovr trees. \n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need in pddl domain format, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\"\"\"\n(define (domain robot_assembly_problem-domain)\n (:requirements :strips :typing)\n (:types\n thing - object\n tool part hand - thing\n )\n (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part))\n (:action pick_up\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part))\n :effect (and (hold ?tool ?part) (not (is_free ?tool))))\n (:action put_down\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (hold ?tool ?part) (hold ?hand ?tool))\n :effect (and (not (hold ?hand ?part)) (is_free ?tool)))\n (:action place\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2)))\n (:action detach\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2))))\n (:action insert\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2)))\n (:action pull\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2))))\n (:action screw\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2)))\n (:action unscrew\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2))))\n (:action load_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (is_equippable ?tool) (is_free ?hand))\n :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool)))\n (:action unload_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (hold ?hand ?tool) (is_free ?tool))\n :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool))))\n)\n\"\"\"\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_free( or ): or is free and can hold something. (default: true)\n- is_equippable(): is ready to be eqquipped by a robot hand. (default: true)\n- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false)\n- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false)\n- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false)\n- can_place_to(, ): can be placed into (placement assembly constraint) (default: false)\n- hold(, ): is holding (default: false)\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding .\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into .\n- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free.\n- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: unload_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to unload the outwardgripper in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outwardgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\"\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding a outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"Unload the outwardgripper in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outwardgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: load_tool(left_hand, parallel_box1)\n---------------\n\n\n Return a JSON object.\n ", "{\n \"summary\": \"Selector to load the parallel_box1 in the left hand\",\n \"name\": \"selector: load_tool(left_hand, parallel_box1)\",\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\"\n },\n {\n \"summary\": \"Sequence to load the tool in the left hand\",\n \"name\": \"sequence: load_tool(left_hand, parallel_box1)\",\n \"children\": [\n {\n \"summary\": \"Check the precondition that the parallel_box1 is equippable\",\n \"name\": \"precondition: is_equippable(parallel_box1)\"\n },\n {\n \"summary\": \"Check the precondition that the left hand is free\",\n \"name\": \"precondition: is_free(left_hand)\"\n },\n {\n \"summary\": \"Load the parallel_box1 in the left hand\",\n \"name\": \"action: load_tool(left_hand, parallel_box1)\"\n }\n ]\n }\n ]\n}"], ["You are an excellent behavior tree builder for robotic assembly tasks", "You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you construct robot task plans represented by behaiovr trees. \n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need in pddl domain format, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\"\"\"\n(define (domain robot_assembly_problem-domain)\n (:requirements :strips :typing)\n (:types\n thing - object\n tool part hand - thing\n )\n (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part))\n (:action pick_up\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part))\n :effect (and (hold ?tool ?part) (not (is_free ?tool))))\n (:action put_down\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (hold ?tool ?part) (hold ?hand ?tool))\n :effect (and (not (hold ?hand ?part)) (is_free ?tool)))\n (:action place\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2)))\n (:action detach\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2))))\n (:action insert\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2)))\n (:action pull\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2))))\n (:action screw\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2)))\n (:action unscrew\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2))))\n (:action load_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (is_equippable ?tool) (is_free ?hand))\n :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool)))\n (:action unload_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (hold ?hand ?tool) (is_free ?tool))\n :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool))))\n)\n\"\"\"\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_free( or ): or is free and can hold something. (default: true)\n- is_equippable(): is ready to be eqquipped by a robot hand. (default: true)\n- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false)\n- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false)\n- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false)\n- can_place_to(, ): can be placed into (placement assembly constraint) (default: false)\n- hold(, ): is holding (default: false)\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding .\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into .\n- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free.\n- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: unload_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to unload the outwardgripper in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outwardgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\"\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding a outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"Unload the outwardgripper in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outwardgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: unload_tool(left_hand, outward_claw)\n---------------\n\n\n Return a JSON object.\n ", "{\n \"summary\": \"Selector to unload the outward_claw in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outward_claw)\",\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\"\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outward_claw)\",\n \"children\": [\n {\n \"summary\": \"Check the precondition that the left hand is holding a outward_claw\",\n \"name\": \"precondition: hold(left_hand, outward_claw)\"\n },\n {\n \"summary\": \"Unload the outward_claw in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outward_claw)\"\n }\n ]\n }\n ]\n}"], ["You are an excellent behavior tree builder for robotic assembly tasks", "You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you construct robot task plans represented by behaiovr trees. \n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need in pddl domain format, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\"\"\"\n(define (domain robot_assembly_problem-domain)\n (:requirements :strips :typing)\n (:types\n thing - object\n tool part hand - thing\n )\n (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part))\n (:action pick_up\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part))\n :effect (and (hold ?tool ?part) (not (is_free ?tool))))\n (:action put_down\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (hold ?tool ?part) (hold ?hand ?tool))\n :effect (and (not (hold ?hand ?part)) (is_free ?tool)))\n (:action place\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2)))\n (:action detach\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2))))\n (:action insert\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2)))\n (:action pull\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2))))\n (:action screw\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2)))\n (:action unscrew\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2))))\n (:action load_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (is_equippable ?tool) (is_free ?hand))\n :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool)))\n (:action unload_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (hold ?hand ?tool) (is_free ?tool))\n :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool))))\n)\n\"\"\"\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_free( or ): or is free and can hold something. (default: true)\n- is_equippable(): is ready to be eqquipped by a robot hand. (default: true)\n- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false)\n- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false)\n- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false)\n- can_place_to(, ): can be placed into (placement assembly constraint) (default: false)\n- hold(, ): is holding (default: false)\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding .\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into .\n- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free.\n- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: unload_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to unload the outwardgripper in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outwardgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\"\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding a outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"Unload the outwardgripper in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outwardgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: pick_up(left_hand, parallel_box1, shaft1)\n---------------\n\n\n Return a JSON object.\n ", "{\n \"summary\": \"Selector to pick up the shaft1 with the parallel_box1 in the left hand\",\n \"name\": \"selector: pick_up(left_hand, parallel_box1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"Check the target that the parallel_box1 is holding the shaft1\",\n \"name\": \"target: hold(parallel_box1, shaft1)\"\n },\n {\n \"summary\": \"Sequence to pick up the shaft1 with the parallel_box1 in the left hand\",\n \"name\": \"sequence: pick_up(left_hand, parallel_box1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"Check the precondition that the left hand is holding the parallel_box1\",\n \"name\": \"precondition: hold(left_hand, parallel_box1)\"\n },\n {\n \"summary\": \"Pick up the shaft1 with the parallel_box1 in the left hand\",\n \"name\": \"action: pick_up(left_hand, parallel_box1, shaft1)\"\n }\n ]\n }\n ]\n}"], ["You are an excellent behavior tree builder for robotic assembly tasks", "You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you construct robot task plans represented by behaiovr trees. \n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need in pddl domain format, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\"\"\"\n(define (domain robot_assembly_problem-domain)\n (:requirements :strips :typing)\n (:types\n thing - object\n tool part hand - thing\n )\n (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part))\n (:action pick_up\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part))\n :effect (and (hold ?tool ?part) (not (is_free ?tool))))\n (:action put_down\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (hold ?tool ?part) (hold ?hand ?tool))\n :effect (and (not (hold ?hand ?part)) (is_free ?tool)))\n (:action place\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2)))\n (:action detach\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2))))\n (:action insert\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2)))\n (:action pull\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2))))\n (:action screw\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2)))\n (:action unscrew\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2))))\n (:action load_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (is_equippable ?tool) (is_free ?hand))\n :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool)))\n (:action unload_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (hold ?hand ?tool) (is_free ?tool))\n :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool))))\n)\n\"\"\"\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_free( or ): or is free and can hold something. (default: true)\n- is_equippable(): is ready to be eqquipped by a robot hand. (default: true)\n- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false)\n- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false)\n- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false)\n- can_place_to(, ): can be placed into (placement assembly constraint) (default: false)\n- hold(, ): is holding (default: false)\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding .\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into .\n- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free.\n- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: unload_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to unload the outwardgripper in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outwardgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\"\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding a outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"Unload the outwardgripper in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outwardgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)\n---------------\n\n\n Return a JSON object.\n ", "{\n \"summary\": \"Selector to insert shaft1 into gearbase_hole1 using parallel_box1 in the left hand\",\n \"name\": \"selector: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)\",\n \"children\": [\n {\n \"summary\": \"Check the target that shaft1 is inserted into gearbase_hole1\",\n \"name\": \"target: is_inserted_to(shaft1, gearbase_hole1)\"\n },\n {\n \"summary\": \"Sequence to insert shaft1 into gearbase_hole1 using parallel_box1 in the left hand\",\n \"name\": \"sequence: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding parallel_box1\",\n \"name\": \"precondition: hold(left_hand, parallel_box1)\"\n },\n {\n \"summary\": \"check the precondition that parallel_box1 is holding shaft1\",\n \"name\": \"precondition: hold(parallel_box1, shaft1)\"\n },\n {\n \"summary\": \"check the precondition that shaft1 can be inserted into gearbase_hole1\",\n \"name\": \"precondition: can_insert_to(shaft1, gearbase_hole1)\"\n },\n {\n \"summary\": \"Insert shaft1 into gearbase_hole1 using parallel_box1 in the left hand\",\n \"name\": \"action: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)\"\n }\n ]\n }\n ]\n}"], ["You are an excellent behavior tree builder for robotic assembly tasks", "You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you construct robot task plans represented by behaiovr trees. \n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need in pddl domain format, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\"\"\"\n(define (domain robot_assembly_problem-domain)\n (:requirements :strips :typing)\n (:types\n thing - object\n tool part hand - thing\n )\n (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part))\n (:action pick_up\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part))\n :effect (and (hold ?tool ?part) (not (is_free ?tool))))\n (:action put_down\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (hold ?tool ?part) (hold ?hand ?tool))\n :effect (and (not (hold ?hand ?part)) (is_free ?tool)))\n (:action place\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2)))\n (:action detach\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2))))\n (:action insert\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2)))\n (:action pull\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2))))\n (:action screw\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2)))\n (:action unscrew\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2))))\n (:action load_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (is_equippable ?tool) (is_free ?hand))\n :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool)))\n (:action unload_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (hold ?hand ?tool) (is_free ?tool))\n :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool))))\n)\n\"\"\"\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_free( or ): or is free and can hold something. (default: true)\n- is_equippable(): is ready to be eqquipped by a robot hand. (default: true)\n- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false)\n- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false)\n- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false)\n- can_place_to(, ): can be placed into (placement assembly constraint) (default: false)\n- hold(, ): is holding (default: false)\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding .\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into .\n- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free.\n- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: unload_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to unload the outwardgripper in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outwardgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\"\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding a outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"Unload the outwardgripper in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outwardgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: load_tool(left_hand, parallel_box1)\n---------------\n\n\n Return a JSON object.\n ", "{\n \"summary\": \"Selector to load the parallel_box1 in the left hand\",\n \"name\": \"selector: load_tool(left_hand, parallel_box1)\",\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is holding parallel_box1\",\n \"name\": \"target: hold(left_hand, parallel_box1)\"\n },\n {\n \"summary\": \"Sequence to load the tool in the left hand\",\n \"name\": \"sequence: load_tool(left_hand, parallel_box1)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is free\",\n \"name\": \"precondition: is_free(left_hand)\"\n },\n {\n \"summary\": \"check the precondition that the parallel_box1 is equippable\",\n \"name\": \"precondition: is_equippable(parallel_box1)\"\n },\n {\n \"summary\": \"Load the parallel_box1 in the left hand\",\n \"name\": \"action: load_tool(left_hand, parallel_box1)\"\n }\n ]\n }\n ]\n}"], ["You are an excellent behavior tree builder for robotic assembly tasks", "You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you construct robot task plans represented by behaiovr trees. \n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need in pddl domain format, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\"\"\"\n(define (domain robot_assembly_problem-domain)\n (:requirements :strips :typing)\n (:types\n thing - object\n tool part hand - thing\n )\n (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part))\n (:action pick_up\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part))\n :effect (and (hold ?tool ?part) (not (is_free ?tool))))\n (:action put_down\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (hold ?tool ?part) (hold ?hand ?tool))\n :effect (and (not (hold ?hand ?part)) (is_free ?tool)))\n (:action place\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2)))\n (:action detach\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2))))\n (:action insert\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2)))\n (:action pull\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2))))\n (:action screw\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2)))\n (:action unscrew\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2))))\n (:action load_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (is_equippable ?tool) (is_free ?hand))\n :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool)))\n (:action unload_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (hold ?hand ?tool) (is_free ?tool))\n :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool))))\n)\n\"\"\"\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_free( or ): or is free and can hold something. (default: true)\n- is_equippable(): is ready to be eqquipped by a robot hand. (default: true)\n- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false)\n- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false)\n- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false)\n- can_place_to(, ): can be placed into (placement assembly constraint) (default: false)\n- hold(, ): is holding (default: false)\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding .\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into .\n- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free.\n- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: unload_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to unload the outwardgripper in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outwardgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\"\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding a outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"Unload the outwardgripper in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outwardgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: unload_tool(left_hand, outward_claw)\n---------------\n\n\n Return a JSON object.\n ", "{\n \"summary\": \"Selector to unload the outward_claw in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outward_claw)\",\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\"\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outward_claw)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding an outward_claw\",\n \"name\": \"precondition: hold(left_hand, outward_claw)\"\n },\n {\n \"summary\": \"Unload the outward_claw in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outward_claw)\"\n }\n ]\n }\n ]\n}"], ["You are an excellent behavior tree builder for robotic assembly tasks", "You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you construct robot task plans represented by behaiovr trees. \n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need in pddl domain format, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\"\"\"\n(define (domain robot_assembly_problem-domain)\n (:requirements :strips :typing)\n (:types\n thing - object\n tool part hand - thing\n )\n (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part))\n (:action pick_up\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part))\n :effect (and (hold ?tool ?part) (not (is_free ?tool))))\n (:action put_down\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (hold ?tool ?part) (hold ?hand ?tool))\n :effect (and (not (hold ?hand ?part)) (is_free ?tool)))\n (:action place\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2)))\n (:action detach\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2))))\n (:action insert\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2)))\n (:action pull\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2))))\n (:action screw\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2)))\n (:action unscrew\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2))))\n (:action load_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (is_equippable ?tool) (is_free ?hand))\n :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool)))\n (:action unload_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (hold ?hand ?tool) (is_free ?tool))\n :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool))))\n)\n\"\"\"\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_free( or ): or is free and can hold something. (default: true)\n- is_equippable(): is ready to be eqquipped by a robot hand. (default: true)\n- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false)\n- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false)\n- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false)\n- can_place_to(, ): can be placed into (placement assembly constraint) (default: false)\n- hold(, ): is holding (default: false)\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding .\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into .\n- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free.\n- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: unload_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to unload the outwardgripper in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outwardgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\"\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding a outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"Unload the outwardgripper in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outwardgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: load_tool(left_hand, parallel_box1)\n---------------\n\n\n Return a JSON object.\n ", "{\n \"summary\": \"Selector to load the parallel_box1 in the left hand\",\n \"name\": \"selector: load_tool(left_hand, parallel_box1)\",\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is holding parallel_box1\",\n \"name\": \"target: hold(left_hand, parallel_box1)\"\n },\n {\n \"summary\": \"Sequence to load the tool in the left hand\",\n \"name\": \"sequence: load_tool(left_hand, parallel_box1)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is free\",\n \"name\": \"precondition: is_free(left_hand)\"\n },\n {\n \"summary\": \"check the precondition that the parallel_box1 is equippable\",\n \"name\": \"precondition: is_equippable(parallel_box1)\"\n },\n {\n \"summary\": \"Load the parallel_box1 in the left hand\",\n \"name\": \"action: load_tool(left_hand, parallel_box1)\"\n }\n ]\n }\n ]\n}"], ["You are an excellent behavior tree builder for robotic assembly tasks", "You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you construct robot task plans represented by behaiovr trees. \n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need in pddl domain format, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\"\"\"\n(define (domain robot_assembly_problem-domain)\n (:requirements :strips :typing)\n (:types\n thing - object\n tool part hand - thing\n )\n (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part))\n (:action pick_up\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part))\n :effect (and (hold ?tool ?part) (not (is_free ?tool))))\n (:action put_down\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (hold ?tool ?part) (hold ?hand ?tool))\n :effect (and (not (hold ?hand ?part)) (is_free ?tool)))\n (:action place\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2)))\n (:action detach\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2))))\n (:action insert\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2)))\n (:action pull\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2))))\n (:action screw\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2)))\n (:action unscrew\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2))))\n (:action load_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (is_equippable ?tool) (is_free ?hand))\n :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool)))\n (:action unload_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (hold ?hand ?tool) (is_free ?tool))\n :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool))))\n)\n\"\"\"\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_free( or ): or is free and can hold something. (default: true)\n- is_equippable(): is ready to be eqquipped by a robot hand. (default: true)\n- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false)\n- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false)\n- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false)\n- can_place_to(, ): can be placed into (placement assembly constraint) (default: false)\n- hold(, ): is holding (default: false)\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding .\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into .\n- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free.\n- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: unload_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to unload the outwardgripper in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outwardgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\"\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding a outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"Unload the outwardgripper in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outwardgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: load_tool(left_hand, parallel_box1)\n---------------\n\n\n Return a JSON object.\n ", "{\n \"summary\": \"Selector to load the parallel_box1 in the left hand\",\n \"name\": \"selector: load_tool(left_hand, parallel_box1)\",\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is holding parallel_box1\",\n \"name\": \"target: hold(left_hand, parallel_box1)\"\n },\n {\n \"summary\": \"Sequence to load the tool in the left hand\",\n \"name\": \"sequence: load_tool(left_hand, parallel_box1)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is free\",\n \"name\": \"precondition: is_free(left_hand)\"\n },\n {\n \"summary\": \"check the precondition that the parallel_box1 is equippable\",\n \"name\": \"precondition: is_equippable(parallel_box1)\"\n },\n {\n \"summary\": \"Load the parallel_box1 in the left hand\",\n \"name\": \"action: load_tool(left_hand, parallel_box1)\"\n }\n ]\n }\n ]\n}"], ["You are an excellent behavior tree builder for robotic assembly tasks", "You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you construct robot task plans represented by behaiovr trees. \n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need in pddl domain format, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\"\"\"\n(define (domain robot_assembly_problem-domain)\n (:requirements :strips :typing)\n (:types\n thing - object\n tool part hand - thing\n )\n (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part))\n (:action pick_up\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part))\n :effect (and (hold ?tool ?part) (not (is_free ?tool))))\n (:action put_down\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (hold ?tool ?part) (hold ?hand ?tool))\n :effect (and (not (hold ?hand ?part)) (is_free ?tool)))\n (:action place\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2)))\n (:action detach\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2))))\n (:action insert\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2)))\n (:action pull\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2))))\n (:action screw\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2)))\n (:action unscrew\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2))))\n (:action load_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (is_equippable ?tool) (is_free ?hand))\n :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool)))\n (:action unload_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (hold ?hand ?tool) (is_free ?tool))\n :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool))))\n)\n\"\"\"\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_free( or ): or is free and can hold something. (default: true)\n- is_equippable(): is ready to be eqquipped by a robot hand. (default: true)\n- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false)\n- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false)\n- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false)\n- can_place_to(, ): can be placed into (placement assembly constraint) (default: false)\n- hold(, ): is holding (default: false)\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding .\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into .\n- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free.\n- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: unload_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to unload the outwardgripper in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outwardgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\"\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding a outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"Unload the outwardgripper in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outwardgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: load_tool(left_hand, parallel_box1)\n---------------\n\n\n Return a JSON object.\n ", "{\n \"summary\": \"Selector to load the parallel_box1 in the left hand\",\n \"name\": \"selector: load_tool(left_hand, parallel_box1)\",\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is holding parallel_box1\",\n \"name\": \"target: hold(left_hand, parallel_box1)\"\n },\n {\n \"summary\": \"Sequence to load the tool in the left hand\",\n \"name\": \"sequence: load_tool(left_hand, parallel_box1)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the parallel_box1 is equippable\",\n \"name\": \"precondition: is_equippable(parallel_box1)\"\n },\n {\n \"summary\": \"check the precondition that the left hand is free\",\n \"name\": \"precondition: is_free(left_hand)\"\n },\n {\n \"summary\": \"Load the parallel_box1 in the left hand\",\n \"name\": \"action: load_tool(left_hand, parallel_box1)\"\n }\n ]\n }\n ]\n}"], ["You are an excellent behavior tree builder for robotic assembly tasks", "You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you construct robot task plans represented by behaiovr trees. \n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need in pddl domain format, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\"\"\"\n(define (domain robot_assembly_problem-domain)\n (:requirements :strips :typing)\n (:types\n thing - object\n tool part hand - thing\n )\n (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part))\n (:action pick_up\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part))\n :effect (and (hold ?tool ?part) (not (is_free ?tool))))\n (:action put_down\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (hold ?tool ?part) (hold ?hand ?tool))\n :effect (and (not (hold ?hand ?part)) (is_free ?tool)))\n (:action place\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2)))\n (:action detach\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2))))\n (:action insert\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2)))\n (:action pull\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2))))\n (:action screw\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2)))\n (:action unscrew\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2))))\n (:action load_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (is_equippable ?tool) (is_free ?hand))\n :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool)))\n (:action unload_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (hold ?hand ?tool) (is_free ?tool))\n :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool))))\n)\n\"\"\"\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_free( or ): or is free and can hold something. (default: true)\n- is_equippable(): is ready to be eqquipped by a robot hand. (default: true)\n- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false)\n- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false)\n- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false)\n- can_place_to(, ): can be placed into (placement assembly constraint) (default: false)\n- hold(, ): is holding (default: false)\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding .\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into .\n- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free.\n- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: unload_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to unload the outwardgripper in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outwardgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\"\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding a outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"Unload the outwardgripper in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outwardgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)\n---------------\n\n\n Return a JSON object.\n ", "{\n \"summary\": \"Selector to insert shaft1 into gearbase_hole1 using parallel_box1 in the left hand\",\n \"name\": \"selector: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)\",\n \"children\": [\n {\n \"summary\": \"Check the target that shaft1 is inserted into gearbase_hole1\",\n \"name\": \"target: is_inserted_to(shaft1, gearbase_hole1)\"\n },\n {\n \"summary\": \"Sequence to insert shaft1 into gearbase_hole1 using parallel_box1 in the left hand\",\n \"name\": \"sequence: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding parallel_box1\",\n \"name\": \"precondition: hold(left_hand, parallel_box1)\"\n },\n {\n \"summary\": \"check the precondition that parallel_box1 is holding shaft1\",\n \"name\": \"precondition: hold(parallel_box1, shaft1)\"\n },\n {\n \"summary\": \"check the precondition that shaft1 can be inserted into gearbase_hole1\",\n \"name\": \"precondition: can_insert_to(shaft1, gearbase_hole1)\"\n },\n {\n \"summary\": \"Insert shaft1 into gearbase_hole1 using parallel_box1 in the left hand\",\n \"name\": \"action: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)\"\n }\n ]\n }\n ]\n}"], ["You are an excellent behavior tree builder for robotic assembly tasks", "You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you construct robot task plans represented by behaiovr trees. \n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need in pddl domain format, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\"\"\"\n(define (domain robot_assembly_problem-domain)\n (:requirements :strips :typing)\n (:types\n thing - object\n tool part hand - thing\n )\n (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part))\n (:action pick_up\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part))\n :effect (and (hold ?tool ?part) (not (is_free ?tool))))\n (:action put_down\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (hold ?tool ?part) (hold ?hand ?tool))\n :effect (and (not (hold ?hand ?part)) (is_free ?tool)))\n (:action place\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2)))\n (:action detach\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2))))\n (:action insert\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2)))\n (:action pull\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2))))\n (:action screw\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2)))\n (:action unscrew\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2))))\n (:action load_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (is_equippable ?tool) (is_free ?hand))\n :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool)))\n (:action unload_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (hold ?hand ?tool) (is_free ?tool))\n :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool))))\n)\n\"\"\"\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_free( or ): or is free and can hold something. (default: true)\n- is_equippable(): is ready to be eqquipped by a robot hand. (default: true)\n- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false)\n- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false)\n- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false)\n- can_place_to(, ): can be placed into (placement assembly constraint) (default: false)\n- hold(, ): is holding (default: false)\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding .\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into .\n- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free.\n- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: unload_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to unload the outwardgripper in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outwardgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\"\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding a outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"Unload the outwardgripper in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outwardgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: load_tool(left_hand, parallel_box1)\n---------------\n\n\n Return a JSON object.\n ", "{\n \"summary\": \"Selector to load the parallel_box1 in the left hand\",\n \"name\": \"selector: load_tool(left_hand, parallel_box1)\",\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is holding parallel_box1\",\n \"name\": \"target: hold(left_hand, parallel_box1)\"\n },\n {\n \"summary\": \"Sequence to load the tool in the left hand\",\n \"name\": \"sequence: load_tool(left_hand, parallel_box1)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is free\",\n \"name\": \"precondition: is_free(left_hand)\"\n },\n {\n \"summary\": \"check the precondition that the parallel_box1 is equippable\",\n \"name\": \"precondition: is_equippable(parallel_box1)\"\n },\n {\n \"summary\": \"Load the parallel_box1 in the left hand\",\n \"name\": \"action: load_tool(left_hand, parallel_box1)\"\n }\n ]\n }\n ]\n}"], ["You are an excellent behavior tree builder for robotic assembly tasks", "You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you construct robot task plans represented by behaiovr trees. \n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need in pddl domain format, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\"\"\"\n(define (domain robot_assembly_problem-domain)\n (:requirements :strips :typing)\n (:types\n thing - object\n tool part hand - thing\n )\n (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part))\n (:action pick_up\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part))\n :effect (and (hold ?tool ?part) (not (is_free ?tool))))\n (:action put_down\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (hold ?tool ?part) (hold ?hand ?tool))\n :effect (and (not (hold ?hand ?part)) (is_free ?tool)))\n (:action place\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2)))\n (:action detach\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2))))\n (:action insert\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2)))\n (:action pull\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2))))\n (:action screw\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2)))\n (:action unscrew\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2))))\n (:action load_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (is_equippable ?tool) (is_free ?hand))\n :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool)))\n (:action unload_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (hold ?hand ?tool) (is_free ?tool))\n :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool))))\n)\n\"\"\"\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_free( or ): or is free and can hold something. (default: true)\n- is_equippable(): is ready to be eqquipped by a robot hand. (default: true)\n- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false)\n- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false)\n- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false)\n- can_place_to(, ): can be placed into (placement assembly constraint) (default: false)\n- hold(, ): is holding (default: false)\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding .\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into .\n- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free.\n- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: unload_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to unload the outwardgripper in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outwardgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\"\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding a outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"Unload the outwardgripper in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outwardgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: unload_tool(left_hand, outward_claw)\n---------------\n\n\n Return a JSON object.\n ", "{\n \"summary\": \"Selector to unload the outward_claw in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outward_claw)\",\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\"\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outward_claw)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding an outward_claw\",\n \"name\": \"precondition: hold(left_hand, outward_claw)\"\n },\n {\n \"summary\": \"Unload the outward_claw in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outward_claw)\"\n }\n ]\n }\n ]\n}"], ["You are an excellent behavior tree builder for robotic assembly tasks", "You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you construct robot task plans represented by behaiovr trees. \n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need in pddl domain format, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\"\"\"\n(define (domain robot_assembly_problem-domain)\n (:requirements :strips :typing)\n (:types\n thing - object\n tool part hand - thing\n )\n (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part))\n (:action pick_up\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part))\n :effect (and (hold ?tool ?part) (not (is_free ?tool))))\n (:action put_down\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (hold ?tool ?part) (hold ?hand ?tool))\n :effect (and (not (hold ?hand ?part)) (is_free ?tool)))\n (:action place\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2)))\n (:action detach\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2))))\n (:action insert\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2)))\n (:action pull\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2))))\n (:action screw\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2)))\n (:action unscrew\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2))))\n (:action load_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (is_equippable ?tool) (is_free ?hand))\n :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool)))\n (:action unload_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (hold ?hand ?tool) (is_free ?tool))\n :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool))))\n)\n\"\"\"\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_free( or ): or is free and can hold something. (default: true)\n- is_equippable(): is ready to be eqquipped by a robot hand. (default: true)\n- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false)\n- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false)\n- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false)\n- can_place_to(, ): can be placed into (placement assembly constraint) (default: false)\n- hold(, ): is holding (default: false)\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding .\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into .\n- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free.\n- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: unload_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to unload the outwardgripper in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outwardgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\"\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding a outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"Unload the outwardgripper in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outwardgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: load_tool(left_hand, parallel_box1)\n---------------\n\n\n Return a JSON object.\n ", "{\n \"summary\": \"Selector to load the parallel_box1 in the left hand\",\n \"name\": \"selector: load_tool(left_hand, parallel_box1)\",\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is holding parallel_box1\",\n \"name\": \"target: hold(left_hand, parallel_box1)\"\n },\n {\n \"summary\": \"Sequence to load the tool in the left hand\",\n \"name\": \"sequence: load_tool(left_hand, parallel_box1)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is free\",\n \"name\": \"precondition: is_free(left_hand)\"\n },\n {\n \"summary\": \"check the precondition that the parallel_box1 is equippable\",\n \"name\": \"precondition: is_equippable(parallel_box1)\"\n },\n {\n \"summary\": \"Load the parallel_box1 in the left hand\",\n \"name\": \"action: load_tool(left_hand, parallel_box1)\"\n }\n ]\n }\n ]\n}"], ["You are an excellent behavior tree builder for robotic assembly tasks", "You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you construct robot task plans represented by behaiovr trees. \n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need in pddl domain format, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\"\"\"\n(define (domain robot_assembly_problem-domain)\n (:requirements :strips :typing)\n (:types\n thing - object\n tool part hand - thing\n )\n (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part))\n (:action pick_up\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part))\n :effect (and (hold ?tool ?part) (not (is_free ?tool))))\n (:action put_down\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (hold ?tool ?part) (hold ?hand ?tool))\n :effect (and (not (hold ?hand ?part)) (is_free ?tool)))\n (:action place\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2)))\n (:action detach\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2))))\n (:action insert\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2)))\n (:action pull\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2))))\n (:action screw\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2)))\n (:action unscrew\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2))))\n (:action load_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (is_equippable ?tool) (is_free ?hand))\n :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool)))\n (:action unload_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (hold ?hand ?tool) (is_free ?tool))\n :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool))))\n)\n\"\"\"\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_free( or ): or is free and can hold something. (default: true)\n- is_equippable(): is ready to be eqquipped by a robot hand. (default: true)\n- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false)\n- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false)\n- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false)\n- can_place_to(, ): can be placed into (placement assembly constraint) (default: false)\n- hold(, ): is holding (default: false)\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding .\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into .\n- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free.\n- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: unload_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to unload the outwardgripper in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outwardgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\"\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding a outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"Unload the outwardgripper in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outwardgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)\n---------------\n\n\n Return a JSON object.\n ", "{\n \"summary\": \"Selector to insert shaft1 into gearbase_hole1 using parallel_box1 in the left hand\",\n \"name\": \"selector: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)\",\n \"children\": [\n {\n \"summary\": \"Check the target that shaft1 is inserted into gearbase_hole1\",\n \"name\": \"target: is_inserted_to(shaft1, gearbase_hole1)\"\n },\n {\n \"summary\": \"Sequence to insert shaft1 into gearbase_hole1 using parallel_box1 in the left hand\",\n \"name\": \"sequence: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding parallel_box1\",\n \"name\": \"precondition: hold(left_hand, parallel_box1)\"\n },\n {\n \"summary\": \"check the precondition that parallel_box1 is holding shaft1\",\n \"name\": \"precondition: hold(parallel_box1, shaft1)\"\n },\n {\n \"summary\": \"check the precondition that shaft1 can be inserted into gearbase_hole1\",\n \"name\": \"precondition: can_insert_to(shaft1, gearbase_hole1)\"\n },\n {\n \"summary\": \"Insert shaft1 into gearbase_hole1 using parallel_box1 in the left hand\",\n \"name\": \"action: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)\"\n }\n ]\n }\n ]\n}"], ["You are an excellent behavior tree builder for robotic assembly tasks", "You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you construct robot task plans represented by behaiovr trees. \n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need in pddl domain format, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\"\"\"\n(define (domain robot_assembly_problem-domain)\n (:requirements :strips :typing)\n (:types\n thing - object\n tool part hand - thing\n )\n (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part))\n (:action pick_up\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part))\n :effect (and (hold ?tool ?part) (not (is_free ?tool))))\n (:action put_down\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (hold ?tool ?part) (hold ?hand ?tool))\n :effect (and (not (hold ?hand ?part)) (is_free ?tool)))\n (:action place\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2)))\n (:action detach\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2))))\n (:action insert\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2)))\n (:action pull\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2))))\n (:action screw\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2)))\n (:action unscrew\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2))))\n (:action load_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (is_equippable ?tool) (is_free ?hand))\n :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool)))\n (:action unload_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (hold ?hand ?tool) (is_free ?tool))\n :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool))))\n)\n\"\"\"\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_free( or ): or is free and can hold something. (default: true)\n- is_equippable(): is ready to be eqquipped by a robot hand. (default: true)\n- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false)\n- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false)\n- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false)\n- can_place_to(, ): can be placed into (placement assembly constraint) (default: false)\n- hold(, ): is holding (default: false)\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding .\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into .\n- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free.\n- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: unload_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to unload the outwardgripper in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outwardgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\"\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding a outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"Unload the outwardgripper in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outwardgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: load_tool(left_hand, parallel_box1)\n---------------\n\n\n Return a JSON object.\n ", "{\n \"summary\": \"Selector to load the parallel_box1 in the left hand\",\n \"name\": \"selector: load_tool(left_hand, parallel_box1)\",\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is holding parallel_box1\",\n \"name\": \"target: hold(left_hand, parallel_box1)\"\n },\n {\n \"summary\": \"Sequence to load the tool in the left hand\",\n \"name\": \"sequence: load_tool(left_hand, parallel_box1)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is free\",\n \"name\": \"precondition: is_free(left_hand)\"\n },\n {\n \"summary\": \"check the precondition that the parallel_box1 is equippable\",\n \"name\": \"precondition: is_equippable(parallel_box1)\"\n },\n {\n \"summary\": \"Load the parallel_box1 in the left hand\",\n \"name\": \"action: load_tool(left_hand, parallel_box1)\"\n }\n ]\n }\n ]\n}"], ["You are an excellent behavior tree builder for robotic assembly tasks", "You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you construct robot task plans represented by behaiovr trees. \n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need in pddl domain format, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\"\"\"\n(define (domain robot_assembly_problem-domain)\n (:requirements :strips :typing)\n (:types\n thing - object\n tool part hand - thing\n )\n (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part))\n (:action pick_up\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part))\n :effect (and (hold ?tool ?part) (not (is_free ?tool))))\n (:action put_down\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (hold ?tool ?part) (hold ?hand ?tool))\n :effect (and (not (hold ?hand ?part)) (is_free ?tool)))\n (:action place\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2)))\n (:action detach\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2))))\n (:action insert\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2)))\n (:action pull\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2))))\n (:action screw\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2)))\n (:action unscrew\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2))))\n (:action load_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (is_equippable ?tool) (is_free ?hand))\n :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool)))\n (:action unload_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (hold ?hand ?tool) (is_free ?tool))\n :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool))))\n)\n\"\"\"\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_free( or ): or is free and can hold something. (default: true)\n- is_equippable(): is ready to be eqquipped by a robot hand. (default: true)\n- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false)\n- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false)\n- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false)\n- can_place_to(, ): can be placed into (placement assembly constraint) (default: false)\n- hold(, ): is holding (default: false)\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding .\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into .\n- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free.\n- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: unload_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to unload the outwardgripper in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outwardgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\"\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding a outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"Unload the outwardgripper in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outwardgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: unload_tool(left_hand, outward_claw)\n---------------\n\n\n Return a JSON object.\n ", "{\n \"summary\": \"Selector to unload the outward_claw in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outward_claw)\",\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\"\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outward_claw)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding an outward_claw\",\n \"name\": \"precondition: hold(left_hand, outward_claw)\"\n },\n {\n \"summary\": \"Unload the outward_claw in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outward_claw)\"\n }\n ]\n }\n ]\n}"], ["You are an excellent behavior tree builder for robotic assembly tasks", "You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you construct robot task plans represented by behaiovr trees. \n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need in pddl domain format, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\"\"\"\n(define (domain robot_assembly_problem-domain)\n (:requirements :strips :typing)\n (:types\n thing - object\n tool part hand - thing\n )\n (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part))\n (:action pick_up\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part))\n :effect (and (hold ?tool ?part) (not (is_free ?tool))))\n (:action put_down\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (hold ?tool ?part) (hold ?hand ?tool))\n :effect (and (not (hold ?hand ?part)) (is_free ?tool)))\n (:action place\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2)))\n (:action detach\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2))))\n (:action insert\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2)))\n (:action pull\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2))))\n (:action screw\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2)))\n (:action unscrew\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2))))\n (:action load_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (is_equippable ?tool) (is_free ?hand))\n :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool)))\n (:action unload_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (hold ?hand ?tool) (is_free ?tool))\n :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool))))\n)\n\"\"\"\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_free( or ): or is free and can hold something. (default: true)\n- is_equippable(): is ready to be eqquipped by a robot hand. (default: true)\n- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false)\n- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false)\n- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false)\n- can_place_to(, ): can be placed into (placement assembly constraint) (default: false)\n- hold(, ): is holding (default: false)\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding .\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into .\n- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free.\n- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: unload_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to unload the outwardgripper in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outwardgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\"\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding a outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"Unload the outwardgripper in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outwardgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: pick_up(left_hand, parallel_box1, shaft1)\n---------------\n\n\n Return a JSON object.\n ", "{\n \"summary\": \"Selector to pick up shaft1 using parallel_box1 in the left hand\",\n \"name\": \"selector: pick_up(left_hand, parallel_box1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"Check the target that parallel_box1 is holding shaft1\",\n \"name\": \"target: hold(parallel_box1, shaft1)\"\n },\n {\n \"summary\": \"Sequence to pick up shaft1 using parallel_box1 in the left hand\",\n \"name\": \"sequence: pick_up(left_hand, parallel_box1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that parallel_box1 is free\",\n \"name\": \"precondition: is_free(parallel_box1)\"\n },\n {\n \"summary\": \"check the precondition that left hand is holding parallel_box1\",\n \"name\": \"precondition: hold(left_hand, parallel_box1)\"\n },\n {\n \"summary\": \"check the precondition that parallel_box1 can manipulate shaft1\",\n \"name\": \"precondition: can_manipulate(parallel_box1, shaft1)\"\n },\n {\n \"summary\": \"Pick up shaft1 using parallel_box1 in the left hand\",\n \"name\": \"action: pick_up(left_hand, parallel_box1, shaft1)\"\n }\n ]\n }\n ]\n}"], ["You are an excellent behavior tree builder for robotic assembly tasks", "You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you construct robot task plans represented by behaiovr trees. \n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need in pddl domain format, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\"\"\"\n(define (domain robot_assembly_problem-domain)\n (:requirements :strips :typing)\n (:types\n thing - object\n tool part hand - thing\n )\n (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part))\n (:action pick_up\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part))\n :effect (and (hold ?tool ?part) (not (is_free ?tool))))\n (:action put_down\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (hold ?tool ?part) (hold ?hand ?tool))\n :effect (and (not (hold ?hand ?part)) (is_free ?tool)))\n (:action place\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2)))\n (:action detach\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2))))\n (:action insert\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2)))\n (:action pull\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2))))\n (:action screw\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2)))\n (:action unscrew\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2))))\n (:action load_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (is_equippable ?tool) (is_free ?hand))\n :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool)))\n (:action unload_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (hold ?hand ?tool) (is_free ?tool))\n :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool))))\n)\n\"\"\"\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_free( or ): or is free and can hold something. (default: true)\n- is_equippable(): is ready to be eqquipped by a robot hand. (default: true)\n- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false)\n- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false)\n- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false)\n- can_place_to(, ): can be placed into (placement assembly constraint) (default: false)\n- hold(, ): is holding (default: false)\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding .\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into .\n- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free.\n- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: unload_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to unload the outwardgripper in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outwardgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\"\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding a outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"Unload the outwardgripper in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outwardgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: put_down(left_hand, parallel_box1, shaft1)\n---------------\n\n\n Return a JSON object.\n ", "{\n \"summary\": \"Selector to put down the shaft1 in the left hand using parallel_box1\",\n \"name\": \"selector: put_down(left_hand, parallel_box1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"Check the target that the parallel_box1 is free\",\n \"name\": \"target: is_free(parallel_box1)\"\n },\n {\n \"summary\": \"Sequence to put down the shaft1 in the left hand using parallel_box1\",\n \"name\": \"sequence: put_down(left_hand, parallel_box1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding a parallel_box1\",\n \"name\": \"precondition: hold(left_hand, parallel_box1)\"\n },\n {\n \"summary\": \"check the precondition that the parallel_box1 is holding a shaft1\",\n \"name\": \"precondition: hold(parallel_box1, shaft1)\"\n },\n {\n \"summary\": \"Put down the shaft1 in the left hand using parallel_box1\",\n \"name\": \"action: put_down(left_hand, parallel_box1, shaft1)\"\n }\n ]\n }\n ]\n}"], ["You are an excellent behavior tree builder for robotic assembly tasks", "You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you construct robot task plans represented by behaiovr trees. \n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need in pddl domain format, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\"\"\"\n(define (domain robot_assembly_problem-domain)\n (:requirements :strips :typing)\n (:types\n thing - object\n tool part hand - thing\n )\n (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part))\n (:action pick_up\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part))\n :effect (and (hold ?tool ?part) (not (is_free ?tool))))\n (:action put_down\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (hold ?tool ?part) (hold ?hand ?tool))\n :effect (and (not (hold ?hand ?part)) (is_free ?tool)))\n (:action place\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2)))\n (:action detach\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2))))\n (:action insert\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2)))\n (:action pull\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2))))\n (:action screw\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2)))\n (:action unscrew\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2))))\n (:action load_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (is_equippable ?tool) (is_free ?hand))\n :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool)))\n (:action unload_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (hold ?hand ?tool) (is_free ?tool))\n :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool))))\n)\n\"\"\"\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_free( or ): or is free and can hold something. (default: true)\n- is_equippable(): is ready to be eqquipped by a robot hand. (default: true)\n- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false)\n- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false)\n- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false)\n- can_place_to(, ): can be placed into (placement assembly constraint) (default: false)\n- hold(, ): is holding (default: false)\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding .\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into .\n- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free.\n- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: unload_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to unload the outwardgripper in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outwardgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\"\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding a outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"Unload the outwardgripper in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outwardgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)\n---------------\n\n\n Return a JSON object.\n ", "{\n \"summary\": \"Selector to insert shaft1 into gearbase_hole1 using parallel_box1 in the left hand\",\n \"name\": \"selector: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)\",\n \"children\": [\n {\n \"summary\": \"Check the target that shaft1 is inserted into gearbase_hole1\",\n \"name\": \"target: is_inserted_to(shaft1, gearbase_hole1)\"\n },\n {\n \"summary\": \"Sequence to insert shaft1 into gearbase_hole1 using parallel_box1 in the left hand\",\n \"name\": \"sequence: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding a parallel_box1\",\n \"name\": \"precondition: hold(left_hand, parallel_box1)\"\n },\n {\n \"summary\": \"check the precondition that the parallel_box1 is holding a shaft1\",\n \"name\": \"precondition: hold(parallel_box1, shaft1)\"\n },\n {\n \"summary\": \"check the precondition that shaft1 can be inserted into gearbase_hole1\",\n \"name\": \"precondition: can_insert_to(shaft1, gearbase_hole1)\"\n },\n {\n \"summary\": \"Insert shaft1 into gearbase_hole1 using parallel_box1 in the left hand\",\n \"name\": \"action: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)\"\n }\n ]\n }\n ]\n}"], ["You are an excellent behavior tree builder for robotic assembly tasks", "You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you construct robot task plans represented by behaiovr trees. \n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need in pddl domain format, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\"\"\"\n(define (domain robot_assembly_problem-domain)\n (:requirements :strips :typing)\n (:types\n thing - object\n tool part hand - thing\n )\n (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part))\n (:action pick_up\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part))\n :effect (and (hold ?tool ?part) (not (is_free ?tool))))\n (:action put_down\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (hold ?tool ?part) (hold ?hand ?tool))\n :effect (and (not (hold ?hand ?part)) (is_free ?tool)))\n (:action place\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2)))\n (:action detach\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2))))\n (:action insert\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2)))\n (:action pull\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2))))\n (:action screw\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2)))\n (:action unscrew\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2))))\n (:action load_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (is_equippable ?tool) (is_free ?hand))\n :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool)))\n (:action unload_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (hold ?hand ?tool) (is_free ?tool))\n :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool))))\n)\n\"\"\"\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_free( or ): or is free and can hold something. (default: true)\n- is_equippable(): is ready to be eqquipped by a robot hand. (default: true)\n- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false)\n- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false)\n- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false)\n- can_place_to(, ): can be placed into (placement assembly constraint) (default: false)\n- hold(, ): is holding (default: false)\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding .\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into .\n- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free.\n- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: unload_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to unload the outwardgripper in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outwardgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\"\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding a outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"Unload the outwardgripper in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outwardgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: load_tool(left_hand, parallel_box1)\n---------------\n\n\n Return a JSON object.\n ", "{\n \"summary\": \"Selector to load the parallel_box1 in the left hand\",\n \"name\": \"selector: load_tool(left_hand, parallel_box1)\",\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is holding a parallel_box1\",\n \"name\": \"target: hold(left_hand, parallel_box1)\"\n },\n {\n \"summary\": \"Sequence to load the tool in the left hand\",\n \"name\": \"sequence: load_tool(left_hand, parallel_box1)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is free\",\n \"name\": \"precondition: is_free(left_hand)\"\n },\n {\n \"summary\": \"check the precondition that the parallel_box1 is equippable\",\n \"name\": \"precondition: is_equippable(parallel_box1)\"\n },\n {\n \"summary\": \"Load the parallel_box1 in the left hand\",\n \"name\": \"action: load_tool(left_hand, parallel_box1)\"\n }\n ]\n }\n ]\n}"], ["You are an excellent behavior tree builder for robotic assembly tasks", "You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you construct robot task plans represented by behaiovr trees. \n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need in pddl domain format, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\"\"\"\n(define (domain robot_assembly_problem-domain)\n (:requirements :strips :typing)\n (:types\n thing - object\n tool part hand - thing\n )\n (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part))\n (:action pick_up\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part))\n :effect (and (hold ?tool ?part) (not (is_free ?tool))))\n (:action put_down\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (hold ?tool ?part) (hold ?hand ?tool))\n :effect (and (not (hold ?hand ?part)) (is_free ?tool)))\n (:action place\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2)))\n (:action detach\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2))))\n (:action insert\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2)))\n (:action pull\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2))))\n (:action screw\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2)))\n (:action unscrew\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2))))\n (:action load_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (is_equippable ?tool) (is_free ?hand))\n :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool)))\n (:action unload_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (hold ?hand ?tool) (is_free ?tool))\n :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool))))\n)\n\"\"\"\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_free( or ): or is free and can hold something. (default: true)\n- is_equippable(): is ready to be eqquipped by a robot hand. (default: true)\n- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false)\n- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false)\n- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false)\n- can_place_to(, ): can be placed into (placement assembly constraint) (default: false)\n- hold(, ): is holding (default: false)\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding .\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into .\n- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free.\n- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: unload_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to unload the outwardgripper in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outwardgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\"\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding a outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"Unload the outwardgripper in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outwardgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: unload_tool(left_hand, outward_claw)\n---------------\n\n\n Return a JSON object.\n ", "{\n \"summary\": \"Selector to unload the outward_claw in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outward_claw)\",\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\"\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outward_claw)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding an outward_claw\",\n \"name\": \"precondition: hold(left_hand, outward_claw)\"\n },\n {\n \"summary\": \"Unload the outward_claw in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outward_claw)\"\n }\n ]\n }\n ]\n}"], ["You are an excellent behavior tree builder for robotic assembly tasks", "You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you construct robot task plans represented by behaiovr trees. \n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need in pddl domain format, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\"\"\"\n(define (domain robot_assembly_problem-domain)\n (:requirements :strips :typing)\n (:types\n thing - object\n tool part hand - thing\n )\n (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part))\n (:action pick_up\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part))\n :effect (and (hold ?tool ?part) (not (is_free ?tool))))\n (:action put_down\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (hold ?tool ?part) (hold ?hand ?tool))\n :effect (and (not (hold ?hand ?part)) (is_free ?tool)))\n (:action place\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2)))\n (:action detach\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2))))\n (:action insert\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2)))\n (:action pull\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2))))\n (:action screw\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2)))\n (:action unscrew\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2))))\n (:action load_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (is_equippable ?tool) (is_free ?hand))\n :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool)))\n (:action unload_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (hold ?hand ?tool) (is_free ?tool))\n :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool))))\n)\n\"\"\"\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_free( or ): or is free and can hold something. (default: true)\n- is_equippable(): is ready to be eqquipped by a robot hand. (default: true)\n- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false)\n- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false)\n- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false)\n- can_place_to(, ): can be placed into (placement assembly constraint) (default: false)\n- hold(, ): is holding (default: false)\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding .\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into .\n- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free.\n- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: unload_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to unload the outwardgripper in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outwardgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\"\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding a outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"Unload the outwardgripper in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outwardgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: load_tool(left_hand, parallel_box1)\n---------------\n\n\n Return a JSON object.\n ", "{\n \"summary\": \"Selector to load the parallel_box1 in the left hand\",\n \"name\": \"selector: load_tool(left_hand, parallel_box1)\",\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is holding parallel_box1\",\n \"name\": \"target: hold(left_hand, parallel_box1)\"\n },\n {\n \"summary\": \"Sequence to load the tool in the left hand\",\n \"name\": \"sequence: load_tool(left_hand, parallel_box1)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is free\",\n \"name\": \"precondition: is_free(left_hand)\"\n },\n {\n \"summary\": \"check the precondition that the parallel_box1 is equippable\",\n \"name\": \"precondition: is_equippable(parallel_box1)\"\n },\n {\n \"summary\": \"Load the parallel_box1 in the left hand\",\n \"name\": \"action: load_tool(left_hand, parallel_box1)\"\n }\n ]\n }\n ]\n}"], ["You are an excellent behavior tree builder for robotic assembly tasks", "You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you construct robot task plans represented by behaiovr trees. \n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need in pddl domain format, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\"\"\"\n(define (domain robot_assembly_problem-domain)\n (:requirements :strips :typing)\n (:types\n thing - object\n tool part hand - thing\n )\n (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part))\n (:action pick_up\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part))\n :effect (and (hold ?tool ?part) (not (is_free ?tool))))\n (:action put_down\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (hold ?tool ?part) (hold ?hand ?tool))\n :effect (and (not (hold ?hand ?part)) (is_free ?tool)))\n (:action place\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2)))\n (:action detach\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2))))\n (:action insert\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2)))\n (:action pull\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2))))\n (:action screw\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2)))\n (:action unscrew\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2))))\n (:action load_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (is_equippable ?tool) (is_free ?hand))\n :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool)))\n (:action unload_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (hold ?hand ?tool) (is_free ?tool))\n :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool))))\n)\n\"\"\"\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_free( or ): or is free and can hold something. (default: true)\n- is_equippable(): is ready to be eqquipped by a robot hand. (default: true)\n- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false)\n- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false)\n- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false)\n- can_place_to(, ): can be placed into (placement assembly constraint) (default: false)\n- hold(, ): is holding (default: false)\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding .\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into .\n- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free.\n- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: unload_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to unload the outwardgripper in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outwardgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\"\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding a outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"Unload the outwardgripper in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outwardgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)\n---------------\n\n\n Return a JSON object.\n ", "{\n \"summary\": \"Selector to insert shaft1 into gearbase_hole1 using parallel_box1 in the left hand\",\n \"name\": \"selector: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)\",\n \"children\": [\n {\n \"summary\": \"Check the target that shaft1 is inserted into gearbase_hole1\",\n \"name\": \"target: is_inserted_to(shaft1, gearbase_hole1)\"\n },\n {\n \"summary\": \"Sequence to insert shaft1 into gearbase_hole1 using parallel_box1 in the left hand\",\n \"name\": \"sequence: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding a parallel_box1\",\n \"name\": \"precondition: hold(left_hand, parallel_box1)\"\n },\n {\n \"summary\": \"check the precondition that the parallel_box1 is holding a shaft1\",\n \"name\": \"precondition: hold(parallel_box1, shaft1)\"\n },\n {\n \"summary\": \"check the precondition that the shaft1 can be inserted into gearbase_hole1\",\n \"name\": \"precondition: can_insert_to(shaft1, gearbase_hole1)\"\n },\n {\n \"summary\": \"Insert shaft1 into gearbase_hole1 using parallel_box1 in the left hand\",\n \"name\": \"action: insert(left_hand, parallel_box1, shaft1, gearbase_hole1)\"\n }\n ]\n }\n ]\n}"], ["You are an excellent behavior tree builder for robotic assembly tasks", "You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you construct robot task plans represented by behaiovr trees. \n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need in pddl domain format, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\"\"\"\n(define (domain robot_assembly_problem-domain)\n (:requirements :strips :typing)\n (:types\n thing - object\n tool part hand - thing\n )\n (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part))\n (:action pick_up\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part))\n :effect (and (hold ?tool ?part) (not (is_free ?tool))))\n (:action put_down\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (hold ?tool ?part) (hold ?hand ?tool))\n :effect (and (not (hold ?hand ?part)) (is_free ?tool)))\n (:action place\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2)))\n (:action detach\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2))))\n (:action insert\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2)))\n (:action pull\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2))))\n (:action screw\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2)))\n (:action unscrew\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2))))\n (:action load_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (is_equippable ?tool) (is_free ?hand))\n :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool)))\n (:action unload_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (hold ?hand ?tool) (is_free ?tool))\n :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool))))\n)\n\"\"\"\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_free( or ): or is free and can hold something. (default: true)\n- is_equippable(): is ready to be eqquipped by a robot hand. (default: true)\n- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false)\n- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false)\n- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false)\n- can_place_to(, ): can be placed into (placement assembly constraint) (default: false)\n- hold(, ): is holding (default: false)\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding .\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into .\n- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free.\n- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: unload_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to unload the outwardgripper in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outwardgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\"\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding a outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"Unload the outwardgripper in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outwardgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: load_tool(left_hand, parallel_box1)\n---------------\n\n\n Return a JSON object.\n ", "{\n \"summary\": \"Selector to load the parallel_box1 in the left hand\",\n \"name\": \"selector: load_tool(left_hand, parallel_box1)\",\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is holding parallel_box1\",\n \"name\": \"target: hold(left_hand, parallel_box1)\"\n },\n {\n \"summary\": \"Sequence to load the tool in the left hand\",\n \"name\": \"sequence: load_tool(left_hand, parallel_box1)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is free\",\n \"name\": \"precondition: is_free(left_hand)\"\n },\n {\n \"summary\": \"check the precondition that the parallel_box1 is equippable\",\n \"name\": \"precondition: is_equippable(parallel_box1)\"\n },\n {\n \"summary\": \"Load the parallel_box1 in the left hand\",\n \"name\": \"action: load_tool(left_hand, parallel_box1)\"\n }\n ]\n }\n ]\n}"], ["You are an excellent behavior tree builder for robotic assembly tasks", "You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you construct robot task plans represented by behaiovr trees. \n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need in pddl domain format, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\"\"\"\n(define (domain robot_assembly_problem-domain)\n (:requirements :strips :typing)\n (:types\n thing - object\n tool part hand - thing\n )\n (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part))\n (:action pick_up\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part))\n :effect (and (hold ?tool ?part) (not (is_free ?tool))))\n (:action put_down\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (hold ?tool ?part) (hold ?hand ?tool))\n :effect (and (not (hold ?hand ?part)) (is_free ?tool)))\n (:action place\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2)))\n (:action detach\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2))))\n (:action insert\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2)))\n (:action pull\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2))))\n (:action screw\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2)))\n (:action unscrew\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2))))\n (:action load_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (is_equippable ?tool) (is_free ?hand))\n :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool)))\n (:action unload_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (hold ?hand ?tool) (is_free ?tool))\n :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool))))\n)\n\"\"\"\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_free( or ): or is free and can hold something. (default: true)\n- is_equippable(): is ready to be eqquipped by a robot hand. (default: true)\n- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false)\n- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false)\n- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false)\n- can_place_to(, ): can be placed into (placement assembly constraint) (default: false)\n- hold(, ): is holding (default: false)\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding .\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into .\n- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free.\n- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: unload_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to unload the outwardgripper in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outwardgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\"\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding a outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"Unload the outwardgripper in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outwardgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: unload_tool(left_hand, outward_claw)\n---------------\n\n\n Return a JSON object.\n ", "{\n \"summary\": \"Selector to unload the outward_claw in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outward_claw)\",\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\"\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outward_claw)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding an outward_claw\",\n \"name\": \"precondition: hold(left_hand, outward_claw)\"\n },\n {\n \"summary\": \"Unload the outward_claw in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outward_claw)\"\n }\n ]\n }\n ]\n}"], ["You are an excellent behavior tree builder for robotic assembly tasks", "You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you construct robot task plans represented by behaiovr trees. \n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need in pddl domain format, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\"\"\"\n(define (domain robot_assembly_problem-domain)\n (:requirements :strips :typing)\n (:types\n thing - object\n tool part hand - thing\n )\n (:predicates (is_free ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part))\n (:action pick_up\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (is_free ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part))\n :effect (and (hold ?tool ?part) (not (is_free ?tool))))\n (:action put_down\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (hold ?tool ?part) (hold ?hand ?tool))\n :effect (and (not (hold ?hand ?part)) (is_free ?tool)))\n (:action place\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_placed_to ?part1 ?part2)))\n (:action detach\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_placed_to ?part1 ?part2))))\n (:action insert\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_inserted_to ?part1 ?part2)))\n (:action pull\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_inserted_to ?part1 ?part2))))\n (:action screw\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_free ?tool) (is_screwed_to ?part1 ?part2)))\n (:action unscrew\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_free ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_free ?tool)) (not (is_screwed_to ?part1 ?part2))))\n (:action load_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (is_equippable ?tool) (is_free ?hand))\n :effect (and (not (is_free ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool)))\n (:action unload_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (hold ?hand ?tool) (is_free ?tool))\n :effect (and (is_free ?hand) (is_equippable ?tool) (not (hold ?hand ?tool))))\n)\n\"\"\"\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_free( or ): or is free and can hold something. (default: true)\n- is_equippable(): is ready to be eqquipped by a robot hand. (default: true)\n- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false)\n- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false)\n- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false)\n- can_place_to(, ): can be placed into (placement assembly constraint) (default: false)\n- hold(, ): is holding (default: false)\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be free to hold something.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding .\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be free to manipulate , and should be screwed into .\n- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be free.\n- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be free.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: unload_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to unload the outwardgripper in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outwardgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is free\",\n \"name\": \"target: is_free(left_hand)\"\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding a outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"Unload the outwardgripper in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outwardgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: pick_up(left_hand, parallel_box1, shaft1)\n---------------\n\n\n Return a JSON object.\n ", "{\n \"summary\": \"Selector to pick up shaft1 using parallel_box1 in the left hand\",\n \"name\": \"selector: pick_up(left_hand, parallel_box1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"Check the target that parallel_box1 is holding shaft1\",\n \"name\": \"target: hold(parallel_box1, shaft1)\"\n },\n {\n \"summary\": \"Sequence to pick up shaft1 using parallel_box1 in the left hand\",\n \"name\": \"sequence: pick_up(left_hand, parallel_box1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that parallel_box1 is free\",\n \"name\": \"precondition: is_free(parallel_box1)\"\n },\n {\n \"summary\": \"check the precondition that the left hand is holding parallel_box1\",\n \"name\": \"precondition: hold(left_hand, parallel_box1)\"\n },\n {\n \"summary\": \"check the precondition that parallel_box1 can manipulate shaft1\",\n \"name\": \"precondition: can_manipulate(parallel_box1, shaft1)\"\n },\n {\n \"summary\": \"Pick up shaft1 using parallel_box1 in the left hand\",\n \"name\": \"action: pick_up(left_hand, parallel_box1, shaft1)\"\n }\n ]\n }\n ]\n}"], ["You are an excellent behavior tree builder for robotic assembly tasks", "You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you construct robot task plans represented by behaiovr trees. \n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need in pddl domain format, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\"\"\"\n(define (domain robot_assembly_problem-domain)\n (:requirements :strips :typing)\n (:types\n thing - object\n tool part hand - thing\n )\n (:predicates (is_empty ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part))\n (:action pick_up\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (is_empty ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part))\n :effect (and (hold ?tool ?part) (not (is_empty ?tool))))\n (:action put_down\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (hold ?tool ?part) (hold ?hand ?tool))\n :effect (and (not (hold ?hand ?part)) (is_empty ?tool)))\n (:action place\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_empty ?tool) (is_placed_to ?part1 ?part2)))\n (:action detach\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_empty ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2))\n :effect (and (hold ?tool ?part1) (not (is_empty ?tool)) (not (is_placed_to ?part1 ?part2))))\n (:action insert\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_empty ?tool) (is_inserted_to ?part1 ?part2)))\n (:action pull\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_empty ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_empty ?tool)) (not (is_inserted_to ?part1 ?part2))))\n (:action screw\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_empty ?tool) (is_screwed_to ?part1 ?part2)))\n (:action unscrew\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_empty ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_empty ?tool)) (not (is_screwed_to ?part1 ?part2))))\n (:action load_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (is_equippable ?tool) (is_empty ?hand))\n :effect (and (not (is_empty ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool)))\n (:action unload_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (hold ?hand ?tool) (is_empty ?tool))\n :effect (and (is_empty ?hand) (is_equippable ?tool) (not (hold ?hand ?tool))))\n)\n\"\"\"\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_empty( or ): or is empty and can hold something. (default: true)\n- is_equippable(): is ready to be eqquipped by a robot hand. (default: true)\n- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false)\n- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false)\n- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false)\n- can_place_to(, ): can be placed into (placement assembly constraint) (default: false)\n- hold(, ): is holding (default: false)\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be empty to hold something.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding .\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be screwed into .\n- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be empty.\n- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be empty.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: unload_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to unload the outwardgripper in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outwardgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is empty\",\n \"name\": \"target: is_empty(left_hand)\"\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding a outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"Unload the outwardgripper in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outwardgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: pick_up(left_hand, parallel_box1, shaft1)\n---------------\n\n\n Return a JSON object.\n ", "{\n \"summary\": \"Selector to pick up the shaft1 with parallel_box1 in the left hand\",\n \"name\": \"selector: pick_up(left_hand, parallel_box1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"Check the target that the parallel_box1 is holding shaft1\",\n \"name\": \"target: hold(parallel_box1, shaft1)\"\n },\n {\n \"summary\": \"Sequence to pick up the shaft1 with parallel_box1 in the left hand\",\n \"name\": \"sequence: pick_up(left_hand, parallel_box1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the parallel_box1 is empty\",\n \"name\": \"precondition: is_empty(parallel_box1)\"\n },\n {\n \"summary\": \"check the precondition that the left hand is holding a parallel_box1\",\n \"name\": \"precondition: hold(left_hand, parallel_box1)\"\n },\n {\n \"summary\": \"check the precondition that the parallel_box1 can manipulate shaft1\",\n \"name\": \"precondition: can_manipulate(parallel_box1, shaft1)\"\n },\n {\n \"summary\": \"Pick up the shaft1 with parallel_box1 in the left hand\",\n \"name\": \"action: pick_up(left_hand, parallel_box1, shaft1)\"\n }\n ]\n }\n ]\n}"], ["You are an excellent behavior tree builder for robotic assembly tasks", "You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you construct robot task plans represented by behaiovr trees. \n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need in pddl domain format, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\"\"\"\n(define (domain robot_assembly_problem-domain)\n (:requirements :strips :typing)\n (:types\n thing - object\n tool part hand - thing\n )\n (:predicates (is_empty ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part))\n (:action pick_up\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (is_empty ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part))\n :effect (and (hold ?tool ?part) (not (is_empty ?tool))))\n (:action put_down\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (hold ?tool ?part) (hold ?hand ?tool))\n :effect (and (not (hold ?hand ?part)) (is_empty ?tool)))\n (:action place\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_empty ?tool) (is_placed_to ?part1 ?part2)))\n (:action detach\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_empty ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2))\n :effect (and (hold ?tool ?part1) (not (is_empty ?tool)) (not (is_placed_to ?part1 ?part2))))\n (:action insert\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_empty ?tool) (is_inserted_to ?part1 ?part2)))\n (:action pull\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_empty ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_empty ?tool)) (not (is_inserted_to ?part1 ?part2))))\n (:action screw\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_empty ?tool) (is_screwed_to ?part1 ?part2)))\n (:action unscrew\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_empty ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_empty ?tool)) (not (is_screwed_to ?part1 ?part2))))\n (:action load_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (is_equippable ?tool) (is_empty ?hand))\n :effect (and (not (is_empty ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool)))\n (:action unload_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (hold ?hand ?tool) (is_empty ?tool))\n :effect (and (is_empty ?hand) (is_equippable ?tool) (not (hold ?hand ?tool))))\n)\n\"\"\"\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_empty( or ): or is empty and can hold something. (default: true)\n- is_equippable(): is ready to be eqquipped by a robot hand. (default: true)\n- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false)\n- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false)\n- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false)\n- can_place_to(, ): can be placed into (placement assembly constraint) (default: false)\n- hold(, ): is holding (default: false)\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be empty to hold something.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding .\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be screwed into .\n- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be empty.\n- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be empty.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: unload_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to unload the outwardgripper in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outwardgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is empty\",\n \"name\": \"target: is_empty(left_hand)\"\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding a outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"Unload the outwardgripper in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outwardgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: pick_up(left_hand, parallel_box1, shaft1)\n---------------\n\n\n Return a JSON object.\n ", "{\n \"summary\": \"Selector to pick up shaft1 with parallel_box1 using left hand\",\n \"name\": \"selector: pick_up(left_hand, parallel_box1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"Check the target that shaft1 is held by parallel_box1\",\n \"name\": \"target: hold(parallel_box1, shaft1)\"\n },\n {\n \"summary\": \"Sequence to pick up shaft1 with parallel_box1 using left hand\",\n \"name\": \"sequence: pick_up(left_hand, parallel_box1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"Check the precondition that left hand is holding parallel_box1 and parallel_box1 is empty\",\n \"name\": \"precondition: hold(left_hand, parallel_box1) and is_empty(parallel_box1)\"\n },\n {\n \"summary\": \"Pick up shaft1 with parallel_box1 using left hand\",\n \"name\": \"action: pick_up(left_hand, parallel_box1, shaft1)\"\n }\n ]\n }\n ]\n}"], ["You are an excellent behavior tree builder for robotic assembly tasks", "You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you construct robot task plans represented by behaiovr trees. \n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need in pddl domain format, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\"\"\"\n(define (domain robot_assembly_problem-domain)\n (:requirements :strips :typing)\n (:types\n thing - object\n tool part hand - thing\n )\n (:predicates (is_empty ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part))\n (:action pick_up\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (is_empty ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part))\n :effect (and (hold ?tool ?part) (not (is_empty ?tool))))\n (:action put_down\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (hold ?tool ?part) (hold ?hand ?tool))\n :effect (and (not (hold ?hand ?part)) (is_empty ?tool)))\n (:action place\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_empty ?tool) (is_placed_to ?part1 ?part2)))\n (:action detach\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_empty ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2))\n :effect (and (hold ?tool ?part1) (not (is_empty ?tool)) (not (is_placed_to ?part1 ?part2))))\n (:action insert\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_empty ?tool) (is_inserted_to ?part1 ?part2)))\n (:action pull\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_empty ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_empty ?tool)) (not (is_inserted_to ?part1 ?part2))))\n (:action screw\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_empty ?tool) (is_screwed_to ?part1 ?part2)))\n (:action unscrew\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_empty ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_empty ?tool)) (not (is_screwed_to ?part1 ?part2))))\n (:action load_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (is_equippable ?tool) (is_empty ?hand))\n :effect (and (not (is_empty ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool)))\n (:action unload_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (hold ?hand ?tool) (is_empty ?tool))\n :effect (and (is_empty ?hand) (is_equippable ?tool) (not (hold ?hand ?tool))))\n)\n\"\"\"\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_empty( or ): or is empty and can hold something. (default: true)\n- is_equippable(): is ready to be eqquipped by a robot hand. (default: true)\n- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false)\n- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false)\n- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false)\n- can_place_to(, ): can be placed into (placement assembly constraint) (default: false)\n- hold(, ): is holding (default: false)\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be empty to hold something.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding .\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be screwed into .\n- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be empty.\n- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be empty.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: unload_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to unload the outwardgripper in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outwardgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is empty\",\n \"name\": \"target: is_empty(left_hand)\"\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding a outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"Unload the outwardgripper in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outwardgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: pick_up(left_hand, parallel_box1, shaft1)\n---------------\n\n\n Return a JSON object.\n ", "{\n \"summary\": \"Selector to pick up shaft1 using parallel_box1 in the left hand\",\n \"name\": \"selector: pick_up(left_hand, parallel_box1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"Check the target that parallel_box1 is holding shaft1\",\n \"name\": \"target: hold(parallel_box1, shaft1)\"\n },\n {\n \"summary\": \"Sequence to pick up shaft1 using parallel_box1 in the left hand\",\n \"name\": \"sequence: pick_up(left_hand, parallel_box1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"Check the precondition that parallel_box1 is empty\",\n \"name\": \"precondition: is_empty(parallel_box1)\"\n },\n {\n \"summary\": \"Check the precondition that left hand is holding parallel_box1\",\n \"name\": \"precondition: hold(left_hand, parallel_box1)\"\n },\n {\n \"summary\": \"Check the precondition that parallel_box1 can manipulate shaft1\",\n \"name\": \"precondition: can_manipulate(parallel_box1, shaft1)\"\n },\n {\n \"summary\": \"Pick up shaft1 using parallel_box1 in the left hand\",\n \"name\": \"action: pick_up(left_hand, parallel_box1, shaft1)\"\n }\n ]\n }\n ]\n}"], ["You are an excellent behavior tree builder for robotic assembly tasks", "You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you construct robot task plans represented by behaiovr trees. \n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need in pddl domain format, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\"\"\"\n(define (domain robot_assembly_problem-domain)\n (:requirements :strips :typing)\n (:types\n thing - object\n tool part hand - thing\n )\n (:predicates (is_empty ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part))\n (:action pick_up\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (is_empty ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part))\n :effect (and (hold ?tool ?part) (not (is_empty ?tool))))\n (:action put_down\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (hold ?tool ?part) (hold ?hand ?tool))\n :effect (and (not (hold ?hand ?part)) (is_empty ?tool)))\n (:action place\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_empty ?tool) (is_placed_to ?part1 ?part2)))\n (:action detach\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_empty ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2))\n :effect (and (hold ?tool ?part1) (not (is_empty ?tool)) (not (is_placed_to ?part1 ?part2))))\n (:action insert\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_empty ?tool) (is_inserted_to ?part1 ?part2)))\n (:action pull\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_empty ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_empty ?tool)) (not (is_inserted_to ?part1 ?part2))))\n (:action screw\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_empty ?tool) (is_screwed_to ?part1 ?part2)))\n (:action unscrew\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_empty ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_empty ?tool)) (not (is_screwed_to ?part1 ?part2))))\n (:action load_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (is_equippable ?tool) (is_empty ?hand))\n :effect (and (not (is_empty ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool)))\n (:action unload_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (hold ?hand ?tool) (is_empty ?tool))\n :effect (and (is_empty ?hand) (is_equippable ?tool) (not (hold ?hand ?tool))))\n)\n\"\"\"\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_empty( or ): or is empty and can hold something. (default: true)\n- is_equippable(): is ready to be eqquipped by a robot hand. (default: true)\n- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false)\n- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false)\n- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false)\n- can_place_to(, ): can be placed into (placement assembly constraint) (default: false)\n- hold(, ): is holding (default: false)\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be empty to hold something.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding .\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be screwed into .\n- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be empty.\n- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be empty.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: unload_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to unload the outwardgripper in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outwardgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is empty\",\n \"name\": \"target: is_empty(left_hand)\"\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding a outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"Unload the outwardgripper in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outwardgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: pick_up(left_hand, parallel_box1, shaft1)\n---------------\n\n\n Return a JSON object.\n ", "{\n \"summary\": \"Selector to pick up shaft1 using parallel_box1 in the left hand\",\n \"name\": \"selector: pick_up(left_hand, parallel_box1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"Check the target that parallel_box1 is holding shaft1\",\n \"name\": \"target: hold(parallel_box1, shaft1)\"\n },\n {\n \"summary\": \"Sequence to pick up shaft1 using parallel_box1 in the left hand\",\n \"name\": \"sequence: pick_up(left_hand, parallel_box1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"Check the precondition that parallel_box1 is empty\",\n \"name\": \"precondition: is_empty(parallel_box1)\"\n },\n {\n \"summary\": \"Check the precondition that left hand is holding parallel_box1\",\n \"name\": \"precondition: hold(left_hand, parallel_box1)\"\n },\n {\n \"summary\": \"Check the precondition that parallel_box1 can manipulate shaft1\",\n \"name\": \"precondition: can_manipulate(parallel_box1, shaft1)\"\n },\n {\n \"summary\": \"Pick up shaft1 using parallel_box1 in the left hand\",\n \"name\": \"action: pick_up(left_hand, parallel_box1, shaft1)\"\n }\n ]\n }\n ]\n}"], ["You are an excellent behavior tree builder for robotic assembly tasks", "You are an excellent task planner for robotic assembly tasks. Given the information about the world states and some instructions, you construct robot task plans represented by behaiovr trees. \n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need in pddl domain format, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\"\"\"\n(define (domain robot_assembly_problem-domain)\n (:requirements :strips :typing)\n (:types\n thing - object\n tool part hand - thing\n )\n (:predicates (is_empty ?p - thing) (is_equippable ?tool - tool) (can_manipulate ?tool - tool ?part - part) (can_insert_to ?part1 - part ?part2 - part) (can_screw_to ?part1 - part ?part2 - part) (can_place_to ?part1 - part ?part2 - part) (hold ?thing1 - thing ?thing2 - thing) (is_inserted_to ?part1 - part ?part2 - part) (is_screwed_to ?part1 - part ?part2 - part) (is_placed_to ?part1 - part ?part2 - part))\n (:action pick_up\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (is_empty ?tool) (hold ?hand ?tool) (can_manipulate ?tool ?part))\n :effect (and (hold ?tool ?part) (not (is_empty ?tool))))\n (:action put_down\n :parameters ( ?hand - hand ?tool - tool ?part - part)\n :precondition (and (hold ?tool ?part) (hold ?hand ?tool))\n :effect (and (not (hold ?hand ?part)) (is_empty ?tool)))\n (:action place\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_place_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_empty ?tool) (is_placed_to ?part1 ?part2)))\n (:action detach\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_empty ?tool) (can_manipulate ?tool ?part1) (is_placed_to ?part1 ?part2))\n :effect (and (hold ?tool ?part1) (not (is_empty ?tool)) (not (is_placed_to ?part1 ?part2))))\n (:action insert\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_insert_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_empty ?tool) (is_inserted_to ?part1 ?part2)))\n (:action pull\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_empty ?tool) (is_inserted_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_empty ?tool)) (not (is_inserted_to ?part1 ?part2))))\n (:action screw\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (hold ?tool ?part1) (can_screw_to ?part1 ?part2))\n :effect (and (not (hold ?tool ?part1)) (is_empty ?tool) (is_screwed_to ?part1 ?part2)))\n (:action unscrew\n :parameters ( ?hand - hand ?tool - tool ?part1 - part ?part2 - part)\n :precondition (and (hold ?hand ?tool) (is_empty ?tool) (is_screwed_to ?part1 ?part2) (can_manipulate ?tool ?part1))\n :effect (and (hold ?tool ?part1) (not (is_empty ?tool)) (not (is_screwed_to ?part1 ?part2))))\n (:action load_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (is_equippable ?tool) (is_empty ?hand))\n :effect (and (not (is_empty ?hand)) (not (is_equippable ?tool)) (hold ?hand ?tool)))\n (:action unload_tool\n :parameters ( ?hand - hand ?tool - tool)\n :precondition (and (hold ?hand ?tool) (is_empty ?tool))\n :effect (and (is_empty ?hand) (is_equippable ?tool) (not (hold ?hand ?tool))))\n)\n\"\"\"\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_empty( or ): or is empty and can hold something. (default: true)\n- is_equippable(): is ready to be eqquipped by a robot hand. (default: true)\n- can_manipulate(, ): can be used to manipulate (manipulation constraint) (default: false)\n- can_insert_to(, ): can be inserted into (insertion assembly constraint) (default: false)\n- can_screw_to(, ): can be screwed into (screw assembly constraint) (default: false)\n- can_place_to(, ): can be placed into (placement assembly constraint) (default: false)\n- hold(, ): is holding (default: false)\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied) (default: false)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied) (default: false)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied) (default: false)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be empty to hold something.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding .\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be screwed into .\n- load_tool(, ): equips to manipulate other objects. As preconditions, should be equippable and should be empty.\n- unload_tool(, ): unloads to manipulate other objects. As preconditions, should be holding and should be empty.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: unload_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to unload the outwardgripper in the left hand\",\n \"name\": \"selector: unload_tool(left_hand, outwardgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is empty\",\n \"name\": \"target: is_empty(left_hand)\"\n },\n {\n \"summary\": \"Sequence to unload the tool in the left hand\",\n \"name\": \"sequence: unload_tool(left_hand, outwardgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding a outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"Unload the outwardgripper in the left hand\",\n \"name\": \"action: unload_tool(left_hand, outwardgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: pick_up(left_hand, parallel_box1, shaft1)\n---------------\n\n\n Return a JSON object.\n ", "{\n \"summary\": \"Selector to pick up shaft1 using parallel_box1 in the left hand\",\n \"name\": \"selector: pick_up(left_hand, parallel_box1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"Check the target that parallel_box1 is holding shaft1\",\n \"name\": \"target: hold(parallel_box1, shaft1)\"\n },\n {\n \"summary\": \"Sequence to pick up shaft1 using parallel_box1 in the left hand\",\n \"name\": \"sequence: pick_up(left_hand, parallel_box1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that parallel_box1 is empty\",\n \"name\": \"precondition: is_empty(parallel_box1)\"\n },\n {\n \"summary\": \"check the precondition that left hand is holding parallel_box1\",\n \"name\": \"precondition: hold(left_hand, parallel_box1)\"\n },\n {\n \"summary\": \"check the precondition that parallel_box1 can manipulate shaft1\",\n \"name\": \"precondition: can_manipulate(parallel_box1, shaft1)\"\n },\n {\n \"summary\": \"Pick up shaft1 using parallel_box1 in the left hand\",\n \"name\": \"action: pick_up(left_hand, parallel_box1, shaft1)\"\n }\n ]\n }\n ]\n}"], ["You are an excellent behavior tree builder for robotic assembly tasks", "You are an excellent behavior tree builder for robotic assembly tasks. Given an action, you construct a unit subtree for the action based on its definition in the domain knowledge.\n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\nThe object types are explained as follows:\n\"\"\"\nthing: The base class for all objects in the world. Its subclasses are: hand, tool, part.\nhand: The robot hand. A empty can only hold one at a time.\ntool: The tool to manipulate . A empty can hold one at a time.\npart: The part to be manipulated.\n\"\"\"\n\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_empty(): is empty and can hold something.\n- can_manipulate(, ): can be used to manipulate (manipulation constraint). This regulates the compatibility between and .\n- can_insert_to(, ): can be inserted into (insertion assembly constraint)\n- can_screw_to(, ): can be screwed into (screw assembly constraint)\n- can_place_to(, ): can be placed into (placement assembly constraint)\n- hold(, ): is holding . It can be holding or holding .\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be empty in order to hold the . The will be held by the as a result of this action.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding . The will be empty and will not hold the anymore as a result of this action.\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be screwed into .\n- change_tool(, , ): change the equipped tool from to in order to manipulate specific parts. As preconditions, should be holding and should be empty.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: change_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to change the tool in the left hand from outwardgripper to defaultgripper\",\n \"name\": \"selector: change_tool(left_hand, outwardgripper, defaultgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is holding the default gripper\",\n \"name\": \"target: hold(left_hand, defaultgripper)\"\n },\n {\n \"summary\": \"Sequence to change the tool in the left hand from outwardgripper to defaultgripper\", \n \"name\": \"sequence: change_tool(left_hand, outwardgripper, defaultgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding the outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"check the precondition that the outwardgripper is empty\",\n \"name\": \"precondition: is_empty(outwardgripper)\"\n },\n {\n \"summary\": \"Action to change the tool in the left hand from outwardgripper to defaultgripper\",\n \"name\": \"action: change_tool(left_hand, outwardgripper, defaultgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: pick_up(left_hand, parallel_box1, shaft1)\n---------------\n\n\n Return a JSON object.\n ", "{\n \"summary\": \"Selector to pick up shaft1 using parallel_box1 in the left hand\",\n \"name\": \"selector: pick_up(left_hand, parallel_box1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"Check the target that parallel_box1 is holding shaft1\",\n \"name\": \"target: hold(parallel_box1, shaft1)\"\n },\n {\n \"summary\": \"Sequence to pick up shaft1 using parallel_box1 in the left hand\", \n \"name\": \"sequence: pick_up(left_hand, parallel_box1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding parallel_box1\",\n \"name\": \"precondition: hold(left_hand, parallel_box1)\"\n },\n {\n \"summary\": \"check the precondition that parallel_box1 is empty\",\n \"name\": \"precondition: is_empty(parallel_box1)\"\n },\n {\n \"summary\": \"Action to pick up shaft1 using parallel_box1 in the left hand\",\n \"name\": \"action: pick_up(left_hand, parallel_box1, shaft1)\"\n }\n ]\n }\n ]\n}"], ["You are an excellent behavior tree builder for robotic assembly tasks", "You are an excellent behavior tree builder for robotic assembly tasks. Given an action, you construct a unit subtree for the action based on its definition in the domain knowledge.\n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\nThe object types are explained as follows:\n\"\"\"\nthing: The base class for all objects in the world. Its subclasses are: hand, tool, part.\nhand: The robot hand. A empty can only hold one at a time.\ntool: The tool to manipulate . A empty can hold one at a time.\npart: The part to be manipulated.\n\"\"\"\n\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_empty(): is empty and can hold something.\n- can_manipulate(, ): can be used to manipulate (manipulation constraint). This regulates the compatibility between and .\n- can_insert_to(, ): can be inserted into (insertion assembly constraint)\n- can_screw_to(, ): can be screwed into (screw assembly constraint)\n- can_place_to(, ): can be placed into (placement assembly constraint)\n- hold(, ): is holding . It can be holding or holding .\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be empty in order to hold the . The will be held by the as a result of this action.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding . The will be empty and will not hold the anymore as a result of this action.\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be screwed into .\n- change_tool(, , ): change the equipped tool from to in order to manipulate specific parts. As preconditions, should be holding and should be empty.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: change_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to change the tool in the left hand from outwardgripper to defaultgripper\",\n \"name\": \"selector: change_tool(left_hand, outwardgripper, defaultgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is holding the default gripper\",\n \"name\": \"target: hold(left_hand, defaultgripper)\"\n },\n {\n \"summary\": \"Sequence to change the tool in the left hand from outwardgripper to defaultgripper\", \n \"name\": \"sequence: change_tool(left_hand, outwardgripper, defaultgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding the outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"check the precondition that the outwardgripper is empty\",\n \"name\": \"precondition: is_empty(outwardgripper)\"\n },\n {\n \"summary\": \"Action to change the tool in the left hand from outwardgripper to defaultgripper\",\n \"name\": \"action: change_tool(left_hand, outwardgripper, defaultgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: insert(left_hand, outward_claw, gear2, shaft2)\n---------------\n\n\n Return a JSON object.\n ", "{\n \"summary\": \"Selector to insert gear2 into shaft2 using the outward claw in the left hand\",\n \"name\": \"selector: insert(left_hand, outward_claw, gear2, shaft2)\",\n \"children\": [\n {\n \"summary\": \"Check the target that gear2 is inserted into shaft2\",\n \"name\": \"target: is_inserted_to(gear2, shaft2)\"\n },\n {\n \"summary\": \"Sequence to insert gear2 into shaft2 using the outward claw in the left hand\", \n \"name\": \"sequence: insert(left_hand, outward_claw, gear2, shaft2)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding the outward claw\",\n \"name\": \"precondition: hold(left_hand, outward_claw)\"\n },\n {\n \"summary\": \"check the precondition that the outward claw is holding gear2\",\n \"name\": \"precondition: hold(outward_claw, gear2)\"\n },\n {\n \"summary\": \"Action to insert gear2 into shaft2 using the outward claw in the left hand\",\n \"name\": \"action: insert(left_hand, outward_claw, gear2, shaft2)\"\n }\n ]\n }\n ]\n}"], ["You are an excellent behavior tree builder for robotic assembly tasks", "You are an excellent behavior tree builder for robotic assembly tasks. Given an action, you construct a unit subtree for the action based on its definition in the domain knowledge.\n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\nThe object types are explained as follows:\n\"\"\"\nthing: The base class for all objects in the world. Its subclasses are: hand, tool, part.\nhand: The robot hand. A empty can only hold one at a time.\ntool: The tool to manipulate . A empty can hold one at a time.\npart: The part to be manipulated.\n\"\"\"\n\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_empty(): is empty and can hold something.\n- can_manipulate(, ): can be used to manipulate (manipulation constraint). This regulates the compatibility between and .\n- can_insert_to(, ): can be inserted into (insertion assembly constraint)\n- can_screw_to(, ): can be screwed into (screw assembly constraint)\n- can_place_to(, ): can be placed into (placement assembly constraint)\n- hold(, ): is holding . It can be holding or holding .\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be empty in order to hold the . The will be held by the as a result of this action.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding . The will be empty and will not hold the anymore as a result of this action.\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be screwed into .\n- change_tool(, , ): change the equipped tool from to in order to manipulate specific parts. As preconditions, should be holding and should be empty.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: change_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to change the tool in the left hand from outwardgripper to defaultgripper\",\n \"name\": \"selector: change_tool(left_hand, outwardgripper, defaultgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is holding the default gripper\",\n \"name\": \"target: hold(left_hand, defaultgripper)\"\n },\n {\n \"summary\": \"Sequence to change the tool in the left hand from outwardgripper to defaultgripper\", \n \"name\": \"sequence: change_tool(left_hand, outwardgripper, defaultgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding the outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"check the precondition that the outwardgripper is empty\",\n \"name\": \"precondition: is_empty(outwardgripper)\"\n },\n {\n \"summary\": \"Action to change the tool in the left hand from outwardgripper to defaultgripper\",\n \"name\": \"action: change_tool(left_hand, outwardgripper, defaultgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: change_tool(left_hand, parallel_box1, outward_claw)\n---------------\n\n\n Return a JSON object.\n ", "{\n \"summary\": \"Selector to change the tool in the left hand from parallel_box1 to outward_claw\",\n \"name\": \"selector: change_tool(left_hand, parallel_box1, outward_claw)\",\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is holding the outward_claw\",\n \"name\": \"target: hold(left_hand, outward_claw)\"\n },\n {\n \"summary\": \"Sequence to change the tool in the left hand from parallel_box1 to outward_claw\", \n \"name\": \"sequence: change_tool(left_hand, parallel_box1, outward_claw)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding the parallel_box1\",\n \"name\": \"precondition: hold(left_hand, parallel_box1)\"\n },\n {\n \"summary\": \"check the precondition that the parallel_box1 is empty\",\n \"name\": \"precondition: is_empty(parallel_box1)\"\n },\n {\n \"summary\": \"Action to change the tool in the left hand from parallel_box1 to outward_claw\",\n \"name\": \"action: change_tool(left_hand, parallel_box1, outward_claw)\"\n }\n ]\n }\n ]\n}"], ["You are an excellent behavior tree builder for robotic assembly tasks", "You are an excellent behavior tree builder for robotic assembly tasks. Given an action, you construct a unit subtree for the action based on its definition in the domain knowledge.\n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\nThe object types are explained as follows:\n\"\"\"\nthing: The base class for all objects in the world. Its subclasses are: hand, tool, part.\nhand: The robot hand. A empty can only hold one at a time.\ntool: The tool to manipulate . A empty can hold one at a time.\npart: The part to be manipulated.\n\"\"\"\n\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_empty(): is empty and can hold something.\n- can_manipulate(, ): can be used to manipulate (manipulation constraint). This regulates the compatibility between and .\n- can_insert_to(, ): can be inserted into (insertion assembly constraint)\n- can_screw_to(, ): can be screwed into (screw assembly constraint)\n- can_place_to(, ): can be placed into (placement assembly constraint)\n- hold(, ): is holding . It can be holding or holding .\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be empty in order to hold the . The will be held by the as a result of this action.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding . The will be empty and will not hold the anymore as a result of this action.\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be screwed into .\n- change_tool(, , ): change the equipped tool from to in order to manipulate specific parts. As preconditions, should be holding and should be empty.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: change_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to change the tool in the left hand from outwardgripper to defaultgripper\",\n \"name\": \"selector: change_tool(left_hand, outwardgripper, defaultgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is holding the default gripper\",\n \"name\": \"target: hold(left_hand, defaultgripper)\"\n },\n {\n \"summary\": \"Sequence to change the tool in the left hand from outwardgripper to defaultgripper\", \n \"name\": \"sequence: change_tool(left_hand, outwardgripper, defaultgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding the outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"check the precondition that the outwardgripper is empty\",\n \"name\": \"precondition: is_empty(outwardgripper)\"\n },\n {\n \"summary\": \"Action to change the tool in the left hand from outwardgripper to defaultgripper\",\n \"name\": \"action: change_tool(left_hand, outwardgripper, defaultgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: pick_up(left_hand, outward_claw, gear2)\n---------------\n\n\n Return a JSON object.\n ", "{\n \"summary\": \"Selector to pick up gear2 with outward_claw using left_hand\",\n \"name\": \"selector: pick_up(left_hand, outward_claw, gear2)\",\n \"children\": [\n {\n \"summary\": \"Check the target that the outward_claw is holding gear2\",\n \"name\": \"target: hold(outward_claw, gear2)\"\n },\n {\n \"summary\": \"Sequence to pick up gear2 with outward_claw using left_hand\", \n \"name\": \"sequence: pick_up(left_hand, outward_claw, gear2)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left_hand is holding the outward_claw\",\n \"name\": \"precondition: hold(left_hand, outward_claw)\"\n },\n {\n \"summary\": \"check the precondition that the outward_claw is empty\",\n \"name\": \"precondition: is_empty(outward_claw)\"\n },\n {\n \"summary\": \"Action to pick up gear2 with outward_claw using left_hand\",\n \"name\": \"action: pick_up(left_hand, outward_claw, gear2)\"\n }\n ]\n }\n ]\n}"], ["You are an excellent behavior tree builder for robotic assembly tasks", "You are an excellent behavior tree builder for robotic assembly tasks. Given an action, you construct a unit subtree for the action based on its definition in the domain knowledge.\n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\nThe object types are explained as follows:\n\"\"\"\nthing: The base class for all objects in the world. Its subclasses are: hand, tool, part.\nhand: The robot hand. A empty can only hold one at a time.\ntool: The tool to manipulate . A empty can hold one at a time.\npart: The part to be manipulated.\n\"\"\"\n\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_empty(): is empty and can hold something.\n- can_manipulate(, ): can be used to manipulate (manipulation constraint). This regulates the compatibility between and .\n- can_insert_to(, ): can be inserted into (insertion assembly constraint)\n- can_screw_to(, ): can be screwed into (screw assembly constraint)\n- can_place_to(, ): can be placed into (placement assembly constraint)\n- hold(, ): is holding . It can be holding or holding .\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be empty in order to hold the . The will be held by the as a result of this action.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding . The will be empty and will not hold the anymore as a result of this action.\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be screwed into .\n- change_tool(, , ): change the equipped tool from to in order to manipulate specific parts. As preconditions, should be holding and should be empty.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: change_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to change the tool in the left hand from outwardgripper to defaultgripper\",\n \"name\": \"selector: change_tool(left_hand, outwardgripper, defaultgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is holding the default gripper\",\n \"name\": \"target: hold(left_hand, defaultgripper)\"\n },\n {\n \"summary\": \"Sequence to change the tool in the left hand from outwardgripper to defaultgripper\", \n \"name\": \"sequence: change_tool(left_hand, outwardgripper, defaultgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding the outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"check the precondition that the outwardgripper is empty\",\n \"name\": \"precondition: is_empty(outwardgripper)\"\n },\n {\n \"summary\": \"Action to change the tool in the left hand from outwardgripper to defaultgripper\",\n \"name\": \"action: change_tool(left_hand, outwardgripper, defaultgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: insert(left_hand, parallel_box1, gear2, shaft2)\n---------------\n\n\n Return a JSON object.\n ", "{\n \"summary\": \"Selector to insert gear2 into shaft2 using parallel_box1 with left hand\",\n \"name\": \"selector: insert(left_hand, parallel_box1, gear2, shaft2)\",\n \"children\": [\n {\n \"summary\": \"Check the target that gear2 is inserted into shaft2\",\n \"name\": \"target: is_inserted_to(gear2, shaft2)\"\n },\n {\n \"summary\": \"Sequence to insert gear2 into shaft2 using parallel_box1 with left hand\",\n \"name\": \"sequence: insert(left_hand, parallel_box1, gear2, shaft2)\",\n \"children\": [\n {\n \"summary\": \"Check the precondition that left hand is holding parallel_box1\",\n \"name\": \"precondition: hold(left_hand, parallel_box1)\"\n },\n {\n \"summary\": \"Check the precondition that parallel_box1 is holding gear2\",\n \"name\": \"precondition: hold(parallel_box1, gear2)\"\n },\n {\n \"summary\": \"Action to insert gear2 into shaft2 using parallel_box1 with left hand\",\n \"name\": \"action: insert(left_hand, parallel_box1, gear2, shaft2)\"\n }\n ]\n }\n ]\n}"], ["You are an excellent behavior tree builder for robotic assembly tasks", "You are an excellent behavior tree builder for robotic assembly tasks. Given an action, you construct a unit subtree for the action based on its definition in the domain knowledge.\n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\nThe object types are explained as follows:\n\"\"\"\nthing: The base class for all objects in the world. Its subclasses are: hand, tool, part.\nhand: The robot hand. A empty can only hold one at a time.\ntool: The tool to manipulate . A empty can hold one at a time.\npart: The part to be manipulated.\n\"\"\"\n\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_empty(): is empty and can hold something.\n- can_manipulate(, ): can be used to manipulate (manipulation constraint). This regulates the compatibility between and .\n- can_insert_to(, ): can be inserted into (insertion assembly constraint)\n- can_screw_to(, ): can be screwed into (screw assembly constraint)\n- can_place_to(, ): can be placed into (placement assembly constraint)\n- hold(, ): is holding . It can be holding or holding .\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be empty in order to hold the . The will be held by the as a result of this action.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding . The will be empty and will not hold the anymore as a result of this action.\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be screwed into .\n- change_tool(, , ): change the equipped tool from to in order to manipulate specific parts. As preconditions, should be holding and should be empty.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: change_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to change the tool in the left hand from outwardgripper to defaultgripper\",\n \"name\": \"selector: change_tool(left_hand, outwardgripper, defaultgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is holding the default gripper\",\n \"name\": \"target: hold(left_hand, defaultgripper)\"\n },\n {\n \"summary\": \"Sequence to change the tool in the left hand from outwardgripper to defaultgripper\", \n \"name\": \"sequence: change_tool(left_hand, outwardgripper, defaultgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding the outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"check the precondition that the outwardgripper is empty\",\n \"name\": \"precondition: is_empty(outwardgripper)\"\n },\n {\n \"summary\": \"Action to change the tool in the left hand from outwardgripper to defaultgripper\",\n \"name\": \"action: change_tool(left_hand, outwardgripper, defaultgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: insert(left_hand, outward_claw, gear2, shaft2)\n---------------\n\n\n Return a JSON object.\n ", "{\n \"summary\": \"Selector to insert gear2 into shaft2 using outward_claw in the left hand\",\n \"name\": \"selector: insert(left_hand, outward_claw, gear2, shaft2)\",\n \"children\": [\n {\n \"summary\": \"Check the target that gear2 is inserted into shaft2\",\n \"name\": \"target: is_inserted_to(gear2, shaft2)\"\n },\n {\n \"summary\": \"Sequence to insert gear2 into shaft2 using outward_claw in the left hand\",\n \"name\": \"sequence: insert(left_hand, outward_claw, gear2, shaft2)\",\n \"children\": [\n {\n \"summary\": \"Check the precondition that the left hand is holding outward_claw\",\n \"name\": \"precondition: hold(left_hand, outward_claw)\"\n },\n {\n \"summary\": \"Check the precondition that outward_claw is holding gear2\",\n \"name\": \"precondition: hold(outward_claw, gear2)\"\n },\n {\n \"summary\": \"Action to insert gear2 into shaft2 using outward_claw in the left hand\",\n \"name\": \"action: insert(left_hand, outward_claw, gear2, shaft2)\"\n }\n ]\n }\n ]\n}"], ["You are an excellent behavior tree builder for robotic assembly tasks", "You are an excellent behavior tree builder for robotic assembly tasks. Given an action, you construct a unit subtree for the action based on its definition in the domain knowledge.\n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\nThe object types are explained as follows:\n\"\"\"\nthing: The base class for all objects in the world. Its subclasses are: hand, tool, part.\nhand: The robot hand. A empty can only hold one at a time.\ntool: The tool to manipulate . A empty can hold one at a time.\npart: The part to be manipulated.\n\"\"\"\n\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_empty(): is empty and can hold something.\n- can_manipulate(, ): can be used to manipulate (manipulation constraint). This regulates the compatibility between and .\n- can_insert_to(, ): can be inserted into (insertion assembly constraint)\n- can_screw_to(, ): can be screwed into (screw assembly constraint)\n- can_place_to(, ): can be placed into (placement assembly constraint)\n- hold(, ): is holding . It can be holding or holding .\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be empty in order to hold the . The will be held by the as a result of this action.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding . The will be empty and will not hold the anymore as a result of this action.\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be screwed into .\n- change_tool(, , ): change the equipped tool from to in order to manipulate specific parts. As preconditions, should be holding and should be empty.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: change_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to change the tool in the left hand from outwardgripper to defaultgripper\",\n \"name\": \"selector: change_tool(left_hand, outwardgripper, defaultgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is holding the default gripper\",\n \"name\": \"target: hold(left_hand, defaultgripper)\"\n },\n {\n \"summary\": \"Sequence to change the tool in the left hand from outwardgripper to defaultgripper\", \n \"name\": \"sequence: change_tool(left_hand, outwardgripper, defaultgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding the outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"check the precondition that the outwardgripper is empty\",\n \"name\": \"precondition: is_empty(outwardgripper)\"\n },\n {\n \"summary\": \"Action to change the tool in the left hand from outwardgripper to defaultgripper\",\n \"name\": \"action: change_tool(left_hand, outwardgripper, defaultgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: change_tool(left_hand, parallel_box1, outward_claw)\n---------------\n\n\n Return a JSON object.\n ", "{\n \"summary\": \"Selector to change the tool in the left hand from outward_claw to parallel_box1\",\n \"name\": \"selector: change_tool(left_hand, outward_claw, parallel_box1)\",\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is holding the parallel_box1\",\n \"name\": \"target: hold(left_hand, parallel_box1)\"\n },\n {\n \"summary\": \"Sequence to change the tool in the left hand from outward_claw to parallel_box1\",\n \"name\": \"sequence: change_tool(left_hand, outward_claw, parallel_box1)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding the outward_claw\",\n \"name\": \"precondition: hold(left_hand, outward_claw)\"\n },\n {\n \"summary\": \"check the precondition that the outward_claw is empty\",\n \"name\": \"precondition: is_empty(outward_claw)\"\n },\n {\n \"summary\": \"Action to change the tool in the left hand from outward_claw to parallel_box1\",\n \"name\": \"action: change_tool(left_hand, outward_claw, parallel_box1)\"\n }\n ]\n }\n ]\n}"], ["You are an excellent behavior tree builder for robotic assembly tasks", "You are an excellent behavior tree builder for robotic assembly tasks. Given an action, you construct a unit subtree for the action based on its definition in the domain knowledge.\n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\nThe object types are explained as follows:\n\"\"\"\nthing: The base class for all objects in the world. Its subclasses are: hand, tool, part.\nhand: The robot hand. A empty can only hold one at a time.\ntool: The tool to manipulate . A empty can hold one at a time.\npart: The part to be manipulated.\n\"\"\"\n\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_empty(): is empty and can hold something.\n- can_manipulate(, ): can be used to manipulate (manipulation constraint). This regulates the compatibility between and .\n- can_insert_to(, ): can be inserted into (insertion assembly constraint)\n- can_screw_to(, ): can be screwed into (screw assembly constraint)\n- can_place_to(, ): can be placed into (placement assembly constraint)\n- hold(, ): is holding . It can be holding or holding .\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be empty in order to hold the . The will be held by the as a result of this action.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding . The will be empty and will not hold the anymore as a result of this action.\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be screwed into .\n- change_tool(, , ): change the equipped tool from to in order to manipulate specific parts. As preconditions, should be holding and should be empty.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: change_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to change the tool in the left hand from outwardgripper to defaultgripper\",\n \"name\": \"selector: change_tool(left_hand, outwardgripper, defaultgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is holding the default gripper\",\n \"name\": \"target: hold(left_hand, defaultgripper)\"\n },\n {\n \"summary\": \"Sequence to change the tool in the left hand from outwardgripper to defaultgripper\", \n \"name\": \"sequence: change_tool(left_hand, outwardgripper, defaultgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding the outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"check the precondition that the outwardgripper is empty\",\n \"name\": \"precondition: is_empty(outwardgripper)\"\n },\n {\n \"summary\": \"Action to change the tool in the left hand from outwardgripper to defaultgripper\",\n \"name\": \"action: change_tool(left_hand, outwardgripper, defaultgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: insert(left_hand, outward_claw, gear2, shaft2)\n---------------\n\n\n Return a JSON object.\n ", "{\n \"summary\": \"Selector to insert gear2 into shaft2 using the outward claw in the left hand\",\n \"name\": \"selector: insert(left_hand, outward_claw, gear2, shaft2)\",\n \"children\": [\n {\n \"summary\": \"Check the target that gear2 is inserted into shaft2\",\n \"name\": \"target: is_inserted_to(gear2, shaft2)\"\n },\n {\n \"summary\": \"Sequence to insert gear2 into shaft2 using the outward claw in the left hand\", \n \"name\": \"sequence: insert(left_hand, outward_claw, gear2, shaft2)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding the outward claw\",\n \"name\": \"precondition: hold(left_hand, outward_claw)\"\n },\n {\n \"summary\": \"check the precondition that the outward claw is holding gear2\",\n \"name\": \"precondition: hold(outward_claw, gear2)\"\n },\n {\n \"summary\": \"Action to insert gear2 into shaft2 using the outward claw in the left hand\",\n \"name\": \"action: insert(left_hand, outward_claw, gear2, shaft2)\"\n }\n ]\n }\n ]\n}"], ["You are an excellent behavior tree builder for robotic assembly tasks", "You are an excellent behavior tree builder for robotic assembly tasks. Given an action, you construct a unit subtree for the action based on its definition in the domain knowledge.\n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\nThe object types are explained as follows:\n\"\"\"\nthing: The base class for all objects in the world. Its subclasses are: hand, tool, part.\nhand: The robot hand. A empty can only hold one at a time.\ntool: The tool to manipulate . A empty can hold one at a time.\npart: The part to be manipulated.\n\"\"\"\n\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_empty(): is empty and can hold something.\n- can_manipulate(, ): can be used to manipulate (manipulation constraint). This regulates the compatibility between and .\n- can_insert_to(, ): can be inserted into (insertion assembly constraint)\n- can_screw_to(, ): can be screwed into (screw assembly constraint)\n- can_place_to(, ): can be placed into (placement assembly constraint)\n- hold(, ): is holding . It can be holding or holding .\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be empty in order to hold the . The will be held by the as a result of this action.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding . The will be empty and will not hold the anymore as a result of this action.\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be screwed into .\n- change_tool(, , ): change the equipped tool from to in order to manipulate specific parts. As preconditions, should be holding and should be empty.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: change_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to change the tool in the left hand from outwardgripper to defaultgripper\",\n \"name\": \"selector: change_tool(left_hand, outwardgripper, defaultgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is holding the default gripper\",\n \"name\": \"target: hold(left_hand, defaultgripper)\"\n },\n {\n \"summary\": \"Sequence to change the tool in the left hand from outwardgripper to defaultgripper\", \n \"name\": \"sequence: change_tool(left_hand, outwardgripper, defaultgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding the outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"check the precondition that the outwardgripper is empty\",\n \"name\": \"precondition: is_empty(outwardgripper)\"\n },\n {\n \"summary\": \"Action to change the tool in the left hand from outwardgripper to defaultgripper\",\n \"name\": \"action: change_tool(left_hand, outwardgripper, defaultgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: change_tool(left_hand, parallel_box1, outward_claw)\n---------------\n\n\n Return a JSON object.\n ", "{\n \"summary\": \"Selector to change the tool in the left hand from parallel_box1 to outward_claw\",\n \"name\": \"selector: change_tool(left_hand, parallel_box1, outward_claw)\",\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is holding the outward_claw\",\n \"name\": \"target: hold(left_hand, outward_claw)\"\n },\n {\n \"summary\": \"Sequence to change the tool in the left hand from parallel_box1 to outward_claw\", \n \"name\": \"sequence: change_tool(left_hand, parallel_box1, outward_claw)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding the parallel_box1\",\n \"name\": \"precondition: hold(left_hand, parallel_box1)\"\n },\n {\n \"summary\": \"check the precondition that the parallel_box1 is empty\",\n \"name\": \"precondition: is_empty(parallel_box1)\"\n },\n {\n \"summary\": \"Action to change the tool in the left hand from parallel_box1 to outward_claw\",\n \"name\": \"action: change_tool(left_hand, parallel_box1, outward_claw)\"\n }\n ]\n }\n ]\n}"], ["You are an excellent behavior tree builder for robotic assembly tasks", "You are an excellent behavior tree builder for robotic assembly tasks. Given an action, you construct a unit subtree for the action based on its definition in the domain knowledge.\n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\nThe object types are explained as follows:\n\"\"\"\nthing: The base class for all objects in the world. Its subclasses are: hand, tool, part.\nhand: The robot hand. A empty can only hold one at a time.\ntool: The tool to manipulate . A empty can hold one at a time.\npart: The part to be manipulated.\n\"\"\"\n\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_empty(): is empty and can hold something.\n- can_manipulate(, ): can be used to manipulate (manipulation constraint). This regulates the compatibility between and .\n- can_insert_to(, ): can be inserted into (insertion assembly constraint)\n- can_screw_to(, ): can be screwed into (screw assembly constraint)\n- can_place_to(, ): can be placed into (placement assembly constraint)\n- hold(, ): is holding . It can be holding or holding .\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be empty in order to hold the . The will be held by the as a result of this action.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding . The will be empty and will not hold the anymore as a result of this action.\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be screwed into .\n- change_tool(, , ): change the equipped tool from to in order to manipulate specific parts. As preconditions, should be holding and should be empty.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: change_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to change the tool in the left hand from outwardgripper to defaultgripper\",\n \"name\": \"selector: change_tool(left_hand, outwardgripper, defaultgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is holding the default gripper\",\n \"name\": \"target: hold(left_hand, defaultgripper)\"\n },\n {\n \"summary\": \"Sequence to change the tool in the left hand from outwardgripper to defaultgripper\", \n \"name\": \"sequence: change_tool(left_hand, outwardgripper, defaultgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding the outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"check the precondition that the outwardgripper is empty\",\n \"name\": \"precondition: is_empty(outwardgripper)\"\n },\n {\n \"summary\": \"Action to change the tool in the left hand from outwardgripper to defaultgripper\",\n \"name\": \"action: change_tool(left_hand, outwardgripper, defaultgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: pick_up(left_hand, outward_claw, gear2)\n---------------\n\n\n Return a JSON object.\n ", "{\n \"summary\": \"Selector to pick up gear2 with outward_claw using left_hand\",\n \"name\": \"selector: pick_up(left_hand, outward_claw, gear2)\",\n \"children\": [\n {\n \"summary\": \"Check the target that the outward_claw is holding gear2\",\n \"name\": \"target: hold(outward_claw, gear2)\"\n },\n {\n \"summary\": \"Sequence to pick up gear2 with outward_claw using left_hand\", \n \"name\": \"sequence: pick_up(left_hand, outward_claw, gear2)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left_hand is holding the outward_claw\",\n \"name\": \"precondition: hold(left_hand, outward_claw)\"\n },\n {\n \"summary\": \"check the precondition that the outward_claw is empty\",\n \"name\": \"precondition: is_empty(outward_claw)\"\n },\n {\n \"summary\": \"Action to pick up gear2 with outward_claw using left_hand\",\n \"name\": \"action: pick_up(left_hand, outward_claw, gear2)\"\n }\n ]\n }\n ]\n}"], ["You are an excellent behavior tree builder for robotic assembly tasks", "You are an excellent behavior tree builder for robotic assembly tasks. Given an action, you construct a unit subtree for the action based on its definition in the domain knowledge.\n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\nThe object types are explained as follows:\n\"\"\"\nthing: The base class for all objects in the world. Its subclasses are: hand, tool, part.\nhand: The robot hand. A empty can only hold one at a time.\ntool: The tool to manipulate . A empty can hold one at a time.\npart: The part to be manipulated.\n\"\"\"\n\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_empty(): is empty and can hold something.\n- can_manipulate(, ): can be used to manipulate (manipulation constraint). This regulates the compatibility between and .\n- can_insert_to(, ): can be inserted into (insertion assembly constraint)\n- can_screw_to(, ): can be screwed into (screw assembly constraint)\n- can_place_to(, ): can be placed into (placement assembly constraint)\n- hold(, ): is holding . It can be holding or holding .\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be empty in order to hold the . The will be held by the as a result of this action.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding . The will be empty and will not hold the anymore as a result of this action.\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be screwed into .\n- change_tool(, , ): change the equipped tool from to in order to manipulate specific parts. As preconditions, should be holding and should be empty.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: change_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to change the tool in the left hand from outwardgripper to defaultgripper\",\n \"name\": \"selector: change_tool(left_hand, outwardgripper, defaultgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is holding the default gripper\",\n \"name\": \"target: hold(left_hand, defaultgripper)\"\n },\n {\n \"summary\": \"Sequence to change the tool in the left hand from outwardgripper to defaultgripper\", \n \"name\": \"sequence: change_tool(left_hand, outwardgripper, defaultgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding the outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"check the precondition that the outwardgripper is empty\",\n \"name\": \"precondition: is_empty(outwardgripper)\"\n },\n {\n \"summary\": \"Action to change the tool in the left hand from outwardgripper to defaultgripper\",\n \"name\": \"action: change_tool(left_hand, outwardgripper, defaultgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: put_down(left_hand, outward_claw, gear2)\n---------------\n\n\n Return a JSON object.\n ", "{\n \"summary\": \"Selector to put down gear2 using the outward claw in the left hand\",\n \"name\": \"selector: put_down(left_hand, outward_claw, gear2)\",\n \"children\": [\n {\n \"summary\": \"Check the target that the outward claw is not holding gear2\",\n \"name\": \"target: not hold(outward_claw, gear2)\"\n },\n {\n \"summary\": \"Sequence to put down gear2 using the outward claw in the left hand\", \n \"name\": \"sequence: put_down(left_hand, outward_claw, gear2)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding the outward claw\",\n \"name\": \"precondition: hold(left_hand, outward_claw)\"\n },\n {\n \"summary\": \"check the precondition that the outward claw is holding gear2\",\n \"name\": \"precondition: hold(outward_claw, gear2)\"\n },\n {\n \"summary\": \"Action to put down gear2 using the outward claw in the left hand\",\n \"name\": \"action: put_down(left_hand, outward_claw, gear2)\"\n }\n ]\n }\n ]\n}"], ["You are an excellent behavior tree builder for robotic assembly tasks", "You are an excellent behavior tree builder for robotic assembly tasks. Given an action, you construct a unit subtree for the action based on its definition in the domain knowledge.\n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\nThe object types are explained as follows:\n\"\"\"\nthing: The base class for all objects in the world. Its subclasses are: hand, tool, part.\nhand: The robot hand. A empty can only hold one at a time.\ntool: The tool to manipulate . A empty can hold one at a time.\npart: The part to be manipulated.\n\"\"\"\n\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_empty(): is empty and can hold something.\n- can_manipulate(, ): can be used to manipulate (manipulation constraint). This regulates the compatibility between and .\n- can_insert_to(, ): can be inserted into (insertion assembly constraint)\n- can_screw_to(, ): can be screwed into (screw assembly constraint)\n- can_place_to(, ): can be placed into (placement assembly constraint)\n- hold(, ): is holding . It can be holding or holding .\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be empty in order to hold the . The will be held by the as a result of this action.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding . The will be empty and will not hold the anymore as a result of this action.\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be screwed into .\n- change_tool(, , ): change the equipped tool from to in order to manipulate specific parts. As preconditions, should be holding and should be empty.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: change_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to change the tool in the left hand from outwardgripper to defaultgripper\",\n \"name\": \"selector: change_tool(left_hand, outwardgripper, defaultgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is holding the default gripper\",\n \"name\": \"target: hold(left_hand, defaultgripper)\"\n },\n {\n \"summary\": \"Sequence to change the tool in the left hand from outwardgripper to defaultgripper\", \n \"name\": \"sequence: change_tool(left_hand, outwardgripper, defaultgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding the outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"check the precondition that the outwardgripper is empty\",\n \"name\": \"precondition: is_empty(outwardgripper)\"\n },\n {\n \"summary\": \"Action to change the tool in the left hand from outwardgripper to defaultgripper\",\n \"name\": \"action: change_tool(left_hand, outwardgripper, defaultgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: insert(left_hand, outward_claw, gear2, shaft2)\n---------------\n\n\n Return a JSON object.\n ", "{\n \"summary\": \"Selector to insert gear2 into shaft2 using the outward claw in the left hand\",\n \"name\": \"selector: insert(left_hand, outward_claw, gear2, shaft2)\",\n \"children\": [\n {\n \"summary\": \"Check the target that gear2 is inserted into shaft2\",\n \"name\": \"target: is_inserted_to(gear2, shaft2)\"\n },\n {\n \"summary\": \"Sequence to insert gear2 into shaft2 using the outward claw in the left hand\", \n \"name\": \"sequence: insert(left_hand, outward_claw, gear2, shaft2)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding the outward claw\",\n \"name\": \"precondition: hold(left_hand, outward_claw)\"\n },\n {\n \"summary\": \"check the precondition that the outward claw is holding gear2\",\n \"name\": \"precondition: hold(outward_claw, gear2)\"\n },\n {\n \"summary\": \"Action to insert gear2 into shaft2 using the outward claw in the left hand\",\n \"name\": \"action: insert(left_hand, outward_claw, gear2, shaft2)\"\n }\n ]\n }\n ]\n}"], ["You are an excellent behavior tree builder for robotic assembly tasks", "You are an excellent behavior tree builder for robotic assembly tasks. Given an action, you construct a unit subtree for the action based on its definition in the domain knowledge.\n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\nThe object types are explained as follows:\n\"\"\"\nthing: The base class for all objects in the world. Its subclasses are: hand, tool, part.\nhand: The robot hand. A empty can only hold one at a time.\ntool: The tool to manipulate . A empty can hold one at a time.\npart: The part to be manipulated.\n\"\"\"\n\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_empty(): is empty and can hold something.\n- can_manipulate(, ): can be used to manipulate (manipulation constraint). This regulates the compatibility between and .\n- can_insert_to(, ): can be inserted into (insertion assembly constraint)\n- can_screw_to(, ): can be screwed into (screw assembly constraint)\n- can_place_to(, ): can be placed into (placement assembly constraint)\n- hold(, ): is holding . It can be holding or holding .\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be empty in order to hold the . The will be held by the as a result of this action.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding . The will be empty and will not hold the anymore as a result of this action.\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be screwed into .\n- change_tool(, , ): change the equipped tool from to in order to manipulate specific parts. As preconditions, should be holding and should be empty.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: change_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to change the tool in the left hand from outwardgripper to defaultgripper\",\n \"name\": \"selector: change_tool(left_hand, outwardgripper, defaultgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is holding the default gripper\",\n \"name\": \"target: hold(left_hand, defaultgripper)\"\n },\n {\n \"summary\": \"Sequence to change the tool in the left hand from outwardgripper to defaultgripper\", \n \"name\": \"sequence: change_tool(left_hand, outwardgripper, defaultgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding the outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"check the precondition that the outwardgripper is empty\",\n \"name\": \"precondition: is_empty(outwardgripper)\"\n },\n {\n \"summary\": \"Action to change the tool in the left hand from outwardgripper to defaultgripper\",\n \"name\": \"action: change_tool(left_hand, outwardgripper, defaultgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: change_tool(left_hand, parallel_box1, outward_claw)\n---------------\n\n\n Return a JSON object.\n ", "{\n \"summary\": \"Selector to change the tool in the left hand from parallel_box1 to outward_claw\",\n \"name\": \"selector: change_tool(left_hand, parallel_box1, outward_claw)\",\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is holding the outward_claw\",\n \"name\": \"target: hold(left_hand, outward_claw)\"\n },\n {\n \"summary\": \"Sequence to change the tool in the left hand from parallel_box1 to outward_claw\", \n \"name\": \"sequence: change_tool(left_hand, parallel_box1, outward_claw)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding the parallel_box1\",\n \"name\": \"precondition: hold(left_hand, parallel_box1)\"\n },\n {\n \"summary\": \"check the precondition that the parallel_box1 is empty\",\n \"name\": \"precondition: is_empty(parallel_box1)\"\n },\n {\n \"summary\": \"Action to change the tool in the left hand from parallel_box1 to outward_claw\",\n \"name\": \"action: change_tool(left_hand, parallel_box1, outward_claw)\"\n }\n ]\n }\n ]\n}"], ["You are an excellent behavior tree builder for robotic assembly tasks", "You are an excellent behavior tree builder for robotic assembly tasks. Given an action, you construct a unit subtree for the action based on its definition in the domain knowledge.\n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\nThe object types are explained as follows:\n\"\"\"\nthing: The base class for all objects in the world. Its subclasses are: hand, tool, part.\nhand: The robot hand. A empty can only hold one at a time.\ntool: The tool to manipulate . A empty can hold one at a time.\npart: The part to be manipulated.\n\"\"\"\n\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_empty(): is empty and can hold something.\n- can_manipulate(, ): can be used to manipulate (manipulation constraint). This regulates the compatibility between and .\n- can_insert_to(, ): can be inserted into (insertion assembly constraint)\n- can_screw_to(, ): can be screwed into (screw assembly constraint)\n- can_place_to(, ): can be placed into (placement assembly constraint)\n- hold(, ): is holding . It can be holding or holding .\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be empty in order to hold the . The will be held by the as a result of this action.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding . The will be empty and will not hold the anymore as a result of this action.\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be screwed into .\n- change_tool(, , ): change the equipped tool from to in order to manipulate specific parts. As preconditions, should be holding and should be empty.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: change_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to change the tool in the left hand from outwardgripper to defaultgripper\",\n \"name\": \"selector: change_tool(left_hand, outwardgripper, defaultgripper)\"\n \"children\": [\n {\n \"summary\": \"Check the target that the left hand is holding the default gripper\",\n \"name\": \"target: hold(left_hand, defaultgripper)\"\n },\n {\n \"summary\": \"Sequence to change the tool in the left hand from outwardgripper to defaultgripper\", \n \"name\": \"sequence: change_tool(left_hand, outwardgripper, defaultgripper)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left hand is holding the outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"check the precondition that the outwardgripper is empty\",\n \"name\": \"precondition: is_empty(outwardgripper)\"\n },\n {\n \"summary\": \"Action to change the tool in the left hand from outwardgripper to defaultgripper\",\n \"name\": \"action: change_tool(left_hand, outwardgripper, defaultgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: pick_up(left_hand, outward_claw, gear2)\n---------------\n\n\n Return a JSON object.\n ", "{\n \"summary\": \"Selector to pick up gear2 with outward_claw using left_hand\",\n \"name\": \"selector: pick_up(left_hand, outward_claw, gear2)\",\n \"children\": [\n {\n \"summary\": \"Check the target that the outward_claw is holding gear2\",\n \"name\": \"target: hold(outward_claw, gear2)\"\n },\n {\n \"summary\": \"Sequence to pick up gear2 with outward_claw using left_hand\", \n \"name\": \"sequence: pick_up(left_hand, outward_claw, gear2)\",\n \"children\": [\n {\n \"summary\": \"check the precondition that the left_hand is holding the outward_claw\",\n \"name\": \"precondition: hold(left_hand, outward_claw)\"\n },\n {\n \"summary\": \"check the precondition that the outward_claw is empty\",\n \"name\": \"precondition: is_empty(outward_claw)\"\n },\n {\n \"summary\": \"Action to pick up gear2 with outward_claw using left_hand\",\n \"name\": \"action: pick_up(left_hand, outward_claw, gear2)\"\n }\n ]\n }\n ]\n}"], ["You are an excellent behavior tree builder for robotic assembly tasks", "You are an excellent behavior tree builder for robotic assembly tasks. Given an action, you construct a unit subtree for the action based on its definition in the domain knowledge.\n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\nThe object types are explained as follows:\n\"\"\"\nthing: The base class for all objects in the world. Its subclasses are: hand, tool, part.\nhand: The robot hand. A empty can only hold one at a time.\ntool: The tool to manipulate . A empty can hold one at a time.\npart: The part to be manipulated.\n\"\"\"\n\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_empty(): is empty and can hold something.\n- can_manipulate(, ): can be used to manipulate (manipulation constraint). This regulates the compatibility between and .\n- can_insert_to(, ): can be inserted into (insertion assembly constraint)\n- can_screw_to(, ): can be screwed into (screw assembly constraint)\n- can_place_to(, ): can be placed into (placement assembly constraint)\n- hold(, ): is holding . It can be holding or holding .\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be empty in order to hold the . The will be held by the as a result of this action.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding . The will be empty and will not hold the anymore as a result of this action.\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be screwed into .\n- change_tool(, , ): change the equipped tool from to in order to manipulate specific parts. As preconditions, should be holding and should be empty.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: change_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to change the tool in the left hand from outwardgripper to defaultgripper\",\n \"name\": \"selector: change_tool(left_hand, outwardgripper, defaultgripper)\"\n \"children\": [\n {\n \"summary\": \"The target is to make the left hand hold the default gripper\",\n \"name\": \"target: hold(left_hand, defaultgripper)\"\n },\n {\n \"summary\": \"Sequence to change the tool in the left hand from outwardgripper to defaultgripper\", \n \"name\": \"sequence: change_tool(left_hand, outwardgripper, defaultgripper)\",\n \"children\": [\n {\n \"summary\": \"A precondition is that the left hand is holding the outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"A precondition is that the outwardgripper is empty\",\n \"name\": \"precondition: is_empty(outwardgripper)\"\n },\n {\n \"summary\": \"Take the action to change the tool in the left hand from outwardgripper to defaultgripper\",\n \"name\": \"action: change_tool(left_hand, outwardgripper, defaultgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: insert(left_hand, defaultgripper, gear1, shaft1)\n---------------\n\n\n Return a JSON object.\n ", "{\n \"summary\": \"Selector to insert gear1 into shaft1 using left hand with defaultgripper\",\n \"name\": \"selector: insert(left_hand, defaultgripper, gear1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"The target is to make gear1 inserted into shaft1\",\n \"name\": \"target: is_inserted_to(gear1, shaft1)\"\n },\n {\n \"summary\": \"Sequence to insert gear1 into shaft1 using left hand with defaultgripper\", \n \"name\": \"sequence: insert(left_hand, defaultgripper, gear1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"A precondition is that the left hand is holding the defaultgripper\",\n \"name\": \"precondition: hold(left_hand, defaultgripper)\"\n },\n {\n \"summary\": \"A precondition is that the defaultgripper is holding gear1\",\n \"name\": \"precondition: hold(defaultgripper, gear1)\"\n },\n {\n \"summary\": \"Take the action to insert gear1 into shaft1 using left hand with defaultgripper\",\n \"name\": \"action: insert(left_hand, defaultgripper, gear1, shaft1)\"\n }\n ]\n }\n ]\n}"], ["You are an excellent behavior tree builder for robotic assembly tasks", "You are an excellent behavior tree builder for robotic assembly tasks. Given an action, you construct a unit subtree for the action based on its definition in the domain knowledge.\n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\nThe object types are explained as follows:\n\"\"\"\nthing: The base class for all objects in the world. Its subclasses are: hand, tool, part.\nhand: The robot hand. A empty can only hold one at a time.\ntool: The tool to manipulate . A empty can hold one at a time.\npart: The part to be manipulated.\n\"\"\"\n\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_empty(): is empty and can hold something.\n- can_manipulate(, ): can be used to manipulate (manipulation constraint). This regulates the compatibility between and .\n- can_insert_to(, ): can be inserted into (insertion assembly constraint)\n- can_screw_to(, ): can be screwed into (screw assembly constraint)\n- can_place_to(, ): can be placed into (placement assembly constraint)\n- hold(, ): is holding . It can be holding or holding .\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be empty in order to hold the . The will be held by the as a result of this action.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding . The will be empty and will not hold the anymore as a result of this action.\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be screwed into .\n- change_tool(, , ): change the equipped tool from to in order to manipulate specific parts. As preconditions, should be holding and should be empty.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: change_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to change the tool in the left hand from outwardgripper to defaultgripper\",\n \"name\": \"selector: change_tool(left_hand, outwardgripper, defaultgripper)\"\n \"children\": [\n {\n \"summary\": \"The target is to make the left hand hold the default gripper\",\n \"name\": \"target: hold(left_hand, defaultgripper)\"\n },\n {\n \"summary\": \"Sequence to change the tool in the left hand from outwardgripper to defaultgripper\", \n \"name\": \"sequence: change_tool(left_hand, outwardgripper, defaultgripper)\",\n \"children\": [\n {\n \"summary\": \"A precondition is that the left hand is holding the outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"A precondition is that the outwardgripper is empty\",\n \"name\": \"precondition: is_empty(outwardgripper)\"\n },\n {\n \"summary\": \"Take the action to change the tool in the left hand from outwardgripper to defaultgripper\",\n \"name\": \"action: change_tool(left_hand, outwardgripper, defaultgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: insert(left_hand, defaultgripper, gear1, shaft1)\n---------------\n\n\n Return a JSON object.\n ", "{\n \"summary\": \"Selector to insert gear1 into shaft1 using left hand with defaultgripper\",\n \"name\": \"selector: insert(left_hand, defaultgripper, gear1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"The target is to make gear1 inserted into shaft1\",\n \"name\": \"target: is_inserted_to(gear1, shaft1)\"\n },\n {\n \"summary\": \"Sequence to insert gear1 into shaft1 using left hand with defaultgripper\", \n \"name\": \"sequence: insert(left_hand, defaultgripper, gear1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"A precondition is that the left hand is holding the defaultgripper\",\n \"name\": \"precondition: hold(left_hand, defaultgripper)\"\n },\n {\n \"summary\": \"A precondition is that the defaultgripper is holding gear1\",\n \"name\": \"precondition: hold(defaultgripper, gear1)\"\n },\n {\n \"summary\": \"Take the action to insert gear1 into shaft1 using left hand with defaultgripper\",\n \"name\": \"action: insert(left_hand, defaultgripper, gear1, shaft1)\"\n }\n ]\n }\n ]\n}"], ["You are an excellent behavior tree builder for robotic assembly tasks", "You are an excellent behavior tree builder for robotic assembly tasks. Given an action, you construct a unit subtree for the action based on its definition in the domain knowledge.\n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\nThe object types are explained as follows:\n\"\"\"\nthing: The base class for all objects in the world. Its subclasses are: hand, tool, part.\nhand: The robot hand. A empty can only hold one at a time.\ntool: The tool to manipulate . A empty can hold one at a time.\npart: The part to be manipulated.\n\"\"\"\n\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_empty(): is empty and can hold something.\n- can_manipulate(, ): can be used to manipulate (manipulation constraint). This regulates the compatibility between and .\n- can_insert_to(, ): can be inserted into (insertion assembly constraint)\n- can_screw_to(, ): can be screwed into (screw assembly constraint)\n- can_place_to(, ): can be placed into (placement assembly constraint)\n- hold(, ): is holding . It can be holding or holding .\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be empty in order to hold the . The will be held by the as a result of this action.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding . The will be empty and will not hold the anymore as a result of this action.\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be screwed into .\n- change_tool(, , ): change the equipped tool from to in order to manipulate specific parts. As preconditions, should be holding and should be empty.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: change_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to change the tool in the left hand from outwardgripper to defaultgripper\",\n \"name\": \"selector: change_tool(left_hand, outwardgripper, defaultgripper)\"\n \"children\": [\n {\n \"summary\": \"The target is to make the left hand hold the default gripper\",\n \"name\": \"target: hold(left_hand, defaultgripper)\"\n },\n {\n \"summary\": \"Sequence to change the tool in the left hand from outwardgripper to defaultgripper\", \n \"name\": \"sequence: change_tool(left_hand, outwardgripper, defaultgripper)\",\n \"children\": [\n {\n \"summary\": \"A precondition is that the left hand is holding the outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"A precondition is that the outwardgripper is empty\",\n \"name\": \"precondition: is_empty(outwardgripper)\"\n },\n {\n \"summary\": \"Take the action to change the tool in the left hand from outwardgripper to defaultgripper\",\n \"name\": \"action: change_tool(left_hand, outwardgripper, defaultgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: insert(left_hand, defaultgripper, gear1, shaft1)\n---------------\n\n\n Return a JSON object.\n ", "{\n \"summary\": \"Selector to insert gear1 into shaft1 using left hand with defaultgripper\",\n \"name\": \"selector: insert(left_hand, defaultgripper, gear1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"The target is to make gear1 inserted into shaft1\",\n \"name\": \"target: is_inserted_to(gear1, shaft1)\"\n },\n {\n \"summary\": \"Sequence to insert gear1 into shaft1 using left hand with defaultgripper\", \n \"name\": \"sequence: insert(left_hand, defaultgripper, gear1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"A precondition is that the left hand is holding the defaultgripper\",\n \"name\": \"precondition: hold(left_hand, defaultgripper)\"\n },\n {\n \"summary\": \"A precondition is that the defaultgripper is holding gear1\",\n \"name\": \"precondition: hold(defaultgripper, gear1)\"\n },\n {\n \"summary\": \"A precondition is that the defaultgripper can manipulate gear1\",\n \"name\": \"precondition: can_manipulate(defaultgripper, gear1)\"\n },\n {\n \"summary\": \"A precondition is that gear1 can be inserted into shaft1\",\n \"name\": \"precondition: can_insert_to(gear1, shaft1)\"\n },\n {\n \"summary\": \"Take the action to insert gear1 into shaft1 using left hand with defaultgripper\",\n \"name\": \"action: insert(left_hand, defaultgripper, gear1, shaft1)\"\n }\n ]\n }\n ]\n}"], ["You are an excellent behavior tree builder for robotic assembly tasks", "You are an excellent behavior tree builder for robotic assembly tasks. Given an action, you construct a unit subtree for the action based on its definition in the domain knowledge.\n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\nThe object types are explained as follows:\n\"\"\"\nthing: The base class for all objects in the world. Its subclasses are: hand, tool, part.\nhand: The robot hand. A empty can only hold one at a time.\ntool: The tool to manipulate . A empty can hold one at a time.\npart: The part to be manipulated.\n\"\"\"\n\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_empty(): is empty and can hold something.\n- can_manipulate(, ): can be used to manipulate (manipulation constraint). This regulates the compatibility between and .\n- can_insert_to(, ): can be inserted into (insertion assembly constraint)\n- can_screw_to(, ): can be screwed into (screw assembly constraint)\n- can_place_to(, ): can be placed into (placement assembly constraint)\n- hold(, ): is holding . It can be holding or holding .\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be empty in order to hold the . The will be held by the as a result of this action.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding . The will be empty and will not hold the anymore as a result of this action.\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be screwed into .\n- change_tool(, , ): change the equipped tool from to in order to manipulate specific parts. As preconditions, should be holding and should be empty.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: change_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to change the tool in the left hand from outwardgripper to defaultgripper\",\n \"name\": \"selector: change_tool(left_hand, outwardgripper, defaultgripper)\"\n \"children\": [\n {\n \"summary\": \"The target is to make the left hand hold the default gripper\",\n \"name\": \"target: hold(left_hand, defaultgripper)\"\n },\n {\n \"summary\": \"Sequence to change the tool in the left hand from outwardgripper to defaultgripper\", \n \"name\": \"sequence: change_tool(left_hand, outwardgripper, defaultgripper)\",\n \"children\": [\n {\n \"summary\": \"A precondition is that the left hand is holding the outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"A precondition is that the outwardgripper is empty\",\n \"name\": \"precondition: is_empty(outwardgripper)\"\n },\n {\n \"summary\": \"Take the action to change the tool in the left hand from outwardgripper to defaultgripper\",\n \"name\": \"action: change_tool(left_hand, outwardgripper, defaultgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: insert(left_hand, defaultgripper, gear1, shaft1)\n---------------\n\n\n Return a JSON object.\n ", "{\n \"summary\": \"Selector to insert gear1 into shaft1 using left hand with defaultgripper\",\n \"name\": \"selector: insert(left_hand, defaultgripper, gear1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"The target is to make gear1 inserted into shaft1\",\n \"name\": \"target: is_inserted_to(gear1, shaft1)\"\n },\n {\n \"summary\": \"Sequence to insert gear1 into shaft1 using left hand with defaultgripper\", \n \"name\": \"sequence: insert(left_hand, defaultgripper, gear1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"A precondition is that the left hand is holding the defaultgripper\",\n \"name\": \"precondition: hold(left_hand, defaultgripper)\"\n },\n {\n \"summary\": \"A precondition is that the defaultgripper is holding gear1\",\n \"name\": \"precondition: hold(defaultgripper, gear1)\"\n },\n {\n \"summary\": \"A precondition is that gear1 can be inserted into shaft1\",\n \"name\": \"precondition: can_insert_to(gear1, shaft1)\"\n },\n {\n \"summary\": \"Take the action to insert gear1 into shaft1 using left hand with defaultgripper\",\n \"name\": \"action: insert(left_hand, defaultgripper, gear1, shaft1)\"\n }\n ]\n }\n ]\n}"], ["You are an excellent behavior tree builder for robotic assembly tasks", "You are an excellent behavior tree builder for robotic assembly tasks. Given an action, you construct a unit subtree for the action based on its definition in the domain knowledge.\n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\nThe object types are explained as follows:\n\"\"\"\nthing: The base class for all objects in the world. Its subclasses are: hand, tool, part.\nhand: The robot hand. A empty can only hold one at a time.\ntool: The tool to manipulate . A empty can hold one at a time.\npart: The part to be manipulated.\n\"\"\"\n\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_empty(): is empty and can hold something.\n- can_manipulate(, ): can be used to manipulate (manipulation constraint). This regulates the compatibility between and .\n- can_insert_to(, ): can be inserted into (insertion assembly constraint)\n- can_screw_to(, ): can be screwed into (screw assembly constraint)\n- can_place_to(, ): can be placed into (placement assembly constraint)\n- hold(, ): is holding . It can be holding or holding .\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be empty in order to hold the . The will be held by the as a result of this action.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding . The will be empty and will not hold the anymore as a result of this action.\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be screwed into .\n- change_tool(, , ): change the equipped tool from to in order to manipulate specific parts. As preconditions, should be holding and should be empty.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: change_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to change the tool in the left hand from outwardgripper to defaultgripper\",\n \"name\": \"selector: change_tool(left_hand, outwardgripper, defaultgripper)\"\n \"children\": [\n {\n \"summary\": \"The target is to make the left hand hold the default gripper\",\n \"name\": \"target: hold(left_hand, defaultgripper)\"\n },\n {\n \"summary\": \"Sequence to change the tool in the left hand from outwardgripper to defaultgripper\", \n \"name\": \"sequence: change_tool(left_hand, outwardgripper, defaultgripper)\",\n \"children\": [\n {\n \"summary\": \"A precondition is that the left hand is holding the outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"A precondition is that the outwardgripper is empty\",\n \"name\": \"precondition: is_empty(outwardgripper)\"\n },\n {\n \"summary\": \"Take the action to change the tool in the left hand from outwardgripper to defaultgripper\",\n \"name\": \"action: change_tool(left_hand, outwardgripper, defaultgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: insert(left_hand, defaultgripper, gear1, shaft1)\n---------------\n\n\n Return a JSON object.\n ", "{\n \"summary\": \"Selector to insert gear1 into shaft1 using left hand with defaultgripper\",\n \"name\": \"selector: insert(left_hand, defaultgripper, gear1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"The target is to make gear1 inserted into shaft1\",\n \"name\": \"target: is_inserted_to(gear1, shaft1)\"\n },\n {\n \"summary\": \"Sequence to insert gear1 into shaft1 using left hand with defaultgripper\", \n \"name\": \"sequence: insert(left_hand, defaultgripper, gear1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"A precondition is that the left hand is holding the defaultgripper\",\n \"name\": \"precondition: hold(left_hand, defaultgripper)\"\n },\n {\n \"summary\": \"A precondition is that the defaultgripper is holding gear1\",\n \"name\": \"precondition: hold(defaultgripper, gear1)\"\n },\n {\n \"summary\": \"Take the action to insert gear1 into shaft1 using left hand with defaultgripper\",\n \"name\": \"action: insert(left_hand, defaultgripper, gear1, shaft1)\"\n }\n ]\n }\n ]\n}"], ["You are an excellent behavior tree builder for robotic assembly tasks", "You are an excellent behavior tree builder for robotic assembly tasks. Given an action, you construct a unit subtree for the action based on its definition in the domain knowledge.\n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\nThe object types are explained as follows:\n\"\"\"\nthing: The base class for all objects in the world. Its subclasses are: hand, tool, part.\nhand: The robot hand. A empty can only hold one at a time.\ntool: The tool to manipulate . A empty can hold one at a time.\npart: The part to be manipulated.\n\"\"\"\n\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_empty(): is empty and can hold something.\n- can_manipulate(, ): can be used to manipulate (manipulation constraint). This regulates the compatibility between and .\n- can_insert_to(, ): can be inserted into (insertion assembly constraint)\n- can_screw_to(, ): can be screwed into (screw assembly constraint)\n- can_place_to(, ): can be placed into (placement assembly constraint)\n- hold(, ): is holding . It can be holding or holding .\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be empty in order to hold the . The will be held by the as a result of this action.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding . The will be empty and will not hold the anymore as a result of this action.\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be screwed into .\n- change_tool(, , ): change the equipped tool from to in order to manipulate specific parts. As preconditions, should be holding and should be empty.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: change_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to change the tool in the left hand from outwardgripper to defaultgripper\",\n \"name\": \"selector: change_tool(left_hand, outwardgripper, defaultgripper)\"\n \"children\": [\n {\n \"summary\": \"The target is to make the left hand hold the default gripper\",\n \"name\": \"target: hold(left_hand, defaultgripper)\"\n },\n {\n \"summary\": \"Sequence to change the tool in the left hand from outwardgripper to defaultgripper\", \n \"name\": \"sequence: change_tool(left_hand, outwardgripper, defaultgripper)\",\n \"children\": [\n {\n \"summary\": \"A precondition is that the left hand is holding the outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"A precondition is that the outwardgripper is empty\",\n \"name\": \"precondition: is_empty(outwardgripper)\"\n },\n {\n \"summary\": \"Take the action to change the tool in the left hand from outwardgripper to defaultgripper\",\n \"name\": \"action: change_tool(left_hand, outwardgripper, defaultgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: insert(left_hand, defaultgripper, gear1, shaft1)\n---------------\n\n\n Return a JSON object.\n ", "{\n \"summary\": \"Selector to insert gear1 into shaft1 using the default gripper in the left hand\",\n \"name\": \"selector: insert(left_hand, defaultgripper, gear1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"The target is to have gear1 inserted into shaft1\",\n \"name\": \"target: is_inserted_to(gear1, shaft1)\"\n },\n {\n \"summary\": \"Sequence to insert gear1 into shaft1 using the default gripper in the left hand\",\n \"name\": \"sequence: insert(left_hand, defaultgripper, gear1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"A precondition is that the left hand is holding the default gripper\",\n \"name\": \"precondition: hold(left_hand, defaultgripper)\"\n },\n {\n \"summary\": \"A precondition is that the default gripper is holding gear1\",\n \"name\": \"precondition: hold(defaultgripper, gear1)\"\n },\n {\n \"summary\": \"Take the action to insert gear1 into shaft1 using the default gripper in the left hand\",\n \"name\": \"action: insert(left_hand, defaultgripper, gear1, shaft1)\"\n }\n ]\n }\n ]\n}"], ["You are an excellent behavior tree builder for robotic assembly tasks", "You are an excellent behavior tree builder for robotic assembly tasks. Given an action, you construct a unit subtree for the action based on its definition in the domain knowledge.\n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\nThe object types are explained as follows:\n\"\"\"\nthing: The base class for all objects in the world. Its subclasses are: hand, tool, part.\nhand: The robot hand. A empty can only hold one at a time.\ntool: The tool to manipulate . A empty can hold one at a time.\npart: The part to be manipulated.\n\"\"\"\n\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_empty(): is empty and can hold something.\n- can_manipulate(, ): can be used to manipulate (manipulation constraint). This regulates the compatibility between and .\n- can_insert_to(, ): can be inserted into (insertion assembly constraint)\n- can_screw_to(, ): can be screwed into (screw assembly constraint)\n- can_place_to(, ): can be placed into (placement assembly constraint)\n- hold(, ): is holding . It can be holding or holding .\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be empty in order to hold the . The will be held by the as a result of this action.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding . The will be empty and will not hold the anymore as a result of this action.\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be screwed into .\n- change_tool(, , ): change the equipped tool from to in order to manipulate specific parts. As preconditions, should be holding and should be empty.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: change_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to change the tool in the left hand from outwardgripper to defaultgripper\",\n \"name\": \"selector: change_tool(left_hand, outwardgripper, defaultgripper)\"\n \"children\": [\n {\n \"summary\": \"The target is to make the left hand hold the default gripper\",\n \"name\": \"target: hold(left_hand, defaultgripper)\"\n },\n {\n \"summary\": \"Sequence to change the tool in the left hand from outwardgripper to defaultgripper\", \n \"name\": \"sequence: change_tool(left_hand, outwardgripper, defaultgripper)\",\n \"children\": [\n {\n \"summary\": \"A precondition is that the left hand is holding the outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"A precondition is that the outwardgripper is empty\",\n \"name\": \"precondition: is_empty(outwardgripper)\"\n },\n {\n \"summary\": \"Take the action to change the tool in the left hand from outwardgripper to defaultgripper\",\n \"name\": \"action: change_tool(left_hand, outwardgripper, defaultgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: insert(left_hand, defaultgripper, gear1, shaft1)\n---------------\n\n\n Return a JSON object.\n ", "{\n \"summary\": \"Selector to insert gear1 into shaft1 using the default gripper in the left hand\",\n \"name\": \"selector: insert(left_hand, defaultgripper, gear1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"The target is to have gear1 inserted into shaft1\",\n \"name\": \"target: is_inserted_to(gear1, shaft1)\"\n },\n {\n \"summary\": \"Sequence to insert gear1 into shaft1 using the default gripper in the left hand\",\n \"name\": \"sequence: insert(left_hand, defaultgripper, gear1, shaft1)\",\n \"children\": [\n {\n \"summary\": \"A precondition is that the left hand is holding the default gripper\",\n \"name\": \"precondition: hold(left_hand, defaultgripper)\"\n },\n {\n \"summary\": \"A precondition is that the default gripper is holding gear1\",\n \"name\": \"precondition: hold(defaultgripper, gear1)\"\n },\n {\n \"summary\": \"Take the action to insert gear1 into shaft1 using the default gripper in the left hand\",\n \"name\": \"action: insert(left_hand, defaultgripper, gear1, shaft1)\"\n }\n ]\n }\n ]\n}"], ["You are an excellent behavior tree builder for robotic assembly tasks", "You are an excellent behavior tree builder for robotic assembly tasks. Given an action, you construct a unit subtree for the action based on its definition in the domain knowledge.\n\n The input are:\naction: The action that you construct the unit subtree from.\n\nYou analyze the action, refer to the domain knowledge to understand the action, and construct a unit subtree for it.\nThe unit subtree should follow the regulated format.\nYou only return the unit subtree as a json object.\n\n Following is the domain knowledge you need, which includes:\n- the object types in the world,\n- the predicates for describing the states of the world,\n- the actions that the robot can apply, with their precondtions and effects defined.\n\nThe object types are explained as follows:\n\"\"\"\nthing: The base class for all objects in the world. Its subclasses are: hand, tool, part.\nhand: The robot hand. A empty can only hold one at a time.\ntool: The tool to manipulate . A empty can hold one at a time.\npart: The part to be manipulated.\n\"\"\"\n\nThe precidates are explained as follows:\n\"\"\"\n\"PREDICATE_LIST\"\n- is_empty(): is empty and can hold something.\n- can_manipulate(, ): can be used to manipulate (manipulation constraint). This regulates the compatibility between and .\n- can_insert_to(, ): can be inserted into (insertion assembly constraint)\n- can_screw_to(, ): can be screwed into (screw assembly constraint)\n- can_place_to(, ): can be placed into (placement assembly constraint)\n- hold(, ): is holding . It can be holding or holding .\n- is_inserted_to(, ): is inserted into (insertion assembly constraint is satisfied)\n- is_screwed_to(, ): is screwed into (screw assembly constraint is satisfied)\n- is_placed_to(, ): is placed into (placement assembly constraint is satisfied)\n\"\"\"\n\nThe actions are explained as follows:\n\"\"\"\n\"ROBOT_ACTION_LIST\"\n- pick_up(, , ): use to pick up . As preconditions, should be holding and should be empty in order to hold the . The will be held by the as a result of this action.\n- put_down(, , ): use to put down . As preconditions, should be holding and should be holding . The will be empty and will not hold the anymore as a result of this action.\n- place(, , , ): use to place to (to satisfy the placement assembly constraint between and ). As preconditions, should be holding and should be holding .\n- detach(, , , ): use to detach from (to lift the existing placement assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be placed into .\n- insert(, , , ): use to insert into (to satisfy the insertion assembly constraint between and ). As preconditions, should be holding and should be holding .\n- pull(, , , ): use to pull from (to lift the existing insertion assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be inserted into .\n- screw(, , , ): use to screw into (to satisfy the screw assembly constraint between and ). As preconditions, should be holding and should be holding .\n- unscrew(, , , ): use to unscrew from (to lift the existing screw assembly constraint between and ). As preconditions, should be holding and should be empty to manipulate , and should be screwed into .\n- change_tool(, , ): change the equipped tool from to in order to manipulate specific parts. As preconditions, should be holding and should be empty.\n\"\"\"\nthe items in <> indicate the type of the instances needed for the predicates and actions.\n\n The behavior tree should be in json dictionary format.\nWhen building a unit subtree, follow these rules:\n1. Every node are described by a summary and a name:\n - \"summary\": a short interpretive description of the node. \n - \"name\": includes the node type and the formal form of the node according to the domain knowledge. Legal node types are \"selector\", \"sequence\", \"condition\" (\"target\", \"precondition\"), \"action\".\n2. The control flow nodes, namely \"selector\" and \"sequence\", can have \"children\", which is a list of children nodes.\n3. The \"condition\" nodes that are children of \"selectors\" are targets. Those that are children of \"sequences\" are preconditions. \n4. All the node should always have their corresponding actions/precidates in the domain knowledge. \n5. Control flow nodes in behavior trees don't have memory. Each tick starts from the root and traverses all nodes, regardless of their previous state.\n6. The basic structure template of a unit subtree is:\n - A selector as the root node. \n - One target condition node as the first child of the selector to check if the target of this subtree has been satisfied.\n - A sequence node as the last child of the selector node to achieve the target condition node.\n - A number of precondition nodes as the first child nodes of the sequence node to check the action's preconditions.\n - One action node as the last child of the sequence node, whose preconditions are checked in the child condition nodes before.\n - The action should have effects that can fulfill the target condition node in the selector.\nFollowing is an example:\naction: change_tool(left_hand, outwardgripper)\nunit subtree:\n{\n \"summary\": \"Selector to change the tool in the left hand from outwardgripper to defaultgripper\",\n \"name\": \"selector: change_tool(left_hand, outwardgripper, defaultgripper)\"\n \"children\": [\n {\n \"summary\": \"The target is to make the left hand hold the default gripper\",\n \"name\": \"target: hold(left_hand, defaultgripper)\"\n },\n {\n \"summary\": \"Sequence to change the tool in the left hand from outwardgripper to defaultgripper\", \n \"name\": \"sequence: change_tool(left_hand, outwardgripper, defaultgripper)\",\n \"children\": [\n {\n \"summary\": \"A precondition is that the left hand is holding the outwardgripper\",\n \"name\": \"precondition: hold(left_hand, outwardgripper)\"\n },\n {\n \"summary\": \"A precondition is that the outwardgripper is empty\",\n \"name\": \"precondition: is_empty(outwardgripper)\"\n },\n {\n \"summary\": \"Take the action to change the tool in the left hand from outwardgripper to defaultgripper\",\n \"name\": \"action: change_tool(left_hand, outwardgripper, defaultgripper)\"\n }\n ]\n }\n ]\n}\n\n\n\n Start working. Resume from the input below.\n---------------\naction: change_tool(left_hand, parallel_box1, defaultgripper)\n---------------\n\n\n Return a JSON object.\n ", "{\n \"summary\": \"Selector to change the tool in the left hand from parallel_box1 to defaultgripper\",\n \"name\": \"selector: change_tool(left_hand, parallel_box1, defaultgripper)\",\n \"children\": [\n {\n \"summary\": \"The target is to make the left hand hold the default gripper\",\n \"name\": \"target: hold(left_hand, defaultgripper)\"\n },\n {\n \"summary\": \"Sequence to change the tool in the left hand from parallel_box1 to defaultgripper\",\n \"name\": \"sequence: change_tool(left_hand, parallel_box1, defaultgripper)\",\n \"children\": [\n {\n \"summary\": \"A precondition is that the left hand is holding the parallel_box1\",\n \"name\": \"precondition: hold(left_hand, parallel_box1)\"\n },\n {\n \"summary\": \"A precondition is that the parallel_box1 is empty\",\n \"name\": \"precondition: is_empty(parallel_box1)\"\n },\n {\n \"summary\": \"Take the action to change the tool in the left hand from parallel_box1 to defaultgripper\",\n \"name\": \"action: change_tool(left_hand, parallel_box1, defaultgripper)\"\n }\n ]\n }\n ]\n}"]]} \ No newline at end of file diff --git a/wandb/run-20240303_162941-ftjob-AncbMuC4XGCe9OOk4zjjAz5g/files/output.log b/wandb/run-20240303_162941-ftjob-AncbMuC4XGCe9OOk4zjjAz5g/files/output.log deleted file mode 100644 index f833bb74..00000000 --- a/wandb/run-20240303_162941-ftjob-AncbMuC4XGCe9OOk4zjjAz5g/files/output.log +++ /dev/null @@ -1,45 +0,0 @@ -wandb: Waiting for the OpenAI fine-tuning job to be finished... -wandb: Fine-tuning finished, logging metrics, model metadata, and more to W&B -wandb: Waiting for artifact train-file-D8aBgTaXbp3g42RcV2bTiKDK to be committed... -wandb: | 2.523 MB of 2.523 MB uploaded -wandb: Committed artifact proneverfake/OpenAI-Fine-Tune/train-file-D8aBgTaXbp3g42RcV2bTiKDK:v0 -Traceback (most recent call last): - File "/home/blackbird/kios/data/finetuning/visualize.py", line 5, in - WandbLogger.sync(fine_tune_job_id="ftjob-AncbMuC4XGCe9OOk4zjjAz5g") - File "/home/blackbird/miniconda3/envs/kios/lib/python3.10/site-packages/wandb/integration/openai/fine_tuning.py", line 148, in sync - cls._log_fine_tune( - File "/home/blackbird/miniconda3/envs/kios/lib/python3.10/site-packages/wandb/integration/openai/fine_tuning.py", line 237, in _log_fine_tune - cls._log_artifacts(fine_tune, project, entity) - File "/home/blackbird/miniconda3/envs/kios/lib/python3.10/site-packages/wandb/integration/openai/fine_tuning.py", line 344, in _log_artifacts - json.dump(dict_fine_tune, f, indent=2) - File "/home/blackbird/miniconda3/envs/kios/lib/python3.10/json/__init__.py", line 179, in dump - for chunk in iterable: - File "/home/blackbird/miniconda3/envs/kios/lib/python3.10/json/encoder.py", line 431, in _iterencode - yield from _iterencode_dict(o, _current_indent_level) - File "/home/blackbird/miniconda3/envs/kios/lib/python3.10/json/encoder.py", line 405, in _iterencode_dict - yield from chunks - File "/home/blackbird/miniconda3/envs/kios/lib/python3.10/json/encoder.py", line 438, in _iterencode - o = _default(o) - File "/home/blackbird/miniconda3/envs/kios/lib/python3.10/json/encoder.py", line 179, in default - raise TypeError(f'Object of type {o.__class__.__name__} ' -TypeError: Object of type Error is not JSON serializable -Traceback (most recent call last): - File "/home/blackbird/kios/data/finetuning/visualize.py", line 5, in - WandbLogger.sync(fine_tune_job_id="ftjob-AncbMuC4XGCe9OOk4zjjAz5g") - File "/home/blackbird/miniconda3/envs/kios/lib/python3.10/site-packages/wandb/integration/openai/fine_tuning.py", line 148, in sync - cls._log_fine_tune( - File "/home/blackbird/miniconda3/envs/kios/lib/python3.10/site-packages/wandb/integration/openai/fine_tuning.py", line 237, in _log_fine_tune - cls._log_artifacts(fine_tune, project, entity) - File "/home/blackbird/miniconda3/envs/kios/lib/python3.10/site-packages/wandb/integration/openai/fine_tuning.py", line 344, in _log_artifacts - json.dump(dict_fine_tune, f, indent=2) - File "/home/blackbird/miniconda3/envs/kios/lib/python3.10/json/__init__.py", line 179, in dump - for chunk in iterable: - File "/home/blackbird/miniconda3/envs/kios/lib/python3.10/json/encoder.py", line 431, in _iterencode - yield from _iterencode_dict(o, _current_indent_level) - File "/home/blackbird/miniconda3/envs/kios/lib/python3.10/json/encoder.py", line 405, in _iterencode_dict - yield from chunks - File "/home/blackbird/miniconda3/envs/kios/lib/python3.10/json/encoder.py", line 438, in _iterencode - o = _default(o) - File "/home/blackbird/miniconda3/envs/kios/lib/python3.10/json/encoder.py", line 179, in default - raise TypeError(f'Object of type {o.__class__.__name__} ' -TypeError: Object of type Error is not JSON serializable \ No newline at end of file diff --git a/wandb/run-20240303_162941-ftjob-AncbMuC4XGCe9OOk4zjjAz5g/files/requirements.txt b/wandb/run-20240303_162941-ftjob-AncbMuC4XGCe9OOk4zjjAz5g/files/requirements.txt deleted file mode 100644 index 63c80899..00000000 --- a/wandb/run-20240303_162941-ftjob-AncbMuC4XGCe9OOk4zjjAz5g/files/requirements.txt +++ /dev/null @@ -1,376 +0,0 @@ -adal==1.2.7 -aenum==3.1.15 -aiohttp==3.9.1 -aiosignal==1.3.1 -alembic==1.13.1 -annotated-types==0.6.0 -ansitable==0.9.10 -anthropic==0.8.1 -antlr4-python3-runtime==4.7.2 -anyio==4.2.0 -appdirs==1.4.4 -applicationinsights==0.11.10 -argcomplete==3.2.2 -asgiref==3.7.2 -asttokens==2.0.5 -async-timeout==4.0.3 -attrs==23.2.0 -azure-ai-generative==1.0.0b2 -azure-ai-ml==1.12.1 -azure-ai-resources==1.0.0b2 -azure-common==1.1.28 -azure-core==1.29.7 -azure-graphrbac==0.61.1 -azure-identity==1.15.0 -azure-keyvault-certificates==4.7.0 -azure-keyvault-keys==4.8.0 -azure-keyvault-secrets==4.7.0 -azure-keyvault==4.2.0 -azure-mgmt-authorization==4.0.0 -azure-mgmt-containerregistry==10.3.0 -azure-mgmt-core==1.4.0 -azure-mgmt-keyvault==10.3.0 -azure-mgmt-network==25.1.0 -azure-mgmt-resource==23.0.1 -azure-mgmt-storage==21.1.0 -azure-search-documents==11.4.0b11 -azure-storage-blob==12.19.0 -azure-storage-file-datalake==12.14.0 -azure-storage-file-share==12.15.0 -azureml-core==1.54.0.post1 -azureml-dataprep-native==41.0.0 -azureml-dataprep-rslex==2.22.2 -azureml-dataprep==5.1.2 -azureml-fsspec==1.3.0 -azureml-metrics==0.0.43 -azureml-mlflow==1.54.0.post1 -azureml-telemetry==1.54.0 -backoff==2.2.1 -backports.tempfile==1.0 -backports.weakref==1.0.post1 -banal==1.0.6 -bcrypt==4.1.2 -beautifulsoup4==4.12.3 -blinker==1.7.0 -bs4==0.0.2 -build==1.0.3 -cachetools==5.3.2 -cattrs==23.2.3 -certifi==2023.11.17 -cffi==1.16.0 -cfgv==3.4.0 -chardet==5.2.0 -charset-normalizer==3.3.2 -chroma-hnswlib==0.7.3 -chromadb==0.4.22 -click==8.1.7 -cloudpickle==2.2.1 -cohere==4.46 -colorama==0.4.6 -colored==1.4.4 -coloredlogs==15.0.1 -comm==0.2.1 -contextlib2==21.6.0 -contourpy==1.2.0 -cryptography==41.0.7 -cycler==0.12.1 -databricks-cli==0.18.0 -dataclasses-json==0.6.3 -dataset==1.6.2 -datasets==2.16.1 -debugpy==1.8.1 -decorator==5.1.1 -deprecated==1.2.14 -dill==0.3.7 -dirtyjson==1.0.8 -distlib==0.3.8 -distro==1.9.0 -dnspython==2.5.0 -docker-pycreds==0.4.0 -docker==6.1.3 -emoji==2.10.0 -entrypoints==0.4 -evaluate==0.4.1 -exceptiongroup==1.2.0 -executing==0.8.3 -fairscale==0.4.13 -faiss-cpu==1.7.4 -fastapi==0.109.1 -fastavro==1.9.3 -filelock==3.13.1 -filetype==1.2.0 -fire==0.5.0 -flask==2.3.3 -flatbuffers==23.5.26 -fonttools==4.47.2 -frozenlist==1.4.1 -fsspec==2023.10.0 -gitdb==4.0.11 -gitpython==3.1.41 -google-api-core==2.15.0 -google-auth==2.27.0 -google-search-results==2.4.1 -googleapis-common-protos==1.62.0 -gradientai==1.7.0 -graphviz==0.20.1 -greenlet==3.0.3 -grpcio-tools==1.60.0 -grpcio==1.60.0 -h11==0.14.0 -httpcore==1.0.2 -httptools==0.6.1 -httpx==0.26.0 -huggingface-hub==0.20.3 -humanfriendly==10.0 -identify==2.5.33 -idna==3.6 -importlib-metadata==6.11.0 -importlib-resources==6.1.1 -iniconfig==2.0.0 -ipykernel==6.29.2 -ipython==8.20.0 -isodate==0.6.1 -itsdangerous==2.1.2 -jaraco.classes==3.3.0 -jedi==0.18.1 -jeepney==0.8.0 -jinja2==3.1.3 -jmespath==1.0.1 -joblib==1.3.2 -jsonpatch==1.33 -jsonpath-python==1.0.6 -jsonpickle==3.0.2 -jsonpointer==2.4 -jsonschema-specifications==2023.12.1 -jsonschema==4.21.1 -jupyter-client==8.6.0 -jupyter-core==5.7.1 -keyring==24.3.0 -kios-bt-planning==0.1.0 -kiwisolver==1.4.5 -knack==0.11.0 -kubernetes==29.0.0 -langchain-anthropic==0.0.1.post1 -langchain-community==0.0.20 -langchain-core==0.1.23 -langchain-experimental==0.0.51 -langchain-openai==0.0.6 -langchain==0.1.7 -langchainhub==0.1.14 -langdetect==1.0.9 -langgraph==0.0.24 -langsmith==0.0.87 -llama-index-agent-openai==0.1.1 -llama-index-core==0.10.3 -llama-index-embeddings-adapter==0.1.0 -llama-index-embeddings-openai==0.1.1 -llama-index-finetuning==0.1.0 -llama-index-legacy==0.9.48 -llama-index-llms-gradient==0.1.0 -llama-index-llms-openai==0.1.1 -llama-index-multi-modal-llms-openai==0.1.1 -llama-index-postprocessor-cohere-rerank==0.1.0 -llama-index-program-openai==0.1.1 -llama-index-question-gen-openai==0.1.1 -llama-index-readers-file==0.1.3 -llama-index==0.10.4 -lxml==5.1.0 -mako==1.3.1 -markdown==3.5.2 -markupsafe==2.1.4 -marshmallow==3.20.2 -matplotlib-inline==0.1.6 -matplotlib==3.8.3 -mlflow-skinny==2.9.2 -mmh3==4.1.0 -monotonic==1.6 -more-itertools==10.2.0 -mpmath==1.3.0 -msal-extensions==1.0.0 -msal==1.26.0 -msrest==0.7.1 -msrestazure==0.6.4 -multidict==6.0.4 -multipledispatch==1.0.0 -multiprocess==0.70.15 -mypy-extensions==1.0.0 -ndg-httpsclient==0.5.1 -neo4j==5.16.0 -nest-asyncio==1.6.0 -networkx==3.2.1 -nltk==3.8.1 -nodeenv==1.8.0 -numpy==1.26.3 -nvidia-cublas-cu12==12.1.3.1 -nvidia-cuda-cupti-cu12==12.1.105 -nvidia-cuda-nvrtc-cu12==12.1.105 -nvidia-cuda-runtime-cu12==12.1.105 -nvidia-cudnn-cu12==8.9.2.26 -nvidia-cufft-cu12==11.0.2.54 -nvidia-curand-cu12==10.3.2.106 -nvidia-cusolver-cu12==11.4.5.107 -nvidia-cusparse-cu12==12.1.0.106 -nvidia-nccl-cu12==2.18.1 -nvidia-nvjitlink-cu12==12.3.101 -nvidia-nvtx-cu12==12.1.105 -oauthlib==3.2.2 -onnxruntime==1.17.0 -openai==1.13.3 -opencensus-context==0.1.3 -opencensus-ext-azure==1.1.13 -opencensus-ext-logging==0.1.1 -opencensus==0.11.4 -opentelemetry-api==1.22.0 -opentelemetry-exporter-otlp-proto-common==1.22.0 -opentelemetry-exporter-otlp-proto-grpc==1.22.0 -opentelemetry-instrumentation-asgi==0.43b0 -opentelemetry-instrumentation-fastapi==0.43b0 -opentelemetry-instrumentation==0.43b0 -opentelemetry-proto==1.22.0 -opentelemetry-sdk==1.22.0 -opentelemetry-semantic-conventions==0.43b0 -opentelemetry-util-http==0.43b0 -overrides==7.7.0 -packaging==23.2 -pandas==2.2.0 -paramiko==3.4.0 -parso==0.8.3 -pathspec==0.12.1 -pexpect==4.8.0 -pgraph-python==0.6.2 -pillow==10.2.0 -pip==23.3.1 -pkginfo==1.9.6 -platformdirs==4.1.0 -pluggy==1.4.0 -portalocker==2.8.2 -posthog==3.3.4 -pre-commit==3.6.0 -progress==1.6 -prompt-toolkit==3.0.43 -promptflow-tools==0.1.0b15 -promptflow-vectordb==0.2.3 -promptflow==0.1.0b8 -protobuf==4.25.2 -psutil==5.9.8 -ptyprocess==0.7.0 -pulsar-client==3.4.0 -pure-eval==0.2.2 -py-trees==2.2.3 -pyarrow-hotfix==0.6 -pyarrow==13.0.0 -pyasn1-modules==0.3.0 -pyasn1==0.5.1 -pycparser==2.21 -pydantic-core==2.14.6 -pydantic==1.10.14 -pydash==6.0.2 -pydot==2.0.0 -pygments==2.15.1 -pyjwt==2.8.0 -pymongo==4.6.1 -pymupdf==1.23.22 -pymupdfb==1.23.22 -pynacl==1.5.0 -pyopenssl==23.3.0 -pyparsing==3.1.1 -pypdf==4.0.1 -pyperplan==2.1 -pypika==0.48.9 -pyproject-hooks==1.0.0 -pysocks==1.7.1 -pytamer==0.1.15 -pytest==8.0.0 -python-dateutil==2.8.2 -python-dotenv==1.0.1 -python-iso639==2024.1.2 -python-magic==0.4.27 -pytz==2023.3.post1 -pyyaml==6.0.1 -pyzmq==25.1.2 -rapidfuzz==3.6.1 -referencing==0.32.1 -regex==2023.12.25 -requests-cache==1.1.1 -requests-oauthlib==1.3.1 -requests==2.31.0 -responses==0.18.0 -roboticstoolbox-python==1.1.0 -rpds-py==0.17.1 -rsa==4.9 -rtb-data==1.0.1 -rtoml==0.9.0 -ruamel.yaml.clib==0.2.8 -ruamel.yaml==0.17.40 -safetensors==0.4.2 -scikit-learn==1.4.0 -scipy==1.12.0 -secretstorage==3.3.3 -sentence-transformers==2.4.0 -sentencepiece==0.1.99 -sentry-sdk==1.40.6 -setproctitle==1.3.3 -setuptools==68.2.2 -six==1.16.0 -smmap==5.0.1 -sniffio==1.3.0 -soupsieve==2.5 -spatialgeometry==1.1.0 -spatialmath-python==1.1.9 -sqlalchemy==2.0.27 -sqlparse==0.4.4 -stack-data==0.2.0 -starlette==0.35.1 -strictyaml==1.7.3 -swift-sim==1.1.0 -sympy==1.12 -tabulate==0.9.0 -tarski==0.8.2 -tavily-python==0.3.1 -tenacity==8.2.3 -termcolor==2.4.0 -threadpoolctl==3.2.0 -tika==2.6.0 -tiktoken==0.5.2 -tokenizers==0.15.1 -tomli==2.0.1 -torch==2.1.2 -tornado==6.4 -tqdm==4.66.1 -traitlets==5.7.1 -transformers==4.37.2 -triton==2.1.0 -typer==0.9.0 -types-requests==2.31.0.20240125 -typing-extensions==4.9.0 -typing-inspect==0.9.0 -tzdata==2023.4 -unified-planning==1.0.0 -unstructured-client==0.15.5 -unstructured==0.12.2 -up-aries==0.3.3 -up-enhsp==0.0.15 -up-fast-downward==0.2.3 -up-fmap==0.0.7 -up-lpg==0.0.7 -up-pyperplan==1.0.0 -up-symk==1.1.0 -up-tamer==1.0.0 -url-normalize==1.4.3 -urllib3==2.1.0 -uvicorn==0.27.0.post1 -uvloop==0.19.0 -virtualenv==20.25.0 -waitress==2.1.2 -wandb==0.16.3 -watchfiles==0.21.0 -wcwidth==0.2.5 -websocket-client==1.7.0 -websockets==12.0 -werkzeug==3.0.1 -wheel==0.41.2 -wikipedia==1.4.0 -wrapt==1.16.0 -xxhash==3.4.1 -yarl==1.9.4 -zipp==3.17.0 \ No newline at end of file diff --git a/wandb/run-20240303_162941-ftjob-AncbMuC4XGCe9OOk4zjjAz5g/files/wandb-metadata.json b/wandb/run-20240303_162941-ftjob-AncbMuC4XGCe9OOk4zjjAz5g/files/wandb-metadata.json deleted file mode 100644 index 9daf3032..00000000 --- a/wandb/run-20240303_162941-ftjob-AncbMuC4XGCe9OOk4zjjAz5g/files/wandb-metadata.json +++ /dev/null @@ -1,80 +0,0 @@ -{ - "os": "Linux-5.15.0-92-generic-x86_64-with-glibc2.31", - "python": "3.10.0", - "heartbeatAt": "2024-03-03T15:29:41.782342", - "startedAt": "2024-03-03T15:29:41.138867", - "docker": null, - "cuda": null, - "args": [], - "state": "running", - "program": "/home/blackbird/kios/data/finetuning/visualize.py", - "codePathLocal": "data/finetuning/visualize.py", - "codePath": "data/finetuning/visualize.py", - "git": { - "remote": "git@gitlab.lrz.de:bblab/ma/kios.git", - "commit": "bd7f44195e793e386f8713bf2d213783c079685a" - }, - "email": "jicong.ao@tum.de", - "root": "/home/blackbird/kios", - "host": "bblab-laptop", - "username": "blackbird", - "executable": "/home/blackbird/miniconda3/envs/kios/bin/python", - "cpu_count": 4, - "cpu_count_logical": 8, - "cpu_freq": { - "current": 2672.6728749999997, - "min": 400.0, - "max": 4200.0 - }, - "cpu_freq_per_core": [ - { - "current": 2400.0, - "min": 400.0, - "max": 4200.0 - }, - { - "current": 3505.852, - "min": 400.0, - "max": 4200.0 - }, - { - "current": 2400.0, - "min": 400.0, - "max": 4200.0 - }, - { - "current": 3475.531, - "min": 400.0, - "max": 4200.0 - }, - { - "current": 2400.0, - "min": 400.0, - "max": 4200.0 - }, - { - "current": 2400.0, - "min": 400.0, - "max": 4200.0 - }, - { - "current": 2400.0, - "min": 400.0, - "max": 4200.0 - }, - { - "current": 2400.0, - "min": 400.0, - "max": 4200.0 - } - ], - "disk": { - "/": { - "total": 467.8895797729492, - "used": 173.48459243774414 - } - }, - "memory": { - "total": 15.418136596679688 - } -} diff --git a/wandb/run-20240303_162941-ftjob-AncbMuC4XGCe9OOk4zjjAz5g/files/wandb-summary.json b/wandb/run-20240303_162941-ftjob-AncbMuC4XGCe9OOk4zjjAz5g/files/wandb-summary.json deleted file mode 100644 index 8f70eb2a..00000000 --- a/wandb/run-20240303_162941-ftjob-AncbMuC4XGCe9OOk4zjjAz5g/files/wandb-summary.json +++ /dev/null @@ -1 +0,0 @@ -{"train_loss": 0.04204, "train_accuracy": 0.98717, "_timestamp": 1709479788.3053255, "_runtime": 7.161970615386963, "_step": 30, "fine_tuned_model": "ft:gpt-3.5-turbo-0125:kifabrik-mirmi:kios-ut-gen-v1:8yhtwL2A", "train_data": {"_type": "table-file", "sha256": "80f7369497bd53a29f9027680db1054d83d8e29f1976df341ea17d26852d9d41", "size": 659184, "path": "media/table/train_data_30_80f7369497bd53a29f90.table.json", "ncols": 3, "nrows": 60}, "_wandb": {"runtime": 11}} \ No newline at end of file diff --git a/wandb/run-20240303_162941-ftjob-AncbMuC4XGCe9OOk4zjjAz5g/run-ftjob-AncbMuC4XGCe9OOk4zjjAz5g.wandb b/wandb/run-20240303_162941-ftjob-AncbMuC4XGCe9OOk4zjjAz5g/run-ftjob-AncbMuC4XGCe9OOk4zjjAz5g.wandb deleted file mode 100644 index 4507e659..00000000 Binary files a/wandb/run-20240303_162941-ftjob-AncbMuC4XGCe9OOk4zjjAz5g/run-ftjob-AncbMuC4XGCe9OOk4zjjAz5g.wandb and /dev/null differ diff --git a/wandb/run-20240304_141307-ftjob-AncbMuC4XGCe9OOk4zjjAz5g/files/conda-environment.yaml b/wandb/run-20240304_141307-ftjob-AncbMuC4XGCe9OOk4zjjAz5g/files/conda-environment.yaml deleted file mode 100644 index 5b7d2077..00000000 --- a/wandb/run-20240304_141307-ftjob-AncbMuC4XGCe9OOk4zjjAz5g/files/conda-environment.yaml +++ /dev/null @@ -1,400 +0,0 @@ -name: kios -channels: - - defaults -dependencies: - - _libgcc_mutex=0.1=main - - _openmp_mutex=5.1=1_gnu - - asttokens=2.0.5=pyhd3eb1b0_0 - - bzip2=1.0.8=h7b6447c_0 - - ca-certificates=2023.12.12=h06a4308_0 - - decorator=5.1.1=pyhd3eb1b0_0 - - exceptiongroup=1.2.0=py310h06a4308_0 - - executing=0.8.3=pyhd3eb1b0_0 - - ipython=8.20.0=py310h06a4308_0 - - jedi=0.18.1=py310h06a4308_1 - - ld_impl_linux-64=2.38=h1181459_1 - - libffi=3.3=he6710b0_2 - - libgcc-ng=11.2.0=h1234567_1 - - libgomp=11.2.0=h1234567_1 - - libstdcxx-ng=11.2.0=h1234567_1 - - libuuid=1.41.5=h5eee18b_0 - - matplotlib-inline=0.1.6=py310h06a4308_0 - - ncurses=6.4=h6a678d5_0 - - openssl=1.1.1w=h7f8727e_0 - - parso=0.8.3=pyhd3eb1b0_0 - - pexpect=4.8.0=pyhd3eb1b0_3 - - pip=23.3.1=py310h06a4308_0 - - prompt-toolkit=3.0.43=py310h06a4308_0 - - prompt_toolkit=3.0.43=hd3eb1b0_0 - - ptyprocess=0.7.0=pyhd3eb1b0_2 - - pure_eval=0.2.2=pyhd3eb1b0_0 - - python=3.10.0=h12debd9_5 - - readline=8.2=h5eee18b_0 - - setuptools=68.2.2=py310h06a4308_0 - - six=1.16.0=pyhd3eb1b0_1 - - sqlite=3.41.2=h5eee18b_0 - - stack_data=0.2.0=pyhd3eb1b0_0 - - tk=8.6.12=h1ccaba5_0 - - traitlets=5.7.1=py310h06a4308_0 - - wcwidth=0.2.5=pyhd3eb1b0_0 - - wheel=0.41.2=py310h06a4308_0 - - xz=5.4.5=h5eee18b_0 - - zlib=1.2.13=h5eee18b_0 - - pip: - - adal==1.2.7 - - aenum==3.1.15 - - aiohttp==3.9.1 - - aiosignal==1.3.1 - - alembic==1.13.1 - - annotated-types==0.6.0 - - ansitable==0.9.10 - - anthropic==0.8.1 - - antlr4-python3-runtime==4.7.2 - - anyio==4.2.0 - - appdirs==1.4.4 - - applicationinsights==0.11.10 - - argcomplete==3.2.2 - - asgiref==3.7.2 - - async-timeout==4.0.3 - - attrs==23.2.0 - - azure-ai-generative==1.0.0b2 - - azure-ai-ml==1.12.1 - - azure-ai-resources==1.0.0b2 - - azure-common==1.1.28 - - azure-core==1.29.7 - - azure-graphrbac==0.61.1 - - azure-identity==1.15.0 - - azure-keyvault==4.2.0 - - azure-keyvault-certificates==4.7.0 - - azure-keyvault-keys==4.8.0 - - azure-keyvault-secrets==4.7.0 - - azure-mgmt-authorization==4.0.0 - - azure-mgmt-containerregistry==10.3.0 - - azure-mgmt-core==1.4.0 - - azure-mgmt-keyvault==10.3.0 - - azure-mgmt-network==25.1.0 - - azure-mgmt-resource==23.0.1 - - azure-mgmt-storage==21.1.0 - - azure-search-documents==11.4.0b11 - - azure-storage-blob==12.19.0 - - azure-storage-file-datalake==12.14.0 - - azure-storage-file-share==12.15.0 - - azureml-core==1.54.0.post1 - - azureml-dataprep==5.1.2 - - azureml-dataprep-native==41.0.0 - - azureml-dataprep-rslex==2.22.2 - - azureml-fsspec==1.3.0 - - azureml-metrics==0.0.43 - - azureml-mlflow==1.54.0.post1 - - azureml-telemetry==1.54.0 - - backoff==2.2.1 - - backports-tempfile==1.0 - - backports-weakref==1.0.post1 - - banal==1.0.6 - - bcrypt==4.1.2 - - beautifulsoup4==4.12.3 - - blinker==1.7.0 - - bs4==0.0.2 - - build==1.0.3 - - cachetools==5.3.2 - - cattrs==23.2.3 - - certifi==2023.11.17 - - cffi==1.16.0 - - cfgv==3.4.0 - - chardet==5.2.0 - - charset-normalizer==3.3.2 - - chroma-hnswlib==0.7.3 - - chromadb==0.4.22 - - click==8.1.7 - - cloudpickle==2.2.1 - - cohere==4.46 - - colorama==0.4.6 - - colored==1.4.4 - - coloredlogs==15.0.1 - - comm==0.2.1 - - contextlib2==21.6.0 - - contourpy==1.2.0 - - cryptography==41.0.7 - - cycler==0.12.1 - - databricks-cli==0.18.0 - - dataclasses-json==0.6.3 - - dataset==1.6.2 - - datasets==2.16.1 - - debugpy==1.8.1 - - deprecated==1.2.14 - - dill==0.3.7 - - dirtyjson==1.0.8 - - distlib==0.3.8 - - distro==1.9.0 - - dnspython==2.5.0 - - docker==6.1.3 - - docker-pycreds==0.4.0 - - emoji==2.10.0 - - entrypoints==0.4 - - evaluate==0.4.1 - - fairscale==0.4.13 - - faiss-cpu==1.7.4 - - fastapi==0.109.1 - - fastavro==1.9.3 - - filelock==3.13.1 - - filetype==1.2.0 - - fire==0.5.0 - - flask==2.3.3 - - flatbuffers==23.5.26 - - fonttools==4.47.2 - - frozenlist==1.4.1 - - fsspec==2023.10.0 - - gitdb==4.0.11 - - gitpython==3.1.41 - - google-api-core==2.15.0 - - google-auth==2.27.0 - - google-search-results==2.4.1 - - googleapis-common-protos==1.62.0 - - gradientai==1.7.0 - - greenlet==3.0.3 - - grpcio==1.60.0 - - grpcio-tools==1.60.0 - - h11==0.14.0 - - httpcore==1.0.2 - - httptools==0.6.1 - - httpx==0.26.0 - - huggingface-hub==0.20.3 - - humanfriendly==10.0 - - identify==2.5.33 - - idna==3.6 - - importlib-metadata==6.11.0 - - importlib-resources==6.1.1 - - iniconfig==2.0.0 - - ipykernel==6.29.2 - - isodate==0.6.1 - - itsdangerous==2.1.2 - - jaraco-classes==3.3.0 - - jeepney==0.8.0 - - jinja2==3.1.3 - - jmespath==1.0.1 - - joblib==1.3.2 - - jsonpatch==1.33 - - jsonpath-python==1.0.6 - - jsonpickle==3.0.2 - - jsonpointer==2.4 - - jsonschema==4.21.1 - - jsonschema-specifications==2023.12.1 - - jupyter-client==8.6.0 - - jupyter-core==5.7.1 - - keyring==24.3.0 - - kiwisolver==1.4.5 - - knack==0.11.0 - - kubernetes==29.0.0 - - langchain==0.1.7 - - langchain-anthropic==0.0.1.post1 - - langchain-community==0.0.20 - - langchain-core==0.1.23 - - langchain-experimental==0.0.51 - - langchain-openai==0.0.6 - - langchainhub==0.1.14 - - langdetect==1.0.9 - - langgraph==0.0.24 - - langsmith==0.0.87 - - llama-index==0.10.4 - - llama-index-agent-openai==0.1.1 - - llama-index-core==0.10.3 - - llama-index-embeddings-adapter==0.1.0 - - llama-index-embeddings-openai==0.1.1 - - llama-index-finetuning==0.1.0 - - llama-index-legacy==0.9.48 - - llama-index-llms-gradient==0.1.0 - - llama-index-llms-openai==0.1.1 - - llama-index-multi-modal-llms-openai==0.1.1 - - llama-index-postprocessor-cohere-rerank==0.1.0 - - llama-index-program-openai==0.1.1 - - llama-index-question-gen-openai==0.1.1 - - llama-index-readers-file==0.1.3 - - lxml==5.1.0 - - mako==1.3.1 - - markdown==3.5.2 - - markupsafe==2.1.4 - - marshmallow==3.20.2 - - matplotlib==3.8.3 - - mlflow-skinny==2.9.2 - - mmh3==4.1.0 - - monotonic==1.6 - - more-itertools==10.2.0 - - mpmath==1.3.0 - - msal==1.26.0 - - msal-extensions==1.0.0 - - msrest==0.7.1 - - msrestazure==0.6.4 - - multidict==6.0.4 - - multipledispatch==1.0.0 - - multiprocess==0.70.15 - - mypy-extensions==1.0.0 - - ndg-httpsclient==0.5.1 - - neo4j==5.16.0 - - nest-asyncio==1.6.0 - - networkx==3.2.1 - - nltk==3.8.1 - - nodeenv==1.8.0 - - numpy==1.26.3 - - nvidia-cublas-cu12==12.1.3.1 - - nvidia-cuda-cupti-cu12==12.1.105 - - nvidia-cuda-nvrtc-cu12==12.1.105 - - nvidia-cuda-runtime-cu12==12.1.105 - - nvidia-cudnn-cu12==8.9.2.26 - - nvidia-cufft-cu12==11.0.2.54 - - nvidia-curand-cu12==10.3.2.106 - - nvidia-cusolver-cu12==11.4.5.107 - - nvidia-cusparse-cu12==12.1.0.106 - - nvidia-nccl-cu12==2.18.1 - - nvidia-nvjitlink-cu12==12.3.101 - - nvidia-nvtx-cu12==12.1.105 - - oauthlib==3.2.2 - - onnxruntime==1.17.0 - - openai==1.13.3 - - opencensus==0.11.4 - - opencensus-context==0.1.3 - - opencensus-ext-azure==1.1.13 - - opencensus-ext-logging==0.1.1 - - opentelemetry-api==1.22.0 - - opentelemetry-exporter-otlp-proto-common==1.22.0 - - opentelemetry-exporter-otlp-proto-grpc==1.22.0 - - opentelemetry-instrumentation==0.43b0 - - opentelemetry-instrumentation-asgi==0.43b0 - - opentelemetry-instrumentation-fastapi==0.43b0 - - opentelemetry-proto==1.22.0 - - opentelemetry-sdk==1.22.0 - - opentelemetry-semantic-conventions==0.43b0 - - opentelemetry-util-http==0.43b0 - - overrides==7.7.0 - - packaging==23.2 - - pandas==2.2.0 - - paramiko==3.4.0 - - pathspec==0.12.1 - - pgraph-python==0.6.2 - - pillow==10.2.0 - - pkginfo==1.9.6 - - platformdirs==4.1.0 - - pluggy==1.4.0 - - portalocker==2.8.2 - - posthog==3.3.4 - - pre-commit==3.6.0 - - progress==1.6 - - promptflow==0.1.0b8 - - promptflow-tools==0.1.0b15 - - promptflow-vectordb==0.2.3 - - protobuf==4.25.2 - - psutil==5.9.8 - - pulsar-client==3.4.0 - - py-trees==2.2.3 - - pyarrow==13.0.0 - - pyarrow-hotfix==0.6 - - pyasn1==0.5.1 - - pyasn1-modules==0.3.0 - - pycparser==2.21 - - pydantic==1.10.14 - - pydantic-core==2.14.6 - - pydash==6.0.2 - - pydot==2.0.0 - - pygments==2.17.2 - - pyjwt==2.8.0 - - pymongo==4.6.1 - - pymupdf==1.23.22 - - pymupdfb==1.23.22 - - pynacl==1.5.0 - - pyopenssl==23.3.0 - - pyparsing==3.1.1 - - pypdf==4.0.1 - - pyperplan==2.1 - - pypika==0.48.9 - - pyproject-hooks==1.0.0 - - pysocks==1.7.1 - - pytamer==0.1.15 - - pytest==8.0.0 - - python-dateutil==2.8.2 - - python-dotenv==1.0.1 - - python-graphviz==0.20.1 - - python-iso639==2024.1.2 - - python-magic==0.4.27 - - pytz==2023.3.post1 - - pyyaml==6.0.1 - - pyzmq==25.1.2 - - rapidfuzz==3.6.1 - - referencing==0.32.1 - - regex==2023.12.25 - - requests==2.31.0 - - requests-cache==1.1.1 - - requests-oauthlib==1.3.1 - - responses==0.18.0 - - roboticstoolbox-python==1.1.0 - - rpds-py==0.17.1 - - rsa==4.9 - - rtb-data==1.0.1 - - rtoml==0.9.0 - - ruamel-yaml==0.17.40 - - ruamel-yaml-clib==0.2.8 - - safetensors==0.4.2 - - scikit-learn==1.4.0 - - scipy==1.12.0 - - secretstorage==3.3.3 - - sentence-transformers==2.4.0 - - sentencepiece==0.1.99 - - sentry-sdk==1.40.6 - - setproctitle==1.3.3 - - smmap==5.0.1 - - sniffio==1.3.0 - - soupsieve==2.5 - - spatialgeometry==1.1.0 - - spatialmath-python==1.1.9 - - sqlalchemy==2.0.27 - - sqlparse==0.4.4 - - starlette==0.35.1 - - strictyaml==1.7.3 - - swift-sim==1.1.0 - - sympy==1.12 - - tabulate==0.9.0 - - tarski==0.8.2 - - tavily-python==0.3.1 - - tenacity==8.2.3 - - termcolor==2.4.0 - - threadpoolctl==3.2.0 - - tika==2.6.0 - - tiktoken==0.5.2 - - tokenizers==0.15.1 - - tomli==2.0.1 - - torch==2.1.2 - - tornado==6.4 - - tqdm==4.66.1 - - transformers==4.37.2 - - triton==2.1.0 - - typer==0.9.0 - - types-requests==2.31.0.20240125 - - typing-extensions==4.9.0 - - typing-inspect==0.9.0 - - tzdata==2023.4 - - unified-planning==1.0.0 - - unstructured==0.12.2 - - unstructured-client==0.15.5 - - up-aries==0.3.3 - - up-enhsp==0.0.15 - - up-fast-downward==0.2.3 - - up-fmap==0.0.7 - - up-lpg==0.0.7 - - up-pyperplan==1.0.0 - - up-symk==1.1.0 - - up-tamer==1.0.0 - - url-normalize==1.4.3 - - urllib3==2.1.0 - - uvicorn==0.27.0.post1 - - uvloop==0.19.0 - - virtualenv==20.25.0 - - waitress==2.1.2 - - wandb==0.16.3 - - watchfiles==0.21.0 - - websocket-client==1.7.0 - - websockets==12.0 - - werkzeug==3.0.1 - - wikipedia==1.4.0 - - wrapt==1.16.0 - - xxhash==3.4.1 - - yarl==1.9.4 - - zipp==3.17.0 -prefix: /home/blackbird/miniconda3/envs/kios diff --git a/wandb/run-20240304_141307-ftjob-AncbMuC4XGCe9OOk4zjjAz5g/files/config.yaml b/wandb/run-20240304_141307-ftjob-AncbMuC4XGCe9OOk4zjjAz5g/files/config.yaml deleted file mode 100644 index 479cb627..00000000 --- a/wandb/run-20240304_141307-ftjob-AncbMuC4XGCe9OOk4zjjAz5g/files/config.yaml +++ /dev/null @@ -1,79 +0,0 @@ -wandb_version: 1 - -_wandb: - desc: null - value: - python_version: 3.10.0 - cli_version: 0.16.3 - is_jupyter_run: false - is_kaggle_kernel: false - start_time: 1709557987.155776 - t: - 1: - - 55 - - 95 - 2: - - 55 - - 95 - 3: - - 13 - - 14 - - 23 - - 37 - - 58 - 4: 3.10.0 - 5: 0.16.3 - 8: - - 5 - 13: linux-x86_64 -id: - desc: null - value: ftjob-AncbMuC4XGCe9OOk4zjjAz5g -created_at: - desc: null - value: '2024-03-03 16:12:46' -error: - desc: null - value: Error(code=None, message=None, param=None, error=None) -fine_tuned_model: - desc: null - value: ft:gpt-3.5-turbo-0125:kifabrik-mirmi:kios-ut-gen-v1:8yhtwL2A -finished_at: - desc: null - value: '2024-03-03 16:18:48' -model: - desc: null - value: gpt-3.5-turbo-0125 -object: - desc: null - value: fine_tuning.job -organization_id: - desc: null - value: org-ygWwisor3PBcF2dPNRDTzzgd -result_files: - desc: null - value: file-5b2b35XQYTgpFUQglt1jjtIY -status: - desc: null - value: succeeded -trained_tokens: - desc: null - value: 500793 -training_file: - desc: null - value: file-D8aBgTaXbp3g42RcV2bTiKDK -validation_file: - desc: null - value: null -user_provided_suffix: - desc: null - value: kios_ut_gen_v1 -hyperparameters: - desc: null - value: - n_epochs: 3 - batch_size: 6 - learning_rate_multiplier: 0.1 -n_train: - desc: null - value: 60 diff --git a/wandb/run-20240304_141307-ftjob-AncbMuC4XGCe9OOk4zjjAz5g/files/output.log b/wandb/run-20240304_141307-ftjob-AncbMuC4XGCe9OOk4zjjAz5g/files/output.log deleted file mode 100644 index b629c13a..00000000 --- a/wandb/run-20240304_141307-ftjob-AncbMuC4XGCe9OOk4zjjAz5g/files/output.log +++ /dev/null @@ -1,42 +0,0 @@ -wandb: Waiting for the OpenAI fine-tuning job to be finished... -wandb: Fine-tuning finished, logging metrics, model metadata, and more to W&B -Traceback (most recent call last): - File "/home/blackbird/kios/data/finetuning/visualize.py", line 5, in - WandbLogger.sync(fine_tune_job_id="ftjob-AncbMuC4XGCe9OOk4zjjAz5g") - File "/home/blackbird/miniconda3/envs/kios/lib/python3.10/site-packages/wandb/integration/openai/fine_tuning.py", line 148, in sync - cls._log_fine_tune( - File "/home/blackbird/miniconda3/envs/kios/lib/python3.10/site-packages/wandb/integration/openai/fine_tuning.py", line 237, in _log_fine_tune - cls._log_artifacts(fine_tune, project, entity) - File "/home/blackbird/miniconda3/envs/kios/lib/python3.10/site-packages/wandb/integration/openai/fine_tuning.py", line 344, in _log_artifacts - json.dump(dict_fine_tune, f, indent=2) - File "/home/blackbird/miniconda3/envs/kios/lib/python3.10/json/__init__.py", line 179, in dump - for chunk in iterable: - File "/home/blackbird/miniconda3/envs/kios/lib/python3.10/json/encoder.py", line 431, in _iterencode - yield from _iterencode_dict(o, _current_indent_level) - File "/home/blackbird/miniconda3/envs/kios/lib/python3.10/json/encoder.py", line 405, in _iterencode_dict - yield from chunks - File "/home/blackbird/miniconda3/envs/kios/lib/python3.10/json/encoder.py", line 438, in _iterencode - o = _default(o) - File "/home/blackbird/miniconda3/envs/kios/lib/python3.10/json/encoder.py", line 179, in default - raise TypeError(f'Object of type {o.__class__.__name__} ' -TypeError: Object of type Error is not JSON serializable -Traceback (most recent call last): - File "/home/blackbird/kios/data/finetuning/visualize.py", line 5, in - WandbLogger.sync(fine_tune_job_id="ftjob-AncbMuC4XGCe9OOk4zjjAz5g") - File "/home/blackbird/miniconda3/envs/kios/lib/python3.10/site-packages/wandb/integration/openai/fine_tuning.py", line 148, in sync - cls._log_fine_tune( - File "/home/blackbird/miniconda3/envs/kios/lib/python3.10/site-packages/wandb/integration/openai/fine_tuning.py", line 237, in _log_fine_tune - cls._log_artifacts(fine_tune, project, entity) - File "/home/blackbird/miniconda3/envs/kios/lib/python3.10/site-packages/wandb/integration/openai/fine_tuning.py", line 344, in _log_artifacts - json.dump(dict_fine_tune, f, indent=2) - File "/home/blackbird/miniconda3/envs/kios/lib/python3.10/json/__init__.py", line 179, in dump - for chunk in iterable: - File "/home/blackbird/miniconda3/envs/kios/lib/python3.10/json/encoder.py", line 431, in _iterencode - yield from _iterencode_dict(o, _current_indent_level) - File "/home/blackbird/miniconda3/envs/kios/lib/python3.10/json/encoder.py", line 405, in _iterencode_dict - yield from chunks - File "/home/blackbird/miniconda3/envs/kios/lib/python3.10/json/encoder.py", line 438, in _iterencode - o = _default(o) - File "/home/blackbird/miniconda3/envs/kios/lib/python3.10/json/encoder.py", line 179, in default - raise TypeError(f'Object of type {o.__class__.__name__} ' -TypeError: Object of type Error is not JSON serializable \ No newline at end of file diff --git a/wandb/run-20240304_141307-ftjob-AncbMuC4XGCe9OOk4zjjAz5g/files/requirements.txt b/wandb/run-20240304_141307-ftjob-AncbMuC4XGCe9OOk4zjjAz5g/files/requirements.txt deleted file mode 100644 index 63c80899..00000000 --- a/wandb/run-20240304_141307-ftjob-AncbMuC4XGCe9OOk4zjjAz5g/files/requirements.txt +++ /dev/null @@ -1,376 +0,0 @@ -adal==1.2.7 -aenum==3.1.15 -aiohttp==3.9.1 -aiosignal==1.3.1 -alembic==1.13.1 -annotated-types==0.6.0 -ansitable==0.9.10 -anthropic==0.8.1 -antlr4-python3-runtime==4.7.2 -anyio==4.2.0 -appdirs==1.4.4 -applicationinsights==0.11.10 -argcomplete==3.2.2 -asgiref==3.7.2 -asttokens==2.0.5 -async-timeout==4.0.3 -attrs==23.2.0 -azure-ai-generative==1.0.0b2 -azure-ai-ml==1.12.1 -azure-ai-resources==1.0.0b2 -azure-common==1.1.28 -azure-core==1.29.7 -azure-graphrbac==0.61.1 -azure-identity==1.15.0 -azure-keyvault-certificates==4.7.0 -azure-keyvault-keys==4.8.0 -azure-keyvault-secrets==4.7.0 -azure-keyvault==4.2.0 -azure-mgmt-authorization==4.0.0 -azure-mgmt-containerregistry==10.3.0 -azure-mgmt-core==1.4.0 -azure-mgmt-keyvault==10.3.0 -azure-mgmt-network==25.1.0 -azure-mgmt-resource==23.0.1 -azure-mgmt-storage==21.1.0 -azure-search-documents==11.4.0b11 -azure-storage-blob==12.19.0 -azure-storage-file-datalake==12.14.0 -azure-storage-file-share==12.15.0 -azureml-core==1.54.0.post1 -azureml-dataprep-native==41.0.0 -azureml-dataprep-rslex==2.22.2 -azureml-dataprep==5.1.2 -azureml-fsspec==1.3.0 -azureml-metrics==0.0.43 -azureml-mlflow==1.54.0.post1 -azureml-telemetry==1.54.0 -backoff==2.2.1 -backports.tempfile==1.0 -backports.weakref==1.0.post1 -banal==1.0.6 -bcrypt==4.1.2 -beautifulsoup4==4.12.3 -blinker==1.7.0 -bs4==0.0.2 -build==1.0.3 -cachetools==5.3.2 -cattrs==23.2.3 -certifi==2023.11.17 -cffi==1.16.0 -cfgv==3.4.0 -chardet==5.2.0 -charset-normalizer==3.3.2 -chroma-hnswlib==0.7.3 -chromadb==0.4.22 -click==8.1.7 -cloudpickle==2.2.1 -cohere==4.46 -colorama==0.4.6 -colored==1.4.4 -coloredlogs==15.0.1 -comm==0.2.1 -contextlib2==21.6.0 -contourpy==1.2.0 -cryptography==41.0.7 -cycler==0.12.1 -databricks-cli==0.18.0 -dataclasses-json==0.6.3 -dataset==1.6.2 -datasets==2.16.1 -debugpy==1.8.1 -decorator==5.1.1 -deprecated==1.2.14 -dill==0.3.7 -dirtyjson==1.0.8 -distlib==0.3.8 -distro==1.9.0 -dnspython==2.5.0 -docker-pycreds==0.4.0 -docker==6.1.3 -emoji==2.10.0 -entrypoints==0.4 -evaluate==0.4.1 -exceptiongroup==1.2.0 -executing==0.8.3 -fairscale==0.4.13 -faiss-cpu==1.7.4 -fastapi==0.109.1 -fastavro==1.9.3 -filelock==3.13.1 -filetype==1.2.0 -fire==0.5.0 -flask==2.3.3 -flatbuffers==23.5.26 -fonttools==4.47.2 -frozenlist==1.4.1 -fsspec==2023.10.0 -gitdb==4.0.11 -gitpython==3.1.41 -google-api-core==2.15.0 -google-auth==2.27.0 -google-search-results==2.4.1 -googleapis-common-protos==1.62.0 -gradientai==1.7.0 -graphviz==0.20.1 -greenlet==3.0.3 -grpcio-tools==1.60.0 -grpcio==1.60.0 -h11==0.14.0 -httpcore==1.0.2 -httptools==0.6.1 -httpx==0.26.0 -huggingface-hub==0.20.3 -humanfriendly==10.0 -identify==2.5.33 -idna==3.6 -importlib-metadata==6.11.0 -importlib-resources==6.1.1 -iniconfig==2.0.0 -ipykernel==6.29.2 -ipython==8.20.0 -isodate==0.6.1 -itsdangerous==2.1.2 -jaraco.classes==3.3.0 -jedi==0.18.1 -jeepney==0.8.0 -jinja2==3.1.3 -jmespath==1.0.1 -joblib==1.3.2 -jsonpatch==1.33 -jsonpath-python==1.0.6 -jsonpickle==3.0.2 -jsonpointer==2.4 -jsonschema-specifications==2023.12.1 -jsonschema==4.21.1 -jupyter-client==8.6.0 -jupyter-core==5.7.1 -keyring==24.3.0 -kios-bt-planning==0.1.0 -kiwisolver==1.4.5 -knack==0.11.0 -kubernetes==29.0.0 -langchain-anthropic==0.0.1.post1 -langchain-community==0.0.20 -langchain-core==0.1.23 -langchain-experimental==0.0.51 -langchain-openai==0.0.6 -langchain==0.1.7 -langchainhub==0.1.14 -langdetect==1.0.9 -langgraph==0.0.24 -langsmith==0.0.87 -llama-index-agent-openai==0.1.1 -llama-index-core==0.10.3 -llama-index-embeddings-adapter==0.1.0 -llama-index-embeddings-openai==0.1.1 -llama-index-finetuning==0.1.0 -llama-index-legacy==0.9.48 -llama-index-llms-gradient==0.1.0 -llama-index-llms-openai==0.1.1 -llama-index-multi-modal-llms-openai==0.1.1 -llama-index-postprocessor-cohere-rerank==0.1.0 -llama-index-program-openai==0.1.1 -llama-index-question-gen-openai==0.1.1 -llama-index-readers-file==0.1.3 -llama-index==0.10.4 -lxml==5.1.0 -mako==1.3.1 -markdown==3.5.2 -markupsafe==2.1.4 -marshmallow==3.20.2 -matplotlib-inline==0.1.6 -matplotlib==3.8.3 -mlflow-skinny==2.9.2 -mmh3==4.1.0 -monotonic==1.6 -more-itertools==10.2.0 -mpmath==1.3.0 -msal-extensions==1.0.0 -msal==1.26.0 -msrest==0.7.1 -msrestazure==0.6.4 -multidict==6.0.4 -multipledispatch==1.0.0 -multiprocess==0.70.15 -mypy-extensions==1.0.0 -ndg-httpsclient==0.5.1 -neo4j==5.16.0 -nest-asyncio==1.6.0 -networkx==3.2.1 -nltk==3.8.1 -nodeenv==1.8.0 -numpy==1.26.3 -nvidia-cublas-cu12==12.1.3.1 -nvidia-cuda-cupti-cu12==12.1.105 -nvidia-cuda-nvrtc-cu12==12.1.105 -nvidia-cuda-runtime-cu12==12.1.105 -nvidia-cudnn-cu12==8.9.2.26 -nvidia-cufft-cu12==11.0.2.54 -nvidia-curand-cu12==10.3.2.106 -nvidia-cusolver-cu12==11.4.5.107 -nvidia-cusparse-cu12==12.1.0.106 -nvidia-nccl-cu12==2.18.1 -nvidia-nvjitlink-cu12==12.3.101 -nvidia-nvtx-cu12==12.1.105 -oauthlib==3.2.2 -onnxruntime==1.17.0 -openai==1.13.3 -opencensus-context==0.1.3 -opencensus-ext-azure==1.1.13 -opencensus-ext-logging==0.1.1 -opencensus==0.11.4 -opentelemetry-api==1.22.0 -opentelemetry-exporter-otlp-proto-common==1.22.0 -opentelemetry-exporter-otlp-proto-grpc==1.22.0 -opentelemetry-instrumentation-asgi==0.43b0 -opentelemetry-instrumentation-fastapi==0.43b0 -opentelemetry-instrumentation==0.43b0 -opentelemetry-proto==1.22.0 -opentelemetry-sdk==1.22.0 -opentelemetry-semantic-conventions==0.43b0 -opentelemetry-util-http==0.43b0 -overrides==7.7.0 -packaging==23.2 -pandas==2.2.0 -paramiko==3.4.0 -parso==0.8.3 -pathspec==0.12.1 -pexpect==4.8.0 -pgraph-python==0.6.2 -pillow==10.2.0 -pip==23.3.1 -pkginfo==1.9.6 -platformdirs==4.1.0 -pluggy==1.4.0 -portalocker==2.8.2 -posthog==3.3.4 -pre-commit==3.6.0 -progress==1.6 -prompt-toolkit==3.0.43 -promptflow-tools==0.1.0b15 -promptflow-vectordb==0.2.3 -promptflow==0.1.0b8 -protobuf==4.25.2 -psutil==5.9.8 -ptyprocess==0.7.0 -pulsar-client==3.4.0 -pure-eval==0.2.2 -py-trees==2.2.3 -pyarrow-hotfix==0.6 -pyarrow==13.0.0 -pyasn1-modules==0.3.0 -pyasn1==0.5.1 -pycparser==2.21 -pydantic-core==2.14.6 -pydantic==1.10.14 -pydash==6.0.2 -pydot==2.0.0 -pygments==2.15.1 -pyjwt==2.8.0 -pymongo==4.6.1 -pymupdf==1.23.22 -pymupdfb==1.23.22 -pynacl==1.5.0 -pyopenssl==23.3.0 -pyparsing==3.1.1 -pypdf==4.0.1 -pyperplan==2.1 -pypika==0.48.9 -pyproject-hooks==1.0.0 -pysocks==1.7.1 -pytamer==0.1.15 -pytest==8.0.0 -python-dateutil==2.8.2 -python-dotenv==1.0.1 -python-iso639==2024.1.2 -python-magic==0.4.27 -pytz==2023.3.post1 -pyyaml==6.0.1 -pyzmq==25.1.2 -rapidfuzz==3.6.1 -referencing==0.32.1 -regex==2023.12.25 -requests-cache==1.1.1 -requests-oauthlib==1.3.1 -requests==2.31.0 -responses==0.18.0 -roboticstoolbox-python==1.1.0 -rpds-py==0.17.1 -rsa==4.9 -rtb-data==1.0.1 -rtoml==0.9.0 -ruamel.yaml.clib==0.2.8 -ruamel.yaml==0.17.40 -safetensors==0.4.2 -scikit-learn==1.4.0 -scipy==1.12.0 -secretstorage==3.3.3 -sentence-transformers==2.4.0 -sentencepiece==0.1.99 -sentry-sdk==1.40.6 -setproctitle==1.3.3 -setuptools==68.2.2 -six==1.16.0 -smmap==5.0.1 -sniffio==1.3.0 -soupsieve==2.5 -spatialgeometry==1.1.0 -spatialmath-python==1.1.9 -sqlalchemy==2.0.27 -sqlparse==0.4.4 -stack-data==0.2.0 -starlette==0.35.1 -strictyaml==1.7.3 -swift-sim==1.1.0 -sympy==1.12 -tabulate==0.9.0 -tarski==0.8.2 -tavily-python==0.3.1 -tenacity==8.2.3 -termcolor==2.4.0 -threadpoolctl==3.2.0 -tika==2.6.0 -tiktoken==0.5.2 -tokenizers==0.15.1 -tomli==2.0.1 -torch==2.1.2 -tornado==6.4 -tqdm==4.66.1 -traitlets==5.7.1 -transformers==4.37.2 -triton==2.1.0 -typer==0.9.0 -types-requests==2.31.0.20240125 -typing-extensions==4.9.0 -typing-inspect==0.9.0 -tzdata==2023.4 -unified-planning==1.0.0 -unstructured-client==0.15.5 -unstructured==0.12.2 -up-aries==0.3.3 -up-enhsp==0.0.15 -up-fast-downward==0.2.3 -up-fmap==0.0.7 -up-lpg==0.0.7 -up-pyperplan==1.0.0 -up-symk==1.1.0 -up-tamer==1.0.0 -url-normalize==1.4.3 -urllib3==2.1.0 -uvicorn==0.27.0.post1 -uvloop==0.19.0 -virtualenv==20.25.0 -waitress==2.1.2 -wandb==0.16.3 -watchfiles==0.21.0 -wcwidth==0.2.5 -websocket-client==1.7.0 -websockets==12.0 -werkzeug==3.0.1 -wheel==0.41.2 -wikipedia==1.4.0 -wrapt==1.16.0 -xxhash==3.4.1 -yarl==1.9.4 -zipp==3.17.0 \ No newline at end of file diff --git a/wandb/run-20240304_141307-ftjob-AncbMuC4XGCe9OOk4zjjAz5g/files/wandb-metadata.json b/wandb/run-20240304_141307-ftjob-AncbMuC4XGCe9OOk4zjjAz5g/files/wandb-metadata.json deleted file mode 100644 index 5a79b753..00000000 --- a/wandb/run-20240304_141307-ftjob-AncbMuC4XGCe9OOk4zjjAz5g/files/wandb-metadata.json +++ /dev/null @@ -1,80 +0,0 @@ -{ - "os": "Linux-5.15.0-92-generic-x86_64-with-glibc2.31", - "python": "3.10.0", - "heartbeatAt": "2024-03-04T13:13:07.700363", - "startedAt": "2024-03-04T13:13:07.152586", - "docker": null, - "cuda": null, - "args": [], - "state": "running", - "program": "/home/blackbird/kios/data/finetuning/visualize.py", - "codePathLocal": "data/finetuning/visualize.py", - "codePath": "data/finetuning/visualize.py", - "git": { - "remote": "git@gitlab.lrz.de:bblab/ma/kios.git", - "commit": "bd7f44195e793e386f8713bf2d213783c079685a" - }, - "email": "jicong.ao@tum.de", - "root": "/home/blackbird/kios", - "host": "bblab-laptop", - "username": "blackbird", - "executable": "/home/blackbird/miniconda3/envs/kios/bin/python", - "cpu_count": 4, - "cpu_count_logical": 8, - "cpu_freq": { - "current": 2505.405125, - "min": 400.0, - "max": 4200.0 - }, - "cpu_freq_per_core": [ - { - "current": 2400.0, - "min": 400.0, - "max": 4200.0 - }, - { - "current": 1963.008, - "min": 400.0, - "max": 4200.0 - }, - { - "current": 2400.0, - "min": 400.0, - "max": 4200.0 - }, - { - "current": 2400.0, - "min": 400.0, - "max": 4200.0 - }, - { - "current": 2400.0, - "min": 400.0, - "max": 4200.0 - }, - { - "current": 2400.0, - "min": 400.0, - "max": 4200.0 - }, - { - "current": 2400.0, - "min": 400.0, - "max": 4200.0 - }, - { - "current": 2400.0, - "min": 400.0, - "max": 4200.0 - } - ], - "disk": { - "/": { - "total": 467.8895797729492, - "used": 173.5617332458496 - } - }, - "memory": { - "total": 15.418136596679688 - } -} diff --git a/wandb/run-20240304_141307-ftjob-AncbMuC4XGCe9OOk4zjjAz5g/files/wandb-summary.json b/wandb/run-20240304_141307-ftjob-AncbMuC4XGCe9OOk4zjjAz5g/files/wandb-summary.json deleted file mode 100644 index e968c85d..00000000 --- a/wandb/run-20240304_141307-ftjob-AncbMuC4XGCe9OOk4zjjAz5g/files/wandb-summary.json +++ /dev/null @@ -1 +0,0 @@ -{"train_loss": 0.04204, "train_accuracy": 0.98717, "_timestamp": 1709557991.884281, "_runtime": 4.72850489616394, "_step": 30, "fine_tuned_model": "ft:gpt-3.5-turbo-0125:kifabrik-mirmi:kios-ut-gen-v1:8yhtwL2A", "_wandb": {"runtime": 4}} \ No newline at end of file diff --git a/wandb/run-20240304_141307-ftjob-AncbMuC4XGCe9OOk4zjjAz5g/run-ftjob-AncbMuC4XGCe9OOk4zjjAz5g.wandb b/wandb/run-20240304_141307-ftjob-AncbMuC4XGCe9OOk4zjjAz5g/run-ftjob-AncbMuC4XGCe9OOk4zjjAz5g.wandb deleted file mode 100644 index e12971af..00000000 Binary files a/wandb/run-20240304_141307-ftjob-AncbMuC4XGCe9OOk4zjjAz5g/run-ftjob-AncbMuC4XGCe9OOk4zjjAz5g.wandb and /dev/null differ diff --git a/wandb/run-20240304_141453-ftjob-kdM1O1dj6DYbYcykXXKyikqt/files/conda-environment.yaml b/wandb/run-20240304_141453-ftjob-kdM1O1dj6DYbYcykXXKyikqt/files/conda-environment.yaml deleted file mode 100644 index 5b7d2077..00000000 --- a/wandb/run-20240304_141453-ftjob-kdM1O1dj6DYbYcykXXKyikqt/files/conda-environment.yaml +++ /dev/null @@ -1,400 +0,0 @@ -name: kios -channels: - - defaults -dependencies: - - _libgcc_mutex=0.1=main - - _openmp_mutex=5.1=1_gnu - - asttokens=2.0.5=pyhd3eb1b0_0 - - bzip2=1.0.8=h7b6447c_0 - - ca-certificates=2023.12.12=h06a4308_0 - - decorator=5.1.1=pyhd3eb1b0_0 - - exceptiongroup=1.2.0=py310h06a4308_0 - - executing=0.8.3=pyhd3eb1b0_0 - - ipython=8.20.0=py310h06a4308_0 - - jedi=0.18.1=py310h06a4308_1 - - ld_impl_linux-64=2.38=h1181459_1 - - libffi=3.3=he6710b0_2 - - libgcc-ng=11.2.0=h1234567_1 - - libgomp=11.2.0=h1234567_1 - - libstdcxx-ng=11.2.0=h1234567_1 - - libuuid=1.41.5=h5eee18b_0 - - matplotlib-inline=0.1.6=py310h06a4308_0 - - ncurses=6.4=h6a678d5_0 - - openssl=1.1.1w=h7f8727e_0 - - parso=0.8.3=pyhd3eb1b0_0 - - pexpect=4.8.0=pyhd3eb1b0_3 - - pip=23.3.1=py310h06a4308_0 - - prompt-toolkit=3.0.43=py310h06a4308_0 - - prompt_toolkit=3.0.43=hd3eb1b0_0 - - ptyprocess=0.7.0=pyhd3eb1b0_2 - - pure_eval=0.2.2=pyhd3eb1b0_0 - - python=3.10.0=h12debd9_5 - - readline=8.2=h5eee18b_0 - - setuptools=68.2.2=py310h06a4308_0 - - six=1.16.0=pyhd3eb1b0_1 - - sqlite=3.41.2=h5eee18b_0 - - stack_data=0.2.0=pyhd3eb1b0_0 - - tk=8.6.12=h1ccaba5_0 - - traitlets=5.7.1=py310h06a4308_0 - - wcwidth=0.2.5=pyhd3eb1b0_0 - - wheel=0.41.2=py310h06a4308_0 - - xz=5.4.5=h5eee18b_0 - - zlib=1.2.13=h5eee18b_0 - - pip: - - adal==1.2.7 - - aenum==3.1.15 - - aiohttp==3.9.1 - - aiosignal==1.3.1 - - alembic==1.13.1 - - annotated-types==0.6.0 - - ansitable==0.9.10 - - anthropic==0.8.1 - - antlr4-python3-runtime==4.7.2 - - anyio==4.2.0 - - appdirs==1.4.4 - - applicationinsights==0.11.10 - - argcomplete==3.2.2 - - asgiref==3.7.2 - - async-timeout==4.0.3 - - attrs==23.2.0 - - azure-ai-generative==1.0.0b2 - - azure-ai-ml==1.12.1 - - azure-ai-resources==1.0.0b2 - - azure-common==1.1.28 - - azure-core==1.29.7 - - azure-graphrbac==0.61.1 - - azure-identity==1.15.0 - - azure-keyvault==4.2.0 - - azure-keyvault-certificates==4.7.0 - - azure-keyvault-keys==4.8.0 - - azure-keyvault-secrets==4.7.0 - - azure-mgmt-authorization==4.0.0 - - azure-mgmt-containerregistry==10.3.0 - - azure-mgmt-core==1.4.0 - - azure-mgmt-keyvault==10.3.0 - - azure-mgmt-network==25.1.0 - - azure-mgmt-resource==23.0.1 - - azure-mgmt-storage==21.1.0 - - azure-search-documents==11.4.0b11 - - azure-storage-blob==12.19.0 - - azure-storage-file-datalake==12.14.0 - - azure-storage-file-share==12.15.0 - - azureml-core==1.54.0.post1 - - azureml-dataprep==5.1.2 - - azureml-dataprep-native==41.0.0 - - azureml-dataprep-rslex==2.22.2 - - azureml-fsspec==1.3.0 - - azureml-metrics==0.0.43 - - azureml-mlflow==1.54.0.post1 - - azureml-telemetry==1.54.0 - - backoff==2.2.1 - - backports-tempfile==1.0 - - backports-weakref==1.0.post1 - - banal==1.0.6 - - bcrypt==4.1.2 - - beautifulsoup4==4.12.3 - - blinker==1.7.0 - - bs4==0.0.2 - - build==1.0.3 - - cachetools==5.3.2 - - cattrs==23.2.3 - - certifi==2023.11.17 - - cffi==1.16.0 - - cfgv==3.4.0 - - chardet==5.2.0 - - charset-normalizer==3.3.2 - - chroma-hnswlib==0.7.3 - - chromadb==0.4.22 - - click==8.1.7 - - cloudpickle==2.2.1 - - cohere==4.46 - - colorama==0.4.6 - - colored==1.4.4 - - coloredlogs==15.0.1 - - comm==0.2.1 - - contextlib2==21.6.0 - - contourpy==1.2.0 - - cryptography==41.0.7 - - cycler==0.12.1 - - databricks-cli==0.18.0 - - dataclasses-json==0.6.3 - - dataset==1.6.2 - - datasets==2.16.1 - - debugpy==1.8.1 - - deprecated==1.2.14 - - dill==0.3.7 - - dirtyjson==1.0.8 - - distlib==0.3.8 - - distro==1.9.0 - - dnspython==2.5.0 - - docker==6.1.3 - - docker-pycreds==0.4.0 - - emoji==2.10.0 - - entrypoints==0.4 - - evaluate==0.4.1 - - fairscale==0.4.13 - - faiss-cpu==1.7.4 - - fastapi==0.109.1 - - fastavro==1.9.3 - - filelock==3.13.1 - - filetype==1.2.0 - - fire==0.5.0 - - flask==2.3.3 - - flatbuffers==23.5.26 - - fonttools==4.47.2 - - frozenlist==1.4.1 - - fsspec==2023.10.0 - - gitdb==4.0.11 - - gitpython==3.1.41 - - google-api-core==2.15.0 - - google-auth==2.27.0 - - google-search-results==2.4.1 - - googleapis-common-protos==1.62.0 - - gradientai==1.7.0 - - greenlet==3.0.3 - - grpcio==1.60.0 - - grpcio-tools==1.60.0 - - h11==0.14.0 - - httpcore==1.0.2 - - httptools==0.6.1 - - httpx==0.26.0 - - huggingface-hub==0.20.3 - - humanfriendly==10.0 - - identify==2.5.33 - - idna==3.6 - - importlib-metadata==6.11.0 - - importlib-resources==6.1.1 - - iniconfig==2.0.0 - - ipykernel==6.29.2 - - isodate==0.6.1 - - itsdangerous==2.1.2 - - jaraco-classes==3.3.0 - - jeepney==0.8.0 - - jinja2==3.1.3 - - jmespath==1.0.1 - - joblib==1.3.2 - - jsonpatch==1.33 - - jsonpath-python==1.0.6 - - jsonpickle==3.0.2 - - jsonpointer==2.4 - - jsonschema==4.21.1 - - jsonschema-specifications==2023.12.1 - - jupyter-client==8.6.0 - - jupyter-core==5.7.1 - - keyring==24.3.0 - - kiwisolver==1.4.5 - - knack==0.11.0 - - kubernetes==29.0.0 - - langchain==0.1.7 - - langchain-anthropic==0.0.1.post1 - - langchain-community==0.0.20 - - langchain-core==0.1.23 - - langchain-experimental==0.0.51 - - langchain-openai==0.0.6 - - langchainhub==0.1.14 - - langdetect==1.0.9 - - langgraph==0.0.24 - - langsmith==0.0.87 - - llama-index==0.10.4 - - llama-index-agent-openai==0.1.1 - - llama-index-core==0.10.3 - - llama-index-embeddings-adapter==0.1.0 - - llama-index-embeddings-openai==0.1.1 - - llama-index-finetuning==0.1.0 - - llama-index-legacy==0.9.48 - - llama-index-llms-gradient==0.1.0 - - llama-index-llms-openai==0.1.1 - - llama-index-multi-modal-llms-openai==0.1.1 - - llama-index-postprocessor-cohere-rerank==0.1.0 - - llama-index-program-openai==0.1.1 - - llama-index-question-gen-openai==0.1.1 - - llama-index-readers-file==0.1.3 - - lxml==5.1.0 - - mako==1.3.1 - - markdown==3.5.2 - - markupsafe==2.1.4 - - marshmallow==3.20.2 - - matplotlib==3.8.3 - - mlflow-skinny==2.9.2 - - mmh3==4.1.0 - - monotonic==1.6 - - more-itertools==10.2.0 - - mpmath==1.3.0 - - msal==1.26.0 - - msal-extensions==1.0.0 - - msrest==0.7.1 - - msrestazure==0.6.4 - - multidict==6.0.4 - - multipledispatch==1.0.0 - - multiprocess==0.70.15 - - mypy-extensions==1.0.0 - - ndg-httpsclient==0.5.1 - - neo4j==5.16.0 - - nest-asyncio==1.6.0 - - networkx==3.2.1 - - nltk==3.8.1 - - nodeenv==1.8.0 - - numpy==1.26.3 - - nvidia-cublas-cu12==12.1.3.1 - - nvidia-cuda-cupti-cu12==12.1.105 - - nvidia-cuda-nvrtc-cu12==12.1.105 - - nvidia-cuda-runtime-cu12==12.1.105 - - nvidia-cudnn-cu12==8.9.2.26 - - nvidia-cufft-cu12==11.0.2.54 - - nvidia-curand-cu12==10.3.2.106 - - nvidia-cusolver-cu12==11.4.5.107 - - nvidia-cusparse-cu12==12.1.0.106 - - nvidia-nccl-cu12==2.18.1 - - nvidia-nvjitlink-cu12==12.3.101 - - nvidia-nvtx-cu12==12.1.105 - - oauthlib==3.2.2 - - onnxruntime==1.17.0 - - openai==1.13.3 - - opencensus==0.11.4 - - opencensus-context==0.1.3 - - opencensus-ext-azure==1.1.13 - - opencensus-ext-logging==0.1.1 - - opentelemetry-api==1.22.0 - - opentelemetry-exporter-otlp-proto-common==1.22.0 - - opentelemetry-exporter-otlp-proto-grpc==1.22.0 - - opentelemetry-instrumentation==0.43b0 - - opentelemetry-instrumentation-asgi==0.43b0 - - opentelemetry-instrumentation-fastapi==0.43b0 - - opentelemetry-proto==1.22.0 - - opentelemetry-sdk==1.22.0 - - opentelemetry-semantic-conventions==0.43b0 - - opentelemetry-util-http==0.43b0 - - overrides==7.7.0 - - packaging==23.2 - - pandas==2.2.0 - - paramiko==3.4.0 - - pathspec==0.12.1 - - pgraph-python==0.6.2 - - pillow==10.2.0 - - pkginfo==1.9.6 - - platformdirs==4.1.0 - - pluggy==1.4.0 - - portalocker==2.8.2 - - posthog==3.3.4 - - pre-commit==3.6.0 - - progress==1.6 - - promptflow==0.1.0b8 - - promptflow-tools==0.1.0b15 - - promptflow-vectordb==0.2.3 - - protobuf==4.25.2 - - psutil==5.9.8 - - pulsar-client==3.4.0 - - py-trees==2.2.3 - - pyarrow==13.0.0 - - pyarrow-hotfix==0.6 - - pyasn1==0.5.1 - - pyasn1-modules==0.3.0 - - pycparser==2.21 - - pydantic==1.10.14 - - pydantic-core==2.14.6 - - pydash==6.0.2 - - pydot==2.0.0 - - pygments==2.17.2 - - pyjwt==2.8.0 - - pymongo==4.6.1 - - pymupdf==1.23.22 - - pymupdfb==1.23.22 - - pynacl==1.5.0 - - pyopenssl==23.3.0 - - pyparsing==3.1.1 - - pypdf==4.0.1 - - pyperplan==2.1 - - pypika==0.48.9 - - pyproject-hooks==1.0.0 - - pysocks==1.7.1 - - pytamer==0.1.15 - - pytest==8.0.0 - - python-dateutil==2.8.2 - - python-dotenv==1.0.1 - - python-graphviz==0.20.1 - - python-iso639==2024.1.2 - - python-magic==0.4.27 - - pytz==2023.3.post1 - - pyyaml==6.0.1 - - pyzmq==25.1.2 - - rapidfuzz==3.6.1 - - referencing==0.32.1 - - regex==2023.12.25 - - requests==2.31.0 - - requests-cache==1.1.1 - - requests-oauthlib==1.3.1 - - responses==0.18.0 - - roboticstoolbox-python==1.1.0 - - rpds-py==0.17.1 - - rsa==4.9 - - rtb-data==1.0.1 - - rtoml==0.9.0 - - ruamel-yaml==0.17.40 - - ruamel-yaml-clib==0.2.8 - - safetensors==0.4.2 - - scikit-learn==1.4.0 - - scipy==1.12.0 - - secretstorage==3.3.3 - - sentence-transformers==2.4.0 - - sentencepiece==0.1.99 - - sentry-sdk==1.40.6 - - setproctitle==1.3.3 - - smmap==5.0.1 - - sniffio==1.3.0 - - soupsieve==2.5 - - spatialgeometry==1.1.0 - - spatialmath-python==1.1.9 - - sqlalchemy==2.0.27 - - sqlparse==0.4.4 - - starlette==0.35.1 - - strictyaml==1.7.3 - - swift-sim==1.1.0 - - sympy==1.12 - - tabulate==0.9.0 - - tarski==0.8.2 - - tavily-python==0.3.1 - - tenacity==8.2.3 - - termcolor==2.4.0 - - threadpoolctl==3.2.0 - - tika==2.6.0 - - tiktoken==0.5.2 - - tokenizers==0.15.1 - - tomli==2.0.1 - - torch==2.1.2 - - tornado==6.4 - - tqdm==4.66.1 - - transformers==4.37.2 - - triton==2.1.0 - - typer==0.9.0 - - types-requests==2.31.0.20240125 - - typing-extensions==4.9.0 - - typing-inspect==0.9.0 - - tzdata==2023.4 - - unified-planning==1.0.0 - - unstructured==0.12.2 - - unstructured-client==0.15.5 - - up-aries==0.3.3 - - up-enhsp==0.0.15 - - up-fast-downward==0.2.3 - - up-fmap==0.0.7 - - up-lpg==0.0.7 - - up-pyperplan==1.0.0 - - up-symk==1.1.0 - - up-tamer==1.0.0 - - url-normalize==1.4.3 - - urllib3==2.1.0 - - uvicorn==0.27.0.post1 - - uvloop==0.19.0 - - virtualenv==20.25.0 - - waitress==2.1.2 - - wandb==0.16.3 - - watchfiles==0.21.0 - - websocket-client==1.7.0 - - websockets==12.0 - - werkzeug==3.0.1 - - wikipedia==1.4.0 - - wrapt==1.16.0 - - xxhash==3.4.1 - - yarl==1.9.4 - - zipp==3.17.0 -prefix: /home/blackbird/miniconda3/envs/kios diff --git a/wandb/run-20240304_141453-ftjob-kdM1O1dj6DYbYcykXXKyikqt/files/config.yaml b/wandb/run-20240304_141453-ftjob-kdM1O1dj6DYbYcykXXKyikqt/files/config.yaml deleted file mode 100644 index 85e49047..00000000 --- a/wandb/run-20240304_141453-ftjob-kdM1O1dj6DYbYcykXXKyikqt/files/config.yaml +++ /dev/null @@ -1,78 +0,0 @@ -wandb_version: 1 - -_wandb: - desc: null - value: - python_version: 3.10.0 - cli_version: 0.16.3 - is_jupyter_run: false - is_kaggle_kernel: false - start_time: 1709558093.756392 - t: - 1: - - 55 - - 95 - 2: - - 55 - - 95 - 3: - - 13 - - 14 - - 23 - - 58 - 4: 3.10.0 - 5: 0.16.3 - 8: - - 5 - 13: linux-x86_64 -id: - desc: null - value: ftjob-kdM1O1dj6DYbYcykXXKyikqt -created_at: - desc: null - value: '2024-03-04 14:03:20' -error: - desc: null - value: Error(code=None, message=None, param=None, error=None) -fine_tuned_model: - desc: null - value: ft:gpt-3.5-turbo-0125:kifabrik-mirmi:kios-ut-gen-v2:8z2KbPsr -finished_at: - desc: null - value: '2024-03-04 14:07:40' -model: - desc: null - value: ft:gpt-3.5-turbo-0125:kifabrik-mirmi:kios-ut-gen-v1:8yhtwL2A -object: - desc: null - value: fine_tuning.job -organization_id: - desc: null - value: org-ygWwisor3PBcF2dPNRDTzzgd -result_files: - desc: null - value: file-tZbs1XUN3ch21P5hGoXSvyCX -status: - desc: null - value: succeeded -trained_tokens: - desc: null - value: 333862 -training_file: - desc: null - value: file-D8aBgTaXbp3g42RcV2bTiKDK -validation_file: - desc: null - value: null -user_provided_suffix: - desc: null - value: kios_ut_gen_v2 -hyperparameters: - desc: null - value: - n_epochs: 2 - batch_size: 6 - learning_rate_multiplier: 0.25 -n_train: - desc: null - value: 60 diff --git a/wandb/run-20240304_141453-ftjob-kdM1O1dj6DYbYcykXXKyikqt/files/output.log b/wandb/run-20240304_141453-ftjob-kdM1O1dj6DYbYcykXXKyikqt/files/output.log deleted file mode 100644 index c177484c..00000000 --- a/wandb/run-20240304_141453-ftjob-kdM1O1dj6DYbYcykXXKyikqt/files/output.log +++ /dev/null @@ -1,42 +0,0 @@ -wandb: Waiting for the OpenAI fine-tuning job to be finished... -wandb: Fine-tuning finished, logging metrics, model metadata, and more to W&B -Traceback (most recent call last): - File "/home/blackbird/kios/data/finetuning/visualize.py", line 5, in - WandbLogger.sync(fine_tune_job_id="ftjob-kdM1O1dj6DYbYcykXXKyikqt") - File "/home/blackbird/miniconda3/envs/kios/lib/python3.10/site-packages/wandb/integration/openai/fine_tuning.py", line 148, in sync - cls._log_fine_tune( - File "/home/blackbird/miniconda3/envs/kios/lib/python3.10/site-packages/wandb/integration/openai/fine_tuning.py", line 237, in _log_fine_tune - cls._log_artifacts(fine_tune, project, entity) - File "/home/blackbird/miniconda3/envs/kios/lib/python3.10/site-packages/wandb/integration/openai/fine_tuning.py", line 344, in _log_artifacts - json.dump(dict_fine_tune, f, indent=2) - File "/home/blackbird/miniconda3/envs/kios/lib/python3.10/json/__init__.py", line 179, in dump - for chunk in iterable: - File "/home/blackbird/miniconda3/envs/kios/lib/python3.10/json/encoder.py", line 431, in _iterencode - yield from _iterencode_dict(o, _current_indent_level) - File "/home/blackbird/miniconda3/envs/kios/lib/python3.10/json/encoder.py", line 405, in _iterencode_dict - yield from chunks - File "/home/blackbird/miniconda3/envs/kios/lib/python3.10/json/encoder.py", line 438, in _iterencode - o = _default(o) - File "/home/blackbird/miniconda3/envs/kios/lib/python3.10/json/encoder.py", line 179, in default - raise TypeError(f'Object of type {o.__class__.__name__} ' -TypeError: Object of type Error is not JSON serializable -Traceback (most recent call last): - File "/home/blackbird/kios/data/finetuning/visualize.py", line 5, in - WandbLogger.sync(fine_tune_job_id="ftjob-kdM1O1dj6DYbYcykXXKyikqt") - File "/home/blackbird/miniconda3/envs/kios/lib/python3.10/site-packages/wandb/integration/openai/fine_tuning.py", line 148, in sync - cls._log_fine_tune( - File "/home/blackbird/miniconda3/envs/kios/lib/python3.10/site-packages/wandb/integration/openai/fine_tuning.py", line 237, in _log_fine_tune - cls._log_artifacts(fine_tune, project, entity) - File "/home/blackbird/miniconda3/envs/kios/lib/python3.10/site-packages/wandb/integration/openai/fine_tuning.py", line 344, in _log_artifacts - json.dump(dict_fine_tune, f, indent=2) - File "/home/blackbird/miniconda3/envs/kios/lib/python3.10/json/__init__.py", line 179, in dump - for chunk in iterable: - File "/home/blackbird/miniconda3/envs/kios/lib/python3.10/json/encoder.py", line 431, in _iterencode - yield from _iterencode_dict(o, _current_indent_level) - File "/home/blackbird/miniconda3/envs/kios/lib/python3.10/json/encoder.py", line 405, in _iterencode_dict - yield from chunks - File "/home/blackbird/miniconda3/envs/kios/lib/python3.10/json/encoder.py", line 438, in _iterencode - o = _default(o) - File "/home/blackbird/miniconda3/envs/kios/lib/python3.10/json/encoder.py", line 179, in default - raise TypeError(f'Object of type {o.__class__.__name__} ' -TypeError: Object of type Error is not JSON serializable \ No newline at end of file diff --git a/wandb/run-20240304_141453-ftjob-kdM1O1dj6DYbYcykXXKyikqt/files/requirements.txt b/wandb/run-20240304_141453-ftjob-kdM1O1dj6DYbYcykXXKyikqt/files/requirements.txt deleted file mode 100644 index 63c80899..00000000 --- a/wandb/run-20240304_141453-ftjob-kdM1O1dj6DYbYcykXXKyikqt/files/requirements.txt +++ /dev/null @@ -1,376 +0,0 @@ -adal==1.2.7 -aenum==3.1.15 -aiohttp==3.9.1 -aiosignal==1.3.1 -alembic==1.13.1 -annotated-types==0.6.0 -ansitable==0.9.10 -anthropic==0.8.1 -antlr4-python3-runtime==4.7.2 -anyio==4.2.0 -appdirs==1.4.4 -applicationinsights==0.11.10 -argcomplete==3.2.2 -asgiref==3.7.2 -asttokens==2.0.5 -async-timeout==4.0.3 -attrs==23.2.0 -azure-ai-generative==1.0.0b2 -azure-ai-ml==1.12.1 -azure-ai-resources==1.0.0b2 -azure-common==1.1.28 -azure-core==1.29.7 -azure-graphrbac==0.61.1 -azure-identity==1.15.0 -azure-keyvault-certificates==4.7.0 -azure-keyvault-keys==4.8.0 -azure-keyvault-secrets==4.7.0 -azure-keyvault==4.2.0 -azure-mgmt-authorization==4.0.0 -azure-mgmt-containerregistry==10.3.0 -azure-mgmt-core==1.4.0 -azure-mgmt-keyvault==10.3.0 -azure-mgmt-network==25.1.0 -azure-mgmt-resource==23.0.1 -azure-mgmt-storage==21.1.0 -azure-search-documents==11.4.0b11 -azure-storage-blob==12.19.0 -azure-storage-file-datalake==12.14.0 -azure-storage-file-share==12.15.0 -azureml-core==1.54.0.post1 -azureml-dataprep-native==41.0.0 -azureml-dataprep-rslex==2.22.2 -azureml-dataprep==5.1.2 -azureml-fsspec==1.3.0 -azureml-metrics==0.0.43 -azureml-mlflow==1.54.0.post1 -azureml-telemetry==1.54.0 -backoff==2.2.1 -backports.tempfile==1.0 -backports.weakref==1.0.post1 -banal==1.0.6 -bcrypt==4.1.2 -beautifulsoup4==4.12.3 -blinker==1.7.0 -bs4==0.0.2 -build==1.0.3 -cachetools==5.3.2 -cattrs==23.2.3 -certifi==2023.11.17 -cffi==1.16.0 -cfgv==3.4.0 -chardet==5.2.0 -charset-normalizer==3.3.2 -chroma-hnswlib==0.7.3 -chromadb==0.4.22 -click==8.1.7 -cloudpickle==2.2.1 -cohere==4.46 -colorama==0.4.6 -colored==1.4.4 -coloredlogs==15.0.1 -comm==0.2.1 -contextlib2==21.6.0 -contourpy==1.2.0 -cryptography==41.0.7 -cycler==0.12.1 -databricks-cli==0.18.0 -dataclasses-json==0.6.3 -dataset==1.6.2 -datasets==2.16.1 -debugpy==1.8.1 -decorator==5.1.1 -deprecated==1.2.14 -dill==0.3.7 -dirtyjson==1.0.8 -distlib==0.3.8 -distro==1.9.0 -dnspython==2.5.0 -docker-pycreds==0.4.0 -docker==6.1.3 -emoji==2.10.0 -entrypoints==0.4 -evaluate==0.4.1 -exceptiongroup==1.2.0 -executing==0.8.3 -fairscale==0.4.13 -faiss-cpu==1.7.4 -fastapi==0.109.1 -fastavro==1.9.3 -filelock==3.13.1 -filetype==1.2.0 -fire==0.5.0 -flask==2.3.3 -flatbuffers==23.5.26 -fonttools==4.47.2 -frozenlist==1.4.1 -fsspec==2023.10.0 -gitdb==4.0.11 -gitpython==3.1.41 -google-api-core==2.15.0 -google-auth==2.27.0 -google-search-results==2.4.1 -googleapis-common-protos==1.62.0 -gradientai==1.7.0 -graphviz==0.20.1 -greenlet==3.0.3 -grpcio-tools==1.60.0 -grpcio==1.60.0 -h11==0.14.0 -httpcore==1.0.2 -httptools==0.6.1 -httpx==0.26.0 -huggingface-hub==0.20.3 -humanfriendly==10.0 -identify==2.5.33 -idna==3.6 -importlib-metadata==6.11.0 -importlib-resources==6.1.1 -iniconfig==2.0.0 -ipykernel==6.29.2 -ipython==8.20.0 -isodate==0.6.1 -itsdangerous==2.1.2 -jaraco.classes==3.3.0 -jedi==0.18.1 -jeepney==0.8.0 -jinja2==3.1.3 -jmespath==1.0.1 -joblib==1.3.2 -jsonpatch==1.33 -jsonpath-python==1.0.6 -jsonpickle==3.0.2 -jsonpointer==2.4 -jsonschema-specifications==2023.12.1 -jsonschema==4.21.1 -jupyter-client==8.6.0 -jupyter-core==5.7.1 -keyring==24.3.0 -kios-bt-planning==0.1.0 -kiwisolver==1.4.5 -knack==0.11.0 -kubernetes==29.0.0 -langchain-anthropic==0.0.1.post1 -langchain-community==0.0.20 -langchain-core==0.1.23 -langchain-experimental==0.0.51 -langchain-openai==0.0.6 -langchain==0.1.7 -langchainhub==0.1.14 -langdetect==1.0.9 -langgraph==0.0.24 -langsmith==0.0.87 -llama-index-agent-openai==0.1.1 -llama-index-core==0.10.3 -llama-index-embeddings-adapter==0.1.0 -llama-index-embeddings-openai==0.1.1 -llama-index-finetuning==0.1.0 -llama-index-legacy==0.9.48 -llama-index-llms-gradient==0.1.0 -llama-index-llms-openai==0.1.1 -llama-index-multi-modal-llms-openai==0.1.1 -llama-index-postprocessor-cohere-rerank==0.1.0 -llama-index-program-openai==0.1.1 -llama-index-question-gen-openai==0.1.1 -llama-index-readers-file==0.1.3 -llama-index==0.10.4 -lxml==5.1.0 -mako==1.3.1 -markdown==3.5.2 -markupsafe==2.1.4 -marshmallow==3.20.2 -matplotlib-inline==0.1.6 -matplotlib==3.8.3 -mlflow-skinny==2.9.2 -mmh3==4.1.0 -monotonic==1.6 -more-itertools==10.2.0 -mpmath==1.3.0 -msal-extensions==1.0.0 -msal==1.26.0 -msrest==0.7.1 -msrestazure==0.6.4 -multidict==6.0.4 -multipledispatch==1.0.0 -multiprocess==0.70.15 -mypy-extensions==1.0.0 -ndg-httpsclient==0.5.1 -neo4j==5.16.0 -nest-asyncio==1.6.0 -networkx==3.2.1 -nltk==3.8.1 -nodeenv==1.8.0 -numpy==1.26.3 -nvidia-cublas-cu12==12.1.3.1 -nvidia-cuda-cupti-cu12==12.1.105 -nvidia-cuda-nvrtc-cu12==12.1.105 -nvidia-cuda-runtime-cu12==12.1.105 -nvidia-cudnn-cu12==8.9.2.26 -nvidia-cufft-cu12==11.0.2.54 -nvidia-curand-cu12==10.3.2.106 -nvidia-cusolver-cu12==11.4.5.107 -nvidia-cusparse-cu12==12.1.0.106 -nvidia-nccl-cu12==2.18.1 -nvidia-nvjitlink-cu12==12.3.101 -nvidia-nvtx-cu12==12.1.105 -oauthlib==3.2.2 -onnxruntime==1.17.0 -openai==1.13.3 -opencensus-context==0.1.3 -opencensus-ext-azure==1.1.13 -opencensus-ext-logging==0.1.1 -opencensus==0.11.4 -opentelemetry-api==1.22.0 -opentelemetry-exporter-otlp-proto-common==1.22.0 -opentelemetry-exporter-otlp-proto-grpc==1.22.0 -opentelemetry-instrumentation-asgi==0.43b0 -opentelemetry-instrumentation-fastapi==0.43b0 -opentelemetry-instrumentation==0.43b0 -opentelemetry-proto==1.22.0 -opentelemetry-sdk==1.22.0 -opentelemetry-semantic-conventions==0.43b0 -opentelemetry-util-http==0.43b0 -overrides==7.7.0 -packaging==23.2 -pandas==2.2.0 -paramiko==3.4.0 -parso==0.8.3 -pathspec==0.12.1 -pexpect==4.8.0 -pgraph-python==0.6.2 -pillow==10.2.0 -pip==23.3.1 -pkginfo==1.9.6 -platformdirs==4.1.0 -pluggy==1.4.0 -portalocker==2.8.2 -posthog==3.3.4 -pre-commit==3.6.0 -progress==1.6 -prompt-toolkit==3.0.43 -promptflow-tools==0.1.0b15 -promptflow-vectordb==0.2.3 -promptflow==0.1.0b8 -protobuf==4.25.2 -psutil==5.9.8 -ptyprocess==0.7.0 -pulsar-client==3.4.0 -pure-eval==0.2.2 -py-trees==2.2.3 -pyarrow-hotfix==0.6 -pyarrow==13.0.0 -pyasn1-modules==0.3.0 -pyasn1==0.5.1 -pycparser==2.21 -pydantic-core==2.14.6 -pydantic==1.10.14 -pydash==6.0.2 -pydot==2.0.0 -pygments==2.15.1 -pyjwt==2.8.0 -pymongo==4.6.1 -pymupdf==1.23.22 -pymupdfb==1.23.22 -pynacl==1.5.0 -pyopenssl==23.3.0 -pyparsing==3.1.1 -pypdf==4.0.1 -pyperplan==2.1 -pypika==0.48.9 -pyproject-hooks==1.0.0 -pysocks==1.7.1 -pytamer==0.1.15 -pytest==8.0.0 -python-dateutil==2.8.2 -python-dotenv==1.0.1 -python-iso639==2024.1.2 -python-magic==0.4.27 -pytz==2023.3.post1 -pyyaml==6.0.1 -pyzmq==25.1.2 -rapidfuzz==3.6.1 -referencing==0.32.1 -regex==2023.12.25 -requests-cache==1.1.1 -requests-oauthlib==1.3.1 -requests==2.31.0 -responses==0.18.0 -roboticstoolbox-python==1.1.0 -rpds-py==0.17.1 -rsa==4.9 -rtb-data==1.0.1 -rtoml==0.9.0 -ruamel.yaml.clib==0.2.8 -ruamel.yaml==0.17.40 -safetensors==0.4.2 -scikit-learn==1.4.0 -scipy==1.12.0 -secretstorage==3.3.3 -sentence-transformers==2.4.0 -sentencepiece==0.1.99 -sentry-sdk==1.40.6 -setproctitle==1.3.3 -setuptools==68.2.2 -six==1.16.0 -smmap==5.0.1 -sniffio==1.3.0 -soupsieve==2.5 -spatialgeometry==1.1.0 -spatialmath-python==1.1.9 -sqlalchemy==2.0.27 -sqlparse==0.4.4 -stack-data==0.2.0 -starlette==0.35.1 -strictyaml==1.7.3 -swift-sim==1.1.0 -sympy==1.12 -tabulate==0.9.0 -tarski==0.8.2 -tavily-python==0.3.1 -tenacity==8.2.3 -termcolor==2.4.0 -threadpoolctl==3.2.0 -tika==2.6.0 -tiktoken==0.5.2 -tokenizers==0.15.1 -tomli==2.0.1 -torch==2.1.2 -tornado==6.4 -tqdm==4.66.1 -traitlets==5.7.1 -transformers==4.37.2 -triton==2.1.0 -typer==0.9.0 -types-requests==2.31.0.20240125 -typing-extensions==4.9.0 -typing-inspect==0.9.0 -tzdata==2023.4 -unified-planning==1.0.0 -unstructured-client==0.15.5 -unstructured==0.12.2 -up-aries==0.3.3 -up-enhsp==0.0.15 -up-fast-downward==0.2.3 -up-fmap==0.0.7 -up-lpg==0.0.7 -up-pyperplan==1.0.0 -up-symk==1.1.0 -up-tamer==1.0.0 -url-normalize==1.4.3 -urllib3==2.1.0 -uvicorn==0.27.0.post1 -uvloop==0.19.0 -virtualenv==20.25.0 -waitress==2.1.2 -wandb==0.16.3 -watchfiles==0.21.0 -wcwidth==0.2.5 -websocket-client==1.7.0 -websockets==12.0 -werkzeug==3.0.1 -wheel==0.41.2 -wikipedia==1.4.0 -wrapt==1.16.0 -xxhash==3.4.1 -yarl==1.9.4 -zipp==3.17.0 \ No newline at end of file diff --git a/wandb/run-20240304_141453-ftjob-kdM1O1dj6DYbYcykXXKyikqt/files/wandb-metadata.json b/wandb/run-20240304_141453-ftjob-kdM1O1dj6DYbYcykXXKyikqt/files/wandb-metadata.json deleted file mode 100644 index ea3e9e3b..00000000 --- a/wandb/run-20240304_141453-ftjob-kdM1O1dj6DYbYcykXXKyikqt/files/wandb-metadata.json +++ /dev/null @@ -1,80 +0,0 @@ -{ - "os": "Linux-5.15.0-92-generic-x86_64-with-glibc2.31", - "python": "3.10.0", - "heartbeatAt": "2024-03-04T13:14:54.301422", - "startedAt": "2024-03-04T13:14:53.751723", - "docker": null, - "cuda": null, - "args": [], - "state": "running", - "program": "/home/blackbird/kios/data/finetuning/visualize.py", - "codePathLocal": "data/finetuning/visualize.py", - "codePath": "data/finetuning/visualize.py", - "git": { - "remote": "git@gitlab.lrz.de:bblab/ma/kios.git", - "commit": "bd7f44195e793e386f8713bf2d213783c079685a" - }, - "email": "jicong.ao@tum.de", - "root": "/home/blackbird/kios", - "host": "bblab-laptop", - "username": "blackbird", - "executable": "/home/blackbird/miniconda3/envs/kios/bin/python", - "cpu_count": 4, - "cpu_count_logical": 8, - "cpu_freq": { - "current": 2315.62725, - "min": 400.0, - "max": 4200.0 - }, - "cpu_freq_per_core": [ - { - "current": 2400.0, - "min": 400.0, - "max": 4200.0 - }, - { - "current": 2400.0, - "min": 400.0, - "max": 4200.0 - }, - { - "current": 2400.0, - "min": 400.0, - "max": 4200.0 - }, - { - "current": 1725.018, - "min": 400.0, - "max": 4200.0 - }, - { - "current": 2400.0, - "min": 400.0, - "max": 4200.0 - }, - { - "current": 2400.0, - "min": 400.0, - "max": 4200.0 - }, - { - "current": 2400.0, - "min": 400.0, - "max": 4200.0 - }, - { - "current": 2400.0, - "min": 400.0, - "max": 4200.0 - } - ], - "disk": { - "/": { - "total": 467.8895797729492, - "used": 173.56134414672852 - } - }, - "memory": { - "total": 15.418136596679688 - } -} diff --git a/wandb/run-20240304_141453-ftjob-kdM1O1dj6DYbYcykXXKyikqt/files/wandb-summary.json b/wandb/run-20240304_141453-ftjob-kdM1O1dj6DYbYcykXXKyikqt/files/wandb-summary.json deleted file mode 100644 index 7e33c30c..00000000 --- a/wandb/run-20240304_141453-ftjob-kdM1O1dj6DYbYcykXXKyikqt/files/wandb-summary.json +++ /dev/null @@ -1 +0,0 @@ -{"train_loss": 0.01838, "train_accuracy": 0.99588, "_timestamp": 1709558098.328243, "_runtime": 4.571851015090942, "_step": 20, "fine_tuned_model": "ft:gpt-3.5-turbo-0125:kifabrik-mirmi:kios-ut-gen-v2:8z2KbPsr", "_wandb": {"runtime": 4}} \ No newline at end of file diff --git a/wandb/run-20240304_141453-ftjob-kdM1O1dj6DYbYcykXXKyikqt/run-ftjob-kdM1O1dj6DYbYcykXXKyikqt.wandb b/wandb/run-20240304_141453-ftjob-kdM1O1dj6DYbYcykXXKyikqt/run-ftjob-kdM1O1dj6DYbYcykXXKyikqt.wandb deleted file mode 100644 index 5a50d72c..00000000 Binary files a/wandb/run-20240304_141453-ftjob-kdM1O1dj6DYbYcykXXKyikqt/run-ftjob-kdM1O1dj6DYbYcykXXKyikqt.wandb and /dev/null differ diff --git a/wandb/run-20240304_142731-ftjob-L36vT4dDnkWzL9GckJ0SssVm/files/conda-environment.yaml b/wandb/run-20240304_142731-ftjob-L36vT4dDnkWzL9GckJ0SssVm/files/conda-environment.yaml deleted file mode 100644 index 5b7d2077..00000000 --- a/wandb/run-20240304_142731-ftjob-L36vT4dDnkWzL9GckJ0SssVm/files/conda-environment.yaml +++ /dev/null @@ -1,400 +0,0 @@ -name: kios -channels: - - defaults -dependencies: - - _libgcc_mutex=0.1=main - - _openmp_mutex=5.1=1_gnu - - asttokens=2.0.5=pyhd3eb1b0_0 - - bzip2=1.0.8=h7b6447c_0 - - ca-certificates=2023.12.12=h06a4308_0 - - decorator=5.1.1=pyhd3eb1b0_0 - - exceptiongroup=1.2.0=py310h06a4308_0 - - executing=0.8.3=pyhd3eb1b0_0 - - ipython=8.20.0=py310h06a4308_0 - - jedi=0.18.1=py310h06a4308_1 - - ld_impl_linux-64=2.38=h1181459_1 - - libffi=3.3=he6710b0_2 - - libgcc-ng=11.2.0=h1234567_1 - - libgomp=11.2.0=h1234567_1 - - libstdcxx-ng=11.2.0=h1234567_1 - - libuuid=1.41.5=h5eee18b_0 - - matplotlib-inline=0.1.6=py310h06a4308_0 - - ncurses=6.4=h6a678d5_0 - - openssl=1.1.1w=h7f8727e_0 - - parso=0.8.3=pyhd3eb1b0_0 - - pexpect=4.8.0=pyhd3eb1b0_3 - - pip=23.3.1=py310h06a4308_0 - - prompt-toolkit=3.0.43=py310h06a4308_0 - - prompt_toolkit=3.0.43=hd3eb1b0_0 - - ptyprocess=0.7.0=pyhd3eb1b0_2 - - pure_eval=0.2.2=pyhd3eb1b0_0 - - python=3.10.0=h12debd9_5 - - readline=8.2=h5eee18b_0 - - setuptools=68.2.2=py310h06a4308_0 - - six=1.16.0=pyhd3eb1b0_1 - - sqlite=3.41.2=h5eee18b_0 - - stack_data=0.2.0=pyhd3eb1b0_0 - - tk=8.6.12=h1ccaba5_0 - - traitlets=5.7.1=py310h06a4308_0 - - wcwidth=0.2.5=pyhd3eb1b0_0 - - wheel=0.41.2=py310h06a4308_0 - - xz=5.4.5=h5eee18b_0 - - zlib=1.2.13=h5eee18b_0 - - pip: - - adal==1.2.7 - - aenum==3.1.15 - - aiohttp==3.9.1 - - aiosignal==1.3.1 - - alembic==1.13.1 - - annotated-types==0.6.0 - - ansitable==0.9.10 - - anthropic==0.8.1 - - antlr4-python3-runtime==4.7.2 - - anyio==4.2.0 - - appdirs==1.4.4 - - applicationinsights==0.11.10 - - argcomplete==3.2.2 - - asgiref==3.7.2 - - async-timeout==4.0.3 - - attrs==23.2.0 - - azure-ai-generative==1.0.0b2 - - azure-ai-ml==1.12.1 - - azure-ai-resources==1.0.0b2 - - azure-common==1.1.28 - - azure-core==1.29.7 - - azure-graphrbac==0.61.1 - - azure-identity==1.15.0 - - azure-keyvault==4.2.0 - - azure-keyvault-certificates==4.7.0 - - azure-keyvault-keys==4.8.0 - - azure-keyvault-secrets==4.7.0 - - azure-mgmt-authorization==4.0.0 - - azure-mgmt-containerregistry==10.3.0 - - azure-mgmt-core==1.4.0 - - azure-mgmt-keyvault==10.3.0 - - azure-mgmt-network==25.1.0 - - azure-mgmt-resource==23.0.1 - - azure-mgmt-storage==21.1.0 - - azure-search-documents==11.4.0b11 - - azure-storage-blob==12.19.0 - - azure-storage-file-datalake==12.14.0 - - azure-storage-file-share==12.15.0 - - azureml-core==1.54.0.post1 - - azureml-dataprep==5.1.2 - - azureml-dataprep-native==41.0.0 - - azureml-dataprep-rslex==2.22.2 - - azureml-fsspec==1.3.0 - - azureml-metrics==0.0.43 - - azureml-mlflow==1.54.0.post1 - - azureml-telemetry==1.54.0 - - backoff==2.2.1 - - backports-tempfile==1.0 - - backports-weakref==1.0.post1 - - banal==1.0.6 - - bcrypt==4.1.2 - - beautifulsoup4==4.12.3 - - blinker==1.7.0 - - bs4==0.0.2 - - build==1.0.3 - - cachetools==5.3.2 - - cattrs==23.2.3 - - certifi==2023.11.17 - - cffi==1.16.0 - - cfgv==3.4.0 - - chardet==5.2.0 - - charset-normalizer==3.3.2 - - chroma-hnswlib==0.7.3 - - chromadb==0.4.22 - - click==8.1.7 - - cloudpickle==2.2.1 - - cohere==4.46 - - colorama==0.4.6 - - colored==1.4.4 - - coloredlogs==15.0.1 - - comm==0.2.1 - - contextlib2==21.6.0 - - contourpy==1.2.0 - - cryptography==41.0.7 - - cycler==0.12.1 - - databricks-cli==0.18.0 - - dataclasses-json==0.6.3 - - dataset==1.6.2 - - datasets==2.16.1 - - debugpy==1.8.1 - - deprecated==1.2.14 - - dill==0.3.7 - - dirtyjson==1.0.8 - - distlib==0.3.8 - - distro==1.9.0 - - dnspython==2.5.0 - - docker==6.1.3 - - docker-pycreds==0.4.0 - - emoji==2.10.0 - - entrypoints==0.4 - - evaluate==0.4.1 - - fairscale==0.4.13 - - faiss-cpu==1.7.4 - - fastapi==0.109.1 - - fastavro==1.9.3 - - filelock==3.13.1 - - filetype==1.2.0 - - fire==0.5.0 - - flask==2.3.3 - - flatbuffers==23.5.26 - - fonttools==4.47.2 - - frozenlist==1.4.1 - - fsspec==2023.10.0 - - gitdb==4.0.11 - - gitpython==3.1.41 - - google-api-core==2.15.0 - - google-auth==2.27.0 - - google-search-results==2.4.1 - - googleapis-common-protos==1.62.0 - - gradientai==1.7.0 - - greenlet==3.0.3 - - grpcio==1.60.0 - - grpcio-tools==1.60.0 - - h11==0.14.0 - - httpcore==1.0.2 - - httptools==0.6.1 - - httpx==0.26.0 - - huggingface-hub==0.20.3 - - humanfriendly==10.0 - - identify==2.5.33 - - idna==3.6 - - importlib-metadata==6.11.0 - - importlib-resources==6.1.1 - - iniconfig==2.0.0 - - ipykernel==6.29.2 - - isodate==0.6.1 - - itsdangerous==2.1.2 - - jaraco-classes==3.3.0 - - jeepney==0.8.0 - - jinja2==3.1.3 - - jmespath==1.0.1 - - joblib==1.3.2 - - jsonpatch==1.33 - - jsonpath-python==1.0.6 - - jsonpickle==3.0.2 - - jsonpointer==2.4 - - jsonschema==4.21.1 - - jsonschema-specifications==2023.12.1 - - jupyter-client==8.6.0 - - jupyter-core==5.7.1 - - keyring==24.3.0 - - kiwisolver==1.4.5 - - knack==0.11.0 - - kubernetes==29.0.0 - - langchain==0.1.7 - - langchain-anthropic==0.0.1.post1 - - langchain-community==0.0.20 - - langchain-core==0.1.23 - - langchain-experimental==0.0.51 - - langchain-openai==0.0.6 - - langchainhub==0.1.14 - - langdetect==1.0.9 - - langgraph==0.0.24 - - langsmith==0.0.87 - - llama-index==0.10.4 - - llama-index-agent-openai==0.1.1 - - llama-index-core==0.10.3 - - llama-index-embeddings-adapter==0.1.0 - - llama-index-embeddings-openai==0.1.1 - - llama-index-finetuning==0.1.0 - - llama-index-legacy==0.9.48 - - llama-index-llms-gradient==0.1.0 - - llama-index-llms-openai==0.1.1 - - llama-index-multi-modal-llms-openai==0.1.1 - - llama-index-postprocessor-cohere-rerank==0.1.0 - - llama-index-program-openai==0.1.1 - - llama-index-question-gen-openai==0.1.1 - - llama-index-readers-file==0.1.3 - - lxml==5.1.0 - - mako==1.3.1 - - markdown==3.5.2 - - markupsafe==2.1.4 - - marshmallow==3.20.2 - - matplotlib==3.8.3 - - mlflow-skinny==2.9.2 - - mmh3==4.1.0 - - monotonic==1.6 - - more-itertools==10.2.0 - - mpmath==1.3.0 - - msal==1.26.0 - - msal-extensions==1.0.0 - - msrest==0.7.1 - - msrestazure==0.6.4 - - multidict==6.0.4 - - multipledispatch==1.0.0 - - multiprocess==0.70.15 - - mypy-extensions==1.0.0 - - ndg-httpsclient==0.5.1 - - neo4j==5.16.0 - - nest-asyncio==1.6.0 - - networkx==3.2.1 - - nltk==3.8.1 - - nodeenv==1.8.0 - - numpy==1.26.3 - - nvidia-cublas-cu12==12.1.3.1 - - nvidia-cuda-cupti-cu12==12.1.105 - - nvidia-cuda-nvrtc-cu12==12.1.105 - - nvidia-cuda-runtime-cu12==12.1.105 - - nvidia-cudnn-cu12==8.9.2.26 - - nvidia-cufft-cu12==11.0.2.54 - - nvidia-curand-cu12==10.3.2.106 - - nvidia-cusolver-cu12==11.4.5.107 - - nvidia-cusparse-cu12==12.1.0.106 - - nvidia-nccl-cu12==2.18.1 - - nvidia-nvjitlink-cu12==12.3.101 - - nvidia-nvtx-cu12==12.1.105 - - oauthlib==3.2.2 - - onnxruntime==1.17.0 - - openai==1.13.3 - - opencensus==0.11.4 - - opencensus-context==0.1.3 - - opencensus-ext-azure==1.1.13 - - opencensus-ext-logging==0.1.1 - - opentelemetry-api==1.22.0 - - opentelemetry-exporter-otlp-proto-common==1.22.0 - - opentelemetry-exporter-otlp-proto-grpc==1.22.0 - - opentelemetry-instrumentation==0.43b0 - - opentelemetry-instrumentation-asgi==0.43b0 - - opentelemetry-instrumentation-fastapi==0.43b0 - - opentelemetry-proto==1.22.0 - - opentelemetry-sdk==1.22.0 - - opentelemetry-semantic-conventions==0.43b0 - - opentelemetry-util-http==0.43b0 - - overrides==7.7.0 - - packaging==23.2 - - pandas==2.2.0 - - paramiko==3.4.0 - - pathspec==0.12.1 - - pgraph-python==0.6.2 - - pillow==10.2.0 - - pkginfo==1.9.6 - - platformdirs==4.1.0 - - pluggy==1.4.0 - - portalocker==2.8.2 - - posthog==3.3.4 - - pre-commit==3.6.0 - - progress==1.6 - - promptflow==0.1.0b8 - - promptflow-tools==0.1.0b15 - - promptflow-vectordb==0.2.3 - - protobuf==4.25.2 - - psutil==5.9.8 - - pulsar-client==3.4.0 - - py-trees==2.2.3 - - pyarrow==13.0.0 - - pyarrow-hotfix==0.6 - - pyasn1==0.5.1 - - pyasn1-modules==0.3.0 - - pycparser==2.21 - - pydantic==1.10.14 - - pydantic-core==2.14.6 - - pydash==6.0.2 - - pydot==2.0.0 - - pygments==2.17.2 - - pyjwt==2.8.0 - - pymongo==4.6.1 - - pymupdf==1.23.22 - - pymupdfb==1.23.22 - - pynacl==1.5.0 - - pyopenssl==23.3.0 - - pyparsing==3.1.1 - - pypdf==4.0.1 - - pyperplan==2.1 - - pypika==0.48.9 - - pyproject-hooks==1.0.0 - - pysocks==1.7.1 - - pytamer==0.1.15 - - pytest==8.0.0 - - python-dateutil==2.8.2 - - python-dotenv==1.0.1 - - python-graphviz==0.20.1 - - python-iso639==2024.1.2 - - python-magic==0.4.27 - - pytz==2023.3.post1 - - pyyaml==6.0.1 - - pyzmq==25.1.2 - - rapidfuzz==3.6.1 - - referencing==0.32.1 - - regex==2023.12.25 - - requests==2.31.0 - - requests-cache==1.1.1 - - requests-oauthlib==1.3.1 - - responses==0.18.0 - - roboticstoolbox-python==1.1.0 - - rpds-py==0.17.1 - - rsa==4.9 - - rtb-data==1.0.1 - - rtoml==0.9.0 - - ruamel-yaml==0.17.40 - - ruamel-yaml-clib==0.2.8 - - safetensors==0.4.2 - - scikit-learn==1.4.0 - - scipy==1.12.0 - - secretstorage==3.3.3 - - sentence-transformers==2.4.0 - - sentencepiece==0.1.99 - - sentry-sdk==1.40.6 - - setproctitle==1.3.3 - - smmap==5.0.1 - - sniffio==1.3.0 - - soupsieve==2.5 - - spatialgeometry==1.1.0 - - spatialmath-python==1.1.9 - - sqlalchemy==2.0.27 - - sqlparse==0.4.4 - - starlette==0.35.1 - - strictyaml==1.7.3 - - swift-sim==1.1.0 - - sympy==1.12 - - tabulate==0.9.0 - - tarski==0.8.2 - - tavily-python==0.3.1 - - tenacity==8.2.3 - - termcolor==2.4.0 - - threadpoolctl==3.2.0 - - tika==2.6.0 - - tiktoken==0.5.2 - - tokenizers==0.15.1 - - tomli==2.0.1 - - torch==2.1.2 - - tornado==6.4 - - tqdm==4.66.1 - - transformers==4.37.2 - - triton==2.1.0 - - typer==0.9.0 - - types-requests==2.31.0.20240125 - - typing-extensions==4.9.0 - - typing-inspect==0.9.0 - - tzdata==2023.4 - - unified-planning==1.0.0 - - unstructured==0.12.2 - - unstructured-client==0.15.5 - - up-aries==0.3.3 - - up-enhsp==0.0.15 - - up-fast-downward==0.2.3 - - up-fmap==0.0.7 - - up-lpg==0.0.7 - - up-pyperplan==1.0.0 - - up-symk==1.1.0 - - up-tamer==1.0.0 - - url-normalize==1.4.3 - - urllib3==2.1.0 - - uvicorn==0.27.0.post1 - - uvloop==0.19.0 - - virtualenv==20.25.0 - - waitress==2.1.2 - - wandb==0.16.3 - - watchfiles==0.21.0 - - websocket-client==1.7.0 - - websockets==12.0 - - werkzeug==3.0.1 - - wikipedia==1.4.0 - - wrapt==1.16.0 - - xxhash==3.4.1 - - yarl==1.9.4 - - zipp==3.17.0 -prefix: /home/blackbird/miniconda3/envs/kios diff --git a/wandb/run-20240304_142731-ftjob-L36vT4dDnkWzL9GckJ0SssVm/files/config.yaml b/wandb/run-20240304_142731-ftjob-L36vT4dDnkWzL9GckJ0SssVm/files/config.yaml deleted file mode 100644 index c9063e9f..00000000 --- a/wandb/run-20240304_142731-ftjob-L36vT4dDnkWzL9GckJ0SssVm/files/config.yaml +++ /dev/null @@ -1,78 +0,0 @@ -wandb_version: 1 - -_wandb: - desc: null - value: - python_version: 3.10.0 - cli_version: 0.16.3 - is_jupyter_run: false - is_kaggle_kernel: false - start_time: 1709558851.777437 - t: - 1: - - 55 - - 95 - 2: - - 55 - - 95 - 3: - - 13 - - 14 - - 23 - - 58 - 4: 3.10.0 - 5: 0.16.3 - 8: - - 5 - 13: linux-x86_64 -id: - desc: null - value: ftjob-L36vT4dDnkWzL9GckJ0SssVm -created_at: - desc: null - value: '2024-03-04 14:19:02' -error: - desc: null - value: Error(code=None, message=None, param=None, error=None) -fine_tuned_model: - desc: null - value: ft:gpt-3.5-turbo-0125:kifabrik-mirmi:kios-ut-gen-v3:8z2ZAg2T -finished_at: - desc: null - value: '2024-03-04 14:22:43' -model: - desc: null - value: ft:gpt-3.5-turbo-0125:kifabrik-mirmi:kios-ut-gen-v2:8z2KbPsr -object: - desc: null - value: fine_tuning.job -organization_id: - desc: null - value: org-ygWwisor3PBcF2dPNRDTzzgd -result_files: - desc: null - value: file-BZkZkwLqXcoQNXF0OkJpwEsr -status: - desc: null - value: succeeded -trained_tokens: - desc: null - value: 166931 -training_file: - desc: null - value: file-D8aBgTaXbp3g42RcV2bTiKDK -validation_file: - desc: null - value: null -user_provided_suffix: - desc: null - value: kios_ut_gen_v3 -hyperparameters: - desc: null - value: - n_epochs: 1 - batch_size: 3 - learning_rate_multiplier: 0.5 -n_train: - desc: null - value: 60 diff --git a/wandb/run-20240304_142731-ftjob-L36vT4dDnkWzL9GckJ0SssVm/files/output.log b/wandb/run-20240304_142731-ftjob-L36vT4dDnkWzL9GckJ0SssVm/files/output.log deleted file mode 100644 index 143831ac..00000000 --- a/wandb/run-20240304_142731-ftjob-L36vT4dDnkWzL9GckJ0SssVm/files/output.log +++ /dev/null @@ -1,42 +0,0 @@ -wandb: Waiting for the OpenAI fine-tuning job to be finished... -wandb: Fine-tuning finished, logging metrics, model metadata, and more to W&B -Traceback (most recent call last): - File "/home/blackbird/kios/data/finetuning/visualize.py", line 5, in - WandbLogger.sync(fine_tune_job_id="ftjob-L36vT4dDnkWzL9GckJ0SssVm") - File "/home/blackbird/miniconda3/envs/kios/lib/python3.10/site-packages/wandb/integration/openai/fine_tuning.py", line 148, in sync - cls._log_fine_tune( - File "/home/blackbird/miniconda3/envs/kios/lib/python3.10/site-packages/wandb/integration/openai/fine_tuning.py", line 237, in _log_fine_tune - cls._log_artifacts(fine_tune, project, entity) - File "/home/blackbird/miniconda3/envs/kios/lib/python3.10/site-packages/wandb/integration/openai/fine_tuning.py", line 344, in _log_artifacts - json.dump(dict_fine_tune, f, indent=2) - File "/home/blackbird/miniconda3/envs/kios/lib/python3.10/json/__init__.py", line 179, in dump - for chunk in iterable: - File "/home/blackbird/miniconda3/envs/kios/lib/python3.10/json/encoder.py", line 431, in _iterencode - yield from _iterencode_dict(o, _current_indent_level) - File "/home/blackbird/miniconda3/envs/kios/lib/python3.10/json/encoder.py", line 405, in _iterencode_dict - yield from chunks - File "/home/blackbird/miniconda3/envs/kios/lib/python3.10/json/encoder.py", line 438, in _iterencode - o = _default(o) - File "/home/blackbird/miniconda3/envs/kios/lib/python3.10/json/encoder.py", line 179, in default - raise TypeError(f'Object of type {o.__class__.__name__} ' -TypeError: Object of type Error is not JSON serializable -Traceback (most recent call last): - File "/home/blackbird/kios/data/finetuning/visualize.py", line 5, in - WandbLogger.sync(fine_tune_job_id="ftjob-L36vT4dDnkWzL9GckJ0SssVm") - File "/home/blackbird/miniconda3/envs/kios/lib/python3.10/site-packages/wandb/integration/openai/fine_tuning.py", line 148, in sync - cls._log_fine_tune( - File "/home/blackbird/miniconda3/envs/kios/lib/python3.10/site-packages/wandb/integration/openai/fine_tuning.py", line 237, in _log_fine_tune - cls._log_artifacts(fine_tune, project, entity) - File "/home/blackbird/miniconda3/envs/kios/lib/python3.10/site-packages/wandb/integration/openai/fine_tuning.py", line 344, in _log_artifacts - json.dump(dict_fine_tune, f, indent=2) - File "/home/blackbird/miniconda3/envs/kios/lib/python3.10/json/__init__.py", line 179, in dump - for chunk in iterable: - File "/home/blackbird/miniconda3/envs/kios/lib/python3.10/json/encoder.py", line 431, in _iterencode - yield from _iterencode_dict(o, _current_indent_level) - File "/home/blackbird/miniconda3/envs/kios/lib/python3.10/json/encoder.py", line 405, in _iterencode_dict - yield from chunks - File "/home/blackbird/miniconda3/envs/kios/lib/python3.10/json/encoder.py", line 438, in _iterencode - o = _default(o) - File "/home/blackbird/miniconda3/envs/kios/lib/python3.10/json/encoder.py", line 179, in default - raise TypeError(f'Object of type {o.__class__.__name__} ' -TypeError: Object of type Error is not JSON serializable \ No newline at end of file diff --git a/wandb/run-20240304_142731-ftjob-L36vT4dDnkWzL9GckJ0SssVm/files/requirements.txt b/wandb/run-20240304_142731-ftjob-L36vT4dDnkWzL9GckJ0SssVm/files/requirements.txt deleted file mode 100644 index 63c80899..00000000 --- a/wandb/run-20240304_142731-ftjob-L36vT4dDnkWzL9GckJ0SssVm/files/requirements.txt +++ /dev/null @@ -1,376 +0,0 @@ -adal==1.2.7 -aenum==3.1.15 -aiohttp==3.9.1 -aiosignal==1.3.1 -alembic==1.13.1 -annotated-types==0.6.0 -ansitable==0.9.10 -anthropic==0.8.1 -antlr4-python3-runtime==4.7.2 -anyio==4.2.0 -appdirs==1.4.4 -applicationinsights==0.11.10 -argcomplete==3.2.2 -asgiref==3.7.2 -asttokens==2.0.5 -async-timeout==4.0.3 -attrs==23.2.0 -azure-ai-generative==1.0.0b2 -azure-ai-ml==1.12.1 -azure-ai-resources==1.0.0b2 -azure-common==1.1.28 -azure-core==1.29.7 -azure-graphrbac==0.61.1 -azure-identity==1.15.0 -azure-keyvault-certificates==4.7.0 -azure-keyvault-keys==4.8.0 -azure-keyvault-secrets==4.7.0 -azure-keyvault==4.2.0 -azure-mgmt-authorization==4.0.0 -azure-mgmt-containerregistry==10.3.0 -azure-mgmt-core==1.4.0 -azure-mgmt-keyvault==10.3.0 -azure-mgmt-network==25.1.0 -azure-mgmt-resource==23.0.1 -azure-mgmt-storage==21.1.0 -azure-search-documents==11.4.0b11 -azure-storage-blob==12.19.0 -azure-storage-file-datalake==12.14.0 -azure-storage-file-share==12.15.0 -azureml-core==1.54.0.post1 -azureml-dataprep-native==41.0.0 -azureml-dataprep-rslex==2.22.2 -azureml-dataprep==5.1.2 -azureml-fsspec==1.3.0 -azureml-metrics==0.0.43 -azureml-mlflow==1.54.0.post1 -azureml-telemetry==1.54.0 -backoff==2.2.1 -backports.tempfile==1.0 -backports.weakref==1.0.post1 -banal==1.0.6 -bcrypt==4.1.2 -beautifulsoup4==4.12.3 -blinker==1.7.0 -bs4==0.0.2 -build==1.0.3 -cachetools==5.3.2 -cattrs==23.2.3 -certifi==2023.11.17 -cffi==1.16.0 -cfgv==3.4.0 -chardet==5.2.0 -charset-normalizer==3.3.2 -chroma-hnswlib==0.7.3 -chromadb==0.4.22 -click==8.1.7 -cloudpickle==2.2.1 -cohere==4.46 -colorama==0.4.6 -colored==1.4.4 -coloredlogs==15.0.1 -comm==0.2.1 -contextlib2==21.6.0 -contourpy==1.2.0 -cryptography==41.0.7 -cycler==0.12.1 -databricks-cli==0.18.0 -dataclasses-json==0.6.3 -dataset==1.6.2 -datasets==2.16.1 -debugpy==1.8.1 -decorator==5.1.1 -deprecated==1.2.14 -dill==0.3.7 -dirtyjson==1.0.8 -distlib==0.3.8 -distro==1.9.0 -dnspython==2.5.0 -docker-pycreds==0.4.0 -docker==6.1.3 -emoji==2.10.0 -entrypoints==0.4 -evaluate==0.4.1 -exceptiongroup==1.2.0 -executing==0.8.3 -fairscale==0.4.13 -faiss-cpu==1.7.4 -fastapi==0.109.1 -fastavro==1.9.3 -filelock==3.13.1 -filetype==1.2.0 -fire==0.5.0 -flask==2.3.3 -flatbuffers==23.5.26 -fonttools==4.47.2 -frozenlist==1.4.1 -fsspec==2023.10.0 -gitdb==4.0.11 -gitpython==3.1.41 -google-api-core==2.15.0 -google-auth==2.27.0 -google-search-results==2.4.1 -googleapis-common-protos==1.62.0 -gradientai==1.7.0 -graphviz==0.20.1 -greenlet==3.0.3 -grpcio-tools==1.60.0 -grpcio==1.60.0 -h11==0.14.0 -httpcore==1.0.2 -httptools==0.6.1 -httpx==0.26.0 -huggingface-hub==0.20.3 -humanfriendly==10.0 -identify==2.5.33 -idna==3.6 -importlib-metadata==6.11.0 -importlib-resources==6.1.1 -iniconfig==2.0.0 -ipykernel==6.29.2 -ipython==8.20.0 -isodate==0.6.1 -itsdangerous==2.1.2 -jaraco.classes==3.3.0 -jedi==0.18.1 -jeepney==0.8.0 -jinja2==3.1.3 -jmespath==1.0.1 -joblib==1.3.2 -jsonpatch==1.33 -jsonpath-python==1.0.6 -jsonpickle==3.0.2 -jsonpointer==2.4 -jsonschema-specifications==2023.12.1 -jsonschema==4.21.1 -jupyter-client==8.6.0 -jupyter-core==5.7.1 -keyring==24.3.0 -kios-bt-planning==0.1.0 -kiwisolver==1.4.5 -knack==0.11.0 -kubernetes==29.0.0 -langchain-anthropic==0.0.1.post1 -langchain-community==0.0.20 -langchain-core==0.1.23 -langchain-experimental==0.0.51 -langchain-openai==0.0.6 -langchain==0.1.7 -langchainhub==0.1.14 -langdetect==1.0.9 -langgraph==0.0.24 -langsmith==0.0.87 -llama-index-agent-openai==0.1.1 -llama-index-core==0.10.3 -llama-index-embeddings-adapter==0.1.0 -llama-index-embeddings-openai==0.1.1 -llama-index-finetuning==0.1.0 -llama-index-legacy==0.9.48 -llama-index-llms-gradient==0.1.0 -llama-index-llms-openai==0.1.1 -llama-index-multi-modal-llms-openai==0.1.1 -llama-index-postprocessor-cohere-rerank==0.1.0 -llama-index-program-openai==0.1.1 -llama-index-question-gen-openai==0.1.1 -llama-index-readers-file==0.1.3 -llama-index==0.10.4 -lxml==5.1.0 -mako==1.3.1 -markdown==3.5.2 -markupsafe==2.1.4 -marshmallow==3.20.2 -matplotlib-inline==0.1.6 -matplotlib==3.8.3 -mlflow-skinny==2.9.2 -mmh3==4.1.0 -monotonic==1.6 -more-itertools==10.2.0 -mpmath==1.3.0 -msal-extensions==1.0.0 -msal==1.26.0 -msrest==0.7.1 -msrestazure==0.6.4 -multidict==6.0.4 -multipledispatch==1.0.0 -multiprocess==0.70.15 -mypy-extensions==1.0.0 -ndg-httpsclient==0.5.1 -neo4j==5.16.0 -nest-asyncio==1.6.0 -networkx==3.2.1 -nltk==3.8.1 -nodeenv==1.8.0 -numpy==1.26.3 -nvidia-cublas-cu12==12.1.3.1 -nvidia-cuda-cupti-cu12==12.1.105 -nvidia-cuda-nvrtc-cu12==12.1.105 -nvidia-cuda-runtime-cu12==12.1.105 -nvidia-cudnn-cu12==8.9.2.26 -nvidia-cufft-cu12==11.0.2.54 -nvidia-curand-cu12==10.3.2.106 -nvidia-cusolver-cu12==11.4.5.107 -nvidia-cusparse-cu12==12.1.0.106 -nvidia-nccl-cu12==2.18.1 -nvidia-nvjitlink-cu12==12.3.101 -nvidia-nvtx-cu12==12.1.105 -oauthlib==3.2.2 -onnxruntime==1.17.0 -openai==1.13.3 -opencensus-context==0.1.3 -opencensus-ext-azure==1.1.13 -opencensus-ext-logging==0.1.1 -opencensus==0.11.4 -opentelemetry-api==1.22.0 -opentelemetry-exporter-otlp-proto-common==1.22.0 -opentelemetry-exporter-otlp-proto-grpc==1.22.0 -opentelemetry-instrumentation-asgi==0.43b0 -opentelemetry-instrumentation-fastapi==0.43b0 -opentelemetry-instrumentation==0.43b0 -opentelemetry-proto==1.22.0 -opentelemetry-sdk==1.22.0 -opentelemetry-semantic-conventions==0.43b0 -opentelemetry-util-http==0.43b0 -overrides==7.7.0 -packaging==23.2 -pandas==2.2.0 -paramiko==3.4.0 -parso==0.8.3 -pathspec==0.12.1 -pexpect==4.8.0 -pgraph-python==0.6.2 -pillow==10.2.0 -pip==23.3.1 -pkginfo==1.9.6 -platformdirs==4.1.0 -pluggy==1.4.0 -portalocker==2.8.2 -posthog==3.3.4 -pre-commit==3.6.0 -progress==1.6 -prompt-toolkit==3.0.43 -promptflow-tools==0.1.0b15 -promptflow-vectordb==0.2.3 -promptflow==0.1.0b8 -protobuf==4.25.2 -psutil==5.9.8 -ptyprocess==0.7.0 -pulsar-client==3.4.0 -pure-eval==0.2.2 -py-trees==2.2.3 -pyarrow-hotfix==0.6 -pyarrow==13.0.0 -pyasn1-modules==0.3.0 -pyasn1==0.5.1 -pycparser==2.21 -pydantic-core==2.14.6 -pydantic==1.10.14 -pydash==6.0.2 -pydot==2.0.0 -pygments==2.15.1 -pyjwt==2.8.0 -pymongo==4.6.1 -pymupdf==1.23.22 -pymupdfb==1.23.22 -pynacl==1.5.0 -pyopenssl==23.3.0 -pyparsing==3.1.1 -pypdf==4.0.1 -pyperplan==2.1 -pypika==0.48.9 -pyproject-hooks==1.0.0 -pysocks==1.7.1 -pytamer==0.1.15 -pytest==8.0.0 -python-dateutil==2.8.2 -python-dotenv==1.0.1 -python-iso639==2024.1.2 -python-magic==0.4.27 -pytz==2023.3.post1 -pyyaml==6.0.1 -pyzmq==25.1.2 -rapidfuzz==3.6.1 -referencing==0.32.1 -regex==2023.12.25 -requests-cache==1.1.1 -requests-oauthlib==1.3.1 -requests==2.31.0 -responses==0.18.0 -roboticstoolbox-python==1.1.0 -rpds-py==0.17.1 -rsa==4.9 -rtb-data==1.0.1 -rtoml==0.9.0 -ruamel.yaml.clib==0.2.8 -ruamel.yaml==0.17.40 -safetensors==0.4.2 -scikit-learn==1.4.0 -scipy==1.12.0 -secretstorage==3.3.3 -sentence-transformers==2.4.0 -sentencepiece==0.1.99 -sentry-sdk==1.40.6 -setproctitle==1.3.3 -setuptools==68.2.2 -six==1.16.0 -smmap==5.0.1 -sniffio==1.3.0 -soupsieve==2.5 -spatialgeometry==1.1.0 -spatialmath-python==1.1.9 -sqlalchemy==2.0.27 -sqlparse==0.4.4 -stack-data==0.2.0 -starlette==0.35.1 -strictyaml==1.7.3 -swift-sim==1.1.0 -sympy==1.12 -tabulate==0.9.0 -tarski==0.8.2 -tavily-python==0.3.1 -tenacity==8.2.3 -termcolor==2.4.0 -threadpoolctl==3.2.0 -tika==2.6.0 -tiktoken==0.5.2 -tokenizers==0.15.1 -tomli==2.0.1 -torch==2.1.2 -tornado==6.4 -tqdm==4.66.1 -traitlets==5.7.1 -transformers==4.37.2 -triton==2.1.0 -typer==0.9.0 -types-requests==2.31.0.20240125 -typing-extensions==4.9.0 -typing-inspect==0.9.0 -tzdata==2023.4 -unified-planning==1.0.0 -unstructured-client==0.15.5 -unstructured==0.12.2 -up-aries==0.3.3 -up-enhsp==0.0.15 -up-fast-downward==0.2.3 -up-fmap==0.0.7 -up-lpg==0.0.7 -up-pyperplan==1.0.0 -up-symk==1.1.0 -up-tamer==1.0.0 -url-normalize==1.4.3 -urllib3==2.1.0 -uvicorn==0.27.0.post1 -uvloop==0.19.0 -virtualenv==20.25.0 -waitress==2.1.2 -wandb==0.16.3 -watchfiles==0.21.0 -wcwidth==0.2.5 -websocket-client==1.7.0 -websockets==12.0 -werkzeug==3.0.1 -wheel==0.41.2 -wikipedia==1.4.0 -wrapt==1.16.0 -xxhash==3.4.1 -yarl==1.9.4 -zipp==3.17.0 \ No newline at end of file diff --git a/wandb/run-20240304_142731-ftjob-L36vT4dDnkWzL9GckJ0SssVm/files/wandb-metadata.json b/wandb/run-20240304_142731-ftjob-L36vT4dDnkWzL9GckJ0SssVm/files/wandb-metadata.json deleted file mode 100644 index d5a4af52..00000000 --- a/wandb/run-20240304_142731-ftjob-L36vT4dDnkWzL9GckJ0SssVm/files/wandb-metadata.json +++ /dev/null @@ -1,80 +0,0 @@ -{ - "os": "Linux-5.15.0-92-generic-x86_64-with-glibc2.31", - "python": "3.10.0", - "heartbeatAt": "2024-03-04T13:27:32.849778", - "startedAt": "2024-03-04T13:27:31.773324", - "docker": null, - "cuda": null, - "args": [], - "state": "running", - "program": "/home/blackbird/kios/data/finetuning/visualize.py", - "codePathLocal": "data/finetuning/visualize.py", - "codePath": "data/finetuning/visualize.py", - "git": { - "remote": "git@gitlab.lrz.de:bblab/ma/kios.git", - "commit": "bd7f44195e793e386f8713bf2d213783c079685a" - }, - "email": "jicong.ao@tum.de", - "root": "/home/blackbird/kios", - "host": "bblab-laptop", - "username": "blackbird", - "executable": "/home/blackbird/miniconda3/envs/kios/bin/python", - "cpu_count": 4, - "cpu_count_logical": 8, - "cpu_freq": { - "current": 2492.59625, - "min": 400.0, - "max": 4200.0 - }, - "cpu_freq_per_core": [ - { - "current": 3140.77, - "min": 400.0, - "max": 4200.0 - }, - { - "current": 2400.0, - "min": 400.0, - "max": 4200.0 - }, - { - "current": 2400.0, - "min": 400.0, - "max": 4200.0 - }, - { - "current": 2400.0, - "min": 400.0, - "max": 4200.0 - }, - { - "current": 2400.0, - "min": 400.0, - "max": 4200.0 - }, - { - "current": 2400.0, - "min": 400.0, - "max": 4200.0 - }, - { - "current": 2400.0, - "min": 400.0, - "max": 4200.0 - }, - { - "current": 2400.0, - "min": 400.0, - "max": 4200.0 - } - ], - "disk": { - "/": { - "total": 467.8895797729492, - "used": 173.56483840942383 - } - }, - "memory": { - "total": 15.418136596679688 - } -} diff --git a/wandb/run-20240304_142731-ftjob-L36vT4dDnkWzL9GckJ0SssVm/files/wandb-summary.json b/wandb/run-20240304_142731-ftjob-L36vT4dDnkWzL9GckJ0SssVm/files/wandb-summary.json deleted file mode 100644 index 95658eb9..00000000 --- a/wandb/run-20240304_142731-ftjob-L36vT4dDnkWzL9GckJ0SssVm/files/wandb-summary.json +++ /dev/null @@ -1 +0,0 @@ -{"train_loss": 0.02223, "train_accuracy": 0.98966, "_timestamp": 1709558857.0419958, "_runtime": 5.264558792114258, "_step": 20, "fine_tuned_model": "ft:gpt-3.5-turbo-0125:kifabrik-mirmi:kios-ut-gen-v3:8z2ZAg2T", "_wandb": {"runtime": 5}} \ No newline at end of file diff --git a/wandb/run-20240304_142731-ftjob-L36vT4dDnkWzL9GckJ0SssVm/run-ftjob-L36vT4dDnkWzL9GckJ0SssVm.wandb b/wandb/run-20240304_142731-ftjob-L36vT4dDnkWzL9GckJ0SssVm/run-ftjob-L36vT4dDnkWzL9GckJ0SssVm.wandb deleted file mode 100644 index 8c1bd837..00000000 Binary files a/wandb/run-20240304_142731-ftjob-L36vT4dDnkWzL9GckJ0SssVm/run-ftjob-L36vT4dDnkWzL9GckJ0SssVm.wandb and /dev/null differ