Add vehicle configuration (#2130)

* Update truck urdf.

* Update truck

* Move assets from ``smarts.core.models` to `smarts.assets`

* Renamed vehicle.urdf to sedan.urdf

* Fix sedan vehicle urdf reference.

* Update substitution configuration loading.

* Allow vehicle composition.

* Add missing scenario_definitions_filepath property

* Add missing glb files.

* Fix trajectory controller test.

* Fix collision test.

* Fix types test.

* Include smarts/assets in manifest.

* Add default truck definition.

* Update test with truck.

* Remove unused files.

* Change [resources] to [assets].

* Retype moving_truck definitions to truck.

* Rename vehicle_filepath -> vehicle_dynamics_filepath

* Move MACOS gui BulletClient.

* Remove color dependency on VEHICLE_CONFIGS.

* Add visual_model_filepath argument to vehicle creation.

* Fix missing argument.

* Fix plan frame generation.

* Update changelog.

* Add type checking improvements.

* Clean up vehicle methods.

* Use PyYAML cast.

* Update configuration with
`SMARTS_ASSETS_DEFAULT_VEHICLE_DEFINITIONS_LIST`

* Attempt typing test fix.

* Attempt to fix typing test.

* Add missing docstring.

* Update AgentInterface with vehicle class.

* Fix pytype types test.

* Move all vehicle related assets to
`smarts.assets.vehicles`.

* Update truck urdf.

* Fix base-tests.

* Make format.

* Fix types test.

* Add moving truck to options.

* Add trucks to tests.

* Add pickup truck urdf.

* Fix test that broke.

* Update to final vehicle set.

* Update documention.

* Update engine configuration documentation.

* Add information on creating a new vehicle
configuration.

* Lessen redirects in test-docs links stage.

* Remove accidental method regression.

CHANGELOG.md

@@ -12,6 +12,8 @@ Copy and pasting the git commit messages is __NOT__ enough.
 ### Added
 - Added a utility method `SMARTS.prepare_observe_from()` which allows safely adding sensors to vehicles.
 - The following methods now exist explicitly `Vehicle.{add_sensor|detach_sensor|subscribed_to|sensor_property|}`.
+- Resources loaded with `load_yaml_config_with_substitution()` now substitute in SMARTS configuration with single squiggle bracket `${}` syntax. This is currently restricted to environment variable names prefixed with `"SMARTS_"`. This extends to benchmark configuration and vehicle configuration.
+- Default vehicle definitions can be now modified using `assets:default_vehicle_definitions_list`/`SMARTS_ASSSETS_DEFAULT_VEHICLE_DEFINITIONS_LIST` or by providing a `vehicle_definitions_list.yaml` in the scenario. These vehicle types are related to the `AgentInterface.vehicle_type` attribute.
 ### Changed
 - `VehicleIndex.build_agent_vehicle()` no longer has `filename` and `surface_patches` parameters.
 - The following modules have been renamed: `envision.types` -> `envision.etypes`, `smarts.core.utils.logging` -> `smarts.core.utils.core_logging`, `smarts.core.utils.math` -> `smarts.core.utils.core_math`, `smarts.sstudio.types` -> `smarts.sstudio.sstypes`. For compatibility reasons they can still be imported by their original module name.
@@ -20,10 +22,21 @@ Copy and pasting the git commit messages is __NOT__ enough.
 - The `examples/e12_rllib` training examples `{pg_example|pg_pbt_example}.py` have been changed to `{ppo_example|ppo_pbt_example}.py`. `Policy Gradients (PG)` has been dropped in favor of the more well documented `Proximal Policy Optimization (PPO)`.
 - Vehicles can now have sensors added to, overwritten, or replaced outright.
 - Logging is now improved to give information about sensor changes in the sensor manager.
+- - Renamed `vehicle.urdf` to `sedan.urdf`.
+- Environment prefix is now configurable for custom `smarts.core.config()` calls.
+- `Vehicle.build_agent_vehicle()` argument `vehicle_filepath` now renamed to `vehicle_dynamics_filepath`.
+- Renamed `MACOS` `pybullet` gui utility from `smarts.core.utils.bullet.BulletClient` to `smarts.core.utils.pybullet.BulletClientMACOS`.
+- `Vehicle.build_agent_vehicle()` and `Vehicle.build_social_vehicle()` moved to `VehicleIndex`.
+- `smarts.core.configuration.Configuration.get_settings()` now uses the `PyYAML` default instead of forcefully casting to `str`.
+- Added `AgentInterface.vehicle_class` which allows selection of a dynamics vehicle from the vehicle definitions list file.
 ### Deprecated
+- Module `smarts.core.models` is now deprecated in favour of `smarts.assets`.
+- Deprecated a few things related to vehicles in the `Scenario` class, including the `vehicle_filepath`, `tire_parameters_filepath`, and `controller_parameters_filepath`. The functionality is now handled through the vehicle definitions.
+- `AgentInterface.vehicle_type` is now deprecated with potential to be restored.
 ### Fixed
 - `SumoTrafficSimulation` gives clearer reasons as to why it failed to connect to the TraCI server.
 - Suppressed an issue where `pybullet_utils.pybullet.BulletClient` would cause an error because it was catching a non `BaseException` type.
+- Fixed a bug where `smarts.core.vehicle_index.VehicleIndex.attach_sensors_to_vehicle()` would pass a method instead of a `PlanFrame` to the generated vehicle `SensorState`.
 - Fixed an issue where `AgentInterface.vehicle_type` would not affect agent vehicles when attempting to take over an existing vehicle.
 - Fixed a case where newly created agent vehicles would have a constant `"sedan"` size instead of the size of `AgentInterface.vehicle_type`.
 - Fixed a case where if vehicles are replaced they would not respect controller and vehicle parameters.

MANIFEST.in

@@ -1,8 +1,7 @@
 include smarts/core/glsl/*.vert smarts/core/glsl/*.frag
-include smarts/core/models/*.glb smarts/core/models/*.urdf
-include smarts/core/models/controller_parameters.yaml
 include envision/web/dist/*
 include smarts/*.ini
+recursive-include smarts/assets *.glb *.urdf *.yaml *.yml
 recursive-include smarts/benchmark *.yaml *.yml
 recursive-include smarts/ros/src *.launch *.msg *.srv package.xml CMakeLists.txt *.py
 recursive-include smarts/scenarios *.xml *.py

docs/benchmarks/benchmark.rst

@@ -108,18 +108,5 @@ The benchmark listing file is organized as below.
            params: # Additional values to pass into the entrypoint as named keyword arguments.
              benchmark_config: ${{smarts.benchmark.driving_smarts.v2022}}/config.yaml
 
-.. note:: 
-
-    Resolving module directories.
 
-    The benchmark configuration directory can be dynamically found through
-    python using an evaluation syntax ``${{}}``. This is experimental and
-    open to change but the following resolves the python module location in
-    loaded configuration files:
-
-    .. code:: yaml
-
-        somewhere_path: ${{module.to.resolve}}/file.txt # resolves to <path>/module/to/resolve/file.txt
-
-    This avoids loading the module into python but resolves to the first
-    path that matches the module.
+See :ref:`engine_configuration` for more details.

docs/conf.py

@@ -47,12 +47,16 @@
 ]
 
 extlinks = {
+    "assets": (
+        "https://github.com/huawei-noah/SMARTS/tree/master/smarts/assets/%s",
+        "%s",
+    ),
     "examples": (
-        "https://github.com/huawei-noah/SMARTS/blob/master/examples/%s",
+        "https://github.com/huawei-noah/SMARTS/tree/master/examples/%s",
         "%s",
     ),
     "scenarios": (
-        "https://github.com/huawei-noah/SMARTS/blob/master/scenarios/%s",
+        "https://github.com/huawei-noah/SMARTS/tree/master/scenarios/%s",
         "%s",
     ),
 }

docs/index.rst

@@ -46,7 +46,8 @@ If you use SMARTS in your research, please cite the `paper <https://arxiv.org/ab
    sim/simulator.rst
    sim/scenario_studio.rst
    sim/bubbles.rst
-   sim/configuration.rst
+   sim/engine_configuration.rst
+   sim/vehicle.rst
 
 .. toctree::
    :hidden:

docs/sim/engine_configuration.rst

@@ -0,0 +1,68 @@
+.. _engine_configuration:
+
+Configuration
+=============
+
+
+SMARTS
+------
+
+You can change the behavior of the underlying SMARTS engine.
+
+Configuration of the engine can come from several sources. These locations take precedence as noted:
+
+1. Individual ``SMARTS_`` prefixed environment variables (e.g. ``SMARTS_SENSOR_WORKER_COUNT``)
+2. Local directory engine configuration (``./smarts_engine.ini``)
+3. Local user engine configuration, ``~/.smarts/engine.ini``, if local directory configuration is not found.
+4. Global engine configuration, ``/etc/smarts/engine.ini``, if local configuration is not found.
+5. Package default configuration, ``$PYTHON_PATH/smarts/engine.ini``, if global configuration is not found.
+
+Note that configuration files resolve all settings at the first found configuration file (they do not layer.)
+
+
+Options
+-------
+
+All settings demonstrated as environment variables are formatted to ``UPPERCASE`` and prefixed with ``SMARTS_`` (e.g. ``[core] logging`` can be configured with ``SMARTS_CORE_LOGGING``)
+
+These settings are as follows:
+
+.. todo::
+
+    List engine settings
+
+
+YAML resources
+--------------
+
+YAML files resolve as `PyYAML.safe_load() <https://pyyaml.org/wiki/PyYAMLDocumentation>` allows with a few extensions.
+
+Dynamic module resolution
+^^^^^^^^^^^^^^^^^^^^^^^^^
+
+The benchmark configuration directory can be dynamically found through
+python using an evaluation syntax ``${{}}``. This is experimental and
+open to change but the following resolves the python module location in
+loaded configuration files:
+
+.. code:: yaml
+
+    somewhere_path: ${{module.to.resolve}}/file.txt # resolves to <path>/module/to/resolve/file.txt
+
+
+This avoids loading the module into python but resolves to the first
+path that matches the module.
+
+Environment variable resolution
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Resolving SMARTS engine configuration.
+
+The engine configuration resolves by referencing the setting through
+the evaluation syntax ``${}``. This is restricted to ``"SMARTS_"``
+prefixed environment variables.
+
+.. code:: yaml
+
+    is_debug: ${SMARTS_CORE_DEBUG} # literal environment variable or engine setting `[core] debug`
+

docs/sim/vehicle.rst

@@ -0,0 +1,150 @@
+.. _vehicle_defaults:
+
+
+Vehicle defaults
+================
+
+``SMARTS`` provides vehicle configuration for agent control.
+
+
+Default agent vehicle details
+-----------------------------
+
+.. list-table::
+   :header-rows: 1
+
+   * - **Vehicle**
+     - Sedan (Default)
+     - Bus (class 4)
+     - Pickup truck (class 2a)
+     - Empty moving truck (class 5)
+     - Loaded moving truck (class 5)
+   * - **Resource**
+     - "sedan" | "generic_sedan"
+     - "bus" | "generic_bus"
+     - "pickup" | "generic_pickup_truck"
+     - "moving_truck_empty"
+     - "moving_truck_loaded"
+   * - **Dimensions** (LWH)
+     - 3.68  1.47  1.30
+     - 7.00  2.20  2.40
+     - 5.00  1.91  1.89
+     - 7.10  2.40  2.40
+     - 7.10  2.40  2.40
+   * - **Mass** (kg)
+     - 2356.00
+     - 6000.00
+     - 2600.00
+     - 6500.00
+     - 8700.00
+
+
+Note that the dimensions do not take into account elevation due to the wheels.
+
+.. note::
+
+    See also :assets:`vehicles/vehicle_definitions_list.yaml` and `truck classifications <https://en.wikipedia.org/wiki/Truck_classification>`.
+
+
+Specifying vehicle definitions
+------------------------------
+
+Vehicles can be configured in a few different ways.
+
+
+Configuration file
+^^^^^^^^^^^^^^^^^^
+
+.. code-block:: ini
+
+    [assets]
+    default_vehicle_definitions_list = path/to/file.yaml
+
+.. note::
+
+    See also :ref:`engine_configuration`.
+
+
+Environment variable
+^^^^^^^^^^^^^^^^^^^^
+
+Setting ``SMARTS_ASSETS_DEFAULT_VEHICLE_DEFINITIONS_LIST`` will cause ``SMARTS`` to use the given vehicle definitions file as the default vehicle definitions.
+
+.. note::
+
+    See also :ref:`engine_configuration`.
+
+
+Scenario
+^^^^^^^^
+
+Including a ``vehicle_definitions_list.yaml`` in your scenario will cause ``SMARTS`` to use those vehicle definitions for the duration of the scenario.
+
+.. code-block:: bash
+
+    $ tree scenarios/sumo/loop
+    scenarios/sumo/loop
+    ├── build
+    │   └── ...
+    ├── map.net.xml
+    ├── rerouter.add.xml
+    ├── scenario.py
+    └── vehicle_definitions_list.yaml # <---
+
+
+Usage
+-----
+
+Agent interface
+^^^^^^^^^^^^^^^
+
+.. code-block:: python
+
+    from smarts.core.agent_interface import AgentInterface
+    from smarts.core.controllers import ActionSpaceType
+
+    agent_interface = AgentInterface(
+        max_episode_steps=1000,
+        waypoint_paths=True,
+        vehicle_class="pickup",
+        action=ActionSpaceType.Continuous,
+    )
+
+.. note::
+
+    See also :ref:`agent`.
+
+
+Syntax
+------
+
+A vehicle can be composed in the following way:
+
+
+.. code-block:: yaml
+
+    # vehicle_definitions_list.yaml
+    f150: /usr/home/dev/vehicles/f150.yaml
+
+
+.. code-block:: yaml
+
+    # /usr/home/dev/vehicles/f150.yaml
+    model: Ford F-150
+    type: truck
+    controller_params: ${SMARTS_ASSETS_PATH}/vehicles/controller_params/generic_pickup_truck.yaml
+    chassis_params: ${SMARTS_ASSETS_PATH}/vehicles/chassis_params/generic_pickup_truck.yaml
+    dynamics_model: /usr/home/dev/vehicles/f150_loaded.urdf
+    visual_model: /usr/home/dev/vehicles/f150.glb
+    tire_params: null # ${SMARTS_ASSETS_PATH}/vehicles/tire_params/base_tire_parameters.yaml
+
+
+.. note::
+
+    See :ref:`engine_configuration` for details about how YAML is resolved.
+
+
+.. note::
+
+    See :assets:`vehicles/controller_params/generic_pickup_truck.yaml` and :assets:`vehicles/chassis_params/generic_pickup_truck.yaml`.
+

envision/server.py

@@ -46,7 +46,7 @@
         "Missing dependencies for Envision. Install them using the command `pip install -e .[envision]` at the source directory."
     )
 
-import smarts.core.models
+import smarts.assets.vehicles.visual_model
 from envision.web import dist as web_dist
 from smarts.core.utils.file import path2hash
 
@@ -473,11 +473,13 @@ def initialize(self):
     async def get(self, id_):
         """Serve the requested model geometry."""
         if id_ not in self._path_map or not pkg_resources.is_resource(
-            smarts.core.models, self._path_map[id_]
+            smarts.assets.vehicles.visual_model, self._path_map[id_]
         ):
             raise tornado.web.HTTPError(404, f"GLB Model `{id_}` not found.")
 
-        with pkg_resources.path(smarts.core.models, self._path_map[id_]) as path:
+        with pkg_resources.path(
+            smarts.assets.vehicles.visual_model, self._path_map[id_]
+        ) as path:
             await self.serve_chunked(path)