• Self Driving Car System Integration
• Team: Total Recall Dryvers
• Structure
  • Perception
  • Planning
  • Control
• Hardware Requirement
• Software Requirement
• Build
  • Without Docker
  • With Docker
• Real world testing

Self Driving Car System Integration

This is the final capstone project in Udacity's Self-Driving Car Nanodegree. In this project, we write code that will automatically drive a "Carla." The Carla is an actual self-driving car equipped with necessary sensors and the drive-by-wire module.

Testing our code on the Udacity simulator

Team: Total Recall Dryvers

Name	Email
Krishtof Korda	[email protected]
Kyung Mo Kweon	[email protected]
Jochen Mombach	[email protected]
Luca Venturi	[email protected]
Tim Aske	[email protected]

Structure

Our project consists of 3 modules: perception, planning, and control.

Perception

The perception module is mainly responsible for detecting roads and its environment. It is also responsible for detecting traffic lights where we used deep learning to classify the traffic lights into 4 different labels.

def get_classification(self, image):
    """Determines the color of the traffic light in the image
    Args:
        image (cv::Mat): image containing the traffic light

    Returns:
        int: ID of traffic light color (specified in styx_msgs/TrafficLight)
    """
    # Used Keras to detect label
    self.predicted_label =  self.ldac.infer(ImageWrap(image), annotate=True, desired_labels=["Red", "Green", "Yellow"])

    # Convert the label to the ID such that the SDC can understand
    label_to_id_map = {"Green": 2, "Yellow": 1, "Red": 0, "Off": 4}
    self.COLORID = label_to_id_map[self.predicted_label]

    return TrafficLight.self.COLORID

More information can be found in tl detector directory.

Planning

The planning module is responsible for generating next trajectories of the vehicle. The main idea is that the car look ahead into future positions and also make sure it plans accordingly when it encounters traffic lights. For example, it should decelerate when there is a traffic light in RED.

Source code can be found in the Waypoint Updater directory.

Control

The Carla is equipped with a drive-by-wire (dbw) system, meaning the throttle, brake, and steering have electronic control Thus, we need to connect our code to the DBW system of the vehicle. It receives information such as current velocity and dbw switch(if the car is under the DBW or human driver). It then publishes throttle, steering, and brake commands.

More information can be found in the Twist Controller directory.

Hardware Requirement

The Carla has the following hardware spec

• 31.4 GiB Memory
• Intel Core i7-6700K CPU @ 4 GHz x 8
• TITAN X Graphics
• 64-bit OS

Software Requirement

Another requirement is that the project cannot include external libraries that are not included in this repository. To run this project, you will need

• Ubuntu 14.04+
• ROS
  • ROS Kinetic if you have Ubuntu 16.04.
  • ROS Indigo if you have Ubuntu 14.04.
• Dataspeed DBW
  • Use this option to install the SDK on a workstation that already has ROS installed: One Line SDK Install binary
• Download the Udacity Simulator.

Build

Without Docker

1. Clone the project repository

   git clone https://github.com/udacity/CarND-Capstone.git

2. Install python dependencies

   cd CarND-Capstone
   pip install -r requirements.txt

3. Make and run styx

   cd ros
   catkin_make
   source devel/setup.sh
   roslaunch launch/styx.launch

4. Run the simulator

   ./run_udacity_simulator

With Docker

Running

docker-compose up

Cleaning

docker-compose down -v
# or if you want to clean up entirely including the build image
docker-compose down -v --rmi all

Real world testing

1. Download training bag that was recorded on the Udacity self-driving car (a bag demonstraing the correct predictions in autonomous mode can be found here

2. Unzip the file

   unzip traffic_light_bag_files.zip

3. Play the bag file

   rosbag play -l traffic_light_bag_files/loop_with_traffic_light.bag

4. Launch your project in site mode

   cd CarND-Capstone/ros
   roslaunch launch/site.launch

5. Confirm that traffic light detection works on real life images