Overview

We use the donkeycar stack on our kart. Donkeycar provides the software architecture for the kart. Donkeycar uses different parts which have inputs and outputs. These parts can be added to a vehicle. The vehicle loop runs the parts in order and stores and modifies data in the vehicle memory.

Here is more information on donkeycar: donkeycar

Here is a link to our current software architecture: github link

The kart currently has these parts:

• Frame Publisher
• Heartbeat
• Segment Model
• Logger
• UART Driver

Frame Publisher

Code Link

Inputs: N/A
Outputs:

• sensors/ZED/RGB/Left
• sensors/ZED/RGB/Right

This part reads in and publishes the current left and right frames on the ZED camera.

Heartbeat

Code Link

Inputs: N/A
Outputs:

• safety/heartbeat

Determines if there is an active connection between the kart and the host. If the connection fails, the kart will automatically stop.

Segment Model

Code Link

Inputs:

• sensors/ZED/RGB/Left

Outputs:

• perception/segmentedTrack
• controls/steering
• controls/throttle
• centroid

Runs the segmentation model on the inputted image from the frame publisher. Calculates the centroid of the detected road from the segmentation model. Determines steering and throttle values based on the location of the centroid. Uses a PID controller to follow the centroid.

Logger

Code Link

Inputs:

• sensors/ZED/RGB/Left
• perception/segmentedTrack
• controls/steering
• controls/throttle
• centroid

Outputs: N/A

Logs the current image, segmented image, centroid, steering, and throttle values. Saves the images in a folder and stores the image paths and other data in a CSV file.

UART Driver

Code Link

Inputs:

• controls/steering
• controls/throttle
• safety/heartbeat

Outputs: N/A

Writes the throttle and steering commands over serial to the STM which actuates the kart. Maps the inputted values from (-1,1) to the expected values for serial communication.

Main Loop

Code Link

1. Instantiates the donkey car vehicle.
2. Adds the heartbeat, frame publisher, segment model, and logger parts to the vehicle.
3. Adds the web controller to the vehicle.
4. Adds the UART driver to the vehicle.
5. Starts the drive loop.

The main loop instantiates and adds all of the parts to the vehicle. It executes each part sequentially in the order they were added to the vehicle. Each loop executes the parts in the same order.

Project

Create the parts and drive loop for simple object detection.

It should contain these parts:

• Image publisher
• Object Detection
• Logger

It also needs a main loop

Installing Donkeycar

Windows: https://docs.robocarstore.com/guide/host_pc/setup_windows/
Other: https://docs.donkeycar.com/guide/install_software/#step-1-install-software-on-host-pc

Image Publisher

Inputs: N/A
Outputs:

• image

This part should read images from your webcam and return them in the run function. The run function is what runs each time the part is called in the drive loop. It takes in inputs and returns outputs in the vehicle memory.

Outline:

• Create a file called image_publisher.py in the parts directory.
• Imports
  • cv2
• Create a class called Image_Publisher
• Create an __init__() method
  • initialize a cv2 VideoCapture()
• Create a run() method
  • read a frame from the VideoCapture()
  • Return the frame if it is not null

Completed Code

Object Detection

Inputs:

• image

Outputs:

• image_cv
• object_x
• object_y
• contour_area

This part should detect objects on the inputted image. It should identify objects in the image and return the coordinates of the largest contour by area. It should also return the area of the largest contour.

Outline:

• Create a file called image_cv.py in the parts directory
• Imports
  • cv2
• Create a class called Object_Detection
• Create a run() method which takes in an image
  • check if the image is not null
  • convert the image to grayscale
  • detect the contours in the grayscale image
  • find and draw largest contour by area
    • Calculate and draw the centroid of the largest contour
  • Return the modified image, the contour x coordinate, the contour y coordinate, and contour area

Completed Code

Logger

Inputs:

• object_x
• object_y
• contour_area

Outputs:

• N/A

Writes the current timestamp, object_x, object_y, and contour_area to a CSV file.

Outline:

• Imports
  • datetime
  • csv
• Create a file called log.py
• Create a class called Logger
• Create an __init__() method
  • Open a file object with filename “logger.csv” in writemode with newline=’’
  • Create a csv writer using the file object
  • Write the fields of the csv: ‘timestamp’, ‘object_x’, ‘object_y’, ‘contour_area’
• Create a run() method which takes in object_x, object_y, and contour_area
  • Check that the inputted values are not null
  • Get the current timestamp
  • Write the row to the csv file: timestamp, object_x, object_y, contour_area

Completed Code

Main Loop

Instantiates the vehicle and adds parts to it. The program runs in an infinite loop, running each part in a sequential order.

Outline:

• Imports
  • donkeycar
    • This will be dependent on the path which donkeycar was cloned into your directory
  • Image_Publisher
  • Object_Detection
  • Logger
    • these will be imported from the files in your parts directory
• if __name__ == “__main__”:
  • Create a donkeycar vehicle
  • instantiate an Image_Publisher() and add it to the vehicle.
    • inputs=[]
    • outputs=[‘image’]
  • instantiate an Object_Detection() and add it to the vehicle
    • inputs=[‘image]
    • outputs=[‘image_cv’, ‘object_x’, ‘object_y’, ‘contour_area’]
  • instantiate a Logger() and add it to the vehicle
    • inputs=[‘object_x’, ‘object_y’, ‘contour_area’]
    • outputs=[]
  • Start the vehicle with rate_hz=30

Completed Code