Running inference

Inference Functions:

• RGB|d|t human detection (process/human_detector.py)
• YOLO object detection with thermal interaction confirmation (coming soon)
• Tri-stream (RGB|d|t) activity detection CNN (initial stages)

human_detector

To run the human detector on a frame, use human_detector(rgb, depth, ir) with rgb, depth, and ir being the raw numpy frames without homography applied. It returns bounding boxes for a strong match (HOG postive from rgb and thermal), bounding boxes from medium match (depth with either rgb or thermal HOG), bounding boxes from HOG rgb detector, bounding boxes from HOG thermal detector, bounding boxes from depth&ir human detection, and an rgb image with homography applied and bounded boxes printed on. This detector has three detection methods:

• RGB: HOG person detection
• t: HOG person detection
• d|t: depth map segmentation and thermal human confirmation (checks if depth segment contains a significant heat signature which could likely be human)

This detector was designed to be able to attempt improve detection over plain RGB detection under dark conditions by using depth and thermal.

interaction_detector

Being implemented in human_detector.py

activity_detector

CNN with temporal memory for activity recognition. Not yet implemented.