Running inference

Inference Functions:

• RGB|d|t human detection (process/human_detector.py)
• YOLO object detection with thermal interaction confirmation (coming soon)
• Tri-stream (RGB|d|t) activity detection CNN

human_detector

To run the human detector on a frame, use human_detector(rgb, depth, ir) with rgb, depth, and ir being the raw numpy frames without homography applied. It returns bounding boxes for a strong match (confirmed from all three modalities), bounding boxes from HOG rgb detector, bounding boxes from depth&ir human detection, bounding boxes for depth&ir false positive, and an rgb image with homography applied and bounded boxes printed on. This detector has two detection methods:

• RGB: HOG person detection
• d|t: depth map segmentation and thermal human confirmation (checks if depth segment contains a significant heat signature which could likely be human)

This detector was designed to be able to attempt improve detection over plain RGB detection under dark conditions by using depth and thermal.