How to setup

Run on machine or RPI

1. Run install scripts in the shells folder.
2. Follow readme in shells/pyyolo folder to install pyyolo.
3. Install imutils through pip.
4. Mount intern account from VRL server as /home/carlos/vrlserver
5. Download homography folder from intern server for corresponding camera set up (1, 2, or 3) or calculate new homography
6. Optional: Create bash alias for running src/showframe.py and src/saveframes.py in order to view and label recordings for training.

Calculate Homography

Run src/get_homographies.py in order to calculate homographies. The camera must be pointed at a 3 x 3 circle grid where there is a distinct color and temperature change. I used a wood grid with a shirt behind and used a hair dryer to warm the short so the thermal camera could register the heat difference. get_homographies.py will start by displaying frames waiting for you to press space to capture a frame and detect the circle grid. The first iteration you want the camera to be close to the grid (homographies vary by the distance so we start from close and move to far). Once you have captured a frame, you select the 3x3 grid by dragging on the rgb window and press s to save and then repeat with the thermal image. Finally you will adjust the sliders until the circles are detected in both images (rgb and ir). If circles are detected and the detected orientation is the same, press s and the homography for that distance will be saved. If you did not get a good image you can press esc to try recapturing for that distance. Continue until homogrpahies for 5 distances have been saved.

Apply Homography

Code for appling homographies between the rgb/depth perspective and the ir perspective are in src/use_homographies. In order there are two types of functions you can use: ir_to_rgb/rgb_to_ir which will just convert and x,y coordinate and rgb_conv/ir_conv which will warp an entire image. Example code (convert depth and rgb to perspective of thermal):

homog = get_pos()

depthm = np.mean(depth_map[70:170,110:210]) #Get the average depth from center of image to find correct homography

depth = homog.rgb_conv(depth,depthm) #IR view

rgb = homog.rgb_conv(rgb,depthm) #IR view