Skip to content

Hand Gesture Detection with TF Lite using Raspberry Pi | Hardware Laboratory Course, Spring 2023

License

Notifications You must be signed in to change notification settings

mofayezi/HW-Lab-Spring23

Repository files navigation

Hand Gesture Detection using Raspberry Pi

This repository contains files of the Hardware Laboratory course, Spring 2023. Although it is more convenient to use the MediaPipe library for hand keypoint detection we do it from scratch and use a pre-trained tf-lite model.

overview

Requirements

Below is a list of the components you will need to get this system up.

  • Raspberry Pi 3 Model B
  • Raspberry Pi Camera Module
  • Micro SD Card
  • Power Supply
  • Monitor (Optional - Otherwise you can use a VNC Viewer)

Initial Set-Up

First, you need to install the operating system on the Raspberry Pi. It is recommended to use the Raspberry Pi Imager to install an operating system onto your SD card. You can follow this tutorial from the official website. Note: Our code has been tested on a 64-bit OS version.

After installing the OS, you need to install the required packages on the system. You can use the following command to install OpenCV and TensorFlow Lite. If you encounter an error while installing the TensorFlow Lite library check the version of python installed on the system and change the tflite_runtime version respectfully.

bash setup.sh

The client code requires Python 3.8 or later. The file requirements.txt contains the full list of required Python modules.

pip install -r requirements.txt

Running the Code

Our system will be able to receive gestures from the Raspberry device by establishing a socket connection between the two. This will enable seamless and efficient communication between the two devices. To begin, it is necessary to modify the IP addresses in both handler.py and client.py.

In order to initiate the server on your Raspberry device, you will need to execute the manager.py file. Additionally, on your client system, you should run the client.py file. This will allow for proper communication between the server and client devices. Finally, you can see the gestures being applied on the client side.

Note: Currently, only 5 gestures are supported:

  • Move Mouse
  • Click Mouse
  • Double Click Mouse
  • Right Click Mouse
  • Take ScreenShot

The gestures have been defined on the handler.py for the server and on the client.py for the actions on the client.

Report

You can find the final report (in Persian) and descriptions of various modules at report.pdf. The tf-lite checkpoints of our hand detection models have been put in the checkpoints directory.

About

Hand Gesture Detection with TF Lite using Raspberry Pi | Hardware Laboratory Course, Spring 2023

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published