This project demonstrates a virtual glasses augmentation application using OpenCV, Mediapipe, and Pyrender. The application detects faces in real-time video frames captured from the webcam and overlays a 3D glasses model on the detected face.
- Python 3.x
- OpenCV
- Mediapipe
- Numpy
- Pyrender
- Trimesh
-
Clone the repository: git clone https://github.com/vrk7/3D-Eyeglass-Frame-Augmentation-on-Camera-Feed.git
-
Install the required dependencies: pip install -r requirements.txt
-
In case any error is coming, being the sudo user intsall the below:
sudo apt-get update sudo apt-get install libgl1-mesa-glx sudo apt-get install libglm-dev
-
Place your 3D glasses model file (e.g.,
oculos.obj) in the project directory. -
Run the script: python glasses_augmentation.py
-
The application will open a window displaying the real-time video feed from your webcam.
-
The glasses will be automatically overlaid on the detected face.
-
Press 'q' to quit the application.
-
The script loads the 3D glasses model using Trimesh and creates a Pyrender mesh from it.
-
It initializes the Mediapipe Face Mesh model for face detection and landmark estimation.
-
A Pyrender scene is created with a camera and the glasses node.
-
The script captures video frames from the webcam using OpenCV.
-
For each frame:
- The frame is flipped horizontally for a mirror effect.
- The frame is converted to RGB color space.
- Faces are detected using the Mediapipe Face Mesh model.
- For each detected face:
- The eye coordinates are extracted from the face landmarks.
- The center of the eyes is calculated.
- The glasses position and orientation are set based on the eye coordinates.
- The scene is rendered using Pyrender.
- The rendered glasses are overlaid on the frame using a mask.
-
The resulting frame with the overlaid glasses is displayed in a window.
-
The process continues until the user presses 'q' to quit the application.