Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix:libcamera support #1

Merged
merged 9 commits into from
Nov 17, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions MANIFEST.in
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
recursive-include locale *
include *.txt
include *.wav
50 changes: 49 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,6 @@ The `settings.json` file allows you to configure the behavior of the Camera Skil
| `play_sound` | Boolean | `true` | Whether to play a sound when a picture is taken. |
| `camera_sound_path` | String | `camera.wav` | Path to the sound file to play when taking a picture. |
| `pictures_folder` | String | `~/Pictures` | Directory where pictures are saved. |
| `keep_camera_open` | Boolean | `false` | Whether to keep the camera stream open after taking a picture. |

### Example `settings.json`

Expand All @@ -33,3 +32,52 @@ The `settings.json` file allows you to configure the behavior of the Camera Skil
"keep_camera_open": false
}
```


### Additional Steps for Raspberry Pi Users

If you plan to use this skill on a Raspberry Pi, it requires access to the `libcamera` package for the Picamera2 library to function correctly. Due to how `libcamera` is installed on the Raspberry Pi (system-wide), additional steps are necessary to ensure compatibility when using a Python virtual environment (venv).

In these examples we use the default .venv location from ovos-installer, `~/.venvs/ovos`, adjust as needed

#### **Steps to Enable `libcamera` in Your Virtual Environment**

1. **Install Required System Packages**
Before proceeding, ensure that `libcamera` and its dependencies are installed on your Raspberry Pi. Run the following commands:
```bash
sudo apt install -y python3-libcamera python3-kms++ libcap-dev
```

2. **Modify the Virtual Environment Configuration**
If you already have a virtual environment set up, enable access to system-wide packages by modifying the `pyvenv.cfg` file in the virtual environment directory:
```bash
nano ~/.venvs/ovos/pyvenv.cfg
```

Add or update the following line:
```plaintext
include-system-site-packages = true
```

Save the file and exit.

3. **Verify Access to `libcamera`**
Activate your virtual environment:
```bash
source ~/.venvs/ovos/bin/activate
```

Check if the `libcamera` package is accessible:
```bash
python3 -c "import libcamera; print('libcamera is accessible')"
```

#### **Why Are These Steps Necessary?**
The `libcamera` package is not available on PyPI and is installed system-wide on the Raspberry Pi. Virtual environments typically isolate themselves from system-wide Python packages, so these adjustments allow the skill to access `libcamera` while still benefiting from the isolation provided by a venv.

#### **Notes**
- These steps are specific to Raspberry Pi users who want to utilize the Picamera2 library for camera functionality. On other platforms, the skill defaults to using OpenCV, which does not require additional configuration.
- Ensure that `libcamera` is installed on your Raspberry Pi before attempting these steps. You can test this by running:
```bash
libcamera-still --version
```
125 changes: 92 additions & 33 deletions __init__.py
Original file line number Diff line number Diff line change
@@ -1,8 +1,10 @@
import os.path
import random
import time
from os.path import dirname, exists, join

import cv2
import numpy as np
from imutils.video import VideoStream
from ovos_utils import classproperty
from ovos_utils.log import LOG
Expand All @@ -12,6 +14,73 @@
from ovos_workshop.skills import OVOSSkill


class Camera:
# TODO - move to a PHAL plugin so camera
# can be shared across components
def __init__(self, camera_index=0):
self.camera_index = camera_index
self._camera = None
self.camera_type = self.detect_camera_type()

@staticmethod
def detect_camera_type() -> str:
"""Detect if running on Raspberry Pi with libcamera or desktop."""
try:
# only works in rpi
import libcamera
return "libcamera"
except:
return "opencv"
JarbasAl marked this conversation as resolved.
Show resolved Hide resolved

def open(self):
"""Open camera based on detected type."""
if self.camera_type == "libcamera":
try:
from picamera2 import Picamera2
self._camera = Picamera2()
self._camera.start()
LOG.info("libcamera initialized")
except Exception as e:
LOG.error(f"Failed to start libcamera: {e}")
return None
elif self.camera_type == "opencv":
try:
self._camera = VideoStream(self.camera_index)
if not self._camera.stream.grabbed:
self._camera = None
raise ValueError("OpenCV Camera stream could not be started")
self._camera.start()
except Exception as e:
LOG.error(f"Failed to start OpenCV camera: {e}")
return None
return self._camera

def get_frame(self) -> np.ndarray:
if self.camera_type == "libcamera":
return self._camera.capture_array()
else:
return self._camera.get()
JarbasAl marked this conversation as resolved.
Show resolved Hide resolved

def close(self):
"""Close the camera."""
if self._camera:
if self.camera_type == "libcamera":
self._camera.close()
elif self.camera_type == "opencv":
self._camera.stop()
self._camera = None

def __enter__(self):
"""Enter the context and open the camera."""
if self.open() is None:
raise RuntimeError("Failed to open the camera")
return self

def __exit__(self, exc_type, exc_value, traceback):
"""Exit the context and close the camera."""
self.close()


class WebcamSkill(OVOSSkill):

@classproperty
Expand All @@ -31,27 +100,7 @@ def initialize(self):
self.settings["video_source"] = 0
if "play_sound" not in self.settings:
self.settings["play_sound"] = True
self._camera = None

def open_camera(self) -> VideoStream:
if self._camera is None:
try:
LOG.info("initializing camera stream")
self._camera = VideoStream(self.settings.get("video_source", 0))
if not self._camera.stream.grabbed:
self._camera = None
raise ValueError("Camera stream could not be started")
self._camera.start()
except Exception as e:
LOG.error(f"Failed to open camera: {e}")
return None
return self._camera

def close_camera(self):
if self._camera:
LOG.info("closing camera stream")
self._camera.stop()
self._camera = None
self.camera = Camera(self.settings.get("video_source", 0))

@property
def pictures_folder(self) -> str:
Expand All @@ -68,18 +117,28 @@ def play_camera_sound(self):

@intent_handler("take_picture.intent")
def handle_take_picture(self, message):
cam = self.open_camera()
if cam is None:
try:
with self.camera as cam:
self.speak_dialog("get_ready", wait=True)
# need time to Allow sensor to stabilize
self.gui.show_text("3")
self.speak("3", wait=True)
self.gui.show_text("2")
self.speak("2", wait=True)
self.gui.show_text("1")
self.speak("1", wait=True)
self.play_camera_sound()
self.gui.clear()
frame = self.camera.get_frame()
pic_path = join(self.pictures_folder, time.strftime("%Y-%m-%d_%H-%M-%S") + ".jpg")
cv2.imwrite(pic_path, frame)
self.gui.show_image(pic_path)
if random.choice([True, False]):
self.speak_dialog("picture")
except RuntimeError as e:
LOG.error(e)
self.speak_dialog("camera_error")
return
self.play_camera_sound()
frame = cam.get().copy()
pic_path = join(self.pictures_folder, time.strftime("%Y-%m-%d_%H-%M-%S") + ".jpg")
cv2.imwrite(pic_path, frame)
self.gui.show_image(pic_path)
self.speak_dialog("picture")
if not self.settings.get("keep_camera_open"):
self.close_camera()

def shutdown(self):
self.close_camera()
# just in case
self.camera.close()
1 change: 1 addition & 0 deletions locale/en-us/get_ready.dialog
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
say cheese
1 change: 1 addition & 0 deletions requirements.txt
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
imutils
ovos_workshop>=0.0.12
opencv-python
picamera2
2 changes: 1 addition & 1 deletion setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@
def find_resource_files():
resource_base_dirs = ("locale",)
base_dir = path.dirname(__file__)
package_data = ["*.json"]
package_data = ["*.json", "*.wav"]
for res in resource_base_dirs:
if path.isdir(path.join(base_dir, res)):
for (directory, _, files) in walk(path.join(base_dir, res)):
Expand Down
1 change: 1 addition & 0 deletions translations/en-us/dialogs.json
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
"camera_error.dialog": [
"I couldn't access the camera."
],
"get_ready.dialog": ["say cheese"],
"picture.dialog": [
"picture taken",
"i like taking pictures"
Expand Down
Loading