Skip to content

Commit

Permalink
refactor!:adopt PHAL plugin (#8)
Browse files Browse the repository at this point in the history
* refactor!:adopt PHAL plugin

https://github.com/OpenVoiceOS/ovos-PHAL-plugin-camera

* refactor!:adopt PHAL plugin

https://github.com/OpenVoiceOS/ovos-PHAL-plugin-camera

* session support

* dont expand home dir , let phal do it so it works in satellites

* better session handling

* warm up time

* fixdialogs

* dont open camera

* fix gui path

* fix
  • Loading branch information
JarbasAl authored Jan 6, 2025
1 parent 8e063c0 commit 37dd762
Show file tree
Hide file tree
Showing 10 changed files with 92 additions and 170 deletions.
2 changes: 1 addition & 1 deletion .github/workflows/publish_stable.yml
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ jobs:
- name: Setup Python
uses: actions/setup-python@v1
with:
python-version: 3.8
python-version: "3.10"
- name: Install Build Tools
run: |
python -m pip install build wheel
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/release_workflow.yml
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ jobs:
- name: Setup Python
uses: actions/setup-python@v1
with:
python-version: 3.8
python-version: "3.10"
- name: Install Build Tools
run: |
python -m pip install build wheel
Expand Down
56 changes: 3 additions & 53 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
# Camera Skill

Camera skill for OpenVoiceOS
Camera skill for OpenVoiceOS, needs the companion [ovos-PHAL-plugin-camera](https://github.com/OpenVoiceOS/ovos-PHAL-plugin-camera)

## Description

This skill allows you to take pictures using a connected webcam. You can configure various settings to customize its behavior.
This skill allows you to ask to take pictures using a connected webcam. You can configure various settings to customize its behavior.

## Examples

Expand All @@ -16,7 +16,6 @@ The `settings.json` file allows you to configure the behavior of the Camera Skil

| Setting Name | Type | Default | Description |
|----------------------|----------|---------------|-----------------------------------------------------------------------------|
| `video_source` | Integer | `0` | Specifies the camera to use. `0` is the default system webcam. |
| `play_sound` | Boolean | `true` | Whether to play a sound when a picture is taken. |
| `camera_sound_path` | String | `camera.wav` | Path to the sound file to play when taking a picture. |
| `pictures_folder` | String | `~/Pictures` | Directory where pictures are saved. |
Expand All @@ -25,59 +24,10 @@ The `settings.json` file allows you to configure the behavior of the Camera Skil

```json
{
"video_source": 0,
"play_sound": true,
"camera_sound_path": "/path/to/camera.wav",
"pictures_folder": "/home/user/Pictures",
"keep_camera_open": false
"pictures_folder": "/home/user/Pictures"
}
```


### Additional Steps for Raspberry Pi Users

If you plan to use this skill on a Raspberry Pi, it requires access to the `libcamera` package for the Picamera2 library to function correctly. Due to how `libcamera` is installed on the Raspberry Pi (system-wide), additional steps are necessary to ensure compatibility when using a Python virtual environment (venv).

In these examples we use the default .venv location from ovos-installer, `~/.venvs/ovos`, adjust as needed

#### **Steps to Enable `libcamera` in Your Virtual Environment**

1. **Install Required System Packages**
Before proceeding, ensure that `libcamera` and its dependencies are installed on your Raspberry Pi. Run the following commands:
```bash
sudo apt install -y python3-libcamera python3-kms++ libcap-dev
```

2. **Modify the Virtual Environment Configuration**
If you already have a virtual environment set up, enable access to system-wide packages by modifying the `pyvenv.cfg` file in the virtual environment directory:
```bash
nano ~/.venvs/ovos/pyvenv.cfg
```

Add or update the following line:
```plaintext
include-system-site-packages = true
```

Save the file and exit.

3. **Verify Access to `libcamera`**
Activate your virtual environment:
```bash
source ~/.venvs/ovos/bin/activate
```

Check if the `libcamera` package is accessible:
```bash
python3 -c "import libcamera; print('libcamera is accessible')"
```

#### **Why Are These Steps Necessary?**
The `libcamera` package is not available on PyPI and is installed system-wide on the Raspberry Pi. Virtual environments typically isolate themselves from system-wide Python packages, so these adjustments allow the skill to access `libcamera` while still benefiting from the isolation provided by a venv.

#### **Notes**
- These steps are specific to Raspberry Pi users who want to utilize the Picamera2 library for camera functionality. On other platforms, the skill defaults to using OpenCV, which does not require additional configuration.
- Ensure that `libcamera` is installed on your Raspberry Pi before attempting these steps. You can test this by running:
```bash
libcamera-still --version
```
158 changes: 53 additions & 105 deletions __init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,86 +2,13 @@
import random
import time
from os.path import dirname, exists, join

import cv2
import numpy as np
from imutils.video import VideoStream
from ovos_bus_client.message import Message
from ovos_bus_client.session import SessionManager, Session
from ovos_utils import classproperty
from ovos_utils.log import LOG
from ovos_utils.process_utils import RuntimeRequirements

from ovos_workshop.decorators import intent_handler
from ovos_workshop.skills import OVOSSkill


class Camera:
# TODO - move to a PHAL plugin so camera
# can be shared across components
def __init__(self, camera_index=0):
self.camera_index = camera_index
self._camera = None
self.camera_type = self.detect_camera_type()

@staticmethod
def detect_camera_type() -> str:
"""Detect if running on Raspberry Pi with libcamera or desktop."""
try:
# only works in rpi
import libcamera
return "libcamera"
except:
return "opencv"

def open(self):
"""Open camera based on detected type."""
if self.camera_type == "libcamera":
try:
from picamera2 import Picamera2
self._camera = Picamera2()
self._camera.start()
LOG.info("libcamera initialized")
except Exception as e:
LOG.error(f"Failed to start libcamera: {e}")
return None
elif self.camera_type == "opencv":
try:
self._camera = VideoStream(self.camera_index)
if not self._camera.stream.grabbed:
self._camera = None
raise ValueError("OpenCV Camera stream could not be started")
self._camera.start()
except Exception as e:
LOG.error(f"Failed to start OpenCV camera: {e}")
return None
return self._camera

def get_frame(self) -> np.ndarray:
if self.camera_type == "libcamera":
frame = self._camera.capture_array() # In RGB format
# Convert RGB to BGR for OpenCV compatibility
frame = cv2.cvtColor(frame, cv2.COLOR_RGB2BGR)
return frame
else:
return self._camera.get()

def close(self):
"""Close the camera."""
if self._camera:
if self.camera_type == "libcamera":
self._camera.close()
elif self.camera_type == "opencv":
self._camera.stop()
self._camera = None

def __enter__(self):
"""Enter the context and open the camera."""
if self.open() is None:
raise RuntimeError("Failed to open the camera")
return self

def __exit__(self, exc_type, exc_value, traceback):
"""Exit the context and close the camera."""
self.close()
from ovos_utils.log import LOG


class WebcamSkill(OVOSSkill):
Expand All @@ -99,17 +26,20 @@ def runtime_requirements(self):
no_gui_fallback=True)

def initialize(self):
if "video_source" not in self.settings:
self.settings["video_source"] = 0
self.sess2cam = {}
if "play_sound" not in self.settings:
self.settings["play_sound"] = True
self.camera = Camera(self.settings.get("video_source", 0))
self.add_event("ovos.phal.camera.pong", self.handle_pong)
self.bus.emit(Message("ovos.phal.camera.ping"))

def handle_pong(self, message: Message):
sess = SessionManager.get(message)
LOG.info(f"Camera available for session: {sess.session_id}")
self.sess2cam[sess.session_id] = True

@property
def pictures_folder(self) -> str:
folder = os.path.expanduser(self.settings.get("pictures_folder", "~/Pictures"))
os.makedirs(folder, exist_ok=True)
return folder
return self.settings.get("pictures_folder", "~/Pictures")

def play_camera_sound(self):
if self.settings["play_sound"]:
Expand All @@ -118,30 +48,48 @@ def play_camera_sound(self):
if exists(s):
self.play_audio(s, instant=True)

def sess_has_camera(self, message) -> bool:
sess = SessionManager.get(message)
# check if this session has the camera PHAL plugin installed
has_camera = (self.sess2cam.get(sess.session_id) is not None or
self.bus.wait_for_response(message.forward("ovos.phal.camera.ping"),
"ovos.phal.camera.pong",
timeout=0.5))
LOG.debug(f"has camera: {has_camera}")
if has_camera and sess.session_id not in self.sess2cam:
self.sess2cam[sess.session_id] = True
return has_camera

@intent_handler("have_camera.intent")
def handle_camera_check(self, message):
if self.sess_has_camera(message):
self.speak_dialog("camera_yes")
else:
self.speak_dialog("camera_error")

@intent_handler("take_picture.intent")
def handle_take_picture(self, message):
try:
with self.camera as cam:
self.speak_dialog("get_ready", wait=True)
# need time to Allow sensor to stabilize
self.gui.show_text("3")
self.speak("3", wait=True)
self.gui.show_text("2")
self.speak("2", wait=True)
self.gui.show_text("1")
self.speak("1", wait=True)
self.play_camera_sound()
self.gui.clear()
frame = self.camera.get_frame()
pic_path = join(self.pictures_folder, time.strftime("%Y-%m-%d_%H-%M-%S") + ".jpg")
cv2.imwrite(pic_path, frame)
self.gui.show_image(pic_path)
if random.choice([True, False]):
self.speak_dialog("picture")
except RuntimeError as e:
LOG.error(e)
if not self.sess_has_camera(message):
self.speak_dialog("camera_error")
return

self.speak_dialog("get_ready", wait=True)

if self.settings.get("countdown"):
# need time to Allow sensor to stabilize
self.gui.show_text("3")
self.speak("3", wait=True)
self.gui.show_text("2")
self.speak("2", wait=True)
self.gui.show_text("1")
self.speak("1", wait=True)

pic_path = join(self.pictures_folder, time.strftime("%Y-%m-%d_%H-%M-%S") + ".jpg")
self.bus.emit(message.forward("ovos.phal.camera.get", {"path": pic_path}))

self.play_camera_sound()

def shutdown(self):
# just in case
self.camera.close()
self.gui.clear()
self.gui.show_image(os.path.expanduser(pic_path))
if random.choice([True, False]):
self.speak_dialog("picture")
5 changes: 4 additions & 1 deletion locale/en-us/camera_error.dialog
Original file line number Diff line number Diff line change
@@ -1 +1,4 @@
I couldn't access the camera.
I can't access the camera.
I couldn't detect a camera.
It seems there is no camera available.
I couldn't find a working camera.
3 changes: 3 additions & 0 deletions locale/en-us/camera_yes.dialog
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
Yes, a camera is available.
My camera is ready to use.
The camera is functioning correctly.
6 changes: 6 additions & 0 deletions locale/en-us/have_camera.intent
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
do you have a camera
can you access the camera
is a camera available
is the camera working
can I use the camera
is there a camera
3 changes: 0 additions & 3 deletions requirements.txt
Original file line number Diff line number Diff line change
@@ -1,4 +1 @@
imutils
ovos_workshop>=0.0.12
opencv-python
picamera2; sys_platform == 'linux' and platform_machine == 'armv7l' or platform_machine == 'aarch64'
15 changes: 11 additions & 4 deletions translations/en-us/dialogs.json
Original file line number Diff line number Diff line change
@@ -1,10 +1,17 @@
{
"camera_error.dialog": [
"I couldn't access the camera."
"I can't access the camera."
],
"camera_yes.dialog": [
"My camera is ready to use.",
"The camera is functioning correctly.",
"Yes, a camera is available."
],
"get_ready.dialog": [
"say cheese"
],
"get_ready.dialog": ["say cheese"],
"picture.dialog": [
"picture taken",
"i like taking pictures"
"i like taking pictures",
"picture taken"
]
}
12 changes: 10 additions & 2 deletions translations/en-us/intents.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,14 @@
{
"have_camera.intent": [
"can I use the camera",
"can you access the camera",
"do you have a camera",
"is a camera available",
"is the camera working",
"is there a camera"
],
"take_picture.intent": [
"take a picture",
"take a photo"
"take a photo",
"take a picture"
]
}

0 comments on commit 37dd762

Please sign in to comment.