Skip to content

Commit

Permalink
Initial setup of repository (#2)
Browse files Browse the repository at this point in the history
# Initial setup of repository

## ♻️ Current situation & Problem
Currently, the repo is empty except for template code.


## ⚙️ Release Notes 
- Initial setup of the `SpeziSpeech` repo which previously was a target
in the [SpeziML](https://github.com/StanfordSpezi/SpeziML) repo
- Upgrade to Spezi v0.8
- Improved README and documentation via DocC


## 📚 Documentation
--


## ✅ Testing
--


## 📝 Code of Conduct & Contributing Guidelines 

By submitting creating this pull request, you agree to follow our [Code
of
Conduct](https://github.com/StanfordSpezi/.github/blob/main/CODE_OF_CONDUCT.md)
and [Contributing
Guidelines](https://github.com/StanfordSpezi/.github/blob/main/CONTRIBUTING.md):
- [X] I agree to follow the [Code of
Conduct](https://github.com/StanfordSpezi/.github/blob/main/CODE_OF_CONDUCT.md)
and [Contributing
Guidelines](https://github.com/StanfordSpezi/.github/blob/main/CONTRIBUTING.md).
  • Loading branch information
philippzagar authored Nov 17, 2023
1 parent 4980e52 commit f629d66
Show file tree
Hide file tree
Showing 21 changed files with 864 additions and 218 deletions.
18 changes: 14 additions & 4 deletions .github/workflows/build-and-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -21,8 +21,8 @@ jobs:
uses: StanfordSpezi/.github/.github/workflows/xcodebuild-or-fastlane.yml@v2
with:
runsonlabels: '["macOS", "self-hosted"]'
scheme: SpeziSpeech
artifactname: SpeziSpeech.xcresult
scheme: SpeziSpeech-Package
artifactname: SpeziSpeech-Package.xcresult
ios:
name: Build and Test iOS
uses: StanfordSpezi/.github/.github/workflows/xcodebuild-or-fastlane.yml@v2
Expand All @@ -31,9 +31,19 @@ jobs:
path: 'Tests/UITests'
scheme: TestApp
artifactname: TestApp.xcresult
ipados:
name: Build and Test iPadOS
uses: StanfordSpezi/.github/.github/workflows/xcodebuild-or-fastlane.yml@v2
with:
runsonlabels: '["macOS", "self-hosted"]'
path: 'Tests/UITests'
scheme: TestApp
resultBundle: TestAppiPadOS.xcresult
destination: 'platform=iOS Simulator,name=iPad mini (6th generation)'
artifactname: TestAppiPadOS.xcresult
uploadcoveragereport:
name: Upload Coverage Report
needs: [packageios, ios]
needs: [packageios, ios, ipados]
uses: StanfordSpezi/.github/.github/workflows/create-and-upload-coverage-report.yml@v2
with:
coveragereports: SpeziSpeech.xcresult TestApp.xcresult
coveragereports: SpeziSpeech-Package.xcresult TestApp.xcresult TestAppiPadOS.xcresult
3 changes: 2 additions & 1 deletion .spi.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,4 +11,5 @@ builder:
configs:
- platform: ios
documentation_targets:
- SpeziSpeech
- SpeziSpeechRecognizer
- SpeziSpeechSynthesizer
6 changes: 6 additions & 0 deletions CITATION.cff
Original file line number Diff line number Diff line change
Expand Up @@ -12,5 +12,11 @@ authors:
- family-names: "Schmiedmayer"
given-names: "Paul"
orcid: "https://orcid.org/0000-0002-8607-9148"
- family-names: "Philipp"
given-names: "Zagar"
orcid: "https://orcid.org/0009-0001-5934-2078"
- family-names: "Adrit"
given-names: "Rao"
orcid: "https://orcid.org/0000-0002-0780-033X"
title: "SpeziSpeech"
url: "https://github.com/StanfordSpezi/SpeziSpeech"
3 changes: 3 additions & 0 deletions CONTRIBUTORS.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,3 +14,6 @@ SpeziSpeech contributors
====================

* [Paul Schmiedmayer](https://github.com/PSchmiedmayer)
* [Vishnu Ravi](https://github.com/vishnuravi)
* [Philipp Zagar](https://github.com/philippzagar)
* [Adrit Rao](https://github.com/AdritRao)
17 changes: 11 additions & 6 deletions Package.swift
Original file line number Diff line number Diff line change
Expand Up @@ -18,25 +18,30 @@ let package = Package(
.iOS(.v17)
],
products: [
.library(name: "SpeziSpeech", targets: ["SpeziSpeech"])
.library(name: "SpeziSpeechRecognizer", targets: ["SpeziSpeechRecognizer"]),
.library(name: "SpeziSpeechSynthesizer", targets: ["SpeziSpeechSynthesizer"])
],
dependencies: [
.package(url: "https://github.com/StanfordSpezi/Spezi", .upToNextMinor(from: "0.8.0"))
],
targets: [
.target(
name: "SpeziSpeech",
name: "SpeziSpeechRecognizer",
dependencies: [
.product(name: "Spezi", package: "Spezi")
]
),
.target(
name: "SpeziSpeechSynthesizer",
dependencies: [
.product(name: "Spezi", package: "Spezi")
],
resources: [
.process("Resources")
]
),
.testTarget(
name: "SpeziSpeechTests",
dependencies: [
.target(name: "SpeziSpeech")
.target(name: "SpeziSpeechRecognizer"),
.target(name: "SpeziSpeechSynthesizer")
]
)
]
Expand Down
96 changes: 81 additions & 15 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,12 +14,12 @@ SPDX-License-Identifier: MIT
[![codecov](https://codecov.io/gh/StanfordSpezi/SpeziSpeech/graph/badge.svg?token=ufmRQvE0Cs)](https://codecov.io/gh/StanfordSpezi/SpeziSpeech)


Enable applications to connect to Speech devices.
Recognize and synthesize natural language speech.


## Overview

...
The Spezi Speech component provides an easy and convenient way to recognize (speech-to-text) and synthesize (text-to-speech) natural language content, facilitating seamless interaction with an application. It builds on top of Apple's [Speech](https://developer.apple.com/documentation/speech/) and [AVFoundation](https://developer.apple.com/documentation/avfoundation/) frameworks.


## Setup
Expand All @@ -32,19 +32,18 @@ You need to add the Spezi Speech Swift package to
[Swift package](https://developer.apple.com/documentation/xcode/creating-a-standalone-swift-package-with-xcode#Add-a-dependency-on-another-Swift-package).

> [!IMPORTANT]
> If your application is not yet configured to use Spezi, follow the [Spezi setup article](https://swiftpackageindex.com/stanfordspezi/spezi/documentation/spezi/initial-setup) setup the core Spezi infrastructure.
> If your application is not yet configured to use Spezi, follow the [Spezi setup article](https://swiftpackageindex.com/stanfordspezi/spezi/documentation/spezi/initial-setup) to setup the core Spezi infrastructure.
### 2. Configure the `SpeechRecognizer` and the `SpeechSynthesizer` in the Spezi `Configuration`

### 2. Register the Module

The `Speech` module needs to be registered in a Spezi-based application using the
[`configuration`](https://swiftpackageindex.com/stanfordspezi/spezi/documentation/spezi/speziappdelegate/configuration) in a
[`SpeziAppDelegate`](https://swiftpackageindex.com/stanfordspezi/spezi/documentation/spezi/speziappdelegate):
The [`SpeechRecognizer`](https://swiftpackageindex.com/stanfordspezi/spezispeech/documentation/spezispeech/speechrecognizer) and [`SpeechSynthesizer`](https://swiftpackageindex.com/stanfordspezi/spezispeech/documentation/spezispeech/speechsynthesizer) modules need to be registered in a Spezi-based application using the [`configuration`](https://swiftpackageindex.com/stanfordspezi/spezi/documentation/spezi/speziappdelegate/configuration)
in a [`SpeziAppDelegate`](https://swiftpackageindex.com/stanfordspezi/spezi/documentation/spezi/speziappdelegate):
```swift
class ExampleAppDelegate: SpeziAppDelegate {
override var configuration: Configuration {
Configuration {
Speech()
SpeechRecognizer()
SpeechSynthesizer()
// ...
}
}
Expand All @@ -54,16 +53,82 @@ class ExampleAppDelegate: SpeziAppDelegate {
> [!NOTE]
> You can learn more about a [`Module` in the Spezi documentation](https://swiftpackageindex.com/stanfordspezi/spezi/documentation/spezi/module).
### 3. Configure target properties

To ensure that your application has the necessary permissions for microphone access and speech recognition, follow the steps below to configure the target properties within your Xcode project:

- Open your project settings in Xcode by selecting *PROJECT_NAME > TARGET_NAME > Info* tab.
- You will need to add two entries to the `Custom iOS Target Properties` (so the `Info.plist` file) to provide descriptions for why your app requires these permissions:
- Add a key named `Privacy - Microphone Usage Description` and provide a string value that describes why your application needs access to the microphone. This description will be displayed to the user when the app first requests microphone access.
- Add another key named `Privacy - Speech Recognition Usage Description` with a string value that explains why your app requires the speech recognition capability. This will be presented to the user when the app first attempts to perform speech recognition.

These entries are mandatory for apps that utilize microphone and speech recognition features. Failing to provide them will result in your app being unable to access these features.

## Example

...
`SpeechTestView` provides a demonstration of the capabilities of the Spezi Speech module.
It showcases the interaction with the [`SpeechRecognizer`](https://swiftpackageindex.com/stanfordspezi/spezispeech/documentation/spezispeech/speechrecognizer) to provide speech-to-text capabilities and the [`SpeechSynthesizer`](https://swiftpackageindex.com/stanfordspezi/spezispeech/documentation/spezispeech/speechsynthesizer) to generate speech from text.


```swift
...
struct SpeechTestView: View {
// Get the `SpeechRecognizer` and `SpeechSynthesizer` from the SwiftUI `Environment`.
@Environment(SpeechRecognizer.self) private var speechRecognizer
@Environment(SpeechSynthesizer.self) private var speechSynthesizer
// The transcribed message from the user's voice input.
@State private var message = ""


var body: some View {
VStack {
// Button used to start and stop recording by triggering the `microphoneButtonPressed()` function.
Button("Record") {
microphoneButtonPressed()
}
.padding(.bottom)

// Button used to start and stop playback of the transcribed message by triggering the `playbackButtonPressed()` function.
Button("Playback") {
playbackButtonPressed()
}
.padding(.bottom)

Text(message)
}
}


private func microphoneButtonPressed() {
if speechRecognizer.isRecording {
// If speech is currently recognized, stop the transcribing.
speechRecognizer.stop()
} else {
// If the recognizer is idle, start a new recording.
Task {
do {
// The `speechRecognizer.start()` function returns an `AsyncThrowingStream` that yields the transcribed text.
for try await result in speechRecognizer.start() {
// Access the string-based result of the transcribed result.
message = result.bestTranscription.formattedString
}
}
}
}
}

private func playbackButtonPressed() {
if speechSynthesizer.isSpeaking {
// If speech is currently synthezized, pause the playback.
speechSynthesizer.pause()
} else {
// If synthesizer is idle, start with the text-to-speech functionality.
speechSynthesizer.speak(message)
}
}
}
```

For more information, please refer to the API documentation.
For more information, please refer to the [API documentation](https://swiftpackageindex.com/StanfordSpezi/SpeziSpeech/documentation).


## Contributing
Expand All @@ -73,7 +138,8 @@ Contributions to this project are welcome. Please make sure to read the [contrib

## License

This project is licensed under the MIT License. See [Licenses](https://github.com/StanfordSpezi/SpeziContact/tree/main/LICENSES) for more information.

![Spezi Footer](https://raw.githubusercontent.com/StanfordSpezi/.github/main/assets/FooterLight.png#gh-light-mode-only)
![Spezi Footer](https://raw.githubusercontent.com/StanfordSpezi/.github/main/assets/FooterDark.png#gh-dark-mode-only)
This project is licensed under the MIT License. See [Licenses](https://github.com/StanfordSpezi/SpeziSpeech/tree/main/LICENSES) for more information.

![Spezi Footer](https://raw.githubusercontent.com/StanfordSpezi/.github/main/assets/Footer.png#gh-light-mode-only)
![Spezi Footer](https://raw.githubusercontent.com/StanfordSpezi/.github/main/assets/Footer~dark.png#gh-dark-mode-only)
9 changes: 0 additions & 9 deletions Sources/SpeziSpeech/Resources/en.lproj/Localizable.strings

This file was deleted.

42 changes: 0 additions & 42 deletions Sources/SpeziSpeech/Speech.swift

This file was deleted.

63 changes: 0 additions & 63 deletions Sources/SpeziSpeech/SpeziSpeech.docc/SpeziSpeech.md

This file was deleted.

Loading

0 comments on commit f629d66

Please sign in to comment.