Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add leading silence detection and removal logic #17648

Merged
merged 28 commits into from
Feb 11, 2025
Merged
Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
bd5eece
Add leading silence detection and removal logic
gexgd0419 Jan 24, 2025
f42ded7
Call startTrimmingLeadingSilence in stop and _idleCheck
gexgd0419 Jan 24, 2025
4157888
Add changelog entry
gexgd0419 Jan 24, 2025
8ac5326
Add generateSilence in SilenceDetect
gexgd0419 Jan 25, 2025
c4b0036
Add pre_synthSpeak extension point in synthDriverHandler
gexgd0419 Jan 26, 2025
55062da
Check speech sequence in nvwave.py to see if leading silence is inser…
gexgd0419 Jan 26, 2025
896a1d3
Add pre_synthSpeak in developerGuide
gexgd0419 Jan 27, 2025
77e93a9
Enable trimming by default for speech only
gexgd0419 Jan 27, 2025
e4b5ec5
Fix
gexgd0419 Jan 27, 2025
5af9500
Clear (initialize) the smp value before reading samples
gexgd0419 Feb 5, 2025
9038869
Add comments for min/max value trimming
gexgd0419 Feb 5, 2025
4faa2c5
Add Changes for Developers entry
gexgd0419 Feb 5, 2025
5985147
Apply suggestions from code review
gexgd0419 Feb 5, 2025
2fc2ed4
Apply suggestions
gexgd0419 Feb 5, 2025
7df9f7a
Revert "Add generateSilence in SilenceDetect"
gexgd0419 Feb 5, 2025
212476d
Add config item speech.trimLeadingSilence in advanced settings
gexgd0419 Feb 5, 2025
c2f98f5
Allow disabling by user
gexgd0419 Feb 5, 2025
9dde0bd
Remove unused code
gexgd0419 Feb 6, 2025
b9d3d94
Remove GUI structure from name "AdvancedSettingsTrimLeadingSilence"
gexgd0419 Feb 6, 2025
8202b44
Update user_docs/en/userGuide.md
gexgd0419 Feb 6, 2025
75c2179
Reload synth when settings changes
gexgd0419 Feb 6, 2025
f65a549
Add copyright header for silenceDetect.h
gexgd0419 Feb 6, 2025
41da574
Pre-commit auto-fix
pre-commit-ci[bot] Feb 6, 2025
648ebd2
Change anchor
SaschaCowley Feb 6, 2025
52f7252
Update user_docs/en/changes.md
seanbudd Feb 7, 2025
a91b6c6
Move speech queue checking into WavePlayer class
gexgd0419 Feb 7, 2025
1bcb48e
Pre-commit auto-fix
pre-commit-ci[bot] Feb 7, 2025
d54df2e
Remove unused function `WaveFormat::convertFrom`
gexgd0419 Feb 7, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
64 changes: 5 additions & 59 deletions nvdaHelper/local/silenceDetect.h
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
// A part of NonVisual Desktop Access (NVDA)
// This file is covered by the GNU General Public License.
// See the file COPYING for more details.
// Copyright (C) 2025 NV Access Limited, gexgd0419

#ifndef SILENCEDETECT_H
#define SILENCEDETECT_H

Expand Down Expand Up @@ -173,42 +178,6 @@ size_t getLeadingSilenceSizeMono(
return size;
}

/**
* Return the trailing silence wave data length, in bytes.
* Assumes the wave data to be of one channel (mono).
* Uses a `WaveFormat` type (`Fmt`) to determine the wave format.
*/
template <class Fmt>
size_t getTrailingSilenceSizeMono(
const unsigned char* waveData,
size_t size,
typename Fmt::SampleType threshold
) {
using SampleType = Fmt::SampleType;
constexpr size_t bytesPerSample = Fmt::bytesPerSample;

if (size < bytesPerSample)
return 0;

constexpr SampleType zeroPoint = Fmt::zeroPoint();
const SampleType minValue = zeroPoint - threshold, maxValue = zeroPoint + threshold;

// Check each sample in reverse order
const unsigned char* p = waveData + (size - (size % bytesPerSample));
SampleType smp = SampleType();
do {
p -= bytesPerSample;
memcpy(&smp, p, bytesPerSample);
smp = Fmt::signExtend(smp);
// this sample is out of range, so the trailing silence starts at the next sample
if (smp < minValue || smp > maxValue)
return size - (p - waveData) - bytesPerSample;
} while (p > waveData);

// The whole data block is silence
return size;
}

/**
* Invoke a functor with an argument of a WaveFormat type that corresponds to the specified WAVEFORMATEX.
* Return false if the WAVEFORMATEX is unknown.
Expand Down Expand Up @@ -272,29 +241,6 @@ inline size_t getLeadingSilenceSize(
return len - len % wfx->nBlockAlign; // round down to block (channel) boundaries
}

/**
* Return the trailing silence wave data length, in bytes.
* Uses a `WAVEFORMATEX` to determine the wave format.
*/
inline size_t getTrailingSilenceSize(
const WAVEFORMATEX* wfx,
const unsigned char* waveData,
size_t size
) {
size_t len;
if (!callByWaveFormat(wfx, [=, &len](auto fmtTag) {
using Fmt = decltype(fmtTag);
len = getTrailingSilenceSizeMono<Fmt>(
waveData, size, Fmt::defaultThreshold());
}))
return 0;

size_t align = wfx->nBlockAlign;
len += align - 1;
len -= len % align; // round up to block (channel) boundaries
return len;
}

} // namespace SilenceDetect

#endif // SILENCEDETECT_H
11 changes: 9 additions & 2 deletions source/gui/settingsDialogs.py
Original file line number Diff line number Diff line change
Expand Up @@ -3755,7 +3755,7 @@ def __init__(self, parent):
# Advanced settings panel.
label = _("Trim leading silence in speech audio")
self.trimLeadingSilenceCheckBox = speechGroup.addItem(wx.CheckBox(speechBox, label=label))
self.bindHelpEvent("AdvancedSettingsTrimLeadingSilence", self.trimLeadingSilenceCheckBox)
self.bindHelpEvent("TrimLeadingSilence", self.trimLeadingSilenceCheckBox)
SaschaCowley marked this conversation as resolved.
Show resolved Hide resolved
self.trimLeadingSilenceCheckBox.SetValue(config.conf["speech"]["trimLeadingSilence"])
self.trimLeadingSilenceCheckBox.defaultValue = self._getDefaultValue(["speech", "trimLeadingSilence"])

Expand Down Expand Up @@ -3985,6 +3985,14 @@ def restoreToDefaults(self):

def onSave(self):
log.debug("Saving advanced config")

if config.conf["speech"]["trimLeadingSilence"] != self.trimLeadingSilenceCheckBox.IsChecked():
# Reload the synthesizer if "trimLeadingSilence" changes
config.conf["speech"]["trimLeadingSilence"] = self.trimLeadingSilenceCheckBox.IsChecked()
currentSynth = getSynth()
if not setSynth(currentSynth.name):
_synthWarningDialog(currentSynth.name)

config.conf["development"]["enableScratchpadDir"] = self.scratchpadCheckBox.IsChecked()
selectiveUIAEventRegistrationChoice = self.selectiveUIAEventRegistrationCombo.GetSelection()
config.conf["UIA"]["eventRegistration"] = self.selectiveUIAEventRegistrationVals[
Expand All @@ -3997,7 +4005,6 @@ def onSave(self):
config.conf["featureFlag"]["cancelExpiredFocusSpeech"] = (
self.cancelExpiredFocusSpeechCombo.GetSelection()
)
config.conf["speech"]["trimLeadingSilence"] = self.trimLeadingSilenceCheckBox.IsChecked()
config.conf["UIA"]["allowInChromium"] = self.UIAInChromiumCombo.GetSelection()
self.enhancedEventProcessingComboBox.saveCurrentValueToConf()
config.conf["terminals"]["speakPasswords"] = self.winConsoleSpeakPasswordsCheckBox.IsChecked()
Expand Down
6 changes: 3 additions & 3 deletions source/nvwave.py
Original file line number Diff line number Diff line change
Expand Up @@ -294,7 +294,9 @@ def __init__(
NVDAHelper.localLib.wasSilence_init(outputDevice)
WasapiWavePlayer._silenceDevice = outputDevice
# Enable trimming by default for speech only
self.enableTrimmingLeadingSilence(purpose is AudioPurpose.SPEECH)
self.enableTrimmingLeadingSilence(
purpose is AudioPurpose.SPEECH and config.conf["speech"]["trimLeadingSilence"],
)
if self._enableTrimmingLeadingSilence:
self.startTrimmingLeadingSilence()

Expand Down Expand Up @@ -481,8 +483,6 @@ def enableTrimmingLeadingSilence(self, enable: bool) -> None:

def startTrimmingLeadingSilence(self, start: bool = True) -> None:
"""Start or stop trimming the leading silence from the next audio chunk."""
if not config.conf["speech"]["trimLeadingSilence"]:
return ## disabled by user
NVDAHelper.localLib.wasPlay_startTrimmingLeadingSilence(self._player, start)

def _setVolumeFromConfig(self):
Expand Down
7 changes: 4 additions & 3 deletions user_docs/en/userGuide.md
Original file line number Diff line number Diff line change
Expand Up @@ -3321,10 +3321,11 @@ This option enables behaviour which attempts to cancel speech for expired focus
In particular moving quickly through messages in Gmail with Chrome can cause NVDA to speak outdated information.
This functionality is enabled by default as of NVDA 2021.1.

##### Trim leading silence in speech audio {#AdvancedSettingsTrimLeadingSilence}
##### Trim leading silence in speech audio {#TrimLeadingSilence}
SaschaCowley marked this conversation as resolved.
Show resolved Hide resolved

This option enables NVDA to trim the silence at the beginning of the speech audio, making the voices respond faster.
This option is enabled by default, and should only affect the silence at the beginning. But if you find that some necessary silence periods are also missing (e.g. pause between two sentences) when using a speech synthesizer add-on, you may try turning off this feature.
When enabled, NVDA will remove silence from the start of speech audio, which may improve the responsiveness of some speech synthesizers.
This option is enabled by default, and should only affect the silence at the beginning of speech.
If you find that some necessary silence periods are also missing (e.g. pause between two sentences) when using a speech synthesizer add-on, you may turn this feature off entirely to resolve the issue.

##### Caret move timeout (in MS) {#AdvancedSettingsCaretMoveTimeout}

Expand Down