Wespeaker Roadmap

Version 2.0 (Time: 2023.12)

This is the roadmap for wespeaker version 2.0.

This is the roadmap for wespeaker version 1.0.

Standard dataset support
- VoxCeleb
- CnCeleb
SOTA models support
- x-vector (tdnn based, milestone deep speaker embedding)
- r-vector (resnet based, winner of voxsrc 2019)
- ecapa-tdnn (variant of tdnn, winner of voxsrc 2020)
Back-end Support
- Cosine
- EER/minDCF
- AS-norm
- PLDA
UIO for effective industrial-scale dataset processing
- Online data augmentation
  - Noise && RIR
  - Speed Perturb
  - Specaug
ONNX support
Triton Server support (GPU)
~~
- Training or finetuning big models such as WavLM might be too costly for current stage
Basic Speaker Diarization Recipe
- Embedding based (more related with our speaker embedding learner toolkit)
Interactive Demo
- Support using features from released pretrained models (hugging face)

Model (SOTA Models)
- Standard X-vector
- ResNet
- ECAPA_TDNN
- RepVGG
- CAM++
- ERes2Net
- Gemini-dfresnet
Pooling Functions
- TAP(mean) / TSDP(std) / TSTP(mean+std)
  - Comparison of mean/std pooling can be found in shuai_iscslp, anna_arxiv
- Attentive Statistics Pooling (ASTP)
  - Mainly for ECAPA_TDNN
- Multi-Query and Multi-Head Attentive Statistics Pooling (MQMHASTP)
  - Details can be found in MQMHASTP
Criteria
Scoring
- Cosine
- PLDA
- Score Normalization (AS-Norm)
- Quality-aware Score Calibration
Metric
- EER
- minDCF
- DER
Online Augmentation
- Noise && RIR
- Speed Perturb
- SpecAug
Training Strategy
- Well-designed Learning Rate and Margin Schedulers
- Large Margin Fine-tuning
- Automatic Mixed Precision (AMP) Training
Runtime
- Python Binding
- Triton Inference Server on verification && diarization in GPU deployment
- C++ Onnxruntime
- MNN
Self-Supervised Learning (SSL)
- DINO
- MoCo
- SimCLR
Literature
- Awesome Speaker Papers