Training GlowTTS for polish lang

[madey83]

(coqui) root@DESKTOP-H42AEU4:/coqui# python train_pl_wsl.py
fatal: not a git repository (or any of the parent directories): .git
fatal: not a git repository (or any of the parent directories): .git

Training Environment:
| > Backend: Torch
| > Mixed precision: False
| > Precision: float32
| > Current device: 0
| > Num. of GPUs: 1
| > Num. of CPUs: 12
| > Num. of Torch Threads: 6
| > Torch seed: 54321
| > Torch CUDNN: True
| > Torch CUDNN deterministic: False
| > Torch CUDNN benchmark: False
| > Torch TF32 MatMul: False
Start Tensorboard: tensorboard --logdir=/coqui/output-glowtts/Glow-TTS-PL-November-08-2024_12+02PM-0000000

Model has 28610449 parameters

EPOCH: 0/1000
--> /coqui/output-glowtts/Glow-TTS-PL-November-08-2024_12+02PM-0000000
48%|████████████████████████████████████████████████████████████████████████████████████████████▊ | 72/149 [00:00<00:01, 68.04it/s]— jˌɛstɛm jˈɛdnɔ̃ s pantsˈɛrnɨx, tˈak jˈak tˈɨ.
Character '—' not found in the vocabulary. Discarding it.
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 149/149 [00:01<00:00, 76.98it/s]

TRAINING (2024-11-08 12:02:06)

--> TIME: 2024-11-08 12:02:09 -- STEP: 0/5 -- GLOBAL_STEP: 0
| > current_lr: 2.5e-07
| > step_time: 1.343 (1.342970371246338)
| > loader_time: 1.7516 (1.7515895366668701)

[!] train_step() retuned None outputs. Skipping training step.
[!] train_step() retuned None outputs. Skipping training step.
[!] train_step() retuned None outputs. Skipping training step.
[!] train_step() retuned None outputs. Skipping training step.
[!] train_step() retuned None outputs. Skipping training step.
/coqui/lib/python3.12/site-packages/torch/optim/lr_scheduler.py:224: UserWarning: Detected call of lr_scheduler.step() before optimizer.step(). In PyTorch 1.1.0 and later, you should call them in the opposite order: optimizer.step() before lr_scheduler.step(). Failure to do this will result in PyTorch skipping the first value of the learning rate schedule. See more details at https://pytorch.org/docs/stable/optim.html#how-to-adjust-learning-rate
warnings.warn(

EVALUATION

/coqui/coqui-ai-TTS/TTS/tts/models/glow_tts.py:418: FutureWarning: torch.cuda.amp.autocast(args...) is deprecated. Please use torch.amp.autocast('cuda', args...) instead.
with autocast(enabled=False): # avoid mixed_precision in criterion
--> STEP: 0
| > loss: 1.9118473529815674 (1.9118473529815674)
| > log_mle: 0.7955801486968994 (0.7955801486968994)
| > loss_dur: 1.116267204284668 (1.116267204284668)

--> EVAL PERFORMANCE
| > avg_loader_time: 0.3790626525878906 (+0)
| > avg_loss: 1.9118473529815674 (+0)
| > avg_log_mle: 0.7955801486968994 (+0)
| > avg_loss_dur: 1.116267204284668 (+0)

EPOCH: 1/1000
--> /coqui/output-glowtts/Glow-TTS-PL-November-08-2024_12+02PM-0000000

TRAINING (2024-11-08 12:02:13)
[!] train_step() retuned None outputs. Skipping training step.
[!] train_step() retuned None outputs. Skipping training step.
[!] train_step() retuned None outputs. Skipping training step.
[!] train_step() retuned None outputs. Skipping training step.

is it normal output?

[Marioando]

vits/vits2 is easier to train than glowtts. I suggest to switch to vits, end to end and better quality.

[madey83]

ok. thanks for answer. can i use vits female polish model from tts models "tts_models--pl--mai_female--vits" and start training my new male model with my custom dataset to not start from beggining but start from some point?

if yes... could you please guide how to do that?

[Marioando]

Yes. You can. You just have to set restore_path in trainer args to the path of the pretrained models.