Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Tokotron: Tokenized TTS for the SpeechBrain benchmark (single speaker) #37

Merged
merged 144 commits into from
Jul 25, 2024
Merged

Tokotron: Tokenized TTS for the SpeechBrain benchmark (single speaker) #37

merged 144 commits into from
Jul 25, 2024

Conversation

flexthink
Copy link
Contributor

No description provided.

@flexthink
Tokotron: Initial import for the Benchmark
d76a3b9
@flexthink
Tokotron: Update hyperparameter defaults
1dcc20c
@flexthink
Tokotron: Add a workaround for concat_padded_features, which cannot b…
7e1ae9b 
…e fixed for the benchmark
@flexthink
Tokotron: Update the default EOS mode
75509fc
@flexthink
Tokotron: Add multispeaker support with LibriTTS
d55a23d
@flexthink
Tokotron: Fix defaults
9dcaf55
@flexthink
Tokotron: LibriTTS fixes
c23f6ce
@flexthink
Tokotron: Fixes for feature extraction
1e187e1
@flexthink
Tokotron: Add support for Encodec
6d1326b
@flexthink
Tokotron: Fix for Encodec
65e0754
@flexthink
Tokotron: Add normalization to embeddings, change the injection strategy
4afb395
@flexthink
DASB: Implement SSL Tacotron (continuous baseline)
0e84349
@flexthink
DASB: Tacotron: Update ljspeech parameters
bc29dfa
@flexthink
DASB: Tacotron: Add a "freezer"
3f9d4bf
@flexthink
DASB: Tacotron: Fixes
3f19048
@flexthink
Tokotron: Add Encodec support
ad5991b
@flexthink
DASB: Tacotron: Fix splits
4fe3a10
@flexthink
DASB: Tacotron device fix
6ac1a13
@flexthink
Tokotron: Fix audio_num_tokens for encodec
31b6080
@flexthink
DASB: Add a MSTacotron2 recipe
ea0afb1
@flexthink
Tokotron: Update LJSpeech to use the new vocoder
4229ef9
@flexthink
Tokotron: Update for the latest vocoders
b456a79
@flexthink
DASB: MSTacotron2: Update for all possible model types
c4a30a2
@flexthink
MSTacotron2: Fixes
2142488
@flexthink
DASB: MSTacotron2: Update a bad reference
bbdd9dc
@flexthink
DASB: MSTacotron2: Fixes
4ec2af8
@flexthink
DASB: MSTacotron: Undo a temporary debugging change
dce0563
@flexthink
MSTacotron2: Remove the test set
7d65efe
@flexthink
DASB: Tokotron: Update hparams
d296345
@flexthink
DASB: Tokotron: Update multispeaker for the latest vocoder
fad4c1c
poonehmousavi
poonehmousavi requested changes Jul 22, 2024
View reviewed changes
Copy link
Collaborator

@poonehmousavi poonehmousavi left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

since util is only used in TTS.. it would be better to move it to TTS folder

poonehmousavi
poonehmousavi requested changes Jul 22, 2024
View reviewed changes
Copy link
Collaborator

@poonehmousavi poonehmousavi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the wrappers added for all tokenzier are only used in TTS.. so maybe better to cerate the custom_model specific to TTS so it is less confusing.. but the diea is really good . in the next refactoring phase, I will try to apply teh changes across all tokenizers

poonehmousavi
poonehmousavi reviewed Jul 22, 2024
View reviewed changes
benchmarks/DASB/model/custom_model.py Outdated
from huggingface_hub import snapshot_download

try:
from speechtokenizer import SpeechTokenizer
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

precommit error for unused lib

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No longer needed given some of the code has been moved out, will update

@poonehmousavi
Copy link
Collaborator

Thanks @flexthink, everything looks really neat .. I did the review... I have fetched the latest update from the DASB main branch... These are the few comments that I have before merging the PR:

  1. There is some recommit error mostly related to unused lib.
  2. The utils and Custim_model should be moved to TTS folder, especially custom_model should be renamed or avoid the link to the main custim_model because the wrappers ae not used for other tasks and it could be confusing,.. we could add your changes later to all tasks, but for now let's keep it only for TTS.
  3. I have added the run_generative_benchmark.sh for SE and SS, please complete it for TTS and make sure that it works.

flexthink and others added 20 commits July 23, 2024 10:29
@flexthink
DASB: Tokotron: Fix flake8 errors
5fc43bc
@flexthink
DASB: Tokotron: Minor refactoring
d50c0df
@flexthink
DASB: Tokotron: Update generative script
29e3bab
@flexthink
DASB: Tokotron: Update script, fix a missing import
e1f6690
@flexthink
DASB: Update the script to allow arbitrary overrides
ea28166
@poonehmousavi
fix discriminative script
a306d3c
@flexthink
DASB: Tokotron: Refactoring
1ba7afd
@flexthink
Merge branch 'DASB-tokotron-clean' of https://github.com/flexthink/be…
586a7a5 
…nchmarks into DASB-tokotron-clean
@flexthink
DASB: Tokotron: Fixes
e767da5
@poonehmousavi
modify bash
6ae2d02
@poonehmousavi
fix bash scripts
a1493a8
@poonehmousavi
add print to script
7f92e6f
@flexthink
DASB: Tokotron: Add the UTMOS path to the shell script
dc1daf3
@flexthink
Merge branch 'DASB-tokotron-clean' of https://github.com/flexthink/be…
f74c40e 
…nchmarks into DASB-tokotron-clean
@flexthink
DASB: Tokotron: Update README for UTMOS
4522953
@poonehmousavi
FIX INDENT IN README
b11c1d2
@flexthink
DASB: Tokotron: Update TTS args
dc0bf29
@poonehmousavi
remove comments
1318aee
@poonehmousavi
fix precommit
e89de0e
@poonehmousavi
fix main readme with main branch
2939ae4
@poonehmousavi poonehmousavi merged commit c3d3e89 into speechbrain:DASB Jul 25, 2024
1 check passed
@poonehmousavi
Copy link
Collaborator

Thanks @flexthink for this PR... everything works now..It merged

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@poonehmousavi poonehmousavi poonehmousavi requested changes

Assignees

@flexthink flexthink

Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@flexthink @poonehmousavi