Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adds multi-language support for VITS onnx, fixes onnx exporting and inference errors #2816

Merged
merged 1 commit into from
Jul 31, 2023

Conversation

SystemPanic
Copy link
Contributor

Tested with multi-speaker and multi-language models, and with single speaker and single language, using the following script:

import torch
import os
import numpy as np
from TTS.tts.models.vits import Vits
from TTS.tts.configs.vits_config import VitsConfig
from TTS.utils.audio.numpy_transforms import save_wav

modelPath = "MULTILANG_MULTISPEAKER_PATH"
speaker_id = 0 '''None if no multi-speaker model'''
language_id = 0 '''None if no multi-language model'''

config = VitsConfig()
config.load_json(os.path.join(modelPath, "config.json"))
vits = Vits.init_from_config(config)

vits.load_onnx(os.path.join(modelPath, "MULTILANG_MULTISPEAKER_PATH.onnx"))

text = "LONG TEXT HERE"
text_inputs = np.asarray(
    vits.tokenizer.text_to_ids(text),
    dtype=np.int64,
)[None, :]

audio = vits.inference_onnx(text_inputs, speaker_id=speaker_id, language_id=language_id)
save_wav(wav=audio[0], path=os.path.join(os.path.dirname(__file__), 'test.wav'), sample_rate=config.audio.sample_rate)

… when speaker_id is None or not passed, fixes onnx exporting for models with init_discriminator=false
@CLAassistant
Copy link

CLAassistant commented Jul 29, 2023

CLA assistant check
All committers have signed the CLA.

@erogol erogol merged commit c140df5 into coqui-ai:dev Jul 31, 2023
41 of 44 checks passed
@erogol
Copy link
Member

erogol commented Jul 31, 2023

Thanks for the PR. There is one CI error but it is not about your PR.

@SystemPanic SystemPanic deleted the onnxmultilang branch August 1, 2023 18:13
Tindell pushed a commit to pugtech-co/TTS that referenced this pull request Sep 4, 2023
… when speaker_id is None or not passed, fixes onnx exporting for models with init_discriminator=false (coqui-ai#2816)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants