jyapayne/mimic3
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Merge mimic3-client into main CLI

Browse source
This commit is contained in:
Michael Hansen 2022-04-11 14:49:31 -04:00
commit 4a738b0cba
19 changed files with 153 additions and 346 deletions
Download patch file Download diff file Expand all files Collapse all files

1
.dockerignore
View file

@ -3,7 +3,6 @@
!install.sh
!pyinstaller/*.py
!pyinstaller/mimic3-server
!pyinstaller/mimic3-client
!pyinstaller/mimic3-download
# opentts-abc

1
Dockerfile.binary
View file

@ -94,7 +94,6 @@ RUN find dist/mimic3/ -wholename '*/gruut_lang_*/espeak' -type d | \
# Copy convenience scripts
COPY pyinstaller/mimic3-server \
pyinstaller/mimic3-client \
pyinstaller/mimic3-download \
dist/mimic3/

10
README.md
View file

@ -76,7 +76,6 @@ pip install mimic3-http[all]
Once installed, the following commands will be available:
* `mimic3-server`
* `mimic3-client`
Language support can be selectively installed by replacing `all` with:
@ -111,7 +110,6 @@ A virtual environment will be created in `mimic3/.venv` and each of the Python m
Once installed, the following commands will be available in `.venv/bin`:
* `mimic3`
* `mimic3-server`
* `mimic3-client`
* `mimic3-download`
@ -185,15 +183,17 @@ See `mimic3-server --help` for the [web server documentation](mimic3-http/) for
#### Web Client
The `mimic3-client` program provides an interface to the Mimic 3 web server that is similar to the `mimic3` command.
The `mimic3` program provides an interface to the Mimic 3 web server when the `--remote` option is given.
Assuming you have started `mimic3-server` and can access `http://localhost:59125`, then:
``` sh
mimic3-client --voice 'en_UK/apope_low' 'My hovercraft is full of eels.' > hovercraft_eels.wav
mimic3 --remote --voice 'en_UK/apope_low' 'My hovercraft is full of eels.' > hovercraft_eels.wav
```
See `mimic3-client --help` for more options.
If your server is somewhere besides `localhost`, use `mimic3 --remote <URL> ...`
See `mimic3 --help` for more options.
## CUDA Acceleration

82
docker/mimic3-client
View file

@ -1,82 +0,0 @@
#!/usr/bin/env bash
# Copyright 2022 Mycroft AI Inc.
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU Affero General Public License for more details.
#
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
# Run mimic3-client --update to update Docker image
args=()
repo='mycroftai/mimic3'
tag='latest'
docker='docker'
port='59125'
while [[ -n "$1" ]]; do
if [[ "$1" == '--update' ]]; then
# Update Docker image
update='1'
elif [[ "$1" == '--port' ]]; then
port="$2"
args+=('--port' "${port}")
shift 1
else
args+=("$1")
fi
shift 1
done
if [[ -n "${update}" ]]; then
docker pull "${repo}:${tag}"
fi
docker_run_args=()
if [[ -d /etc/ssl/certs ]]; then
# This directory seems to usually have symlinks to other directories
docker_run_args+=('-v' '/etc/ssl/certs:/etc/ssl/certs:ro')
# Create temp file with all certificate directories found
cert_dirs_file="$(mktemp)"
function finish {
rm -rf "${cert_dirs_file}"
}
trap finish EXIT
while read -r cert_path; do
# Follow symlinks and record directory paths
cert_path="$(readlink -f "${cert_path}")"
cert_dir="$(dirname "${cert_path}")"
echo $cert_dir >> "${cert_dirs_file}"
done < <(find /etc/ssl/certs -name '*.pem' -type l)
# Map unique certificate directories
while read -r cert_dir; do
docker_run_args+=('-v' "${cert_dir}:${cert_dir}:ro")
done < <(sort < "${cert_dirs_file}" | uniq)
fi
"${docker}" run \
-i \
--network host \
-e "HOME=${HOME}" \
-v "$HOME:${HOME}" \
-w "${PWD}" \
--user "$(id -u):$(id -g)" "${docker_run_args[@]}" \
--entrypoint '/home/mimic3/app/.venv/bin/python3' \
"${repo}:${tag}" \
-m mimic3_http.client \
--stdout \
"${args[@]}"

2
examples/speech-dispatcher/mimic3-generic.conf
View file

@ -1,2 +1,2 @@
GenericExecuteSynth "printf %s \'$DATA\' | /path/to/mimic3-client --voice \'$VOICE\' --stdout | $PLAY_COMMAND"
GenericExecuteSynth "printf %s \'$DATA\' | /path/to/mimic3 --remote --voice \'$VOICE\' --stdout | $PLAY_COMMAND"
AddVoice "en-us" "MALE1" "en_UK/apope_low"

6
mimic3-http/README.md
View file

@ -42,10 +42,12 @@ Using [nvidia-docker](https://github.com/NVIDIA/nvidia-docker) is highly recomme
Assuming you have started `mimic3-server` and can access `http://localhost:59125`, then:
``` sh
mimic3-client --voice 'en_UK/apope_low' 'My hovercraft is full of eels.' > hovercraft_eels.wav
mimic3 --remote --voice 'en_UK/apope_low' 'My hovercraft is full of eels.' > hovercraft_eels.wav
```
See `mimic3-client --help` for more options.
If your server is somewhere besides `localhost`, use `mimic3 --remote <URL> ...`
See `mimic3 --help` for more options.
## MaryTTS Compatibility

9
mimic3-http/mimic3_http/app.py
View file

@ -26,7 +26,7 @@ from urllib.parse import parse_qs
from uuid import uuid4
import quart_cors
from mimic3_tts import Mimic3Settings, Mimic3TextToSpeechSystem
from mimic3_tts import DEFAULT_VOICE, Mimic3Settings, Mimic3TextToSpeechSystem
from quart import (
Quart,
Response,
@ -159,9 +159,8 @@ def get_app(args: argparse.Namespace, request_queue: Queue, temp_dir: str):
_LOGGER.debug("Request args: %s", request.args)
voice = request.args.get("voice")
if voice is not None:
tts_args["voice"] = str(voice)
voice = request.args.get("voice") or args.voice or DEFAULT_VOICE
tts_args["voice"] = str(voice)
# TTS settings
noise_scale = request.args.get("noiseScale")
@ -226,7 +225,7 @@ def get_app(args: argparse.Namespace, request_queue: Queue, temp_dir: str):
text = request.args.get("INPUT_TEXT", "")
voice = str(request.args.get("VOICE", voice)).strip()
voice = voice or args.voice
voice = voice or args.voice or DEFAULT_VOICE
# Assume SSML if text begins with an angle bracket
ssml = text.strip().startswith("<")

175
mimic3-http/mimic3_http/client.py
View file

@ -1,175 +0,0 @@
#!/usr/bin/env python3
# Copyright 2022 Mycroft AI Inc.
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU Affero General Public License for more details.
#
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
import argparse
import logging
import os
import shlex
import shutil
import subprocess
import sys
import tempfile
import typing
from pathlib import Path
import requests
_PACKAGE = "mimic3_http.client"
_LOGGER = logging.getLogger(_PACKAGE)
_DEFAULT_PLAY_PROGRAMS = ["paplay", "play -q", "aplay -q"]
# -----------------------------------------------------------------------------
def main(argv=None):
args = get_args(argv)
if args.output:
args.output = Path(args.output)
args.output.parent.mkdir(parents=True, exist_ok=True)
if args.ssml:
headers = {"Content-Type": "application/ssml+xml"}
else:
headers = {"Content-Type": "text/plain"}
params: typing.Dict[str, str] = {}
if args.voice:
params["voice"] = args.voice
if args.length_scale:
params["lengthScale"] = args.length_scale
if args.noise_scale:
params["noiseScale"] = args.noise_scale
if args.noise_w:
params["noiseW"] = args.noise_w
if args.text:
data = "\n".join(args.text)
else:
if os.isatty(sys.stdin.fileno()):
print("Reading text from stdin...", file=sys.stderr)
data = sys.stdin.read()
wav_bytes = requests.post(
args.url, headers=headers, params=params, data=data
).content
if args.output:
args.output.write_bytes(wav_bytes)
_LOGGER.info("Wrote WAV data to %s", args.output)
elif args.stdout or (not sys.stdout.isatty()):
_LOGGER.debug("Writing WAV data to stdout")
sys.stdout.buffer.write(wav_bytes)
else:
play_wav_bytes(args, wav_bytes)
# -----------------------------------------------------------------------------
def play_wav_bytes(args: argparse.Namespace, wav_bytes: bytes):
with tempfile.NamedTemporaryFile(mode="wb+", suffix=".wav") as wav_file:
wav_file.write(wav_bytes)
wav_file.seek(0)
for play_program in reversed(args.play_program):
play_cmd = shlex.split(play_program)
if not shutil.which(play_cmd[0]):
continue
play_cmd.append(wav_file.name)
_LOGGER.debug("Playing WAV file: %s", play_cmd)
subprocess.check_output(play_cmd)
break
# -----------------------------------------------------------------------------
def get_args(argv) -> argparse.Namespace:
parser = argparse.ArgumentParser(
prog=_PACKAGE, description="Client for Mimic 3 HTTP web server"
)
parser.add_argument(
"text", nargs="*", help="Text to convert to speech (default: stdin)"
)
parser.add_argument(
"--url",
"-u",
default="http://localhost:59125/api/tts",
help="URL of mimic3 HTTP server (default: http://localhost:59125/api/tts)",
)
parser.add_argument(
"--voice",
"-v",
help="Name of voice (expected in <voices-dir>/<language>)",
)
parser.add_argument(
"--output",
"-o",
help="Path to write WAV file (default: play audio)",
)
parser.add_argument(
"--stdout",
action="store_true",
help="Write WAV data to stdout",
)
parser.add_argument(
"--noise-scale",
type=float,
help="Noise scale [0-1], default is 0.667",
)
parser.add_argument(
"--length-scale",
type=float,
help="Length scale (1.0 is default speed, 0.5 is 2x faster)",
)
parser.add_argument(
"--noise-w",
type=float,
help="Variation in cadence [0-1], default is 0.8",
)
parser.add_argument(
"--play-program",
action="append",
default=_DEFAULT_PLAY_PROGRAMS,
help="Program(s) used to play WAV files",
)
parser.add_argument("--ssml", action="store_true", help="Input text is SSML")
parser.add_argument(
"--debug", action="store_true", help="Print DEBUG messages to the console"
)
args = parser.parse_args(args=argv)
if args.debug:
logging.basicConfig(level=logging.DEBUG)
else:
logging.basicConfig(level=logging.INFO)
return args
# -----------------------------------------------------------------------------
if __name__ == "__main__":
main()

2
mimic3-http/mimic3_http/examples/speech-dispatcher/mimic3-generic.conf
View file

@ -1,2 +0,0 @@
GenericExecuteSynth "printf %s \'$DATA\' | /home/hansenm/opt/mimic3/mimic3-http/client.sh --voice \'$VOICE\' --length-scale 0.5 --stdout | $PLAY_COMMAND"
AddVoice "en-us" "FEMALE1" "en_US/amy_low"

2
mimic3-http/mimic3_http/examples/speech-dispatcher/speechd.conf
View file

@ -1,2 +0,0 @@
DefaultVoiceType "FEMALE1"
DefaultModule mimic3-generic

6
mimic3-http/mypy.ini
View file

@ -1,11 +1,5 @@
[mypy]
[mypy-playsound.*]
ignore_missing_imports = True
[mypy-requests.*]
ignore_missing_imports = True
[mypy-setuptools.*]
ignore_missing_imports = True

1
mimic3-http/requirements.txt
View file

@ -1,5 +1,4 @@
mimic3-tts<1.0
quart>=0.16,<1.0
quart-cors
requests>=2,<3
swagger-ui-py>=21,<22

1
mimic3-http/setup.py
View file

@ -95,7 +95,6 @@ setup(
entry_points={
"console_scripts": [
"mimic3-server = mimic3_http.__main__:main",
"mimic3-client = mimic3_http.client:main",
]
},
classifiers=[

4
mimic3-tts/README.md
View file

@ -214,11 +214,11 @@ sudo apt-get install speech-dispatcher
Create the file `/etc/speech-dispatcher/modules/mimic3-generic.conf` with the contents:
``` text
GenericExecuteSynth "printf %s \'$DATA\' | /path/to/mimic3-client --voice \'$VOICE\' --stdout | $PLAY_COMMAND"
GenericExecuteSynth "printf %s \'$DATA\' | /path/to/mimic3 --remote --voice \'$VOICE\' --stdout | $PLAY_COMMAND"
AddVoice "en-us" "MALE1" "en_UK/apope_low"
```
You will need `sudo` access to do this. Make sure to change `/path/to/mimic3-client` to wherever you installed Mimic 3.
You will need `sudo` access to do this. Make sure to change `/path/to/mimic3` to wherever you installed Mimic 3. Note that the `--remote` option is used to connect to a local Mimic 3 web server (use `--remote <URL>` if your server is somewhere besides `localhost`).
To change the voice later, you only need to replace `en_UK/apope_low`.

1
mimic3-tts/mimic3_tts/__init__.py
View file

@ -13,6 +13,7 @@ from opentts_abc import (
from opentts_abc.ssml import SSMLSpeaker
from ._resources import __version__
from .const import DEFAULT_VOICE
from .tts import Mimic3Settings, Mimic3TextToSpeechSystem
__author__ = "Michael Hansen"

167
mimic3-tts/mimic3_tts/__main__.py
View file

@ -213,6 +213,14 @@ def initialize_args(state: CommandLineInterfaceState):
state.texts = process_on_blank_line(state.texts)
if args.remote and args.remote.endswith("/"):
# Ensure no slash
args.remote = args.remote[:-1]
if (not args.speaker) and args.voice and ("#" in args.voice):
# Split apart voice
args.voice, args.speaker = args.voice.split("#", maxsplit=1)
def initialize_tts(state: CommandLineInterfaceState):
"""Create Mimic 3 TTS from command-line arguments"""
@ -220,24 +228,29 @@ def initialize_tts(state: CommandLineInterfaceState):
args = state.args
state.tts = Mimic3TextToSpeechSystem(
Mimic3Settings(
voices_directories=args.voices_dir, speaker=args.speaker, use_cuda=args.cuda
if not args.remote:
# Local TTS
state.tts = Mimic3TextToSpeechSystem(
Mimic3Settings(
voices_directories=args.voices_dir,
speaker=args.speaker,
use_cuda=args.cuda,
)
)
)
if args.voices:
# Don't bother with the rest of the initialization
return
if state.args.voice:
# Set default voice
state.tts.voice = state.args.voice
if state.tts:
if state.args.voice:
# Set default voice
state.tts.voice = state.args.voice
if state.args.preload_voice:
for voice_key in state.args.preload_voice:
_LOGGER.debug("Preloading voice: %s", voice_key)
state.tts.preload_voice(voice_key)
if state.args.preload_voice:
for voice_key in state.args.preload_voice:
_LOGGER.debug("Preloading voice: %s", voice_key)
state.tts.preload_voice(voice_key)
state.result_queue = Queue(maxsize=args.result_queue_size)
@ -326,31 +339,52 @@ def process_line(
line_id: str = "",
line_voice: typing.Optional[str] = None,
):
from mimic3_tts import SSMLSpeaker
assert state.tts is not None
assert state.result_queue is not None
args = state.args
if state.tts:
# Local TTS
from mimic3_tts import SSMLSpeaker
if line_voice:
if line_voice.startswith("#"):
# Same voice, but different speaker
state.tts.speaker = line_voice[1:]
assert state.tts is not None
args = state.args
if line_voice:
if line_voice.startswith("#"):
# Same voice, but different speaker
state.tts.speaker = line_voice[1:]
else:
# Different voice
state.tts.voice = line_voice
if args.ssml:
results = SSMLSpeaker(state.tts).speak(line)
else:
# Different voice
state.tts.voice = line_voice
state.tts.begin_utterance()
if args.ssml:
results = SSMLSpeaker(state.tts).speak(line)
# TODO: text language
state.tts.speak_text(line)
results = state.tts.end_utterance()
else:
state.tts.begin_utterance()
# Remote TTS
from mimic3_tts import AudioResult
# TODO: text language
state.tts.speak_text(line)
results = state.tts.end_utterance()
# Get remote WAV data and repackage as AudioResult
wav_bytes = get_remote_wav_bytes(state, line)
with io.BytesIO(wav_bytes) as wav_io:
wav_reader: wave.Wave_read = wave.open(wav_io, "rb")
with wav_reader as wav_file:
results = [
AudioResult(
sample_rate_hz=wav_file.getframerate(),
sample_width_bytes=wav_file.getsampwidth(),
num_channels=wav_file.getnchannels(),
audio_bytes=wav_file.readframes(wav_file.getnframes()),
)
]
# Add results to processing queue
for result in results:
state.result_queue.put(
ResultToProcess(
@ -361,8 +395,9 @@ def process_line(
)
# Restore voice/speaker
state.tts.voice = args.voice
state.tts.speaker = args.speaker
if state.tts:
state.tts.voice = args.voice
state.tts.speaker = args.speaker
def process_lines(state: CommandLineInterfaceState):
@ -434,7 +469,7 @@ def process_lines(state: CommandLineInterfaceState):
def shutdown_tts(state: CommandLineInterfaceState):
if state.tts is not None:
if state.tts:
state.tts.shutdown()
state.tts = None
@ -456,10 +491,13 @@ def play_wav_bytes(args: argparse.Namespace, wav_bytes: bytes):
def print_voices(state: CommandLineInterfaceState):
assert state.tts is not None
voices = list(state.tts.get_voices())
voices = sorted(voices, key=lambda v: v.key)
if state.tts:
# Local TTS
voices = list(state.tts.get_voices())
voices = sorted(voices, key=lambda v: v.key)
else:
# Remove TTS
voices = get_remote_voices(state)
writer = csv.writer(sys.stdout, delimiter="\t")
writer.writerow(("KEY", "LANGUAGE", "NAME", "DESCRIPTION", "LOCATION"))
@ -472,6 +510,59 @@ def print_voices(state: CommandLineInterfaceState):
# -----------------------------------------------------------------------------
def get_remote_voices(state: CommandLineInterfaceState) -> typing.List:
import requests
from mimic3_tts import Voice
args = state.args
url = f"{args.remote}/api/voices"
_LOGGER.debug("Getting voices from remote server at %s", url)
voices_json = requests.get(url).json()
return [Voice(**voice_args) for voice_args in voices_json]
def get_remote_wav_bytes(state: CommandLineInterfaceState, text: str) -> bytes:
import requests
args = state.args
if args.ssml:
headers = {"Content-Type": "application/ssml+xml"}
else:
headers = {"Content-Type": "text/plain"}
params: typing.Dict[str, str] = {}
if args.voice:
if args.speaker:
params["voice"] = f"{args.voice}#{args.speaker}"
else:
params["voice"] = args.voice
if args.length_scale:
params["lengthScale"] = args.length_scale
if args.noise_scale:
params["noiseScale"] = args.noise_scale
if args.noise_w:
params["noiseW"] = args.noise_w
url = f"{args.remote}/api/tts"
_LOGGER.debug("Synthesizing text remotely at %s", url)
wav_bytes = requests.post(url, headers=headers, params=params, data=text).content
return wav_bytes
# -----------------------------------------------------------------------------
def get_args(argv=None):
"""Parse command-line arguments"""
parser = argparse.ArgumentParser(
@ -480,6 +571,12 @@ def get_args(argv=None):
parser.add_argument(
"text", nargs="*", help="Text to convert to speech (default: stdin)"
)
parser.add_argument(
"--remote",
nargs="?",
const="http://localhost:59125",
help="Connect to Mimic 3 HTTP web server for synthesis (default: localhost)",
)
parser.add_argument(
"--stdin-format",
choices=[str(v.value) for v in StdinFormat],

3
mimic3-tts/mypy.ini
View file

@ -9,6 +9,9 @@ ignore_missing_imports = True
[mypy-onnxruntime.*]
ignore_missing_imports = True
[mypy-requests.*]
ignore_missing_imports = True
[mypy-tqdm.*]
ignore_missing_imports = True

1
mimic3-tts/requirements.txt
View file

@ -6,5 +6,6 @@ numpy<2.0
onnxruntime>=1.6,<2.0
opentts_abc<1.0
phonemes2ids<2.0
requests>=2,<3
tqdm>=4,<5
xdgenvpy>2.0,<3

25
pyinstaller/mimic3-client
View file

@ -1,25 +0,0 @@
#!/usr/bin/env bash
# Copyright 2022 Mycroft AI Inc.
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU Affero General Public License for more details.
#
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
# -----------------------------------------------------------------------------
#
# Stub script to execute web client in PyInstaller distribution.
#
# Directory of *this* script
this_dir="$( cd "$( dirname "$0" )" && pwd )"
"${this_dir}/mimic3" --subprogram client "$@"