Centralize documentation
This commit is contained in:
Michael Hansen
parent
4a781a1a64
commit
07f3c00d9b
5 changed files with 47 additions and 683 deletions
332
README.md
332
README.md
|
|
@ -5,99 +5,32 @@
|
||||||
A fast and local neural text to speech system developed by [Mycroft](https://mycroft.ai/) for the [Mark II](https://mycroft.ai/product/mark-ii/).
|
A fast and local neural text to speech system developed by [Mycroft](https://mycroft.ai/) for the [Mark II](https://mycroft.ai/product/mark-ii/).
|
||||||
|
|
||||||
* [Available voices](https://github.com/MycroftAI/mimic3-voices)
|
* [Available voices](https://github.com/MycroftAI/mimic3-voices)
|
||||||
* [How does it work?](mimic3_tts/#architecture)
|
* [Documentation](https://mycroft-ai.gitbook.io/docs/mycroft-technologies/mimic-tts/coming-soon-mimic-3)
|
||||||
|
* [How does it work?](https://mycroft-ai.gitbook.io/docs/mycroft-technologies/mimic-tts/coming-soon-mimic-3#how-it-works)
|
||||||
|
|
||||||
|
|
||||||
## Use Cases
|
## Quickstart
|
||||||
|
|
||||||
* [Mycroft TTS plugin](#mycroft-tts-plugin)
|
|
||||||
* `mycroft-say 'Hello world.'`
|
|
||||||
* [Web server](#web-server-and-client)
|
|
||||||
* `curl -X POST --data 'Hello world.' --output - localhost:59125/api/tts | aplay`
|
|
||||||
* Drop-in [replacement for MaryTTS](#marytts-compatibility)
|
|
||||||
* [Command-line tool](#command-line-tools)
|
|
||||||
* `mimic3 'Hello world.' | aplay`
|
|
||||||
* [Voice for screen reader](mimic3_tts/#speech-dispatcher)
|
|
||||||
* `spd-say 'Hello world.'`
|
|
||||||
|
|
||||||
|
|
||||||
## Dependencies
|
|
||||||
|
|
||||||
Mimic 3 requires:
|
|
||||||
|
|
||||||
* Python 3.7 or higher
|
|
||||||
* The [Onnx runtime](https://onnxruntime.ai/)
|
|
||||||
* [gruut](https://github.com/rhasspy/gruut) or [eSpeak-ng](https://github.com/espeak-ng/espeak-ng) or [epitran](https://github.com/dmort27/epitran/) (depending on the voice)
|
|
||||||
|
|
||||||
|
|
||||||
## Installation
|
|
||||||
|
|
||||||
|
|
||||||
### eSpeak
|
|
||||||
|
|
||||||
Some voices depend on [eSpeak-ng](https://github.com/espeak-ng/espeak-ng), specifically `libespeak-ng.so`. For those voices, make sure that libespeak-ng is installed with:
|
|
||||||
|
|
||||||
``` sh
|
|
||||||
sudo apt-get install libespeak-ng1
|
|
||||||
```
|
|
||||||
|
|
||||||
On 32-bit ARM platforms (a.k.a. `armv7l` or `armhf`), you will also need some extra libraries:
|
|
||||||
|
|
||||||
``` sh
|
|
||||||
sudo apt-get install libatomic1 libgomp1 libatlas-base-dev
|
|
||||||
```
|
|
||||||
|
|
||||||
|
|
||||||
### Mycroft TTS Plugin
|
### Mycroft TTS Plugin
|
||||||
|
|
||||||
Install the plugin:
|
|
||||||
|
|
||||||
``` sh
|
``` sh
|
||||||
|
# Install system packages
|
||||||
|
sudo apt-get install libespeak-ng1
|
||||||
|
|
||||||
|
# Install plugin
|
||||||
mycroft-pip install mycroft-plugin-tts-mimic3[all]
|
mycroft-pip install mycroft-plugin-tts-mimic3[all]
|
||||||
```
|
|
||||||
|
|
||||||
Enable the plugin in your [mycroft.conf](https://mycroft-ai.gitbook.io/docs/using-mycroft-ai/customizations/mycroft-conf) file:
|
# Activate plugin
|
||||||
|
|
||||||
``` sh
|
|
||||||
mycroft-config set tts.module mimic3_tts_plug
|
mycroft-config set tts.module mimic3_tts_plug
|
||||||
|
|
||||||
|
# Start mycroft
|
||||||
|
mycroft-start all
|
||||||
```
|
```
|
||||||
|
|
||||||
or you can manually add the following to `mycroft.conf` with `mycroft-config edit user`:
|
See [documentation](https://mycroft-ai.gitbook.io/docs/mycroft-technologies/mimic-tts/coming-soon-mimic-3#tts-plugin-for-mycroft-ai) for more details.
|
||||||
|
|
||||||
``` json
|
|
||||||
"tts": {
|
|
||||||
"module": "mimic3_tts_plug"
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
|
|
||||||
See the [plugin's documentation](https://github.com/MycroftAI/plugin-tts-mimic3) for more options.
|
### Web Server
|
||||||
|
|
||||||
|
|
||||||
### Docker image
|
|
||||||
|
|
||||||
A pre-built Docker image is available for the following platforms:
|
|
||||||
|
|
||||||
* `linux/amd64`
|
|
||||||
* For desktops and laptops (`x86_64` CPUs)
|
|
||||||
* `linux/arm64`
|
|
||||||
* For Raspberry 3/4 and Zero 2 with [64-bit Pi OS](https://www.raspberrypi.com/news/raspberry-pi-os-64-bit/)
|
|
||||||
* `linux/arm/v7`
|
|
||||||
* For Raspberry Pi 1/2/3/4 and Zero 2 with 32-bit Pi OS
|
|
||||||
|
|
||||||
Install/update with:
|
|
||||||
|
|
||||||
``` sh
|
|
||||||
docker pull mycroftai/mimic3
|
|
||||||
```
|
|
||||||
|
|
||||||
Once installed, check out the following scripts for running:
|
|
||||||
|
|
||||||
* [`mimic3`](docker/mimic3)
|
|
||||||
* [`mimic3-server`](docker/mimic3-server)
|
|
||||||
* [`mimic3-download`](docker/mimic3-download)
|
|
||||||
|
|
||||||
Or you can manually run the web server with:
|
|
||||||
|
|
||||||
``` sh
|
``` sh
|
||||||
docker run \
|
docker run \
|
||||||
|
|
@ -107,234 +40,43 @@ docker run \
|
||||||
'mycroftai/mimic3'
|
'mycroftai/mimic3'
|
||||||
```
|
```
|
||||||
|
|
||||||
Voices will be automatically downloaded to `${HOME}/.local/share/mycroft/mimic3/voices`
|
Visit [http://localhost:59125](http://localhost:59125) or from another terminal:
|
||||||
|
|
||||||
|
|
||||||
### Debian Package
|
|
||||||
|
|
||||||
Grab the Debian package from the [latest release](https://github.com/mycroftAI/mimic3/releases) for your platform:
|
|
||||||
|
|
||||||
* `mycroft-mimic3-tts_<version>_amd64.deb`
|
|
||||||
* For desktops and laptops (`x86_64` CPUs)
|
|
||||||
* `mycroft-mimic3-tts_<version>_arm64.deb`
|
|
||||||
* For Raspberry 3/4 and Zero 2 with [64-bit Pi OS](https://www.raspberrypi.com/news/raspberry-pi-os-64-bit/)
|
|
||||||
* `mycroft-mimic3-tts_<version>_armhf.deb`
|
|
||||||
* For Raspberry Pi 1/2/3/4 and Zero 2 with 32-bit Pi OS
|
|
||||||
|
|
||||||
Once downloaded, install the package with (note the `./`):
|
|
||||||
|
|
||||||
``` sh
|
|
||||||
sudo apt install ./mycroft-mimic3-tts_<version>_<platform>.deb
|
|
||||||
```
|
|
||||||
|
|
||||||
Once installed, the following commands will be available in `/usr/bin`:
|
|
||||||
|
|
||||||
* `mimic3`
|
|
||||||
* `mimic3-server`
|
|
||||||
* `mimic3-download`
|
|
||||||
|
|
||||||
|
|
||||||
### Using pip
|
|
||||||
|
|
||||||
Install the command-line tool:
|
|
||||||
|
|
||||||
``` sh
|
|
||||||
pip install mycroft-mimic3-tts[all]
|
|
||||||
```
|
|
||||||
|
|
||||||
Once installed, the following commands will be available:
|
|
||||||
|
|
||||||
* `mimic3`
|
|
||||||
* `mimic3-download`
|
|
||||||
* `mimic3-server`
|
|
||||||
|
|
||||||
Language support can be selectively installed by replacing `all` with:
|
|
||||||
|
|
||||||
* `de` - German
|
|
||||||
* `es` - Spanish
|
|
||||||
* `fa` - Farsi
|
|
||||||
* `fr` - French
|
|
||||||
* `it` - Italian
|
|
||||||
* `nl` - Dutch
|
|
||||||
* `ru` - Russian
|
|
||||||
* `sw` - Kiswahili
|
|
||||||
|
|
||||||
Excluding `[..]` entirely will install support for English only.
|
|
||||||
|
|
||||||
|
|
||||||
### From Source
|
|
||||||
|
|
||||||
Clone the repository:
|
|
||||||
|
|
||||||
``` sh
|
|
||||||
git clone https://github.com/MycroftAI/mimic3.git
|
|
||||||
```
|
|
||||||
|
|
||||||
Run the install script:
|
|
||||||
|
|
||||||
``` sh
|
|
||||||
cd mimic3/
|
|
||||||
./install.sh
|
|
||||||
```
|
|
||||||
|
|
||||||
A virtual environment will be created in `mimic3/.venv` and each of the Python modules will be installed in editiable mode (`pip install -e`).
|
|
||||||
|
|
||||||
Once installed, the following commands will be available in `.venv/bin`:
|
|
||||||
|
|
||||||
* `mimic3`
|
|
||||||
* `mimic3-server`
|
|
||||||
* `mimic3-download`
|
|
||||||
|
|
||||||
|
|
||||||
## Voice Keys
|
|
||||||
|
|
||||||
Mimic 3 references voices with the format:
|
|
||||||
|
|
||||||
* `<language>_<region>/<dataset>_<quality>` for single speaker voices, and
|
|
||||||
* `<language>_<region>/<dataset>_<quality>#<speaker>` for multi-speaker voices
|
|
||||||
* `<speaker>` can be a name or number starting at 0
|
|
||||||
* Speaker names come from a voice's `speakers.txt` file
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
For example, the default [Alan Pope](https://popey.me/) voice key is `en_UK/apope_low`.
|
|
||||||
The [CMU Arctic voice](https://github.com/MycroftAI/mimic3-voices/tree/master/voices/en_US/cmu-arctic_low) contains multiple speakers, with a commonly used voice being `en_US/cmu-arctic_low#slt`.
|
|
||||||
|
|
||||||
Voices are automatically downloaded from [Github](https://github.com/MycroftAI/mimic3-voices) and stored in `${HOME}/.local/share/mycroft/mimic3` (technically `${XDG_DATA_HOME}/mycroft/mimic3`). You can also [manually download them](#downloading-voices).
|
|
||||||
|
|
||||||
|
|
||||||
## Running
|
|
||||||
|
|
||||||
|
|
||||||
### Command-Line Tools
|
|
||||||
|
|
||||||
The `mimic3` command can be used to synthesize audio on the command line:
|
|
||||||
|
|
||||||
``` sh
|
|
||||||
mimic3 --voice 'en_UK/apope_low' 'My hovercraft is full of eels.' > hovercraft_eels.wav
|
|
||||||
```
|
|
||||||
|
|
||||||
See [voice keys](#voice-keys) for how to reference voices and speakers.
|
|
||||||
|
|
||||||
See `mimic3 --help` or the [CLI documentation](mimic3_tts/) for more details.
|
|
||||||
|
|
||||||
|
|
||||||
#### Downloading Voices
|
|
||||||
|
|
||||||
Mimic 3 automatically downloads voices when they're first used, but you can manually download them too with `mimic3-download`.
|
|
||||||
|
|
||||||
For example:
|
|
||||||
|
|
||||||
``` sh
|
|
||||||
mimic3-download 'en_US/*'
|
|
||||||
```
|
|
||||||
|
|
||||||
will download all U.S. English voices to `${HOME}/.local/share/mycroft/mimic3/voices`.
|
|
||||||
|
|
||||||
See `mimic3-download --help` for more options.
|
|
||||||
|
|
||||||
|
|
||||||
### Web Server and Client
|
|
||||||
|
|
||||||
Start a web server with `mimic3-server` and visit `http://localhost:59125` to view the web UI.
|
|
||||||
|
|
||||||
To access the server from outside your device, run `mimic3-server --host 0.0.0.0` (see also `--port <PORT>`).
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
The following endpoints are available:
|
|
||||||
|
|
||||||
* `/api/tts`
|
|
||||||
* `POST` text or [SSML](#ssml) and receive WAV audio back
|
|
||||||
* Use `?voice=` to select a different [voice/speaker](#voice-keys)
|
|
||||||
* Set `Content-Type` to `application/ssml+xml` (or use `?ssml=1`) for [SSML](#ssml) input
|
|
||||||
* `/api/voices`
|
|
||||||
* Returns a JSON list of available voices
|
|
||||||
|
|
||||||
An [OpenAPI](https://www.openapis.org/) test page is also available at `http://localhost:59125/openapi`
|
|
||||||
|
|
||||||
See `mimic3-server --help` for the [web server documentation](mimic3_http/) for more details.
|
|
||||||
|
|
||||||
|
|
||||||
#### Web Client
|
|
||||||
|
|
||||||
The `mimic3` program provides an interface to the Mimic 3 web server when the `--remote` option is given.
|
|
||||||
|
|
||||||
Assuming you have started `mimic3-server` and can access `http://localhost:59125`, then:
|
|
||||||
|
|
||||||
``` sh
|
|
||||||
mimic3 --remote --voice 'en_UK/apope_low' 'My hovercraft is full of eels.' > hovercraft_eels.wav
|
|
||||||
```
|
|
||||||
|
|
||||||
If your server is somewhere besides `localhost`, use `mimic3 --remote <URL> ...`
|
|
||||||
|
|
||||||
See `mimic3 --help` for more options.
|
|
||||||
|
|
||||||
|
|
||||||
## CUDA Acceleration
|
|
||||||
|
|
||||||
If you have a GPU with support for CUDA, you can accelerate synthesis with the `--cuda` flag when running `mimic3` or `mimic3-server`. This requires you to install the [onnxruntime-gpu](https://pypi.org/project/onnxruntime-gpu/) Python package.
|
|
||||||
|
|
||||||
Using [nvidia-docker](https://github.com/NVIDIA/nvidia-docker) is highly recommended. See [ Dockerfile.gpu](Dockerfile.gpu) for an example of how to build a compatible container.
|
|
||||||
|
|
||||||
|
|
||||||
## MaryTTS Compatibility
|
|
||||||
|
|
||||||
Use the Mimic 3 web server as a drop-in replacement for [MaryTTS](http://mary.dfki.de/), for example with [Home Assistant](https://www.home-assistant.io/integrations/marytts/).
|
|
||||||
|
|
||||||
Make sure to use a Mimic 3 [voice key](#voice-keys) like `en_UK/apope_low` instead of a MaryTTS voice name.
|
|
||||||
|
|
||||||
For Mycroft, you can use this instead of [the plugin](https://github.com/MycroftAI/plugin-tts-mimic3) by running:
|
|
||||||
|
|
||||||
|
|
||||||
``` sh
|
``` sh
|
||||||
mycroft-config edit user
|
curl -X POST --data 'Hello world.' --output - localhost:59125/api/tts | aplay
|
||||||
|
|
||||||
```
|
```
|
||||||
|
|
||||||
and then adding the following:
|
See [documentation](https://mycroft-ai.gitbook.io/docs/mycroft-technologies/mimic-tts/coming-soon-mimic-3#web-server) for more details.
|
||||||
|
|
||||||
``` json
|
|
||||||
"tts": {
|
### Command-Line Tool
|
||||||
"module": "marytts",
|
|
||||||
"marytts": {
|
``` sh
|
||||||
"url": "http://localhost:59125",
|
# Install system packages
|
||||||
"voice": "en_UK/apope_low"
|
sudo apt-get install libespeak-ng1
|
||||||
}
|
|
||||||
|
# Create virtual environment
|
||||||
|
python3 -m venv .venv
|
||||||
|
source .venv/bin/activate
|
||||||
|
pip3 install --upgrade pip
|
||||||
|
|
||||||
|
pip3 install mycroft-mimic3-tts[all]
|
||||||
```
|
```
|
||||||
|
|
||||||
|
Now you can run:
|
||||||
|
|
||||||
## SSML
|
``` sh
|
||||||
|
mimic3 'Hello world.' | aplay
|
||||||
A [subset of SSML](mimic3_tts/#SSML) (Speech Synthesis Markup Language) is supported.
|
|
||||||
|
|
||||||
For example:
|
|
||||||
|
|
||||||
``` xml
|
|
||||||
<speak>
|
|
||||||
<voice name="en_UK/apope_low">
|
|
||||||
<s>
|
|
||||||
Welcome to the world of speech synthesis.
|
|
||||||
</s>
|
|
||||||
</voice>
|
|
||||||
<break time="3s" />
|
|
||||||
<voice name="en_US/cmu-arctic_low#slt">
|
|
||||||
<s>
|
|
||||||
<prosody volume="soft" rate="150%">
|
|
||||||
This is a <say-as interpret-as="number" format="ordinal">2</say-as> voice.
|
|
||||||
</prosody>
|
|
||||||
</s>
|
|
||||||
</voice>
|
|
||||||
</speak>
|
|
||||||
```
|
```
|
||||||
|
|
||||||
will speak the two sentences with different voices and a 3 second second pause in between. The second sentence will also have the number "2" pronounced as "second" (ordinal form).
|
Use `mimic3-server` and `mimic3 --remote ...` for repeated usage (much faster).
|
||||||
|
|
||||||
SSML `<say-as>` support varies between voice types:
|
See [documentation](https://mycroft-ai.gitbook.io/docs/mycroft-technologies/mimic-tts/coming-soon-mimic-3#command-line-interface) for more details.
|
||||||
|
|
||||||
* [gruut](https://github.com/rhasspy/gruut/#ssml)
|
|
||||||
* [eSpeak-ng](http://espeak.sourceforge.net/ssml.html)
|
---
|
||||||
* [epitran](https://github.com/dmort27/epitran/) voices do not currently support `<say-as>`
|
|
||||||
* Character-based voices do not currently support `<say-as>`
|
|
||||||
|
|
||||||
|
|
||||||
## License
|
## License
|
||||||
|
|
|
||||||
Binary file not shown.
|
Before Width: | Height: | Size: 58 KiB After Width: | Height: | Size: 58 KiB |
|
|
@ -2,56 +2,12 @@
|
||||||
|
|
||||||
A small HTTP web server for the [Mimic 3](https://github.com/MycroftAI/mimic3) text to speech system.
|
A small HTTP web server for the [Mimic 3](https://github.com/MycroftAI/mimic3) text to speech system.
|
||||||
|
|
||||||
[Available voices](https://github.com/MycroftAI/mimic3-voices)
|
* [Available voices](https://github.com/MycroftAI/mimic3-voices)
|
||||||
|
* [Documentation](https://mycroft-ai.gitbook.io/docs/mycroft-technologies/mimic-tts/coming-soon-mimic-3)
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
## Running the Server
|
## License
|
||||||
|
|
||||||
``` sh
|
Mimic 3 is available under the [AGPL v3 license](LICENSE)
|
||||||
mimic3-server
|
|
||||||
```
|
|
||||||
|
|
||||||
This will start a web server at `http://localhost:59125`
|
|
||||||
|
|
||||||
See `mimic3-server --debug` for more options.
|
|
||||||
|
|
||||||
|
|
||||||
### Endpoints
|
|
||||||
|
|
||||||
* `/api/tts`
|
|
||||||
* `POST` text or [SSML](#ssml) and receive WAV audio back
|
|
||||||
* Use `?voice=` to select a different [voice/speaker](#voice-keys)
|
|
||||||
* Set `Content-Type` to `application/ssml+xml` (or use `?ssml=1`) for [SSML](#ssml) input
|
|
||||||
* `/api/voices`
|
|
||||||
* Returns a JSON list of available voices
|
|
||||||
|
|
||||||
An [OpenAPI](https://www.openapis.org/) test page is also available at `http://localhost:59125/openapi`
|
|
||||||
|
|
||||||
|
|
||||||
### CUDA Acceleration
|
|
||||||
|
|
||||||
If you have a GPU with support for CUDA, you can accelerate synthesis with the `--cuda` flag. This requires you to install the [onnxruntime-gpu](https://pypi.org/project/onnxruntime-gpu/) Python package.
|
|
||||||
|
|
||||||
Using [nvidia-docker](https://github.com/NVIDIA/nvidia-docker) is highly recommended. See the `Dockerfile.gpu` file in the parent repository for an example of how to build a compatible container.
|
|
||||||
|
|
||||||
|
|
||||||
## Running the Client
|
|
||||||
|
|
||||||
Assuming you have started `mimic3-server` and can access `http://localhost:59125`, then:
|
|
||||||
|
|
||||||
``` sh
|
|
||||||
mimic3 --remote --voice 'en_UK/apope_low' 'My hovercraft is full of eels.' > hovercraft_eels.wav
|
|
||||||
```
|
|
||||||
|
|
||||||
If your server is somewhere besides `localhost`, use `mimic3 --remote <URL> ...`
|
|
||||||
|
|
||||||
See `mimic3 --help` for more options.
|
|
||||||
|
|
||||||
|
|
||||||
## MaryTTS Compatibility
|
|
||||||
|
|
||||||
Use the Mimic 3 web server as a drop-in replacement for [MaryTTS](http://mary.dfki.de/), for example with [Home Assistant](https://www.home-assistant.io/integrations/marytts/).
|
|
||||||
|
|
||||||
Make sure to use a compatible [voice key](#voice-keys) like `en_UK/apope_low`.
|
|
||||||
|
|
|
||||||
|
|
@ -1,348 +1,11 @@
|
||||||
# Mimic 3
|
# Mimic 3
|
||||||
|
|
||||||
A fast and local neural text to speech system for [Mycroft](https://mycroft.ai/) and the [Mark II](https://mycroft.ai/product/mark-ii/).
|
A fast and local neural text to speech system developed by [Mycroft](https://mycroft.ai/) for the [Mark II](https://mycroft.ai/product/mark-ii/).
|
||||||
|
|
||||||
* [Available voices](https://github.com/MycroftAI/mimic3-voices)
|
* [Available voices](https://github.com/MycroftAI/mimic3-voices)
|
||||||
* [Mimic 3 Architecture](#architecture)
|
* [Documentation](https://mycroft-ai.gitbook.io/docs/mycroft-technologies/mimic-tts/coming-soon-mimic-3)
|
||||||
|
|
||||||
|
|
||||||
## Command-Line Tools
|
|
||||||
|
|
||||||
|
|
||||||
### mimic3
|
|
||||||
|
|
||||||
|
|
||||||
#### Basic Synthesis
|
|
||||||
|
|
||||||
```sh
|
|
||||||
mimic3 --voice <voice> "<text>" > output.wav
|
|
||||||
```
|
|
||||||
|
|
||||||
where `<voice>` is a [voice key](https://github.com/MycroftAI/mimic3/#voice-keys) like `en_UK/apope_low`.
|
|
||||||
`<TEXT>` may contain multiple sentences, which will be combined in the final output WAV file. These can also be [split into separate WAV files](#multiple-wav-output).
|
|
||||||
|
|
||||||
|
|
||||||
#### SSML Synthesis
|
|
||||||
|
|
||||||
```sh
|
|
||||||
mimic3 --ssml --voice <voice> "<ssml>" > output.wav
|
|
||||||
```
|
|
||||||
|
|
||||||
where `<ssml>` is valid [SSML](https://www.w3.org/TR/speech-synthesis11/). Not all SSML features are supported, see [the documentation](#ssml) for details.
|
|
||||||
|
|
||||||
If your SSML contains `<mark>` tags, add `--mark-file <file>` to the command-line and use `--interactive` mode. As the marks are encountered, their names will be written on separate lines to the file:
|
|
||||||
|
|
||||||
```sh
|
|
||||||
mimic3 --ssml --interactive --mark-file - '<speak>Test 1. <mark name="here" /> Test 2.</speak>'
|
|
||||||
```
|
|
||||||
|
|
||||||
|
|
||||||
#### Long Texts
|
|
||||||
|
|
||||||
If your text is very long, and you would like to listen to it as its being synthesized, use `--interactive` mode:
|
|
||||||
|
|
||||||
```sh
|
|
||||||
mimic3 --interactive < long.txt
|
|
||||||
```
|
|
||||||
|
|
||||||
Each input line will be synthesized and played (see `--play-program`). By default, 5 sentences will be kept in an output queue, only blocking synthesis when the queue is full. You can adjust this value with `--result-queue-size`.
|
|
||||||
|
|
||||||
If your long text is fixed-width with blank lines separating paragraphs like those from [Project Gutenberg](https://www.gutenberg.org/), use the `--process-on-blank-line` option so that sentences will not be broken at line boundaries. For example, you can listen to "Alice in Wonderland" like this:
|
|
||||||
|
|
||||||
```sh
|
|
||||||
curl --output - 'https://www.gutenberg.org/files/11/11-0.txt' | \
|
|
||||||
mimic3 --interactive --process-on-blank-line
|
|
||||||
```
|
|
||||||
|
|
||||||
|
|
||||||
#### Multiple WAV Output
|
|
||||||
|
|
||||||
With `--output-dir` set to a directory, Mimic 3 will output a separate WAV file for each sentence:
|
|
||||||
|
|
||||||
```sh
|
|
||||||
mimic3 'Test 1. Test 2.' --output-dir /path/to/wavs
|
|
||||||
```
|
|
||||||
|
|
||||||
By default, each WAV file will be named using the (slightly modified) text of the sentence. You can have WAV files named using a timestamp instead with `--output-naming time`. For full control of the output naming, the `--csv` command-line flag indicates that each sentence is of the form `id|text` where `id` will be the name of the WAV file.
|
|
||||||
|
|
||||||
```sh
|
|
||||||
cat << EOF |
|
|
||||||
s01|The birch canoe slid on the smooth planks.
|
|
||||||
s02|Glue the sheet to the dark blue background.
|
|
||||||
s03|It's easy to tell the depth of a well.
|
|
||||||
s04|These days a chicken leg is a rare dish.
|
|
||||||
s05|Rice is often served in round bowls.
|
|
||||||
s06|The juice of lemons makes fine punch.
|
|
||||||
s07|The box was thrown beside the parked truck.
|
|
||||||
s08|The hogs were fed chopped corn and garbage.
|
|
||||||
s09|Four hours of steady work faced us.
|
|
||||||
s10|Large size in stockings is hard to sell.
|
|
||||||
EOF
|
|
||||||
mimic3 --csv --output-dir /path/to/wavs
|
|
||||||
```
|
|
||||||
|
|
||||||
You can adjust the delimiter with `--csv-delimiter <delimiter>`.
|
|
||||||
|
|
||||||
Additionally, you can use the `--csv-voice` option to specify a different voice or speaker for each line:
|
|
||||||
|
|
||||||
```sh
|
|
||||||
cat << EOF |
|
|
||||||
s01|#awb|The birch canoe slid on the smooth planks.
|
|
||||||
s02|#rms|Glue the sheet to the dark blue background.
|
|
||||||
s03|#slt|It's easy to tell the depth of a well.
|
|
||||||
s04|#ksp|These days a chicken leg is a rare dish.
|
|
||||||
s05|#clb|Rice is often served in round bowls.
|
|
||||||
s06|#aew|The juice of lemons makes fine punch.
|
|
||||||
s07|#bdl|The box was thrown beside the parked truck.
|
|
||||||
s08|#lnh|The hogs were fed chopped corn and garbage.
|
|
||||||
s09|#jmk|Four hours of steady work faced us.
|
|
||||||
s10|en_UK/apope_low|Large size in stockings is hard to sell.
|
|
||||||
EOF
|
|
||||||
mimic3 --voice 'en_US/cmu-arctic_low' --csv-voice --output-dir /path/to/wavs
|
|
||||||
```
|
|
||||||
|
|
||||||
The second contain can contain a `#<speaker>` or an entirely different voice!
|
|
||||||
|
|
||||||
|
|
||||||
#### Interactive Mode
|
|
||||||
|
|
||||||
With `--interactive`, Mimic 3 will switch into interactive mode. After entering a sentence, it will be played with `--play-program`.
|
|
||||||
|
|
||||||
```sh
|
|
||||||
mimic3 --interactive
|
|
||||||
Reading text from stdin...
|
|
||||||
Hello world!<ENTER>
|
|
||||||
```
|
|
||||||
|
|
||||||
Use `CTRL+D` or `CTRL+C` to exit.
|
|
||||||
|
|
||||||
|
|
||||||
#### Noise and Length Settings
|
|
||||||
|
|
||||||
Synthesis has the following additional parameters:
|
|
||||||
|
|
||||||
* `--noise-scale` and `--noise-w`
|
|
||||||
* Determine the speaker volatility during synthesis
|
|
||||||
* 0-1, default is 0.667 and 0.8 respectively
|
|
||||||
* `--length-scale` - makes the voice speaker slower (> 1) or faster (< 1)
|
|
||||||
|
|
||||||
Individual voices have default settings for these parameters in their `config.json` files (under `inference`).
|
|
||||||
|
|
||||||
|
|
||||||
#### List Voices
|
|
||||||
|
|
||||||
```sh
|
|
||||||
mimic3 --voices
|
|
||||||
```
|
|
||||||
|
|
||||||
|
|
||||||
#### CUDA Acceleration
|
|
||||||
|
|
||||||
If you have a GPU with support for CUDA, you can accelerate synthesis with the `--cuda` flag. This requires you to install the [onnxruntime-gpu](https://pypi.org/project/onnxruntime-gpu/) Python package.
|
|
||||||
|
|
||||||
Using [nvidia-docker](https://github.com/NVIDIA/nvidia-docker) is highly recommended. See the `Dockerfile.gpu` file in the parent repository for an example of how to build a compatible container.
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
### mimic3-download
|
|
||||||
|
|
||||||
Mimic 3 automatically downloads voices when they're first used, but you can manually download them too with `mimic3-download`.
|
|
||||||
|
|
||||||
For example:
|
|
||||||
|
|
||||||
``` sh
|
|
||||||
mimic3-download 'en_US/*'
|
|
||||||
```
|
|
||||||
|
|
||||||
will download all U.S. English voices to `${HOME}/.local/share/mycroft/mimic3` (technically `${XDG_DATA_HOME}/mimic3`).
|
|
||||||
|
|
||||||
See `mimic3-download --help` for more options.
|
|
||||||
|
|
||||||
|
|
||||||
## SSML
|
|
||||||
|
|
||||||
A subset of [SSML](https://www.w3.org/TR/speech-synthesis11/) (Speech Synthesis Markup Language) is supported:
|
|
||||||
|
|
||||||
* `<speak>` - wrap around SSML text
|
|
||||||
* `lang` - set language for document
|
|
||||||
* `<s>` - sentence (disables automatic sentence breaking)
|
|
||||||
* `lang` - set language for sentence
|
|
||||||
* `<w>` / `<token>` - word (disables automatic tokenization)
|
|
||||||
* `<voice name="...">` - set voice of inner text
|
|
||||||
* `voice` - name or language of voice
|
|
||||||
* Name format is `tts:voice` (e.g., "glow-speak:en-us_mary_ann") or `tts:voice#speaker_id` (e.g., "coqui-tts:en_vctk#p228")
|
|
||||||
* If one of the supported languages, a preferred voice is used (override with `--preferred-voice <lang> <voice>`)
|
|
||||||
* `<prosody attribute="value">` - change speaking attributes
|
|
||||||
* Supported `attribute` names:
|
|
||||||
* `volume` - speaking volume
|
|
||||||
* number in [0, 100] - 0 is silent, 100 is loudest (default)
|
|
||||||
* +X, -X, +X%, -X% - absolute/percent offset from current volume
|
|
||||||
* one of "default", "silent", "x-loud", "loud", "medium", "soft", "x-soft"
|
|
||||||
* `rate` - speaking rate
|
|
||||||
* number - 1 is default rate, < 1 is slower, > 1 is faster
|
|
||||||
* X% - 100% is default rate, 50% is half speed, 200% is twice as fast
|
|
||||||
* one of "default", "x-fast", "fast", "medium", "slow", "x-slow"
|
|
||||||
* `<say-as interpret-as="">` - force interpretation of inner text
|
|
||||||
* `interpret-as` one of "spell-out", "date", "number", "time", or "currency"
|
|
||||||
* `format` - way to format text depending on `interpret-as`
|
|
||||||
* number - one of "cardinal", "ordinal", "digits", "year"
|
|
||||||
* date - string with "d" (cardinal day), "o" (ordinal day), "m" (month), or "y" (year)
|
|
||||||
* `<break time="">` - Pause for given amount of time
|
|
||||||
* time - seconds ("123s") or milliseconds ("123ms")
|
|
||||||
* `<sub alias="">` - substitute `alias` for inner text
|
|
||||||
* `<phoneme ph="">` - supply phonemes for inner text
|
|
||||||
* See `phonemes.txt` in voice directory for available phonemes
|
|
||||||
* Phonemes may need to be separated by whitespace
|
|
||||||
|
|
||||||
SSML `<say-as>` support varies between voice types:
|
|
||||||
|
|
||||||
* [gruut](https://github.com/rhasspy/gruut/#ssml)
|
|
||||||
* [eSpeak-ng](http://espeak.sourceforge.net/ssml.html)
|
|
||||||
* Character-based voices do not currently support `<say-as>`
|
|
||||||
|
|
||||||
|
|
||||||
## Speech Dispatcher
|
|
||||||
|
|
||||||
Mimic 3 can be used with the [Orca screen reader](https://help.gnome.org/users/orca/stable/) for Linux via [speech-dispatcher](https://github.com/brailcom/speechd).
|
|
||||||
|
|
||||||
After [installing Mimic 3](https://github.com/MycroftAI/mimic3/#installation), make sure you also have speech-dispatcher installed:
|
|
||||||
|
|
||||||
``` sh
|
|
||||||
sudo apt-get install speech-dispatcher
|
|
||||||
```
|
|
||||||
|
|
||||||
Create the file `/etc/speech-dispatcher/modules/mimic3-generic.conf` with the contents:
|
|
||||||
|
|
||||||
``` text
|
|
||||||
GenericExecuteSynth "printf %s \'$DATA\' | /path/to/mimic3 --remote --voice \'$VOICE\' --stdout | $PLAY_COMMAND"
|
|
||||||
AddVoice "en-us" "MALE1" "en_UK/apope_low"
|
|
||||||
```
|
|
||||||
|
|
||||||
You will need `sudo` access to do this. Make sure to change `/path/to/mimic3` to wherever you installed Mimic 3. Note that the `--remote` option is used to connect to a local Mimic 3 web server (use `--remote <URL>` if your server is somewhere besides `localhost`).
|
|
||||||
|
|
||||||
To change the voice later, you only need to replace `en_UK/apope_low`.
|
|
||||||
|
|
||||||
Next, edit the existing file `/etc/speech-dispatcher/speechd.conf` and ensure the following settings are present:
|
|
||||||
|
|
||||||
``` text
|
|
||||||
DefaultVoiceType "MALE1"
|
|
||||||
DefaultModule mimic3-generic
|
|
||||||
```
|
|
||||||
|
|
||||||
Restart speech-dispatcher with:
|
|
||||||
|
|
||||||
``` sh
|
|
||||||
sudo systemctl restart speech-dispatcher
|
|
||||||
```
|
|
||||||
|
|
||||||
and test it out with:
|
|
||||||
|
|
||||||
``` sh
|
|
||||||
spd-say 'Hello from speech dispatcher.'
|
|
||||||
```
|
|
||||||
|
|
||||||
|
|
||||||
### Systemd Service
|
|
||||||
|
|
||||||
To ensure that Mimic 3 runs at boot, create a systemd service at `$HOME/.config/systemd/user/mimic3.service` with the contents:
|
|
||||||
|
|
||||||
``` text
|
|
||||||
[Unit]
|
|
||||||
Description=Run Mimic 3 web server
|
|
||||||
Documentation=https://github.com/MycroftAI/mimic3
|
|
||||||
|
|
||||||
[Service]
|
|
||||||
ExecStart=/path/to/mimic3-server
|
|
||||||
|
|
||||||
[Install]
|
|
||||||
WantedBy=default.target
|
|
||||||
```
|
|
||||||
|
|
||||||
Make sure to change `/path/to/mimic3-server` to wherever you installed Mimic 3.
|
|
||||||
|
|
||||||
Refresh the systemd services:
|
|
||||||
|
|
||||||
``` sh
|
|
||||||
systemctl --user daemon-reload
|
|
||||||
```
|
|
||||||
|
|
||||||
Now try starting the service:
|
|
||||||
|
|
||||||
``` sh
|
|
||||||
systemctl --user start mimic3
|
|
||||||
```
|
|
||||||
|
|
||||||
If that's successful, ensure it starts at boot:
|
|
||||||
|
|
||||||
``` sh
|
|
||||||
systemctl --user enable mimic3
|
|
||||||
```
|
|
||||||
|
|
||||||
|
|
||||||
## Architecture
|
|
||||||
|
|
||||||
Mimic 3 uses the [VITS](https://arxiv.org/abs/2106.06103), a "Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speech". VITS is a combination of the [GlowTTS duration predictor](https://arxiv.org/abs/2005.11129) and the [HiFi-GAN vocoder](https://arxiv.org/abs/2010.05646).
|
|
||||||
|
|
||||||
Our implementation is heavily based on [Jaehyeon Kim's PyTorch model](https://github.com/jaywalnut310/vits), with the addition of [Onnx runtime](https://onnxruntime.ai/) export for speed.
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
### Phoneme Ids
|
|
||||||
|
|
||||||
At a high level, Mimic 3 performs two important tasks:
|
|
||||||
|
|
||||||
1. Converting raw text to numeric input for the VITS TTS model, and
|
|
||||||
2. Using the model to transform numeric input into audio output
|
|
||||||
|
|
||||||
The second step is the same for every voice, but the first step (text to numbers) varies. There are currently three implementations of step 1, described below.
|
|
||||||
|
|
||||||
|
|
||||||
### gruut Phoneme-based Voices
|
|
||||||
|
|
||||||
Voices that use [gruut](https://github.com/rhasspy/gruut/) for phonemization.
|
|
||||||
|
|
||||||
gruut normalizes text and phonemizes words according to a lexicon, with a pre-trained grapheme-to-phoneme model used to guess unknown word pronunciations.
|
|
||||||
|
|
||||||
|
|
||||||
### eSpeak Phoneme-based Voices
|
|
||||||
|
|
||||||
Voices that use [eSpeak-ng](https://github.com/espeak-ng/espeak-ng) for phonemization (via [espeak-phonemizer](https://github.com/rhasspy/espeak-phonemizer)).
|
|
||||||
|
|
||||||
eSpeak-ng normalizes and phonemizes text using internal rules and lexicons. It supports a large number of languages, and can handle many textual forms.
|
|
||||||
|
|
||||||
|
|
||||||
### Character-based Voices
|
|
||||||
|
|
||||||
Voices whose "phonemes" are characters from an alphabet, typically with some punctuation.
|
|
||||||
|
|
||||||
For voices whose orthography (writing system) is close enough to its spoken form, character-based voices allow for skipping the phonemization step. However, these voices do not support text normalization, so numbers, dates, etc. must be written out.
|
|
||||||
|
|
||||||
|
|
||||||
### Epitran-based Voices
|
|
||||||
|
|
||||||
Voices that use [epitran](https://github.com/dmort27/epitran/) for phonemization.
|
|
||||||
|
|
||||||
epitran uses rules to generate phonetic pronunciations from text. It does not support text normalization, however, so numbers, dates, etc. must be written out.
|
|
||||||
|
|
||||||
|
|
||||||
### Components of a Voice Model
|
|
||||||
|
|
||||||
Voice models are stored in a directory with a specific layout:
|
|
||||||
|
|
||||||
* `<language>_<region>` (e.g., `en_UK`)
|
|
||||||
* `<voice-name>_<quality>` (e.g., `apope_low`)
|
|
||||||
* `ALIASES` - alternative names for the voice, one per line (optional)
|
|
||||||
* `config.json` - training/inference configuration (see [code](https://github.com/MycroftAI/mimic3/blob/master/mimic3-tts/mimic3_tts/config.py) for details)
|
|
||||||
* `generator.onnx` - exported inference model (see `ids_to_audio` method in [`voice.py`](https://github.com/MycroftAI/mimic3/blob/master/mimic3-tts/mimic3_tts/voice.py))
|
|
||||||
* `LICENSE` - text, name, or URL of voice model license
|
|
||||||
* `phoneme_map.txt` - mapping from source phoneme to destination phoneme(s) (optional)
|
|
||||||
* `phonemes.txt` - mapping from integer ids to phonemes (`_` = padding, `^` = beginning of utterance, `$` = end of utterance, `#` = word break)
|
|
||||||
* `README.md` - description of the voice
|
|
||||||
* `SOURCE` - URL(s) of the dataset(s) this voice was trained on
|
|
||||||
* `VERSION` - version of the voice in the format "MAJOR.Minor.bugfix" (e.g. "1.0.2")
|
|
||||||
|
|
||||||
|
|
||||||
## License
|
## License
|
||||||
|
|
||||||
See [license file](LICENSE)
|
Mimic 3 is available under the [AGPL v3 license](LICENSE)
|
||||||
|
|
|
||||||
3
opentts_abc/README.md
Normal file
3
opentts_abc/README.md
Normal file
|
|
@ -0,0 +1,3 @@
|
||||||
|
# OpenTTS ABC
|
||||||
|
|
||||||
|
Abstract base classes used by the [Mimic 3](https://github.com/MycroftAI/mimic3) text to speech system.
|
||||||
Loading…
Add table
Add a link
Reference in a new issue