Centralize documentation
This commit is contained in:
Michael Hansen
parent
4a781a1a64
commit
07f3c00d9b
5 changed files with 47 additions and 683 deletions
332
README.md
332
README.md
|
|
@ -5,99 +5,32 @@
|
|||
A fast and local neural text to speech system developed by [Mycroft](https://mycroft.ai/) for the [Mark II](https://mycroft.ai/product/mark-ii/).
|
||||
|
||||
* [Available voices](https://github.com/MycroftAI/mimic3-voices)
|
||||
* [How does it work?](mimic3_tts/#architecture)
|
||||
* [Documentation](https://mycroft-ai.gitbook.io/docs/mycroft-technologies/mimic-tts/coming-soon-mimic-3)
|
||||
* [How does it work?](https://mycroft-ai.gitbook.io/docs/mycroft-technologies/mimic-tts/coming-soon-mimic-3#how-it-works)
|
||||
|
||||
|
||||
## Use Cases
|
||||
|
||||
* [Mycroft TTS plugin](#mycroft-tts-plugin)
|
||||
* `mycroft-say 'Hello world.'`
|
||||
* [Web server](#web-server-and-client)
|
||||
* `curl -X POST --data 'Hello world.' --output - localhost:59125/api/tts | aplay`
|
||||
* Drop-in [replacement for MaryTTS](#marytts-compatibility)
|
||||
* [Command-line tool](#command-line-tools)
|
||||
* `mimic3 'Hello world.' | aplay`
|
||||
* [Voice for screen reader](mimic3_tts/#speech-dispatcher)
|
||||
* `spd-say 'Hello world.'`
|
||||
|
||||
|
||||
## Dependencies
|
||||
|
||||
Mimic 3 requires:
|
||||
|
||||
* Python 3.7 or higher
|
||||
* The [Onnx runtime](https://onnxruntime.ai/)
|
||||
* [gruut](https://github.com/rhasspy/gruut) or [eSpeak-ng](https://github.com/espeak-ng/espeak-ng) or [epitran](https://github.com/dmort27/epitran/) (depending on the voice)
|
||||
|
||||
|
||||
## Installation
|
||||
|
||||
|
||||
### eSpeak
|
||||
|
||||
Some voices depend on [eSpeak-ng](https://github.com/espeak-ng/espeak-ng), specifically `libespeak-ng.so`. For those voices, make sure that libespeak-ng is installed with:
|
||||
|
||||
``` sh
|
||||
sudo apt-get install libespeak-ng1
|
||||
```
|
||||
|
||||
On 32-bit ARM platforms (a.k.a. `armv7l` or `armhf`), you will also need some extra libraries:
|
||||
|
||||
``` sh
|
||||
sudo apt-get install libatomic1 libgomp1 libatlas-base-dev
|
||||
```
|
||||
|
||||
## Quickstart
|
||||
|
||||
### Mycroft TTS Plugin
|
||||
|
||||
Install the plugin:
|
||||
|
||||
``` sh
|
||||
# Install system packages
|
||||
sudo apt-get install libespeak-ng1
|
||||
|
||||
# Install plugin
|
||||
mycroft-pip install mycroft-plugin-tts-mimic3[all]
|
||||
```
|
||||
|
||||
Enable the plugin in your [mycroft.conf](https://mycroft-ai.gitbook.io/docs/using-mycroft-ai/customizations/mycroft-conf) file:
|
||||
|
||||
``` sh
|
||||
# Activate plugin
|
||||
mycroft-config set tts.module mimic3_tts_plug
|
||||
|
||||
# Start mycroft
|
||||
mycroft-start all
|
||||
```
|
||||
|
||||
or you can manually add the following to `mycroft.conf` with `mycroft-config edit user`:
|
||||
|
||||
``` json
|
||||
"tts": {
|
||||
"module": "mimic3_tts_plug"
|
||||
}
|
||||
```
|
||||
See [documentation](https://mycroft-ai.gitbook.io/docs/mycroft-technologies/mimic-tts/coming-soon-mimic-3#tts-plugin-for-mycroft-ai) for more details.
|
||||
|
||||
|
||||
See the [plugin's documentation](https://github.com/MycroftAI/plugin-tts-mimic3) for more options.
|
||||
|
||||
|
||||
### Docker image
|
||||
|
||||
A pre-built Docker image is available for the following platforms:
|
||||
|
||||
* `linux/amd64`
|
||||
* For desktops and laptops (`x86_64` CPUs)
|
||||
* `linux/arm64`
|
||||
* For Raspberry 3/4 and Zero 2 with [64-bit Pi OS](https://www.raspberrypi.com/news/raspberry-pi-os-64-bit/)
|
||||
* `linux/arm/v7`
|
||||
* For Raspberry Pi 1/2/3/4 and Zero 2 with 32-bit Pi OS
|
||||
|
||||
Install/update with:
|
||||
|
||||
``` sh
|
||||
docker pull mycroftai/mimic3
|
||||
```
|
||||
|
||||
Once installed, check out the following scripts for running:
|
||||
|
||||
* [`mimic3`](docker/mimic3)
|
||||
* [`mimic3-server`](docker/mimic3-server)
|
||||
* [`mimic3-download`](docker/mimic3-download)
|
||||
|
||||
Or you can manually run the web server with:
|
||||
### Web Server
|
||||
|
||||
``` sh
|
||||
docker run \
|
||||
|
|
@ -107,234 +40,43 @@ docker run \
|
|||
'mycroftai/mimic3'
|
||||
```
|
||||
|
||||
Voices will be automatically downloaded to `${HOME}/.local/share/mycroft/mimic3/voices`
|
||||
|
||||
|
||||
### Debian Package
|
||||
|
||||
Grab the Debian package from the [latest release](https://github.com/mycroftAI/mimic3/releases) for your platform:
|
||||
|
||||
* `mycroft-mimic3-tts_<version>_amd64.deb`
|
||||
* For desktops and laptops (`x86_64` CPUs)
|
||||
* `mycroft-mimic3-tts_<version>_arm64.deb`
|
||||
* For Raspberry 3/4 and Zero 2 with [64-bit Pi OS](https://www.raspberrypi.com/news/raspberry-pi-os-64-bit/)
|
||||
* `mycroft-mimic3-tts_<version>_armhf.deb`
|
||||
* For Raspberry Pi 1/2/3/4 and Zero 2 with 32-bit Pi OS
|
||||
|
||||
Once downloaded, install the package with (note the `./`):
|
||||
|
||||
``` sh
|
||||
sudo apt install ./mycroft-mimic3-tts_<version>_<platform>.deb
|
||||
```
|
||||
|
||||
Once installed, the following commands will be available in `/usr/bin`:
|
||||
|
||||
* `mimic3`
|
||||
* `mimic3-server`
|
||||
* `mimic3-download`
|
||||
|
||||
|
||||
### Using pip
|
||||
|
||||
Install the command-line tool:
|
||||
|
||||
``` sh
|
||||
pip install mycroft-mimic3-tts[all]
|
||||
```
|
||||
|
||||
Once installed, the following commands will be available:
|
||||
|
||||
* `mimic3`
|
||||
* `mimic3-download`
|
||||
* `mimic3-server`
|
||||
|
||||
Language support can be selectively installed by replacing `all` with:
|
||||
|
||||
* `de` - German
|
||||
* `es` - Spanish
|
||||
* `fa` - Farsi
|
||||
* `fr` - French
|
||||
* `it` - Italian
|
||||
* `nl` - Dutch
|
||||
* `ru` - Russian
|
||||
* `sw` - Kiswahili
|
||||
|
||||
Excluding `[..]` entirely will install support for English only.
|
||||
|
||||
|
||||
### From Source
|
||||
|
||||
Clone the repository:
|
||||
|
||||
``` sh
|
||||
git clone https://github.com/MycroftAI/mimic3.git
|
||||
```
|
||||
|
||||
Run the install script:
|
||||
|
||||
``` sh
|
||||
cd mimic3/
|
||||
./install.sh
|
||||
```
|
||||
|
||||
A virtual environment will be created in `mimic3/.venv` and each of the Python modules will be installed in editiable mode (`pip install -e`).
|
||||
|
||||
Once installed, the following commands will be available in `.venv/bin`:
|
||||
|
||||
* `mimic3`
|
||||
* `mimic3-server`
|
||||
* `mimic3-download`
|
||||
|
||||
|
||||
## Voice Keys
|
||||
|
||||
Mimic 3 references voices with the format:
|
||||
|
||||
* `<language>_<region>/<dataset>_<quality>` for single speaker voices, and
|
||||
* `<language>_<region>/<dataset>_<quality>#<speaker>` for multi-speaker voices
|
||||
* `<speaker>` can be a name or number starting at 0
|
||||
* Speaker names come from a voice's `speakers.txt` file
|
||||
|
||||
|
||||
|
||||
For example, the default [Alan Pope](https://popey.me/) voice key is `en_UK/apope_low`.
|
||||
The [CMU Arctic voice](https://github.com/MycroftAI/mimic3-voices/tree/master/voices/en_US/cmu-arctic_low) contains multiple speakers, with a commonly used voice being `en_US/cmu-arctic_low#slt`.
|
||||
|
||||
Voices are automatically downloaded from [Github](https://github.com/MycroftAI/mimic3-voices) and stored in `${HOME}/.local/share/mycroft/mimic3` (technically `${XDG_DATA_HOME}/mycroft/mimic3`). You can also [manually download them](#downloading-voices).
|
||||
|
||||
|
||||
## Running
|
||||
|
||||
|
||||
### Command-Line Tools
|
||||
|
||||
The `mimic3` command can be used to synthesize audio on the command line:
|
||||
|
||||
``` sh
|
||||
mimic3 --voice 'en_UK/apope_low' 'My hovercraft is full of eels.' > hovercraft_eels.wav
|
||||
```
|
||||
|
||||
See [voice keys](#voice-keys) for how to reference voices and speakers.
|
||||
|
||||
See `mimic3 --help` or the [CLI documentation](mimic3_tts/) for more details.
|
||||
|
||||
|
||||
#### Downloading Voices
|
||||
|
||||
Mimic 3 automatically downloads voices when they're first used, but you can manually download them too with `mimic3-download`.
|
||||
|
||||
For example:
|
||||
|
||||
``` sh
|
||||
mimic3-download 'en_US/*'
|
||||
```
|
||||
|
||||
will download all U.S. English voices to `${HOME}/.local/share/mycroft/mimic3/voices`.
|
||||
|
||||
See `mimic3-download --help` for more options.
|
||||
|
||||
|
||||
### Web Server and Client
|
||||
|
||||
Start a web server with `mimic3-server` and visit `http://localhost:59125` to view the web UI.
|
||||
|
||||
To access the server from outside your device, run `mimic3-server --host 0.0.0.0` (see also `--port <PORT>`).
|
||||
|
||||
|
||||
|
||||
The following endpoints are available:
|
||||
|
||||
* `/api/tts`
|
||||
* `POST` text or [SSML](#ssml) and receive WAV audio back
|
||||
* Use `?voice=` to select a different [voice/speaker](#voice-keys)
|
||||
* Set `Content-Type` to `application/ssml+xml` (or use `?ssml=1`) for [SSML](#ssml) input
|
||||
* `/api/voices`
|
||||
* Returns a JSON list of available voices
|
||||
|
||||
An [OpenAPI](https://www.openapis.org/) test page is also available at `http://localhost:59125/openapi`
|
||||
|
||||
See `mimic3-server --help` for the [web server documentation](mimic3_http/) for more details.
|
||||
|
||||
|
||||
#### Web Client
|
||||
|
||||
The `mimic3` program provides an interface to the Mimic 3 web server when the `--remote` option is given.
|
||||
|
||||
Assuming you have started `mimic3-server` and can access `http://localhost:59125`, then:
|
||||
|
||||
``` sh
|
||||
mimic3 --remote --voice 'en_UK/apope_low' 'My hovercraft is full of eels.' > hovercraft_eels.wav
|
||||
```
|
||||
|
||||
If your server is somewhere besides `localhost`, use `mimic3 --remote <URL> ...`
|
||||
|
||||
See `mimic3 --help` for more options.
|
||||
|
||||
|
||||
## CUDA Acceleration
|
||||
|
||||
If you have a GPU with support for CUDA, you can accelerate synthesis with the `--cuda` flag when running `mimic3` or `mimic3-server`. This requires you to install the [onnxruntime-gpu](https://pypi.org/project/onnxruntime-gpu/) Python package.
|
||||
|
||||
Using [nvidia-docker](https://github.com/NVIDIA/nvidia-docker) is highly recommended. See [ Dockerfile.gpu](Dockerfile.gpu) for an example of how to build a compatible container.
|
||||
|
||||
|
||||
## MaryTTS Compatibility
|
||||
|
||||
Use the Mimic 3 web server as a drop-in replacement for [MaryTTS](http://mary.dfki.de/), for example with [Home Assistant](https://www.home-assistant.io/integrations/marytts/).
|
||||
|
||||
Make sure to use a Mimic 3 [voice key](#voice-keys) like `en_UK/apope_low` instead of a MaryTTS voice name.
|
||||
|
||||
For Mycroft, you can use this instead of [the plugin](https://github.com/MycroftAI/plugin-tts-mimic3) by running:
|
||||
Visit [http://localhost:59125](http://localhost:59125) or from another terminal:
|
||||
|
||||
|
||||
``` sh
|
||||
mycroft-config edit user
|
||||
curl -X POST --data 'Hello world.' --output - localhost:59125/api/tts | aplay
|
||||
|
||||
```
|
||||
|
||||
and then adding the following:
|
||||
See [documentation](https://mycroft-ai.gitbook.io/docs/mycroft-technologies/mimic-tts/coming-soon-mimic-3#web-server) for more details.
|
||||
|
||||
``` json
|
||||
"tts": {
|
||||
"module": "marytts",
|
||||
"marytts": {
|
||||
"url": "http://localhost:59125",
|
||||
"voice": "en_UK/apope_low"
|
||||
}
|
||||
|
||||
### Command-Line Tool
|
||||
|
||||
``` sh
|
||||
# Install system packages
|
||||
sudo apt-get install libespeak-ng1
|
||||
|
||||
# Create virtual environment
|
||||
python3 -m venv .venv
|
||||
source .venv/bin/activate
|
||||
pip3 install --upgrade pip
|
||||
|
||||
pip3 install mycroft-mimic3-tts[all]
|
||||
```
|
||||
|
||||
Now you can run:
|
||||
|
||||
## SSML
|
||||
|
||||
A [subset of SSML](mimic3_tts/#SSML) (Speech Synthesis Markup Language) is supported.
|
||||
|
||||
For example:
|
||||
|
||||
``` xml
|
||||
<speak>
|
||||
<voice name="en_UK/apope_low">
|
||||
<s>
|
||||
Welcome to the world of speech synthesis.
|
||||
</s>
|
||||
</voice>
|
||||
<break time="3s" />
|
||||
<voice name="en_US/cmu-arctic_low#slt">
|
||||
<s>
|
||||
<prosody volume="soft" rate="150%">
|
||||
This is a <say-as interpret-as="number" format="ordinal">2</say-as> voice.
|
||||
</prosody>
|
||||
</s>
|
||||
</voice>
|
||||
</speak>
|
||||
``` sh
|
||||
mimic3 'Hello world.' | aplay
|
||||
```
|
||||
|
||||
will speak the two sentences with different voices and a 3 second second pause in between. The second sentence will also have the number "2" pronounced as "second" (ordinal form).
|
||||
Use `mimic3-server` and `mimic3 --remote ...` for repeated usage (much faster).
|
||||
|
||||
SSML `<say-as>` support varies between voice types:
|
||||
See [documentation](https://mycroft-ai.gitbook.io/docs/mycroft-technologies/mimic-tts/coming-soon-mimic-3#command-line-interface) for more details.
|
||||
|
||||
* [gruut](https://github.com/rhasspy/gruut/#ssml)
|
||||
* [eSpeak-ng](http://espeak.sourceforge.net/ssml.html)
|
||||
* [epitran](https://github.com/dmort27/epitran/) voices do not currently support `<say-as>`
|
||||
* Character-based voices do not currently support `<say-as>`
|
||||
|
||||
---
|
||||
|
||||
|
||||
## License
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue