Mimic 3

A fast and local neural text to speech system developed by Mycroft for the Mark II.

• Available voices
• How does it work?

Use Cases

• Mycroft TTS plugin
  • mycroft-say 'Hello world.'
• Web server
  • curl -X POST --data 'Hello world.' --output - localhost:59125/api/tts | aplay
  • Drop-in replacement for MaryTTS
• Command-line tool
  • mimic3 'Hello world.' | aplay
• Voice for screen reader
  • spd-say 'Hello world.'

Dependencies

Mimic 3 requires:

• Python 3.7 or higher
• The Onnx runtime
• gruut or eSpeak-ng or epitran (depending on the voice)

Installation

eSpeak

Some voices depend on eSpeak-ng, specifically libespeak-ng.so. For those voices, make sure that libespeak-ng is installed with:

sudo apt-get install libespeak-ng1

On 32-bit ARM platforms (a.k.a. armv7l or armhf), you will also need some extra libraries:

sudo apt-get install libatomic1 libgomp1 libatlas-base-dev

Mycroft TTS Plugin

Install the plugin:

mycroft-pip install mycroft-plugin-tts-mimic3[all]

Enable the plugin in your mycroft.conf file:

mycroft-config set tts.module mimic3_tts_plug

or you can manually add the following to mycroft.conf with mycroft-config edit user:

"tts": {
  "module": "mimic3_tts_plug"
}

See the plugin's documentation for more options.

Docker image

A pre-built Docker image is available for the following platforms:

• linux/amd64
  • For desktops and laptops (x86_64 CPUs)
• linux/arm64
  • For Raspberry 3/4 and Zero 2 with 64-bit Pi OS
• linux/arm/v7
  • For Raspberry Pi 1/2/3/4 and Zero 2 with 32-bit Pi OS

Install/update with:

docker pull mycroftai/mimic3

Once installed, check out the following scripts for running:

• mimic3
• mimic3-server
• mimic3-download

Or you can manually run the web server with:

docker run \
       -it \
       -p 59125:59125 \
       -v "${HOME}/.local/share/mycroft/mimic3:/home/mimic3/.local/share/mycroft/mimic3" \
       'mycroftai/mimic3'

Voices will be automatically downloaded to ${HOME}/.local/share/mycroft/mimic3/voices

Debian Package

Grab the Debian package from the latest release for your platform:

• mycroft-mimic3-tts_<version>_amd64.deb
  • For desktops and laptops (x86_64 CPUs)
• mycroft-mimic3-tts_<version>_arm64.deb
  • For Raspberry 3/4 and Zero 2 with 64-bit Pi OS
• mycroft-mimic3-tts_<version>_armhf.deb
  • For Raspberry Pi 1/2/3/4 and Zero 2 with 32-bit Pi OS

Once downloaded, install the package with (note the ./):

sudo apt install ./mycroft-mimic3-tts_<version>_<platform>.deb

Once installed, the following commands will be available in /usr/bin:

• mimic3
• mimic3-server
• mimic3-download

Using pip

Install the command-line tool:

pip install mycroft-mimic3-tts[all]

Once installed, the following commands will be available:

• mimic3
• mimic3-download
• mimic3-server

Language support can be selectively installed by replacing all with:

• de - German
• es - Spanish
• fa - Farsi
• fr - French
• it - Italian
• nl - Dutch
• ru - Russian
• sw - Kiswahili

Excluding [..] entirely will install support for English only.

From Source

Clone the repository:

git clone https://github.com/MycroftAI/mimic3.git

Run the install script:

cd mimic3/
./install.sh

A virtual environment will be created in mimic3/.venv and each of the Python modules will be installed in editiable mode (pip install -e).

Once installed, the following commands will be available in .venv/bin:

• mimic3
• mimic3-server
• mimic3-download

Voice Keys

Mimic 3 references voices with the format:

• <language>_<region>/<dataset>_<quality> for single speaker voices, and
• <language>_<region>/<dataset>_<quality>#<speaker> for multi-speaker voices
  • <speaker> can be a name or number starting at 0
  • Speaker names come from a voice's speakers.txt file

For example, the default Alan Pope voice key is en_UK/apope_low. The CMU Arctic voice contains multiple speakers, with a commonly used voice being en_US/cmu-arctic_low#slt.

Voices are automatically downloaded from Github and stored in ${HOME}/.local/share/mycroft/mimic3 (technically ${XDG_DATA_HOME}/mycroft/mimic3). You can also manually download them.

Running

Command-Line Tools

The mimic3 command can be used to synthesize audio on the command line:

mimic3 --voice 'en_UK/apope_low' 'My hovercraft is full of eels.' > hovercraft_eels.wav

See voice keys for how to reference voices and speakers.

See mimic3 --help or the CLI documentation for more details.

Downloading Voices

Mimic 3 automatically downloads voices when they're first used, but you can manually download them too with mimic3-download.

For example:

mimic3-download 'en_US/*'

will download all U.S. English voices to ${HOME}/.local/share/mycroft/mimic3/voices.

See mimic3-download --help for more options.

Web Server and Client

Start a web server with mimic3-server and visit http://localhost:59125 to view the web UI.

The following endpoints are available:

• /api/tts
  • POST text or SSML and receive WAV audio back
  • Use ?voice= to select a different voice/speaker
  • Set Content-Type to application/ssml+xml (or use ?ssml=1) for SSML input
• /api/voices
  • Returns a JSON list of available voices

An OpenAPI test page is also available at http://localhost:59125/openapi

See mimic3-server --help for the web server documentation for more details.

Web Client

The mimic3 program provides an interface to the Mimic 3 web server when the --remote option is given.

Assuming you have started mimic3-server and can access http://localhost:59125, then:

mimic3 --remote --voice 'en_UK/apope_low' 'My hovercraft is full of eels.' > hovercraft_eels.wav

If your server is somewhere besides localhost, use mimic3 --remote <URL> ...

See mimic3 --help for more options.

CUDA Acceleration

If you have a GPU with support for CUDA, you can accelerate synthesis with the --cuda flag when running mimic3 or mimic3-server. This requires you to install the onnxruntime-gpu Python package.

Using nvidia-docker is highly recommended. See Dockerfile.gpu for an example of how to build a compatible container.

MaryTTS Compatibility

Use the Mimic 3 web server as a drop-in replacement for MaryTTS, for example with Home Assistant.

Make sure to use a compatible voice key like en_UK/apope_low.

For Mycroft, you can use this instead of the plugin by running:

mycroft-config edit user

and then adding the following:

"tts": {
"module": "marytts",
"marytts": {
    "url": "http://localhost:59125",
    "voice": "en_UK/apope_low"
}

SSML

A subset of SSML (Speech Synthesis Markup Language) is supported.

For example:

<speak>
  <voice name="en_UK/apope_low">
    <s>
      Welcome to the world of speech synthesis.
    </s>
  </voice>
  <break time="3s" />
  <voice name="en_US/cmu-arctic_low#slt">
    <s>
      <prosody volume="soft" rate="150%">
        This is a <say-as interpret-as="number" format="ordinal">2</say-as> voice.
      </prosody>
    </s>
  </voice>
</speak>

will speak the two sentences with different voices and a 3 second second pause in between. The second sentence will also have the number "2" pronounced as "second" (ordinal form).

SSML <say-as> support varies between voice types:

• gruut
• eSpeak-ng
• epitran voices do not currently support <say-as>
• Character-based voices do not currently support <say-as>

License

See license file