Assist, the voice assistant built into Home Assistant, can run without any cloud service. A voice command then goes through four stages on your own hardware: wake word detection, speech-to-text, the conversation agent that works out what you meant, and text-to-speech for the reply. Home Assistant calls this chain a pipeline, and each stage can use a different engine. The setup itself is a few clicks. Choosing the engines is where it goes right or wrong: the wrong speech-to-text engine on a Raspberry Pi turns every command into a long wait.

The steps below assume Home Assistant OS, where the engines are installed as apps (formerly called add-ons). On other installation types the same engines run as separate containers and connect through the Wyoming integration, as described near the end.

Choosing speech-to-text: Speech-to-Phrase or Whisper

The Home Assistant documentation lists two local options, and they behave very differently.

Speech-to-Phrase doesn't try to transcribe everything you say. It recognises only the sentences it knows, built from Assist's sentence templates and the names of the entities, areas and floors you exposed to Assist, plus your own sentence triggers. The app trains itself on those names and retrains when they change. According to the documentation it transcribes in under a second even on a Home Assistant Green or a Raspberry Pi 4. The price is coverage: only a subset of Assist commands works, and open-ended ones such as adding items to a shopping list, naming a timer or broadcasts don't work out of the box.

Whisper is an open-ended model that tries to transcribe anything. The documentation gives about 8 seconds per command on a Raspberry Pi 4 and under a second on an Intel NUC. It recommends Whisper only if you have powerful hardware or plan to go beyond home control, for example by connecting an LLM-based conversation agent.

For plain home control on a Pi or a Green, Speech-to-Phrase is the sensible default. Whisper makes sense on a mini PC, or on a separate, stronger computer, which the last section covers.

Check your language before you decide. The Speech-to-Phrase app in the official repository (version 1.4.x at the time of writing) lists 21 supported languages, Polish among them. Its developers are working on a rewrite in a separate repository, and their README says the rewrite currently serves only eight languages (Catalan, Czech, German, English, Spanish, French, Italian and Dutch). If your language isn't among them, read the changelog before updating the app later.

Text-to-speech: Piper

Piper is the local text-to-speech engine from the same team. Voice names follow the pattern <language>_<REGION>-<name>-<quality>, for example en_US-lessac-medium, which is the app's default. Polish voices, for example, include pl_PL-gosia-medium, pl_PL-darkman-medium and pl_PL-mc_speech-medium. Quality comes in four levels: x_low and low at 16 kHz, medium and high at 22.05 kHz. The app documentation says models up to medium run at usable speed on a Raspberry Pi 4, and the voice docs give about 1.6 seconds of speech generated per second on a Pi with medium models. Listen to the samples on the Piper samples page before choosing, because quality differs a lot between voices and languages.

The Whisper, Speech-to-Phrase and Piper apps are built for amd64 and aarch64 only, so a 32-bit ARM system won't run them.

Installing the apps and building the pipeline

  1. Go to Settings > Apps > Install app and install Speech-to-Phrase or Whisper, then Piper.
  2. Start both apps.
  3. Open Settings > Devices & services. The Wyoming integration should show both services as discovered. Select Add for each.
  4. Go to Settings > Voice assistants and select Add assistant.
  5. Give it a name and choose the language. Set Conversation agent to Home Assistant, Speech-to-text to the engine you installed, and Text-to-speech to Piper, then pick a voice. Set the language at each stage.

If Settings > Voice assistants shows no assistants at all, your configuration doesn't load the defaults. The documentation's fix is to add assist_pipeline: to configuration.yaml.

Two optional settings are worth knowing about. Acts as user tells Home Assistant whose request this is. A voice satellite has no user of its own, and the assistant can do only what the selected user is allowed to do. If you later switch the conversation agent to an LLM, Prefer handling commands locally sends each command to the local agent first and passes it on only when the local agent can't handle it.

For Whisper, set the app's language option explicitly. The auto setting works, but the app documentation warns that it is much slower because the model detects the language on every request. The default model: auto picks tiny-int8 on ARM devices such as the Raspberry Pi 4 and base-int8 on other hardware. Models are excluded from backups and downloaded again after a restore.

Exposing devices and naming them well

Assist controls only entities you have exposed to it. The documentation explains this as protection against controlling sensitive devices, such as locks and garage doors, by accident. Open Settings > Voice assistants, go to the Expose tab, and expose what you want to control by voice.

Names matter more than in the dashboard. Assist matches the names of entities and areas exactly as they are defined in Home Assistant, and the troubleshooting page gives a plain example: with an area called "bathroom", "turn on the lights in the bath" won't work. Aliases fix that. Entities, areas and floors can all have several of them, and Assist treats all aliases equally.

With Speech-to-Phrase, good names matter even more, because the model only knows the names it was trained on. In inflected languages there is one more catch: the Polish test sentences in the Home Assistant intents repository use area and device names in the form they have in Home Assistant ("włącz światła w Salon"). If a form you naturally say isn't recognised, add it as an alias.

Testing without saying a word

Two tools save a lot of talking to the microphone:

  • Settings > Tools > Assist (the sentences parser) shows which intent a typed sentence triggers and which entities it targets, without executing anything.
  • In Settings > Voice assistants, open your assistant and select Debug. Here you can pick a past run and see each stage, or use Run text pipeline to type a command that is actually executed.

If the text pipeline works but speech doesn't, the problem is in speech-to-text or in the audio. If even typed text fails, fix names, aliases or exposure first.

Where the microphone comes from

The pipeline needs something that listens. The documentation lists the Home Assistant companion app on Android and iOS, the Home Assistant Voice Preview Edition, and satellites you build yourself with ESPHome.

Wake words work in two ways. openWakeWord runs inside Home Assistant: the satellite streams audio and the server checks it for the wake word. The documentation says a Raspberry Pi 4 can handle 5 satellites streaming at once, and that openWakeWord currently works only for English wake words. microWakeWord runs on the device itself, on ESP32-S3 hardware and Android phones, and the wake word page lists three models for it: "okay nabu", "hey jarvis" and "alexa". For a non-English household this means the wake word stays an English phrase, while the commands after it can be in your language.

When the assistant understands but stays silent

The reply audio is generated in Home Assistant, and the satellite downloads it from a URL that Home Assistant gives it. If that URL is wrong, the command runs but no answer plays. Check Settings > System > Network, where the local network URL should be one the satellite can reach. The documentation recommends the Automatic option for most setups. With a YAML configuration that doesn't use default_config:, also make sure media_source: is present.

For an ESPHome satellite that hears badly or plays too quietly, the troubleshooting page shows how to record what Home Assistant actually receives (debug_recording_dir under assist_pipeline:) and how to tune the voice_assistant: block on the device: noise_suppression_level up to 4, auto_gain up to 31 dBFS, and volume_multiplier, which has no upper limit but distorts at high values. Delete the recordings when you're done, because they take disk space.

Running Whisper on a stronger computer

You don't have to run speech-to-text on the Home Assistant machine. The Wyoming integration can connect to a service running elsewhere on the local network: add the Wyoming Protocol integration manually and enter the host and port. The documentation's example is Whisper, the most demanding of the engines: it can run on a desktop or mini PC while Home Assistant stays on a Pi. The commands still never leave your home network, and the slow step runs on hardware that can handle it.