Skip to content

Commit

Permalink
Merge pull request #67 from FutureProofHomes/develop
Browse files Browse the repository at this point in the history
Adding first version of Voice Pipeline Setup
  • Loading branch information
that1guy authored Dec 9, 2024
2 parents 029f780 + e2c6ca8 commit 7b9a478
Show file tree
Hide file tree
Showing 18 changed files with 103 additions and 10 deletions.
Binary file added docs/assets/.png
Binary file not shown.
Binary file added docs/assets/Expose Entities.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/assets/Expose Settings.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/assets/Pipeline0.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/assets/Pipeline1.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/assets/Pipeline2.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/assets/Pipeline3.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/assets/Pipeline4.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/assets/Pipeline5.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/assets/Pipeline7.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/assets/Pipeline8.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/assets/googleGPT0.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/assets/googleGPT1.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/assets/googleGPT2.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
16 changes: 14 additions & 2 deletions docs/faqs.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,19 @@
<a id="faq_anchor_factory_reset"></a>
???+ note "I messed something up, How do I start over?"

[Perform a Factory Reset](/troubleshooting/#resetting-to-factory-settings)
[Perform a Factory Reset](/troubleshooting/#resetting-to-factory-settings){ .md-button .md-button--primary }

<a id="faq_anchor_inspect_logs"></a>
???+ note "How do I access the Sat1 logs?"

[If you want to look at the logs, click here](/troubleshooting/#inspecting-logs)
[If you want to look at the logs, click here](/troubleshooting/#inspecting-logs){ .md-button .md-button--primary }

<a id="faq_anchor_report_issue"></a>
???+ note "How do I get help or report issues?"

[There are many ways to contact us, click here](/troubleshooting/#need-more-help){ .md-button .md-button--primary }

<a id="faq_anchor_custom_wake_word"></a>
???+ note "How do I build my own custom wake word?"

[This will require time and GPU horsepower.](https://github.com/kahrendt/microWakeWord/blob/november-update/notebooks/basic_training_notebook.ipynb){ .md-button .md-button--primary }
6 changes: 5 additions & 1 deletion docs/index.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,9 @@
## Introduction
Congrats on your shiny new Satellite1 Voice Assistant & Multi-Sensor hardware! In this opening section we will teach you how to assemble your Satellite1 and get it fully functional. This guide is presented in several steps. Each step will bring more functionality to your new toy.

Use the links in the naviation bar on the left of this screen to walk through each section to get your device up and running. In this section, we'll get it assembled, powered on and recognized by Home Assistant.

## Assembling the Satellite1
Congrats on your shiny new Satellite1 Voice Assistant & Multi-Sensor hardware! In this opening section we will teach you how to assemble your Satellite1 and get it fully functional.

<figure markdown="span">
<video width="100%" class="video-js" loop="" autoplay="" preload="none" muted="" playsinline="" poster="https://futureproofhomes.net/cdn/shop/files/0001.jpg?v=1732152100">
Expand Down
11 changes: 6 additions & 5 deletions docs/streaming-music.md
Original file line number Diff line number Diff line change
@@ -1,12 +1,13 @@
## Set Up Music Assistant

Your new Satellite1 can control your home now but what about streaming music? The good news is there's a giant 25 Watt amplifier built into satellite one and if you have a good speaker or headphones (and the appropriate [power supply](Accessories.md)) you'll be able to stream high-quality music to it with music assistant from Home Assistant. Follow this excellent guide to get Music Assistant (MA) installed and configured to stream to your Satellite1.
Your new Satellite1 can control your home now but what about streaming music? The good news is there's a giant 25 Watt amplifier built into Satellite1 and if you have a good speaker or headphones (and the appropriate [power supply](/recommended-accessories/#power-supply)) you'll be able to stream high-quality music to it with music assistant from Home Assistant. Follow this excellent guide to get Music Assistant (MA) installed and configured to stream to your Satellite1.

A couple of tips:
- Spotify ONLY works if you are a paying subscriber. Free accounts will not work. This is a limitation of MA, and may or may not be addressed in future versions.
- Watch your volume! 25 watts of power can be LOUD!
!!! tip "A Couple Tips:"

[Install Music Assistant](https://music-assistant.io/installation/)
1. Spotify ONLY works if you are a paying subscriber. Free accounts will not work. This may or may not be addressed in future versions of MA.
2. Watch your volume! 25 watts of power can be LOUD!

[Install Music Assistant](https://music-assistant.io/installation/){ .md-button .md-button--primary }

To add your Satellite1 to MA, do the following:

Expand Down
80 changes: 78 additions & 2 deletions docs/voice-pipeline.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,79 @@
<!-- <iframe width="560" height="315" src="https://www.youtube.com/embed/yqWX86uT5jM?si=qK_A1XmaSsqYQ9js" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe> -->
## What is a Voice Pipeline?

Coming soon...
Think about your interaction with any voice assistant:

1. **Wake Word** - You say a special phrase, like "Hey, Jarvis!"
2. **Speech-to-Text** - Your voice command is recorded and converted to a text transcription.
3. **Conversation Agent** - The text transcription is processed by a rules-based engine (or perhaps an LLM) which executes your command and generates a text response.
4. **Text-to-Speech** - The text response is converted into a synthetic voice response that is played back through the speaker.

That's a voice pipeline. It's the backbone of any voice assistant. Each step in a voice pipeline can be modified and customized to fit your needs. What wake word do you want? What language are you speaking? Do you want a standard voice response or to hear Arnold Schwarzenegger speak back to you? Do you want Home Assistant, Google, or OpenAI to process and execute your command? Follow the steps below to set up a voice pipeline for your Satellite1.

Please read all of the below. Then we highly recommend **[upgrading to a magical fallback conversation agent](/voice-pipeline/#magical-fall-back-conversation-agent)**. Also, don't forget to **[expose entities to your Satellite1 voice assistant!](/voice-pipeline/#exposing-entities)**

## General Steps to Create a Voice Pipeline:

1. Go to **Settings -> Voice Assistants -> Add Assistant**
<img width="60%" alt="image" src="/assets/Pipeline0.png">

2. Name your pipeline. Select your preferred Conversation Agent, Speech-to-Text, and Text-to-Speech engine.
<img width="60%" alt="image" src="/assets/Pipeline1.png">

## Standard Home Assistant Conversation Agents

There are two standard voice pipelines we recommend trying out to get your feet wet:

???+ abstract "Home Assistant's Cloud Assist Pipeline (Requires paid Home Assistant Cloud account, response times are fast!)"
[Set up Cloud Assist Pipeline](https://www.home-assistant.io/voice_control/voice_remote_cloud_assistant/){ .md-button .md-button--primary }

???+ abstract "Home Assistant's Local Assist Pipeline (Free and completely private, response times depend on your hardware)"
[Set up Local Assist Pipeline](https://www.home-assistant.io/voice_control/voice_remote_local_assistant/){ .md-button .md-button--primary }

## Generative A.I. Conversation Agents

Once you have one of the standard pipelines above operational, you can decide to upgrade to a Generative AI conversation agent.

???+ abstract "Ollama AI Powered Conversation Agent (Free, requires a GPU, and can be hard to set up with proper function calling.)"
[Set up Ollama Conversation Agent](https://www.home-assistant.io/integrations/ollama/){ .md-button .md-button--primary }

???+ abstract "Google A.I Powered Pipeline (Free, but will collect your data)"
???+ danger "Warning: You're giving a company with non-deterministic artificial intelligence ability control of your home and collect data on your usage. Be cautious!"
[Set up Google AI Conversation Agent](https://www.home-assistant.io/integrations/google_generative_ai_conversation/){ .md-button .md-button--primary }

???+ abstract "OpenAI ChatGPT Powered Pipeline (Expensive, and not open at all, despite the marketing name.)"
???+ danger "Warning: You're giving a company with non-deterministic artificial intelligence ability control of your home and collect data on your usage. Be cautious!"
[Set up OpenAI ChatGPT Conversation Agent](https://www.home-assistant.io/integrations/openai_conversation/){ .md-button .md-button--primary }

## Magical "Fall back" Conversation Agent

???+ abstract "Combine a standard conversation agent with an AI conversation agent (It's like magic!)"
???+ tip "Requires Home Assistant version 2024.12.1 or later"
A "fall back" pipeline will first use a non-AI conversation agent to process your request, and if that fails it will _fall back_ to your preferred AI conversation agent. Combining these two conversation agents results in an almost magical voice experience and is highly recommended.

Simply toggle on the **"Prefer handling commands locally"** switch underneath your Generative AI conversation agent:
<img width="60%" alt="image" src="/assets/googleGPT0.png">

[Read more about this feature release](https://www.home-assistant.io/blog/2024/12/04/release-202412/#let-your-voice-assistant-fall-back-to-an-llm-based-agent){ .md-button .md-button--primary }

## Assign a Voice Pipeline & Wake Word to your Satellite1

1. Go to **Settings -> Devices & Services -> ESPHome** and click "1 device" under your Satellite1 device.
<img width="60%" alt="image" src="/assets/Pipeline4.png">

2. In the Configuration section, select your Voice Pipeline.
<img width="60%" alt="image" src="/assets/Pipeline5.png">

3. You can also set your preferred wake word. (NOTE: If you want to build your own custom wake word, then [read here](/faqs#faq_anchor_custom_wake_word))
<img width="60%" alt="image" src="/assets/Pipeline7.png">

Congratulations! You've created your own voice pipeline for your Satellite1.

## Exposing Entities

Your home assistant likely has hundreds if not thousands of entities. If you want to use them in your voice assistant, then you need to expose them to your voice assistant. Here's how:

1. Go to **Settings -> Voice Assistants -> Expose** and select the entities you want to control via your voice assistant.
<img width="60%" alt="image" src="/assets/Expose Entities.png">

2. Enable the "Assist" toggle switch and consider adding alias names that you might use when referring to the entity.
<img width="60%" alt="image" src="/assets/Expose Settings.png">

0 comments on commit 7b9a478

Please sign in to comment.