With HA 2024.7 Local Intents (specifically setting timers via built in intents) are not working

[admn-monkey]

Ultimately, I loaded the default ESP32-S3-Box-3 install via the Home Assistant instructions to test if it was related to the esphome_audio repo. With the HA ESPHOME default esp32-s3-box-3.yaml firmware the ESP32 works as expected with now built in timer via voice support.

With the esphome_audio, it appears none of the built in intents are working.

I will admit the voice recognition and audio capabilities are so much better with the esphome_audio load, but local intents are critical for ongoing functionality.

Am I missing something obvious for how to make built in intents work with esphome_audio?

[gnumpi]

Can you be more specific with what is not working with esphome_audio? Could you provide configs and logs for that issue? Thx.

[formatBCE]

For me, timers are still working fine. After HA 2024.7.1 my yelling about having default intents overriding custom ones was heard and fixed, so now it tried to run my own timers logic. But when I disabled my custom intent for that, timers started working on device again.

[admn-monkey]

For me, timers are still working fine. After HA 2024.7.1 my yelling about having default intents overriding custom ones was heard and fixed, so now it tried to run my own timers logic. But when I disabled my custom intent for that, timers started working on device again.

@formatBCE

How did you disable your custom intents? Was that by just disabling the related automation(s)?

@gnumpi

I'm seeing this in my main HA logs when I ask the device "Start a X minute timer":

2024-07-06 11:22:07.950 ERROR (MainThread) [homeassistant.components.conversation.default_agent] Intent handling error Traceback (most recent call last): File "/usr/src/homeassistant/homeassistant/components/conversation/default_agent.py", line 347, in async_process intent_response = await intent.async_handle( ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/homeassistant/homeassistant/helpers/intent.py", line 140, in async_handle result = await handler.async_handle(intent) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/homeassistant/homeassistant/components/intent/timers.py", line 836, in async_handle raise TimersNotSupportedError(intent_obj.device_id) homeassistant.components.intent.timers.TimersNotSupportedError: Device does not support timers: device_id=_xxxxxxxxxxxxxxxxxxxxxxxxxxx_

[formatBCE]

@admn-monkey I just commented out the custom sentence part, and reloaded conversation in Dev tools.
But I suppose it's not your case, since your log is showing that device doesn't support timers. This happens, if you don't have at least on_timer_finished section in your voice_assistant config. I had it before.

[admn-monkey]

@gnumpi

YAML for the device is attached. There's nothing special about this; outside an announcement speaker on response, it's nearly exactly default to yours.

ESP32Config_redacted.yaml.txt

[formatBCE]

@gnumpi

YAML for the device is attached. There's nothing special about this; outside an announcement speaker on response, it's nearly exactly default to yours.

ESP32Config_redacted.yaml.txt

Yup, as I said, add lambda to notify user on timer finished to VA, and it will work.

[admn-monkey]

@gnumpi
YAML for the device is attached. There's nothing special about this; outside an announcement speaker on response, it's nearly exactly default to yours.
ESP32Config_redacted.yaml.txt

Yup, as I said, add lambda to notify user on timer finished to VA, and it will work.

@formatBCE

May I ask a favor? Could you copy in your on_timer_finished VA section from your device?

The documentation on that section is somewhat lacking. Plus, after messing with this for 2 hours now, it's not even moderately easy to just take the section from the ESPHome firmware ESP32-S3-Box-3 yaml and integrate it with the esphome_audio yaml. While I did get it in there and sloppily working, my attempt isn't functional. It will start the timer, yet when it's finished I appear to be locking out the assist voice recognition state.

@gnumpi

I do think the issue remains open considering that the esphome_audio yaml does not contain the on_timer_finished section and the one from any of the standard esphome reference firmwares are not structurally compatible without considerable effort on the end user.

[formatBCE]

@admn-monkey i kept it simple and stupid:

on_timer_finished:
    - script.execute: stop_voice_assistant
    - lambda: id(voice_assistant_phase) = ${voice_assist_timer_finished_phase_id};
    - script.execute: draw_display
    - wait_until:
        not:
          microphone.is_capturing:
    - media_player.play_media: http://homeassistant.local/local/sounds/timer-ding.mp3
    - delay: 2s
    - media_player.play_media: http://homeassistant.local/local/sounds/timer-ding.mp3
    - delay: 2s
    - media_player.play_media: http://homeassistant.local/local/sounds/timer-ding.mp3
    - delay: 2s
    - wait_until:
        not:
          media_player.is_playing:
    - script.execute: start_voice_assistant
    - script.execute: draw_display

It's pretty close to official S3-Box firmware here, but i decided to go with 3 "dings" instead of continuous dinging.

P.S. i don't think this is current repo's issue, as timers are additional functionality, and requires additional work outside of ESPHome.

[formatBCE]

@gnumpi by the way, probably you will have some suggestions on how to deal with situation, if player is playing when timer is finished?

[admn-monkey]

@formatBCE

Ah, thanks! It appears you've integrated the necessary core components of the esphome_audio firmware into something close to the default esphome wake-word-voice-assistant esp32-s3-box-3 firmware or similar. Rather than the other way around which is what I did. Is that a good assumption of your configuration?

The following do not exist in the esphome_audio fw yaml and of course aren't in my current one either:
voice_assistant_phase
draw_display
start_voice_assistant

I believe it will be necessary for me to completely start over and work in the direction I assume you took.

[formatBCE]

@admn-monkey yes, i used VA firmware for ESP32-S3-Box, and replaced their I2S configuration with this one. Was easier for me, since i've already had it done with default I2S.
You can find all missing chunks on the link i provided, draw_display script is just updating display, and start_voice_assistant deals with tasks on starting wake word or VA itself.

[admn-monkey]

@gnumpi

I'm going to leave this open since the current esphome_audio fw now excludes the major set timer feature of HA voice assist.

[formatBCE]

@gnumpi i guess whole part of working with voice assistant is a bit unclear so far.
Like, is there mechanism to pause player and then resume with same content on interaction with VA? E.g. pause on wake word detected, and resume on VA going to idle? Can it be done automatically, or manually from code?

[gnumpi]

@gnumpi by the way, probably you will have some suggestions on how to deal with situation, if player is playing when timer is finished?

We will need to add an option to the play_media action to play it as an announcement. Then you can pause the MP, and then play your "dings" as announcements, which makes the MP continue with the current song when the announcement is done.
Hope to find some time to add this additional option.

[formatBCE]

@gnumpi by the way, probably you will have some suggestions on how to deal with situation, if player is playing when timer is finished?

We will need to add an option to the play_media action to play it as an announcement. Then you can pause the MP, and then play your "dings" as announcements, which makes the MP continue with the current song when the announcement is done. Hope to find some time to add this additional option.

Thinking on it more, voice assistant component just takes media player, and we have no control how exactly it plays VA audio there. We can't set it to be played as announcement, even if media player itself will allow that.

I tried to make player pause on wake word detected, and it resumed playback automatically after TTS end, but looks like resumed from different place...
Timer "dings" are clearing the queue, so when timer is finished, it "dings" and stops playback after that.

Interesting, that while there's TTS or "ding", Music Assistant still registers it like current song playback (e.g. "pause" button shown, and timeline is updating itself.
A lot of inconsistency....