Investigate supporting power management in arduino-esp32 (DFS + tickless idle)

[igrr]

Related area

Power management

Hardware specification

All ESP32 series chips

Is your feature request related to a problem?

Recently a community member has mentioned that the current consumption of ESP32 connected to Wi-Fi is too high. While it is possible to reduce current consumption with ESP-IDF, this option is not accessible to Arduino users.

Describe the solution you'd like

To reduce current consumption in Wi-Fi connected state, several things are required:

• Power management should be enabled (CONFIG_PM_ENABLE).
  • Investigate what are the downsides enabling it if it is not turned on at run time.
• FreeRTOS tickless idle should be enabled (CONFIG_FREERTOS_USE_TICKLESS_IDLE).
  • Investigate what are the downsides enabling it if the DFS and automatic light sleep are not enabled at runtime.
• Enabling DFS and automatic light sleep at run time via some API.
• An easy-to-understand example showing how to achieve lower current consumption when Wi-Fi is connected.
• Documentation explaining the caveats (mostly related to peripherals and timers) of enabling the above mentioned features.

Describe alternatives you've considered

Keep not supporting DFS and tickless idle in Arduino. But then everyone keeps thinking that it's not possible to have Wi-Fi connection with lower average current.

Additional context

No response

I have checked existing list of Feature requests and the Contribution Guide

• I confirm I have checked existing list of Feature requests and Contribution Guide.

[our30K]

just want to pump this up , is there anyone working on this ? it has been a long while :)

[tshcherban]

i can take a look at this. recently i've been playing around with IDF and power management API, can try applying this to the arduino core.
@igrr @our30K can i sometimes ping you here if any questions?

[our30K]

that would be great, I am interested in both low power WIFI and BLE via arudino.

[tshcherban]

I've added arduino core and some of the arduino libs (OneWire, DallasTemperature) as a components to ESP-IDF.
Currently enabled CONFIG_PM_ENABLE, CONFIG_FREERTOS_USE_TICKLESS_IDLE and used automatic light-sleep with esp_wifi_set_ps(wifi_ps_type_t::WIFI_PS_MAX_MODEM) and esp_pm_configure.
Used a simple HttpClient request and OneWire library to test if it's alive. millis works as expected since it relies on esp_timer_get_time.
Will try using those APIs at runtime, enabling/disabling power-saving on demand. And i have nordic power profiler 2 kit, so can investigate how much gain from DFS vs static frequency.
Do you have any good ideas of what stuff can be broken most probably? so i can add those to my 'test sketch'
P.S. preliminary results on enabling DFS (no significant improvement in powersaving)

160 MHz, ~33 mA

10..160 MHz, ~33 mA

80..160 MHz, ~34 mA

On 10..160 MHz range started to observe Serial.print problems, especially on large amounts of text (I'm using it to print the results of a vTaskList function):

Serial log

Time since bootup: 17369385 us
Lock stats:
Name            Type            Arg    Active    Total_count    Time(us)  Time(%)
wifi            APB_FREQ_MAX    0      0         63             2268808   14 %
rtos1           CPU_FREQ_MAX    0      1         2955           4704990   28 %
rtos0           CPU_FREQ_MAX    0      1         2330           1252689   8  %

Mode stats:
Mode      CPU_freq    Time(us)    Time(%)
SLEEP     10 M        12339504    70%
APB_MIN   10 M        0           0 %
APB_MAX   80 M        1451702     8 %
CPU_MAX   160M        3167449     18%

Sleep stats:
light_sleep_counts:68  light_sleep_reject_counts:0
Task Name       Status  Prio    HWM     Task    Affinity
main            X       1       2164    4
IDLE            R       0       556     6
IDLE            R       0       544     5
tiT             B       18      1896    8
ipc0            S       24      520     1
Tmr Svc         B       1       1320    7
ipc1            S       24      484     ����e��������������er          S       22      3088    3
wifi            B       23      3752    11
arduino_events  B       �����J10
sys_evt         B       20      588     9

[tshcherban]

Also compared existing API from WiFiGenericClass::setSleep (withoud IDF menuconfig modifications), value WIFI_PS_MIN_MODEM is used by default (not for ESP32-S2 though)

[TD-er]

On 10..160 MHz range started to observe Serial.print problems, especially on large amounts of text (I'm using it to print the results of a vTaskList function):

Which is totally expected as below 80 MHz the clock source is likely to change from PLL_CLK to XTAL_CLK or RC_FAST_CLK.
I'm not sure all timings related to APB clock are updated when switching clock source or going below 80 MHz CPU frequency.
Well... I'm quite sure there are things getting out of sync between Arduino and ESP-IDF when you change it, but not sure what effect this will have.
Both ESP-IDF and Arduino tend to keep their own administration on this matter and I know there are parts of the code where both are not notifying each other (or rather listening to notifications).

N.B. It might be possible this "80 MHz" threshold differs among ESP32-variants as the ESP32-C2 for example does have a clock based on multiples of 60 MHz.

I can imagine the issues you saw with serial also depend on the used baud rate as not every baud rate can be perfectly matched with an integer factor from each clock frequency.
So some ratios may lead to a bigger "error" in matching clock frequencies.

[tshcherban]

Yep, below 80 MHz APB starts to change also. However Serial itself keeps working even on stable 10 MHz without DFS (mostly, i've experienced some problems with Serial and SIM800 modem, characters were lost. Had to rise freq to 20 MHz to keep it stable). For DFS-related issues I assume the problems begin when (the frequency) switching occurs in the middle of processing a Serial buffer.
I'll try inserting some APB_MAX lock until TX buffer is empty.

Both ESP-IDF and Arduino tend to keep their own administration on this matter and I know there are parts of the code where both are not notifying each other (or rather listening to notifications).

not sure what is this. Does the Arduino core have any notifications on clocks change?

[TD-er]

A few weeks ago I was looking into the wrongly reported APB frequency on an ESP32-C2 and then noticed some code in IDF which can send notifications on changes in these frequencies, but no code in Arduino to act on it.
And also some code in Arduino which computes timing parameters based on reported APB frequency.
If there is no synchronization between Arduino and IDF code about these changes, then I suspect some timings will be incorrect after run-time changes of the frequency.

[tshcherban]

I am interested in both low power WIFI and BLE via arudino

For BLE power consumption testing I've used a simple listener with interval 150 and window 75. Of course windows/interval ratio directly affects how much power would be needed for the radio.
I was not able to enable BLE light sleep on ESP32 (MH-ET Live D1 mini), seems it lacks 32 KHz crystal because I'm getting
W (618) clk: 32 kHz XTAL not found, switching to internal 150 kHz oscillator

For the ESP32-S3 (DevKit with power LED cut off) options are better, light sleep works ok for both Main XTAL and Internal 150 KHz oscillator.

power options used	bluetooth sleep clock	current, mA
-	-	99
modem sleep	Main XTAL	70
modem sleep	Internal 150 KHz	70
modem sleep, automatic sleep	Main XTAL	56
modem sleep, automatic sleep	Internal 150 KHz	54

However the internal one accuracy is lower:
W (480) BLE_INIT: Internal 150kHz RC osciallator. The accuracy of this clock is a lot larger than 500ppm which is required in Bluetooth communication, so don't select this option in scenarios such as BLE connection state.
so it might be better (safer) to use main XTAL by default, and for advanced power savings (2 mA only for listening mode) leave an option to explicitly enable BLE using internal oscillator.
Do we have a possibility to change some menuconfig options via Arduino IDE menus? or IDF libraris have to be built once with a predefined values?

[tshcherban]

@TD-er i think we can make UART work with DFS by selecting it's clock source from the default (APB) to RC_FAST/XTAL for S3 or UART_SCLK_REF_TICK for ESP32. At least it started printing large amounts without garbage for my S3 with DFS set to 10..160 MHz.

However on S3 I'm facing another problem: sometimes assert
configASSERT( ( xTickCount + xTicksToJump ) <= xNextTaskUnblockTime );
bites here
vTaskStepTick at /vscode-idf/data/esp-idf/src/components/freertos/FreeRTOS-Kernel/tasks.c:3061 (discriminator 1)
does it ring a bell for you? Now Im excluding code parts (from my test sketch) one by one to find a reason.

[TD-er]

Doesn't really ring a bell, like I haven't seen it before.
But basing on the assert, I would think your RTOS task takes longer than anticipated by the RTOS scheduler.
So it seems like a perfect example of some timings not being updated.
Probably an administrative mismatch between RTOS and ESP-IDF.

My first instinct would be to check if millis() or micros() are still in sync with reality.
Perhaps nothing is referencing the PLL clock anymore and thus at some layer it was decided to turn it off and thus derive clocks from another clock source.

Is the baud rate still correct?
Maybe you can try running at different baud rates to see if there is some threshold where things are being initialized differently?
Or change the TX buffer size of the serial port to see if your code may be waiting to store data in the buffer?
I think you can also set the threshold of when to generate events for when to send an event based on buffer full (RX) or empty (TX), though I'm not sure the latter one exists.

[tshcherban]

millis() / micros() are perfectly fine.
Seems its a known problem.

[TD-er]

Hmm interesting.
Seems like the S3 does perhaps do things slightly different at a hardware level compared to the others.
If I'm not mistaken, there was also something with Serial2 on the S3?

Maybe handling all those extra GPIO's takes some extra steps or perhaps things have been split into different (power) domains which need to be kept in sync?
Or it could be something as basic as extremely timing critical stuff which could even be caused by using different flash brands. That would be really nasty to reproduce and debug.

[tshcherban]

something with Serial2 on the S3

Not sure, I'm rather at level 'compile Hello World Blink -> 59 errors, 107 warnings' ))
And Serial2 isnt involved anyway.

[TD-er]

OK, if you're not using Serial2, then it is probably unrelated, but just for completeness, this is the issue I was referring to:
#9020 (comment)

[tshcherban]

Seems it has something to do with BLE, because once I've disabled listener - that assert stopped annoying.

[tshcherban]

Is the baud rate still correct?

here was a mention that with a REF_TICK clock baudrate 921600 wouldn't be stable. So for the old ESP32 I would add log warning for unsupported combinations and note somewhere in documentation/examples that automatic lightsleep and high speeds are not compatible.
For S3 and UART_SCLK_RTC / UART_SCLK_XTAL it works ok, checked 921600 and 460800, slower would obviously be OK.

@igrr @TD-er I'm going to prepare 2 separate PRs.
one for:

• power management enable (sdkconfig.defaults)
• arduino API for automatic lightsleep enable/disable
• WiFi modem sleep example
• bluetooth modem sleep enable (sdkconfig.defaults for S3/C3)
• arduino API wrap for esp_bt_sleep_enable

another for:

• DFS APIs
• Serial clock source API extend (to specify Serial clock source)

First one is relatively safe to enable, peripherals are somewhat aware of a lightsleep and can put locks if necessary to prevent sleeping in the middle of important stuff like transactions. Timing as well as WiFi, BLE, SPI, I2C seems to work OK with automatic lightsleep.
And the second one with DFS is a subject to investigate a bit more (IDFGH-8319 for S3). At a first glimpse it's feasible, and even most of the stuff works if not set too low frequency, but it does not give as much power savings as a modem+CPU sleep.

Not sure if update Serial library to automatically detect if DFS is enabled and update its internals accordingly?
Or just provide another begin parameter or separate function to set appropriate clock source?

[tshcherban]

Seems we've been unblocked with a IDFGH-11870 completed. Will check those changes out and continue working on a PR for a PowerManagement support.

[me-no-dev]

needs to be first backported to 5.1 and 5.3