Best of two worlds: Menuconfig options for ESP32 Arduino and Platformio – Speed up the boot time

Published by cneuhaus on

Speed Up Boot-Time with Menuconfig (ESP32)

When you read this, you may have been already going a long way … and I am still walking.. so this is just my latest understanding. Feel free to correct and help me to further detail this out – and be ready that not all below is really correct… but at least … I got it working this way.. 🙂

Why does it take a second to boot?

Boot-Time becomes critical for any embedded device, when you need wake-up from deep sleep and immediate action.  I needed a short boot time  for the Goodwatch project, to make sure the clock is awake super fast when I want her to show the time in seconds on a “hands move” (watch the video here.).

On the  Wemos D32 Pro it took approx 1 second to  boot – that is to long for many tasks.

During this 1 second the processor e.g. runs some selfcheck for ram (PSRAM) and some other stuff.

So – I was thinking, how can I skip this?

If you ever looked at your PC BIOS you could see some options there, something similar exists for the ESP32: Menuconfig is a configuration utility provided by Espressif to fine-tune settings for the ESP32 processor family – looking a bit like BIOS settings:

But with menuconfig you can do much more, so – what I did for Goodwatch

  • Speedup boot as described in : https://esp32.com/viewtopic.php?t=9448
  • Support for external RAM: Y
  • Run memory Test on SPI Init: N
  • Faster SPI: Serial Flasher Config” QIO
  • Double Flash Read Speed: Set CONFIG_ESPTOOLPY_FLASHFREQ to 80 MH from Serial

As a result, the boot-time reduced dramatically – seconds now are displayed nearly instant 🙂

But here comes my problem – I did not manage to get menuconfig installed and working together with my Arduino build system and libraries….

I am using an e-Ink display and quite a few sensors – so it is great to have access to all the Arduino libraries  – just thinking about  ESP-Audio from Schreibfaul that converts the ESP into a radio or voice speaker are  worth using  Arduino, also the great e-Ink library from ZinggJM.

Understanding the Build System and “Frameworks”

Setting up that build system is complicated task. To build the Arduino projects I am using Platformio.io – a build and development platform for embedded devices, that supports Arduino but also many other frameworks.

It helps to understand this terms:

  • Framework: Arduino or espidf: what type of libraries are used to communicate with the hardware. That is the most foundational decision.
  • Platform: That is the core from Platformio – a set of scripts, files that do all the hard work and set-up the build system for you. The platform determines which version of the
  • Components: Code blocks that can be added to the framework to extend the functionality (libraries). Arduino can be used as a component.  Arduino specific Components (libraries wrapping ESP calls into Arduino) are provided by ESP-IDF here. and can be addded here

The default Platformio configuration is stored the platformio.ini for a simple “Arduiono” project is set-up like this:

platform = espressif32
board = lolin_d32_pro
framework = arduino

The arduinoespressif32 Framework does all the work – it allows the device independent Arduino libraries to access the device specific ESP-IDF and ESP32 hardware functions.

When running the build with Platformio you see the following:

PLATFORM: Espressif 32 (5.1.1+sha.4901957) > WEMOS LOLIN D32 PRO
HARDWARE: ESP32 240MHz, 320KB RAM, 16MB Flash
DEBUG: Current (cmsis-dap) External (cmsis-dap, esp-bridge, esp-prog, iot-bus-jtag, jlink, minimodule, olimex-arm-usb-ocd, olimex-arm-usb-ocd-h, olimex-arm-usb-tiny-h, olimex-jtag-tiny, tumpa)
PACKAGES: -framework-arduinoespressif32 @ 3.20004.220825 (2.0.4)

It using a version 2.04  of arduinoespressif32 , for ease of all the developers this is a “precompiled/bundled” version that does not need the full ESP32 build-chain to be installed.

That explains, why menuconfig doesn’t work, it is changing the settings in the framework.  This is great to kick-off development (no issues with matching versions) – but there is  no way to change any settings 🙁

Escape the cave – using ESP Framework with Components

The only way to get menuconfig working and to have full access to all the ESP32 feature is moving to the Espressif IoT Development Framework. This comes direct from the vendor and allows configuration of the build-chain with menuconfig.

To keep the best of both worlds, we must still use the Arduino libraries by adding them as Components. How this works is described here. In nutshell the project is not set-up using the Arduino build system and approach, but as the full Espressif chain.

A folder “components” contains the libraries that map Arduino into ESP32 “language”. All Arduino libraries will keep working, calling the ESP32 functions via the components wrapper  – that call the core esp-functions.

The challenge – the matching game & magic of Platformio

But now –  at the end, all must match: Arduino version -> Component -> Espressif Framework… here the nightmare starts…

Espressif already gives an important information and provides the component at this page.

The components can be installed from here: https://github.com/espressif/arduino-esp32.git

It also says the work together with ESP-IDF v4.4.

Now we need to find the matching framework from espidf that supports ESP-IDF v4.4

First Option – Just try the Standard

So – lets update platformio.ini with the espidf framework. Should work, we have the components ready

framework = espidf
platform = espressif32
board = lolin_d32_pro

Result:

PLATFORM: Espressif 32 (5.1.1+sha.4901957) > WEMOS LOLIN D32 PRO
HARDWARE: ESP32 240MHz, 320KB RAM, 16MB Flash
DEBUG: Current (cmsis-dap) External .....
PACKAGES:  - framework-espidf @ 3.40401.0 (4.4.1)

Ups…this didn’t work – its building for the ESP-IDF 4.4.1, and that is not aligned with our components. At the end of the build we get an error message: “undefined reference to `uart_get_tx_buffer_free_size'”

Second Option – Pick the right platform for IDF v4.4
framework = espidf
platform = https://github.com/platformio/platform-espressif32/archive/refs/tags/v4.4.0.zip
board = lolin_d32_pro

This looks better, we are now at ESP-IDF 4.4.0

PLATFORM: Espressif 32 (4.4.0) > WEMOS LOLIN D32 PRO
HARDWARE: ESP32 240MHz, 320KB RAM, 16MB Flash
PACKAGES:  - framework-espidf @ 3.40302.0 (4.3.2)

Arrgh..but another error-message:

Arduino-esp32 can be used with ESP-IDF versions between 4.4.0 and 4.4.99,
but an older version is detected: 4.3.2

That’s strange, also the platform version indicates using 4.4  –  it seems that the framework selected  remains still on 4.3.2 level  – that is not matching to our components

Third Option – Using the Tasmota Platform

Cant remember anymore where I found that tip -but it saved my live. Tasmota is so kind of providing another platform file for Platformio that works. Many thanks to Jason2866 (who maintains this at Github) for all his work – he really makes this possible!!!!!

framework = espidf
platform =https://github.com/tasmota/platform-espressif32
board = lolin_d32_pro

The result looks better:

PLATFORM: Espressif 32 (2.0.4) > WEMOS LOLIN D32 PRO
HARDWARE: ESP32 240MHz, 320KB RAM, 16MB Flash
PACKAGES:  - framework-espidf @ 3.40403.0 <strong>(4.4.3) </strong>

The “2.0.4” is the Tasmota versioning of the Platform-File (not the ESP-IDF version).But we see happily a framework with 4.4.3 selected. Hurray!!!!!!

Honestly I do not understand the full magic in the Tasmota Platform configuration…. lets see what we have

Deep Dive.. what is different from Tasmota to Platformio platform file

Each platform file has a json, that defines the elements to be used for the build chain.

Lets look at this file:

"version": "2.0.4+1",
  "frameworks": {
    "arduino": {
      "package": "framework-arduinoespressif32",
      "script": "builder/frameworks/arduino.py"
    },
    "espidf": {
      "package": "framework-espidf",
      "script": "builder/frameworks/espidf.py",
      "description": "ESP-IDF is the official development framework for the ESP32 and ESP32-S Series SoCs.",
      "homepage": "https://docs.espressif.com/projects/esp-idf/en/latest/esp32/",
      "title": "Espressif IoT Development Framework"
    }
  },
  "packages": {
    "framework-arduinoespressif32": {
      "type": "framework",
      "optional": true,
      "owner": "tasmota",
      "version": "https://github.com/tasmota/arduino-esp32/releases/download/2.0.4.1/framework-arduinoespressif32.tar.gz"
    },
    "framework-espidf": {
      "type": "framework",
      "optional": true,
      "owner": "tasmota",
      "version": "https://github.com/tasmota/esp-idf/releases/download/v4.4.3/esp-idf-v4.4.3.zip"
    },

What can we see here?

  • The “version” 2.0.4 seems to be Tasmota specfic and does not indicate the ESP-IDF version
  • The Tasmote package supports two framewors: “Arduino” and “espidf” – we look for espidf
  • The  framework-espidf framework points indead to a specific framework from tasmota with version 4.4.3  -> and that is the trick: Tasmota build its own framewor.
  • Looking at the framework form Tasmota here: https://github.com/tasmota/esp-idf we see its a fork from espressif/esp-idf version 4.4.3.
  • Comparing both forks I see they look the same. So I guess the differences between the Tasmota package and the Platformio Package are minimal, eventually just the version settings in the json.file

Thats enough for me – now I have an easy config working 🙂

How to do it – Tips & Tricks

This is not a tutorial how to set-up the full build environment. So, to get there:

  1. Setup a normal development project with Platformio and following platformio.ini:
    framework = espidf
    platform =https://github.com/tasmota/platform-espressif32
    board = lolin_d32_pro
  2. Add the matching components from espidf to the folder structure as desribed here:https://espressif-docs.readthedocs-hosted.com/projects/arduino-esp32/en/latest/esp-idf_component.html
  3. I am running Platformio in CLION and get not working cmake file genertated after pio init. I use the following “CMakeLists.txt”: 
    cmake_minimum_required(VERSION 3.16.0)
include($ENV{IDF_PATH}/tools/cmake/project.cmake)
list(APPEND EXTRA_COMPONENT_DIRS arduino)
project(GoodWatch_espif)
  4. Run platformio build

Long story … short… but works 🙂

 

Categories: goodwatch

0 Comments

Leave a Reply

Avatar placeholder