Goodwatch – eInk Alarm Clock with Voice Recognition

Goodwatch – The eInk Alarm Clock that listens to my voice – cloudfree and running on battery for months!

I built already the MCW – an eInk alarm clock running on an Arduino Pro Mini – and got fascinated by the great look and power-saving of eInk papers. One issue I could not solve really efficient was alarm programming. Always many buttons… so I want to have my clock understanding me – my words in German: Numbers zero to nine and the words “yes=ja” and “no=nein”.

One easy option would be to build in a microphone and let the watch connect to Googles SpeechToText API – getting the text back – and set-up the alarm. But really, who wants to have a direct connection from his sleeping room into the cloud. I could also just use Alexa – not my style.

So, I started 3 projects:

 

https://youtu.be/W6pIDDZe-aQ
Watch it – 30 sec video showing speech recognition working on an ESP32

Hardware…Hardware…Hardware……

The key to success – is picking the right hardware – that was a good lessons learned from my last project. Here is my pick:

eInk Display

  • Good Display – E-ink epd display 4.2 inch e-ink raspberry GDEW042T2
    The eInk display supports partial refresh and communication via I2C bus – important not to use to many IO pins from the ESP32.
  • DESPI-C02 – a little board to connect the e-Paper with the ESP32 (no connection for power to the FrontLight Panel) and DESPI-C03 (with connection). Caution: The DESPI-C03 allows also to plugin the both FFC cables for the display and the front-light. If you think you can power-both at the same time, you kill your display

eInk FrontLight

If you want to read the time and night – you need light. With eInk that is a bit complicated, you need a special front-light. Its basically a piece of glass with LEDs at the edge. I was thinking to build it by myself. But I think buying it, was a much better idea. Still getting the result I wanted was a bit more complicated:

  • Good Display – FrontLight Pannel <- these are the trick and I think it just looks cool at night The FrontLight must be an exact fit to the display. I had quite some emails with Good Display to select this combination and I was very surprised by their great customer service. At the end it came direct from China 🙂
  • FFC Adapter RM 2.54 to connect the FrontLight Pannel with the power support (see above)
  • HW-045 DC-DC boost voltage regulator: The FrontLigh Pannel has 7 LEDs of 200 Ohm in series , if they need 15mA to shine – you need 21V. Uups – one more device needed, a power booster.
  • IRF3708 Transistor N-MOSFET 30V 62A 87W TO220AB – dimming the front-light, of course it needs to adjust to outside light condition. A first idea to use PWM as input for the voltage regulator was a bit naive, so the ESP32 is using the MOSFET to PWM the output of the voltage regulator before powering the display. To keep it running on battery for a long time, each part of hardware needs to be in low power consumption mode when the ESP32 is on sleep. For the amplifier I found the following solution, watch the tiny blue cable – that pulls the amplifier into deep sleep mode also:
  • MAX98357A
Power Consumption

Additional Hardware and Sensor

Here comes the rest, I had to do several rounds until I found the best pick – but I am really happy with below selection:

  • ESP32 as https://www.wemos.cc/en/latest/d32/d32_pro.html. (320kb RAM, 4MP PSRAM, 16MB Flash).  Two very important things, my one had 16MB Flash (program code, SPIFF), what is great to have when you want to work with Machine Learning and do some more fancy stuff, build in LIPO charger and 4MB PSRAM (that is also needed by the sound library and for ML)
  • Adafruit DS3231 – Precision Clock: An alarm clock needs time and alarms, both can be done the DS3231- supported by a little battery, so time and alarms can survive a short power outage. I finally picked the more expensive version from Adafruit, as two much cheaper versions from the internet just kept running to fast – approx 10s per day – no option 🙂 The DS3231 triggers and interrupt every 5min to show the time and of course when there is an alarm.
  • CS43434 I2S Digital Microphone – there are several different microphones on the market, I tried them all. At least for that one, I can say its compatible with the ESP32 I2S bus and really records in good quality. My one I got over Tindie.
  • PIR HC-SR602 Infrarot sensor: You can see in the video that I disable the alarm with a movement of my hand over the clock, that is a PIR sensor. Amazing – super, super low power consumption and so sensitive. The PIR sensor triggers and interrupt on the ESP32 for wake-up.
  • Adafruit I2S 3W Class D Amplifier Breakout – MAX98357A: It needs to play a nice sound when waking up, and the ESP32 has enough computing power to even decode MP3 files or stream from a radio station. Also this section is on hardware – https://github.com/schreibfaul1/ESP32-audioI2S to get sound into the alarm clock with 5 lines of code.
  • APDS-9960 – Digital Proximity, Ambient Light, RGB and Gesture Sensor: I wanted to use gestures to wake-up the alarm clock and decide between different operating modes. The APDS-9960 is really smart, it detects different hand-moves and measures distance. In addition it can check the light strength and color. I spent a lot of time, but its still consuming a lot of power also in low-energy mode. The only way to use this efficient: Let the PIR sensor (see above) run always – and then start the measurement. Unfortunately this causes a delay in processing – here we come to my next project: Wake up with my voice.

 

Software

Initially I have been using Arduino Platform to build this project. This platform includes a pre-compiled set of the ESP IDF. The ESP-IDF is Espressif’s official IoT Development Framework for the ESP32. As a consequence any settings used to compile the ESP IDF are fixed and can not be changed.

Boot-Speed is King

When waking up the Goodwatch, I want to see quick reaction – but noticed a 1-2 second break before the ESP32 got active.  The ESP is using this time to validate the Flash and RAM. As the Wemos D1 Pro has 4MP PSRAM and 16MB Flash – this takes a while.

With a change in the compiler-options, this check could be disabled.

Arduino as a Component

The trick is to install Arduino as a component to the main framework used for compilation is the ESP IDF and it includes the Arduino libraries as a subfolder.  This provides access to an utility “menueconfig”, that generates a config file, e.g. sdkconfig.lolin_d32_pro.

The problem is, that Arduino and ESP-IDF use different build-tool. After a lot of research I managed to get this working.  Speed is KING again, I would say less then 500ms to react on eInk is good.

The source code can be found here. I have put some comments in the platformio.ini on the build.

It’s all on Github: https://github.com/happychriss/GoodWatch_espidf

Thinking in Circles…

I had some fun on painting the circle-arcs – that brought me back some memories from school with my friends “sinus” and “cosinus”. I am sure there is much more elegant solution – but at one point of time, I was just happy to see the circles. In total I think I found a quite efficient way. Look as this “naughty” peace of code to paint an arc – and the painting:

void DrawArcCircle_Q3(GxEPD2_GFX &my_display, st_pwin *pwin, int x0, int y0, double start_angle, double end_angle, int ri, int ro) {

    start_angle = (270 - start_angle) * (M_PI / 180);
    end_angle = (270 - end_angle) * (M_PI / 180);
    double tan_start_angel = tan(start_angle);
    double tan_end_angel = tan(end_angle);

    int ri2 = ri * ri;
    int ro2 = ro * ro;
    for (int cy = 0; cy <= ro; cy++) {
        int cx_i;
        if (abs(ri) < abs(cy)) {
            cx_i = -1;
        } else {
            cx_i = (int) (sqrt(ri2 - cy * cy) + 0.5);
        }
        int cx_o = (int) (sqrt(ro2 - cy * cy) + 0.5);
        int cyy = cy + y0;

        int dx_1 = cy / tan_end_angel;
        int dx_2 = cy / tan_start_angel;

        if (dx_1 < cx_o) { cx_o = dx_1; };
        if (dx_2 > cx_i) { cx_i = dx_2; };

        if (cx_i <= cx_o) {
            gfx_line(my_display, pwin, x0 - cx_i, cyy, x0 - cx_o, cyy);
        }

    }
}

Time takes Time – and calculating with days and months – even more

Another simple topic – that got me some time to get it right: Calculating with time. My alarm watch has 5 different wake-up alarms, the DS3231 can manage 2 alarms and create an interrupt signal to wake-up the ESP32. One alarm is gone for a wake-up every 5 minutes to paint a new arc. The other one needs to be updated on time – so, you need to sort the alarms to find out the next possible. Now you have alarms on weekdays and on weekend, repeating alarms and one-time alarms – you get the message…..I am not proud of my code here – but sorting was easy at the end.

struct strct_alarm {
    uint8_t i =0;
    bool active = {false};
    bool valid = {false};
    DateTime time;
};

int SortTimes(void const *a, void const *b) {
    const struct strct_alarm *ad = (struct strct_alarm *) a;
    const struct strct_alarm *bd = (struct strct_alarm *) b;
    return (int) (ad->time.secondstime() - bd->time.secondstime());
}

// Calling the qsort
qsort(rtcData.d.alarms, ALARM_NUMBERS_DISPLAY, sizeof(strct_alarm), SortTimes);

Flash – Keep the Memory

The alarm times need to be stored on the ESP32 and should survive a power-outage (similar to the time of the clock). The ESP32 has FLASH memory available, perfect – all what I needed. After fiddling around with CRC values to check if the data in the flash is valid and addresses..I learned about an ESP32 feature called “NVS” on the ESP Documentation. – where it got really easy. It even checks, if the flash is consistent.

   esp_err_t err = nvs_flash_init();
    if (err == ESP_ERR_NVS_NO_FREE_PAGES || err == ESP_ERR_NVS_NEW_VERSION_FOUND) {
        // NVS partition was truncated and needs to be erased Retry nvs_flash_init
        ESP_ERROR_CHECK(nvs_flash_erase());
        err = nvs_flash_init();
    }
    ESP_ERROR_CHECK( err );

// Write:
       err = nvs_open(STORAGE_NAMESPACE, NVS_READWRITE, &my_handle);
        size_t required_size = sizeof(d);
        if (err != ESP_OK) DPL("NVS_OPEN ERROR");
        err = nvs_set_blob(my_handle, STORAGE_NAMESPACE, (uint8_t *) &d, required_size);
        if (err != ESP_OK) DPL("NVS_WRITE ERROR");
        // Commit
        err = nvs_commit(my_handle);
        if (err != ESP_OK) DPL("NVS Commit Error");

// Read: 
       err = nvs_get_blob(my_handle, STORAGE_NAMESPACE, (uint8_t *) this, &required_size);
        if (err != ESP_OK && err != ESP_ERR_NVS_NOT_FOUND) {
            DPL("NVS READ ERROR");
        }

Programming the eInk Display

This would have been the most complicated, but thanks to: https://github.com/ZinggJM/GxEPD2 it was really easy. It was a bit of work to find this library and to get the concept. There is also a support-forum and the author is really helpfull. THANK YOU.

Its worth to note that the display is connected to the ESP32 via a simple adapter DESPI-C02 or DESPI-C023 using an FCC cable, from here its connected to the ESP. Most reasons for error is the wiring and correct association of the PINS – and I mixed up MISO and MOSI — took my so long 🙂

// From my hardware
// EPD Pins - GxEPD2_display_selection_new_style.h using SPI
// EPD Pins****
// CS=5, DC=0, RST=2,BUSY=15;
// SPI: SCK=18, SDI=MISO=23
// BUSY -> 15, RST -> 2, DC -> 0, CS -> 5, CLK -> SCK(18), DIN -> MOSI(23), GND -> GND, 3.3V -> 3.3V

#define EPD_CS 5 // Chip Select -
#define EPD_DC 0
#define EPD_RST 2
#define EPD_BUSY 15

GxEPD2_DISPLAY_CLASS<GxEPD2_DRIVER_CLASS, MAX_HEIGHT(GxEPD2_DRIVER_CLASS)> 
display(GxEPD2_DRIVER_CLASS(/*CS=5*/ EPD_CS, /*DC=*/ EPD_DC, /*RST=*/ EPD_RST, /*BUSY=*/ EPD_BUSY));

Latest News
  • Best of two worlds: Menuconfig options for ESP32 Arduino and Platformio – Speed up the boot time (28.08.2022)

    Speed Up Boot-Time with Menuconfig (ESP32)

    When you read this, you may have been already going a long way … and I am still walking.. so this is just my latest understanding. Feel free to correct and help me to further detail this out – and be ready that not all below is really correct… but at least … I got it working this way.. 🙂

    Why does it take a second to boot?

    Boot-Time becomes critical for any embedded device, when you need wake-up from deep sleep and immediate action.  I needed a short boot time  for the Goodwatch project, to make sure the clock is awake super fast when I want her to show the time in seconds on a “hands move” (watch the video here.). On the  Wemos D32 Pro it took approx 1 second to  boot – that is to long for many tasks. During this 1 second the processor e.g. runs some selfcheck for ram (PSRAM) and some other stuff. So – I was thinking, how can I skip this? If you ever looked at your PC BIOS you could see some options there, something similar exists for the ESP32: Menuconfig is a configuration utility provided by Espressif to fine-tune settings for the ESP32 processor family – looking a bit like BIOS settings: But with menuconfig you can do much more, so – what I did for Goodwatch
    • Speedup boot as described in : https://esp32.com/viewtopic.php?t=9448
    • Support for external RAM: Y
    • Run memory Test on SPI Init: N
    • Faster SPI: Serial Flasher Config” QIO
    • Double Flash Read Speed: Set CONFIG_ESPTOOLPY_FLASHFREQ to 80 MH from Serial
    As a result, the boot-time reduced dramatically – seconds now are displayed nearly instant 🙂
    But here comes my problem – I did not manage to get menuconfig installed and working together with my Arduino build system and libraries….
    I am using an e-Ink display and quite a few sensors – so it is great to have access to all the Arduino libraries  – just thinking about  ESP-Audio from Schreibfaul that converts the ESP into a radio or voice speaker are  worth using  Arduino, also the great e-Ink library from ZinggJM.

    Understanding the Build System and “Frameworks”

    Setting up that build system is complicated task. To build the Arduino projects I am using Platformio.io – a build and development platform for embedded devices, that supports Arduino but also many other frameworks. It helps to understand this terms:
    • Framework: Arduino or espidf: what type of libraries are used to communicate with the hardware. That is the most foundational decision.
    • Platform: That is the core from Platformio – a set of scripts, files that do all the hard work and set-up the build system for you. The platform determines which version of the
    • Components: Code blocks that can be added to the framework to extend the functionality (libraries). Arduino can be used as a component.  Arduino specific Components (libraries wrapping ESP calls into Arduino) are provided by ESP-IDF here. and can be addded here
    The default Platformio configuration is stored the platformio.ini for a simple “Arduiono” project is set-up like this: 
    platform = espressif32
board = lolin_d32_pro
framework = arduino
    The arduinoespressif32 Framework does all the work – it allows the device independent Arduino libraries to access the device specific ESP-IDF and ESP32 hardware functions. When running the build with Platformio you see the following: 
    PLATFORM: Espressif 32 (5.1.1+sha.4901957) > WEMOS LOLIN D32 PRO
HARDWARE: ESP32 240MHz, 320KB RAM, 16MB Flash
DEBUG: Current (cmsis-dap) External (cmsis-dap, esp-bridge, esp-prog, iot-bus-jtag, jlink, minimodule, olimex-arm-usb-ocd, olimex-arm-usb-ocd-h, olimex-arm-usb-tiny-h, olimex-jtag-tiny, tumpa)
PACKAGES: -framework-arduinoespressif32 @ 3.20004.220825 (2.0.4)
    It using a version 2.04  of arduinoespressif32 , for ease of all the developers this is a “precompiled/bundled” version that does not need the full ESP32 build-chain to be installed. That explains, why menuconfig doesn’t work, it is changing the settings in the framework.  This is great to kick-off development (no issues with matching versions) – but there is  no way to change any settings 🙁

    Escape the cave – using ESP Framework with Components

    The only way to get menuconfig working and to have full access to all the ESP32 feature is moving to the Espressif IoT Development Framework. This comes direct from the vendor and allows configuration of the build-chain with menuconfig. To keep the best of both worlds, we must still use the Arduino libraries by adding them as Components. How this works is described here. In nutshell the project is not set-up using the Arduino build system and approach, but as the full Espressif chain. A folder “components” contains the libraries that map Arduino into ESP32 “language”. All Arduino libraries will keep working, calling the ESP32 functions via the components wrapper  – that call the core esp-functions.

    The challenge – the matching game & magic of Platformio

    But now –  at the end, all must match: Arduino version -> Component -> Espressif Framework… here the nightmare starts…
    Espressif already gives an important information and provides the component at this page. The components can be installed from here: https://github.com/espressif/arduino-esp32.git It also says the work together with ESP-IDF v4.4. Now we need to find the matching framework from espidf that supports ESP-IDF v4.4
    First Option – Just try the Standard
    So – lets update platformio.ini with the espidf framework. Should work, we have the components ready framework = espidf platform = espressif32 board = lolin_d32_pro Result: 
    PLATFORM: Espressif 32 (5.1.1+sha.4901957) > WEMOS LOLIN D32 PRO
HARDWARE: ESP32 240MHz, 320KB RAM, 16MB Flash
DEBUG: Current (cmsis-dap) External .....
PACKAGES:  - framework-espidf @ 3.40401.0 (4.4.1)
    Ups…this didn’t work – its building for the ESP-IDF 4.4.1, and that is not aligned with our components. At the end of the build we get an error message: “undefined reference to `uart_get_tx_buffer_free_size'”
    Second Option – Pick the right platform for IDF v4.4
    framework = espidf
platform = https://github.com/platformio/platform-espressif32/archive/refs/tags/v4.4.0.zip
board = lolin_d32_pro
    This looks better, we are now at ESP-IDF 4.4.0 
    PLATFORM: Espressif 32 (4.4.0) > WEMOS LOLIN D32 PRO
HARDWARE: ESP32 240MHz, 320KB RAM, 16MB Flash
PACKAGES:  - framework-espidf @ 3.40302.0 (4.3.2)
    Arrgh..but another error-message: Arduino-esp32 can be used with ESP-IDF versions between 4.4.0 and 4.4.99, but an older version is detected: 4.3.2 That’s strange, also the platform version indicates using 4.4  –  it seems that the framework selected  remains still on 4.3.2 level  – that is not matching to our components
    Third Option – Using the Tasmota Platform
    Cant remember anymore where I found that tip -but it saved my live. Tasmota is so kind of providing another platform file for Platformio that works. Many thanks to Jason2866 (who maintains this at Github) for all his work – he really makes this possible!!!!! 
    framework = espidf
platform =https://github.com/tasmota/platform-espressif32
board = lolin_d32_pro
    The result looks better: 
    PLATFORM: Espressif 32 (2.0.4) > WEMOS LOLIN D32 PRO
HARDWARE: ESP32 240MHz, 320KB RAM, 16MB Flash
PACKAGES:  - framework-espidf @ 3.40403.0 <strong>(4.4.3) </strong>
    The “2.0.4” is the Tasmota versioning of the Platform-File (not the ESP-IDF version).But we see happily a framework with 4.4.3 selected. Hurray!!!!!!
    Honestly I do not understand the full magic in the Tasmota Platform configuration…. lets see what we have
    Deep Dive.. what is different from Tasmota to Platformio platform file
    Each platform file has a json, that defines the elements to be used for the build chain. Lets look at this file: 
    "version": "2.0.4+1",
  "frameworks": {
    "arduino": {
      "package": "framework-arduinoespressif32",
      "script": "builder/frameworks/arduino.py"
    },
    "espidf": {
      "package": "framework-espidf",
      "script": "builder/frameworks/espidf.py",
      "description": "ESP-IDF is the official development framework for the ESP32 and ESP32-S Series SoCs.",
      "homepage": "https://docs.espressif.com/projects/esp-idf/en/latest/esp32/",
      "title": "Espressif IoT Development Framework"
    }
  },
  "packages": {
    "framework-arduinoespressif32": {
      "type": "framework",
      "optional": true,
      "owner": "tasmota",
      "version": "https://github.com/tasmota/arduino-esp32/releases/download/2.0.4.1/framework-arduinoespressif32.tar.gz"
    },
    "framework-espidf": {
      "type": "framework",
      "optional": true,
      "owner": "tasmota",
      "version": "https://github.com/tasmota/esp-idf/releases/download/v4.4.3/esp-idf-v4.4.3.zip"
    },
    What can we see here?
    • The “version” 2.0.4 seems to be Tasmota specfic and does not indicate the ESP-IDF version
    • The Tasmote package supports two framewors: “Arduino” and “espidf” – we look for espidf
    • The  framework-espidf framework points indead to a specific framework from tasmota with version 4.4.3  -> and that is the trick: Tasmota build its own framewor.
    • Looking at the framework form Tasmota here: https://github.com/tasmota/esp-idf we see its a fork from espressif/esp-idf version 4.4.3.
    • Comparing both forks I see they look the same. So I guess the differences between the Tasmota package and the Platformio Package are minimal, eventually just the version settings in the json.file
    Thats enough for me – now I have an easy config working 🙂

    How to do it – Tips & Tricks

    This is not a tutorial how to set-up the full build environment. So, to get there:
    1. Setup a normal development project with Platformio and following platformio.ini: framework = espidf platform =https://github.com/tasmota/platform-espressif32 board = lolin_d32_pro
    2. Add the matching components from espidf to the folder structure as desribed here:https://espressif-docs.readthedocs-hosted.com/projects/arduino-esp32/en/latest/esp-idf_component.html
    3. I am running Platformio in CLION and get not working cmake file genertated after pio init. I use the following “CMakeLists.txt”: 
      cmake_minimum_required(VERSION 3.16.0)
include($ENV{IDF_PATH}/tools/cmake/project.cmake)
list(APPEND EXTRA_COMPONENT_DIRS arduino)
project(GoodWatch_espif)
    4. Run platformio build
    Long story … short… but works 🙂