SenseBoxOTA: use SSD1306 display if available

arduino/samd/libraries/SenseBoxOTA/.gitignore

@@ -0,0 +1 @@
+extra/ota_boot/build

arduino/samd/libraries/SenseBoxOTA/README.md

@@ -1,5 +1,7 @@
 # senseBox OTA
 This library provides over the air programming for the senseBox MCU.
+
+## usage
 To enable this operating mode, just include the following line in your sketch:
 
 ```c
@@ -14,7 +16,7 @@ Your MCU will now have a secondary OTA bootloader, which enables a secondary ope
     In this mode a WiFi hotspot with a webserver is started, where sketch binaries can be sent to.
 - This mode can be entered manually by holding down the grey button on the MCU ("switch") while starting / resetting.
 
-## uploading a sketch
+### uploading a sketch
 To upload a sketch, make sure...
 - the MCU is in OTA mode & you are connected to its WiFi AP,
 - your sketch contains the line `#include <SenseBoxOTA.h>`.
@@ -25,24 +27,59 @@ Export your compiled sketch (`ctrl+alt+s`). Now you can upload from your sketch
 curl 192.168.1.1/sketch --data-binary @<your-sketchname>ino.sensebox_mcu.bin
 ```
 
+Other known-working clients:
+- [senseBox Blockly app](https://github.com/sensebox/blockly-app)
+
+## how it works
+
+This library provides a second stage ('userspace') bootloader, that runs after the first stage bootloader (which provides the USB mass-storage update facility), but runs before the user-provided code.
+This works by inserting the OTA functionality at the start of the userspace.
+This position in flash storage is defined for the symbol name `.sketch_boot` in the linker script (`variants/sensebox_mcu/linker_scripts/gcc/flash_with_bootloader.ld`).
+The symbol `.sketch_boot` is defined in `src/SenseboxOTA.cpp` and contains the compiled sketch of the bootloader (see section [development].
+
+Internally, this bootloader works quite similar as Arduino's [`SDU` library][sdu], except for swapping the SD reading functionality with a webserver:
+The OTA bootloader directly hands over to a user application if one is present, otherwise starts a WiFi accesspoint and webserver.
+On this WiFi accesspoint clients can send new sketches to the MCU via HTTP POST to http://192.168.1.1:80/sketch.
+
+```
++---------------------------+
+|   8KB  primary bootloader |
++---------------------------+
+|  64KB  OTA bootloader     | // this section only exists, when .sketch_boot is defined,
+|        .sketch_boot       | // i.e. when the user sketch contains `#include <SenseBoxOTA.h>` 
++---------------------------+
+| 184KB  userspace code     |
+|                           |
++---------------------------+
+```
+
 ## development
-This library has development dependencies on `WiFi101.h` and `FlashStorage.h` (during build time only).
+- directory layout:
+  - `examples/`: simple usage example
+  - `extra/ota_boot/`: the bootloader
+  - `src/` contains the runtime with the compiled bootloader, included by users via `#include <SenseBoxOTA.h>`
 
-To apply changes made to the `ota_boot.ino` sketch, the `./build.sh` script has to be run first.
-For development you can enable DEBUG logging via the `OTA_DEBUG` define in `conf.h`; note that in debug mode the bootloader will wait until the Serial Monitor is opened before starting operation.
+- To apply changes made to the `ota_boot.ino` sketch, the OTA bootloader needs to be built:
+    Run `./build_cli.sh` to update the bootloader that users include via `#include <SenseBoxOTA.h>`.
 
-Internally, it works quite similar as Arduino's `SDU.h` library, except for swapping the SD reading functionality with a webserver.
-To understand what is happening the following hints may help:
+- This library has build-time dependencies on `sh`, `xxd`, and some arduino libraries that are not vendored. Check the file [`src/boot/buildinfo.txt`](src/boot/buildinfo.txt) for information on the library versions used during the last build.
 
-- The linker script reserves the first section of flash storage for the OTA functionality via the symbol name `.sketch_boot`.
-  If this symbol is missing, the memory is as usual (8KB bootloader, then user code).
-- The OTA functionality is defined in the code in `extras/ota_boot`, and put into the folder `src/boot/` in compiled binary form.
-- The OTA bootloader directly hands over to a user application if one is present, otherwise starts a hotspot and webserver.
+- The recommended build tool is [arduino-cli][cli], but Arduino IDE may work too, use the respective `build_*.sh` script.
+
+- For development you can enable DEBUG logging via the `OTA_DEBUG` define in `OTA.h`;
+    note that in debug mode the bootloader will wait until the Serial Monitor is opened before starting operation!
 
 ## known issues
-- accepts only one wifi client
-- webserver does not respond after one wifi disconnect
-- no code checksumming
+- no checksumming or signature check on the received binary
+- OTA bootloader takes up almost 64KB of flash - most of it is the Wifi101 library.
+    If this can be replaced  with a slimmer library, more space for user sketches will remain.
+- OTA_DEBUG logging is silent after ~2 seconds without communication - cause unknown.
+    Output is re-enabled by sending a message from the host.
 
 ## license
 GPL-3.0, Norwin Roosen
+
+[sdu]: https://github.com/arduino/ArduinoCore-samd/tree/master/libraries/SDU
+[cli]: https://github.com/arduino/arduino-cli
+[libwifi]: https://github.com/arduino-libraries/WiFi101
+[libflash]: https://github.com/cmaglie/FlashStorage

arduino/samd/libraries/SenseBoxOTA/extra/ota_boot/OTA.cpp

@@ -1,13 +1,47 @@
 #include "OTA.h"
 
+#include <senseBoxIO.h>
+
 #include <SPI.h>
 #include <Arduino.h>
 #include <FlashStorage.h>
 
-OTA::OTA() : server(80), status(WL_IDLE_STATUS) {}
+// support for the display. uses around 10k bytes (with lots of room for optimization)
+#include <Wire.h>
+#include <Adafruit_GFX.h>
+#include <Adafruit_SSD1306.h>
+#define DISPLAY_MAXX 128
+#define DISPLAY_MAXY 64
+#define PRINT_DISPLAY(ROWS, CMDS)               \
+    if (displayEnabled) {                       \
+      display.setTextSize(1);                   \
+      display.setCursor(0,DISPLAY_MAXY-ROWS*8); \
+      CMDS;                                     \
+      display.display();                        \
+    }
+
+OTA::OTA() :
+  server(80),
+  status(WL_IDLE_STATUS),
+  display(DISPLAY_MAXX, DISPLAY_MAXY, &Wire, -1) {}
 
 void OTA::begin(bool accessPointMode)
 {
+  // NOTE: the display needs around 340ms after power-up to be available.
+  //   enabling XBee1 introduces a delay of 500ms, so we're good here.
+  senseBoxIO.powerI2C(true); // display
+  senseBoxIO.powerXB1(true); // wifi
+  displayEnabled = display.begin(SSD1306_SWITCHCAPVCC, 0x3D);
+  if (displayEnabled) {
+    display.setTextColor(WHITE,BLACK);
+    display.setTextSize(2);
+    display.println("OTA update");
+    display.display();
+  } else {
+    LOG.println("failed to init display, maybe no display connected.");
+    senseBoxIO.powerI2C(false);
+  }
+
   // switch modes: also allow no access point (for use with just webserver in an existing network)
   if (accessPointMode)
     createAccessPoint();
@@ -25,50 +59,59 @@ void OTA::createAccessPoint()
 {
   if (WiFi.status() == WL_NO_SHIELD)
   {
-    Serial.println("WiFi shield not present");
+    LOG.println("error: WiFi shield not present");
+    PRINT_DISPLAY(2, display.println("  error:  WiFi shield\n     not present"))
     while (true)
       ;
   }
-  // Assign mac address to byte array & push it to a char array
-  WiFi.macAddress(mac);
-  String mac_str = String(mac[1], HEX) + String(mac[0], HEX);
-  mac_str.toUpperCase();
-  String ssid_string = String("senseBox:" + mac_str);
+  // get wifi mac address and use last bytes to generate the wifi SSID
   char ssid[20];
-  ssid_string.toCharArray(ssid, 20);
+  byte mac[6];
+  WiFi.macAddress(mac);
+  sprintf(ssid, "senseBox:%.2X%.2X", mac[1], mac[0]);
 
-  LOG.print("Creating access point named: ");
+  LOG.print("creating access point named ");
   LOG.println(ssid);
-
   // initialize wifi: set SSID based on last 4 bytes of MAC address
   status = WiFi.beginAP(ssid);
   if (status != WL_AP_LISTENING)
   {
-    LOG.println("Creating access point failed");
+    LOG.println("error: creating wifi ap failed");
+    PRINT_DISPLAY(2, display.println("  error:  failed to  \n   create wifi ap   "))
     while (true)
       ;
   }
 
+  PRINT_DISPLAY(2,
+    display.print("connect to wifi named\n    ");
+    display.println(ssid);
+  )
+
   pinMode(LED_BUILTIN2, OUTPUT);
 }
 
 void OTA::pollWifiState()
 {
   // blink faster if a device is connected to the access point
-  if (status != WiFi.status())
+  uint8_t newStatus = WiFi.status();
+  if (status != newStatus)
   {
     status = WiFi.status();
 
     if (status == WL_AP_CONNECTED)
     {
       LOG.println("device connected to AP");
-      led_interval = 900;
+      PRINT_DISPLAY(2, display.print("      connected.     \n awaiting new sketch"))
+      led_interval = 700;
     }
-    else
+    else if (status == WL_AP_LISTENING)
     {
       // a device has disconnected from the AP, and we are back in listening mode
-      LOG.println("Device disconnected from AP");
+      LOG.println("device disconnected from AP");
+      PRINT_DISPLAY(2, display.print("    disconnected.    \n      reconnect?     "))
       led_interval = 2000;
+      // needed according to https://github.com/arduino-libraries/WiFi101/issues/110#issuecomment-256662397
+      server.begin();
     }
   }
 
@@ -88,7 +131,9 @@ void OTA::pollWifiState()
 
 void OTA::pollWebserver()
 {
-  // listen for incoming clients
+  // listen for one (!) incoming client. having more than one client
+  // connected is not intended, as they may interfere, posting
+  // different sketches.
   WiFiClient client = server.available();
   if (!client)
     return;
@@ -98,6 +143,7 @@ void OTA::pollWebserver()
   bool flashSuccess = false;
   bool currentLineIsBlank = true;
   String req_str = "";
+  req_str.reserve(256);
 
   while (client.connected())
   {
@@ -107,10 +153,6 @@ void OTA::pollWebserver()
     char c = client.read();
     req_str += c;
 
-    // if you've gotten to the end of the line (received a newline
-    // character) and the line is blank, the http request has ended,
-    // so you can send a reply
-
     // POST request also needs to handle self update
     if (c == '\n' && currentLineIsBlank && req_str.startsWith("POST"))
     {
@@ -121,6 +163,9 @@ void OTA::pollWebserver()
     }
 
     if (c == '\n' && currentLineIsBlank)
+      // if you've gotten to the end of the line (received a newline
+      // character) and the line is blank, the http request has ended,
+      // so you can send a reply
       break;
     if (c == '\n')
       currentLineIsBlank = true;
@@ -129,6 +174,7 @@ void OTA::pollWebserver()
   }
 
   sendResponse(client, req_str.startsWith("GET") || flashSuccess);
+  delay(50); // give client time to receive response & disconnect
   client.stop();
   LOG.println("client disconnected");
 
@@ -147,40 +193,54 @@ void OTA::pollWebserver()
  */
 bool OTA::handlePostSketch(WiFiClient &client, String &req_str)
 {
+  LOG.print("[OTA] handling POST /sketch request from ");
+  LOG.println(client.remoteIP());
   // extract length of body
   int contentLengthPos = req_str.indexOf("Content-Length:");
   if (contentLengthPos <= 0)
   {
     LOG.println("Content-Length is missing, ignoring request");
+    PRINT_DISPLAY(2, display.print("error invalid request\nmissing contentlength"))
     return false;
   }
   String tmp = req_str.substring(contentLengthPos + 15);
   tmp.trim();
   uint32_t contentLength = tmp.toInt();
+  LOG.print("Content-Length: ");
   LOG.println(contentLength);
 
-  // if (contentLength <= OTA_SIZE) {
-  //   LOG.println("update is too small, ignoring");
-  //   return false;
-  // }
-  if (contentLength > (FLASH_SIZE - 0x2000))
+  if (contentLength < OTA_SIZE) {
+    LOG.println("update is too small, ignoring");
+    PRINT_DISPLAY(2, display.print("error invalid request\nupdate is too small"))
+    return false;
+  }
+  if (contentLength > (FLASH_SIZE - OTA_START))
   {
     LOG.println("update is too large, ignoring");
+    PRINT_DISPLAY(2, display.print("error invalid request\nupdate is too large"))
     return false;
   }
 
+  PRINT_DISPLAY(1, display.print("receiving new sketch"))
+
   // skip the first part of the sketch which contains the OTA code we're currently running from.
   // the new sketch needs to still include this section in order for internal memory adresses to
-  // be compiled correctly (unconfirmed, but it didn't work withou)
+  // be compiled correctly.
   uint32_t updateSize = contentLength - OTA_SIZE;
+  LOG.print("skipping ");
+  LOG.print(contentLength - updateSize);
+  LOG.println(" bytes");
   while (updateSize < contentLength)
   {
-    if (!client.available())
+    if (!client.available()) {
+      LOG.println("waiting for client...");
       continue;
+    }
     contentLength--;
     char c = client.read();
-    req_str += c;
   }
+  LOG.print("skipped ");
+  LOG.println(updateSize + OTA_SIZE - contentLength);
 
   // write the body to flash, page by page
   FlashClass flash;
@@ -194,15 +254,22 @@ bool OTA::handlePostSketch(WiFiClient &client, String &req_str)
                                ? FLASH_PAGE_SIZE
                                : updateSize % FLASH_PAGE_SIZE;
 
+  LOG.print("expecting to write ");
+  LOG.print(numPages);
+  LOG.println(" pages.");
+
   for (uint32_t i = 0; i < numPages; i++)
   {
+    LOG.print("filling buffer for page ");
+    LOG.println(i+1);
     // fill the page buffer, reading one byte at a time.
     uint32_t bufferIndex = 0;
     uint32_t bytesToRead = i == numPages - 1 ? lastPageBytes : FLASH_PAGE_SIZE;
     while (bufferIndex < bytesToRead)
     {
       while (!client.available())
       {
+        LOG.println("waiting for data from client...");
         ;
       } // don't continue until we received new data
       flashbuffer[bufferIndex] = client.read();
@@ -214,6 +281,15 @@ bool OTA::handlePostSketch(WiFiClient &client, String &req_str)
 
     flash.write((void *)flashAddress, flashbuffer, sizeof(flashbuffer));
     flashAddress += sizeof(flashbuffer);
+
+    if (displayEnabled) {
+      display.fillRect(
+        map(i, 0, numPages, 0, DISPLAY_MAXX), 16,
+        map(i+1, 0, numPages, 0, DISPLAY_MAXX), DISPLAY_MAXY,
+        WHITE
+      );
+      display.display();
+    }
   }
 
   LOG.print("FLASH at 0x12000: 0x");
@@ -251,4 +327,6 @@ void OTA::stopHardware()
   LOG.end();
   WiFi.end();
   digitalWrite(LED_BUILTIN2, LOW);
+  senseBoxIO.powerXB1(false);
+  senseBoxIO.powerI2C(false);
 }