Merge branch 'wellenvogel:master' into master

norbert-walter · Nov 30, 2024 · 4f0fd97 · 4f0fd97
2 parents fd896e0 + 5adad32
commit 4f0fd97
Show file tree

Hide file tree

Showing 41 changed files with 790 additions and 564 deletions.
diff --git a/Readme.md b/Readme.md
@@ -170,6 +170,17 @@ For details refer to the [example description](lib/exampletask/Readme.md).
 
 Changelog
 ---------
+[20241128](../../releases/tag/20241128)
+*********
+* additional correction for: USB connection on S3 stops [#81](../../issues/81)
+* [#71](../../pull/71): add BMP280 to [IIC Sensors](doc/Sensors.md), send 130311 for BMP380 and BME380
+* add an api function to add [own Sensors](doc/Sensors.md)
+* use a lock on the USB connection write site to avoid problems with NMEA and logs at the same time
+* allow to show unmapped XDR values in the data display
+* fix a bug that made the dashboard page disappear after a restart of the device
+* correctly handle empty fields in RMB messages
+* call the newly introduced web request handler for user tasks outside of an API lock
+
 [20241114](../../releases/tag/20241114)
 **********
 * UDP writer and reader - [#79](../../issues/79)

diff --git a/doc/Sensors.md b/doc/Sensors.md
@@ -1,6 +1,6 @@
 Sensors
 =======
-The software contains support for a couple of sensors (starting from [20231228](../../releases/tag/20231228) and extend in consecutive releases).
+The software contains support for a couple of sensors (starting from [20231228](../../releases/tag/20231228) and extended in consecutive releases).
 This includes some I2C Sensors and SSI rotary encoders.
 To connect sensors the following steps are necessary:
 
@@ -29,9 +29,57 @@ Bus Usage
 ---------
 When selecting sensors to be connected at the M5 grove ports in the [online build service](BuildService.md) the system will select the appropriate bus (i2c-1, i2c-2) by it's own. As you can have up to 4 grove ports (one at the device and 3 by using the [M5 Atomic PortABC](https://shop.m5stack.com/products/atomic-portabc-extension-base)) you can use both available i2c buses (and still utilize other groves for serial or CAN).
 
+Implementing Own Sensors
+---------------------
+To add an own sensor implementation you typically need to handle the following parts:
+* (opt) add a library that supports your sensor
+* add some [XDR Mapping](./XdrMappings.md) that will convert your generated NMEA2000 message into an NMEA0183 XDR record and ensure the display on the data page
+* implement the sensor initialization
+* implement the measurement and generating the NMEA2000 message
+
+You typically would do this in a [user task](../lib/exampletask/Readme.md).<br>
+You can either just implement everything by your own or reuse the existing infrastructure for sensors.
+
+OwnImplementation
+__________________
+
+To implement everything by your own just create a config.json for the parameters you need, add an XDR mapping in a task init function (see e.g. [PressureXdr](../lib/iictask/GwIicSensors.h#L27)).
+In your user taks just initialize the sensor using your config values and add a loop that periodically measures the sensor value and sends out an nmea2000 message (using the [api->sendN2KMessage](../lib/api/GwApi.h#L137)).
+To display some information on the status page just add a countergroup in your task init function ([api->addCounter](../lib/api/GwApi.h#L170)) and increment a counter of this group on every measure ([api->increment](../lib/api/GwApi.h#L171)).
+To utilize a bus you typically would need to add the related library to your environment and add some bus initialization to define the pins that are used for this particular bus.
+Be carefull if you additionally would like to use sensors from the core as the core potentially would already initialize some bus - depending on the compile flags you provide.
+If you need additional libraries for your sensor just add a platformio.ini to your usertask and define an environment that contains the additional libraries.
+If you would like to compile also other environments (i.e. without the additional libraries) you should wrap all the code that references the additional libraries with some #ifdef and add a define to your environment (see the implementations of the [sensors in the core](../lib/iictask/)).
+
+Using the core infrastructure
+_____________________________
+For sensors of bus types that are already supported by the core (mainly I2C) you can simplify your implementation.
+Just also start with a [usertask](../lib/exampletask/Readme.md). But you only need the task init function, a config.json and potentially a platformio.ini.
+In your task code just implement a class that handles the sensor - it should inherit from [SensorBase](../lib/sensors/GwSensor.h#L20) or from [IICSensorBase](../lib/iictask/GwIicSensors.h#L16).<br>
+You need to implement:
+* _readConfig_ - just read your configuration and fill attributes of your class. Especially set the "ok" to true and fill the interval field to define your measure interval.
+* _preinit_ - check if your snesor is configured ,add necessary XDR mappings and return true if your sensor is active. Do __not__ yet initialize the sensor hardware.
+* _isActive_ - return true if your sensor is active
+* _initDevice_ - init your sensor hardware and return true if this was ok
+* _measure_ - read the sensor data, send NMEA2000 messages and increment 
+counters
+
+The busType and busId fields of your imnplementation have to be set correctly.<br>
+In your task init function add the sensors you would like to be handled using [api->addSensor](../lib/api/GwApi.h#L218).
+
+All the internal sensors are implemented using this approach - e.g. [BME280](../lib/iictask/GwBME280.cpp#L23).<br>
+Do not get confused by all the different defines and the special config handling - this is only there to be as generic as possible - typically not necessary for your own sensor implementation.
+
+To use an IIC bus you need to compile with flags that define the pins to be used for the IIC bus:
+* busId 1 (IIC bus 1): GWIIC_SDA, GWIIC_SCL
+* busId 2 (IIC bus 2): GWIIC_SDA2, GWIIC_SCL2
+
+So you would need to add such definitions to your environment in your platformio.ini.
+
 Implemented Sensors
 -------------------
-* [BME280](https://www.bosch-sensortec.com/media/boschsensortec/downloads/datasheets/bst-bme280-ds002.pdf): temperature/humidity/pressure [PGNs: 130314,130312, 130313]
+* [BME280](https://www.bosch-sensortec.com/media/boschsensortec/downloads/datasheets/bst-bme280-ds002.pdf): temperature/humidity/pressure [PGNs: 130314,130312, 130313, 130311 since 20241128 ]
+* [BMP280](https://www.bosch-sensortec.com/media/boschsensortec/downloads/datasheets/bst-bmp280-ds001.pdf) [since 20241128]: temperature/pressure [PGNs: 130314,130312, 130311]
 * [QMP6988](https://m5stack.oss-cn-shenzhen.aliyuncs.com/resource/docs/datasheet/unit/enviii/QMP6988%20Datasheet.pdf): pressure [PGN: 130314]
 * [SHT30](https://m5stack.oss-cn-shenzhen.aliyuncs.com/resource/docs/datasheet/unit/SHT3x_Datasheet_digital.pdf): temperature and humidity [PGNs: 130312, 130313]
 * [M5-ENV3](https://docs.m5stack.com/en/unit/envIII): combination of QMP6988 and SHT30 [PGNs: 130314,130312, 130313]

diff --git a/lib/api/GwApi.h b/lib/api/GwApi.h
@@ -9,6 +9,9 @@
 #include "GwSynchronized.h"
 #include <map>
 #include <ESPAsyncWebServer.h>
+#include "GwSensor.h"
+#include <functional>
+#include <memory>
 class GwApi;
 typedef void (*GwUserTaskFunction)(GwApi *);
 //API to be used for additional tasks
@@ -43,11 +46,7 @@ class GwApi{
          * the core part will not handle this data at all but
          * this interface ensures that there is a correct locking of the
          * data to correctly handle multi threading
-         * The user code should not use this intterface directly
-         * but instead it should use the static functions
-         * apiGetXXX(GwApi *,...)
-         * apiSetXXX(GwApi *,...)
-         * that will be created by the macro DECLARE_TASK_INTERFACE
+         * there is no protection - i.e. every task can get and set the data
          */
         class TaskInterfaces
         {
@@ -61,9 +60,9 @@ class GwApi{
             };
             using Ptr = std::shared_ptr<Base>;
         protected:
-            virtual bool iset(const String &file, const String &name, Ptr v) = 0;
+            virtual bool iset(const String &name, Ptr v) = 0;
             virtual Ptr iget(const String &name, int &result) = 0;
-            virtual bool iclaim(const String &name, const String &task)=0;
+            virtual bool iupdate(const String &name,std::function<Ptr(Ptr v)>)=0;
         public:
             template <typename T>
             bool set(const T &v){
@@ -76,6 +75,10 @@ class GwApi{
             }
             template <typename T>
             bool claim(const String &task){
+                return true;
+            }
+            template <typename T>
+            bool update(std::function<bool(T *)>){
                 return false;
             }
         };
@@ -206,7 +209,13 @@ class GwApi{
          * @param name: the config name this value is used for
          * @param value: the current value
         */
-        virtual void setCalibrationValue(const String &name, double value);
+        virtual void setCalibrationValue(const String &name, double value)=0;
+        /**
+         * add a sensor
+         * depending on the type it will be added to the appropriate task
+         * @param sensor: created sensor config
+        */
+        virtual void addSensor(SensorBase* sensor,bool readConfig=true){};
 
         /**
          * not thread safe methods
@@ -231,7 +240,10 @@ static void checkDef(T... args){};
 #define DECLARE_USERTASK_PARAM(task,...)
 #endif
 #ifndef DECLARE_INITFUNCTION
-#define DECLARE_INITFUNCTION(task)
+#define DECLARE_INITFUNCTION(task,...)
+#endif
+#ifndef DECLARE_INITFUNCTION_ORDER
+#define DECLARE_INITFUNCTION_ORDER(task,...)
 #endif
 #ifndef DECLARE_CAPABILITY
 #define DECLARE_CAPABILITY(name,value)
@@ -255,27 +267,20 @@ static void checkDef(T... args){};
  *          int ival2=99;
  *          String sval="unset";
  * };
- * DECLARE_TASKIF(testTask,TestTaskApi);
- * The macro will generate 2 static funtions:
+ * DECLARE_TASKIF(TestTaskApi);
  * 
- * bool apiSetTestTaskApi(GwApi *api, const TestTaskApi &v);
- * TestTaskApi apiGetTestTaskApi(GwApi *api, int &result);
- * 
- * The setter will return true on success.
- * It is intended to be used by the task that did declare the api.
- * The getter will set the result to -1 if no data is available, otherwise
- * it will return the update count (so you can check if there was a change 
- * compared to the last call).
  * It is intended to be used by any task.
  * Be aware that all the apis share a common namespace - so be sure to somehow
  * make your API names unique.
- * 
+ * To utilize this interface a task can call:
+ * api->taskInterfaces()->get<TestTaskApi>(res) //and check the result in res
+ * api->taskInterfaces()->set<TestTaskApi>(value)
  * 
 */
 #define DECLARE_TASKIF_IMPL(type) \
     template<> \
     inline bool GwApi::TaskInterfaces::set(const type & v) {\
-        return iset(__FILE__,#type,GwApi::TaskInterfaces::Ptr(new type(v))); \
+        return iset(#type,GwApi::TaskInterfaces::Ptr(new type(v))); \
     }\
     template<> \
     inline type GwApi::TaskInterfaces::get<type>(int &result) {\
@@ -286,13 +291,42 @@ static void checkDef(T... args){};
         }\
         type *tp=(type*)ptr.get(); \
         return type(*tp); \
-    }\
+    } \
     template<> \
-    inline bool GwApi::TaskInterfaces::claim<type>(const String &task) {\
-        return iclaim(#type,task);\
-    }\
+    inline bool GwApi::TaskInterfaces::update(std::function<bool(type *)> f) { \
+        return iupdate(#type,[f](GwApi::TaskInterfaces::Ptr cp)->GwApi::TaskInterfaces::Ptr{ \
+            if (cp) { \
+                f((type *)cp.get()); \
+                return cp; \
+            } \
+            type * et=new type(); \
+            bool res=f(et); \
+            if (! res){ \
+                delete et; \
+                return GwApi::TaskInterfaces::Ptr(); \
+            } \
+            return GwApi::TaskInterfaces::Ptr(et); \
+        }); \
+    } \
+
+
 
 #ifndef DECLARE_TASKIF
     #define DECLARE_TASKIF(type) DECLARE_TASKIF_IMPL(type)
 #endif
+
+/**
+ * do not use this interface directly
+ * instead use the API function addSensor
+*/
+class ConfiguredSensors : public GwApi::TaskInterfaces::Base{
+    public:
+    SensorList sensors;
+};
+DECLARE_TASKIF(ConfiguredSensors);    
+
+//order for late init functions
+//all user tasks should have lower orders (default: 0)
+#define GWLATEORDER 9999
+
 #endif
diff --git a/lib/channel/GwChannelList.cpp b/lib/channel/GwChannelList.cpp
@@ -427,6 +427,7 @@ void GwChannelList::begin(bool fallbackSerial){
     if (! fallbackSerial){
         GwSerial *usbSerial=createSerialImpl(config, logger,USB_CHANNEL_ID,GWUSB_RX,GWUSB_TX,true);
         if (usbSerial != nullptr){
+            usbSerial->enableWriteLock(); //as it is used for logging we need this additionally
             GwChannel *usbChannel=createChannel(logger,config,USB_CHANNEL_ID,usbSerial,GWSERIAL_TYPE_BI);
             if (usbChannel != nullptr){
                 addChannel(usbChannel);

diff --git a/lib/config/GwConverterConfig.h b/lib/config/GwConverterConfig.h
@@ -71,9 +71,12 @@ class GwConverterConfig{
       int rmcInterval=1000;
       int rmcCheckTime=4000;
       int winst312=256;
+      bool unmappedXdr=false;
+      unsigned long xdrTimeout=60000;
       std::vector<WindMapping> windMappings;
       void init(GwConfigHandler *config, GwLog*logger){
         minXdrInterval=config->getInt(GwConfigDefinitions::minXdrInterval,100);
+        xdrTimeout=config->getInt(GwConfigDefinitions::timoSensor);
         starboardRudderInstance=config->getInt(GwConfigDefinitions::stbRudderI,0);
         portRudderInstance=config->getInt(GwConfigDefinitions::portRudderI,-1);
         min2KInterval=config->getInt(GwConfigDefinitions::min2KInterval,50);
@@ -83,6 +86,7 @@ class GwConverterConfig{
         rmcInterval=config->getInt(GwConfigDefinitions::sendRMCi,1000);
         if (rmcInterval < 0) rmcInterval=0;
         if (rmcInterval > 0 && rmcInterval <100) rmcInterval=100;
+        unmappedXdr=config->getBool(GwConfigDefinitions::unknownXdr);
         winst312=config->getInt(GwConfigDefinitions::winst312,256);
         for (auto && it:windConfigs){
           String cfg=config->getString(it.second);

diff --git a/lib/exampletask/GwExampleTask.cpp b/lib/exampletask/GwExampleTask.cpp
@@ -156,11 +156,11 @@ class ExampleWebData{
         vSemaphoreDelete(lock);
     }
     void set(int v){
-        GWSYNCHRONIZED(&lock);
+        GWSYNCHRONIZED(lock);
         data=v;
     }
     int get(){
-        GWSYNCHRONIZED(&lock);
+        GWSYNCHRONIZED(lock);
         return data;
     }
 };

diff --git a/lib/gwwifi/GwWifi.cpp b/lib/gwwifi/GwWifi.cpp
@@ -108,7 +108,7 @@ void GwWifi::loop(){
         }
         else{
             if (! clientIsConnected){
-                LOG_DEBUG(GwLog::LOG,"wifiClient %s now connected to",wifiSSID->asCString());
+                LOG_DEBUG(GwLog::LOG,"wifiClient now connected to %s at %s",wifiSSID->asCString(),WiFi.localIP().toString().c_str());
                 clientIsConnected=true;
             }
         }

diff --git a/lib/hardware/GwM5Grove.in b/lib/hardware/GwM5Grove.in
@@ -64,15 +64,15 @@
 #endif
 
 #GROVE
-#ifdef M5_ENV4$GS$
-  #ifndef M5_GROOVEIIC$GS$
-    #define M5_GROOVEIIC$GS$
-  #endif
-  GROOVE_IIC(SHT3X,$Z$,1)
-  GROOVE_IIC(BMP280,$Z$,1)
-  #define _GWSHT3X
-  #define _GWBMP280
-#endif
+//#ifdef M5_ENV4$GS$
+//  #ifndef M5_GROOVEIIC$GS$
+//    #define M5_GROOVEIIC$GS$
+//  #endif
+//  GROOVE_IIC(SHT3X,$Z$,1)
+//  GROOVE_IIC(BMP280,$Z$,1)
+//  #define _GWSHT3X
+//  #define _GWBMP280
+//#endif
 
 #GROVE
 //example: -DSHT3XG1_A : defines STH3Xn1 on grove A - x depends on the other devices