Skip to content

Commit

Permalink
Merge pull request #18 from jibrilsharafi/development
Browse files Browse the repository at this point in the history
Add callback (#17)
  • Loading branch information
jibrilsharafi authored Nov 3, 2024
2 parents c9f2f96 + 1d75033 commit 52dadc7
Show file tree
Hide file tree
Showing 6 changed files with 387 additions and 127 deletions.
16 changes: 13 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@

[![BuyMeACoffee](https://img.shields.io/badge/Buy%20Me%20a%20Coffee-ffdd00?style=for-the-badge&logo=buy-me-a-coffee&logoColor=black)](https://buymeacoffee.com/jibrilsharafi)

A **simple** logging library capable of **saving logs to memory** and with a **comprehensive format** capable of including all the accessory information to each message.
A **simple** logging library capable of **saving logs to memory** and with a **comprehensive format** capable of including all the accessory information to each message. Easily integrated into any project, the **callback function** allows for a wide range of applications, from sending logs to a server to displaying them on a screen.

## Installing

Expand All @@ -31,6 +31,7 @@ Alternatively, the library can be installed manually by downloading the latest r
The library so far has been successfully tested on the following microcontrollers:

- ESP32S3
- ESP-WROVER

## Usage

Expand Down Expand Up @@ -92,6 +93,15 @@ The library provides the following public methods:
- `clearLog()`: clear the log.
- `dump(Stream& stream)`: dump the log to a stream, such as the Serial or an opened file.
- `setDefaultConfig()`: set the default configuration.
- `setCallback(LogCallback callback)`: Register a callback function that will be called whenever a log message is generated. The callback receives the following parameters:
- `timestamp`: Current formatted timestamp
- `millisEsp`: System uptime in milliseconds
- `level`: Log level as a lowercase string
- `coreId`: The core ID that generated the log
- `function`: Name of the function that generated the log
- `message`: The actual log message

Example of using the callback:

For a detailed example, see the [basicUsage](examples/basicUsage/basicUsage.ino) and [basicServer](examples/basicServer/basicServer.ino) in the examples folder.

Expand All @@ -104,11 +114,11 @@ For more information regarding the necessity of this library, see the [WHY.md](W
### What's next?

- [x] **Automatic log clearing**: if the free memory is less than a certain threshold, the oldest logs should be deleted, keeping the memory usage under control.
- [ ] **Log to SD card**: the ability to log to an external SD card would be a great addition, as it would allow to store a much larger amount of logs.
- [x] **Log to SD card**: the ability to log to an external SD card would be a great addition, as it would allow to store a much larger amount of logs.
- [x] **Remove ArduinoJson dependency**: the library is used only for the configuration file, and as such it could be removed by implementing a simpler configuration in .txt format.
- [ ] **Upgrade to LittleFS**: the SPIFFS library is deprecated, and as such it should be replaced with the LittleFS library. Some effort has been made in this direction, but the nested folder approach gave some problems.
- [ ] **Test other microcontrollers**: the library is currently tested only on the ESP32, but it should be tested on other microcontrollers to ensure compatibility. The *ESP8266* has been tested, but full compatibility has not been achieved yet.
- [ ] **MQTT integration**: the ability to send logs to an MQTT server would be a great addition, as it would allow to monitor the device remotely.
- [x] **MQTT integration**: the ability to send logs to an MQTT server would be a great addition, as it would allow to monitor the device remotely.
- [x] **Consistent spacing**: the spacing between the different parts of the log should be consistent, to make it easier to read.
- [ ] **Buffered logging**: the ability to buffer the logs and send them in chunks would be a great addition, as it would allow to save power and reduce the number of writes to the memory. This has been tested, but the results did not show any significant improvement in speed.
- [x] **New APIs**: the logging APIs should be updated to be more consistent with the usual logging APIs, such as log.debug(...), log.info(...), log.error(...), etc.
Expand Down
236 changes: 236 additions & 0 deletions examples/callbackHttpMqtt/callbackHttpMqtt.ino
Original file line number Diff line number Diff line change
@@ -0,0 +1,236 @@
/*
* File: callbackHttpMqtt.ino
* ----------------------------
* This example demonstrates integrating AdvancedLogger with MQTT and HTTP logging.
* It shows how to:
* - Forward logs to an HTTP endpoint
* - Send logs to a local MQTT broker
* - Track logging performance metrics
* - Handle network reconnections
* - Format logs as JSON
*
* Author: Jibril Sharafi, @jibrilsharafi
* Created: 21/03/2024
* Last modified: 22/05/2024
* GitHub repository: https://github.com/jibrilsharafi/AdvancedLogger
*
* This library is licensed under the MIT License. See the LICENSE file for more information.
*
* This example covers the addition of a simple web server to the basicUsage, which allows
* the user to explore the log and configuration files remotely.
*
* All the other advanced usage features are reported in the basicUsage example.
*/

#include <Arduino.h>
#include <SPIFFS.h>
#include <WiFi.h>
#include <HTTPClient.h>
#include <PubSubClient.h>

#include "AdvancedLogger.h"

// HTTP configuration
const String serverEndpoint = "YOUR_IP"; // **** CHANGE THIS TO YOUR SERVER ****
HTTPClient http;

// MQTT configuration
const char* mqttServer = "YOUR_BROKER"; // **** CHANGE THIS TO YOUR BROKER ****
const unsigned int mqttPort = 1883;
const char* mainTopic = "advancedlogger/log";
const unsigned int bufferSize = 1024;

// **** CHANGE THESE TO YOUR SSID AND PASSWORD ****
const char *ssid = "SSID";
const char *password = "PASSWORD";

const char *customLogPath = "/customPath/log.txt";
const char *customConfigPath = "/customPath/config.txt";
const char *customTimestampFormat = "%Y-%m-%d %H:%M:%S";
AdvancedLogger logger(
customLogPath,
customConfigPath,
customTimestampFormat);

const int timeZone = 0; // UTC. In milliseconds
const int daylightOffset = 0; // No daylight saving time. In milliseconds
const char *ntpServer1 = "pool.ntp.org";
const char *ntpServer2 = "time.nist.gov";
const char *ntpServer3 = "time.windows.com";

unsigned long lastMillisLogClear = 0;
unsigned long intervalLogClear = 60000;
unsigned int maxLogLines = 100; // Low value for testing purposes

// Add after WiFi client
WiFiClient espClient;
PubSubClient mqttClient(espClient);

// Modified callback function to make the JSON in the callback faster
char jsonBuffer[512]; // Pre-allocated buffer
String cachedDeviceId;
String cachedTopicPrefix;

// Get device ID from MAC
String getDeviceId() {
return String((uint32_t)ESP.getEfuseMac(), HEX);
}

/*
This callback function will be called by the AdvancedLogger
whenever a log is generated. It will pass the log information,
then the function will decide what to do with it (eg. based on
the level, it may decide to send it to an HTTP endpoint or to
set a flag).
In this example, the function will:
- Format the log as JSON
- Send the log to an HTTP endpoint
- Publish the log to an MQTT topic
The function will also measure the time taken to format the JSON,
send the HTTP request, and publish the MQTT message.
*/
void callback(
const char* timestamp,
unsigned long millisEsp,
const char* level,
unsigned int coreId,
const char* function,
const char* message
) {
if (WiFi.status() != WL_CONNECTED) return;

unsigned long startJson = micros();

snprintf(jsonBuffer, sizeof(jsonBuffer),
"{\"timestamp\":\"%s\","
"\"millis\":%lu,"
"\"level\":\"%s\","
"\"core\":%u,"
"\"function\":\"%s\","
"\"message\":\"%s\"}",
timestamp,
millisEsp,
level,
coreId,
function,
message);

unsigned long jsonTime = micros() - startJson;

// HTTP POST
unsigned long startHttp = micros();
HTTPClient http;
http.begin(serverEndpoint);
http.addHeader("Content-Type", "application/json");
int httpCode = http.POST(jsonBuffer);
if (httpCode != HTTP_CODE_OK) {
Serial.printf("HTTP POST failed: %s\n", http.errorToString(httpCode).c_str());
}
http.end();
unsigned long httpTime = micros() - startHttp;

// MQTT Publish
unsigned long startMqtt = micros();
if (mqttClient.connected()) {
if (cachedDeviceId.isEmpty()) {
cachedDeviceId = getDeviceId();
cachedTopicPrefix = String(mainTopic) + "/" + cachedDeviceId + "/log/";
}

String topic = cachedTopicPrefix + String(level);

if (!mqttClient.publish(topic.c_str(), jsonBuffer)) {
Serial.printf("MQTT publish failed to %s. Error: %d\n",
topic.c_str(), mqttClient.state());
}
}
unsigned long mqttTime = micros() - startMqtt;

Serial.printf("Durations - JSON: %lu µs, HTTP: %lu µs, MQTT: %lu µs\n",
jsonTime, httpTime, mqttTime);
}

void reconnectMQTT() {
while (!mqttClient.connected()) {
String clientId = "ESP32Client-" + getDeviceId();
if (mqttClient.connect(clientId.c_str())) {
logger.info("MQTT Connected", "reconnectMQTT");
} else {
logger.error("MQTT Connection failed, rc=%d", "reconnectMQTT", mqttClient.state());
}
}
}

void setup()
{
// Initialize Serial and SPIFFS (mandatory for the AdvancedLogger library)
// --------------------
Serial.begin(115200);

if (!SPIFFS.begin(true)) // Setting to true will format the SPIFFS if mounting fails
{
Serial.println("An Error has occurred while mounting SPIFFS");
}

logger.begin();
logger.setMaxLogLines(maxLogLines);
logger.setCallback(callback);

logger.debug("AdvancedLogger setup done!", "basicServer::setup");

// Connect to WiFi
// --------------------
// Connect to the specified SSID
WiFi.begin(ssid, password);

while (WiFi.status() != WL_CONNECTED)
{
delay(1000);
logger.info("Connecting to WiFi... SSID: %s | Password: %s", "basicServer::setup", ssid, password);
}

logger.info(("IP address: " + WiFi.localIP().toString()).c_str(), "basicServer::setup");

mqttClient.setServer(mqttServer, mqttPort);
mqttClient.setBufferSize(bufferSize); // Raise the buffer size as the standard one is only 256 bytes
reconnectMQTT();

configTime(timeZone, daylightOffset, ntpServer1, ntpServer2, ntpServer3);

logger.info("Server started!", "basicServer::setup");

logger.info("Setup done!", "basicServer::setup");
}

void loop()
{
if (!mqttClient.connected()) {
reconnectMQTT();
}
mqttClient.loop();

// Test a burst of messages to see the performance
for (int i = 0; i < 10; i++) {
logger.verbose("[BURST] This is a verbose message", "basicServer::loop");
logger.debug("[BURST] This is a debug message!", "basicServer::loop");
logger.info("[BURST] This is an info message!!", "basicServer::loop");
logger.warning("[BURST] This is a warning message!!!", "basicServer::loop");
logger.error("[BURST] This is a error message!!!!", "basicServer::loop");
logger.fatal("[BURST] This is a fatal message!!!!!", "basicServer::loop");
}

logger.debug("This is a debug message!", "basicServer::loop");
delay(500);
logger.info("This is an info message!!", "basicServer::loop");
delay(500);
logger.warning("This is a warning message!!!", "basicServer::loop");
delay(500);
logger.error("This is a error message!!!!", "basicServer::loop");
delay(500);
logger.fatal("This is a fatal message!!!!!", "basicServer::loop");
delay(500);
logger.info("This is an info message!!", "basicServer::loop", true);
delay(1000);
}
2 changes: 1 addition & 1 deletion library.json
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
}
],
"dependencies": {},
"version": "1.2.2",
"version": "1.3.0",
"frameworks": "arduino",
"platforms": "*"
}
2 changes: 1 addition & 1 deletion library.properties
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
name=AdvancedLogger
version=1.2.2
version=1.3.0
author=Jibril Sharafi <[email protected]>
maintainer=Jibril Sharafi <[email protected]>
sentence=Library for simple logging to memory with comprehensive format.
Expand Down
Loading

0 comments on commit 52dadc7

Please sign in to comment.