Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

initial version of logging feature request #206

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions docs/features/analysis-infra/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -16,4 +16,8 @@ Analysis Infrastructure
#######################

.. toctree::
:titlesonly:
:maxdepth: 1

logging/index.rst
tracing/index.rst
10 changes: 6 additions & 4 deletions docs/features/analysis-infra/logging/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -12,9 +12,11 @@
# SPDX-License-Identifier: Apache-2.0
# *******************************************************************************

Logging
#######
.. Logging
.. #######

.. toctree::
console-output/index.rst
mw-log-api/index.rst
:titlesonly:
:maxdepth: 1

./mw-fr_logging.rst
202 changes: 202 additions & 0 deletions docs/features/analysis-infra/logging/mw-fr_logging.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,202 @@
..
# *******************************************************************************
# Copyright (c) 2024 Contributors to the Eclipse Foundation
#
# See the NOTICE file(s) distributed with this work for additional
# information regarding copyright ownership.
#
# This program and the accompanying materials are made available under the
# terms of the Apache License Version 2.0 which is available at
# https://www.apache.org/licenses/LICENSE-2.0
#
# SPDX-License-Identifier: Apache-2.0
# *******************************************************************************

Logging
#######

.. document:: Logging
:id: DOC_Logging
:status: draft
:safety: ASIL_B
:tags: feature_request


Feature flag
============

To activate this feature, use the following feature flag:

``experimental_logging``

Abstract
========

This feature request proposes the development of a safe, efficient and robust logging framework for embedded systems.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are we general enough that 'embedded system' applies here, or do we just focus on score?

- This feature request proposes the development of a safe, efficient and robust logging framework for embedded systems.
+ This feature request proposes the development of a safe, efficient and robust logging framework within score.



Motivation
==========

- currently no solution available in the score platform

Rationale
=========

Logging provides the possibility to understand the running system by capturing detailed information about system
events and application-level activities.

Specification
=============

[Describe the requirements, architecture of any new feature.] [or]
[Describe the change to requirements, architecture, implementation, process, documentation, infrastructure of any change request.]

.. note::
A Feature Request shall specify the stakeholder requirements as part of our platform/project.
Thereby the :need:`RL_technical_lead` will approve these requirements as part of accepting the Feature Request (e.g. merging the PR with the Feature Request).

Functionality
-------------

Logging has to support the following features:

- Timestamping

- local timestamp for each log entry
- original timestamp for routed log entries
- timestamp synchronization for log entries coming from different logging nodes-

- Log severity levels
- Log prioritization in case of resource conflicts
- logging of early startup events
- Log filtering

- Support application and context identifier like in DLT
- Filtering by log levels for components and application level
- Out of scope "time" (tooling topic)
- logs filtering by logging entity ID (e.g., ECU ID, Application ID, Context ID such as in DLT)
- logs filtering by log level (e.g., FATAL, ERROR, WARN, INFO, DEBUG, VERBOSE such as in DLT)
- Filtering options: it should have filter on app id and more fine-tuned filters on context ids.

- message loss detection
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Message loss detection use cases:

  • discarded log messages in case of internal buffer full
  • lost messages detection based on global message counter (e.g. MCNT from DLT protocol)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hint: prio logic needs to define late


- optional functionality for logging. Important for tracing.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am really not sure, if it is a good idea to have message loss detection only optional for logging. How trustworthy is a log that has for example only every second message in it? Or misses like 10 messages inbetween without a notice. And me, as a log consumer, I have no clue about it? I think this is the complete log.

- would become mandatory in case verification is done via logging feature (not recommended)

- Context specific log level activation at runtime

Log Sources
-----------

- user application
- component features & platform

Copy link

@palexuc palexuc Jan 29, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Log Sources (additional)

  • Networking (e.g., CAN, DBUS, MOST, IPC)
  • Files (e.g., crash reports)
  • Processes
  • different types of system traces (depending on the OS) which need to be routed as standardized log messages (e.g., traces from the Bootloader, from the Kernel, from the native logger of the Operating System, from stdin)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • covert by using of logging APIs in Application Layer and MW Layer applications.
  • record in-/out-put states of an application belongs to recompute/replay
  • not scope of logging
  • log collection is not part of basis logging system

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As discussed in the workshop today:

[the logs from] the Kernel, from the native logger of the Operating System

can be provided via the interfaces specified here in 0.5 by implemeting a bridge that is reading the sys level logs on the one side and writing them into score on the other side.
It does not have to be part of the specification for 0.5.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about

  • early startup

as a log source also? We did mention it in the Functionality section.

Log Sinks
---------
- log sinks shall be transparent to the application (e.g. by a facade )
- console
- File system

- local
- external mounted PCIe Drive

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd not be specific to 'PCIe' here. Also drive is maybe not the right term?

Suggested change
- external mounted PCIe Drive
- external mounted storage

- Cloud native drive via network

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- Cloud native drive via network
- cloud storage via network


- Network (incl. second dedicated Ethernet Channel)
- Logs shall appear on stdout when running unit tests

Configuration
-------------

- Log level
- Log Sinks

- Log storage device
- Log storage strategy (e.g., on system shutdown, on each message, on demand, on file size, on cache size, with circular or linear buffer)

- Buffer size
- Storage size (e.g. storage size of log files)
- Permission settings

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We did not mention 'permission' until here.
What is meant by permission? If we want to have permissions, should we add it to the functionality section as well then?

- log filter
- Logging entity ID (e.g., ECU ID, Application ID, Context ID such as in DLT)
- On-demand functionality (e.g., enable / disable the log storage)
- Provide fallback configs. E.g.: App, System-wide

- Possibility to provide an extension for custom types

Error handling
--------------

- reboot/reset
- On logging errors, the framework should continue if recoverable; otherwise, deactivate silently and set an error state reported on shutdown.
- The user application should not be affected by logging framework errors.

Context
-------

- Supported operating system: QNX, Linux (encapsulation via OSAL)
- Supported programming languages: C++, Rust, Python (e.g. for tests)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For Rust, we would like to be able to use the log crate. It provides an (allocation-free) facade without forcing a specific logger implementation. Advantages of using this facade:

  • We stay with language best-practices.
  • New contributors directly feel at home with the logging.
  • Any library dependency which we might use probably logs against this facade. So we would get logs of dependencies for free by using it as well and implementing a logger for the facade.

There might be comparable defacto standards for C++ and Python.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Design decision for detailed design => copy there

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Python comes with logging out of the box and the concepts are similar to Rust, where you can add custom 'handlers'.
C++ doesn't really have such a standardized interface. There are libraries like log4cplus or spdlog or even boost logging which are all more or less common. But nothing as standardized as with the other two.

Copy link

@palexuc palexuc Jan 29, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Context (additional)

  • Supported shell scripting (depending on OS)
  • Supported documentation tools (e.g., Doxygen, Confluence)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+plantuml

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not in scope of logging. belongs to Process / Docu / Tools

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think C is a score supported language also. Should we have it also on our radar?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Kinda relates to https://github.com/eclipse-score/score/pull/206/files#r1952715564 as it probably would be convenient to have a logging interface that works with c++ and c or is at least similar


Resource consumption
--------------------

- Storage
- Communication Channel
- Runtime resources
- Low impact on overall performance ---> QoS for handling overflows/dropping log messages

Norms/Standards
---------------

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

@arsibo arsibo Feb 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • check for IP
  • compatibility to DLT has to be add


Backwards Compatibility
=======================
\-

Security Impact
===============

- only authorized users should be able to access the log files

Safety Impact
=============

- The interface should support the ASIL level of the user function.
- Dependent on the Safety concept of the over all system, the logging should be classified accordingly.
That could be the case if the logging information are part of the verification strategy..

License Impact
==============

- not known

How to Teach This
=================

How to Use: Logging Guideline

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
=============================


Definitions
===========

Logging
-------

Write out some message
General progress description (incl. Context)
- state changes
- Infos
- errors

When logging from a library, it shall be possible to easily associate the logs to the library. A solution could be to provide contexts.

Tracing
-------

Example - User need to correlate kernel traces to user traces for debugging , analyzing etc
Follow the control flow (on function call basis)of the application.

Recompute
---------

All data to recompute a module (Input, Output, States)
44 changes: 44 additions & 0 deletions docs/features/analysis-infra/logging/mw-fr_logging_DD_Input.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
..
# *******************************************************************************
# Copyright (c) 2024 Contributors to the Eclipse Foundation
#
# See the NOTICE file(s) distributed with this work for additional
# information regarding copyright ownership.
#
# This program and the accompanying materials are made available under the
# terms of the Apache License Version 2.0 which is available at
# https://www.apache.org/licenses/LICENSE-2.0
#
# SPDX-License-Identifier: Apache-2.0
# *******************************************************************************

This document includes the detailed design findings from the feature request review.

Logging
#######

Specification
=============
Context
-------
.. related feature from feature request
- Supported programming languages: C++, Rust, Python (e.g. for tests)

.. detailed design finding
For Rust, we would like to be able to use the log crate. It provides an (allocation-free) facade without forcing a specific logger implementation. Advantages of using this facade:

We stay with language best-practices.
New contributors directly feel at home with the logging.
Any library dependency which we might use probably logs against this facade. So we would get logs of dependencies for free by using it as well and implementing a logger for the facade.
There might be comparable defacto standards for C++ and Python.


Resource consumption
--------------------
.. related feature from feature request
- Runtime ressources
- Low impact on overall performance ---> QoS for handling overflows/dropping log messages

.. detailed design finding
Ideally, in a language like Rust, logging messages should be possible without allocations for individual logs.
Using the defacto standard log crate as facade, implementors of this facade should not allocate to write / forward log messages.
Comment on lines +1 to +44

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the purpose of this file? It does not seem to be referenced from somewhere.

16 changes: 0 additions & 16 deletions docs/features/analysis-infra/logging/mw-log-api/index.rst

This file was deleted.

Original file line number Diff line number Diff line change
Expand Up @@ -12,5 +12,11 @@
# SPDX-License-Identifier: Apache-2.0
# *******************************************************************************

Console Output
##############
.. Tracing
.. #######

.. toctree::
:titlesonly:
:maxdepth: 1

./mw-fr_tracing.rst
Loading