Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Use journald for system module on Debian 12 #41061

Merged
merged 38 commits into from
Oct 14, 2024
Merged

Use journald for system module on Debian 12 #41061

merged 38 commits into from
Oct 14, 2024

Commits on Oct 11, 2024

  1. Use journald for system/syslog module on Debian 12 

    This commit adds osVersion and osFamily to the builtin field used to
render the modules template. The system/syslog template is updated to
use journald for Debian 12.
    @belimawr
    belimawr committed Oct 11, 2024
    Configuration menu
    Copy the full SHA
    7677ba6 View commit details
    Browse the repository at this point in the history

  2. Add variables to force logs our journald

    @belimawr
    belimawr committed Oct 11, 2024
    Configuration menu
    Copy the full SHA
    eb0a798 View commit details
    Browse the repository at this point in the history

  3. Update docs

    @belimawr
    belimawr committed Oct 11, 2024
    Configuration menu
    Copy the full SHA
    5165201 View commit details
    Browse the repository at this point in the history

  4. [journald] enable facility filtering

    @belimawr
    belimawr committed Oct 11, 2024
    Configuration menu
    Copy the full SHA
    ba0b713 View commit details
    Browse the repository at this point in the history

  5. Add facility filtering to system module + journald

    @belimawr
    belimawr committed Oct 11, 2024
    Configuration menu
    Copy the full SHA
    1503234 View commit details
    Browse the repository at this point in the history

  6. Different pipelines for journald and files

    @belimawr
    belimawr committed Oct 11, 2024
    Configuration menu
    Copy the full SHA
    9300272 View commit details
    Browse the repository at this point in the history

  7. Implement journald ingest pipeline

    @belimawr
    belimawr committed Oct 11, 2024
    Configuration menu
    Copy the full SHA
    198c632 View commit details
    Browse the repository at this point in the history

  8. Implement Journald system/auth ingest pipeline

    @belimawr
    belimawr committed Oct 11, 2024
    Configuration menu
    Copy the full SHA
    15e56bf View commit details
    Browse the repository at this point in the history

  9. Add paths config for system module with journald input

    @belimawr
    belimawr committed Oct 11, 2024
    Configuration menu
    Copy the full SHA
    f1d10d6 View commit details
    Browse the repository at this point in the history

  10. Add tests for system.auth and fix ingest pipelines

    @belimawr
    belimawr committed Oct 11, 2024
    Configuration menu
    Copy the full SHA
    8e013bc View commit details
    Browse the repository at this point in the history

  11. Use .log and .journal files in tests

    @belimawr
    belimawr committed Oct 11, 2024
    Configuration menu
    Copy the full SHA
    e6905f5 View commit details
    Browse the repository at this point in the history

  12. fix syslog journald ingest pipeline

    @belimawr
    belimawr committed Oct 11, 2024
    Configuration menu
    Copy the full SHA
    84e2554 View commit details
    Browse the repository at this point in the history

  13. Add journald tests for system.syslog

    @belimawr
    belimawr committed Oct 11, 2024
    Configuration menu
    Copy the full SHA
    1841a02 View commit details
    Browse the repository at this point in the history

  14. Fix lint issues, update docs and changelog

    @belimawr
    belimawr committed Oct 11, 2024
    Configuration menu
    Copy the full SHA
    ae126c8 View commit details
    Browse the repository at this point in the history

  15. fix tests and improve docs

    @belimawr
    belimawr committed Oct 11, 2024
    Configuration menu
    Copy the full SHA
    4b06572 View commit details
    Browse the repository at this point in the history

  16. Improve template documentation

    @belimawr
    belimawr committed Oct 11, 2024
    Configuration menu
    Copy the full SHA
    dd7b43a View commit details
    Browse the repository at this point in the history

  17. Update docs and tests

    @belimawr
    belimawr committed Oct 11, 2024
    Configuration menu
    Copy the full SHA
    24f9895 View commit details
    Browse the repository at this point in the history

  18. update generated files

    @belimawr
    belimawr committed Oct 11, 2024
    Configuration menu
    Copy the full SHA
    f14f41c View commit details
    Browse the repository at this point in the history

  19. Fix lint issues

    @belimawr
    belimawr committed Oct 11, 2024
    Configuration menu
    Copy the full SHA
    d7d37d6 View commit details
    Browse the repository at this point in the history

  20. Update journal files to a compatible format

    @belimawr
    belimawr committed Oct 11, 2024
    Configuration menu
    Copy the full SHA
    fcc6cac View commit details
    Browse the repository at this point in the history

  21. Improve system test error handling 

    Set Filebeat to write logs to file where the framework can find it,
this avoids the 'file not found' exception thrown when trying to print
logs on test failures form masking the actual test failure.
    @belimawr
    belimawr committed Oct 11, 2024
    Configuration menu
    Copy the full SHA
    fe5bdb1 View commit details
    Browse the repository at this point in the history

  22. Create a new input to instantiate a jouranld or log input

    @belimawr
    belimawr committed Oct 11, 2024
    Configuration menu
    Copy the full SHA
    e86d97e View commit details
    Browse the repository at this point in the history

  23. Choose between journald and log input based on flag, update module 

    The module configuration template has been updated to use the new
input, the new input can select journald or traditional log files
based on the flags `use_journald` or `use_files`.
    @belimawr
    belimawr committed Oct 11, 2024
    Configuration menu
    Copy the full SHA
    5c84f4a View commit details
    Browse the repository at this point in the history

  24. Update configuration and docs

    @belimawr
    belimawr committed Oct 11, 2024
    Configuration menu
    Copy the full SHA
    bcc81e5 View commit details
    Browse the repository at this point in the history

  25. Update configs

    @belimawr
    belimawr committed Oct 11, 2024
    Configuration menu
    Copy the full SHA
    333cf0b View commit details
    Browse the repository at this point in the history

  26. update pipeline and tests

    @belimawr
    belimawr committed Oct 11, 2024
    Configuration menu
    Copy the full SHA
    0a1d441 View commit details
    Browse the repository at this point in the history

  27. Add processors

    @belimawr
    belimawr committed Oct 11, 2024
    Configuration menu
    Copy the full SHA
    d8fe075 View commit details
    Browse the repository at this point in the history

  28. Fix test_module.py 

    Fix test_module.py by passing the `-once` flag when the test file is
not a journal and by killing (SIGKILL) the Filebeat process if it does
not terminate gracefully in 15s.
    @belimawr
    belimawr committed Oct 11, 2024
    Configuration menu
    Copy the full SHA
    607ef82 View commit details
    Browse the repository at this point in the history

  29. Check if files exist to choose between jouranld and files

    @belimawr
    belimawr committed Oct 11, 2024
    Configuration menu
    Copy the full SHA
    d9e297d View commit details
    Browse the repository at this point in the history

  30. Finish toJournaldConfig and toFilesConfig

    @belimawr
    belimawr committed Oct 11, 2024
    Configuration menu
    Copy the full SHA
    4fce6a0 View commit details
    Browse the repository at this point in the history

  31. Update pipelines and config not to rely on non-input fields/config

    @belimawr
    belimawr committed Oct 11, 2024
    Configuration menu
    Copy the full SHA
    056d9e7 View commit details
    Browse the repository at this point in the history

  32. Fix tests and ingest pipeline

    @belimawr
    belimawr committed Oct 11, 2024
    Configuration menu
    Copy the full SHA
    9270dc3 View commit details
    Browse the repository at this point in the history

  33. Update golden files

    @belimawr
    belimawr committed Oct 11, 2024
    Configuration menu
    Copy the full SHA
    3328fce View commit details
    Browse the repository at this point in the history

  34. Update golden files to new input type

    @belimawr
    belimawr committed Oct 11, 2024
    Configuration menu
    Copy the full SHA
    648200d View commit details
    Browse the repository at this point in the history

  35. set input type back to the old config

    @belimawr
    belimawr committed Oct 11, 2024
    Configuration menu
    Copy the full SHA
    2bafcf0 View commit details
    Browse the repository at this point in the history

  36. Remove un-used code and small updates

    @belimawr
    belimawr committed Oct 11, 2024
    Configuration menu
    Copy the full SHA
    11e6655 View commit details
    Browse the repository at this point in the history

  37. move grok step to a separated pipeline

    @belimawr
    belimawr committed Oct 11, 2024
    Configuration menu
    Copy the full SHA
    fbfdbbe View commit details
    Browse the repository at this point in the history

  38. Add build constraints to support non-linux systems

    @belimawr
    belimawr committed Oct 11, 2024
    Configuration menu
    Copy the full SHA
    ea2e833 View commit details
    Browse the repository at this point in the history