Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multiline logging failed to take effect #486

Open
jkqjkq opened this issue Aug 27, 2020 · 3 comments
Open

Multiline logging failed to take effect #486

jkqjkq opened this issue Aug 27, 2020 · 3 comments

Comments

@jkqjkq
Copy link

jkqjkq commented Aug 27, 2020

Judging from the left side of the picture, these log messages are not sent together, sent line by line
image

configuration information

docker run -d --name="logspout" --volume=/var/run/docker.sock:/var/run/docker.sock \
--volume=/etc/hostname:/etc/host_hostname:ro -p 8000:80 \
-e MULTILINE_PATTERN='^\[\d{4}-\d{2}-\d{2}' \
-e LOGSPOUT=ignore  \
-e MULTILINE_MATCH=first \
gliderlabs/logspout \
multiline+syslog+tcp://172.17.0.1:5000
@jkqjkq
Copy link
Author

jkqjkq commented Aug 28, 2020

With "tcp", there will be a problem, after removing it, use the default "udp", there is no problem

@lukasa1993
Copy link

same problem with tls

@mauropanzeriitaliaonline
Copy link

mauropanzeriitaliaonline commented Jan 20, 2022

resuscitating this issue for whomever come here searching for a solution to this very problem: thoroughly search for non-printable characters in your logs.
In my case, docker container's emitting the logs, were prepending an ANSI color at the beginning of each line, followed by a date.
so after a lot of trial and error i came up with my specific case:

      - MULTILINE_MATCH=first
      - MULTILINE_PATTERN=^(.\[2m)(\d{4}-\d{2}-\d{2})

.\[2m is the "color" for my case, it could change for you.
try to capture the raw log from the original docker container to catch these hidden chars.

A more general regex to detect ANSI color is:

\x1B\[([0-9]{1,2}(;[0-9]{1,2})?)?[m|K]

(also useful to remove them with a mutate => gsub to remove them in logstash)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants