Skip to content

[QUESTION]:ilogtail采集json日志时,ES无法识别timestamp字段 #1136

Closed Answered by shalousun
opshongwu asked this question in Help
Discussion options

You must be logged in to vote

日志采集组件在采集日志时有一个采集时间(和业务日志时间没关系,采集延迟时采集时间晚于业务日志里的时间),filebeat在推送采集日志时采集时间字段是@timestamp。es也正好需要@timestamp。其实也没有什么特殊意义,只是因为两者属于一个生态的,也就是一种约定。

对于ilogtail也有一个采集时间的字段,叫做time,time和业务日志时间也没有关系,ilogtail如果输入到es希望添加@timestamp字段,则有两个思路:

  1. 从自己的日志里提取时间然后重命名为@timestamp,这个主要根据自己的日志情况选择合适的ilogtail processor插件去处理。可以看https://ilogtail.gitbook.io/ilogtail-docs/data-pipeline/processor

  2. 直接重命名ilogtail的采集字段为@timestamp

Convert:
      ProtocolFieldsRename:
        time: '@timestamp'

日志延迟的时间第二种不推荐,异步写入的日志也不推荐第二种方式。在es中查询会混乱。

Replies: 1 comment

Comment options

You must be logged in to vote
0 replies
Answer selected by linrunqi08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Category
Help
Labels
question Further information is requested
2 participants
Converted from issue

This discussion was converted from issue #1134 on September 20, 2023 02:12.