Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

In self-managed elastic, the agent's logs are not visible in kibana #3415

Open
zjk231851 opened this issue Mar 29, 2024 · 5 comments
Open

In self-managed elastic, the agent's logs are not visible in kibana #3415

zjk231851 opened this issue Mar 29, 2024 · 5 comments

Comments

@zjk231851
Copy link

zjk231851 commented Mar 29, 2024 •
edited
Loading

Hello, everyone.
I set up a cluster with three Linux servers using elastic version 8.11.1. The information for each server is as follows: Server 1 ip-10.150.3.12 elasticsearch+kibana, server 2 ip-10.150.3.15 elasticsearch+elastic agent, and server 3 ip 10.150.3.17 elasticsearch+fleetserver. They are self-managed clusters and are configured with TSL, using self-generated certificates. When installing fleetserver and agent, I didn't do any configuration for Elasti-agent.yml, just tar and install. fleetserver and agent are up and running, but in kibana, don't see logs. May I ask where I need to troubleshoot the problem next?
I uploaded my diagnostics log at the end, hoping to help with this problem. Thank you so much!

1
2
3
4
5
6

 sudo ./elastic-agent install --url=https://10.150.3.17:8220 \
>   --fleet-server-es=https://10.150.3.17:9200 \
>   --fleet-server-service-token=AAEAAWVsYXN0aWMvZmxlZXQtc2VydmVyL3Rva2VuLTE3MTE5NTgxNDE1NjE6Zi1LN0hJQVZUSUdlMjJZejJ5QU9YQQ \
>   --fleet-server-policy=5906c780-ec1c-11ee-9cd9-39245a862599 \
>   --fleet-server-es-ca-trusted-fingerprint=BC8F64D445A43597BE161251826AAA1B302C5C76BDE744C479E36C8481E10704 \
>   --certificate-authorities=/home/elastic/downloads/beats/elastic-agent/fleet-server/ca/ca.crt \
>   --fleet-server-cert=/home/elastic/downloads/beats/elastic-agent/fleet-server/ca/fleet-server.crt \
>   --fleet-server-cert-key=/home/elastic/downloads/beats/elastic-agent/fleet-server/ca/fleet-server.key \
>   --fleet-server-port=8220 \
>   --fleet-server-es-insecure
Elastic Agent will be installed at /opt/Elastic/Agent and will run as a service. Do you want to continue? [Y/n]:y
Copying files......................... DONE
Installing service... DONE
Starting service... DONE
Enrolling Elastic Agent with Fleet...................{"log.level":"info","@timestamp":"2024-04-01T15:56:31.046+0800","log.origin":{"file.name":"cmd/enroll_cmd.go","file.line":774},"message":"Waiting for Elastic Agent to start Fleet Server","ecs.version":"1.6.0"}
...............{"log.level":"info","@timestamp":"2024-04-01T15:56:35.050+0800","log.origin":{"file.name":"cmd/enroll_cmd.go","file.line":788},"message":"Fleet Server - Running on policy with Fleet Server integration: 5906c780-ec1c-11ee-9cd9-39245a862599; missing config fleet.agent.id (expected during bootstrap process)","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2024-04-01T15:56:35.276+0800","log.origin":{"file.name":"cmd/enroll_cmd.go","file.line":479},"message":"Starting enrollment to URL: https://10.150.3.17:8220/","ecs.version":"1.6.0"}
....{"log.level":"info","@timestamp":"2024-04-01T15:56:36.315+0800","log.origin":{"file.name":"cmd/enroll_cmd.go","file.line":277},"message":"Successfully triggered restart on running Elastic Agent.","ecs.version":"1.6.0"}
Successfully enrolled the Elastic Agent.
 DONE
Elastic Agent has been successfully installed.


[[email protected] elastic-agent-03ef9d]# ./elastic-agent status
┌─ fleet
│  └─ status: (HEALTHY) Connected
└─ elastic-agent
   └─ status: (HEALTHY) Running
[[email protected] elastic-agent-03ef9d]# ./elastic-agent status --output=json
{
    "info": {
        "id": "2ca3cb94-9bb4-43a8-b4b5-44a312a45de6",
        "version": "8.11.1",
        "commit": "03ef9d8d6a894af61a77d78983a735d263b414ac",
        "build_time": "2023-11-11 13:29:06 +0000 UTC",
        "snapshot": false
    },
    "state": 2,
    "message": "Running",
    "components": [
        {
            "id": "beat/metrics-monitoring",
            "name": "beat/metrics",
            "state": 2,
            "message": "Healthy: communicating with pid '47052'",
            "units": [
                {
                    "unit_id": "beat/metrics-monitoring",
                    "unit_type": 1,
                    "state": 2,
                    "message": "Healthy"
                },
                {
                    "unit_id": "beat/metrics-monitoring-metrics-monitoring-beats",
                    "unit_type": 0,
                    "state": 2,
                    "message": "Healthy"
                }
            ],
            "version_info": {
                "name": "beat-v2-client",
                "version": "8.11.1",
                "meta": {
                    "build_time": "2023-11-10 19:26:08 +0000 UTC",
                    "commit": "19c8672c0a5bf2fb15648c0caf62d985af5a987f"
                }
            }
        },
        {
            "id": "filestream-monitoring",
            "name": "filestream",
            "state": 2,
            "message": "Healthy: communicating with pid '47041'",
            "units": [
                {
                    "unit_id": "filestream-monitoring",
                    "unit_type": 1,
                    "state": 2,
                    "message": "Healthy"
                },
                {
                    "unit_id": "filestream-monitoring-filestream-monitoring-agent",
                    "unit_type": 0,
                    "state": 2,
                    "message": "Healthy"
                }
            ],
            "version_info": {
                "name": "beat-v2-client",
                "version": "8.11.1",
                "meta": {
                    "build_time": "2023-11-10 19:23:39 +0000 UTC",
                    "commit": "19c8672c0a5bf2fb15648c0caf62d985af5a987f"
                }
            }
        },
        {
            "id": "fleet-server-default",
            "name": "fleet-server",
            "state": 2,
            "message": "Healthy: communicating with pid '46959'",
            "units": [
                {
                    "unit_id": "fleet-server-default",
                    "unit_type": 1,
                    "state": 2,
                    "message": "Running on policy with Fleet Server integration: 5906c780-ec1c-11ee-9cd9-39245a862599"
                },
                {
                    "unit_id": "fleet-server-default-fleet-server-fleet_server-0c1e5097-b750-400f-8983-0c1fec8b58c8",
                    "unit_type": 0,
                    "state": 2,
                    "message": "Running on policy with Fleet Server integration: 5906c780-ec1c-11ee-9cd9-39245a862599"
                }
            ],
            "version_info": {
                "name": "fleet-server",
                "version": "8.11.1",
                "meta": {
                    "build_time": "2023-11-09 06:45:02 +0000 UTC",
                    "commit": "8a5688dc"
                }
            }
        },
        {
            "id": "http/metrics-monitoring",
            "name": "http/metrics",
            "state": 2,
            "message": "Healthy: communicating with pid '47063'",
            "units": [
                {
                    "unit_id": "http/metrics-monitoring",
                    "unit_type": 1,
                    "state": 2,
                    "message": "Healthy"
                },
                {
                    "unit_id": "http/metrics-monitoring-metrics-monitoring-agent",
                    "unit_type": 0,
                    "state": 2,
                    "message": "Healthy"
                }
            ],
            "version_info": {
                "name": "beat-v2-client",
                "version": "8.11.1",
                "meta": {
                    "build_time": "2023-11-10 19:26:08 +0000 UTC",
                    "commit": "19c8672c0a5bf2fb15648c0caf62d985af5a987f"
                }
            }
        },
        {
            "id": "log-default",
            "name": "log",
            "state": 2,
            "message": "Healthy: communicating with pid '47013'",
            "units": [
                {
                    "unit_id": "log-default",
                    "unit_type": 1,
                    "state": 2,
                    "message": "Healthy"
                },
                {
                    "unit_id": "log-default-logfile-system-e6016c3e-b207-40b3-a6dc-1f9a3cd1bec1",
                    "unit_type": 0,
                    "state": 2,
                    "message": "Healthy"
                }
            ],
            "version_info": {
                "name": "beat-v2-client",
                "version": "8.11.1",
                "meta": {
                    "build_time": "2023-11-10 19:23:39 +0000 UTC",
                    "commit": "19c8672c0a5bf2fb15648c0caf62d985af5a987f"
                }
            }
        },
        {
            "id": "system/metrics-default",
            "name": "system/metrics",
            "state": 2,
            "message": "Healthy: communicating with pid '47025'",
            "units": [
                {
                    "unit_id": "system/metrics-default",
                    "unit_type": 1,
                    "state": 2,
                    "message": "Healthy"
                },
                {
                    "unit_id": "system/metrics-default-system/metrics-system-e6016c3e-b207-40b3-a6dc-1f9a3cd1bec1",
                    "unit_type": 0,
                    "state": 2,
                    "message": "Healthy"
                }
            ],
            "version_info": {
                "name": "beat-v2-client",
                "version": "8.11.1",
                "meta": {
                    "build_time": "2023-11-10 19:26:08 +0000 UTC",
                    "commit": "19c8672c0a5bf2fb15648c0caf62d985af5a987f"
                }
            }
        }
    ],
    "FleetState": 2,
    "FleetMessage": "Connected"
}



{"log.level":"info","@timestamp":"2024-04-01T07:56:32.352Z","message":"Listening on localhost:8221","component":{"binary":"fleet-server","dataset":"elastic_agent.fleet_server","id":"fleet-server-default","type":"fleet-server"},"log":{"source":"fleet-server-default"},"ecs.version":"1.6.0","service.name":"fleet-server","service.type":"fleet-server","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2024-04-01T07:56:32.352Z","message":"Listening on 0.0.0.0:8220","component":{"binary":"fleet-server","dataset":"elastic_agent.fleet_server","id":"fleet-server-default","type":"fleet-server"},"log":{"source":"fleet-server-default"},"ecs.version":"1.6.0","service.name":"fleet-server","service.type":"fleet-server","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2024-04-01T07:56:32.878Z","message":"Running on policy with Fleet Server integration: 5906c780-ec1c-11ee-9cd9-39245a862599; missing config fleet.agent.id (expected during bootstrap process)","component":{"binary":"fleet-server","dataset":"elastic_agent.fleet_server","id":"fleet-server-default","type":"fleet-server"},"log":{"source":"fleet-server-default"},"service.name":"fleet-server","service.type":"fleet-server","state":"DEGRADED","ecs.version":"1.6.0","ecs.version":"1.6.0"}
{"log.level":"warn","@timestamp":"2024-04-01T07:56:32.878Z","log.origin":{"file.name":"coordinator/coordinator.go","file.line":558},"message":"Unit state changed fleet-server-default (STARTING->DEGRADED): Running on policy with Fleet Server integration: 5906c780-ec1c-11ee-9cd9-39245a862599; missing config fleet.agent.id (expected during bootstrap process)","log":{"source":"elastic-agent"},"component":{"id":"fleet-server-default","state":"HEALTHY"},"unit":{"id":"fleet-server-default","type":"output","state":"DEGRADED","old_state":"STARTING"},"ecs.version":"1.6.0"}
{"log.level":"warn","@timestamp":"2024-04-01T07:56:32.878Z","log.origin":{"file.name":"coordinator/coordinator.go","file.line":558},"message":"Unit state changed fleet-server-default-fleet-server (STARTING->DEGRADED): Running on policy with Fleet Server integration: 5906c780-ec1c-11ee-9cd9-39245a862599; missing config fleet.agent.id (expected during bootstrap process)","log":{"source":"elastic-agent"},"component":{"id":"fleet-server-default","state":"HEALTHY"},"unit":{"id":"fleet-server-default-fleet-server","type":"input","state":"DEGRADED","old_state":"STARTING"},"ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2024-04-01T07:56:35.505Z","message":"Checking enrollment key from database j85c2GqpRw-x2JEQAmEHjQ","component":{"binary":"fleet-server","dataset":"elastic_agent.fleet_server","id":"fleet-server-default","type":"fleet-server"},"log":{"source":"fleet-server-default"},"mod":"enroll","fleet.enroll.apikey.id":"oQQ9f44B6eQfthdqM8OM","ecs.version":"1.6.0","service.name":"fleet-server","http.request.id":"01HTCAHZ9966N7HWFVQZV7ZBD3","server.address":"","service.type":"fleet-server","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2024-04-01T07:56:35.758Z","message":"Found enrollment key b1FROWY0NEI2ZVFmdGhkcU04T006ajg1YzJHcXBSdy14MkpFUUFtRUhqUQ==","component":{"binary":"fleet-server","dataset":"elastic_agent.fleet_server","id":"fleet-server-default","type":"fleet-server"},"log":{"source":"fleet-server-default"},"ecs.version":"1.6.0","service.type":"fleet-server","server.address":"","mod":"enroll","fleet.enroll.apikey.id":"oQQ9f44B6eQfthdqM8OM","service.name":"fleet-server","http.request.id":"01HTCAHZ9966N7HWFVQZV7ZBD3","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2024-04-01T07:56:36.051Z","message":"Elastic Agent successfully enrolled","component":{"binary":"fleet-server","dataset":"elastic_agent.fleet_server","id":"fleet-server-default","type":"fleet-server"},"log":{"source":"fleet-server-default"},"mod":"enroll","fleet.access.apikey.id":"7aGomI4B2evC5-sW_pEv","event.duration":549099726,"service.name":"fleet-server","http.request.id":"01HTCAHZ9966N7HWFVQZV7ZBD3","fleet.enroll.apikey.id":"oQQ9f44B6eQfthdqM8OM","fleet.agent.id":"2ca3cb94-9bb4-43a8-b4b5-44a312a45de6","service.type":"fleet-server","server.address":"","http.response.body.bytes":1134,"ecs.version":"1.6.0","fleet.policy.id":"5906c780-ec1c-11ee-9cd9-39245a862599","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2024-04-01T07:56:36.314Z","log.origin":{"file.name":"cmd/run.go","file.line":302},"message":"reexec shutdown channel triggered","log":{"source":"elastic-agent"},"ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2024-04-01T07:56:36.315Z","message":"On signal","component":{"binary":"fleet-server","dataset":"elastic_agent.fleet_server","id":"fleet-server-default","type":"fleet-server"},"log":{"source":"fleet-server-default"},"service.name":"fleet-server","service.type":"fleet-server","sig":"terminated","ecs.version":"1.6.0","ecs.version":"1.6.0"}


{"log.level":"info","@timestamp":"2024-04-01T07:56:30.817Z","message":"Uptime: 3.097177795s","component":{"binary":"metricbeat","dataset":"elastic_agent.metricbeat","id":"system/metrics-default","type":"system/metrics"},"log":{"source":"system/metrics-default"},"log.logger":"monitoring","log.origin":{"file.line":196,"file.name":"log/log.go"},"service.name":"metricbeat","ecs.version":"1.6.0","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2024-04-01T07:56:30.817Z","message":"Stopping metrics logging.","component":{"binary":"metricbeat","dataset":"elastic_agent.metricbeat","id":"system/metrics-default","type":"system/metrics"},"log":{"source":"system/metrics-default"},"ecs.version":"1.6.0","log.logger":"monitoring","log.origin":{"file.line":163,"file.name":"log/log.go"},"service.name":"metricbeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2024-04-01T07:56:30.817Z","message":"Stats endpoint (/opt/Elastic/Agent/data/tmp/iThI_df0cBKC6YUNGGlKscMkOfz3FBH3.sock) finished: accept unix /opt/Elastic/Agent/data/tmp/iThI_df0cBKC6YUNGGlKscMkOfz3FBH3.sock: use of closed network connection","component":{"binary":"metricbeat","dataset":"elastic_agent.metricbeat","id":"system/metrics-default","type":"system/metrics"},"log":{"source":"system/metrics-default"},"log.origin":{"file.line":73,"file.name":"api/server.go"},"service.name":"metricbeat","ecs.version":"1.6.0","log.logger":"api","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2024-04-01T07:56:30.817Z","message":"metricbeat stopped.","component":{"binary":"metricbeat","dataset":"elastic_agent.metricbeat","id":"system/metrics-default","type":"system/metrics"},"log":{"source":"system/metrics-default"},"log.origin":{"file.line":531,"file.name":"instance/beat.go"},"service.name":"metricbeat","ecs.version":"1.6.0","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2024-04-01T07:56:30.983Z","message":"add_cloud_metadata: hosting provider type not detected.","component":{"binary":"metricbeat","dataset":"elastic_agent.metricbeat","id":"http/metrics-monitoring","type":"http/metrics"},"log":{"source":"http/metrics-monitoring"},"log.logger":"add_cloud_metadata","log.origin":{"file.line":100,"file.name":"add_cloud_metadata/add_cloud_metadata.go"},"service.name":"metricbeat","ecs.version":"1.6.0","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2024-04-01T07:56:30.985Z","message":"Total metrics","component":{"binary":"metricbeat","dataset":"elastic_agent.metricbeat","id":"http/metrics-monitoring","type":"http/metrics"},"log":{"source":"http/metrics-monitoring"},"log.logger":"monitoring","log.origin":{"file.line":195,"file.name":"log/log.go"},"service.name":"metricbeat","monitoring":{"ecs.version":"1.6.0","metrics":{"beat":{"cpu":{"system":{"ticks":80,"time":{"ms":80}},"total":{"ticks":210,"time":{"ms":210},"value":210},"user":{"ticks":130,"time":{"ms":130}}},"handles":{"limit":{"hard":65536,"soft":65536},"open":10},"info":{"ephemeral_id":"a7add003-1f25-4a1e-97bd-84a95e33eb20","name":"metricbeat","uptime":{"ms":3090},"version":"8.11.1"},"memstats":{"gc_next":47007672,"memory_alloc":37410120,"memory_sys":58586376,"memory_total":52203896,"rss":90165248},"runtime":{"goroutines":29}},"libbeat":{"config":{"module":{"running":2,"starts":2,"stops":0},"reloads":0,"scans":0},"output":{"batches":{"split":0},"events":{"acked":0,"active":0,"batches":0,"dropped":0,"duplicates":0,"failed":0,"toomany":0,"total":0},"read":{"bytes":0,"errors":0},"type":"elasticsearch","write":{"bytes":0,"errors":0}},"pipeline":{"clients":0,"events":{"active":1,"dropped":0,"failed":1,"filtered":0,"published":1,"retry":0,"total":2},"queue":{"acked":0,"max_events":4096}}},"processor":{"add_host_metadata":{"fqdn_lookup_failed":0}},"system":{"cpu":{"cores":8},"load":{"1":0.67,"15":0.63,"5":0.62,"norm":{"1":0.0838,"15":0.0788,"5":0.0775}}}}},"ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2024-04-01T07:56:30.985Z","message":"Uptime: 3.092760384s","component":{"binary":"metricbeat","dataset":"elastic_agent.metricbeat","id":"http/metrics-monitoring","type":"http/metrics"},"log":{"source":"http/metrics-monitoring"},"ecs.version":"1.6.0","log.logger":"monitoring","log.origin":{"file.line":196,"file.name":"log/log.go"},"service.name":"metricbeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2024-04-01T07:56:30.985Z","message":"Stopping metrics logging.","component":{"binary":"metricbeat","dataset":"elastic_agent.metricbeat","id":"http/metrics-monitoring","type":"http/metrics"},"log":{"source":"http/metrics-monitoring"},"log.logger":"monitoring","log.origin":{"file.line":163,"file.name":"log/log.go"},"service.name":"metricbeat","ecs.version":"1.6.0","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2024-04-01T07:56:30.985Z","message":"Stats endpoint (/opt/Elastic/Agent/data/tmp/akSPbdqgaHaTY0_J01-dsfYK6JpMz2zn.sock) finished: accept unix /opt/Elastic/Agent/data/tmp/akSPbdqgaHaTY0_J01-dsfYK6JpMz2zn.sock: use of closed network connection","component":{"binary":"metricbeat","dataset":"elastic_agent.metricbeat","id":"http/metrics-monitoring","type":"http/metrics"},"log":{"source":"http/metrics-monitoring"},"log.logger":"api","log.origin":{"file.line":73,"file.name":"api/server.go"},"service.name":"metricbeat","ecs.version":"1.6.0","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2024-04-01T07:56:30.985Z","message":"metricbeat stopped.","component":{"binary":"metricbeat","dataset":"elastic_agent.metricbeat","id":"http/metrics-monitoring","type":"http/metrics"},"log":{"source":"http/metrics-monitoring"},"service.name":"metricbeat","ecs.version":"1.6.0","log.origin":{"file.line":531,"file.name":"instance/beat.go"},"ecs.version":"1.6.0"}

elastic-agent-diagnostics-2024-04-01T08-21-38Z-00.zip
@zjk231851
Copy link
Author

@michel-laterman I upgraded to 8.14.1 but the problem persists. Can you help me?

@cmacknz
Copy link
Member

cmacknz commented Jul 22, 2024

The data streams have data in them, what logs are you expecting to see that are missing, and where you are looking?

This is probably better suited to being a post in https://discuss.elastic.co/c/elastic-stack/elastic-agent

@zjk231851
Copy link
Author

zjk231851 commented Jul 23, 2024 •
edited
Loading

The data streams have data in them, what logs are you expecting to see that are missing, and where you are looking?

This is probably better suited to being a post in https://discuss.elastic.co/c/elastic-stack/elastic-agent

I want to see the fleet-server and agent logs in [Monitor Elastic Agents | Fleet and Elastic Agent Guide [8.14] | Elastic 3](url) But now it's empty
1721698272386

@michel-laterman
Copy link
Contributor

michel-laterman commented Jul 23, 2024 •
edited
Loading

I took a quick look at the diagnostics you provided, nothing in the configs stuck out to me.
The filestream-monitoring is present, and the inputs shows the paths to the agent logs.

The logs also didn't have anything in the way of errors (nothing to indicate that it failed to connect to ES), the only thing i wound was a warning and inputs loading messages from filestream-monitoring, but i'm not sure if this is expected behaviour.

{"log.level":"warn","@timestamp":"2024-04-01T07:56:37.548Z","message":"Filebeat is unable to load the ingest pipelines for the configured modules because the Elasticsearch output is not configured/enabled. If you have already loaded the ingest pipelines or are using Logstash pipelines, you can ignore this warning.","component":{"binary":"filebeat","dataset":"elastic_agent.filebeat","id":"filestream-monitoring","type":"filestream"},"log":{"source":"filestream-monitoring"},"service.name":"filebeat","ecs.version":"1.6.0","log.origin":{"file.line":331,"file.name":"beater/filebeat.go"},"ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2024-04-01T07:56:37.548Z","message":"creating new InputManager","component":{"binary":"filebeat","dataset":"elastic_agent.filebeat","id":"filestream-monitoring","type":"filestream"},"log":{"source":"filestream-monitoring"},"log.logger":"input","log.origin":{"file.line":55,"file.name":"shipper/input.go"},"service.name":"filebeat","ecs.version":"1.6.0","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2024-04-01T07:56:37.548Z","message":"States Loaded from registrar: 0","component":{"binary":"filebeat","dataset":"elastic_agent.filebeat","id":"filestream-monitoring","type":"filestream"},"log":{"source":"filestream-monitoring"},"log.logger":"registrar","log.origin":{"file.line":107,"file.name":"registrar/registrar.go"},"service.name":"filebeat","ecs.version":"1.6.0","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2024-04-01T07:56:37.548Z","message":"Loading Inputs: 0","component":{"binary":"filebeat","dataset":"elastic_agent.filebeat","id":"filestream-monitoring","type":"filestream"},"log":{"source":"filestream-monitoring"},"service.name":"filebeat","ecs.version":"1.6.0","log.logger":"crawler","log.origin":{"file.line":71,"file.name":"beater/crawler.go"},"ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2024-04-01T07:56:37.548Z","message":"Loading and starting Inputs completed. Enabled inputs: 0","component":{"binary":"filebeat","dataset":"elastic_agent.filebeat","id":"filestream-monitoring","type":"filestream"},"log":{"source":"filestream-monitoring"},"log.logger":"crawler","log.origin":{"file.line":106,"file.name":"beater/crawler.go"},"service.name":"filebeat","ecs.version":"1.6.0","ecs.version":"1.6.0"}

Do any logs show up in the discover view? Do the indcies/datastreams show up ion the management views?

@zjk231851
Copy link
Author

I took a quick look at the diagnostics you provided, nothing in the configs stuck out to me. The filestream-monitoring is present, and the inputs shows the paths to the agent logs.

The logs also didn't have anything in the way of errors (nothing to indicate that it failed to connect to ES), the only thing i wound was a warning and inputs loading messages from filestream-monitoring, but i'm not sure if this is expected behaviour.

{"log.level":"warn","@timestamp":"2024-04-01T07:56:37.548Z","message":"Filebeat is unable to load the ingest pipelines for the configured modules because the Elasticsearch output is not configured/enabled. If you have already loaded the ingest pipelines or are using Logstash pipelines, you can ignore this warning.","component":{"binary":"filebeat","dataset":"elastic_agent.filebeat","id":"filestream-monitoring","type":"filestream"},"log":{"source":"filestream-monitoring"},"service.name":"filebeat","ecs.version":"1.6.0","log.origin":{"file.line":331,"file.name":"beater/filebeat.go"},"ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2024-04-01T07:56:37.548Z","message":"creating new InputManager","component":{"binary":"filebeat","dataset":"elastic_agent.filebeat","id":"filestream-monitoring","type":"filestream"},"log":{"source":"filestream-monitoring"},"log.logger":"input","log.origin":{"file.line":55,"file.name":"shipper/input.go"},"service.name":"filebeat","ecs.version":"1.6.0","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2024-04-01T07:56:37.548Z","message":"States Loaded from registrar: 0","component":{"binary":"filebeat","dataset":"elastic_agent.filebeat","id":"filestream-monitoring","type":"filestream"},"log":{"source":"filestream-monitoring"},"log.logger":"registrar","log.origin":{"file.line":107,"file.name":"registrar/registrar.go"},"service.name":"filebeat","ecs.version":"1.6.0","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2024-04-01T07:56:37.548Z","message":"Loading Inputs: 0","component":{"binary":"filebeat","dataset":"elastic_agent.filebeat","id":"filestream-monitoring","type":"filestream"},"log":{"source":"filestream-monitoring"},"service.name":"filebeat","ecs.version":"1.6.0","log.logger":"crawler","log.origin":{"file.line":71,"file.name":"beater/crawler.go"},"ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2024-04-01T07:56:37.548Z","message":"Loading and starting Inputs completed. Enabled inputs: 0","component":{"binary":"filebeat","dataset":"elastic_agent.filebeat","id":"filestream-monitoring","type":"filestream"},"log":{"source":"filestream-monitoring"},"log.logger":"crawler","log.origin":{"file.line":106,"file.name":"beater/crawler.go"},"service.name":"filebeat","ecs.version":"1.6.0","ecs.version":"1.6.0"}

Do any logs show up in the discover view? Do the indcies/datastreams show up ion the management views?

discover view and management views It works
1721809369807
1721809304458

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@cmacknz @zjk231851 @michel-laterman