Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Need custom parser to filter the vcenter logs #2553

Closed
mpaidela opened this issue Aug 9, 2024 · 5 comments
Closed

Need custom parser to filter the vcenter logs #2553

mpaidela opened this issue Aug 9, 2024 · 5 comments
Assignees

Comments

@mpaidela
Copy link

mpaidela commented Aug 9, 2024

What is the sc4s version? 3.27.0

Is there a pcap available? If so, would you prefer to attach it to this issue or send it to Splunk support?

What the vendor name? Vmware

What's the product name? vpshre

If you're requesting support for a new vendor, do you have any preferences regarding the default index and sourcetype for their events?

Do you have syslog documentation or a manual for that device??

Feature Request description:

Do you want to have it for local usage or prepare a github PR?

we have ingested vcenter logs into Splunk using sc4s, but user wants to drop some unwanted logs.

Could someone please assist us in creating a Custom parser?

@cwadhwani-splunk cwadhwani-splunk self-assigned this Aug 13, 2024
@cwadhwani-splunk cwadhwani-splunk added the enhancement New feature or request label Aug 13, 2024
@cwadhwani-splunk
Copy link
Collaborator

Here is what you can refer to:
https://splunk.github.io/splunk-connect-for-syslog/main/sources/#filtering-events-from-output

If you are still facing an issue with creating a parser please create a support ticket so that we can collect all the necessary information like sample logs to further assist you.

@mpaidela
Copy link
Author

Currently, we are receiving over 100 different source types of logs into Splunk, but the user wants the logs below.

Sourcetype: vmware:vclog:vpxd
vmware:vclog:vpxd-main
vclog:applmgmt-audit
vmware:vclog:vmafdd
vmware:vclog:vpxd-svcs-access
vmware:esxlog:vmkernel
vmware:esxlog:hostd
vmware:esxlog:vmauthd

Can you please provide the sample parser for this?

@cwadhwani-splunk
Copy link
Collaborator

cwadhwani-splunk commented Aug 20, 2024

Hi @mpaidela
Based on the details provided I have created a parser that will only ingest the logs with the above sourcetypes. Create a file to /opt/sc4s/local/config/app_parsers/vmware_vsphere_block_sourcetype-postfilter.conf

block parser vmware_vsphere_block_sourcetype-postfilter() {
    channel {
        rewrite(r_set_dest_splunk_null_queue);
   };
};
application vmware_vsphere_block_sourcetype-postfilter[sc4s-postfilter] {
 filter {
        "${fields.sc4s_vendor}" eq "vmware" and
        not (
            match("vmware:vclog:vpxd", value('.splunk.sourcetype'), type(string)) or
            match("vmware:vclog:vpxd-main", value('.splunk.sourcetype'), type(string)) or
            match("vclog:applmgmt-audit", value('.splunk.sourcetype'), type(string)) or
            match("vmware:vclog:vmafdd", value('.splunk.sourcetype'), type(string)) or
            match("vmware:vclog:vpxd-svcs-access", value('.splunk.sourcetype'), type(string)) or
            match("vmware:esxlog:vmkernel", value('.splunk.sourcetype'), type(string)) or
            match("vmware:esxlog:hostd", value('.splunk.sourcetype'), type(string)) or
            match("vmware:esxlog:vmauthd", value('.splunk.sourcetype'), type(string))
        )
    };
    parser { vmware_vsphere_block_sourcetype-postfilter(); };
};

Please make sure to restart your sc4s service after making these changes.

@cwadhwani-splunk cwadhwani-splunk removed the enhancement New feature or request label Aug 20, 2024
@mpaidela
Copy link
Author

Thank you @cwadhwani-splunk

can you please provide the sample parser to filter nix logs with source?

@cwadhwani-splunk
Copy link
Collaborator

Hi @mpaidela

I am closing this GitHub issue as the initial request is fulfilled.
Could you please create a support case for this and we will proceed accordingly.
Note: GitHub issues are for bugs or enhancements.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants