Skip to content

Commit

Permalink
新增pod日志采集(EFK方案)
Browse files Browse the repository at this point in the history
  • Loading branch information
cuiliang0302 committed Oct 10, 2023
1 parent 76a14b8 commit a24aa2b
Show file tree
Hide file tree
Showing 38 changed files with 3,391 additions and 1 deletion.
6 changes: 5 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,4 +22,8 @@

* pod日志采集(Fluent Bit方案)

[pod日志采集(Fluent Bit方案)-崔亮的博客 (cuiliangblog.cn)](https://www.cuiliangblog.cn/detail/article/75)
[pod日志采集(Fluent Bit方案)-崔亮的博客 (cuiliangblog.cn)](https://www.cuiliangblog.cn/detail/article/75)

* pod日志采集(EFK方案)

[pod日志采集(EFK方案)-崔亮的博客 (cuiliangblog.cn)](https://www.cuiliangblog.cn/detail/article/76)
23 changes: 23 additions & 0 deletions pod日志采集(EFK方案)/fluent-bit/.helmignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
# Patterns to ignore when building packages.
# This supports shell glob matching, relative path matching, and
# negation (prefixed with !). Only one pattern per line.
.DS_Store
# Common VCS dirs
.git/
.gitignore
.bzr/
.bzrignore
.hg/
.hgignore
.svn/
# Common backup files
*.swp
*.bak
*.tmp
*.orig
*~
# Various IDEs
.project
.idea/
*.tmproj
.vscode/
27 changes: 27 additions & 0 deletions pod日志采集(EFK方案)/fluent-bit/Chart.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
annotations:
artifacthub.io/changes: |
- kind: - kind: changed
description: "Updated Fluent Bit OCI image to v2.1.9."
apiVersion: v1
appVersion: 2.1.9
description: Fast and lightweight log processor and forwarder or Linux, OSX and BSD
family operating systems.
home: https://fluentbit.io/
icon: https://raw.githubusercontent.com/cncf/artwork/master/projects/fluentd/fluentbit/icon/fluentbit-icon-color.svg
keywords:
- logging
- fluent-bit
- fluentd
maintainers:
- email: [email protected]
name: edsiper
- email: [email protected]
name: naseemkullah
- email: [email protected]
name: Towmeykaw
- email: [email protected]
name: stevehipwell
name: fluent-bit
sources:
- https://github.com/fluent/fluent-bit/
version: 0.38.0
57 changes: 57 additions & 0 deletions pod日志采集(EFK方案)/fluent-bit/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
# Fluent Bit Helm chart

[Fluent Bit](https://fluentbit.io) is a fast and lightweight log processor and forwarder or Linux, OSX and BSD family operating systems.

## Installation

To add the `fluent` helm repo, run:

```sh
helm repo add fluent https://fluent.github.io/helm-charts
```

To install a release named `fluent-bit`, run:

```sh
helm install fluent-bit fluent/fluent-bit
```

## Chart values

```sh
helm show values fluent/fluent-bit
```

## Using Lua scripts
Fluent Bit allows us to build filter to modify the incoming records using custom [Lua scripts.](https://docs.fluentbit.io/manual/pipeline/filters/lua)

### How to use Lua scripts with this Chart

First, you should add your Lua scripts to `luaScripts` in values.yaml, for example:

```yaml
luaScripts:
filter_example.lua: |
function filter_name(tag, timestamp, record)
-- put your lua code here.
end
```
After that, the Lua scripts will be ready to be used as filters. So next step is to add your Fluent bit [filter](https://docs.fluentbit.io/manual/concepts/data-pipeline/filter) to `config.filters` in values.yaml, for example:

```yaml
config:
filters: |
[FILTER]
Name lua
Match <your-tag>
script /fluent-bit/scripts/filter_example.lua
call filter_name
```
Under the hood, the chart will:
- Create a configmap using `luaScripts`.
- Add a volumeMounts for each Lua scripts using the path `/fluent-bit/scripts/<script>`.
- Add the Lua script's configmap as volume to the pod.

### Note
Remember to set the `script` attribute in the filter using `/fluent-bit/scripts/`, otherwise the file will not be found by fluent bit.
4 changes: 4 additions & 0 deletions pod日志采集(EFK方案)/fluent-bit/ci/ci-values.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
logLevel: debug

dashboards:
enabled: true
Loading

0 comments on commit a24aa2b

Please sign in to comment.