From cd45857243db58189acdc4541b1e05052e871df5 Mon Sep 17 00:00:00 2001 From: Ueli Graf Date: Tue, 1 Oct 2024 09:42:41 +0200 Subject: [PATCH 1/4] out_influxdb: add documentation and example use of strip_prefix Signed-off-by: Ueli Graf --- pipeline/outputs/influxdb.md | 35 +++++++++++++++++++++++++++++++++++ 1 file changed, 35 insertions(+) diff --git a/pipeline/outputs/influxdb.md b/pipeline/outputs/influxdb.md index 2b59703f4..9854318fa 100644 --- a/pipeline/outputs/influxdb.md +++ b/pipeline/outputs/influxdb.md @@ -12,6 +12,7 @@ The **influxdb** output plugin, allows to flush your records into a [InfluxDB](h | Bucket | InfluxDB bucket name where records will be inserted - if specified, `database` is ignored and v2 of API is used | | | Org | InfluxDB organization name where the bucket is \(v2 only\) | fluent | | Sequence\_Tag | The name of the tag whose value is incremented for the consecutive simultaneous events. | \_seq | +| Strip\_Prefix | String to be stripped from the front of _tag_ when writing influxdb measurement names | | | HTTP\_User | Optional username for HTTP Basic Authentication | | | HTTP\_Passwd | Password for user defined in HTTP\_User | | | HTTP\_Token | Authentication token used with InfluDB v2 - if specified, both HTTP\_User and HTTP\_Passwd are ignored | | @@ -109,6 +110,40 @@ Basic example of `Tags_List_Key` usage: Tag_Keys level status ``` +### Prefix stripping + +When collecting data from many inputs into many buckets, it can be helpful to remove a common prefix using _Strip_prefix_. + +```python +[INPUT] + Name cpu + Tag cpu.one + +[INPUT] + Name cpu + Tag cpu.two + +[INPUT] + Name cpu + Tag gpu.one + +[INPUT] + Name cpu + Tag gpu.two + +[OUTPUT] + Name influxdb + Match cpu* + Bucket cpubucket + Strip_prefix cpu. + +[OUTPUT] + Name influxdb + Match gpu* + Bucket gpubucket + Strip_prefix gpu. +``` + ### Testing Before to start Fluent Bit, make sure the target database exists on InfluxDB, using the above example, we will insert the data into a _fluentbit_ database. From 4bff80f43b4049062a2a506989bbf72446154f28 Mon Sep 17 00:00:00 2001 From: Ueli Graf Date: Tue, 1 Oct 2024 09:54:28 +0200 Subject: [PATCH 2/4] Add comment about bucket versus measurement naming Signed-off-by: Ueli Graf --- pipeline/outputs/influxdb.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/pipeline/outputs/influxdb.md b/pipeline/outputs/influxdb.md index 9854318fa..24171c018 100644 --- a/pipeline/outputs/influxdb.md +++ b/pipeline/outputs/influxdb.md @@ -144,6 +144,8 @@ When collecting data from many inputs into many buckets, it can be helpful to re Strip_prefix gpu. ``` +This will result in the buckets _cpubucket_, _gpubucket_ each containing two measurement streams named _one_, _two_. Without prefix stripping, the measurement names would be _cpu.one_, _cpu.two_ (stored in _cpubucket_) and _gpu.one_, _gpu.two_ (stored in _gpubucket_). + ### Testing Before to start Fluent Bit, make sure the target database exists on InfluxDB, using the above example, we will insert the data into a _fluentbit_ database. From ce52150ee615841669c2c99b075a2e3e303b5bb0 Mon Sep 17 00:00:00 2001 From: Lynette Miles <6818907+esmerel@users.noreply.github.com> Date: Wed, 22 Oct 2025 14:18:43 -0700 Subject: [PATCH 3/4] Apply suggestions from code review Signed-off-by: Lynette Miles <6818907+esmerel@users.noreply.github.com> --- pipeline/outputs/influxdb.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/pipeline/outputs/influxdb.md b/pipeline/outputs/influxdb.md index 8b2269f33..bc413939f 100644 --- a/pipeline/outputs/influxdb.md +++ b/pipeline/outputs/influxdb.md @@ -14,7 +14,7 @@ This plugin supports the following parameters: | `Bucket` | InfluxDB bucket name where records will be inserted. If specified, `database` is ignored and v2 of API is used. | _none_ | | `Org` | InfluxDB organization name where the bucket is (v2 only). | `fluent` | | `Sequence_Tag` | The name of the tag whose value is incremented for the consecutive simultaneous events. | `_seq` | -| `Strip_Prefix` | String to be stripped from the front of `tag` when writing influxdb measurement names. | _none_ | +| `Strip_Prefix` | String to be stripped from the front of `tag` when writing InfluxDB measurement names. | _none_ | | `HTTP_User` | Optional username for HTTP Basic Authentication. | _none_ | | `HTTP_Passwd` | Password for user defined in `HTTP_User`. | _none_ | | `HTTP_Token` | Authentication token used with InfluxDB v2. If specified, both `HTTP_User` and `HTTP_Passwd` are ignored. | _none_ | @@ -226,7 +226,7 @@ When collecting data from many inputs into many buckets, it can be helpful to re Strip_prefix gpu. ``` -This will result in the buckets `cpubucket` and `gpubucket` each containing two measurement streams named `one` and `two`. Without prefix stripping, the measurement names would be `cpu.one` and `cpu.two (stored in `cpubucket`), and `gpu.one`, `gpu.two` (stored in `gpubucket`). +This will result in the buckets `cpubucket` and `gpubucket` each containing two measurement streams named `one` and `two`. Without prefix stripping, the measurement names would be `cpu.one` and `cpu.two` (stored in `cpubucket`), and `gpu.one`, `gpu.two` (stored in `gpubucket`). ### Testing From f8865b49d48fd7308f5d1d4ec3dc2b50096003ed Mon Sep 17 00:00:00 2001 From: Ueli Graf Date: Sun, 9 Nov 2025 14:25:38 +0100 Subject: [PATCH 4/4] Add yaml config example, clarify behaviour of config option Signed-off-by: Ueli Graf --- pipeline/outputs/influxdb.md | 42 ++++++++++++++++++++++++++++++++++-- 1 file changed, 40 insertions(+), 2 deletions(-) diff --git a/pipeline/outputs/influxdb.md b/pipeline/outputs/influxdb.md index bc413939f..46ae0aa3d 100644 --- a/pipeline/outputs/influxdb.md +++ b/pipeline/outputs/influxdb.md @@ -14,7 +14,7 @@ This plugin supports the following parameters: | `Bucket` | InfluxDB bucket name where records will be inserted. If specified, `database` is ignored and v2 of API is used. | _none_ | | `Org` | InfluxDB organization name where the bucket is (v2 only). | `fluent` | | `Sequence_Tag` | The name of the tag whose value is incremented for the consecutive simultaneous events. | `_seq` | -| `Strip_Prefix` | String to be stripped from the front of `tag` when writing InfluxDB measurement names. | _none_ | +| `Strip_Prefix` | String prefix to be removed from the front of `tag` when writing InfluxDB measurement names. `Strip_Prefix` is removed only if it matches exactly at the beginning of and is strictly shorther than `tag` (`tag` length will not be reduced to zero if `Strip_Prefix` and `tag` are identical strings). `Strip_Prefix` defaults to empty string, which does nothing. | _none_ | | `HTTP_User` | Optional username for HTTP Basic Authentication. | _none_ | | `HTTP_Passwd` | Password for user defined in `HTTP_User`. | _none_ | | `HTTP_Token` | Authentication token used with InfluxDB v2. If specified, both `HTTP_User` and `HTTP_Passwd` are ignored. | _none_ | @@ -196,7 +196,41 @@ pipeline: When collecting data from many inputs into many buckets, it can be helpful to remove a common prefix using `Strip_prefix`. -```python +{% tabs %} +{% tab title="fluent-bit.yaml" %} + +```yaml +pipeline: + inputs: + - name: cpu + tag: cpu.one + - name: cpu + tag: cpu.two + - name: cpu + tag: gpu.one + - name: cpu + tag: gpu.two + + + outputs: + - name: influxdb + match: 'cpu*' + host: 127.0.0.1 + port: 8086 + bucket: cpubucket + strip_prefix: cpu. + - name: influxdb + match: 'gpu*' + host: 127.0.0.1 + port: 8086 + bucket: gpubucket + strip_prefix: gpu. +``` + +{% endtab %} +{% tab title="fluent-bit.conf" %} + +```text [INPUT] Name cpu Tag cpu.one @@ -226,6 +260,10 @@ When collecting data from many inputs into many buckets, it can be helpful to re Strip_prefix gpu. ``` +{% endtab %} +{% endtabs %} + + This will result in the buckets `cpubucket` and `gpubucket` each containing two measurement streams named `one` and `two`. Without prefix stripping, the measurement names would be `cpu.one` and `cpu.two` (stored in `cpubucket`), and `gpu.one`, `gpu.two` (stored in `gpubucket`).