Skip to content

Commit b76587b

Browse files
add similar section to aws_vpcflow_otel
1 parent 60fb2d3 commit b76587b

File tree

3 files changed

+61
-1
lines changed

3 files changed

+61
-1
lines changed

packages/aws_vpcflow_otel/changelog.yml

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,9 @@
11
# newer versions go on top
2+
- version: "0.1.1"
3+
changes:
4+
- description: Add "Alternative setup using awss3receiver" section to README
5+
type: enhancement
6+
link: https://github.com/elastic/integrations/pull/15844
27
- version: "0.1.0"
38
changes:
49
- description: Initial draft of the AWS VPC Flow Logs OpenTelemetry Assets Package

packages/aws_vpcflow_otel/docs/README.md

Lines changed: 55 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,61 @@ From the AWS side, to collect VPC Flow logs, you need:
2121
For step-by-step instructions on how to set up an EDOT Cloud Forwarder for AWS, see the
2222
[EDOT Cloud Forwarder for AWS](https://www.elastic.co/docs/reference/opentelemetry/edot-cloud-forwarder/aws) guide.
2323

24+
## Alternative setup using [AWS S3 receiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/awss3receiver)
25+
26+
Alternative setup allows to bypass "Serverless only" limitation.
27+
28+
### Prerequisites
29+
30+
- A Virtual Private Cloud (VPC)
31+
- An S3 bucket for storing flow logs
32+
- A flow log configured with the S3 bucket as the destination
33+
- An SQS queue receiving notifications on object creation in the S3 bucket
34+
- `awss3receiver` and `awslogsencodingextension`
35+
36+
### Config example
37+
38+
For details on configuration refer to corresponding docs: [awss3receiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/awss3receiver), [awslogsencodingextension](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/extension/encoding/awslogsencodingextension), [elasticsearchexporter](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/exporter/elasticsearchexporter)
39+
40+
```yaml
41+
extensions:
42+
awslogs_encoding/vpcflow:
43+
format: vpcflow
44+
vpcflow:
45+
file_format: plain-text
46+
47+
receivers:
48+
awss3:
49+
sqs:
50+
queue_url: "<sqs-url>"
51+
region: "<region>"
52+
s3downloader:
53+
region: "<region>"
54+
s3_bucket: '<bucket_name>'
55+
s3_prefix: 'AWSLogs/<account-id>'
56+
encodings:
57+
- extension: awslogs_encoding/vpcflow
58+
59+
exporters:
60+
debug:
61+
verbosity: detailed
62+
elasticsearch/otel:
63+
endpoints: https://<host>:<port>
64+
user: elastic
65+
password: <password>
66+
mapping:
67+
mode: otel
68+
metrics_dynamic_index:
69+
enabled: true
70+
71+
service:
72+
extensions: [awslogs_encoding/vpcflow]
73+
pipelines:
74+
logs:
75+
exporters: [debug, elasticsearch/otel]
76+
receivers: [awss3]
77+
```
78+
2479
## Logs Reference
2580
2681
For a complete list of all available logs and their detailed descriptions, refer to the [OpenTelemetry AWS Logs encoding extension](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/extension/encoding/awslogsencodingextension#vpc-flow-log-record-fields)

packages/aws_vpcflow_otel/manifest.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
format_version: 3.5.0
22
name: aws_vpcflow_otel
33
title: "AWS VPC Flow Logs OpenTelemetry Assets"
4-
version: 0.1.0
4+
version: 0.1.1
55
source:
66
license: "Elastic-2.0"
77
description: "AWS VPC Flow Logs OpenTelemetry Assets"

0 commit comments

Comments
 (0)