You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: packages/aws_vpcflow_otel/docs/README.md
+55Lines changed: 55 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -21,6 +21,61 @@ From the AWS side, to collect VPC Flow logs, you need:
21
21
For step-by-step instructions on how to set up an EDOT Cloud Forwarder for AWS, see the
22
22
[EDOT Cloud Forwarder for AWS](https://www.elastic.co/docs/reference/opentelemetry/edot-cloud-forwarder/aws) guide.
23
23
24
+
## Alternative setup using [AWS S3 receiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/awss3receiver)
25
+
26
+
Alternative setup allows to bypass "Serverless only" limitation.
27
+
28
+
### Prerequisites
29
+
30
+
- A Virtual Private Cloud (VPC)
31
+
- An S3 bucket for storing flow logs
32
+
- A flow log configured with the S3 bucket as the destination
33
+
- An SQS queue receiving notifications on object creation in the S3 bucket
34
+
-`awss3receiver` and `awslogsencodingextension`
35
+
36
+
### Config example
37
+
38
+
For details on configuration refer to corresponding docs: [awss3receiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/awss3receiver), [awslogsencodingextension](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/extension/encoding/awslogsencodingextension), [elasticsearchexporter](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/exporter/elasticsearchexporter)
39
+
40
+
```yaml
41
+
extensions:
42
+
awslogs_encoding/vpcflow:
43
+
format: vpcflow
44
+
vpcflow:
45
+
file_format: plain-text
46
+
47
+
receivers:
48
+
awss3:
49
+
sqs:
50
+
queue_url: "<sqs-url>"
51
+
region: "<region>"
52
+
s3downloader:
53
+
region: "<region>"
54
+
s3_bucket: '<bucket_name>'
55
+
s3_prefix: 'AWSLogs/<account-id>'
56
+
encodings:
57
+
- extension: awslogs_encoding/vpcflow
58
+
59
+
exporters:
60
+
debug:
61
+
verbosity: detailed
62
+
elasticsearch/otel:
63
+
endpoints: https://<host>:<port>
64
+
user: elastic
65
+
password: <password>
66
+
mapping:
67
+
mode: otel
68
+
metrics_dynamic_index:
69
+
enabled: true
70
+
71
+
service:
72
+
extensions: [awslogs_encoding/vpcflow]
73
+
pipelines:
74
+
logs:
75
+
exporters: [debug, elasticsearch/otel]
76
+
receivers: [awss3]
77
+
```
78
+
24
79
## Logs Reference
25
80
26
81
For a complete list of all available logs and their detailed descriptions, refer to the [OpenTelemetry AWS Logs encoding extension](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/extension/encoding/awslogsencodingextension#vpc-flow-log-record-fields)
0 commit comments