[aws_elb_otel] [aws_vpcflow_otel] [docs] add section "alternative setup using awss3receiver"

packages/aws_elb_otel/changelog.yml

@@ -1,4 +1,9 @@
 # newer versions go on top
+- version: "0.1.1"
+  changes:
+    - description: Add "Alternative setup using awss3receiver" section to README
+      type: enhancement
+      link: https://github.com/elastic/integrations/pull/15844
 - version: "0.1.0"
   changes:
     - description: Initial draft of the AWS ELB Access Logs OpenTelemetry Assets package

packages/aws_elb_otel/docs/README.md

@@ -23,6 +23,58 @@ The EDOT Cloud Forwarder for AWS supports collecting logs from:
 
 For the full documentation on how to set up the EDOT Cloud Forwarder, follow this link: [EDOT Cloud Forwarder for AWS](https://www.elastic.co/docs/reference/opentelemetry/edot-cloud-forwarder/aws).
 
+## Alternative setup using [AWS S3 receiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/awss3receiver)
+
+The alternative setup allows you to bypass the "Serverless only" limitation.
+
+### Prerequisites
+
+- An S3 bucket for storing the logs
+- A load balancer configured to export access logs to the S3 bucket. Check the [official AWS docs](https://docs.aws.amazon.com/elasticloadbalancing/latest/application/enable-access-logging.html) on how to export access logs
+- An SQS queue receiving notifications on object creation in the S3 bucket
+- `awss3receiver` and `awslogsencodingextension`
+
+### Configuration example
+
+For details on configuration refer to the following documentation: [awss3receiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/awss3receiver#aws-s3-receiver), [awslogsencodingextension](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/extension/encoding/awslogsencodingextension#aws-logs-encoding-extension), [elasticsearchexporter](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/exporter/elasticsearchexporter#configuration-options)
+
+```yaml
+extensions:
+  awslogs_encoding/elbaccess:
+    format: elbaccess
+
+receivers:
+  awss3:
+    sqs:
+      queue_url: "<sqs-url>"
+      region: "<region>"
+    s3downloader:
+      region: "<region>"
+      s3_bucket: '<bucket_name>'
+      s3_prefix: 'AWSLogs/<account-id>'
+    encodings:
+      - extension: awslogs_encoding/elbaccess
+
+exporters:
+  debug:
+    verbosity: detailed
+  elasticsearch/otel:
+    endpoints: https://<host>:<port>
+    user: elastic
+    password: <password>
+    mapping:
+      mode: otel
+    metrics_dynamic_index:
+      enabled: true
+
+service:
+  extensions: [awslogs_encoding/elbaccess]
+  pipelines:
+    logs:
+      exporters: [debug, elasticsearch/otel]
+      receivers: [awss3]
+```
+
 ## Logs reference
 
 ### AWS ELB access logs

packages/aws_elb_otel/manifest.yml

@@ -1,7 +1,7 @@
 format_version: 3.5.0
 name: aws_elb_otel
 title: "AWS ELB OpenTelemetry Assets"
-version: 0.1.0
+version: 0.1.1
 source:
   license: "Elastic-2.0"
 description: "AWS ELB logs for OpenTelemetry Collector"

packages/aws_vpcflow_otel/changelog.yml

@@ -1,4 +1,9 @@
 # newer versions go on top
+- version: "0.1.1"
+  changes:
+    - description: Add "Alternative setup using awss3receiver" section to README
+      type: enhancement
+      link: https://github.com/elastic/integrations/pull/15844
 - version: "0.1.0"
   changes:
     - description: Initial draft of the AWS VPC Flow Logs OpenTelemetry Assets Package

packages/aws_vpcflow_otel/docs/README.md

@@ -21,6 +21,61 @@ From the AWS side, to collect VPC Flow logs, you need:
 For step-by-step instructions on how to set up an EDOT Cloud Forwarder for AWS, see the
 [EDOT Cloud Forwarder for AWS](https://www.elastic.co/docs/reference/opentelemetry/edot-cloud-forwarder/aws) guide.
 
+## Alternative setup using [AWS S3 receiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/awss3receiver)
+
+The alternative setup allows you to bypass the "Serverless only" limitation.
+
+### Prerequisites
+
+- A Virtual Private Cloud (VPC)
+- An S3 bucket for storing flow logs
+- A flow log configured with the S3 bucket as the destination
+- An SQS queue receiving notifications on object creation in the S3 bucket
+- `awss3receiver` and `awslogsencodingextension`
+
+### Configuration example
+
+For details on configuration refer to the following documentation: [awss3receiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/awss3receiver#aws-s3-receiver), [awslogsencodingextension](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/extension/encoding/awslogsencodingextension#aws-logs-encoding-extension), [elasticsearchexporter](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/exporter/elasticsearchexporter#configuration-options)
+
+```yaml
+extensions:
+  awslogs_encoding/vpcflow:
+    format: vpcflow
+    vpcflow:
+      file_format: plain-text
+
+receivers:
+  awss3:
+    sqs:
+      queue_url: "<sqs-url>"
+      region: "<region>"
+    s3downloader:
+      region: "<region>"
+      s3_bucket: '<bucket_name>'
+      s3_prefix: 'AWSLogs/<account-id>'
+    encodings:
+      - extension: awslogs_encoding/vpcflow
+
+exporters:
+  debug:
+    verbosity: detailed
+  elasticsearch/otel:
+    endpoints: https://<host>:<port>
+    user: elastic
+    password: <password>
+    mapping:
+      mode: otel
+    metrics_dynamic_index:
+      enabled: true
+
+service:
+  extensions: [awslogs_encoding/vpcflow]
+  pipelines:
+    logs:
+      exporters: [debug, elasticsearch/otel]
+      receivers: [awss3]
+```
+
 ## Logs Reference
 
 For a complete list of all available logs and their detailed descriptions, refer to the [OpenTelemetry AWS Logs encoding extension](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/extension/encoding/awslogsencodingextension#vpc-flow-log-record-fields)

packages/aws_vpcflow_otel/manifest.yml

@@ -1,7 +1,7 @@
 format_version: 3.5.0
 name: aws_vpcflow_otel
 title: "AWS VPC Flow Logs OpenTelemetry Assets"
-version: 0.1.0
+version: 0.1.1
 source:
   license: "Elastic-2.0"
 description: "AWS VPC Flow Logs OpenTelemetry Assets"