Skip to content

Conversation

@viv-4
Copy link
Contributor

@viv-4 viv-4 commented Jul 18, 2021

The issue of logspout using large amounts of cpu was caused by the source container restarting which is now resolved, 8f1ef39

Still, as these containers are not usually needed they have been removed from the default docker-compose.yml to be added in by install when ENABLE_ELASTIC_STACK=true in .env

@viv-4 viv-4 requested review from caspiano and w-le July 18, 2021 23:58
w-le
w-le previously approved these changes Jul 20, 2021
Copy link
Contributor

@w-le w-le left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks lgtm,
non-urgently, can you also add a convenience improvement where the elastic stack can be installed when a COMMAND LINE PARAM is specified, not just a change in.env? You should be able to copy the method used in ./install:33-37

@caspiano
Copy link
Contributor

Closes #7

placeos:
volumes:
- /var/run/docker.sock:/var/run/docker.sock
command: gelf://${LOGSTASH_HOST}:${LOGSTASH_PORT}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if the gelf version supports this, but if this can be filtered to only PlaceOS containers it will protected against trawling all logs.

@viv-4 viv-4 requested review from caspiano and kimburgess August 27, 2021 00:35
@caspiano
Copy link
Contributor

caspiano commented Aug 28, 2021

Made a bunch of changes.

Fixed

  • Ensure all images use dockerhub image host
  • Enabling kibana by environment works now

Added

  • Adds dependency between the logstash and logstash config validator

Changed

  • kibana uses the open distro for elasticsearch kibana
  • elasticsearch uses the open distro for elasticsearch
  • Use an alpine image for logstash

Removed

  • start-services and scripts related to elasticsearch docker-compose files

@caspiano
Copy link
Contributor

Since we're now using the Open Distro for Elasticsearch containers, this issue is not blocked #69

@caspiano
Copy link
Contributor

Also looks like logstash and logspout fail to hit elastic

@caspiano
Copy link
Contributor

caspiano commented Aug 28, 2021

Open distro for elasticsearch is SUPER heavy.
Each image is around 1.x GB in size, so we should try to resolve this to decrease spin-up time.

To reduce the size of the images, we can use the open distro plugins on top of blacktop’s alpine kibana/elastic container.

@caspiano caspiano dismissed w-le’s stale review September 2, 2021 02:41

Stale review

@caspiano caspiano added focus: devops Focus on DevOps/Infra type: refactor labels Nov 24, 2021
@caspiano caspiano marked this pull request as draft March 15, 2022 00:39
@caspiano
Copy link
Contributor

What's left of this PR's move to add opendistro for elasticsearch has been shifted to a separate PR, #100

@caspiano caspiano closed this Mar 15, 2022
@caspiano caspiano deleted the feat/logging branch March 15, 2022 07:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

focus: devops Focus on DevOps/Infra type: refactor

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants