Skip to content

HOWTO dev docker_usage_for_GogglesMain

Jump to bottom
steveoro edited this page Apr 11, 2022 · 1 revision

HOW-TO: Docker usage with GogglesMain as example

References

This guide assumes Docker already installed & running and a basic knowledge of Docker usage.

For a more in-depth guide with docker commands for individual containers, check out GogglesAPI Docker usage.

Index

Getting started: setup and usage as a composed Docker service

The main composed Docker service requires also both services goggles-db & api already up and running.

To have each individual service running, the docker container will require access to its related Rails project master.key for secrets decryption and writable folders for storing the database volume, the logs and an additional folder for the backups needed by some maintenance jobs.

The docker-compose.dev.yml file stored within the goggles_main project takes care of most of the setup, assuming everything needed is kept under a single goggles_deploy folder that conveniently separates the storage areas needed by each different service in different sub-folders:

 --- goggles_deploy (any required files, like master-api.key & master-main.key)
            |
            +--- db.prod     (MySQL files)
            +--- db.staging
            +--- backups     (any stored DB dumps, like test.sql.bz2)
            +--- log.prod    (application logs)
            +--- log.staging

Using docker-compose is basically the way to go to have all 3 services up and running.

We have 3 different docker-compose.yml files, one for each main environment (development, staging & production). All 3 could be copied into the common goggles_deploy folder and possibly edited for convenience, depending on the actual folder & file setup used.

Refer to the GogglesAPI Docker usage guide for a more in-depth look on folder & file naming conventions and meaning.

0. Retrieve the latest Docker images (app + api)

Unless you're building Docker images locally, you'll need to retrieve the Docker images for the containers.

Log into DockerHub first and pull both images from our official DockerHub repository:

echo $DOCKERHUB_PASSWORD | docker login --username "$DOCKERHUB_USERNAME" --password-stdin
docker pull steveoro/goggles-main:latest
docker pull steveoro/goggles-api:latest

To rebuild locally the goggles-api & goggles-main images you'll need a local clone of both repositories, all properly set up, and then run a docker-compose up --build or a docker build for both. (Refer to the GogglesAPI Docker usage guide for more details.)

1. Bring up the composed service (app + api + DB)

Let's choose the dev configuration as an example.

Check if any previous instance is running locally with a docker ps -a and stop them, if running, with a composed down command:

$> docker-compose -f docker-compose.dev.yml down

Then, to run any configuration use either one of:

(Run in foreground)

$> docker-compose -f docker-compose.dev.yml up

(Run in background, detached)

$> docker-compose -f docker-compose.dev.yml up -d

2. Make sure a DB dump is available

The docker-compose.dev.yml file mounts the goggles_deploy/backups folder as db/dump for the app inside the container.

Copy the test.sql.bz2 dump from goggles_db (inside spec/dummy/db/dump) into goggles_deploy/backups so that the container will be able to read it.

3. Execute db:rebuild on the container

Open another console and run:

$> docker exec -it goggles-main.dev sh -c 'bundle exec rails db:rebuild from=test to=development'

Then, move to the console of the running service, stop it with CTRL-C and restart it, so that it may clear any caching and reload correctly with a full accessible database.

Bring up the composed service again and test it with something like this (Wiki).

Troubleshooting

Typical issues

  1. Out of memory errors and/or reduced amount of disk space.

Remember to remove unused or older Docker images from localhost with docker rmi <DOCKER_IMAGE_TAG>; the DOCKER_IMAGE_TAG can be read with a docker images command, which list all available images on localhost.

Each Docker app image can take up to a couple of GiB, so it's easy to run out of disk space after a bunch of months of intense localhost development and testing.

This must be done on the remote server as well, possibly after each release, as currently there's no automated check that does this sort of disk clean-up and the server instance is pretty small.

  1. Composed App service starts, but restarts in an infinite loop.

If something happens during the startup procedure as implemented by the Docker entrypoint of each container image (see goggles_main/entrypoints/docker.dev.sh), the service will try to restart itself as per configuration.

The infinite restart loop is usually due to a DB migration issue. Stop the infinite restart loop by stopping the service:

$> docker stop goggles-main.dev

Check the actual Docker output log with a:

$> docker logs goggles-main.dev

If indeed it's a DB migration issue, you'll need to run fix the issue manually on the project.

For the remote server a simple commit & push of the fix is all it takes to automatically re-deploy the corrected version.

But for a locally-tested image, a manual rebuild of the offending image is needed.

Clone this wiki locally