Skip to content

HOWTO devops logrotate

Jump to bottom
steveoro edited this page Apr 26, 2021 · 1 revision

HOW-TO: logorotate configuration.

References:

Set up:

Create a logrotate configuration file that can be uploaded to the server:

$> cd ~/Projects/goggles_deploy
$> vi ~/Projects/goggles_deploy/logrotate.conf

The .conf will compress & rotate all the logs on a daily basis for 7 days, using a date timestamp extension, skipping empty logs and creating new ones after rotation:

/home/deploy/Projects/goggles_deploy/log.prod/*.log /home/deploy/Projects/goggles_deploy/log.prod/api/*.log /home/deploy/Projects/goggles_deploy/log.staging/*.log /home/deploy/Projects/goggles_deploy/log.staging/api/*.log {
    daily
    compress
    dateext
    dateyesterday
    missingok
    notifempty
    rotate 7
    create
}

Use the same ownership and permissions of the user running the container (default: root). This should include also the log subfolders even though the apps running inside the container might have not complained yet about a different group ownership (deploy:deploy) when compared to the process actually writing the logs (root:root).

For that matter, logrotate will righteously complain; so, stick to 0644 for the configuration file & set the actual owner & group for the log files if not yet done:

$> chmod 0644 logrotate*
$> sudo chown -R root:root log*

Test execution (add a --force to actually force the rotation):

$> sudo logrotate --state /home/deploy/logrotate-state --force --verbose /home/deploy/logrotate.conf

Copy the configuration file to the server:

$> scp logrotate.conf [email protected]:~

Log in to the server, fix ownership and permissions as above if not yet done before & set-up a proper cron job:

$> ssh [email protected]
# ...

$> sudo crontab -e

Edit the cron table so that it hands out a notification email on each run (@ ~3.00am).

Use a 2>&1 redirection at the end of the logrotate statement to set the verbose output from stderr to stdout, so that cron may catch it maintaining a common output priority , in case all the output from the cron table execution needs to be sent as mail text.

Also, typically a deploy server will have a UTC difference of +/- some hours from your typical timezone: remember to take that too into account when setting the timing for the crontab.

# Uncomment to toggle e-mail notification of each execution (overkill for most cases):
# MAILTO=<MAILTO_ADDRESS_FOR_NOTIFICATIONS>

# Main app log rotation & DB backups:
00 1 * * * /usr/sbin/logrotate --state /home/deploy/logrotate-state --verbose /home/deploy/logrotate.conf

# Run any recurrent or additional crontab scripts:
00 6 * * * /bin/bash -l /home/deploy/crontab_check.sh
00 12 * * * /bin/bash -l /home/deploy/crontab_check.sh
00 20 * * * /bin/bash -l /home/deploy/crontab_check.sh

# Adjust ntpdate (needs ntpdate installed):
# 0 0 * * * /etc/network/ip-up.d/ntpdate

Reload the crontab with sudo service cron reload.

Extra: synchronize Date & Time using the NTP protocol

References:

Chrony is now the default NTP implementation package on the latest versions of Linux operating systems.

Before installing Chrony, make sure the NTP service isn't already enabled by checking the output of timedatectl (which should report it as "NTP service active: yes").

Chrony installation & setup:

$> sudo apt-get install chrony
$> sudo systemctl enable --now chronyd

# Check daemon status:
$> systemctl status chronyd

# Check activity:
$> chronyc activity

# Check time sources:
$> chronyc sources

# Check tracking:
$> chronyc tracking

# Check system date & time:
$> timedatectl

Done!

Clone this wiki locally