Logrotate

[mbtamuli]

We need to run logrotate periodically on the host to check and rotate log files.

[mbtamuli]

We can use this as the base image https://github.com/samsung-cnct/logrotate

[sagarnasit]

@mbtamuli I'm mounting /opt/easyengine path with logrotate container. Logs are being archived as expected for ee.logs but, it can not able archive logs for site and global service specific logs because path for those logs are docker volumes. When i try to cd to mounted path it gives an error No such file or directoey.Check EasyEngine/dockerfiles#76 for logrotate Dockerfile.

[mbtamuli]

The log volumes are accesible directly in the directory /var/lib/docker/volumes

# ls -al /var/lib/docker/volumes/examplecom_log_php/_data/
total 12
drwxr-xr-x  3 root     root     4096 Dec 27 16:35 .
drwx------ 77 root     root     4096 Jan  4 07:58 ..
drwxr-xr-x  2 www-data www-data 4096 Dec 27 16:35 _data
# ls -al /var/lib/docker/volumes/example1com_log_php/_data/
total 92
drwxr-xr-x 2 www-data www-data  4096 Dec 27 16:35 .
drwxr-xr-x 3 root     root      4096 Dec 27 16:35 ..
-rw------- 1 www-data www-data 35375 Jan  3 17:46 access.log
-rw------- 1 www-data www-data 40619 Jan  3 17:45 error.log

So in this case, we have to mount /var/lib/docker and then, possibly give the
following as a regex to logrotate

/var/lib/docker/volumes/*log*/_data/

[sagarnasit]

/var/lib/docker/volumes/*log*/_data/ this regex string working fine with logrotate. After testing, logs have been archived from docker volumes as well. We can move to further implementation of logrotate.

[sagarnasit]

Logrotate is added as global service with global-logrotate service name. logrotate service container will require two host path to be mounted.

1./opt/easyengine path of root directory of easyengine.
2./var/lib/docker/volumes path of docker volumes directory.

docker-compose.yml content for logrotate service will be like following.

 global-logrotate:
    image: easyengine/logrotate:v4.0.0
    restart: always
    volumes:
      - "/opt/easyengine:/opt/easyengine"
      - "/var/lib/docker/volumes:/var/lib/docker/volumes"

[sagarnasit]

@mbtamuli
Container for logrotate service keeps restarting. I and @mrrobot47 tried to debug the issue but no luck with it. Can you check what is the issue?
Check Dockerfile and script added recently in PR for more detail.

[sagarnasit]

@mbtamuli Check this out. Maybe helpful for us.
https://docs.docker.com/config/containers/logging/configure/

[dhsathiya]

1. Works with n number of sites.
2. Reloads containers only 1 time after rotation.
3. 1 time configuration and copy-paste will work.
4. Logs the rotation in /opt/easyengine/logs/ee.log

/var/lib/docker/volumes/*log_php/_data/*.log                                                                                                                                                  
/var/lib/docker/volumes/*log_nginx/_data/*.log {
        daily
        missingok
        rotate 30
        compress
        delaycompress
        notifempty
        sharedscripts
        postrotate
            for site in $(/usr/local/bin/ee site list --format=text --enabled); do
                absolute_site_php=$(echo $site | sed -e 's/\.//g' )_php_1
                absolute_site_nginx=$(echo $site | sed -e 's/\.//g' )_nginx_1
                docker inspect -f '{{ .State.Pid }}' $absolute_site_nginx | xargs kill -USR1
                docker inspect -f '{{ .State.Pid }}' $absolute_site_php | xargs kill -USR1
                echo "$(date +'[%d/%m/%Y %H:%M:%S]') LogRotate.INFO: Rotated logs for $site" >> /opt/easyengine/logs/ee.log
            done
        endscript
}

Example log

[27/05/2020 09:30:47] LogRotate.INFO: Rotated logs for example1.com
[27/05/2020 09:30:47] LogRotate.INFO: Rotated logs for example2.com

Nginx-proxy

/opt/easyengine/services/nginx-proxy/logs/*.log {
    daily
    missingok
    copytruncate
    rotate 30
    compress
    delaycompress
    notifempty
    create 0640 www-data adm
    sharedscripts
}

[fabiomontefuscolo]

Why in the first config you have to reload services and in the second you simply use copytruncate?

[dhsathiya]

@fabiomontefuscolo The idea by then was to move the Nginx-proxy to reload only. But I decided to do it when implementing it in EE.

The difference between reload and copytruncate is here

Due to less overhead & faster speed, I am more leaning towards reload method only.

[danish17]

logrotate --force $CONF threw an error while trying to fetch the php container for non-PHP sites. Replaced:

docker inspect -f '{{ .State.Pid }}' $absolute_site_nginx | xargs kill -USR1
docker inspect -f '{{ .State.Pid }}' $absolute_site_php | xargs kill -USR1

with

$(docker inspect -f '{{ .State.Pid }}' $absolute_site_nginx | xargs kill -USR1) || echo "ok"
$(docker inspect -f '{{ .State.Pid }}' $absolute_site_php | xargs kill -USR1) || echo "ok"

Thank you, @dhsathiya