Linux Log Rotate and archive rotated file.
I was checking an article on how to upload archived logs to azure blob.
As initial set up I’ve updated logrotate.conf(/etc/logrotate.conf) with below settings.
file to edit:Permalink
/etc/logrotate.conf
| # see "man logrotate" for details | |
| # rotate log files weekly | |
| weekly | |
| # keep 4 weeks worth of backlogs | |
| rotate 1 | |
| # create new (empty) log files after rotating old ones | |
| create | |
| # use date as a suffix of the rotated file | |
| dateext | |
| dateformat -%Y-%m-%d-%s | |
| # uncomment this if you want your log files compressed | |
| #nocompress | |
| # RPM packages drop log rotation information into this directory | |
| include /etc/logrotate.d | |
| # no packages own wtmp and btmp -- we'll rotate them here | |
| /var/log/wtmp { | |
| monthly | |
| create 0664 root utmp | |
| minsize 1M | |
| rotate 1 | |
| } | |
| /var/log/btmp { | |
| missingok | |
| monthly | |
| create 0600 root utmp | |
| rotate 1 | |
| } |
Then updated syslog rotate config to test on how messages files can be archived. Make sure the destination path is created. In this case I’ve got /opt/logs/ referenced.
file edited
/opt/logrotate.d/syslog
| /var/log/cron | |
| /var/log/maillog | |
| /var/log/messages | |
| /var/log/secure | |
| /var/log/spooler | |
| { | |
| missingok | |
| sharedscripts | |
| postrotate | |
| /bin/kill -HUP `cat /var/run/syslogd.pid 2> /dev/null` 2> /dev/null || true | |
| tar -zcvf /opt/logs/messages-$(date +%F-%H-%M).tar.gz $(find /var/log/|grep messages-| head -2|tail -1) | |
| endscript | |
| } |
This will ensure the last rotated log file would be copied to /opt/logs/. You can have any other scripts configured to upload to blob from here. Once these settings are in place test run(force run) logrotate -f /etc/logrotate.conf. Result of the operation can be found below
view Current Dir # pwd
/opt/logs
view files in current directory ll
total 0
Force run logrotate # logrotate -f /etc/logrotate.conf
tar: Removing leading `/' from member names
/var/log/messages-2018-05-08-1525798193 -- Note the time stamp
View files in /opt/logs/ # ll
total 4
-rw-r--r--. 1 root root 257 May 8 16:56 messages-2018-05-08-16-56.tar.gz
View the file next to be rotated # ll /var/log/|grep messages-
-rw-------. 1 root root 147 May 8 16:49 messages-2018-05-08-1525798616
Test Results :)
If you are able to follow the bash commands you can see the archived file has the same timestamp as first created log.
[root@localhost logs]# cat /var/log/messages-2018-05-08-1525799265
May 8 17:05:18 localhost rsyslogd: [origin software="rsyslogd" swVersion="8.24.0" x-pid="553" x-info="http://www.rsyslog.com"] rsyslogd was HUPed
May 8 17:05:31 localhost chronyd[561]: Selected source 123.108.200.124
May 8 17:06:36 localhost su: (to root) vagrant on pts/0
May 8 17:07:40 localhost chronyd[561]: Selected source 139.59.43.68
[root@localhost logs]# logrotate -f /etc/logrotate.conf
tar: Removing leading `/' from member names
/var/log/messages-2018-05-08-1525799265
[root@localhost logs]# ll
total 4
-rw-r--r--. 1 root root 342 May 8 17:08 messages-2018-05-08-17-08.tar.gz
[root@localhost logs]# tar -zxvf *.tar.gz
var/log/messages-2018-05-08-1525799265
[root@localhost logs]# cat var/log/messages-2018-05-08-1525799265
May 8 17:05:18 localhost rsyslogd: [origin software="rsyslogd" swVersion="8.24.0" x-pid="553" x-info="http://www.rsyslog.com"] rsyslogd was HUPed
May 8 17:05:31 localhost chronyd[561]: Selected source 123.108.200.124
May 8 17:06:36 localhost su: (to root) vagrant on pts/0
May 8 17:07:40 localhost chronyd[561]: Selected source 139.59.43.68
If you want to implement logrotate to blog storage, the reference URL is dowdandassociates.com/blog/ .
Above link is about logrotate to S3 hourly using s3cmd. I would be using azcopy for blob storage upload.