«

»

Dec 06 2013

Fail2ban not reading log file pointed by symlink

I am using ISPConfig 3.0.5.3 and Ubuntu 12.04.1 LTS x64 server, with a few custom Fail2Ban filters.

One of the filters is designed to protect my server from different IP’s stealing bandwidth.
Looking in my webalizer logs i noticed several IP’s (mostly from CHINA) downloading files I’ve attached to posts, usually drivers or manuals.

My Apache log looks like this :

175.176.236.18 - - [06/Dec/2013:13:02:41 +0200] "GET /wp-content/uploads/2013/09/KX-TE-Maintenance-Console-V3.0.exe_.zip HTTP/1.1" 304 0 "http://www.evilbox.ro/windows/panasonic-kx-tes824-usb-driver-windows-7-64bit/" "Mozilla/5.0 (Windows NT 6.2; WOW64; rv:25.0) Gecko/20100101 Firefox/25.0"

175.176.236.18 - - [06/Dec/2013:13:03:11 +0200] "GET /wp-content/uploads/2013/09/KX-TE-Maintenance-Console-V3.0.exe_.zip HTTP/1.1" 304 0 "http://www.evilbox.ro/windows/panasonic-kx-tes824-usb-driver-windows-7-64bit/" "Mozilla/5.0 (Windows NT 6.2; WOW64; rv:25.0) Gecko/20100101 Firefox/25.0"

175.176.236.18 - - [06/Dec/2013:13:03:41 +0200] "GET /wp-content/uploads/2013/09/KX-TE-Maintenance-Console-V3.0.exe_.zip HTTP/1.1" 304 0 "http://www.evilbox.ro/windows/panasonic-kx-tes824-usb-driver-windows-7-64bit/" "Mozilla/5.0 (Windows NT 6.2; WOW64; rv:25.0) Gecko/20100101 Firefox/25.0"

So after a bit of research I’ve decided to use Fail2Ban to check my Apache’s log for multiple GET requests for files (zip files especially).

So i’ve created the following filter in /etc/fail2ban/jail.local :

[http-get-dos]
enabled = true
port = http
filter = http-get-dos
logpath = /var/www/clients/client10/web13/log/access.log
maxretry = 3

And the following file : /etc/fail2ban/filter.d/http-get-dos.conf

# Fail2Ban configuration file
#
# Author: http://www.go2linux.org
#
[Definition]

# Option: failregex
# Note: This regex will match any GET entry in your logs, so basically all valid and not valid entries are a match.
# You should set up in the jail.conf file, the maxretry and findtime carefully in order to avoid false positives.

failregex = ^ -.*"\GET.*.zip.*

# Option: ignoreregex
# Notes.: regex to ignore. If this regex matches, the line is ignored.
# Values: TEXT
#
ignoreregex =

My problem is that the file /var/www/clients/client10/web13/log/access.log is a symlink to the file /var/www/clients/client10/web13/log/20131206-access.log and Fail2Ban doesn’t seem to notice the file being modified realtime. It only senses the file modifications at intervals of 5 minutes.
To get the detailed log (verbose) for Fail2Ban you need to modify loglevel=4 in /etc/fail2ban/fail2ban.conf

Then in /var/log/fail2ban.log you will get outupt like this :

2013-12-06 15:00:02,388 fail2ban.filter : DEBUG Got event: 1 for /var/log/apache2/access.log
2013-12-06 15:00:02,388 fail2ban.filter : DEBUG File changed: /var/log/apache2/access.log
2013-12-06 15:05:03,698 fail2ban.filter : DEBUG Got event: 1 for /var/log/apache2/access.log
2013-12-06 15:05:03,698 fail2ban.filter : DEBUG File changed: /var/log/apache2/access.log
2013-12-06 15:10:02,970 fail2ban.filter : DEBUG Got event: 1 for /var/log/apache2/access.log
2013-12-06 15:10:02,970 fail2ban.filter : DEBUG File changed: /var/log/apache2/access.log
2013-12-06 15:15:03,283 fail2ban.filter : DEBUG Got event: 1 for /var/log/apache2/access.log
2013-12-06 15:15:03,283 fail2ban.filter : DEBUG File changed: /var/log/apache2/access.log
2013-12-06 15:16:30,374 fail2ban.filter : DEBUG Got event: 1 for /var/log/apache2/access.log
2013-12-06 15:16:30,374 fail2ban.filter : DEBUG File changed: /var/log/apache2/access.log

But, access.log changes each few seconds or a few times a second.

Testing the fail2ban-regex works :

#fail2ban-regex /var/www/evilbox.ro/log/access.log /etc/fail2ban/filter.d/http-get-dos.conf
=============

Use regex file : /etc/fail2ban/filter.d/http-get-dos.conf
Use log file : /var/www/evilbox.ro/log/access.log
 ...
 176.223.62.242 (Fri Dec 06 14:23:51 2013)
 176.223.62.242 (Fri Dec 06 14:23:53 2013)
 176.223.62.242 (Fri Dec 06 14:24:46 2013)
 176.223.62.242 (Fri Dec 06 14:24:58 2013)

Date template hits:
0 hit(s): MONTH Day Hour:Minute:Second
0 hit(s): WEEKDAY MONTH Day Hour:Minute:Second Year
0 hit(s): WEEKDAY MONTH Day Hour:Minute:Second
0 hit(s): Year/Month/Day Hour:Minute:Second
0 hit(s): Day/Month/Year Hour:Minute:Second
0 hit(s): Day/Month/Year Hour:Minute:Second
6729 hit(s): Day/MONTH/Year:Hour:Minute:Second
0 hit(s): Month/Day/Year:Hour:Minute:Second
0 hit(s): Year-Month-Day Hour:Minute:Second
0 hit(s): Year.Month.Day Hour:Minute:Second
0 hit(s): Day-MONTH-Year Hour:Minute:Second[.Millisecond]
0 hit(s): Day-Month-Year Hour:Minute:Second
0 hit(s): TAI64N
0 hit(s): Epoch
0 hit(s): ISO 8601
0 hit(s): Hour:Minute:Second
0 hit(s): <[email protected]:Minute:Second>

Success, the total number of match is 408

However, look at the above section 'Running tests' which could contain important
information.

So the regex works, but the Fail2Ban doesn’t pickup the changes in access.log. Theese changes are realtime, i’ve checked them with #tail -f /var/www/evilbox.ro/log/access.log.

If I modify the path in /etc/fail2ban/jail.local from :

logpath = /var/www/clients/client10/web13/log/access.log

to

logpath = /var/www/clients/client10/web13/log/20131206-access.log

then Fail2Ban picks up the changes instantly.
Note : 20131206-access.log corresponds to today’s date.

I need to make Fail2Ban to read the symlink real-time. The symlink’s date and time are set each night when cron rotates the logs.
I’ve tried using softlinks and hardlinks, but the result is the same.

The only temporary solution i’ve found is to use a wildcard like this :

logpath = /var/www/clients/client10/web13/log/*-access.log

The log folder contains only 3 files that could match, the day before yesterday and yestarday’s logs.
The other logs are gzipped.

A (bad) solution would be to use the log from /var/log/apache2/other_vhosts_access.log, but that log is for all the websites hosted by ISPConfig.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>

*