Configure Fail2Ban for Zimbra Server with route instead of iptables to block IPs

1. Zimbra Tech Center
2. Community Sandbox
3. Configure Fail2Ban for Zimbra Server with route instead of iptables to block IPs

Configure Fail2Ban for Zimbra Server with route instead of iptables to block IPs

---

---

This article is a how-to guide on installing Fail2Ban to block attacking hosts using a null route or blackhole routes. This can help mitigate brute force attacks on Zimbra. Especially brute force attacks on SMTP are very common.

Prerequisite:

Fail2ban has been tested in combination with netfilter-persistent and iptables. If you use ufw or firewalld you may see errors when trying to ban/unban such as ERROR Failed to execute ban jail …​ action 'route'. This article has been validated using a set-up installed using https://github.com/Zimbra/zinstaller which you can use to test fail2ban before applying to your production environment.

It is required the OIP configuration must be done before configuring Fail2Ban service.

For a Single-Server Setup:
If you are running nginx on the same node as the mailstore, you will need to add both 127.0.0.1 and the real IP address of that node:

sudo -u zimbra -
zmprov mcf +zimbraMailTrustedIP 127.0.0.1 +zimbraMailTrustedIP {IP of Server}
zmcontrol restart

For a Multi-Server Setup:

sudo -u zimbra -
zmprov mcf +zimbraHttpThrottleSafeIPs {IP of Mailbox-1}
zmprov mcf +zimbraHttpThrottleSafeIPs {IP of Mailbox-2}
zmprov mcf +zimbraMailTrustedIP {IP of Proxy-1}
zmprov mcf +zimbraMailTrustedIP {IP of Proxy-2}
zmcontrol restart

Installation and Configuration of Fail2Ban

1) Install Fail2Ban Package

On RHEL/CentOS 7/8:

yum install epel-release -y
yum install fail2ban -y

On Ubuntu 18/20:

apt-get clean all ; apt-get update
apt-get install fail2ban -y

2) Create a file /etc/fail2ban/jail.local and it will override the default conf file /etc/fail2ban/jail.conf.
Add the local IP address of the Zimbra server in ignoreip =. You can also add other IP addresses to ignore from Fail2Ban checking.
On a multi-server setup, add all server’s IP in ignoreip list.

nano /etc/fail2ban/jail.local 

[DEFAULT]
# "ignoreip" can be a list of IP addresses, CIDR masks or DNS hosts.
# Fail2ban will not ban a host which matches an address in this list.
# Several addresses can be defined using space (and/or comma) separator.
#ignoreip = 127.0.0.1/8 ::1 10.137.26.29/32
ignoreip = 127.0.0.1/8 IP-ADDRESS-OF-ZIMBRA-SERVER/32

banaction = route

3) Create a jail file for Zimbra services.

nano /etc/fail2ban/jail.d/zimbra.local

[zimbra-smtp]
enabled = true
filter = zimbra-smtp
port = 25,465,587
logpath = /var/log/zimbra.log
maxretry = 3
findtime = 86400
bantime = 86400
action = route

[zimbra-web]
enabled = true
filter = zimbra-web
port = 80,443,7071,9071
logpath = /opt/zimbra/log/mailbox.log
maxretry = 5
findtime = 86400
bantime = 86400
action = route

Update: This article uses a regular expression that should work on most Zimbra deployments. To avoid double banning/unbanning which may lead to unpredictable results and errors this article combines the WebUI and Admin WebUI into a single jail called zimbra-web. This does mean that for most deployments a failed login will be counted double. So maxretry = 5 actually means you can try 3 times before being banned.

Property	Description
ignoreip	This parameter identifies IP address that should be ignored by the banning system. By default, this is just set to ignore traffic coming from the machine itself, which is a pretty good setting to have.
banaction	This sets the action that will be used when the threshold is reached. There is actually the name of a file located in ’`/etc/fail2ban/action.d/'’ which calls the configured action using the .conf file. Here we configured route which calls route.conf to handle the routing table manipulation to ban an IP address.
findtime	This parameter sets the window that fail2ban will pay attention to when looking for repeated failed authentication attempts. The default is set to 600 seconds (10 minutes again), which means that the software will count the number of failed attempts in the last 10 minutes.
bantime	This parameter sets the length of a ban, in seconds.
maxretry	This sets the number of failed attempts that will be tolerated within the findtime window before a ban is instituted.

4) [Optional]
If you want to apply Fail2Ban for SSH then create jail file sshd.local.
(No need to create filter rules for SSH, Fail2ban by default shipped with filter rules for SSH)
On Ubuntu systems, SSH jail is by default enabled within the jail file "/etc/fail2ban/jail.d/defaults-debian.conf".

nano /etc/fail2ban/jail.d/sshd.local

[sshd]
enabled = true
port = 22
maxretry = 3
findtime = 600
bantime = 3600

5) Create filters for Zimbra services.

nano /etc/fail2ban/filter.d/zimbra-web.conf 

[Definition]
failregex = .*ip=<HOST>;.*authentication failed for .*$

ignoreregex =

nano /etc/fail2ban/filter.d/zimbra-smtp.conf

[Definition]
failregex = postfix\/submission\/smtpd\[\d+\]: warning: .*\[<HOST>\]: SASL \w+ authentication failed: authentication failure$
            postfix\/smtps\/smtpd\[\d+\]: warning: .*\[<HOST>\]: SASL \w+ authentication failed: authentication failure$

ignoreregex =

6) Restart the Fail2ban service and enable it to start after system reboot.

systemctl restart fail2ban
systemctl status fail2ban
systemctl enable fail2ban

7) Check the status of the Fail2Ban jails.

fail2ban-client status

The result should be similar to this:

[root@centos8 ~]# fail2ban-client status
Status
|- Number of jail:      3
`- Jail list:   sshd, zimbra-smtp, zimbra-web
[root@centos8 ~]#
[root@centos8 ~]# fail2ban-client status sshd
Status for the jail: sshd
|- Filter
|  |- Currently failed: 0
|  |- Total failed:     14
|  `- Journal matches:  _SYSTEMD_UNIT=sshd.service + _COMM=sshd
`- Actions
   |- Currently banned: 1
   |- Total banned:     2
   `- Banned IP list:   10.137.26.29

8) Check banned IP in routing table.

ip r

route -n

The result should be similar to this:

[root@centos8 ~]# ip r
default via 10.0.10.1 dev ens3
10.0.10.0/24 dev ens3  proto kernel  scope link  src 10.0.10.67
unreachable 10.137.26.29
[root@centos8 ~]#
[root@centos8 ~]# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         10.0.10.1       0.0.0.0         UG    0      0        0 ens3
10.0.10.0       0.0.0.0         255.255.255.0   U     0      0        0 ens3
10.137.26.29    -               255.255.255.255 !H    0      -        0 -
[root@centos8 ~]#

9) Ban and unban an IP manually.

Ban an IP address.

fail2ban-client set "Jail-Name" banip "IP-Address"

Example:

fail2ban-client set sshd banip 10.137.26.29

Unban an IP address.

fail2ban-client set "Jail-Name" unbanip "Banned IP-Address"

Example:

[root@centos8 ~]# fail2ban-client set sshd unbanip 10.137.26.29

Unban everyone.

Can be useful when something goes wrong with creating new RegEx filter:

fail2ban-client unban --all

Debugging of Fail2Ban:

The loglevel and target are configured in /etc/fail2ban/fail2ban.conf you can also obtain the log level and log target by running:

fail2ban-client get loglevel
fail2ban-client get logtarget

To watch the log for debugging purpose you can run:

tail -f $(fail2ban-client get logtarget | grep "\`" | awk '{ print $2; }')

Fail2ban works by parsing log files using regular expressions, you can test the regular expression by using fail2ban-regex like this:

fail2ban-regex /opt/zimbra/log/mailbox.log /etc/fail2ban/filter.d/zimbra-web.conf

Multi server and centralized syslog

Fail2ban is designed to work on the local server. So it does it’s ban actions on the same server where it reads the logs. This can be a problem if you run Zimbra in a multi server scenario, where you can read the logs on the mailbox server, but want to apply the ban on the proxy server.

In addition you may want to use a centralized logging server and if you decide to ban a bad actor, deny access to all servers in your environment.

To do this you would need to create a custom fail2ban action. And set up SSH public key authentication so the server where you run fail2ban can connect to the server where the ban action needs to be applied. This article is not meant to cover all possible scenarios, but to get you started here is a basic example:

Create a new action by copying the default route action:

cp /etc/fail2ban/action.d/route.conf /etc/fail2ban/action.d/remote-route.conf

Next replace the local ip route command with an SSH command to run remotely, from this:

[Definition]
actionban   = ip route add <blocktype> <ip>
actionunban = ip route del <blocktype> <ip>
actioncheck =
actionstart =
actionstop =

[Init]

# Option:  blocktype
# Note:    Type can be blackhole, unreachable and prohibit. Unreachable and prohibit correspond to the ICMP reject messages.
# Values:  STRING
blocktype = unreachable

To this:

[Definition]
actionban   = ssh root@remote-server -C ip route add <blocktype> <ip>
actionunban = ssh root@remote-server -C ip route del <blocktype> <ip>
actioncheck =
actionstart =
actionstop =

[Init]
blocktype = unreachable

Then configure fail2ban to use the new action, in /etc/fail2ban/jail.local and /etc/fail2ban/jail.d/zimbra.local change

banaction = route
...
action = route

to:

banaction = remote-route
...
action = remote-route

Please note that this is meant to be a simple example to get you started, it is probably best NOT to use the root account on the remote server. But for testing this is the easiest. Once you have an idea of how it works, you will probably want to wrap the remote banaction into a script and use sudo on an account with limited access.

Which would lead to something like:

[Definition]
actionban   = /usr/local/sbin/my-banaction-script <blocktype> <ip>
actionunban = /usr/local/sbin/my-unbanaction-script <blocktype> <ip>
...

Then in /usr/local/sbin/my-banaction-script you could run the banaction to any number over servers over SSH, something like:

#!/bin/bash

ssh banuser@remote-proxy1 -C sudo ip route del $1 $2 &
ssh banuser@remote-proxy2 -C sudo ip route del $1 $2 &
...etc

Submitted by: Heera Singh Koranga