Configure Fail2Ban for Zimbra Server with route instead of iptables to block IPs: Difference between revisions
No edit summary |
No edit summary |
||
(10 intermediate revisions by 3 users not shown) | |||
Line 4: | Line 4: | ||
==Configure Fail2Ban for Zimbra Server with route instead of iptables to block IPs== | ==Configure Fail2Ban for Zimbra Server with route instead of iptables to block IPs== | ||
<hr> | <hr> | ||
{{KB|{{WIP | {{KB|{{WIP}}|{{ZCS 9.0}}|}} | ||
<hr> | <hr> | ||
This article is a how-to guide on installing Fail2Ban to block attacking hosts using a null route or blackhole routes. This can help mitigate brute force attacks on Zimbra. Especially brute force attacks on SMTP are very common. | |||
= | = Prerequisite: = | ||
It is required the OIP configuration must be done before configuring Fail2Ban service. | |||
'''For a Single-Server Setup:'''<br /> | |||
If you are running nginx on the same node as the mailstore, you will need to add both 127.0.0.1 and the real IP address of that node: | |||
: | |||
<pre>sudo -u zimbra - | |||
zmprov mcf +zimbraMailTrustedIP 127.0.0.1 +zimbraMailTrustedIP {IP of Server} | |||
zmcontrol restart</pre> | |||
'''For a Multi-Server Setup:''' | |||
<pre>sudo -u zimbra - | |||
zmprov mcf +zimbraHttpThrottleSafeIPs {IP of Mailbox-1} | |||
zmprov mcf +zimbraHttpThrottleSafeIPs {IP of Mailbox-2} | |||
zmprov mcf +zimbraMailTrustedIP {IP of Proxy-1} | |||
zmprov mcf +zimbraMailTrustedIP {IP of Proxy-2} | |||
zmcontrol restart</pre> | |||
= Installation and Configuration of Fail2Ban = | |||
''' | '''1)''' Install Fail2Ban Package | ||
'''On RHEL/CentOS 7/8:''' | |||
''' | <pre>yum install epel-release -y | ||
yum install fail2ban -y</pre> | |||
'''On Ubuntu 18/20:''' | |||
<pre>apt-get clean all ; apt-get update | |||
apt-get install fail2ban -y</pre> | |||
'''2)''' Create a file '''/etc/fail2ban/jail.local''' and it will override the default conf file '''/etc/fail2ban/jail.conf'''.<br /> | |||
Add the local IP address of the Zimbra server in '''ignoreip ='''. You can also add other IP addresses to ignore from Fail2Ban checking.<br /> | |||
On a multi-server setup, add all server’s IP in ignoreip list. | |||
<code>nano /etc/fail2ban/jail.local </code> | |||
; | <pre>[DEFAULT] | ||
# "ignoreip" can be a list of IP addresses, CIDR masks or DNS hosts. Fail2ban will not ban a host which matches an address in this list. | |||
# Several addresses can be defined using space (and/or comma) separator. | |||
#ignoreip = 127.0.0.1/8 ::1 10.137.26.29/32 | |||
ignoreip = 127.0.0.1/8 "IP-ADDRESS-OF-ZIMBRA-SERVER/32" | |||
banaction = route</pre> | |||
'''3)''' Create a jail file for Zimbra services. | |||
<code>nano /etc/fail2ban/jail.d/zimbra.local</code> | |||
<pre>[zimbra-smtp] | |||
enabled = true | |||
filter = zimbra-smtp | |||
port = 25,465,587 | |||
logpath = /var/log/zimbra.log | |||
maxretry = 3 | |||
findtime = 86400 | |||
bantime = 86400 | |||
action = route | |||
[zimbra-webmail] | |||
enabled = true | |||
filter = zimbra-webmail | |||
port = 80,443 | |||
logpath = /opt/zimbra/log/mailbox.log | |||
maxretry = 3 | |||
findtime = 86400 | |||
bantime = 86400 | |||
action = route | |||
{| | [zimbra-admin] | ||
enabled = true | |||
filter = zimbra-admin | |||
port = 7071,9071 | |||
logpath = /opt/zimbra/log/mailbox.log | |||
maxretry = 3 | |||
findtime = 86400 | |||
bantime = 86400 | |||
action = route</pre> | |||
{| | |||
!width="6%"| Property | |||
!width="94%"| Description | |||
|- | |- | ||
| | | ignoreip | ||
|This parameter identifies IP address that should be ignored by the banning system. By default, this is just set to ignore traffic coming from the machine itself, which is a pretty good setting to have. | | This parameter identifies IP address that should be ignored by the banning system. By default, this is just set to ignore traffic coming from the machine itself, which is a pretty good setting to have. | ||
|- | |- | ||
| | | banaction | ||
|This sets the action that will be used when the threshold is reached. There is actually the name of a file located in | | This sets the action that will be used when the threshold is reached. There is actually the name of a file located in ’`/etc/fail2ban/action.d/'’ which calls the configured action using the .conf file. Here we configured route which calls route.conf to handle the routing table manipulation to ban an IP address. | ||
Here we configured | |||
|- | |- | ||
| | | findtime | ||
|This parameter sets the window that fail2ban will pay attention to when looking for repeated failed authentication attempts. The default is set to 600 seconds (10 minutes again), which means that the software will count the number of failed attempts in the last 10 minutes. | | This parameter sets the window that fail2ban will pay attention to when looking for repeated failed authentication attempts. The default is set to 600 seconds (10 minutes again), which means that the software will count the number of failed attempts in the last 10 minutes. | ||
|- | |- | ||
| | | bantime | ||
|This parameter sets the length of a ban, in seconds | | This parameter sets the length of a ban, in seconds. | ||
|- | |- | ||
| | | maxretry | ||
|This sets the number of failed attempts that will be tolerated within the findtime window before a ban is instituted. | | This sets the number of failed attempts that will be tolerated within the findtime window before a ban is instituted. | ||
|} | |} | ||
'''4)''' '''[Optional]'''<br /> | |||
If you want to apply Fail2Ban for SSH then create jail file '''''sshd.local'''''.<br /> | |||
(No need to create filter rules for SSH, Fail2ban by default shipped with filter rules for SSH)<br /> | |||
On Ubuntu systems, SSH jail is by default enabled within the jail file "/etc/fail2ban/jail.d/defaults-debian.conf". | |||
<code>nano /etc/fail2ban/jail.d/sshd.local</code> | |||
<pre>[sshd] | |||
enabled = true | |||
port = 22 | |||
maxretry = 3 | |||
findtime = 600 | |||
bantime = 3600</pre> | |||
'''5)''' Create filters for Zimbra services. | |||
<code>nano /etc/fail2ban/filter.d/zimbra-webmail.conf </code> | |||
<pre>[Definition] | |||
failregex = .*oip=<HOST>;.*authentication failed for .*$ | |||
ignoreregex =</pre> | |||
<code>nano /etc/fail2ban/filter.d/zimbra-smtp.conf</code> | |||
<pre>[Definition] | |||
failregex = postfix\/submission\/smtpd\[\d+\]: warning: .*\[<HOST>\]: SASL \w+ authentication failed: authentication failure$ | |||
postfix\/smtps\/smtpd\[\d+\]: warning: .*\[<HOST>\]: SASL \w+ authentication failed: authentication failure$ | |||
ignoreregex =</pre> | |||
<code>nano /etc/fail2ban/filter.d/zimbra-admin.conf</code> | |||
<pre>[Definition] | |||
failregex = .*ip=<HOST>;.*authentication failed for .*$ | |||
ignoreregex =</pre> | |||
'''6)''' Restart the Fail2ban service and enable it to start after system reboot. | |||
''' | <pre>systemctl restart fail2ban | ||
systemctl status fail2ban | |||
systemctl enable fail2ban</pre> | |||
'''7)''' Check the status of the Fail2Ban jails. | |||
<code>fail2ban-client status</code> | |||
The result should be similar to this: | |||
<pre>[root@centos8 ~]# fail2ban-client status | |||
Status | |||
|- Number of jail: 4 | |||
`- Jail list: sshd, zimbra-admin, zimbra-smtp, zimbra-webmail | |||
[root@centos8 ~]# | |||
[root@centos8 ~]# fail2ban-client status sshd | |||
Status for the jail: sshd | |||
|- Filter | |||
| |- Currently failed: 0 | |||
| |- Total failed: 14 | |||
| `- Journal matches: _SYSTEMD_UNIT=sshd.service + _COMM=sshd | |||
`- Actions | |||
|- Currently banned: 1 | |||
|- Total banned: 2 | |||
`- Banned IP list: 10.137.26.29</pre> | |||
'''8)''' Check banned IP in routing table. | |||
<code>ip r</code> | |||
<code>route -n</code> | |||
The result should be similar to this: | |||
<pre>[root@centos8 ~]# ip r | |||
default via 10.0.10.1 dev ens3 | |||
10.0.10.0/24 dev ens3 proto kernel scope link src 10.0.10.67 | |||
unreachable 10.137.26.29 | |||
[root@centos8 ~]# | |||
[root@centos8 ~]# route -n | |||
Kernel IP routing table | |||
Destination Gateway Genmask Flags Metric Ref Use Iface | |||
0.0.0.0 10.0.10.1 0.0.0.0 UG 0 0 0 ens3 | |||
10.0.10.0 0.0.0.0 255.255.255.0 U 0 0 0 ens3 | |||
10.137.26.29 - 255.255.255.255 !H 0 - 0 - | |||
[root@centos8 ~]#</pre> | |||
'''9)''' Ban and unban an IP manually. | |||
'''Ban an IP address.''' | |||
<code>fail2ban-client set "Jail-Name" banip "IP-Address"</code> | |||
'''Example:''' | |||
<code>fail2ban-client set sshd banip 10.137.26.29</code> | |||
'''Unban an IP address.''' | |||
<code>fail2ban-client set "Jail-Name" unbanip "Banned IP-Address"</code> | |||
'''Example:''' | '''Example:''' | ||
<code>[root@centos8 ~]# fail2ban-client set sshd unbanip 10.137.26.29</code> | |||
'''Unban everyone.''' | |||
Can be useful when something goes wrong with creating new RegEx filter: | |||
<code>fail2ban-client unban --all</code> | |||
= Debugging of Fail2Ban: = | |||
The loglevel and target are configured in <code>/etc/fail2ban/fail2ban.conf</code> you can also obtain the log level and log target by running: | |||
<pre>fail2ban-client get loglevel | |||
fail2ban-client get logtarget</pre> | |||
To watch the log for debugging purpose you can run: | |||
<pre>tail -f $(fail2ban-client get logtarget | grep "\`" | awk '{ print $2; }')</pre> | |||
Fail2ban works by parsing log files using regular expressions, you can test the regular expression by using <code>fail2ban-regex</code> like this: | |||
<pre>fail2ban-regex /opt/zimbra/log/mailbox.log /etc/fail2ban/filter.d/zimbra-webmail.conf</pre> | |||
= Multi server and centralized syslog = | |||
Fail2ban is designed to work on the local server. So it does it’s ban actions on the same server where it reads the logs. This can be a problem if you run Zimbra in a multi server scenario, where you can read the logs on the mailbox server, but want to apply the ban on the proxy server. | |||
In addition you may want to use a centralized logging server and if you decide to ban a bad actor, deny access to all servers in your environment. | |||
To do this you would need to create a custom fail2ban action. And set up SSH public key authentication so the server where you run fail2ban can connect to the server where the ban action needs to be applied. This article is not meant to cover all possible scenarios, but to get you started here is a basic example: | |||
Create a new action by copying the default route action: | |||
<pre>cp /etc/fail2ban/action.d/route.conf /etc/fail2ban/action.d/remote-route.conf</pre> | |||
Next replace the local <code>ip route</code> command with an SSH command to run remotely, from this: | |||
<pre>[Definition] | |||
actionban = ip route add <blocktype> <ip> | |||
actionunban = ip route del <blocktype> <ip> | |||
actioncheck = | |||
actionstart = | |||
actionstop = | |||
[Init] | |||
# Option: blocktype | |||
# Note: Type can be blackhole, unreachable and prohibit. Unreachable and prohibit correspond to the ICMP reject messages. | |||
# Values: STRING | |||
blocktype = unreachable</pre> | |||
To this: | |||
<pre>[Definition] | |||
actionban = ssh root@remote-server -C ip route add <blocktype> <ip> | |||
actionunban = ssh root@remote-server -C ip route del <blocktype> <ip> | |||
actioncheck = | |||
actionstart = | |||
actionstop = | |||
[Init] | |||
blocktype = unreachable</pre> | |||
Then configure fail2ban to use the new action, in <code>/etc/fail2ban/jail.local</code> and <code>/etc/fail2ban/jail.d/zimbra.local</code> change | |||
<pre>banaction = route | |||
... | |||
action = route</pre> | |||
to: | |||
<pre>banaction = remote-route | |||
... | |||
action = remote-route</pre> | |||
Please note that this is meant to be a simple example to get you started, it is probably best NOT to use the root account on the remote server. But for testing this is the easiest. Once you have an idea of how it works, you will probably want to wrap the remote banaction into a script and use <code>sudo</code> on an account with limited access. | |||
Which would lead to something like: | |||
<pre>[Definition] | |||
actionban = /usr/local/sbin/my-banaction-script <blocktype> <ip> | |||
actionunban = /usr/local/sbin/my-unbanaction-script <blocktype> <ip> | |||
...</pre> | |||
Then in <code>/usr/local/sbin/my-banaction-script</code> you could run the banaction to any number over servers over SSH, something like: | |||
<pre>#!/bin/bash | |||
; | ssh banuser@remote-proxy1 -C sudo ip route del $1 $2 & | ||
ssh banuser@remote-proxy2 -C sudo ip route del $1 $2 & | |||
...etc</pre> | |||
Line 254: | Line 290: | ||
|'''Submitted by''': Heera Singh Koranga | |'''Submitted by''': Heera Singh Koranga | ||
|} | |} | ||
{{Article Footer|ZCS 9.0 | {{Article Footer|ZCS 9.0|2020-12-09}} |
Revision as of 07:21, 19 August 2022
Configure Fail2Ban for Zimbra Server with route instead of iptables to block IPs
This article is a how-to guide on installing Fail2Ban to block attacking hosts using a null route or blackhole routes. This can help mitigate brute force attacks on Zimbra. Especially brute force attacks on SMTP are very common.
Prerequisite:
It is required the OIP configuration must be done before configuring Fail2Ban service.
For a Single-Server Setup:
If you are running nginx on the same node as the mailstore, you will need to add both 127.0.0.1 and the real IP address of that node:
sudo -u zimbra - zmprov mcf +zimbraMailTrustedIP 127.0.0.1 +zimbraMailTrustedIP {IP of Server} zmcontrol restart
For a Multi-Server Setup:
sudo -u zimbra - zmprov mcf +zimbraHttpThrottleSafeIPs {IP of Mailbox-1} zmprov mcf +zimbraHttpThrottleSafeIPs {IP of Mailbox-2} zmprov mcf +zimbraMailTrustedIP {IP of Proxy-1} zmprov mcf +zimbraMailTrustedIP {IP of Proxy-2} zmcontrol restart
Installation and Configuration of Fail2Ban
1) Install Fail2Ban Package
On RHEL/CentOS 7/8:
yum install epel-release -y yum install fail2ban -y
On Ubuntu 18/20:
apt-get clean all ; apt-get update apt-get install fail2ban -y
2) Create a file /etc/fail2ban/jail.local and it will override the default conf file /etc/fail2ban/jail.conf.
Add the local IP address of the Zimbra server in ignoreip =. You can also add other IP addresses to ignore from Fail2Ban checking.
On a multi-server setup, add all server’s IP in ignoreip list.
nano /etc/fail2ban/jail.local
[DEFAULT] # "ignoreip" can be a list of IP addresses, CIDR masks or DNS hosts. Fail2ban will not ban a host which matches an address in this list. # Several addresses can be defined using space (and/or comma) separator. #ignoreip = 127.0.0.1/8 ::1 10.137.26.29/32 ignoreip = 127.0.0.1/8 "IP-ADDRESS-OF-ZIMBRA-SERVER/32" banaction = route
3) Create a jail file for Zimbra services.
nano /etc/fail2ban/jail.d/zimbra.local
[zimbra-smtp] enabled = true filter = zimbra-smtp port = 25,465,587 logpath = /var/log/zimbra.log maxretry = 3 findtime = 86400 bantime = 86400 action = route [zimbra-webmail] enabled = true filter = zimbra-webmail port = 80,443 logpath = /opt/zimbra/log/mailbox.log maxretry = 3 findtime = 86400 bantime = 86400 action = route [zimbra-admin] enabled = true filter = zimbra-admin port = 7071,9071 logpath = /opt/zimbra/log/mailbox.log maxretry = 3 findtime = 86400 bantime = 86400 action = route
Property | Description |
---|---|
ignoreip | This parameter identifies IP address that should be ignored by the banning system. By default, this is just set to ignore traffic coming from the machine itself, which is a pretty good setting to have. |
banaction | This sets the action that will be used when the threshold is reached. There is actually the name of a file located in ’`/etc/fail2ban/action.d/'’ which calls the configured action using the .conf file. Here we configured route which calls route.conf to handle the routing table manipulation to ban an IP address. |
findtime | This parameter sets the window that fail2ban will pay attention to when looking for repeated failed authentication attempts. The default is set to 600 seconds (10 minutes again), which means that the software will count the number of failed attempts in the last 10 minutes. |
bantime | This parameter sets the length of a ban, in seconds. |
maxretry | This sets the number of failed attempts that will be tolerated within the findtime window before a ban is instituted. |
4) [Optional]
If you want to apply Fail2Ban for SSH then create jail file sshd.local.
(No need to create filter rules for SSH, Fail2ban by default shipped with filter rules for SSH)
On Ubuntu systems, SSH jail is by default enabled within the jail file "/etc/fail2ban/jail.d/defaults-debian.conf".
nano /etc/fail2ban/jail.d/sshd.local
[sshd] enabled = true port = 22 maxretry = 3 findtime = 600 bantime = 3600
5) Create filters for Zimbra services.
nano /etc/fail2ban/filter.d/zimbra-webmail.conf
[Definition] failregex = .*oip=<HOST>;.*authentication failed for .*$ ignoreregex =
nano /etc/fail2ban/filter.d/zimbra-smtp.conf
[Definition] failregex = postfix\/submission\/smtpd\[\d+\]: warning: .*\[<HOST>\]: SASL \w+ authentication failed: authentication failure$ postfix\/smtps\/smtpd\[\d+\]: warning: .*\[<HOST>\]: SASL \w+ authentication failed: authentication failure$ ignoreregex =
nano /etc/fail2ban/filter.d/zimbra-admin.conf
[Definition] failregex = .*ip=<HOST>;.*authentication failed for .*$ ignoreregex =
6) Restart the Fail2ban service and enable it to start after system reboot.
systemctl restart fail2ban systemctl status fail2ban systemctl enable fail2ban
7) Check the status of the Fail2Ban jails.
fail2ban-client status
The result should be similar to this:
[root@centos8 ~]# fail2ban-client status Status |- Number of jail: 4 `- Jail list: sshd, zimbra-admin, zimbra-smtp, zimbra-webmail [root@centos8 ~]# [root@centos8 ~]# fail2ban-client status sshd Status for the jail: sshd |- Filter | |- Currently failed: 0 | |- Total failed: 14 | `- Journal matches: _SYSTEMD_UNIT=sshd.service + _COMM=sshd `- Actions |- Currently banned: 1 |- Total banned: 2 `- Banned IP list: 10.137.26.29
8) Check banned IP in routing table.
ip r
route -n
The result should be similar to this:
[root@centos8 ~]# ip r default via 10.0.10.1 dev ens3 10.0.10.0/24 dev ens3 proto kernel scope link src 10.0.10.67 unreachable 10.137.26.29 [root@centos8 ~]# [root@centos8 ~]# route -n Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface 0.0.0.0 10.0.10.1 0.0.0.0 UG 0 0 0 ens3 10.0.10.0 0.0.0.0 255.255.255.0 U 0 0 0 ens3 10.137.26.29 - 255.255.255.255 !H 0 - 0 - [root@centos8 ~]#
9) Ban and unban an IP manually.
Ban an IP address.
fail2ban-client set "Jail-Name" banip "IP-Address"
Example:
fail2ban-client set sshd banip 10.137.26.29
Unban an IP address.
fail2ban-client set "Jail-Name" unbanip "Banned IP-Address"
Example:
[root@centos8 ~]# fail2ban-client set sshd unbanip 10.137.26.29
Unban everyone.
Can be useful when something goes wrong with creating new RegEx filter:
fail2ban-client unban --all
Debugging of Fail2Ban:
The loglevel and target are configured in /etc/fail2ban/fail2ban.conf
you can also obtain the log level and log target by running:
fail2ban-client get loglevel fail2ban-client get logtarget
To watch the log for debugging purpose you can run:
tail -f $(fail2ban-client get logtarget | grep "\`" | awk '{ print $2; }')
Fail2ban works by parsing log files using regular expressions, you can test the regular expression by using fail2ban-regex
like this:
fail2ban-regex /opt/zimbra/log/mailbox.log /etc/fail2ban/filter.d/zimbra-webmail.conf
Multi server and centralized syslog
Fail2ban is designed to work on the local server. So it does it’s ban actions on the same server where it reads the logs. This can be a problem if you run Zimbra in a multi server scenario, where you can read the logs on the mailbox server, but want to apply the ban on the proxy server.
In addition you may want to use a centralized logging server and if you decide to ban a bad actor, deny access to all servers in your environment.
To do this you would need to create a custom fail2ban action. And set up SSH public key authentication so the server where you run fail2ban can connect to the server where the ban action needs to be applied. This article is not meant to cover all possible scenarios, but to get you started here is a basic example:
Create a new action by copying the default route action:
cp /etc/fail2ban/action.d/route.conf /etc/fail2ban/action.d/remote-route.conf
Next replace the local ip route
command with an SSH command to run remotely, from this:
[Definition] actionban = ip route add <blocktype> <ip> actionunban = ip route del <blocktype> <ip> actioncheck = actionstart = actionstop = [Init] # Option: blocktype # Note: Type can be blackhole, unreachable and prohibit. Unreachable and prohibit correspond to the ICMP reject messages. # Values: STRING blocktype = unreachable
To this:
[Definition] actionban = ssh root@remote-server -C ip route add <blocktype> <ip> actionunban = ssh root@remote-server -C ip route del <blocktype> <ip> actioncheck = actionstart = actionstop = [Init] blocktype = unreachable
Then configure fail2ban to use the new action, in /etc/fail2ban/jail.local
and /etc/fail2ban/jail.d/zimbra.local
change
banaction = route ... action = route
to:
banaction = remote-route ... action = remote-route
Please note that this is meant to be a simple example to get you started, it is probably best NOT to use the root account on the remote server. But for testing this is the easiest. Once you have an idea of how it works, you will probably want to wrap the remote banaction into a script and use sudo
on an account with limited access.
Which would lead to something like:
[Definition] actionban = /usr/local/sbin/my-banaction-script <blocktype> <ip> actionunban = /usr/local/sbin/my-unbanaction-script <blocktype> <ip> ...
Then in /usr/local/sbin/my-banaction-script
you could run the banaction to any number over servers over SSH, something like:
#!/bin/bash ssh banuser@remote-proxy1 -C sudo ip route del $1 $2 & ssh banuser@remote-proxy2 -C sudo ip route del $1 $2 & ...etc
Submitted by: Heera Singh Koranga |