Difference between revisions of "DoSFilter"

 
(9 intermediate revisions by 2 users not shown)
Line 1: Line 1:
{{ZC}}{{Article Infobox|{{admin}}|{{ZCS 8.0}}|||}}=DoSFilter Throttling Mechanism=
+
{{BC|Certified}}
 +
__FORCETOC__
 +
<div class="col-md-12 ibox-content">
 +
=DoSFilter Throttling Mechanism=
 +
{{KB|{{ZC}}|{{ZCS 8.0}}||}}
  
The denial-of-service filter or DoSFilter was added to the mailbox server in ZCS 8.0 to throttle clients sending a large number of requests over a very short period of time. The DoSFilter is applied to all requests for service, mailbox and admin. This feature was added with the completion of [https://bugzilla.zimbra.com/show_bug.cgi?id=66921 bug 66921].
+
The denial-of-service filter or DoSFilter was added to the mailbox server in ZCS 8.0 to throttle clients sending a large number of requests over a very short period of time. The DoSFilter is applied to mailboxd requests for service, mailbox and admin. This feature was added with the completion of [https://bugzilla.zimbra.com/show_bug.cgi?id=66921 bug 66921].
  
 
DoS filtering is enabled by default once ZCS 8 is installed. It may be necessary to adjust the configuration to accommodate specific environmental needs. Disabling DoSFilter is not recommended.
 
DoS filtering is enabled by default once ZCS 8 is installed. It may be necessary to adjust the configuration to accommodate specific environmental needs. Disabling DoSFilter is not recommended.
Line 34: Line 38:
  
 
====DoSFilter IP Addresses Whitelist - zimbraHttpThrottleSafeIPs====
 
====DoSFilter IP Addresses Whitelist - zimbraHttpThrottleSafeIPs====
 +
'''Warning: Zimbra Collaboration previous 8.7 doesn't support CIDR, so you must add the IPs individually.'''
 +
 
IP addresses to ignore when applying Jetty DosFilter. This attribute does not have a default value, however these loopback IPs are whitelisted by default:
 
IP addresses to ignore when applying Jetty DosFilter. This attribute does not have a default value, however these loopback IPs are whitelisted by default:
 
* 127.0.0.1
 
* 127.0.0.1
Line 45: Line 51:
 
Note: proxy nodes should not need to be whitelisted, as long as the Originating-IP feature is correctly configured in ZCS: https://wiki.zimbra.com/wiki/Log_Files#Logging_the_Originating_IP
 
Note: proxy nodes should not need to be whitelisted, as long as the Originating-IP feature is correctly configured in ZCS: https://wiki.zimbra.com/wiki/Log_Files#Logging_the_Originating_IP
  
IP addresses should be supplied in the multi-valued zimbraHttpThrottleSafeIPs attribute. CIDR notation can be used. To modify:
+
IP addresses should be supplied in the multi-valued zimbraHttpThrottleSafeIPs attribute. '''CIDR notation can be used in ZCS 8.7 and above. Please note in 8.6, 8.5 and 8.0 you must use it IP by IP'''.
  zmprov mcf zimbraHttpThrottleSafeIPs 10.1.2.3 zimbraHttpThrottleSafeIPs 192.168.4.5 zimbraHttpThrottleSafeIPs 192.168.1.0/24
+
=====Examples in ZCS 8.7 or above=====
 +
  zmprov mcf zimbraHttpThrottleSafeIPs 10.1.2.3/32 zimbraHttpThrottleSafeIPs 192.168.4.0/24
 +
or to append to an existing list of multi-valued zimbraHttpThrottleSafeIPs
 +
zmprov mcf +zimbraHttpThrottleSafeIPs 10.1.2.3/32
 +
zmprov mcf +zimbraHttpThrottleSafeIPs 192.168.4.0/24
  
 +
=====Examples in ZCS 8.6, 8.5 or previous=====
 +
zmprov mcf zimbraHttpThrottleSafeIPs 10.1.2.3 zimbraHttpThrottleSafeIPs 192.168.4.5
 
or to append to an existing list of multi-valued zimbraHttpThrottleSafeIPs
 
or to append to an existing list of multi-valued zimbraHttpThrottleSafeIPs
 
 
  zmprov mcf +zimbraHttpThrottleSafeIPs 10.1.2.3
 
  zmprov mcf +zimbraHttpThrottleSafeIPs 10.1.2.3
 +
zmprov mcf +zimbraHttpThrottleSafeIPs 10.1.2.50
 
  zmprov mcf +zimbraHttpThrottleSafeIPs 192.168.4.5
 
  zmprov mcf +zimbraHttpThrottleSafeIPs 192.168.4.5
  zmprov mcf +zimbraHttpThrottleSafeIPs 192.168.1.0/24
+
  zmprov mcf +zimbraHttpThrottleSafeIPs 192.168.4.6
  
 
====Mailbox server restart====
 
====Mailbox server restart====
Line 67: Line 79:
 
  zimbraInvalidLoginFilterReinstateIpTaskIntervalInMin: 5
 
  zimbraInvalidLoginFilterReinstateIpTaskIntervalInMin: 5
  
''zimbraInvalidLoginFilterDelayInMinBetwnRegBeforeReinstating'' sets how long an IP is blocked.
+
''zimbraInvalidLoginFilterDelayInMinBetwnReqBeforeReinstating'' sets how long an IP is blocked.
  
 
''zimbraInvalidLoginFilterMaxFailedLogin'' sets the number of failed logins before an IP is blocked.
 
''zimbraInvalidLoginFilterMaxFailedLogin'' sets the number of failed logins before an IP is blocked.
Line 77: Line 89:
 
To set the DoS filter to block an IP after 5 failed login attempts for 25 minutes, you would do this:
 
To set the DoS filter to block an IP after 5 failed login attempts for 25 minutes, you would do this:
  
  zmprov mcf zimbraInvalidLoginFilterDelayInMinBetwnRegBeforeReinstating 25
+
  zmprov mcf zimbraInvalidLoginFilterDelayInMinBetwnReqBeforeReinstating 25
 
  zmprov mcf zimbraInvalidLoginFilterMaxFailedLogin 5
 
  zmprov mcf zimbraInvalidLoginFilterMaxFailedLogin 5
 
  zmmailboxdctl restart
 
  zmmailboxdctl restart
Line 90: Line 102:
 
==ZCS 8.0.0 - 8.0.2==
 
==ZCS 8.0.0 - 8.0.2==
 
See this [http://www.zimbra.com/forums/announcements/60397-zcs-dosfilter-workaround-zcs-8-0-1-8-0-2-a.html link] on the Zimbra forums for information on configuring DoSFilter for ZCS 8 versions prior to 8.0.3.
 
See this [http://www.zimbra.com/forums/announcements/60397-zcs-dosfilter-workaround-zcs-8-0-1-8-0-2-a.html link] on the Zimbra forums for information on configuring DoSFilter for ZCS 8 versions prior to 8.0.3.
 +
 +
==References==
 +
* [https://bugzilla.zimbra.com/show_bug.cgi?id=85183 '''Bug 85183''' - Allow CIDR network addresses in throttle configuration]
  
 
{{Article_Footer|ZCS 8.0.x|03/20/2013}}
 
{{Article_Footer|ZCS 8.0.x|03/20/2013}}

Latest revision as of 15:13, 2 June 2020

DoSFilter Throttling Mechanism

   KB 20398        Last updated on 2020-06-2  




0.00
(0 votes)

The denial-of-service filter or DoSFilter was added to the mailbox server in ZCS 8.0 to throttle clients sending a large number of requests over a very short period of time. The DoSFilter is applied to mailboxd requests for service, mailbox and admin. This feature was added with the completion of bug 66921.

DoS filtering is enabled by default once ZCS 8 is installed. It may be necessary to adjust the configuration to accommodate specific environmental needs. Disabling DoSFilter is not recommended.

Identifying False Positives

It is possible for clients such as Zimbra Connector for Outlook (ZCO), mobile ActiveSync, zmprov, etc. to trigger the DoSFilter. To the client, the Zimbra mailbox service is unavailable. DoSFilter can be identified on the server in the following ways:

/opt/zimbra/log/sync.log

2013-01-15 15:52:20,426 WARN [qtp1635701107-91:https://10.10.0.54:443/Microsoft-Server-ActiveSync?User=zsupport2&DeviceId=Appl5K0113UN3NR&DeviceType=iPhone&Cmd=FolderSync][name=zsupport2@domain.com;mid=64;ip=71.194.89.54;Cmd=FolderSync;DeviceID=Appl5K0113UN3NR;Version=12.1;] sync - Service exception
com.zimbra.common.service.ServiceException: error while proxying request to target server: HTTP/1.1 503 Service Unavailable
ExceptionId:qtp1635701107-91:https://10.10.0.54:443/Microsoft-Server-ActiveSync?User=zsupport2&DeviceId=Appl5K0113UN3NR&DeviceType=iPhone&Cmd=FolderSync:1358286740426:c5ca7f36bb0a038f Code:service.PROXY_ERROR Arg:(url, STR,"http://mail.domain.com:80/service/soap/SyncRequest")

/opt/zimbra/log/zmmailboxd.out

2013-01-15 15:57:32.537:WARN:oejs.DoSFilter:DOS ALERT:ip=127.0.1.1,session=null,user=null

Configuration

The configuration attributes zimbraHttpDosFilterDelayMillis, zimbraHttpDosFilterMaxRequestsPerSec and zimbraHttpThrottleSafeIPs. Each attribute is server inherited from global or is configurable at the sever level. Our recommendation is to preserve the default configuration whenever possible.

DoSFilter Delay (milliseconds) - zimbraHttpDosFilterDelayMillis

Delay imposed on all requests over the rate limit, before they are considered at all. -1 = Reject request, 0 = No delay, any other value = Delay in ms. The default is -1.

To modify in the global configuration; e.g. set the delay to 20ms:

zmprov mcf zimbraHttpDosFilterDelayMillis 20

DoSFilter Maximum Requests Per Second - zimbraHttpDosFilterMaxRequestsPerSec

Maximum number of requests from a connection per second. Requests in excess of this are throttled. The default is 30 and the minimum is 1.

To set the maximum number for requests in the global configuration:

zmprov mcf zimbraHttpDosFilterMaxRequestsPerSec 100

DoSFilter IP Addresses Whitelist - zimbraHttpThrottleSafeIPs

Warning: Zimbra Collaboration previous 8.7 doesn't support CIDR, so you must add the IPs individually.

IP addresses to ignore when applying Jetty DosFilter. This attribute does not have a default value, however these loopback IPs are whitelisted by default:

  • 127.0.0.1
  •  ::1
  • 0:0:0:0:0:0:0:1
  • All mailboxd servers

You can check if these hosts have been correctly whitelisted by the log entry in /opt/zimbra/log/mailbox.log. This log entry should contain all of the default whitelisted hosts as well as any IPs added to zimbraHttpThrottleSafeIPs:

2014-09-09 10:33:47,772 INFO  [main] [] misc - DoSFilter: Configured whitelist IPs = 192.168.234.130,127.0.0.1,::1,0:0:0:0:0:0:0:1

Note: proxy nodes should not need to be whitelisted, as long as the Originating-IP feature is correctly configured in ZCS: https://wiki.zimbra.com/wiki/Log_Files#Logging_the_Originating_IP

IP addresses should be supplied in the multi-valued zimbraHttpThrottleSafeIPs attribute. CIDR notation can be used in ZCS 8.7 and above. Please note in 8.6, 8.5 and 8.0 you must use it IP by IP.

Examples in ZCS 8.7 or above
zmprov mcf zimbraHttpThrottleSafeIPs 10.1.2.3/32 zimbraHttpThrottleSafeIPs 192.168.4.0/24

or to append to an existing list of multi-valued zimbraHttpThrottleSafeIPs

zmprov mcf +zimbraHttpThrottleSafeIPs 10.1.2.3/32
zmprov mcf +zimbraHttpThrottleSafeIPs 192.168.4.0/24
Examples in ZCS 8.6, 8.5 or previous
zmprov mcf zimbraHttpThrottleSafeIPs 10.1.2.3 zimbraHttpThrottleSafeIPs 192.168.4.5

or to append to an existing list of multi-valued zimbraHttpThrottleSafeIPs

zmprov mcf +zimbraHttpThrottleSafeIPs 10.1.2.3
zmprov mcf +zimbraHttpThrottleSafeIPs 10.1.2.50
zmprov mcf +zimbraHttpThrottleSafeIPs 192.168.4.5
zmprov mcf +zimbraHttpThrottleSafeIPs 192.168.4.6

Mailbox server restart

A mailbox server restart is required when modifying these attributes.

zmmailboxdctl restart

Using the DoSFilter To Block IPs on Repeated Failed Login - ZCS 8.5+ Only

Starting in ZCS 8.5, you can block IPs for a period of time after a number of failed login attempts. Note that this honors zimbraHttpThrottleSafeIPs, so if set, it will not block the IPs whitelisted there.

You will be looking at these values:

zimbraInvalidLoginFilterDelayInMinBetwnReqBeforeReinstating: 15
zimbraInvalidLoginFilterMaxFailedLogin: 10
zimbraInvalidLoginFilterReinstateIpTaskIntervalInMin: 5

zimbraInvalidLoginFilterDelayInMinBetwnReqBeforeReinstating sets how long an IP is blocked.

zimbraInvalidLoginFilterMaxFailedLogin sets the number of failed logins before an IP is blocked.

zimbraInvalidLoginFilterReinstateIpTaskIntervalInMin sets how long between running the process to unblock IPs.

Examples

To set the DoS filter to block an IP after 5 failed login attempts for 25 minutes, you would do this:

zmprov mcf zimbraInvalidLoginFilterDelayInMinBetwnReqBeforeReinstating 25
zmprov mcf zimbraInvalidLoginFilterMaxFailedLogin 5
zmmailboxdctl restart

Tuning Considerations - 8.0.3 and later

ZCS Member Servers

ZCS servers under the control of a single master LDAP server are automatically whitelisted by IP address. These hosts are discovered using a GetAllServersRequest call; i.e., zmprov gas.

External Provisioning Hosts/SOAP API

External provisioning hosts may be added to the IP whitelist to ensure DoSFilter does not block some requests. For example, a mailbox reindex may make several calls per second that can trigger DoSFilter.

ZCS 8.0.0 - 8.0.2

See this link on the Zimbra forums for information on configuring DoSFilter for ZCS 8 versions prior to 8.0.3.

References

Verified Against: ZCS 8.0.x Date Created: 03/20/2013
Article ID: https://wiki.zimbra.com/index.php?title=DoSFilter Date Modified: 2020-06-02



Try Zimbra

Try Zimbra Collaboration with a 60-day free trial.
Get it now »

Want to get involved?

You can contribute in the Community, Wiki, Code, or development of Zimlets.
Find out more. »

Looking for a Video?

Visit our YouTube channel to get the latest webinars, technology news, product overviews, and so much more.
Go to the YouTube channel »

Jump to: navigation, search