Ajcody-Proxy-Guide-Rewrite-Project: Difference between revisions

mNo edit summary
 
(21 intermediate revisions by 2 users not shown)
Line 1: Line 1:
{{BC|Community Sandbox}}
__FORCETOC__
<div class="col-md-12 ibox-content">
=Ajcody Proxy Guide Rewrite Project=
{{KB|{{Unsupported}}|{{ZCS 8.0}}|{{ZCS 7.0}}|}}
{{WIP}}
= Overview And Planning For Zimbra Proxy=
= Overview And Planning For Zimbra Proxy=
Moved to:
Moved to:
Line 17: Line 23:
* https://wiki.zimbra.com/wiki/Zimbra_Proxy_Manual:Troubleshooting_Zimbra_Proxy
* https://wiki.zimbra.com/wiki/Zimbra_Proxy_Manual:Troubleshooting_Zimbra_Proxy


= Advance Topics For Zimbra Proxy - Miscellaneous Topics =
= Advance Topics For Zimbra Proxy - Configuration And Template Files And Proxy Related Variables =


== Miscellaneous Topics ==
https://wiki.zimbra.com/wiki/Zimbra_Proxy_Manual:Configuration_And_Template_Files_And_Proxy_Related_Variables


===  Set Up Proxy to use Clear Text for Upstream Connections ===
Merged and Updated the following pages below and then set a REDIRECT to the main page above:
* https://wiki.zimbra.com/wiki/NGINX_Configuration_Structure
* https://wiki.zimbra.com/wiki/NGINX_Configuration_Directive_Reference
* https://wiki.zimbra.com/wiki/Zimbra_Proxy_Attributes-Detailed


Source : Admin Guide Draft, 'Set Up Proxy to use Clear Text for Upstream Connections'
= Advance Topics For Zimbra Proxy - Advanced Proxy Configuration Examples via CLI=


When setting up the proxy to use clear text for upstream connections, set zimbraReverseProxySSLToUpstreamEnabled to FALSE.
Created :
* https://wiki.zimbra.com/wiki/Zimbra_Proxy_Manual:Advanced_Proxy_Configuration_Examples_via_CLI


This attribute defaults to TRUE. In an "out of the box" proxy set up, the upstream communication defaults to SSL.
= Advance Topics For Zimbra Proxy - Miscellaneous Topics =


=== REST URL Generation ===
Moved To:
 
* https://wiki.zimbra.com/wiki/Zimbra_Proxy_Manual:Miscellaneous_Topics
Source : Admin Guide Draft, 'REST URL Generation'
 
For REST URL, you set the host name, service protocol, and services port globally or for a specific domain from the following attributes.
 
* zimbraPublicServiceHostname
* zimbraPublicServiceProtocol
* zimbraPublicServicePort
 
When generating REST URL’s:
 
* If domain.zimbraPublicServiceHostname is set, use zimbraPublicServiceProtocol + zimbraPublicServiceHostname + zimbraPublicServicePort
* Otherwise it falls back to the server (account's home server) attributes:
** protocol is computed from server.zimbraMailMode
** hostname is server.zimbraServiceHostname
** port is computed from the protocol.
 
'''Note:''' Why use zimbraMailReferMode - In earlier versions, a local config variable called zimbra_auth_always_send_refer determined which action the back-end server took when a user’s mailbox did not reside on the server that the user logged in to. The default value of FALSE redirected the user if the user was logging in on the wrong backend host.
 
On a multiserver ZCS, if a load balanced name was needed to create a friendly landing page, a user would always have to be redirected. In that case, zimbra_auth_always_send_refer was set to TRUE.
 
Now with a full-fledged reverse proxy, users do not need to be redirected. The localconfig variable zimbraMailReferMode is used with nginx reverse proxy.
 
=== Set Proxy Trusted IP Addresses ===
 
Source : Admin Guide Draft, 'Set Proxy Trusted IP Addresses'
 
When a proxy is configured with ZCS, each proxy server’s IP address must be configured in LDAP attribute zimbraMailTrustedIP to identify the proxy addresses as trusted when users log in through the proxy. The proxy IP address is added to the X-Forwarded-For header information. The X-Forwarded-For header is automatically added to the localconfig zimbra_http_originating_ip_header attribute. When a user logs in, this IP address and the user’s address are verified in the Zimbra mailbox log.
 
Set each proxy IP address in the attribute. For example, if you have two proxy servers:
zmprov mcf +zimbraMailTrustedIP {IP of nginx-1} +zimbraMailTrustedIP {IP of nginx-2}
 
'''Note:''' To verify that X-Forwarded-For was correctly added to the localconfig, type zmlocalconfig | grep -i http. You should see zimbra_http originating_ip_header = X-Forwarded-For.
 
== Zimbra Proxy Performance Tuning ==
 
See the following unresolved RFE:
 
* "add nginx section to the performance tunning guide"
** https://bugzilla.zimbra.com/show_bug.cgi?id=26418
 
 
 
== Details On Some Of The Zimbra Customization To The Proxy Components ==
 
Source : [Internal Eng. Wiki /index.php/Zimbra/Admin/Proxy/Overview#Motivation_behind_Expanding_Nginx ]
 
===Motivation behind Expanding Nginx===
 
Although official Nginx is powerful, it can't meed all the needs of Zimbra.
 
===AUTH_HTTP===
 
Nginx's AUTH_HTTP protocol (described in http://wiki.nginx.org/NginxMailCoreModule#Authentication) is borrowed to implement the lookup with NLE. However, official Nginx's mail AUTH_HTTP config can only specify one NLE URL (auth_http directive). We wish to extend this support so that NZ can use more than one NLE. When a great load of user logins come, NZ can dispatch the load to multiple NLE running in different zimbra servers. This will end up not overloading a single NLE.  In a production set-up, it is likely that there will be many different NZ identically configured, and so the relationship between the NZ set and the server set running NLE is a many-to-many relationship.
 
Besides, the lookup is also necessary for zimbra web accessing, which is not supported by official Nginx. Therefore, this lookup and upstream choose function must also be implemented as an nginx http module.
 
Finally, because Zimbra doesn't use AUTH_HTTP protocol for real authentication, it might add some custom HTTP headers (such as "Auth-ID") and avoid some useless content transmit (such as password via "Auth-Pass").


All of these above have to be extended by Zimbra.
= Ports Scratchpad =


===Memcache Module===
To see ports available on your server, you can do as '''root''' :


It is likely that a single user may log in many times via NZ in a short time, or one http request will generate a great many of sub requests, all of which contains the same login information and thus will get the same route lookup result. In such cases, NZ will have to contact the NLE many times. It is not likely that the upstream server information for a particular client will change frequently. (A change in the upstream server usually corresponds to a user's mailbox being migrated to a different server). And at the same time NLE is relative much slower than Nginx, becoming the bottle neck.
netstat -anltp | egrep '^tcp' | grep LISTEN | awk '{print $4 " "$7}' | sed -e 's/.*://' | sort -n | uniq


As such, it will be beneficial if NZ is able to cache such information so that repeated trips to the NLE servers are avoided. Also, this cached information must be made available to possibly multiple NZ instances. Hence, "memcached" (http://www.danga.com/memcached/) was introduced for this purpose. Memcached is a high-performance, distributed memory object caching system.
{| class="wikitable sortable"
! Port !! If Proxied [Defaults] !! PID Name !! Pid Name If Proxied !! Package Name !! Package Name If Proxied !! zmprov related Variables !! Description !! Comments !! Binds To localhost Or Network Interface !! Open Or Routed Through Firewall
|-  style="background:white; color:black"
| 22 ||  || sshd ||  || sshd - from OS ||  || zimbraRemoteManagementPort || Remote Management Port ||  ||  ||
|-  style="background:white; color:black"
| 25 ||  || master ||  || mta ||  || zimbraSmtpPort || SMTP || Incoming mail to postfix ||  ||
|-  style="background:white; color:black"
| 53 ||  || unbound ||  || dnscache ||  || || DNS Cache Server || Comments || localhost ||
|-  style="background:white; color:black"
| 80 || 8080 || java || nginx  || store || proxy || zimbraMailPort ; '''''zimbraMailProxyPort'''''  || HTTP ; '''''HTTP Backend (when proxied)''''' || Comments ||  ||
|-  style="background:white; color:black"
| 443 || 8443 || java || nginx || store || proxy || zimbraMailSSLPort ; '''''zimbraMailSSLProxyPort''''' || HTTPS ; '''''HTTPS Backend (when proxied''''') || Comments ||  ||
|-  style="background:white; color:black"
|  || 11211 ||  || memcached ||  || proxy || zmprov related Variables || Memcached || Comments ||  ||
|-  style="background:white; color:black"
| 7072 ||  || java ||  || store ||  ||  || Route Lookup Handler || ZCS Nginx Lookup (backend http service for nginx lookup/authentication) ||  ||
|-  style="background:white; color:black"
| 3443 || 9443 || ? || nginx || ? || nginx || zimbraMailSSLClientCertPort ; '''''zimbraMailSSLProxyClientCertPort''''' || Mail Client Cert ; '''''Mail Client Cert Backend (when proxied)''''' || Comments ||  ||
|-  style="background:white; color:black"
| 110 || 7110 || java || nginx || store || nginx || zimbraMailProxyPort ; '''''zimbraMailSSLProxyPort''''' || POP3 ; '''''POP3 Backend (when proxied)''''' || Comments ||  ||
|-  style="background:white; color:black"
| 995 || 7995 || java || nginx || store || nginx || zimbraPop3SSLBindPort ; '''''zimbraPop3SSLProxyBindPort''''' || POP3S (Secure POP3) ; '''''POP3S Backend (when proxied)''''' || POP over SSL ||  ||
|-  style="background:white; color:black"
| 143 || 7143 || java || nginx || store || nginx || zimbraImapBindPort ; '''''zimbraImapProxyBindPort''''' || IMAP ; '''''IMAP Backend (when proxied)''''' || Comments ||  ||
|-  style="background:white; color:black"
| 993 || 7993 || java || nginx || store || nginx || zimbraImapSSLBindPort ; '''''zimbraImapSSLProxyBindPort''''' || IMAPS (Secure IMAP) ; '''''IMAPS Backend (when proxied)''''' || IMAP over SSL ||  ||
|-  style="background:white; color:black"
| 7071 || 9071 || java || nginx || store || nginx || zimbraAdminPort ; '''''zimbraAdminProxyPort''''' ; '''''zimbraReverseProxyAdminEnabled [default FALSE]''''' || Admin Console ; '''''Admin Console Through Proxy [If Enabled]''''' || HTTPS [nginx => mailbox when enabled]  ||  ||
|-  style="background:white; color:black"
| 465 ||  || master ||  || mta ||  ||  || SMTPS || Incoming mail to postfix over ssl (Legacy Outlook only?) If possible, use 587 instead) ||  ||
|-  style="background:white; color:black"
| 587 ||  || master ||  || mta ||  ||  || SMTP || Mail submission over TLS ||  ||
|-  style="background:white; color:black"
| 3310 ||  || clamd ||  || mta ||  || zimbraClamAVListenPort || ClamAV || Comments ||  ||
|-  style="background:white; color:black"
| 7025 ||  || java ||  || store ||  || zimbraLmtpBindPort || LMTP || Local mail delivery ||  ||
|-  style="background:white; color:black"
| 8465 ||  || opendkim ||  || mta ||  || zmprov related Variables || Description || Comments ||  ||
|-  style="background:white; color:black"
| 10024 ||  || amavisd ||  || mta ||  || zmprov related Variables || SMTP || To Amavis from Postfix ||  ||
|-  style="background:white; color:black"
| 10025 ||  || master ||  || mta ||  || zmprov related Variables || SMTP || To Postfix from Amavis ||  ||
|-  style="background:white; color:black"
| 10026 ||  || amavisd ||  || mta ||  || zmprov related Variables || Description || Comments ||  ||
|-  style="background:white; color:black"
| 10027 ||  || master ||  || mta ||  || zmprov related Variables || Description || Comments ||  ||
|-  style="background:white; color:black"
| 10028 ||  || master ||  || mta ||  || zmprov related Variables || Description || Comments ||  ||
|-  style="background:white; color:black"
| 10029 ||  || master ||  || mta ||  || zmprov related Variables || Description || Comments ||  ||
|-  style="background:white; color:black"
| 10030 ||  || master ||  || mta ||  || zmprov related Variables || Description || Comments ||  ||
|-  style="background:white; color:black"
| 10031 ||  ||  ||  || mta ||  || zimbraCBPolicydBindPort || CB Policy || Comments ||  ||
|-  style="background:white; color:black"
| 10032 ||  || amavisd ||  || mta ||  || zmprov related Variables || Description || Comments ||  ||
|-  style="background:white; color:black"
| 389 ||  || slapd ||  || ldap ||  ||  || LDAP || Comments ||  ||
|-  style="background:white; color:black"
| 636 ||  || slapd ||  || ldap ||  ||  || LDAPS || If enabled. ||  ||
|-  style="background:white; color:black"
| 7047 ||  || httpd ||  || convertd ||  || zmprov related Variables || Conversion server || Comments ||  ||
|-  style="background:white; color:black"
| 7306 ||  || mysqld ||  || store ||  || zmprov related Variables || Mysql || Comments ||  ||
|-  style="background:white; color:black"
| 7780 ||  || httpd ||  || spell ||  || zmprov related Variables || Spell check || Comments ||  ||
|- style="background:white; color:black"
| Port || If Proxied || PID Name || Pid Name If Proxied || Package Name || Package Name If Proxied || zmprov related Variables || Description || Comments ||  ||
|}
{{Article Footer|Zimbra Collaboration 8.0, 7.0|04/16/2014}}


Although official Nginx has a build-in "ngx_http_memcache" module, it can only be used for "URL->web page" cache. Therefore, a global "ngx_memcache" module is implemented in NZ which could be accessed by both web and mail modules. Memcached is used to cache 5 kinds of information as follows. The 4th and 5th are used in mail throttle control, which will be described in [Zimbra/Admin/Proxy/Throttle Control]:
----
* alias-->account name
* account name->route information
* account id-->route information
* login IP address->login count (IP throttle control)
* account name->login count (user throttle control)


===Enhanced Mail Protocol Support===
[[Category: Community Sandbox]]
 
[[Category: Author:Ajcody]]
Compared to the web proxy part, official nginx is relative weak in mail proxy, such as:
* Official nginx cannot completely support IMAP ID command, which is required for the logging of upstream zimbra servers
* Official nginx can't echo the [ALERT] msg before authenticating to the upstream server
* Official nginx does not support GSSAPI authentications.
* Official nginx does not support upstream SSL connection, that is, using POP3S and IMAPS protocol connecting nginx and upstream server.
 
These features are critical for a mail system of enterprise level. ZCS' mail server support these, so NZ has to support them as well.
 
Unlike web proxy, during which nginx simply proxies all the requests, in mail proxy nginx has to handle some mail protocol commands rather than proxy them to upstream before login is given. The reason is before getting route from NLE, NZ doesn't know to use which upstream server. These commands include "ID", "NOOP", "LOGIN", "CAPABILITY", "AUTHENTICATE", "LOGOUT", "STARTTLS" for IMAP, and "USER", "CAPA", "PASS", "NOOP", "STLS", "QUIT" for POP3. Although official nginx support most of them, but it has several bugs (like IMAP tag/untag responses), which are fixed by NZ.
 
SMTP proxy os nginx is not used in Zimbra so this part is not touched. Besides, some authentication methods supported by official nginx like SASL LOGIN, SALS CRAM-MD5 and APOP won't work because NLE can't support them.
 
===Throttle Control===
 
To reject DoS like attack, NZ will count the number of logins per IP/username in periods and deny the login if it's beyond a certain configured threshold value. This functionality is called Throttle Control. There are 2 kinds of them: IP Throttle Control and User Throttle Control. IP Throttle Control counts the login by client's IP address. But sometimes, this throttle control is not suitable for some cases (for example, many clients who locate in an internal network and thus have the same outside IP address of the gateway). Therefore, User Throttle Control is also introduced, which counts the login by client's user name. All these counters are stored in memcached so it's not persistent. And its algorithm is not quite accurate, but simple and feasible in practice. It's beneficial to implement Throttle Control in NZ because it can block the attacks before they touch real upstream mail servers.
 
Currently Throttle Control is only implemented for mail proxy. For web proxy, users can manually configure nginx with "deny" & "allow" directive. The following LDAP attributes have to be set to use this feature which get written using the corresponding nginx directives added for this module in /opt/zimbra/conf/nginx/templates/nginx.conf.mail.template configuration file.
zimbraReverseProxyIPLoginLimit    ->  mail_login_ip_max              ${mail.ipmax};
zimbraReverseProxyIPLoginLimitTime    ->  mail_login_ip_ttl              ${mail.ipttl};
zimbraReverseProxyIpThrottleMsg    ->  mail_login_ip_rejectmsg        "${mail.iprej}";
zimbraReverseProxyUserLoginLimit    ->  mail_login_user_max            ${mail.usermax};
zimbraReverseProxyUserLoginLimitTime    ->  mail_login_user_ttl            ${mail.userttl};
zimbraReverseProxyUserThrottleMsg    ->  mail_login_user_rejectmsg      "${mail.userrej}";
 
===Cert Per Virtual Host Name===
 
Many ZCS host thousands of domains, each of which has several virtual host names. Each domain may own its own SSL certificate/private key. If nginx is deployed, nginx can't proxy the SSL handshake to the upstream. So NZ has to support this "cert per domain" feature. Actually, the upstream Jetty server doesn't implement this feature. Therefore, NZ is critical in this use case. See the following for [[Ajcody-Proxy-Guide-Rewrite-Project#SSL_Certificates_Per_Domain_Set_Up|SSL Certificates Per Domain Set Up]].
 
= Advance Topics For Zimbra Proxy - Configuration And Template Files And Proxy Related Variables =
 
https://wiki.zimbra.com/wiki/Advance_Topics_For_Zimbra_Proxy_-_Configuration_And_Template_Files_And_Proxy_Related_Variables
 
Merged and Updated the following pages below and then set a REDIRECT to the main page above:
* https://wiki.zimbra.com/wiki/NGINX_Configuration_Structure
* https://wiki.zimbra.com/wiki/NGINX_Configuration_Directive_Reference
* https://wiki.zimbra.com/wiki/Zimbra_Proxy_Attributes-Detailed
 
 
= Advance Topics For Zimbra Proxy - Advanced Proxy Configuration Examples via CLI=
 
Created :
* https://wiki.zimbra.com/wiki/Advance_Topics_For_Zimbra_Proxy_-_Advanced_Proxy_Configuration_Examples_via_CLI

Latest revision as of 22:00, 7 June 2016

Ajcody Proxy Guide Rewrite Project

   KB 21113        Last updated on 2016-06-7  




0.00
(0 votes)

Overview And Planning For Zimbra Proxy

Moved to:

Installing , Configuring, Disabling the Zimbra Proxy

Moved To:

Zimbra Proxy Related CLI Commands

Moved To:

Troubleshooting Zimbra Proxy

Moved TO:

Advance Topics For Zimbra Proxy - Configuration And Template Files And Proxy Related Variables

https://wiki.zimbra.com/wiki/Zimbra_Proxy_Manual:Configuration_And_Template_Files_And_Proxy_Related_Variables

Merged and Updated the following pages below and then set a REDIRECT to the main page above:

Advance Topics For Zimbra Proxy - Advanced Proxy Configuration Examples via CLI

Created :

Advance Topics For Zimbra Proxy - Miscellaneous Topics

Moved To:

Ports Scratchpad

To see ports available on your server, you can do as root :

netstat -anltp | egrep '^tcp' | grep LISTEN | awk '{print $4 " "$7}' | sed -e 's/.*://' | sort -n | uniq
Port If Proxied [Defaults] PID Name Pid Name If Proxied Package Name Package Name If Proxied zmprov related Variables Description Comments Binds To localhost Or Network Interface Open Or Routed Through Firewall
22 sshd sshd - from OS zimbraRemoteManagementPort Remote Management Port
25 master mta zimbraSmtpPort SMTP Incoming mail to postfix
53 unbound dnscache DNS Cache Server Comments localhost
80 8080 java nginx store proxy zimbraMailPort ; zimbraMailProxyPort HTTP ; HTTP Backend (when proxied) Comments
443 8443 java nginx store proxy zimbraMailSSLPort ; zimbraMailSSLProxyPort HTTPS ; HTTPS Backend (when proxied) Comments
11211 memcached proxy zmprov related Variables Memcached Comments
7072 java store Route Lookup Handler ZCS Nginx Lookup (backend http service for nginx lookup/authentication)
3443 9443 ? nginx ? nginx zimbraMailSSLClientCertPort ; zimbraMailSSLProxyClientCertPort Mail Client Cert ; Mail Client Cert Backend (when proxied) Comments
110 7110 java nginx store nginx zimbraMailProxyPort ; zimbraMailSSLProxyPort POP3 ; POP3 Backend (when proxied) Comments
995 7995 java nginx store nginx zimbraPop3SSLBindPort ; zimbraPop3SSLProxyBindPort POP3S (Secure POP3) ; POP3S Backend (when proxied) POP over SSL
143 7143 java nginx store nginx zimbraImapBindPort ; zimbraImapProxyBindPort IMAP ; IMAP Backend (when proxied) Comments
993 7993 java nginx store nginx zimbraImapSSLBindPort ; zimbraImapSSLProxyBindPort IMAPS (Secure IMAP) ; IMAPS Backend (when proxied) IMAP over SSL
7071 9071 java nginx store nginx zimbraAdminPort ; zimbraAdminProxyPort ; zimbraReverseProxyAdminEnabled [default FALSE] Admin Console ; Admin Console Through Proxy [If Enabled] HTTPS [nginx => mailbox when enabled]
465 master mta SMTPS Incoming mail to postfix over ssl (Legacy Outlook only?) If possible, use 587 instead)
587 master mta SMTP Mail submission over TLS
3310 clamd mta zimbraClamAVListenPort ClamAV Comments
7025 java store zimbraLmtpBindPort LMTP Local mail delivery
8465 opendkim mta zmprov related Variables Description Comments
10024 amavisd mta zmprov related Variables SMTP To Amavis from Postfix
10025 master mta zmprov related Variables SMTP To Postfix from Amavis
10026 amavisd mta zmprov related Variables Description Comments
10027 master mta zmprov related Variables Description Comments
10028 master mta zmprov related Variables Description Comments
10029 master mta zmprov related Variables Description Comments
10030 master mta zmprov related Variables Description Comments
10031 mta zimbraCBPolicydBindPort CB Policy Comments
10032 amavisd mta zmprov related Variables Description Comments
389 slapd ldap LDAP Comments
636 slapd ldap LDAPS If enabled.
7047 httpd convertd zmprov related Variables Conversion server Comments
7306 mysqld store zmprov related Variables Mysql Comments
7780 httpd spell zmprov related Variables Spell check Comments
Port If Proxied PID Name Pid Name If Proxied Package Name Package Name If Proxied zmprov related Variables Description Comments
Verified Against: Zimbra Collaboration 8.0, 7.0 Date Created: 04/16/2014
Article ID: https://wiki.zimbra.com/index.php?title=Ajcody-Proxy-Guide-Rewrite-Project Date Modified: 2016-06-07



Try Zimbra

Try Zimbra Collaboration with a 60-day free trial.
Get it now »

Want to get involved?

You can contribute in the Community, Wiki, Code, or development of Zimlets.
Find out more. »

Looking for a Video?

Visit our YouTube channel to get the latest webinars, technology news, product overviews, and so much more.
Go to the YouTube channel »




Jump to: navigation, search