Ajcody-Proxy-Guide-Rewrite-Project: Difference between revisions

mNo edit summary
 
(27 intermediate revisions by 2 users not shown)
Line 1: Line 1:
{{BC|Community Sandbox}}
__FORCETOC__
<div class="col-md-12 ibox-content">
=Ajcody Proxy Guide Rewrite Project=
{{KB|{{Unsupported}}|{{ZCS 8.0}}|{{ZCS 7.0}}|}}
{{WIP}}
= Overview And Planning For Zimbra Proxy=
= Overview And Planning For Zimbra Proxy=
 
Moved to:
Source: http://wiki.zimbra.com/wiki/Zimbra_Proxy_Guide
* https://wiki.zimbra.com/wiki/Zimbra_Proxy_Manual:Overview_And_Planning_For_Zimbra_Proxy
 
== What Is Zimbra Proxy ==
 
Source:
* http://wiki.zimbra.com/wiki/Zimbra_Proxy_Guide#What_is_Zimbra_Proxy
* Admin Guide Draft - 'Zimbra Proxy Chapter'
 
<span style="color:#DC143C">* Needs to be proofed. </span>
 
<span style="color:#DC143C">* Make references to pop,imap,http as IMAP[S]/POP[S]/HTTP[S] </span>
 
Zimbra Proxy is a high-performance reverse proxy service for passing IMAP[S]/POP[S]/HTTP[S] client requests to other internal ZCS services. A reverse proxy server is an Internet-facing services that protects and manages client connections to your internal services. It can also provide functions like: GSSAPI authentication, throttle control, SSL connection with different certificates for different virtual host names, and other features described later below. When discussing reverse proxy topics, you'll see references to downstream and upstream. Downstream is in reference to the end-user client and the software/computer they are using. Upstream is in reference to the actually server or service that actually handles the request - often times this is the Zimbra mailstore or webapp server the users account resides on.
 
 
The Zimbra Proxy services allows other zimbra services [mailbox, webapp, etc.] that can be hidden from the Internet by acting as a reverse proxy for the other services and allows end-users to access the mail system via a single Login URL [for example, userA and userB can both use mail.domain.com] instead of knowing their specific mailbox hostnames [for example, userA uses mail12.domain.com and userB is uses mail13.domain.com]. It acts as the first entry point for all the HTTP/IMAP/POP traffic and then intelligently routes all static UI (HTML/CSS/JS etc) and dynamic (SOAP/REST/IMAP/POP) requests to the appropriate upstream server. The proxy configuration options for POP/IMAP allows end users to configure their POP/IMAP clients to use this single mail server hostname. The proxy configuration options around HTTP[S] allows end-users to use this single hostname for ZWC, REST, CalDAV, Zimbra Connector for Outlook, Zimbra Connector for BES, Zimbra Mobile Sync connections and so forth. The Zimbra Proxy service will also do URL rewriting so these internal hostnames that are providing the various services aren't exposed to the public or end-clients. For example, a Zimbra briefcase share from userA on mail12.domain.com to userB on mail13.domain.com will simply have mail.domain.com [per our example above] within the url. [ see [https://bugzilla.zimbra.com/show_bug.cgi?id=82236 bug 82236] ]
 
 
With ZCS 8.5, the proxy components are installed by default and therefor would be enabled on a single ZCS server deployment. Normally though, the proxy services would be installed on Internet-facing servers and the other specific ZCS services [zimbra-store for example] would be on installed on internal servers. Generally, these packages are installed on dedicated Internet-facing servers intended to just do the proxy services or those that also run the zimbra-mta package. When the Zimbra Proxy package is installed, the proxy feature is enabled with default values that normally require no modification.
 
 
The Zimbra Proxy installation components consists of the zimbra-proxy and zimbra-memcached options during the installation package choice menu. The zimbra-proxy packages is Zimbra’s modified version of Nginx [http://nginx.org/], pronounced like “engine-ex”. And the zimbra-memcached packages is our modified version of memcached [http://memcached.org], pronounced like “memcache-dee”. A third component needed for our proxy environment to work is the Zimbra Proxy Route Lookup Handler or called the “Nginx Lookup Extension” (NLE for short) if some other references. This is a java servlet that is installed from the zimbra-store package on the Zimbra mailbox servers. This servlet handles nginx queries for the user account route information (the server and port number where the user account resides).
 
 
In a typical use case, the proxy services extracts the user login details and then fetches the route to the upstream mail server or web servers’ address from the Zimbra Proxy Route Lookup Handler [the NLE servlet], and finally proxies the interactions between clients and upstream ZCS servers. To accelerate the speed of future route lookups for a user, memcached caches the lookup results. Therefore, the subsequent login with the same username will directly be proxied without calling to the Zimbra Proxy Route Lookup Handler.
 
== Benefits And Reasons To Use ==
 
Source http://wiki.zimbra.com/wiki/Zimbra_Proxy_Guide#Benefits_and_reasons_to_use
 
<span style="color:#DC143C">* Needs to be proofed. </span>
 
# Zimbra Proxy centralizes access to Mailbox servers
#* Zimbra Proxy allows mailbox servers to be hidden from public internet by acting as a reverse proxy & also allowing end users to access mail system via single Login URL instead of knowing their mailbox hostnames. It acts as the first entry point for all the IMAP[S]/POP[S]/HTTP[S] traffic and then intelligently routes all kind of static UI requests (HTML/CSS/JS etc) and Dynamic requests (SOAP/REST/IMAP/POP) to the appropriate upstream server. Here are some other benefits of using the proxy as well
# Load Balancing
#* This is the reverse proxy function that people are most familiar with. Here the proxy routes incoming HTTP requests to a number of identical mail servers. The upstream mail server selection can be based on a simple client IP hash or round-robin algorithm. It’s such a common function that load balancing reverse proxies are usually just referred to as ‘load balancers’. There are specialized load balancing products available, but many general purpose reverse proxies also provide load balancing functionality.
# Security
#* A reverse proxy can hide the topology and characteristics of your back-end servers by removing the need for direct internet access to them. You can place your reverse proxy in an internet facing DMZ, but hide your web servers inside a non-public subnet.
# Authentication
#* You can use your reverse proxy to provide a single point of authentication for all HTTP requests. Although in case of ZCS, the authentication is provided by upstream mailstores where the user accounts reside.
# SSL Termination
#* Here the reverse proxy handles incoming HTTPS connections, decrypting the requests and passing unencrypted requests on to the web servers.
# Caching
#* Currently we use the memcached module with proxy to achieve caching of upstream routes to mailstores on a per end-client basis. This significantly reduces the route lookup time thereby improving the total time required to process the request and boost performance.
# Centralized Logging and Auditing
#* Because all HTTP requests are routed through the reverse proxy, it makes an excellent point for logging and auditing.
# URL Rewriting
#* Sometimes the URL scheme that a legacy application presents is not ideal for discovery or search engine optimization. A reverse proxy can rewrite URLs before passing them on to your back-end servers. To an external customer it appears that they are simply navigating a single website, but internally the organisation is maintaining three entirely separate sites. This approach can work extremely well for web service APIs where the reverse proxy provides a consistent single public facade to an internal distributed component oriented architecture.
 
== Zimbra Proxy Components ==
 
Source: http://wiki.zimbra.com/wiki/Zimbra_Proxy_Guide#Zimbra_Proxy_Components
 
<span style="color:#DC143C">* Needs to be proofed.</span>
 
<span style="color:#DC143C">* Use style guide for hostname references.</span>
 
Zimbra Proxy is designed to provide a HTTP/POP/IMAP proxy that is quick, reliable, and scalable. Zimbra Proxy includes the following:
 
* Nginx. A high performance IMAP[S]/POP[S]/HTTP[S] proxy server which handles all incoming IMAP[S]/POP[S]/HTTP[S] requests.
* Memcached. A high performance, distributed memory object caching system. Route information is cached for further use in order to increase performance.
* Zimbra Proxy Route Lookup Handler. This is a servlet, Nginx Lookup Extension (NLE), located on the ZCS mailbox server - accessed by http://<host>:7072/service/extension/nginx-lookup . This servlet handles queries for the user account route information (the server and port number where the user account resides).
 
== Architecture and Flow ==
 
Source: http://wiki.zimbra.com/wiki/Zimbra_Proxy_Guide#Architecture_and_Flow
 
<span style="color:#DC143C">* Needs to be proofed. </span>
 
<span style="color:#DC143C">* Merge the two sources of information into one. </span>
 
<span style="color:#DC143C">* Reference customer dependencies - Firewall, load-balancer , round-robin DNS to make single-zimbra-hostname use all the proxy servers in question either by ip or dns . </span>
 
The following sequence shows the architecture and flow of Zimbra Proxy.
 
# End clients connect to Zimbra Proxy using IMAP[S]/POP[S]/HTTP[S] ports.
# When Zimbra Proxy receives an incoming connection, the Nginx component sends an HTTP request to Zimbra Proxy Route Lookup Handler component.
# Zimbra Proxy Route Lookup Handler locates the route information for the account being accessed and returns this to Nginx.
# The Memcached component stores the route information for the configured period of time (by default, this time is one hour). Nginx will use this route information instead of querying the Zimbra Proxy Route Lookup Handler until the default period of time has expired.
# Nginx uses the route information to connect to Zimbra Mailbox.
# Zimbra Proxy connects to Zimbra Mailbox and initiates the web/mail proxy session. The end client behaves as if it is connecting directly to Zimbra Mailbox.
 
Source : Admin Guide Draft , 'Configure Zimbra HTTP Proxy'
 
HTTP reverse proxy routes requests as follows:
# If the requesting URL can be examined to determine the user name, then the request is routed to the backend mailbox server of the user in the URL. REST, Ca lDAV, and Zimbra Mobile Sync are supported through this mechanism.
# If the request has an auth token cookie (ZM_AUTH_TOKEN), the request is routed to the backend mailbox server of the authenticated user.
# If the above methods do not work, the IP hash method is used to load balance the requests across the backend mailbox servers which are able to handle the request or do any necessary internal proxying.
 
== Position In ZCS Runtime ==
 
Source: http://wiki.zimbra.com/wiki/Zimbra_Proxy_Guide#Position_in_ZCS_Runtime
 
<span style="color:#DC143C">* Needs to be proofed. </span>
 
<span style="color:#DC143C">* Get graphic working. </span>
 
<span style="color:#DC143C">* Reference customer dependencies - Firewall, load-balancer , round-robin DNS to make single-zimbra-hostname use all the proxy servers in question either by ip or dns . Expand graphic to include this reference. </span>
 
The figure to the right show the positions of NZ and its relationships to other components of ZCS.[[File:Proxy_position_in_ZCS.png|200px|thumb|right|Proxy position in ZCS runtime. Click to see full sized : https://wiki.zimbra.com/images/f/f2/Proxy_position_in_ZCS-2.png]] With Zimbra Proxy, a general workflow of login is like this. On a client login, NZ will attempt to contact a memcached server (the exact server will be elected from the available candidates, using an round-robin algorithm). The memcached server is expected to return the upstream route information for that particular client. If the information is not present in that memcached server, then this will be a cache-miss, so Zimbra Proxy will proceed to contact an available NLE (elected by Round-Robin), to look up the upstream server information.
 
 
Once the upstream server is known, Zimbra Proxy will immediately initiate the proxy session. And then, it will cache the upstream information into the memcached server. The next time the user logs in, the memcached server will have the upstream information available in its cache, and so Zimbra Proxy will not need to contact NLE.
 
== Zimbra Proxy Ports ==
 
<span style="color:#DC143C">* Need reference to memcached port ,  admin ports  ,  zimbraPublicServicePort  . </span>
 
<span style="color:#DC143C">* Do we clarify "open" ports on the firewall also? --Adam </span>
 
Source: http://wiki.zimbra.com/wiki/Zimbra_Proxy_Guide#Zimbra_Proxy_Ports
 
The following ports are used either by Zimbra Proxy or by Zimbra Mailbox (if Proxy is not configured). If you have any other services running on these ports, turn them off.
 
End clients connect directly to Zimbra Proxy, using the Zimbra Proxy Ports. Zimbra Proxy connects to the Route Lookup Handler or Zimbra Mailbox using the Zimbra Mailbox Ports.
 
 
{| style="width:30%" border="1" cellpadding="5" cellspacing="0"
! Zimbra Proxy Ports (External to ZCS)
! Port
|-
| HTTP
| 80
|-
| HTTPS
| 443
|-
| POP3
| 110
|-
| POP3S (Secure POP3)
| 995
|-
| IMAP
| 143
|-
| IMAPS (Secure IMAP)
| 993
|-
! Zimbra Mailbox Ports (Internal to ZCS)
! Port
|-
| Route Lookup Handler
| 7072
|-
| HTTP Backend (if Proxy configured)
| 8080
|-
| HTTPS Backend (if Proxy configured)
| 8443
|-
| POP3 Backend (if Proxy configured)
| 7110
|-
| POP3S Backend (if Proxy configured)
| 7995
|-
| IMAP Backend (if Proxy configured)
| 7143
|-
| IMAPS Backend (if Proxy configured)
| 7993
|}
 
== Deployment Strategies, Server Specifications, Impact To Other Non-Proxy Hosts ==
 
Source: http://wiki.zimbra.com/wiki/Zimbra_Proxy_Guide#Deployment_strategies.2C_server_specifications.2C_impact_to_other_non-proxy_hosts
 
<span style="color:#DC143C">* Needs more details, incomplete right now. </span>
 
 
The deployment strategy Zimbra actively suggests is to use Zimbra Proxy on the edge with mailbox servers behind the firewall. Please note that the Zimbra Proxy package does not act as a firewall and needs to be behind the firewall in customer deployments.


= Installing , Configuring, Disabling the Zimbra Proxy =
= Installing , Configuring, Disabling the Zimbra Proxy =
Moved To:
* https://wiki.zimbra.com/wiki/Zimbra_Proxy_Manual:Installing_,_Configuring,_Disabling_the_Zimbra_Proxy


== Things To Review First ==
= Zimbra Proxy Related CLI Commands =
 
=== Prerequisite Variables To Check First ===
 
==== zimbraPublicServiceHostname zimbraPublicServiceProtocol and zimbraPublicServicePort ====
 
<span style="color:#DC143C">* Needs more details, incomplete right now. </span>
 
In order for the change password link, calendar launching in separate window, and other various functionality to work correctly - meaning, to use the proxy instead of mailbox server, the following LDAP attributes have to be set to the proxy values:
 
* zimbraPublicServiceHostname(Name to be used in public API such as REST or SOAP proxy) - proxy hostname
* zimbraPublicServiceProtocol(Protocol to be used in public API such as REST or SOAP proxy) - proxy protocol (http or https)
* zimbraPublicServicePort(Port to be used in public API such as REST or SOAP proxy) - proxy port
 
==== zimbraVirtualHostname ====
 
<span style="color:#DC143C">* Needs more details, incomplete right now. </span>
 
==== zimbra_auth_always_send_refer ====
 
The zmlocalconfig key zimbra_auth_always_send_refer is now obsolete. Its been replaced by LDAP attribute zimbraMailReferMode. Now with a full-fledged reverse proxy, users do not need to be redirected. The LDAP attribute zimbraMailReferMode is used directly by the Nginx reverse proxy.
 
=== zmtlsctl ===
 
<span style="color:#DC143C">* Needs more details, incomplete right now. </span>
 
zmtlsctl sets the zimbraMailMode , this is different than the zimbraReverseProxyMailMode .
<pre>
zmtlsctl help
Usage: /opt/zimbra/bin/zmtlsctl [mixed|both|http|https|redirect]
</pre>
 
<pre>
$ zmprov desc -a zimbraMailMode               
zimbraMailMode
    whether to run HTTP or HTTPS or both/mixed mode or redirect mode. See
    also related attributes zimbraMailPort and zimbraMailSSLPort
 
              type : enum
              value : http,https,both,mixed,redirect
          callback : LocalBind
          immutable : false
        cardinality : single
        requiredIn :
        optionalIn : globalConfig,server
              flags : serverInherited
          defaults :
                min :
                max :
                id : 308
    requiresRestart :
              since :
    deprecatedSince :
</pre>
 
<pre>
$ zmprov desc -a zimbraReverseProxyMailMode
zimbraReverseProxyMailMode
    whether to run proxy in HTTP, HTTPS, both, mixed, or redirect mode.
    See also related attributes zimbraMailProxyPort and
    zimbraMailSSLProxyPort
 
              type : enum
              value : http,https,both,mixed,redirect
          callback :
          immutable : false
        cardinality : single
        requiredIn :
        optionalIn : globalConfig,server
              flags : serverInherited
          defaults :
                min :
                max :
                id : 685
    requiresRestart : nginxproxy
              since : 5.0.7
    deprecatedSince :
</pre>
 
From a single ZCS 8.5 server install:
 
<pre>
$ zmprov gs `zmhostname` | grep MailMode
zimbraMailMode: https
zimbraReverseProxyMailMode: https
</pre>
 
=== The New WebApp Services In ZCS 8.5 and zm_auth_token ===
 
Source: From admin guide draft under 'Configur Zimbra HTTP Proxy'
 
<span style="color:#DC143C">* Needs more details, incomplete right now. </span>
 
'''Note''' - <<need to add split node feature info in this section. Affects zm_auth_token>> [From Admin Guide Draft]
 
== New ZCS Deployment ==
 
=== Single ZCS Server Environment ===
 
<span style="color:#DC143C">* Needs more details, incomplete right now. </span>
 
 
=== Multi-Server ZCS Environment ===
 
<span style="color:#DC143C">* Needs more details, incomplete right now. </span>
 
== Adding Zimbra Proxy Services To Existing Non-Proxy Environments via ZCS Installer [Recommended Method] ==
=== Using New Servers ===
 
Source: http://wiki.zimbra.com/wiki/Enabling_Zimbra_Proxy#Using_new_servers
 
Here you are installing the proxy on a brand new server and having all your existing mailbox servers being accessed through the proxy on this new server. Simply use the installer script (install.sh) and select the proxy and memcached packages ('Y' by default with ZCS 8.5+, just need to hit enter). This will ask you for LDAP hostname/password, Bind password for nginx ldap user which you need to provide (do 'zmlocalconfig -s ldap_nginx_password' on the host running ldap to get this) and then the Zimbra Proxy configuration menu would be displayed which would look like this.
 
Proxy configuration
 
  1) Status:                                  Enabled                     
  2) Enable POP/IMAP Proxy:                  TRUE                         
  3) IMAP server port:                        7143                         
  4) IMAP server SSL port:                    7993                         
  5) IMAP proxy port:                        143                         
  6) IMAP SSL proxy port:                    993                         
  7) POP server port:                        7110                         
  8) POP server SSL port:                    7995                         
  9) POP proxy port:                          110                         
  10) POP SSL proxy port:                      995                         
  11) Bind password for nginx ldap user:      set                         
  12) Enable HTTP[S] Proxy:                    TRUE                         
  13) Web server HTTP port:                    8080                         
  14) Web server HTTPS port:                  8443                         
  15) HTTP proxy port:                        80                           
  16) HTTPS proxy port:                        443                         
  17) Proxy server mode:                      https         
 
If you need to change any of these intentionally, you can do that now by selecting the corresponding config item from the menu (say for eg. to disable POP/IMAP proxy, select '2' from the above menu). Otherwise, just proceed with all the defaults and you would have the proxy+memcached installed on this new server.
Now, to have all the mailbox servers use the proxy, simply set the zimbraMailReferMode to reverse-proxied on each mailbox server and restart mailboxd to have all the traffic go through the proxy.
 
=== Using Existing Servers ===
 
<span style="color:#DC143C">* Needs more details, incomplete right now. </span>
 
<span style="color:#DC143C">* /opt/zimbra/libexec/zmsetup.pl vs ./install.sh ? --Adam </span>
 
== Adding Zimbra Proxy Services To Existing Non-Proxy Environments via CLI [Advanced Method] ==
=== Using New Servers ===
=== Using Existing Servers ===
 
Source: http://wiki.zimbra.com/wiki/Enabling_Zimbra_Proxy#Using_existing_servers
 
Assuming you are running a 8.0 or earlier version ZCS with no proxy/memcached, zimbraMailMode as https and now want to upgrade to 8.5+ along with adding proxy & memcached, you need to follow the following steps 
Start 8.5+ installer (install.sh script)
 
<span style="color:#DC143C">* /opt/zimbra/libexec/zmsetup.pl vs ./install.sh ? --Adam </span>
Do you wish to upgrade? [Y] y
Install zimbra-memcached [N] y
 
Install zimbra-proxy [N] y
After install is done, enable web/mail proxy, and set the proxy mode and ports:
 
*If localconfig key 'zimbra_require_interprocess_security' is set, Only "https" and "both" are valid modes
/opt/zimbra/libexec/zmproxyconfig -e -w -o -a 8080:80:8443:443 -x <https/both>  -H `zmhostname`
 
*Else if 'zimbra_require_interprocess_security' is unset, Only "http" and "both" are valid modes
/opt/zimbra/libexec/zmproxyconfig -e -w -o -a 8080:80:8443:443 -x <http/both>  -H `zmhostname`
 
*Set the mail proxy ports
/opt/zimbra/libexec/zmproxyconfig -e -m -o -i 7143:143:7993:993 -p 7110:110:7995:995 -H `zmhostname`
 
Now, to have all the mailbox servers use the proxy, simply set the zimbraMailReferMode to reverse-proxied on each mailbox server and restart mailboxd to have all the traffic go through the proxy. Do a 'zmcontrol restart' on this node and you should be up and running.
 
== Manually Modifying Zimbra Proxy Services And Related Variables via CLI ==
 
Source: http://wiki.zimbra.com/wiki/Enabling_Zimbra_Proxy#Manually_Modifying_Proxy_.26_related_Variables_via_CLI
 
=== Simple Command With Defaults ===
 
Source: http://wiki.zimbra.com/wiki/Enabling_Zimbra_Proxy#Simple_Command_With_Defaults
 
The zmproxyconfig command can be run with limited arguments if the command defaults are acceptable. Run /opt/zimbra/libexec/zmproxyconfig to view all the argument options and the usage
 
=== Protocol Requirements Including HTTPS Redirect ===
 
Source: http://wiki.zimbra.com/wiki/Enabling_Zimbra_Proxy#Protocol_Requirements_Including_HTTPS_Redirect
 
HTTP proxy can support protocol modes for HTTP or HTTPS only, both HTTP and HTTPS, mixed HTTP and HTTPS or HTTPS redirect from HTTP. Redirect is a popular configuration. This configuration must be made to the proxy servers.
 
* HTTPS redirect from HTTP
zmprov ms proxy.server.name zimbraReverseProxyMailMode redirect
 
* HTTP and HTTPS (support both)
zmprov ms proxy.server.name zimbraReverseProxyMailMode both
 
* HTTPS only
zmprov ms proxy.server.name zimbraReverseProxyMailMode https
 
* HTTP only
zmprov ms proxy.server.name zimbraReverseProxyMailMode http
 
* "mixed" will cause only authentication to be sent over HTTPS
zmprov ms proxy.server.name zimbraReverseProxyMailMode mixed
 
 
=== Documents & Sharing - The zimbraPublicService variables ===
 
Source: http://wiki.zimbra.com/wiki/Enabling_Zimbra_Proxy#Documents_.26_Sharing
 
It is important to consider access to documents (Briefcase) and shares when setting up HTTP proxy. A publicly reachable address must be configured to be used for the REST and SOAP proxy interfaces otherwise components requiring access to these interfaces will fail. Calendar sharing is an example of one component. Set '''zimbraPublicServiceHostname''', '''zimbraPublicServiceProtocol''', and '''zimbraPublicServicePort''' when applicable. These values are usually not required without proxy since the REST and SOAP proxy interfaces take the value of the Zimbra mailbox service hostname by default. These attributes can be set globally to be inherited by all domains or per domain.
 
Set zimbraPublicServiceHostname to the value of the host that will be used in the URL for access to the HTTP proxy.
* This command sets ''mail.domain.com'' as the public hostname to be used for access to all domains in the Zimbra directory:
zmprov mcf zimbraPublicServiceHostname mail.domain.com
 
* This command sets ''mail.domaina.com'' as the public hostname to be used for access to ''domaina.com'' domain:
zmprov md domaina.com zimbraPublicServiceHostname mail.domaina.com
 
* Set zimbraPublicServiceProtocol to ''http'' or ''https'' depending on the protocol requirements for HTTP proxy:
zmprov md domaina.com zimbraPublicServiceProtocol https
 
* Set zimbraPublicServicePort to the value that corresponds to the HTTP proxy port used in the URL (optional if standard ports 80 or 443 are used for proxy listeners):
zmprov md domaina.com zimbraPublicServicePort 443
 
== Disabling Zimbra Proxy ==
 
=== Completely Disable Proxy In Single ZCS Server Environment ===
 
 
<span style="color:#DC143C">* '''Note''' - Recommend /opt/zimbra/libexec/zmsetup.pl as default method to do this? --Adam </span>
 
=== Completely Disable Proxy In Multi-Server ZCS Environment ===
 
<span style="color:#DC143C">* '''Note''' - Recommend /opt/zimbra/libexec/zmsetup.pl as default method to do this? --Adam </span>
 
=== Disable POP/IMAP Proxy In Single ZCS Server Environment ===
 
Source: http://wiki.zimbra.com/wiki/Ajcody-Proxy-Notes#Need_To_Disable_Pop.2FImap_Proxy_And_Use_POP.2FIMAP_Normally
 
 
<span style="color:#DC143C">* '''Note''' - Recommend /opt/zimbra/libexec/zmsetup.pl as default method to do this? --Adam </span>
 
 
Sometimes, people install/setup proxy services on their single ZCS server and they don't need them. Here's how you would disable the proxy stuff and get imap/pop working over the default ports.
 
<pre>
do a zmprov -l gs `zmhostname` | grep -i port
 
get the ports, then set variables to port 0:
 
zmprov ms `zmhostname` zimbraImapProxyBindPort 0
zmprov ms `zmhostname` zimbraImapSSLProxyBindPort 0
zmprov ms `zmhostname` zimbraPop3ProxyBindPort 0
zmprov ms `zmhostname` zimbraPop3SSLProxyBindPort 0
 
then, set the non "Proxy" ports to the desired standard ports
 
zmprov ms `zmhostname` zimbraImapBindPort 143
zmprov ms `zmhostname` zimbraImapSSLBindPort 993
zmprov ms `zmhostname` zimbraPop3BindPort 110
zmprov ms `zmhostname` zimbraPop3SSLBindPort 995


once complete:
Moved To:
 
*  https://wiki.zimbra.com/wiki/Zimbra_Proxy_Manual:Zimbra_Proxy_Related_CLI_Commands
zmprov ms `zmhostname` -zimbraServiceEnabled memcached
zmprov ms `zmhostname` -zimbraServiceEnabled imapproxy
 
zmproxyctl stop
zmmemcachedctl stop
zmmailboxdctl stop
zmmailboxdctl start
</pre>


= Troubleshooting Zimbra Proxy =
= Troubleshooting Zimbra Proxy =


Source: http://wiki.zimbra.com/wiki/Zimbra_Proxy_Guide#Troubleshooting
Moved TO:
* https://wiki.zimbra.com/wiki/Zimbra_Proxy_Manual:Troubleshooting_Zimbra_Proxy


== Proxy Related Log Files ==
= Advance Topics For Zimbra Proxy - Configuration And Template Files And Proxy Related Variables =


Source: http://wiki.zimbra.com/wiki/Zimbra_Proxy_Guide#Troubleshooting
https://wiki.zimbra.com/wiki/Zimbra_Proxy_Manual:Configuration_And_Template_Files_And_Proxy_Related_Variables


Proxy related errors can be found in the following logs on each proxy server:
Merged and Updated the following pages below and then set a REDIRECT to the main page above:
* https://wiki.zimbra.com/wiki/NGINX_Configuration_Structure
* https://wiki.zimbra.com/wiki/NGINX_Configuration_Directive_Reference
* https://wiki.zimbra.com/wiki/Zimbra_Proxy_Attributes-Detailed


* /opt/zimbra/log/nginx.log
= Advance Topics For Zimbra Proxy - Advanced Proxy Configuration Examples via CLI=
* /opt/zimbra/log/nginx.access.log


== Slow Proxy Logins - Route Lookup Handles (garpu) ==
Created :
 
* https://wiki.zimbra.com/wiki/Zimbra_Proxy_Manual:Advanced_Proxy_Configuration_Examples_via_CLI
Source:
* http://wiki.zimbra.com/wiki/Enabling_Zimbra_Proxy#Proxy_Login_Slow
* http://wiki.zimbra.com/index.php?title=NGINX_Configuration_Tips#View.2FModify_Route_Lookup_Handlers_.28garpu.29
 
A common nginx misconfiguration is to have incorrectly designated non-mailbox servers as routing/zmlookup handlers. Only mailbox servers can perform route handler functions. To view the zmlookup lookup handlers, review the zm_lookup_handlers parameter in /opt/zimbra/conf/nginx/includes/nginx.conf.zmlookup
<pre>
grep zm_lookup_handlers /opt/zimbra/conf/nginx/includes/nginx.conf.zmlookup
</pre>
 
If a non-mailbox server is listed, set the zimbraReverseProxyLookupTarget server configuration attribute to FALSE for that server.
<pre>
zmprov ms `zmhostname` zimbraReverseProxyLookupTarget FALSE
</pre>
 
Additionally, zimbraReverseProxyLookupTarget is a server inherited attribute from the global configuration, so check if zimbraReverseProxyLookupTarget has been incorrectly designated in global config.
<pre>
zmprov gcf zimbraReverseProxyLookupTarget
</pre>
 
== No Route To Host Errors ==
 
Source:  
* http://wiki.zimbra.com/wiki/Ajcody-Proxy-Notes#No_Route_To_Host
* http://wiki.zimbra.com/wiki/Zimbra_Proxy_Guide#No_route_to_host_errors
 
If your getting "No route to host" errors in /opt/zimbra/log/nginx.log files on the proxy servers, you should check:
 
# If the resolution [DNS] to the host from the proxy servers is correct and working. This oversight might when you've deployed a new ZCS server in your environment.
# Following step one, you also have to confirm there is port level access between proxy servers and the server it's trying to reach. memcache port is 11211 .
# The server/s [mailbox/webapp servers] are too busy to serve the request.
# You have a server in the lookup pool list that shouldn't be there or you should remove the trouble server from the pool to avoid any more customer issues to deescalate the situation.
 
== 5xx Errors ==
 
Source:
* http://wiki.zimbra.com/wiki/Ajcody-Proxy-Notes#50X_Errors
* http://wiki.zimbra.com/wiki/Zimbra_Proxy_Guide#5xx_Errors
 
* 500 Internal Server Error:
** A generic error message, given when an unexpected condition was encountered and no more specific message is suitable. This might be due to several reasons due to failure on one of the upstream mailstore servers.
* 502 Bad Gateway:
** The server was acting as a gateway or proxy and received an invalid response from the upstream server.
* 503 Service Unavailable:
** The most common reason for this is that the jetty mailboxd process is down on the mailstore server and hence is unable to process the request. Check using 'netstat -an | grep <port>' to see if the IMAP/POP/HTTP ports used by mailboxd are up and listening for incoming connections. Also check using 'ps -eaf | grep mailboxd'
* 504 Gateway Timeout:
** The most common reason for this is that the upstream mailstore server took longer than the configured timeout value and hence the client closed the upstream connection. Try changing the timeout values to a higher value if its expected to take longer for specific kind of requests. If this is not the case, then we should look for reasons why it took longer for a request to get processed.
 
These proxy configuration changes can improve proxy behavior related to timeouts:
 
* This will configure proxy to immediately reconnect on any failure
** <pre>$ zmprov mcf zimbraMailProxyReconnectTimeout 0 </pre>
** If necessary, on each proxy
*** <pre>$ zmprov ms `zmhostname` zimbraMailProxyReconnectTimeout 0 </pre>
* This will configure proxy to ignore failures in regards to disconnects
** <pre>$ zmprov mcf zimbraMailProxyMaxFails 0 </pre>
** If necessary, on each proxy
*** <pre>$ zmprov ms `zmhostname` zimbraMailProxyMaxFails 0 </pre>
* Then, restart the processes on all proxies:
* <pre> $ zmproxyctl restart </pre>
 
See the following bug for more details on this recommendation:
* "Improved proxy timeout defaults"
** https://bugzilla.zimbra.com/show_bug.cgi?id=80135
 
== Third Party Issues - Load Balancers, DNS, Etc. ==
 
== Bad/Invalid command when proxying to external POP/IMAP servers ==
 
Source : http://wiki.zimbra.com/index.php?title=NGINX_Configuration_Tips#Bad.2FInvalid_command_when_proxying_to_external_POP.2FIMAP_servers
 
 
Nginx issues the XOIP command to the upstream POP3 server, and the ID command to the upstream IMAP server, before logging in to upstream. This is for auditing purposes so that the client's IP address is known to the upstream server. The global config attributes '''zimbraReverseProxySendPop3Xoip''' and '''zimbraReverseProxySendImapId''' control this aspect.
 
However, some external IMAP servers may not implement the ID command, and some external POP3 servers may not implement the XOIP command.
 
To turn off sending the XOIP command, set zimbraReverseProxySendPop3Xoip to false. To turn off sending the IMAP command, set zimbraReverseProxySendImapId to false.
 
: zmprov mcf zimbraReverseProxySendPop3Xoip FALSE
: zmprov mcf zimbraReverseProxySendImapId FALSE
 
== LDAP/Nginx Won't Start And Asks For A Password ==
 
Source: http://wiki.zimbra.com/wiki/LDAP/Nginx_won%27t_start_and_asks_for_a_password
 
This tends to happen because Zimbra can't read the key or the certificate of the LDAP/IMAP proxy service on startup. This issue is specific to commercial certificates.
 
Here is what you can do:
 
* You want to check to make sure the private key is not encrypted.
* Manually link the cert to the services OR simply redeploy the certificate to relink the cert to the services.
 
cp /opt/zimbra/ssl/zimbra/commercial/commercial.key /opt/zimbra/conf/slapd.key<br>
cp /opt/zimbra/ssl/zimbra/commercial/commercial.crt /opt/zimbra/conf/slapd.crt<br>
cp /opt/zimbra/ssl/zimbra/commercial/commercial.key /opt/zimbra/conf/nginx.key<br>
cp /opt/zimbra/ssl/zimbra/commercial/commercial.crt /opt/zimbra/conf/nginx.crt<br>
 
* Restart the Zimbra services.


= Advance Topics For Zimbra Proxy - Miscellaneous Topics =
= Advance Topics For Zimbra Proxy - Miscellaneous Topics =


== Miscellaneous Topics ==
Moved To:
 
* https://wiki.zimbra.com/wiki/Zimbra_Proxy_Manual:Miscellaneous_Topics
===  Set Up Proxy to use Clear Text for Upstream Connections ===
 
Source : Admin Guide Draft, 'Set Up Proxy to use Clear Text for Upstream Connections'
 
When setting up the proxy to use clear text for upstream connections, set zimbraReverseProxySSLToUpstreamEnabled to FALSE.
 
This attribute defaults to TRUE. In an "out of the box" proxy set up, the upstream communication defaults to SSL.
 
=== REST URL Generation ===
 
Source : Admin Guide Draft, 'REST URL Generation'
 
For REST URL, you set the host name, service protocol, and services port globally or for a specific domain from the following attributes.
 
* zimbraPublicServiceHostname
* zimbraPublicServiceProtocol
* zimbraPublicServicePort
 
When generating REST URL’s:
 
* If domain.zimbraPublicServiceHostname is set, use zimbraPublicServiceProtocol + zimbraPublicServiceHostname + zimbraPublicServicePort
* Otherwise it falls back to the server (account's home server) attributes:
** protocol is computed from server.zimbraMailMode
** hostname is server.zimbraServiceHostname
** port is computed from the protocol.
 
'''Note:''' Why use zimbraMailReferMode - In earlier versions, a local config variable called zimbra_auth_always_send_refer determined which action the back-end server took when a user’s mailbox did not reside on the server that the user logged in to. The default value of FALSE redirected the user if the user was logging in on the wrong backend host.
 
On a multiserver ZCS, if a load balanced name was needed to create a friendly landing page, a user would always have to be redirected. In that case, zimbra_auth_always_send_refer was set to TRUE.
 
Now with a full-fledged reverse proxy, users do not need to be redirected. The localconfig variable zimbraMailReferMode is used with nginx reverse proxy.
 
=== Set Proxy Trusted IP Addresses ===
 
Source : Admin Guide Draft, 'Set Proxy Trusted IP Addresses'
 
When a proxy is configured with ZCS, each proxy server’s IP address must be configured in LDAP attribute zimbraMailTrustedIP to identify the proxy addresses as trusted when users log in through the proxy. The proxy IP address is added to the X-Forwarded-For header information. The X-Forwarded-For header is automatically added to the localconfig zimbra_http_originating_ip_header attribute. When a user logs in, this IP address and the user’s address are verified in the Zimbra mailbox log.
 
Set each proxy IP address in the attribute. For example, if you have two proxy servers:
zmprov mcf +zimbraMailTrustedIP {IP of nginx-1} +zimbraMailTrustedIP {IP of nginx-2}
 
'''Note:''' To verify that X-Forwarded-For was correctly added to the localconfig, type zmlocalconfig | grep -i http. You should see zimbra_http originating_ip_header = X-Forwarded-For.
 
== Zimbra Proxy Performance Tuning ==
 
See the following unresolved RFE:
 
* "add nginx section to the performance tunning guide"
** https://bugzilla.zimbra.com/show_bug.cgi?id=26418
 
== Zimbra Proxy Related CLI Commands ==
 
=== zmproxyconfig ===
 
----
 
Source: http://wiki.zimbra.com/wiki/Zimbra_Proxy_Guide#Proxy_config_rewrite.28zmproxyconfgen.29.2C_zmproxyconfig_and_zmproxyctl
 
==== Syntax And Usage ====
 
/opt/zimbra/libexec/zmproxyconfig help
 
Usage: /opt/zimbra/libexec/zmproxyconfig [-h] [-o] [-m] [-w] [-d [-r] [-s] [-a w1:w2:w3:w4] [-c [-n n1:n2]] [-i p1:p2:p3:p4] [-p p1:p2:p3:p4] [-x mailmode]] [-e [-a w1:w2:w3:w4] -C] [-n n1:n2 [-i p1:p2:p3:p4] [-p p1:p2:p3:p4] [-u|-U] [-x mailmode]] [-f] -H hostname
 
==== Description ====
 
-h: display this help message
 
-H: Hostname of server on which enable/disable proxy functionality.
 
-a: Colon separated list of Web ports to use. Format: HTTP-STORE:HTTP-PROXY:HTTPS-STORE:HTTPS-PROXY (Ex: 8080:80:8443:443)
 
-d: disable proxy
 
-e: enable proxy
 
-f: Full reset on memcached port and search queries and POP/IMAP throttling.
 
-i: Colon separated list of IMAP ports to use. Format: IMAP-STORE:IMAP-PROXY:IMAPS-STORE:IMAPS-PROXY (Ex: 7143:143:7993:993)
 
-m: Toggle mail proxy portions
 
-o: Override enabled checks
 
-p: Colon separated list of POP ports to use. Format: POP-STORE:POP-PROXY:POPS-STORE:POPS-PROXY (Ex: 7110:110:7995:995)
 
-r: Run against a remote host.  Note that this requires the server to be properly configured in the LDAP master.
 
-s: Set cleartext to FALSE (secure mode) on disable
 
-t: Disable reverse proxy lookup target for store server.  Only valid with -d.  Be sure that you intend for all proxy function for the server to be disabled


-w: Toggle Web proxy portions
= Ports Scratchpad =


-c: Disable Admin Console proxy portions.
To see ports available on your server, you can do as '''root''' :


-C: Enable Admin Console proxy portions.
netstat -anltp | egrep '^tcp' | grep LISTEN | awk '{print $4 " "$7}' | sed -e 's/.*://' | sort -n | uniq


-n: Colon separated list of Admin Console ports to use. Format: ADMIN-CONSOLE-STORE:ADMIN-CONSOLE-PROXY (Ex: 7071:9071)
{| class="wikitable sortable"
 
! Port !! If Proxied [Defaults] !! PID Name !! Pid Name If Proxied !! Package Name !! Package Name If Proxied !! zmprov related Variables !! Description !! Comments !! Binds To localhost Or Network Interface !! Open Or Routed Through Firewall
-x: the proxy mail mode when enable proxy, or the store mail mode when disable proxy (Both default: http).
|- style="background:white; color:black"
 
| 22 ||  || sshd ||  || sshd - from OS ||  || zimbraRemoteManagementPort || Remote Management Port ||  ||  ||
-u: disable SSL connection from proxy to mail store.
|- style="background:white; color:black"
 
| 25 ||  || master ||  || mta ||  || zimbraSmtpPort || SMTP || Incoming mail to postfix ||  ||
-U: enable SSL connection from proxy to mail store.
|- style="background:white; color:black"
 
| 53 ||  || unbound ||  || dnscache ||  || || DNS Cache Server || Comments || localhost ||
hostname is the value of the zimbra_server_hostname LC key for the server being modified.
|- style="background:white; color:black"
 
| 80 || 8080 || java || nginx  || store || proxy || zimbraMailPort ; '''''zimbraMailProxyPort'''''  || HTTP ; '''''HTTP Backend (when proxied)''''' || Comments ||  ||
Required options are -f by itself, or -f with -d or -e.
|- style="background:white; color:black"
 
| 443 || 8443 || java || nginx || store || proxy || zimbraMailSSLPort ; '''''zimbraMailSSLProxyPort''''' || HTTPS ; '''''HTTPS Backend (when proxied''''') || Comments ||  ||
Note that -d or -e require one or both of -m and -w.
|- style="background:white; color:black"
 
|  || 11211 ||  || memcached ||  || proxy || zmprov related Variables || Memcached || Comments ||  ||
Note that -i or -p require -m.
|- style="background:white; color:black"
 
| 7072 ||  || java ||  || store ||  ||  || Route Lookup Handler || ZCS Nginx Lookup (backend http service for nginx lookup/authentication) ||  ||
Note that -a requires -w.
|- style="background:white; color:black"
 
| 3443 || 9443 || ? || nginx || ? || nginx || zimbraMailSSLClientCertPort ; '''''zimbraMailSSLProxyClientCertPort''''' || Mail Client Cert ; '''''Mail Client Cert Backend (when proxied)''''' || Comments ||  ||
Note that -c/-C requires -w, and -n requires -c/-C. When disabling web proxy, admin console proxy will be automatically disabled.
|- style="background:white; color:black"
 
| 110 || 7110 || java || nginx || store || nginx || zimbraMailProxyPort ; '''''zimbraMailSSLProxyPort''''' || POP3 ; '''''POP3 Backend (when proxied)''''' || Comments ||  ||
Note that -u or -U are only available when proxy is enabled by -e.
|- style="background:white; color:black"
 
| 995 || 7995 || java || nginx || store || nginx || zimbraPop3SSLBindPort ; '''''zimbraPop3SSLProxyBindPort''''' || POP3S (Secure POP3) ; '''''POP3S Backend (when proxied)''''' || POP over SSL ||  ||
Note that -x requires -w and -d for store.
|- style="background:white; color:black"
 
| 143 || 7143 || java || nginx || store || nginx || zimbraImapBindPort ; '''''zimbraImapProxyBindPort''''' || IMAP ; '''''IMAP Backend (when proxied)''''' || Comments ||  ||
Note that -x requires -w for proxy.
|- style="background:white; color:black"
 
| 993 || 7993 || java || nginx || store || nginx || zimbraImapSSLBindPort ; '''''zimbraImapSSLProxyBindPort''''' || IMAPS (Secure IMAP) ; '''''IMAPS Backend (when proxied)''''' || IMAP over SSL ||  ||
Note that no matter what mail mode is set by -x and no matter proxy is enabled or disabled, admin console's mode is always https.
|- style="background:white; color:black"
 
| 7071 || 9071 || java || nginx || store || nginx || zimbraAdminPort ; '''''zimbraAdminProxyPort''''' ; '''''zimbraReverseProxyAdminEnabled [default FALSE]''''' || Admin Console ; '''''Admin Console Through Proxy [If Enabled]''''' || HTTPS [nginx => mailbox when enabled]  ||  ||
The following are the defaults for -a, -i, -p, and -x if they are not supplied as options.
|- style="background:white; color:black"
 
| 465 ||  || master ||  || mta ||  ||  || SMTPS || Incoming mail to postfix over ssl (Legacy Outlook only?) If possible, use 587 instead) ||  ||
-a default on enable: 8080:80:8443:443
|-  style="background:white; color:black"
 
| 587 ||  || master ||  || mta ||  ||  || SMTP || Mail submission over TLS || ||
-a default on disable: 80:0:443:0
|- style="background:white; color:black"
 
| 3310 ||  || clamd ||  || mta ||  || zimbraClamAVListenPort || ClamAV || Comments ||  ||
-i default on enable: 7143:143:7993:993
|-  style="background:white; color:black"
 
| 7025 ||  || java ||  || store ||  || zimbraLmtpBindPort || LMTP || Local mail delivery ||  ||
-i default on disable: 143:7143:993:7993
|-  style="background:white; color:black"
 
| 8465 ||  || opendkim ||  || mta ||  || zmprov related Variables || Description || Comments ||  ||
-p default on enable: 7110:110:7995:995
|- style="background:white; color:black"
 
| 10024 ||  || amavisd ||  || mta ||  || zmprov related Variables || SMTP || To Amavis from Postfix || ||
-p default on disable: 110:7110:995:7995
|-  style="background:white; color:black"
 
| 10025 ||  || master ||  || mta ||  || zmprov related Variables || SMTP || To Postfix from Amavis ||  ||
-n default on enable: 7071:9071
|- style="background:white; color:black"
 
| 10026 ||  || amavisd ||  || mta ||  || zmprov related Variables || Description || Comments ||  ||
-n default on disable: 7071:9071
|style="background:white; color:black"
 
| 10027 ||  || master ||  || mta ||  || zmprov related Variables || Description || Comments ||  ||
-x default on store disable: http
|-  style="background:white; color:black"
 
| 10028 ||  || master ||  || mta ||  || zmprov related Variables || Description || Comments ||  ||
-x default on proxy enable/disable: http, but -x default on proxy enable when upstream ssl connection is enabled: https
|-  style="background:white; color:black"
 
| 10029 ||  || master ||  || mta ||  || zmprov related Variables || Description || Comments ||  ||
eg. To change the value of zimbraReverseProxySSLToUpstreamEnabled from TRUE(default) to FALSE (i.e use http for the proxy<->mailstore connections instead of https), execute this on the server running the proxy
|-  style="background:white; color:black"
 
| 10030 ||  || master ||  || mta ||  || zmprov related Variables || Description || Comments ||  ||
/opt/zimbra/libexec/zmproxyconfig -e -m -w -H <hostname> -u
|- style="background:white; color:black"
 
| 10031 ||  ||  ||  || mta ||  || zimbraCBPolicydBindPort || CB Policy || Comments ||  ||
You will have to restart the proxy after this by running 'zmproxyctl restart'
|- style="background:white; color:black"
 
| 10032 ||  || amavisd ||  || mta ||  || zmprov related Variables || Description || Comments ||  ||
*Note: zmproxyconfig is no longer required to enable the proxy if you are doing a fresh installation of ZCS with proxy. The installer already runs this and the proxy is enabled for imap/pop/https access by default. Although it is needed while trying to deploy proxy in existing non-proxy environments when using the existing servers. Kindly refer http://wiki.zimbra.com/wiki/Enabling_Zimbra_Proxy#Using_existing_servers
|- style="background:white; color:black"
 
| 389 ||  || slapd ||  || ldap ||  ||  || LDAP || Comments ||  ||
=== zmproxyctl ===
|- style="background:white; color:black"
 
| 636 ||  || slapd ||  || ldap ||  ||  || LDAPS || If enabled. ||  ||
----
|- style="background:white; color:black"
 
| 7047 ||  || httpd ||  || convertd ||  || zmprov related Variables || Conversion server || Comments ||  ||
==== Syntax ====
|- style="background:white; color:black"
 
| 7306 ||  || mysqld ||  || store ||  || zmprov related Variables || Mysql || Comments ||  ||
zmproxyctl help
|- style="background:white; color:black"
 
| 7780 ||  || httpd ||  || spell ||  || zmprov related Variables || Spell check || Comments ||  ||
Usage : /opt/zimbra/bin/zmproxyctl start|stop|restart|reload|status
|- style="background:white; color:black"
 
| Port || If Proxied || PID Name || Pid Name If Proxied || Package Name || Package Name If Proxied || zmprov related Variables || Description || Comments ||  ||
==== Description ====
 
=== zmprov ===
 
----
 
==== Syntax ====
 
zmprov [cmd]
 
==== Description ====
 
{| style="width:50%" border="1" cellpadding="5" cellspacing="0"
! Long Name
! Short Name
! Description
|-
| --getAllReverseProxyURLs
| -garpu
| Used to list all the upstream mailstore servers (NLEs) that should be used for reverse proxy lookup by the proxy
|-
| --getAllReverseProxyBackends
| -garpb
| Used to list all the upstream mailstore servers that are reverse-proxied by the proxy
|-
| --getAllMtaAuthURLs
| -gamau
| Used to publish into saslauthd.conf the servers that should be used for saslauthd.conf MTA auth
|-
| --getAllMemcachedServers
| -gamcs
| Used to list memcached servers (for Zimbra Proxy use)
|}
|}
 
{{Article Footer|Zimbra Collaboration 8.0, 7.0|04/16/2014}}
=== zmproxyconfgen ===


----
----


==== Syntax ====
[[Category: Community Sandbox]]
 
[[Category: Author:Ajcody]]
==== Description ====
 
 
=== zmnginxconf ===
 
----
 
==== Syntax ====
 
==== Description ====
 
Will be functional again with [https://bugzilla.zimbra.com/show_bug.cgi?id=95169 Bug 95169] being fixed.
 
=== zmproxypurge ===
 
----
 
==== Syntax ====
 
==== Description ====
 
=== zmproxyinit [ deprecated since ???] ===
 
----
 
==== Description ====
 
This command is no longer used on any supported versions of ZCS. See zmproxyconfig .
 
== Details On Some Of The Zimbra Customization To The Proxy Components ==
 
Source : [Internal Eng. Wiki /index.php/Zimbra/Admin/Proxy/Overview#Motivation_behind_Expanding_Nginx ]
 
===Motivation behind Expanding Nginx===
 
Although official Nginx is powerful, it can't meed all the needs of Zimbra.
 
===AUTH_HTTP===
 
Nginx's AUTH_HTTP protocol (described in http://wiki.nginx.org/NginxMailCoreModule#Authentication) is borrowed to implement the lookup with NLE. However, official Nginx's mail AUTH_HTTP config can only specify one NLE URL (auth_http directive). We wish to extend this support so that NZ can use more than one NLE. When a great load of user logins come, NZ can dispatch the load to multiple NLE running in different zimbra servers. This will end up not overloading a single NLE.  In a production set-up, it is likely that there will be many different NZ identically configured, and so the relationship between the NZ set and the server set running NLE is a many-to-many relationship.
 
Besides, the lookup is also necessary for zimbra web accessing, which is not supported by official Nginx. Therefore, this lookup and upstream choose function must also be implemented as an nginx http module.
 
Finally, because Zimbra doesn't use AUTH_HTTP protocol for real authentication, it might add some custom HTTP headers (such as "Auth-ID") and avoid some useless content transmit (such as password via "Auth-Pass").
 
All of these above have to be extended by Zimbra.
 
===Memcache Module===
 
It is likely that a single user may log in many times via NZ in a short time, or one http request will generate a great many of sub requests, all of which contains the same login information and thus will get the same route lookup result. In such cases, NZ will have to contact the NLE many times. It is not likely that the upstream server information for a particular client will change frequently. (A change in the upstream server usually corresponds to a user's mailbox being migrated to a different server). And at the same time NLE is relative much slower than Nginx, becoming the bottle neck.
 
As such, it will be beneficial if NZ is able to cache such information so that repeated trips to the NLE servers are avoided. Also, this cached information must be made available to possibly multiple NZ instances. Hence, "memcached" (http://www.danga.com/memcached/) was introduced for this purpose. Memcached is a high-performance, distributed memory object caching system.
 
Although official Nginx has a build-in "ngx_http_memcache" module, it can only be used for "URL->web page" cache. Therefore, a global "ngx_memcache" module is implemented in NZ which could be accessed by both web and mail modules. Memcached is used to cache 5 kinds of information as follows. The 4th and 5th are used in mail throttle control, which will be described in [Zimbra/Admin/Proxy/Throttle Control]:
* alias-->account name
* account name->route information
* account id-->route information
* login IP address->login count (IP throttle control)
* account name->login count (user throttle control)
 
===Enhanced Mail Protocol Support===
 
Compared to the web proxy part, official nginx is relative weak in mail proxy, such as:
* Official nginx cannot completely support IMAP ID command, which is required for the logging of upstream zimbra servers
* Official nginx can't echo the [ALERT] msg before authenticating to the upstream server
* Official nginx does not support GSSAPI authentications.
* Official nginx does not support upstream SSL connection, that is, using POP3S and IMAPS protocol connecting nginx and upstream server.
 
These features are critical for a mail system of enterprise level. ZCS' mail server support these, so NZ has to support them as well.
 
Unlike web proxy, during which nginx simply proxies all the requests, in mail proxy nginx has to handle some mail protocol commands rather than proxy them to upstream before login is given. The reason is before getting route from NLE, NZ doesn't know to use which upstream server. These commands include "ID", "NOOP", "LOGIN", "CAPABILITY", "AUTHENTICATE", "LOGOUT", "STARTTLS" for IMAP, and "USER", "CAPA", "PASS", "NOOP", "STLS", "QUIT" for POP3. Although official nginx support most of them, but it has several bugs (like IMAP tag/untag responses), which are fixed by NZ.
 
SMTP proxy os nginx is not used in Zimbra so this part is not touched. Besides, some authentication methods supported by official nginx like SASL LOGIN, SALS CRAM-MD5 and APOP won't work because NLE can't support them.
 
===Throttle Control===
 
To reject DoS like attack, NZ will count the number of logins per IP/username in periods and deny the login if it's beyond a certain configured threshold value. This functionality is called Throttle Control. There are 2 kinds of them: IP Throttle Control and User Throttle Control. IP Throttle Control counts the login by client's IP address. But sometimes, this throttle control is not suitable for some cases (for example, many clients who locate in an internal network and thus have the same outside IP address of the gateway). Therefore, User Throttle Control is also introduced, which counts the login by client's user name. All these counters are stored in memcached so it's not persistent. And its algorithm is not quite accurate, but simple and feasible in practice. It's beneficial to implement Throttle Control in NZ because it can block the attacks before they touch real upstream mail servers.
 
Currently Throttle Control is only implemented for mail proxy. For web proxy, users can manually configure nginx with "deny" & "allow" directive. The following LDAP attributes have to be set to use this feature which get written using the corresponding nginx directives added for this module in /opt/zimbra/conf/nginx/templates/nginx.conf.mail.template configuration file.
zimbraReverseProxyIPLoginLimit    ->  mail_login_ip_max              ${mail.ipmax};
zimbraReverseProxyIPLoginLimitTime    ->  mail_login_ip_ttl              ${mail.ipttl};
zimbraReverseProxyIpThrottleMsg    ->  mail_login_ip_rejectmsg        "${mail.iprej}";
zimbraReverseProxyUserLoginLimit    ->  mail_login_user_max            ${mail.usermax};
zimbraReverseProxyUserLoginLimitTime    ->  mail_login_user_ttl            ${mail.userttl};
zimbraReverseProxyUserThrottleMsg    ->  mail_login_user_rejectmsg      "${mail.userrej}";
 
===Cert Per Virtual Host Name===
 
Many ZCS host thousands of domains, each of which has several virtual host names. Each domain may own its own SSL certificate/private key. If nginx is deployed, nginx can't proxy the SSL handshake to the upstream. So NZ has to support this "cert per domain" feature. Actually, the upstream Jetty server doesn't implement this feature. Therefore, NZ is critical in this use case. See the following for [[Ajcody-Proxy-Guide-Rewrite-Project#SSL_Certificates_Per_Domain_Set_Up|SSL Certificates Per Domain Set Up]].
 
= Advance Topics For Zimbra Proxy - Advanced Proxy Configuration Examples via CLI=
 
Created :
* https://wiki.zimbra.com/wiki/Advance_Topics_For_Zimbra_Proxy_-_Advanced_Proxy_Configuration_Examples_via_CLI

Latest revision as of 22:00, 7 June 2016

Ajcody Proxy Guide Rewrite Project

   KB 21113        Last updated on 2016-06-7  




0.00
(0 votes)

Overview And Planning For Zimbra Proxy

Moved to:

Installing , Configuring, Disabling the Zimbra Proxy

Moved To:

Zimbra Proxy Related CLI Commands

Moved To:

Troubleshooting Zimbra Proxy

Moved TO:

Advance Topics For Zimbra Proxy - Configuration And Template Files And Proxy Related Variables

https://wiki.zimbra.com/wiki/Zimbra_Proxy_Manual:Configuration_And_Template_Files_And_Proxy_Related_Variables

Merged and Updated the following pages below and then set a REDIRECT to the main page above:

Advance Topics For Zimbra Proxy - Advanced Proxy Configuration Examples via CLI

Created :

Advance Topics For Zimbra Proxy - Miscellaneous Topics

Moved To:

Ports Scratchpad

To see ports available on your server, you can do as root :

netstat -anltp | egrep '^tcp' | grep LISTEN | awk '{print $4 " "$7}' | sed -e 's/.*://' | sort -n | uniq
Port If Proxied [Defaults] PID Name Pid Name If Proxied Package Name Package Name If Proxied zmprov related Variables Description Comments Binds To localhost Or Network Interface Open Or Routed Through Firewall
22 sshd sshd - from OS zimbraRemoteManagementPort Remote Management Port
25 master mta zimbraSmtpPort SMTP Incoming mail to postfix
53 unbound dnscache DNS Cache Server Comments localhost
80 8080 java nginx store proxy zimbraMailPort ; zimbraMailProxyPort HTTP ; HTTP Backend (when proxied) Comments
443 8443 java nginx store proxy zimbraMailSSLPort ; zimbraMailSSLProxyPort HTTPS ; HTTPS Backend (when proxied) Comments
11211 memcached proxy zmprov related Variables Memcached Comments
7072 java store Route Lookup Handler ZCS Nginx Lookup (backend http service for nginx lookup/authentication)
3443 9443 ? nginx ? nginx zimbraMailSSLClientCertPort ; zimbraMailSSLProxyClientCertPort Mail Client Cert ; Mail Client Cert Backend (when proxied) Comments
110 7110 java nginx store nginx zimbraMailProxyPort ; zimbraMailSSLProxyPort POP3 ; POP3 Backend (when proxied) Comments
995 7995 java nginx store nginx zimbraPop3SSLBindPort ; zimbraPop3SSLProxyBindPort POP3S (Secure POP3) ; POP3S Backend (when proxied) POP over SSL
143 7143 java nginx store nginx zimbraImapBindPort ; zimbraImapProxyBindPort IMAP ; IMAP Backend (when proxied) Comments
993 7993 java nginx store nginx zimbraImapSSLBindPort ; zimbraImapSSLProxyBindPort IMAPS (Secure IMAP) ; IMAPS Backend (when proxied) IMAP over SSL
7071 9071 java nginx store nginx zimbraAdminPort ; zimbraAdminProxyPort ; zimbraReverseProxyAdminEnabled [default FALSE] Admin Console ; Admin Console Through Proxy [If Enabled] HTTPS [nginx => mailbox when enabled]
465 master mta SMTPS Incoming mail to postfix over ssl (Legacy Outlook only?) If possible, use 587 instead)
587 master mta SMTP Mail submission over TLS
3310 clamd mta zimbraClamAVListenPort ClamAV Comments
7025 java store zimbraLmtpBindPort LMTP Local mail delivery
8465 opendkim mta zmprov related Variables Description Comments
10024 amavisd mta zmprov related Variables SMTP To Amavis from Postfix
10025 master mta zmprov related Variables SMTP To Postfix from Amavis
10026 amavisd mta zmprov related Variables Description Comments
10027 master mta zmprov related Variables Description Comments
10028 master mta zmprov related Variables Description Comments
10029 master mta zmprov related Variables Description Comments
10030 master mta zmprov related Variables Description Comments
10031 mta zimbraCBPolicydBindPort CB Policy Comments
10032 amavisd mta zmprov related Variables Description Comments
389 slapd ldap LDAP Comments
636 slapd ldap LDAPS If enabled.
7047 httpd convertd zmprov related Variables Conversion server Comments
7306 mysqld store zmprov related Variables Mysql Comments
7780 httpd spell zmprov related Variables Spell check Comments
Port If Proxied PID Name Pid Name If Proxied Package Name Package Name If Proxied zmprov related Variables Description Comments
Verified Against: Zimbra Collaboration 8.0, 7.0 Date Created: 04/16/2014
Article ID: https://wiki.zimbra.com/index.php?title=Ajcody-Proxy-Guide-Rewrite-Project Date Modified: 2016-06-07



Try Zimbra

Try Zimbra Collaboration with a 60-day free trial.
Get it now »

Want to get involved?

You can contribute in the Community, Wiki, Code, or development of Zimlets.
Find out more. »

Looking for a Video?

Visit our YouTube channel to get the latest webinars, technology news, product overviews, and so much more.
Go to the YouTube channel »




Jump to: navigation, search