Difference between revisions of "Ajcody-Proxy-Guide-Rewrite-Project"

(zimbraPublicServiceHostname zimbraPublicServiceProtocol and zimbraPublicServicePort)
m (Overview And Planning)
Line 1: Line 1:
= Overview And Planning =
+
= Overview And Planning For Zimbra Proxy=
== Overview Of Proxy And Related Components ==
+
 
 +
Source: http://wiki.zimbra.com/wiki/Zimbra_Proxy_Guide
 +
 
 +
== What Is Zimbra Proxy ==
 +
 
 +
Source: http://wiki.zimbra.com/wiki/Zimbra_Proxy_Guide#What_is_Zimbra_Proxy
  
 
Zimbra Proxy is a high-performance reverse proxy service for passing IMAP/POP3/HTTP[S] client requests to other internal ZCS services. A reverse proxy server is an Internet-facing services that protects and manages client connections to your internal services. It can also provide functions like: GSSAPI authentication, throttle control, SSL connection with different certificates for different virtual host names, and other features described later below.
 
Zimbra Proxy is a high-performance reverse proxy service for passing IMAP/POP3/HTTP[S] client requests to other internal ZCS services. A reverse proxy server is an Internet-facing services that protects and manages client connections to your internal services. It can also provide functions like: GSSAPI authentication, throttle control, SSL connection with different certificates for different virtual host names, and other features described later below.
Line 16: Line 21:
 
In a typical use case, the proxy services extracts the user login details and then fetches the route to the upstream mail server or web servers’ address from the Zimbra Proxy Route Lookup Handler [the NLE servlet], and finally proxies the interactions between clients and upstream ZCS servers. To accelerate the speed of future route lookups for a user, memcached caches the lookup results. Therefore, the subsequent login with the same username will directly be proxied without calling to the Zimbra Proxy Route Lookup Handler.
 
In a typical use case, the proxy services extracts the user login details and then fetches the route to the upstream mail server or web servers’ address from the Zimbra Proxy Route Lookup Handler [the NLE servlet], and finally proxies the interactions between clients and upstream ZCS servers. To accelerate the speed of future route lookups for a user, memcached caches the lookup results. Therefore, the subsequent login with the same username will directly be proxied without calling to the Zimbra Proxy Route Lookup Handler.
  
== What Is Zimbra Proxy ==
 
 
== Benefits And Reasons To Use ==
 
== Benefits And Reasons To Use ==
 +
 +
Source http://wiki.zimbra.com/wiki/Zimbra_Proxy_Guide#Benefits_and_reasons_to_use
 +
 +
1. Zimbra proxy centralizes access to Mailbox servers
 +
Zimbra Proxy allows mailbox servers to be hidden from public internet by acting as a reverse proxy & also allowing end users to access mail system via single Login URL instead of knowing their mailbox hostnames. It acts as the first entry point for all the HTTP/IMAP/POP traffic and then intelligently routes all kind of static UI requests (HTML/CSS/JS etc) and Dynamic requests (SOAP/REST/IMAP/POP) to the appropriate upstream server. Here are some other benefits of using the proxy as well
 +
2. Load Balancing
 +
This is the reverse proxy function that people are most familiar with. Here the proxy routes incoming HTTP requests to a number of identical mail servers. The upstream mail server selection can be based on a simple client IP hash or round-robin algorithm. It’s such a common function that load balancing reverse proxies are usually just referred to as ‘load balancers’. There are specialized load balancing products available, but many general purpose reverse proxies also provide load balancing functionality.
 +
3. Security
 +
A reverse proxy can hide the topology and characteristics of your back-end servers by removing the need for direct internet access to them. You can place your reverse proxy in an internet facing DMZ, but hide your web servers inside a non-public subnet.
 +
4. Authentication
 +
You can use your reverse proxy to provide a single point of authentication for all HTTP requests. Although in case of ZCS, the authentication is provided by upstream mailstores where the user accounts reside.
 +
5. SSL Termination
 +
Here the reverse proxy handles incoming HTTPS connections, decrypting the requests and passing unencrypted requests on to the web servers.
 +
6. Caching
 +
Currently we use the memcached module with proxy to achieve caching of upstream routes to mailstores on a per end-client basis. This significantly reduces the route lookup time thereby improving the total time required to process the request and boost performance.
 +
7. Centralised Logging and Auditing
 +
Because all HTTP requests are routed through the reverse proxy, it makes an excellent point for logging and auditing.
 +
8. URL Rewriting
 +
Sometimes the URL scheme that a legacy application presents is not ideal for discovery or search engine optimisation. A reverse proxy can rewrite URLs before passing them on to your back-end servers.
 +
To an external customer it appears that they are simply navigating a single website, but internally the organisation is maintaining three entirely separate sites. This approach can work extremely well for web service APIs where the reverse proxy provides a consistent single public facade to an internal distributed component oriented architecture.
 +
 
== Zimbra Proxy Components ==
 
== Zimbra Proxy Components ==
 +
 +
Source: http://wiki.zimbra.com/wiki/Zimbra_Proxy_Guide#Zimbra_Proxy_Components
 +
 +
Zimbra Proxy is designed to provide a HTTP/POP/IMAP proxy that is quick, reliable, and scalable. Zimbra Proxy includes the following:
 +
Nginx. A high performance HTTP/IMAP/POP3 proxy server which handles all incoming HTTP/POP/IMAP requests.
 +
Memcached. A high performance, distributed memory object caching system. Route information is cached for further use in order to increase performance.
 +
Zimbra Proxy Route Lookup Handler. This is a servlet located on the ZCS mailbox server. This servlet handles queries for the user account route information (the server and port number where the user account resides).
 +
 
== Architecture and Flow ==
 
== Architecture and Flow ==
 +
 +
Source: http://wiki.zimbra.com/wiki/Zimbra_Proxy_Guide#Architecture_and_Flow
 +
 +
The following sequence shows the architecture and flow of Zimbra Proxy.
 +
 +
* End clients connect to Zimbra Proxy using HTTP/HTTPS/POP/IMAP ports.
 +
* When Zimbra Proxy receives an incoming connection, the Nginx component sends an HTTP request to Zimbra Proxy Route Lookup Handler component.
 +
* Zimbra Proxy Route Lookup Handler locates the route information for the account being accessed and returns this to Nginx.
 +
* The Memcached component stores the route information for the configured period of time (by default, this time is one hour). Nginx will use this route information instead of querying the Zimbra Proxy Route Lookup Handler until the default period of time has expired.
 +
* Nginx uses the route information to connect to Zimbra Mailbox.
 +
* Zimbra Proxy connects to Zimbra Mailbox and initiates the web/mail proxy session. The end client behaves as if it is connecting directly to Zimbra Mailbox.
 +
 +
 
== Position In ZCS Runtime ==
 
== Position In ZCS Runtime ==
 +
 +
Source: http://wiki.zimbra.com/wiki/Zimbra_Proxy_Guide#Position_in_ZCS_Runtime
 +
 +
The figure to the right show the positions of NZ and its relationships to other components of ZCS.
 +
Error creating thumbnail: /var/www/html/com.zimbra.wiki/bin/ulimit4.sh: line 4: /usr/bin/convert: No such file or directory
 +
 +
Proxy position in ZCS runtime
 +
With Zimbra Proxy, a general workflow of login is like this. On a client login, NZ will attempt to contact a memcached server (the exact server will be elected from the available candidates, using an round-robin algorithm). The memcached server is expected to return the upstream route information for that particular client. If the information is not present in that memcached server, then this will be a cache-miss, so Zimbra Proxy will proceed to contact an available NLE (elected by Round-Robin), to look up the upstream server information.
 +
 +
 +
Once the upstream server is known, Zimbra Proxy will immediately initiate the proxy session. And then, it will cache the upstream information into the memcached server. The next time the user logs in, the memcached server will have the upstream information available in its cache, and so Zimbra Proxy will not need to contact NLE.
 +
 
== Zimbra Proxy Ports ==
 
== Zimbra Proxy Ports ==
 +
 +
Source: http://wiki.zimbra.com/wiki/Zimbra_Proxy_Guide#Zimbra_Proxy_Ports
 +
 +
The following ports are used either by Zimbra Proxy or by Zimbra Mailbox (if Proxy is not configured). If you have any other services running on these ports, turn them off.
 +
 +
End clients connect directly to Zimbra Proxy, using the Zimbra Proxy Ports. Zimbra Proxy connects to the Route Lookup Handler or Zimbra Mailbox using the Zimbra Mailbox Ports.
 +
 +
{| style="width:30%" border="1" cellpadding="5" cellspacing="0"
 +
! Zimbra Proxy Ports (External to ZCS)
 +
! Port
 +
|-
 +
| HTTP
 +
| 80
 +
|-
 +
| HTTPS
 +
| 443
 +
|-
 +
| POP3
 +
| 110
 +
|-
 +
| POP3S (Secure POP3)
 +
| 995
 +
|-
 +
| IMAP
 +
| 143
 +
|-
 +
| IMAPS (Secure IMAP)
 +
| 993
 +
|-
 +
! Zimbra Mailbox Ports (Internal to ZCS)
 +
! Port
 +
|-
 +
| Route Lookup Handler
 +
| 7072
 +
|-
 +
| HTTP Backend (if Proxy configured)
 +
| 8080
 +
|-
 +
| HTTPS Backend (if Proxy configured)
 +
| 8443
 +
|-
 +
| POP3 Backend (if Proxy configured)
 +
| 7110
 +
|-
 +
| POP3S Backend (if Proxy configured)
 +
| 7995
 +
|-
 +
| IMAP Backend (if Proxy configured)
 +
| 7143
 +
|-
 +
| IMAPS Backend (if Proxy configured)
 +
| 7993
 +
|}
 +
 +
 
== Deployment Strategies, Server Specifications, Impact To Other Non-Proxy Hosts ==
 
== Deployment Strategies, Server Specifications, Impact To Other Non-Proxy Hosts ==
 +
 +
Source: http://wiki.zimbra.com/wiki/Zimbra_Proxy_Guide#Deployment_strategies.2C_server_specifications.2C_impact_to_other_non-proxy_hosts
 +
 +
The deployment strategy Zimbra actively suggests is to use Proxy on the edge with mailbox servers behind the firewall. Please note that the Zimbra Proxy package does not act as a firewall and needs to be behind the firewall in customer deployments.
  
 
= Installing And Configuring Zimbra Proxy =
 
= Installing And Configuring Zimbra Proxy =

Revision as of 15:40, 9 August 2014

Contents

Overview And Planning For Zimbra Proxy

Source: http://wiki.zimbra.com/wiki/Zimbra_Proxy_Guide

What Is Zimbra Proxy

Source: http://wiki.zimbra.com/wiki/Zimbra_Proxy_Guide#What_is_Zimbra_Proxy

Zimbra Proxy is a high-performance reverse proxy service for passing IMAP/POP3/HTTP[S] client requests to other internal ZCS services. A reverse proxy server is an Internet-facing services that protects and manages client connections to your internal services. It can also provide functions like: GSSAPI authentication, throttle control, SSL connection with different certificates for different virtual host names, and other features described later below.


The Zimbra Proxy services allows other zimbra services [mailbox, webapp, etc.] that can be hidden from the Internet by acting as a reverse proxy for the other services and allows end-users to access the mail system via a single Login URL [for example, userA and userB can both use mail.domain.com] instead of knowing their specific mailbox hostnames [for example, userA uses mail12.domain.com and userB is uses mail13.domain.com]. It acts as the first entry point for all the HTTP/IMAP/POP traffic and then intelligently routes all static UI (HTML/CSS/JS etc) and dynamic (SOAP/REST/IMAP/POP) requests to the appropriate upstream server. The proxy configuration options for POP/IMAP allows end users to configure their POP/IMAP clients to use this single mail server hostname. The proxy configuration options around HTTP[S] allows end-users to use this single hostname for ZWC, REST, CalDAV, Zimbra Connector for Outlook, Zimbra Connector for BES, Zimbra Mobile Sync connections and so forth. The Zimbra Proxy service will also do URL rewriting so these internal hostnames that are providing the various services aren't exposed to the public or end-clients. For example, a Zimbra briefcase share from userA on mail12.domain.com to userB on mail13.domain.com will simply have mail.domain.com [per our example above] within the url. [ see bug 82236 ]


With ZCS 8.5, the proxy components are installed by default and therefor would be enabled on a single ZCS server deployment. Normally though, the proxy services would be installed on Internet-facing servers and the other specific ZCS services [zimbra-store for example] would be on installed on internal servers. Generally, these packages are installed on dedicated Internet-facing servers intended to just do the proxy services or those that also run the zimbra-mta package. When the Zimbra Proxy package is installed, the proxy feature is enabled with default values that normally require no modification.


The Zimbra Proxy installation components consists of the zimbra-proxy and zimbra-memcached options during the installation package choice menu. The zimbra-proxy packages is Zimbra’s modified version of Nginx [1], pronounced like “engine-ex”. And the zimbra-memcached packages is our modified version of memcached [2], pronounced like “memcache-dee”. A third component needed for our proxy environment to work is the Zimbra Proxy Route Lookup Handler or called the “Nginx Lookup Extension” (NLE for short) if some other references. This is a java servlet that is installed from the zimbra-store package on the Zimbra mailbox servers. This servlet handles nginx queries for the user account route information (the server and port number where the user account resides).


In a typical use case, the proxy services extracts the user login details and then fetches the route to the upstream mail server or web servers’ address from the Zimbra Proxy Route Lookup Handler [the NLE servlet], and finally proxies the interactions between clients and upstream ZCS servers. To accelerate the speed of future route lookups for a user, memcached caches the lookup results. Therefore, the subsequent login with the same username will directly be proxied without calling to the Zimbra Proxy Route Lookup Handler.

Benefits And Reasons To Use

Source http://wiki.zimbra.com/wiki/Zimbra_Proxy_Guide#Benefits_and_reasons_to_use

1. Zimbra proxy centralizes access to Mailbox servers Zimbra Proxy allows mailbox servers to be hidden from public internet by acting as a reverse proxy & also allowing end users to access mail system via single Login URL instead of knowing their mailbox hostnames. It acts as the first entry point for all the HTTP/IMAP/POP traffic and then intelligently routes all kind of static UI requests (HTML/CSS/JS etc) and Dynamic requests (SOAP/REST/IMAP/POP) to the appropriate upstream server. Here are some other benefits of using the proxy as well 2. Load Balancing This is the reverse proxy function that people are most familiar with. Here the proxy routes incoming HTTP requests to a number of identical mail servers. The upstream mail server selection can be based on a simple client IP hash or round-robin algorithm. It’s such a common function that load balancing reverse proxies are usually just referred to as ‘load balancers’. There are specialized load balancing products available, but many general purpose reverse proxies also provide load balancing functionality. 3. Security A reverse proxy can hide the topology and characteristics of your back-end servers by removing the need for direct internet access to them. You can place your reverse proxy in an internet facing DMZ, but hide your web servers inside a non-public subnet. 4. Authentication You can use your reverse proxy to provide a single point of authentication for all HTTP requests. Although in case of ZCS, the authentication is provided by upstream mailstores where the user accounts reside. 5. SSL Termination Here the reverse proxy handles incoming HTTPS connections, decrypting the requests and passing unencrypted requests on to the web servers. 6. Caching Currently we use the memcached module with proxy to achieve caching of upstream routes to mailstores on a per end-client basis. This significantly reduces the route lookup time thereby improving the total time required to process the request and boost performance. 7. Centralised Logging and Auditing Because all HTTP requests are routed through the reverse proxy, it makes an excellent point for logging and auditing. 8. URL Rewriting Sometimes the URL scheme that a legacy application presents is not ideal for discovery or search engine optimisation. A reverse proxy can rewrite URLs before passing them on to your back-end servers. To an external customer it appears that they are simply navigating a single website, but internally the organisation is maintaining three entirely separate sites. This approach can work extremely well for web service APIs where the reverse proxy provides a consistent single public facade to an internal distributed component oriented architecture.

Zimbra Proxy Components

Source: http://wiki.zimbra.com/wiki/Zimbra_Proxy_Guide#Zimbra_Proxy_Components

Zimbra Proxy is designed to provide a HTTP/POP/IMAP proxy that is quick, reliable, and scalable. Zimbra Proxy includes the following: Nginx. A high performance HTTP/IMAP/POP3 proxy server which handles all incoming HTTP/POP/IMAP requests. Memcached. A high performance, distributed memory object caching system. Route information is cached for further use in order to increase performance. Zimbra Proxy Route Lookup Handler. This is a servlet located on the ZCS mailbox server. This servlet handles queries for the user account route information (the server and port number where the user account resides).

Architecture and Flow

Source: http://wiki.zimbra.com/wiki/Zimbra_Proxy_Guide#Architecture_and_Flow

The following sequence shows the architecture and flow of Zimbra Proxy.

  • End clients connect to Zimbra Proxy using HTTP/HTTPS/POP/IMAP ports.
  • When Zimbra Proxy receives an incoming connection, the Nginx component sends an HTTP request to Zimbra Proxy Route Lookup Handler component.
  • Zimbra Proxy Route Lookup Handler locates the route information for the account being accessed and returns this to Nginx.
  • The Memcached component stores the route information for the configured period of time (by default, this time is one hour). Nginx will use this route information instead of querying the Zimbra Proxy Route Lookup Handler until the default period of time has expired.
  • Nginx uses the route information to connect to Zimbra Mailbox.
  • Zimbra Proxy connects to Zimbra Mailbox and initiates the web/mail proxy session. The end client behaves as if it is connecting directly to Zimbra Mailbox.


Position In ZCS Runtime

Source: http://wiki.zimbra.com/wiki/Zimbra_Proxy_Guide#Position_in_ZCS_Runtime

The figure to the right show the positions of NZ and its relationships to other components of ZCS. Error creating thumbnail: /var/www/html/com.zimbra.wiki/bin/ulimit4.sh: line 4: /usr/bin/convert: No such file or directory

Proxy position in ZCS runtime With Zimbra Proxy, a general workflow of login is like this. On a client login, NZ will attempt to contact a memcached server (the exact server will be elected from the available candidates, using an round-robin algorithm). The memcached server is expected to return the upstream route information for that particular client. If the information is not present in that memcached server, then this will be a cache-miss, so Zimbra Proxy will proceed to contact an available NLE (elected by Round-Robin), to look up the upstream server information.


Once the upstream server is known, Zimbra Proxy will immediately initiate the proxy session. And then, it will cache the upstream information into the memcached server. The next time the user logs in, the memcached server will have the upstream information available in its cache, and so Zimbra Proxy will not need to contact NLE.

Zimbra Proxy Ports

Source: http://wiki.zimbra.com/wiki/Zimbra_Proxy_Guide#Zimbra_Proxy_Ports

The following ports are used either by Zimbra Proxy or by Zimbra Mailbox (if Proxy is not configured). If you have any other services running on these ports, turn them off.

End clients connect directly to Zimbra Proxy, using the Zimbra Proxy Ports. Zimbra Proxy connects to the Route Lookup Handler or Zimbra Mailbox using the Zimbra Mailbox Ports.

Zimbra Proxy Ports (External to ZCS) Port
HTTP 80
HTTPS 443
POP3 110
POP3S (Secure POP3) 995
IMAP 143
IMAPS (Secure IMAP) 993
Zimbra Mailbox Ports (Internal to ZCS) Port
Route Lookup Handler 7072
HTTP Backend (if Proxy configured) 8080
HTTPS Backend (if Proxy configured) 8443
POP3 Backend (if Proxy configured) 7110
POP3S Backend (if Proxy configured) 7995
IMAP Backend (if Proxy configured) 7143
IMAPS Backend (if Proxy configured) 7993


Deployment Strategies, Server Specifications, Impact To Other Non-Proxy Hosts

Source: http://wiki.zimbra.com/wiki/Zimbra_Proxy_Guide#Deployment_strategies.2C_server_specifications.2C_impact_to_other_non-proxy_hosts

The deployment strategy Zimbra actively suggests is to use Proxy on the edge with mailbox servers behind the firewall. Please note that the Zimbra Proxy package does not act as a firewall and needs to be behind the firewall in customer deployments.

Installing And Configuring Zimbra Proxy

Things To Review First

Prerequisite Variables To Check First

zimbraPublicServiceHostname zimbraPublicServiceProtocol and zimbraPublicServicePort

In order for the change password link, calendar launching in separate window, and other various functionality to work correctly - meaning, to use the proxy instead of mailbox server, the following LDAP attributes have to be set to the proxy values:

  • zimbraPublicServiceHostname(Name to be used in public API such as REST or SOAP proxy) - proxy hostname
  • zimbraPublicServiceProtocol(Protocol to be used in public API such as REST or SOAP proxy) - proxy protocol (http or https)
  • zimbraPublicServicePort(Port to be used in public API such as REST or SOAP proxy) - proxy port

zimbraVirtualHostname

zimbra_auth_always_send_refer

The zmlocalconfig key zimbra_auth_always_send_refer is now obsolete. Its been replaced by LDAP attribute zimbraMailReferMode. Now with a full-fledged reverse proxy, users do not need to be redirected. The LDAP attribute zimbraMailReferMode is used directly by the nginx reverse proxy.

zmtlsctl

New ZCS Deployment

Single ZCS Server Environment

Multi-Server ZCS Environment

Adding Proxy Services To Existing Non-Proxy Environments via ZCS Installer [Recommended Method]

Using New Servers

Using Existing Servers

Adding Proxy Services To Existing Non-Proxy Environments via CLI [Advanced Method]

Using New Servers

Using Existing Servers

Manually Modifying Proxy Services And Related Variables via CLI

Simple Command With Defaults

Protocol Requirements Including HTTPS Redirect

Documents & Sharing

Troubleshooting

Proxy Related Log Files

Proxy Login Slow

No Route To Host Errors

5xx Errors

Advance Topics

Ldap Attributes For Proxy

Pre 8.5 Attributes

New 8.5 Attributes

Proxy Configuration And Template Files

Understanding The Proxy Config Rewrite (zmproxyconfgen) Process (zmproxyconfig And zmproxyctl)

Config Files And Config Templates

Config File Hierarchy

Description of Config Files

Proxy Related CLI Commands

zmproxyconfig

Syntax

Description

zmproxyctl

Syntax

Description

zmprov

Syntax

Description

Advanced Proxy Configuration Examples via CLI

Configure Zimbra Proxy For POP[S] And IMAP[S] Only

Disabling Zimbra Proxy For POP[S] And IMAP[S] Only

Configure Zimbra Proxy For POP[S] Only

Disabling Zimbra Proxy For POP[S] Only

Configure Zimbra Proxy For POP[S] And HTTP[S] Only

Disabling Zimbra Proxy For POP[S] And HTTP[S] Only

Configure Zimbra Proxy For IMAP[S] Only

Disabling Zimbra Proxy For IMAP[S] Only

Configure Zimbra Proxy For IMAP[S] And HTTP[S] Only

Disabling Zimbra Proxy For IMAP[S] And HTTP[S] Only

Configure Zimbra Proxy For HTTP[S] Only

Disabling Zimbra Proxy For HTTP[S] Only

Configure Or Customize The Zimbra Proxy For The Admin Console

Disabling Or Customize The Zimbra Proxy For The Admin Console

Configure Zimbra Proxy For Kerberos Authentication

Disabling Zimbra Proxy For Kerberos Authentication

Jump to: navigation, search