Zimbra Proxy Manual:Troubleshooting Zimbra Proxy
|General Proxy Overview|
|Overview And Planning For Zimbra Proxy||Installing , Configuring, Disabling the Zimbra Proxy||Zimbra Proxy Related CLI Commands||Troubleshooting Zimbra Proxy|
|Configuration And Template Files And Proxy Related Variables||Advanced Proxy Configuration Examples via CLI||Miscellaneous Topics||Zimbra Proxy Manual:Adding Additional Reverse Proxy To Zimbra Proxy|
|This article applies to the following ZCS versions.|
- 1 Troubleshooting Zimbra Proxy
Troubleshooting Zimbra Proxy
Proxy Related Log Files
Proxy related errors can be found in the following logs on each proxy server:
Slow Proxy Logins - Route Lookup Handles (garpu)
A common nginx misconfiguration is to have incorrectly designated non-mailbox servers as routing/zmlookup handlers. Only mailbox servers can perform route handler functions. To view the zmlookup lookup handlers, review the zm_lookup_handlers parameter in /opt/zimbra/conf/nginx/includes/nginx.conf.zmlookup
grep zm_lookup_handlers /opt/zimbra/conf/nginx/includes/nginx.conf.zmlookup
If a non-mailbox server is listed, set the zimbraReverseProxyLookupTarget server configuration attribute to FALSE for that server.
zmprov ms `zmhostname` zimbraReverseProxyLookupTarget FALSE
Additionally, zimbraReverseProxyLookupTarget is a server inherited attribute from the global configuration, so check if zimbraReverseProxyLookupTarget has been incorrectly designated in global config.
zmprov gcf zimbraReverseProxyLookupTarget
No Route To Host Errors
If your getting "No route to host" errors in /opt/zimbra/log/nginx.log files on the proxy servers, you should check:
- If the resolution [DNS] to the host from the proxy servers is correct and working. This oversight might when you've deployed a new ZCS server in your environment.
- Following step one, you also have to confirm there is port level access between proxy servers and the server it's trying to reach. memcache port is 11211 .
- The server/s [mailbox/webapp servers] are too busy to serve the request.
- You have a server in the lookup pool list that shouldn't be there or you should remove the trouble server from the pool to avoid any more customer issues to deescalate the situation.
- 500 Internal Server Error:
- A generic error message, given when an unexpected condition was encountered and no more specific message is suitable. This might be due to several reasons due to failure on one of the upstream mailstore servers.
- 502 Bad Gateway:
- The server was acting as a gateway or proxy and received an invalid response from the upstream server.
- 503 Service Unavailable:
- The most common reason for this is that the jetty mailboxd process is down on the mailstore server and hence is unable to process the request. Check using 'netstat -an | grep <port>' to see if the IMAP/POP/HTTP ports used by mailboxd are up and listening for incoming connections. Also check using 'ps -eaf | grep mailboxd'
- 504 Gateway Timeout:
- The most common reason for this is that the upstream mailstore server took longer than the configured timeout value and hence the client closed the upstream connection. Try changing the timeout values to a higher value if its expected to take longer for specific kind of requests. If this is not the case, then we should look for reasons why it took longer for a request to get processed.
These following configuration changes for an upstream mailbox server can improve proxy behavior related to timeouts:
- This will configure proxy to immediately reconnect to all mailbox server on any failure
$ zmprov mcf zimbraMailProxyReconnectTimeout 0
- If necessary, for each mailbox server
$ zmprov ms <mailbox server hostname> zimbraMailProxyReconnectTimeout 0
- This will configure proxy to ignore failures in regards to disconnects to all mailbox servers
$ zmprov mcf zimbraMailProxyMaxFails 0
- If necessary, for each mailbox server
$ zmprov ms <mailbox server hostname> zimbraMailProxyMaxFails 0
- Then, restart all the proxies to regenerate the nginx config files with the updated values for 'fail_timeout' & 'max_fails':
$ zmproxyctl restart
See the following bug for more details on this recommendation:
- "Improved proxy timeout defaults"
Third Party Issues - Load Balancers, DNS, Etc.
Bad/Invalid command when proxying to external POP/IMAP servers
Nginx issues the XOIP command to the upstream POP3 server, and the ID command to the upstream IMAP server, before logging in to upstream. This is for auditing purposes so that the client's IP address is known to the upstream server. The global config attributes zimbraReverseProxySendPop3Xoip and zimbraReverseProxySendImapId control this aspect.
However, some external IMAP servers may not implement the ID command, and some external POP3 servers may not implement the XOIP command.
To turn off sending the XOIP command, set zimbraReverseProxySendPop3Xoip to false. To turn off sending the IMAP command, set zimbraReverseProxySendImapId to false.
- zmprov mcf zimbraReverseProxySendPop3Xoip FALSE
- zmprov mcf zimbraReverseProxySendImapId FALSE
LDAP/Nginx Won't Start And Asks For A Password
This tends to happen because Zimbra can't read the key or the certificate of the LDAP/IMAP proxy service on startup. This issue is specific to commercial certificates.
Here is what you can do:
- You want to check to make sure the private key is not encrypted.
- Manually link the cert to the services OR simply redeploy the certificate to relink the cert to the services.
cp /opt/zimbra/ssl/zimbra/commercial/commercial.key /opt/zimbra/conf/slapd.key
cp /opt/zimbra/ssl/zimbra/commercial/commercial.crt /opt/zimbra/conf/slapd.crt
cp /opt/zimbra/ssl/zimbra/commercial/commercial.key /opt/zimbra/conf/nginx.key
cp /opt/zimbra/ssl/zimbra/commercial/commercial.crt /opt/zimbra/conf/nginx.crt
- Restart the Zimbra services.