ZCS88-zxbackup-Disaster-Recovery: Difference between revisions

 
(16 intermediate revisions by the same user not shown)
Line 119: Line 119:
At this point, you should be up and running; however, as an option, you may want to check the user BLOBs, you can refer to this [https://wiki.zimbra.com/wiki/ZCS88-zxbackup-Missing-BLOB guide].<br>
At this point, you should be up and running; however, as an option, you may want to check the user BLOBs, you can refer to this [https://wiki.zimbra.com/wiki/ZCS88-zxbackup-Missing-BLOB guide].<br>


==<font size="6">Scenario II - Single Server with corrupted mysql beyond repair</font>==
==<font size="6">Scenario II - Mysql Server with corrupted mysql data that is beyond repair</font>==
You have been running ZCS-88 using the Next Generation Suite, and you have just discovered a lot of errors in the /opt/zimbra/mysql_error.log file, and the mysql service is not running<br>
You have been running ZCS-88 using the Next Generation Suite, and you have just discovered a lot of errors in the /opt/zimbra/mysql_error.log file, and the mysql service is not running<br>


Line 144: Line 144:
<code><pre>
<code><pre>
zmcontrol start
zmcontrol start
zmcontrol status   
</pre></code>
</pre></code>
<strong><span style="color:red">REMEMBER!</span></strong><br>
The mailboxd service must be running in order to use the <strong>zxsuite</strong> command.<br>


===Run the external restore command to rebuild the mysql data===
===Run the external restore command to rebuild the mysql data===
<code><pre>
<code><pre>
zxsuite backup doExternalRestore /opt/zimbra/backup/
zxsuite backup doExternalRestore /opt/zimbra/backup/
</pre></code>
====Import an export mysql retention file if you have one====
If you had exported your mysql data <strong>prior</strong> to the mysql outage...<br>
<code><pre>
source ~/bin/zmshutil; zmsetvars
mysqldump -u root --password=$mysql_root_password --socket=$mysql_socket --all-databases --single-transaction >> /tmp/mysqlinfo.sql
</pre></code>
Then you can simply reimport the data into mysql.<br>
<code><pre>
mysql < /tmp/mysqlinfo.sql
</pre></code>
====Add the mysql export command to Zimbra's crontab====
As a mere suggestion, it might be a good idea to export the mysql data on a nightly basis.<br>
Should you need to rebuild the mysql data, having an exported mysql data file may be handy; could shorten your downtime.<br>
Add the following to zimbra's crontab<br>
<code><pre>
30 2 * * *  source ~/bin/zmshutil; zmsetvars; /opt/zimbra/common/bin/mysqldump -u root --password=$mysql_root_password --socket=$mysql_socket --all-databases --single-transaction | gzip -c >> /opt/zimbra/data/tmp/mysqlinfo_`date +%F`.sql.gz
</pre></code>
</pre></code>


===Scenario II: Summary===
===Scenario II: Summary===
If mysql service is not running because of corruption, simply move aside the data directory, and run the zmmyinit script to bring up mysql service into a new clean state. Followed up by running the doExternalRestore option.


==<font size="6">Scenario III - Multi Server: Restore LDAP Service</font>==
==<font size="6">Scenario III - Multi Server: Restore LDAP Service</font>==
<font size="5">Two Node Setup</font><br>
<font size="5">Two Node Setup</font><br>
Server 1 - LDAP + Proxy + MTA<br>
<strong>Server 1</strong> - LDAP + Proxy + MTA<br>
Server 2 - Mailboxd  (User mail store)<br>
<strong>Server 2</strong> - Mailboxd  (User mail store)<br>


If LDAP services are needed to be restored, you can do the following.<br>
If LDAP services are needed to be restored, you can do the following.<br>
Line 199: Line 185:
</pre></code>
</pre></code>


====Extract the ldap.ldif file====
====Extract the ldap.ldif file AND ldap-config file====
<code><pre>
<code><pre>
tar -zxvf ldap_23_08_17#04_01_00.tar.gz ldap.ldif
tar -zxvf ldap_23_08_17#04_01_00.tar.gz ldap.ldif
tar -zxvf ldap_23_08_17#04_01_00.tar.gz ldap-config.ldif
</pre></code>
</pre></code>


Line 207: Line 195:
<code><pre>
<code><pre>
cd /tmp/ldap/
cd /tmp/ldap/
tar -zxvf ldap_23_08_17#04_01_00.tar.gz ldap-config.ldif


mv /opt/zimbra/data/ldap/config/  /opt/zimbra/data/ldap/OLDconfig/
mv /opt/zimbra/data/ldap/config/  /opt/zimbra/data/ldap/OLDconfig/
Line 217: Line 203:
</pre></code>
</pre></code>


===Use zmslapadd to add the ldap data===
====Use zmslapadd to add the ldap data====
<code><pre>
<code><pre>
/opt/zimbra/libexec/zmslapadd /tmp/ldap/ldap.ldif
/opt/zimbra/libexec/zmslapadd /tmp/ldap/ldap.ldif

Latest revision as of 16:14, 5 September 2017

ZCS 8.8 Article ZCS 8.8


Scenario I - Single Server

You have been running ZCS-88 using the Next Generation Suite, and you have a dedicated disk for /opt/zimbra/, something like...

/dev/mapper/vg_zcs-lv_root
xxG  xxxG   xxG  xx% /
tmpfs                /dev/shm
/dev/sda1            /boot
/dev/sda2            /opt/zimbra/

But something has happened to the server node, effecting the operating system, which will require a reinstall of the operating system, and ZCS-88.

Reinstall the Operating System

When you reinstall the operating system, make sure the disk containing /opt/zimbra/ is not mounted yet.

Reinstall ZCS-88 with Next Generation Modules

When you are ready to reinstall ZCS-88, make sure the /opt/zimbra/ is now mounted.

If it's possible, rename your production /opt/zimbra/ directory.

mv /opt/zimbra/    mv /opt/zimbraSAVE/


Now, run the installer as the root user using the software option.

./install.sh -s

*OR*

./install.sh --software

During the ZCS-88 Software Reinstall

Observe at the end of the software install, you will see the following

Running Post Installation Configuration:

Software Installation complete!

Operations logged to /tmp/install.log.SviTipi8

Now observe, you have two zimbra directories

drwxr-xr-x  30 root root 4096 Aug 24 14:29 zimbra
drwxr-xr-x  30 root root 4096 Aug 17 13:20 zimbraSAVE

Since the software install had created the /opt/zimbra/ directory, lets move it out of the way

mv /opt/zimbra/   /opt/zimbraSOFTWARE_INSTALL/

Now, lets put our production directory back into place

mv /opt/zimbraSAVE/   /opt/zimbra/

After putting your production /opt/zimbra/ directory back into place, switch to the zimbra account, and start Zimbra

su - zimbra
zmcontrol start
Host test3.zextras.local
	Starting ldap...Done.
	Starting zmconfigd...Done.
	Starting logger...Done.
	Starting convertd...Done.
	Starting mailbox...Done.
	Starting memcached...Done.
	Starting proxy...Done.
	Starting amavis...Done.
	Starting antispam...Done.
	Starting antivirus...Done.
	Starting opendkim...Done.
	Starting snmp...Done.
	Starting spell...Done.
	Starting mta...Done.
	Starting stats...Done.
	Starting service webapp...Done.
	Starting zimbra webapp...Done.
	Starting zimbraAdmin webapp...Done.
	Starting zimlet webapp...Done.
	Starting imapd...Done.

Check the status

zmcontrol status
Host test3.zextras.local
	amavis                  Running
	antispam                Running
	antivirus               Running
	convertd                Running
	imapd                   Running
	ldap                    Running
	logger                  Running
	mailbox                 Running
	memcached               Running
	mta                     Running
	opendkim                Running
	proxy                   Running
	service webapp          Running
	snmp                    Running
	spell                   Running
	stats                   Running
	zimbra webapp           Running
	zimbraAdmin webapp      Running
	zimlet webapp           Running
	zmconfigd               Running

Scenario I - Summary

Operating system needed to be installed, moving aside production Zimbra, and doing a software install, and placing production Zimbra directory back into place.

At this point, you should be up and running; however, as an option, you may want to check the user BLOBs, you can refer to this guide.

Scenario II - Mysql Server with corrupted mysql data that is beyond repair

You have been running ZCS-88 using the Next Generation Suite, and you have just discovered a lot of errors in the /opt/zimbra/mysql_error.log file, and the mysql service is not running

You even tried to export and reimport the mysql data to no avail.

Stop Zimbra Services

zmcontrol stop

Move aside your old mysql directory

mv /opt/zimbra/db/data/   /opt/zimbra/db/OLD_data/

mkdir /opt/zimbra/db/data/

Start up mysql in a clean state

/opt/zimbra/zmmyinit

Start up the rest of the ZCS services

zmcontrol start

zmcontrol status    

REMEMBER!
The mailboxd service must be running in order to use the zxsuite command.

Run the external restore command to rebuild the mysql data

zxsuite backup doExternalRestore /opt/zimbra/backup/

Scenario II: Summary

If mysql service is not running because of corruption, simply move aside the data directory, and run the zmmyinit script to bring up mysql service into a new clean state. Followed up by running the doExternalRestore option.

Scenario III - Multi Server: Restore LDAP Service

Two Node Setup
Server 1 - LDAP + Proxy + MTA
Server 2 - Mailboxd (User mail store)

If LDAP services are needed to be restored, you can do the following.

Method A

Stop ldap and move the old database aside

ldap stop

mv /opt/zimbra/data/ldap/mdb/db  /opt/zimbra/data/ldap/mdb/OLD_db/

mkdir -p  /opt/zimbra/data/ldap/mdb/db

Copy the backed up ldap data from Server 2

mkdir /tmp/ldap/

scp root@server2.example.com:/opt/zimbra/backup/ldap/ldap_23_08_17#04_01_00.tar.gz  /tmp/ldap/

chown -R zimbra:zimbra /tmp/ldap/

Extract the ldap.ldif file AND ldap-config file

tar -zxvf ldap_23_08_17#04_01_00.tar.gz ldap.ldif

tar -zxvf ldap_23_08_17#04_01_00.tar.gz ldap-config.ldif

Restore the ldap-config

cd /tmp/ldap/

mv /opt/zimbra/data/ldap/config/  /opt/zimbra/data/ldap/OLDconfig/

mkdir /opt/zimbra/data/ldap/config/

/opt/zimbra/libexec/zmslapadd -c /tmp/ldap/ldap-config.ldif

Use zmslapadd to add the ldap data

/opt/zimbra/libexec/zmslapadd /tmp/ldap/ldap.ldif

ldap start

Method B

Use zmrestoreldap

List all labels

zmrestoreldap -lbs

**Should see something like...**
full-20170824.050005.077
full-20170823.050004.754
full-20170822.050004.779
full-20170821.050004.736
full-20170820.050005.331

Then use zmrestoreldap with the label name to restore the ldap data.

zmrestoreldap -lb full-20170824.050005.077

If ldap is not starting, use the long command to start ldap

sudo /opt/zimbra/libexec/zmslapd -l LOCAL0 -u zimbra -h 'ldap://ldap.example.com:389 ldapi:///' -F /opt/zimbra/data/ldap/config -d -4

The bottom portion of the output will usually provide a clue on why ldap is not starting.

Section III: Summary

Method A is to illustrate you can copy the backed up ldap from another node with Next Generation Module installed, extract the ldap data, and manually add ldap back into the ldap service.

Method B is to illustrate the old standard way of restoring ldap service using the zmrestoreldap command.

REMEMBER!
Next Generation modules are only installed on Zimbra nodes running mailboxd service; and by convention, nodes running mailboxd will need to remove zmbackup from cron.
However, in a multi node setup, Next Generation modules would not be installed on a node running ldap service. Therefore, the zmbackup command can still be in place in cron to save the ldap config.

Jump to: navigation, search