Difference between revisions of "Zimbra Next Generation Modules/DR/Disaster Recovery - MariaDB data loss or corruption"

 
Line 1: Line 1:
<div class="col-md-12"><br></div>
+
#REDIRECT [[Zimbra_NG_Modules/DR/Disaster_Recovery_-_MariaDB_data_loss_or_corruption]]
<div class="col-md-12"><br></div>
 
<ol class="breadcrumb">
 
  <li>[[Main Page|Zimbra Wiki]]</li>
 
  <li>[[Zimbra_Next_Generation_Modules]]</li>
 
  <li>[[Zimbra_NG_Backup]]</li>
 
  <li class="active">Disaster Recovery - MariaDB data loss or corruption</li>
 
</ol>
 
__NOTOC__
 
<div class="col-md-12"><br /></div>
 
<div class="col-md-9">
 
    <h2 class="title-header" style="padding-bottom: 9px; border-bottom: 4px solid #0087c3;">Disaster Recovery - MariaDB data loss or corruption</h2>
 
    <div class="col-md-12">
 
        <div class="ibox-content">
 
            <div class="post animated fadeInLeft animation-delay-8" style="padding-top:5px">
 
                <div class="panel panel-default">
 
                    <div class="panel-body">
 
                        <div class="row">
 
 
 
=Introduction=
 
This step-by-step guide contains all the information needed to restore lost or corrupted MariaDB data. It applies to both single-server and multi-server infrastructures with only a single MariaDB server.
 
 
 
=== Darabase loss and Zimbra Volumes===
 
Due to the fact that volume information for all items is stored in the database, which in this specific scenario has been lost, your current volumes will not be of any use. Please make sure you have enough disk space before you continue, if not either delete or move to another storage your current volumes.
 
 
 
=Step 1: Database re-initialisation=
 
The first step in order to restore the database to a consistent state is to re-initialise the database itself to its initial state.
 
 
 
To do this:
 
* Stop all services by running <code>zmcontrol stop</code> as the zimbra user.
 
* Rename *or* empty the current Database folder - default <code>/opt/zimbra/db/data/</code>
 
** Rename: <code>mv /opt/zimbra/db/data/ /opt/zimbra/db/data_old/</code>
 
** Empty: <code>rm -rf /opt/zimbra/db/data/*</code>
 
* After cleaning up the database, re-initialise it by running <code>/opt/zimbra/libexec/zmmyinit</code> as the zimbra user:
 
<pre>
 
* Creating required directories
 
* Generating mysql config /opt/zimbra/conf/my.cnf
 
* Creating database in /opt/zimbra/db/data
 
* Starting mysql server
 
* Loading schema /opt/zimbra/db/db.sql
 
* Loading version from /opt/zimbra/db/versions-init.sql
 
* Loading version from /opt/zimbra/db/backup-version-init.sql
 
* Setting random passwd for mysql root user in zimbra localconfig
 
* Setting random passwd for mysql zimbra user in zimbra localconfig
 
* Changing mysql root user password
 
* Changing mysql zimbra user password
 
* Changed zimbra mysql user password
 
</pre>
 
Database passwords as well as mailbox IDs will not be changed.
 
* After re-initialising the database, start all services by running <code>zmcontrol start</code> as the zimbra user
 
 
 
If everything succeeded, after all services are initialised you should see that mailboxgroups have been created anew:
 
<pre>
 
ls -la /opt/zimbra/db/data/
 
total 1099868
 
drwxr-x--- 16 zimbra zimbra      4096 Aug 28 12:09 .
 
drwxrwxr-x  3 zimbra zimbra      4096 Aug 25 18:03 ..
 
drwx------  2 zimbra zimbra      4096 Aug 28 12:09 chat
 
-rw-rw----  1 zimbra zimbra 524288000 Aug 28 12:11 ib_logfile0
 
-rw-rw----  1 zimbra zimbra 524288000 Aug 28 12:03 ib_logfile1
 
-rw-rw----  1 zimbra zimbra  77594624 Aug 28 12:11 ibdata1
 
drwx------  2 zimbra zimbra      4096 Aug 28 12:09 mboxgroup1
 
drwx------  2 zimbra zimbra      4096 Aug 28 12:09 mboxgroup2
 
drwx------  2 zimbra zimbra      4096 Aug 28 12:09 mboxgroup3
 
drwx------  2 zimbra zimbra      4096 Aug 28 12:09 mboxgroup4
 
drwx------  2 zimbra zimbra      4096 Aug 28 12:09 mboxgroup5
 
drwx------  2 zimbra zimbra      4096 Aug 28 12:09 mboxgroup6
 
drwx------  2 zimbra zimbra      4096 Aug 28 12:09 mboxgroup7
 
drwx------  2 zimbra zimbra      4096 Aug 28 12:09 mboxgroup8
 
drwx------  2 zimbra zimbra      4096 Aug 28 12:09 mboxgroup9
 
-rw-rw----  1 zimbra zimbra        0 Aug 28 12:03 multi-master.info
 
drwx------  2 zimbra zimbra      4096 Aug 28 12:03 mysql
 
drwx------  2 zimbra zimbra      4096 Aug 28 12:03 performance_schema
 
-rw-rw----  1 zimbra zimbra    24576 Aug 28 12:03 tc.log
 
drwx------  2 zimbra zimbra      4096 Aug 28 12:03 test
 
drwx------  2 zimbra zimbra      4096 Aug 28 12:03 zimbra
 
</pre>
 
=Step 2: Data Restore=
 
At this point of the process the database is up and running, but mostly empty. You can check this by looking at any mailbox's content using zmprov:
 
<pre>
 
zmmailbox -z -m rick@domain.local gaf
 
        Id  View      Unread  Msg Count  Path
 
----------  ----  ----------  ----------  ----------
 
        1  unkn          0          0  /
 
        16  docu          0          0  /Briefcase
 
        10  appo          0          0  /Calendar
 
        14  mess          0          0  /Chats
 
        7  cont          0          0  /Contacts
 
        6  mess          0          0  /Drafts
 
        13  cont          0          0  /Emailed Contacts
 
        2  mess          0          0  /Inbox
 
        4  mess          0          0  /Junk
 
        5  mess          0          0  /Sent
 
        15  task          0          0  /Tasks
 
        3  unkn          0          0  /Trash
 
</pre>
 
To restore the data, the doExternalRestore feature of Backup NG will be used.
 
 
 
* To start the restore, simply start a complete External Restore by running <code>zxsuite backup doExternalRestore [your backup path here]</code> as the zimbra user
 
** e.g. if your backup path is /opt/zimbra/backup/ then run <code>zxsuite backup doExternalRestore /opt/zimbra/backup/</code>
 
* Once the restore has started, you'll find an "External Restore Started" notification in the "Notifications" section of the NG Adminitration Zimlet.
 
** That notification, along with the output of the command, will include the "Operation ID" of the restore - make sure to keep it at hand.
 
* To verify the progress of the restore, you can check the dedicated restore log found in <code>/opt/zimbra/logs/</code> called <code>op_ExternalRestore_[operation ID].log</code>
 
* After  the External Restore has been completed, you will receive an "External Restore Completed" notification.
 
** You can now verify that data has been restored by checking the content of a mailbox with zmprov:
 
<pre>
 
zmmailbox -z -m rick@domain.local gaf
 
        Id  View      Unread  Msg Count  Path
 
----------  ----  ----------  ----------  ----------
 
        1  unkn          0          0  /
 
        16  docu          0          0  /Briefcase
 
        10  appo          0          0  /Calendar
 
        14  mess          0          0  /Chats
 
        7  cont          0          0  /Contacts
 
        6  mess          0          0  /Drafts
 
        13  cont          0          0  /Emailed Contacts
 
        2  mess        5159        6258  /Inbox
 
        4  mess          0          0  /Junk
 
        5  mess          0        118  /Sent
 
        15  task          0          0  /Tasks
 
        3  unkn          28          29  /Trash
 
</pre>
 
* In several cases, one run of the External Restore is not enough to restore all items, so run the very same restore command again (this will have a new Operation ID and a new restore log).
 
* Once the second restore is completed, all data will have been restored.
 
 
 
=Step 3: Volume Deduplication and aftermath=
 
After the data restore is completed, all items will be stored in the Current Primary volume of your server. This step describes some follow-up operations that can be useful to get your volume configuration back to optimal (and save some disk space in the process).
 
 
 
===Run a Volume Deduplication===
 
When items are restored using the External Restore , there is a very high chance for the cache-based deduplication not to catch many duplicates: you can fix this by running a volume-wide deduplication on your primary volume using the <code>zxsuite hsm doDeduplicate [volume ID]</code>, where [Volume ID] is the ID of the volume you wish to scan for duplicates.
 
 
 
===Create your Secondary Volumes and apply the HSM Policy===
 
If you used to have Secondary Volumes on your system, you can now create those again and apply the HSM policy to move all of your data. To run the HSM from the command line, use the <code>zxsuite hsm doMoveBlobs</code> as the zimbra user.
 
 
 
===Compress your data===
 
If you don't have Secondary Volumes but want to compress your primary volume's data to save disk space you need to:
 
* Create a new Primary Volume, flagging it as both "Compressed" and "Current"
 
* Use the <code>zxsuite hsm doVolumeToVolumeMove</code> command as the zimbra user to move all contents of the original volume to the new one. This will compress items in the process, since the new volume is marked as Compressed.
 
* Once the Volume-To-Volume Move is completed, delete the original volume, which should now be empty
 
 
 
 
 
                        </div>
 
                    </div>
 
                    <div class="col-md-9">
 
                        <div class="panel-footer">
 
                            <p><i class="fa fa-clock-o"></i> Aug 30, 2017 - [https://www.zimbra.com/email-server-software/ Know more »]</p>
 
                        </div>
 
                    </div>
 
                </div>
 
            </div>
 
        </div>
 
    </div>
 
</div>
 
<div class="col-md-3"><br /></div>
 
<div class="col-md-3">
 
    <div class="panel panel-zimbrared-light-border">
 
        <div class="panel-heading">
 
            <h3 class="panel-title"><i class="fa fa-gear pull-left"></i> Zimbra NG Modules</h3>
 
        </div>
 
        <div class="panel-body">
 
            {{ZNG}}
 
        </div>
 
    </div>
 
</div>
 
<div class="col-md-3">
 
    <div class="panel panel-primary-light-border">
 
        <div class="panel-heading">
 
            <h3 class="panel-title"><i class="fa fa-info-circle pull-left"></i> Zimbra NG Modules Resources</h3>
 
        </div>
 
        <div class="panel-body">
 
            {{ZNGL}}
 
        </div>
 
    </div>
 
</div>
 
<div class="clearfix"></div>
 
<div class="col-md-12"><br></div>
 
{{FH}}
 

Latest revision as of 12:16, 29 November 2017

Jump to: navigation, search