How to merge two independent ZCS servers into one: Difference between revisions
No edit summary |
|||
(2 intermediate revisions by one other user not shown) | |||
Line 1: | Line 1: | ||
=How to merge two independent ZCS servers into ONE= | =How to merge two independent ZCS servers into ONE= | ||
{{KB|{{ | {{KB|{{Unsupported}}|{{ZCS 8.6}}|{{ZCS 8.5}}|{{ZCS 8.0}}|}} | ||
Line 18: | Line 18: | ||
*The "source" and the "client" must have the '''same hostnames''' prior this operation. For more information check the "Additional Content" section at the bottom of the page. | *The "source" and the "client" must have the '''same hostnames''' prior this operation. For more information check the "Additional Content" section at the bottom of the page. | ||
*The OS versions of the "source" and the "client" can be different. Eg. migrate from centos to ubuntu is ok. | *The OS versions of the "source" and the "client" can be different. Eg. migrate from centos to ubuntu is ok. | ||
<br> | <br> | ||
During the steps below, there are couple of scripts to be run. Make sure you make them executable (chmod +x file.sh) before that. If you are using '''vim''' to paste the text, use ''':set paste''' before pasting, otherwise when you paste the text the formatting will be gone. | During the steps below, there are couple of scripts to be run. Make sure you make them executable (chmod +x file.sh) before that. If you are using '''vim''' to paste the text, use ''':set paste''' before pasting, otherwise when you paste the text the formatting will be gone. | ||
Line 329: | Line 328: | ||
done | done | ||
------- | |||
'''10. Restore forwarding addresses | |||
#!/bin/bash | |||
mkdir /tmp/script | |||
for i in `cat /tmp/script/accounts.txt` | |||
do | |||
fwd_emails=`cat /tmp/script/$i|awk '{print}' ORS=' zimbraMailForwardingAddress '|sed 's/.\{28\}$//' ` | |||
zmprov ma $i zimbraMailForwardingAddress $fwd_emails | |||
done | |||
''' | ------- | ||
'''11. Restore distribution list/s aliases | |||
#!/bin/bash | |||
mkdir /tmp/dist | |||
for i in `cat /tmp/dist/distlist.txt` | |||
do | |||
for j in `cat /tmp/dist/$i` | |||
do | |||
zmprov adla $i $j | |||
done | |||
done | |||
------- | |||
'''12. Restore user notes | |||
#!/bin/bash | |||
mkdir /tmp/notes | |||
for i in `cat /tmp/notes/accounts.txt` | |||
do | |||
note=`cat /tmp/notes/$i` | |||
zmprov ma $i zimbraNotes "$note" | |||
done | |||
------- | |||
'''13. Post steps.''' | |||
*Shut down the "client" server. | *Shut down the "client" server. |
Latest revision as of 10:57, 12 January 2022
How to merge two independent ZCS servers into ONE
Purpose
This article covers the following:
Case scenario: There are two separate zimbra server, not connected in any way. The aim is to move the data from one of the servers to the other.
The steps below can be used to move a server to another new server too, or to another ZCS server in multi server environment.
Prerequisites
- Lets call the server that the data will be moved to the "SOURCE", and the server that will move its data to the "source" the "CLIENT".
- The "source" and the "client" must have the same hostnames prior this operation. For more information check the "Additional Content" section at the bottom of the page.
- The OS versions of the "source" and the "client" can be different. Eg. migrate from centos to ubuntu is ok.
During the steps below, there are couple of scripts to be run. Make sure you make them executable (chmod +x file.sh) before that. If you are using vim to paste the text, use :set paste before pasting, otherwise when you paste the text the formatting will be gone.
Resolution
Overview of the steps:
On both servers create base directories for the migration:
# mkdir /migration # chown zimbra.zimbra /migration
On the old server (CLIENT):
1. Become zimbra.
# su - zimbra
2. Backup the data from the "client".
2.1. Backup the domains:
cd /migration zmprov gad > domains.txt
(open the domains.txt and remove all domains/subdomains that already exists on the "source")
2.2 Backup all admins.
cd /migration zmprov gaaa > admins.txt
2.3 Backup all accounts.
cd /migration zmprov -l gaa > accounts.txt
(open the file accounts.txt and remove the accounts starting with: spam,ham,virus,galsync)
2.4. Add the calendar resources to the accounts file.
zmprov gacr >> accounts.txt
2.5 Backup all distribution lists.
cd /migration zmprov gadl > dlist.txt
2.6 Backup all members of the distribution lists.
cd /migration mkdir dl for i in `cat /migration/dlist.txt`; do zmprov gdlm $i > /migration/dl/$i.txt ;echo "$i"; done
4. Backup all account passwords.
cd /migration mkdir userpass for i in `cat accounts.txt`; do zmprov -l ga $i userPassword | grep userPassword: | awk '{ print $2}' > userpass/$i.shadow; done
5. Backup all user,display and given names.
cd /migration mkdir userdata for i in `cat accounts.txt`; do zmprov ga $i | grep -i Name: > userdata/$i.txt ; done
6. Backup all accounts data.
cd /migration mkdir accounts for account in `cat /migration/accounts.txt`; do /opt/zimbra/bin/zmmailbox -z -m $account -t 0 getRestURL "//?fmt=tgz" > accounts/$account.tgz ; echo $account done; done
7. Backup aliases.
cd /migration mkdir alias for i in `cat accounts.txt`; do zmprov ga $i | grep zimbraMailAlias |awk '{print $2}' > alias/$i.txt ;echo $i ;done
Remove the ones that are empty files:
find /migration/alias/* -size 0 | xargs rm -f
8. Backup user's emails that are set as "forwarding"
#/bin/bash mkdir /tmp/script zmprov -l gaa > /tmp/script/accounts.txt sed -i '/galsync*/d' /tmp/script/accounts.txt sed -i '/ham*/d' /tmp/script/accounts.txt sed -i '/spam*/d' /tmp/script/accounts.txt sed -i '/virus-quarantine*/d' /tmp/script/accounts.txt for i in `cat /tmp/script/accounts.txt` do addresses=`zmprov ga $i zimbraMailForwardingAddress|sed '1d'|awk '{print $2}'` touch /tmp/script/$i echo -e "$addresses \n" >> /tmp/script/$i truncate -s -1 /tmp/script/$i done find /tmp/script -size -3c | xargs -n1 rm -v ls -l /tmp/script | awk '{print $9}'|sed '1,2d' > /tmp/script/accounts.txt
9. Backup distribution lists aliases
#!/bin/bash mkdir /tmp/dist zmprov gadl > /tmp/dist/distlist.txt for i in `cat /tmp/dist/distlist.txt` do distlist=`zmprov gdl $i | grep zimbraMailAlias |awk '{print $2}'|sed '1d'` touch /tmp/dist/$i echo -e "$distlist \n" >> /tmp/dist/$i truncate -s -1 /tmp/dist/$i done find /tmp/dist -size -3c | xargs -n1 rm -v ls -l /tmp/dist | awk '{print $9}'|sed '1,2d' > /tmp/dist/distlist.txt
10. Backup user's notes
#!/bin/bash mkdir /tmp/notes zmprov -l gaa > /tmp/notes/accounts.txt sed -i '/galsync*/d' /tmp/notes/accounts.txt sed -i '/ham*/d' /tmp/notes/accounts.txt sed -i '/spam*/d' /tmp/notes/accounts.txt sed -i '/virus-quarantine*/d' /tmp/notes/accounts.txt for i in `cat /tmp/notes/accounts.txt` do note=`zmprov ga $i zimbraNotes |sed '1d'| cut -d ":" -f2-| sed '/^$/d' |sed -e 's/^[ \t]*//'` touch /tmp/notes/$i echo "$note" >> /tmp/notes/$i done find /tmp/notes -size -3c | xargs -n1 rm -v ls -l /tmp/notes | awk '{print $9}'|sed '1,2d' > /tmp/notes/accounts.txt
11. Backup COS (if different from the "source" server).
If you have only the default COS on the "client", but its with different settings from the COS on the "source" server, you have to rename it and then import it under different name.
- Download JXplorer from http://jxplorer.org/downloads/users.html
Create new entry as follows:
Host: "client" server IP Port 389 ; Protocol v3 Base DN: cn=zimbra
Authentication:
Level: User + Password User DN: cn=config Password is the output from: zmlocalconfig -s ldap_root_password
Click OK.
- From the left Tree View, expand cos.
- Select the COS that you want to export&move to the "source" server.
- From the File menu line above, click on LDIF > Export Subtree > Save on the local disk.
- Do that with all COS you want to move.
(leave the JXplorer open for the import)
12. Create files for each COS, and their respective user members.
mkdir /migration/cos_users
#!/bin/bash # The result is a file/s for each COS, and the users within this specific COS # Note: If there are no custom created COS, there will be no users listed in the default and default_external files. That # means all users are in the "default" COS. # Since the default and the default_external files will contain no users, their respective files are deleted at # the end. Zimbra only detect users that are not in the default COS. Thus, every non detected user is in the default COS. source ~/bin/zmshutil; zmsetvars for i in `zmprov gac` do id=`zmprov gc $i | grep -e "\bzimbraId\b"|cut -d " " -f 2` ldapsearch -x -H $ldap_master_url -D $zimbra_ldap_userdn -w $zimbra_ldap_password| grep -e $id -e "dn: uid=" -e mail:|grep -v Email| grep -e zimbraCOSId -B1 -A1|grep mail|awk '{print $2}' > /migration/cos_users/$i /bin/rm -rf /migration/cos_users/default* done
13. Backup Resource/s.
The resource backup is the same as the COS explained in point 8 above. The one difference is that you need to specify different "Base DN:" when you login. The Resource is type of a user, and to see the users, we need to use the following bind dn:
dc=domain,dc=com
As you substitute the above "domain" and "com" with your specific information. The rest is the same as in point 8, as you click on the resource users in order to export them.
14. RSYNC the backed up data to the new server.
rsync -e ssh -axvzKHS /migration/ root@10.0.0.1:/migration/
The 10.0.0.1 is the IP of the "source" server.
On the new server (SOURCE)
1. Restore COS.
To import to the "source" server:
- Create new entry in the JXplorer as in point 8, but this time specify the "source" server IP.
- Click on the "cos"
- LDIF > Import File > Select File > Import
Do that with all COS you want to import.
2. Restore Resource/s.
Same as restoring COS, except the difference in the binding base dn, explained above in point 10.
3. Recreate all domains.
cd /migration for i in `cat /migration/domains.txt`; do zmprov cd $i zimbraAuthMech zimbra ;echo $i ;done
4. Create the accounts and set the original passwords.
You may have a warning that the resource users are already created, ignore it.
#!/bin/bash # Creates the users and sets the old passwords. # Use dummy password "zimbra" during the user creation if [ $(whoami) != 'zimbra' ]; then echo "Must be zimbra to run $0" exit 1; fi USERPASS="/migration/userpass" USERDDATA="/migration/userdata" USERS="/migration/accounts.txt" for i in `cat $USERS` do givenName=$(grep givenName: $USERDDATA/$i.txt | cut -d ":" -f2) displayName=$(grep displayName: $USERDDATA/$i.txt | cut -d ":" -f2) shadowpass=$(cat $USERPASS/$i.shadow) zmprov ca $i zimbra cn "$givenName" displayName "$displayName" givenName "$givenName" zmprov ma $i userPassword "$shadowpass" done
5. Restore accounts data from the tgz files.
for i in `cat /migration/accounts.txt`; do zmmailbox -z -m $i postRestURL "/?fmt=tgz&resolve=skip" /migration/accounts/$i.tgz ; echo "$i done "; done
6. Add the migrated users from the "client" to their respective COS on the "source".
#!/bin/bash # The script first list the exported cos names, then iterate through all members within this cos and finally assigns the # user to the COS. for cos in `/bin/ls -1 /migration/cos_users | tr " " "\n"` do for i in `cat /migration/cos_users/$cos` do zmprov sac $i $cos done done
7. Recreate the distribution lists.
for i in `cat dlist.txt`; do zmprov cdl $i ; echo "$i done " ; done
8. Add members to the distribution lists.
#!/bin/bash # add all memebers to each of these distribution lists if [ $(whoami) != 'zimbra' ]; then echo "Must be zimbra to run $0" exit 1; fi for i in `cat /migration/dlist.txt` do for j in `grep -v '#' dl/$i.txt |grep '@'` do zmprov adlm $i $j echo " $j has been added to list $i" done done
9. Restore the aliases.
#!/bin/bash if [ $(whoami) != 'zimbra' ]; then echo "Must be zimbra to run $0" exit 1; fi for i in `cat /migration/accounts.txt` do if [ -f "alias/$i.txt" ]; then for j in `grep '@' /migration/alias/$i.txt` do zmprov aaa $i $j echo "$i has alias $j" done fi done
10. Restore forwarding addresses
#!/bin/bash mkdir /tmp/script for i in `cat /tmp/script/accounts.txt` do fwd_emails=`cat /tmp/script/$i|awk '{print}' ORS=' zimbraMailForwardingAddress '|sed 's/.\{28\}$//' ` zmprov ma $i zimbraMailForwardingAddress $fwd_emails done
11. Restore distribution list/s aliases
#!/bin/bash mkdir /tmp/dist for i in `cat /tmp/dist/distlist.txt` do for j in `cat /tmp/dist/$i` do zmprov adla $i $j done done
12. Restore user notes
#!/bin/bash mkdir /tmp/notes for i in `cat /tmp/notes/accounts.txt` do note=`cat /tmp/notes/$i` zmprov ma $i zimbraNotes "$note" done
13. Post steps.
- Shut down the "client" server.
- Fix DNS.
- Confirm your accounts and domains are present and show active, as zimbra run: zmaccts.
- Confirm accounts show mailbox quota usage appropriately: zmprov gqu `zmhostname`.
- Confirm COS shows the correct data: zmprov cta domain.com.
- Send emails to test the new users.
Additional Content
Similar articles:
- https://wiki.zimbra.com/wiki/Network_Edition_Disaster_Recovery
- https://wiki.zimbra.com/wiki/How_to_move_ZCS_to_another_server
Benefits:
- Do not depend on the OS. You can migrate from CentOS to Ubuntu. Only requirement is ZCS versions to be same.
- Does not require running the installation script after the migration
- Most of the process can be scripted (except the cos/resources section).
Changing the hostname.
1. Stop the flow of mail/Zimbra and take backup.
2. The new hostname must exist in the DNS before zmsetservername is run.
Note: If the box is duplicate of production server, prevent this server from affecting the production server by halting communications between test and live servers.
3. Execute the following commands:
su - zimbra /opt/zimbra/libexec/zmsetservername -o old.server.com -n new.server.com
4. Reconfigure the system for its new hostname as necessary for the operating system, (It is very important to do that AFTER step 3 - edit /etc/hosts & /etc/sysconfig/network files).
5. Reboot.
6. Recreate the certificates for the new hostname:
/opt/zimbra/bin/zmcertmgr createca -new /opt/zimbra/bin/zmcertmgr createcrt -new -subjectAltNames <servername> -days 365 (select the other servers as subjAltNames, if its multi-server env) /opt/zimbra/bin/zmcertmgr deploycrt self /opt/zimbra/bin/zmcertmgr deployca /opt/zimbra/bin/zmcertmgr viewdeployedcrt zmcontrol restart
Additionally check: https://wiki.zimbra.com/wiki/ZmSetServerName#Clean_the_zmloggerhostmap