Difference between revisions of "How to merge two independent ZCS servers into one"

(How to merge two independent ZCS servers into ONE)
(How to merge two independent ZCS servers into ONE)
Line 37: Line 37:
 
  # su - zimbra
 
  # su - zimbra
  
3. Backup the data from the "client".
+
2. Backup the data from the "client".
  
3.1. backup the domains:
+
2.1. Backup the domains:
 
  cd /migration
 
  cd /migration
 
  zmprov gad > domains.txt
 
  zmprov gad > domains.txt
 
(open the domains.txt and remove all domains/subdomains that already exists on the "source")
 
(open the domains.txt and remove all domains/subdomains that already exists on the "source")
  
3.2 backup all admins
+
2.2 Backup all admins
 
  cd /migration
 
  cd /migration
 
  zmprov gaaa > admins.txt
 
  zmprov gaaa > admins.txt
  
3.3 backup all accounts
+
2.3 Backup all accounts
 
  cd /migration
 
  cd /migration
 
  zmprov -l gaa > accounts.txt
 
  zmprov -l gaa > accounts.txt
 
(open the file accounts.txt and remove the accounts starting with: spam,ham,virus,galsync)
 
(open the file accounts.txt and remove the accounts starting with: spam,ham,virus,galsync)
  
3.4. add the calendar resources to the accounts file
+
2.4. Add the calendar resources to the accounts file
 
  zmprov gacr >> accounts.txt
 
  zmprov gacr >> accounts.txt
  
3.5 backup all distribution lists
+
2.5 Backup all distribution lists
 
  cd /migration
 
  cd /migration
 
  zmprov gadl > dlist.txt
 
  zmprov gadl > dlist.txt
  
3.6 backup all members of the distribution lists
+
2.6 Backup all members of the distribution lists
 
  cd /migration
 
  cd /migration
 
  mkdir dl
 
  mkdir dl
 
  for i in `cat /migration/dlist.txt`; do zmprov gdlm $i > /migration/dl/$i.txt ;echo "$i"; done
 
  for i in `cat /migration/dlist.txt`; do zmprov gdlm $i > /migration/dl/$i.txt ;echo "$i"; done
  
4. backup all account passwords
+
4. Backup all account passwords
 
  cd /migration
 
  cd /migration
 
  mkdir userpass
 
  mkdir userpass
 
  for i in `cat accounts.txt`; do zmprov  -l ga $i userPassword | grep userPassword: | awk '{ print $2}' > userpass/$i.shadow; done
 
  for i in `cat accounts.txt`; do zmprov  -l ga $i userPassword | grep userPassword: | awk '{ print $2}' > userpass/$i.shadow; done
  
5. backup all user,display and given names
+
5. Backup all user,display and given names
 
  cd /migration
 
  cd /migration
 
  mkdir userdata
 
  mkdir userdata
 
  for i in `cat accounts.txt`; do zmprov ga $i  | grep -i Name: > userdata/$i.txt ; done
 
  for i in `cat accounts.txt`; do zmprov ga $i  | grep -i Name: > userdata/$i.txt ; done
  
6. backup all accounts data
+
6. Backup all accounts data
 
  cd /migration
 
  cd /migration
 
  mkdir accounts
 
  mkdir accounts
 
  for account in `cat /migration/accounts.txt`; do /opt/zimbra/bin/zmmailbox -z -m $account -t 0 getRestURL "//?fmt=tgz" > accounts/$account.tgz ;  echo $account done; done
 
  for account in `cat /migration/accounts.txt`; do /opt/zimbra/bin/zmmailbox -z -m $account -t 0 getRestURL "//?fmt=tgz" > accounts/$account.tgz ;  echo $account done; done
  
7. backup aliases
+
7. Backup aliases
 
  cd /migration
 
  cd /migration
 
  mkdir alias
 
  mkdir alias
Line 88: Line 88:
 
  find /migration/alias/* -size 0 | xargs rm -f
 
  find /migration/alias/* -size 0 | xargs rm -f
  
8. backup COS (if different from the "source" server)
+
8. Backup COS (if different from the "source" server)
 +
 
 
If you have only the default COS on the "client", but its with different settings from the COS on the "source" server, you have to rename it and then import it under different name.  
 
If you have only the default COS on the "client", but its with different settings from the COS on the "source" server, you have to rename it and then import it under different name.  
  
*download JXPlorer from http://jxplorer.org/downloads/users.html  (it is free software)
+
*Download JXPlorer from http://jxplorer.org/downloads/users.html  (it is free software)
 +
 
 
Create new entry as follows:  
 
Create new entry as follows:  
*Host: "client" server IP
+
Host: "client" server IP
*Port 389 ; Protocol v3
+
Port 389 ; Protocol v3
*Base DN: cn=zimbra
+
Base DN: cn=zimbra
 
Authentication:
 
Authentication:
*Level: User + Password
+
Level: User + Password
*User DN: cn=config
+
User DN: cn=config
*Password is the output from: zmlocalconfig -s ldap_root_password
+
Password is the output from: zmlocalconfig -s ldap_root_password
 
   
 
   
 
Click OK.  
 
Click OK.  
Line 110: Line 112:
  
 
9. Create files for each COS, and their respective user members.  
 
9. Create files for each COS, and their respective user members.  
 
 
  mkdir /migration/cos_users
 
  mkdir /migration/cos_users
 
  
 
SCRIPT:
 
SCRIPT:
Line 139: Line 139:
  
  
10. backup Resource
+
10. Backup Resource/s
  
 
The resource backup is the same as the COS explained in point 8 above. The one difference is that you need to specify different "Base DN:" when you login. The Resource is type of a user, and to see the users, we need to use the following bind dn:
 
The resource backup is the same as the COS explained in point 8 above. The one difference is that you need to specify different "Base DN:" when you login. The Resource is type of a user, and to see the users, we need to use the following bind dn:
Line 145: Line 145:
 
  dc=domain,dc=com
 
  dc=domain,dc=com
  
Az you substitute the above "domain" and "com" with your specific information. The rest is the same as in point 8.  
+
As you substitute the above "domain" and "com" with your specific information. The rest is the same as in point 8.  
 
 
--
 
  
11. RSYNC to the new server.
+
11. RSYNC the backed up data to the new server.
  
 
  rsync -e ssh -axvzKHS  /migration/ root@10.0.0.1:/migration/
 
  rsync -e ssh -axvzKHS  /migration/ root@10.0.0.1:/migration/
  
 
The 10.0.0.1 is the IP of the "source" server.
 
The 10.0.0.1 is the IP of the "source" server.
 
+
----
  
 
====On the new server (SOURCE)====
 
====On the new server (SOURCE)====
  
 
0. Restore COS
 
0. Restore COS
 +
 
To import to the "source" server:
 
To import to the "source" server:
 
*Create new entry in the Softerra LDAP Admin browser as in the import, but this time specify the "source" server IP.  
 
*Create new entry in the Softerra LDAP Admin browser as in the import, but this time specify the "source" server IP.  
Line 165: Line 164:
  
 
Do that with all COS you want to import.  
 
Do that with all COS you want to import.  
 
  
 
1. Restore Resource/s
 
1. Restore Resource/s
Line 177: Line 175:
  
 
3.1. Remove the resources from the accounts.txt file
 
3.1. Remove the resources from the accounts.txt file
 
  
 
SCRIPT:
 
SCRIPT:
Line 205: Line 202:
 
  for i in `cat /migration/accounts.txt`; do zmmailbox -z -m $i postRestURL "/?fmt=tgz&resolve=skip" /migration/accounts/$i.tgz ; echo "$i done "; done
 
  for i in `cat /migration/accounts.txt`; do zmmailbox -z -m $i postRestURL "/?fmt=tgz&resolve=skip" /migration/accounts/$i.tgz ; echo "$i done "; done
  
5. Add the moved users to their respecitve COS
+
5. Add the moved users to their respective COS
 
 
  
 
SCRIPT:
 
SCRIPT:
Line 224: Line 220:
  
  
6. Recreate distribution lists
+
6. Recreate the distribution lists
 
  for i in `cat dlist.txt`; do zmprov cdl $i ; echo "$i done " ; done
 
  for i in `cat dlist.txt`; do zmprov cdl $i ; echo "$i done " ; done
  
Line 249: Line 245:
 
  done
 
  done
  
8. Restore aliases
+
8. Restore the aliases
  
 
SCRIPT:
 
SCRIPT:

Revision as of 01:20, 24 December 2016

How to merge two independent ZCS servers into ONE

   KB 23120        Last updated on 2016-12-24  




0.00
(0 votes)


Purpose

This article covers the following:

Case scenario: There are two separate zimbra server, not connected in any way. The aim is to move the data from one of the servers to the other.

The steps below can be used to move a server to another new server too, or to another ZCS server in multi server environment.

Prerequisites

Lets call the server that the data will be moved to the "SOURCE", and the server that will move its data to the "source" the "CLIENT". The "source" and the "client" must have the same hostnames prior this operation. The OS versions of the "source" and the "client" can be different. Eg. migrate from centos to ubuntu is ok. The Zimbra versions must be the same, because we will be doing ldap export and import, which doesn't work if the ldap schema is different.

During the steps below, there are couple of scripts to be run. Make sure you make them executable (chmod +x file.sh) before that. If you are using vim to paste the text, use :set paste before pasting, otherwise when you paste the text the formatting will be gone.


Resolution

Overview of the steps:


On both servers create base directories for the migration:

# mkdir /migration
# chown zimbra.zimbra /migration


On the old server (CLIENT):


1. Become zimbra

# su - zimbra

2. Backup the data from the "client".

2.1. Backup the domains:

cd /migration
zmprov gad > domains.txt

(open the domains.txt and remove all domains/subdomains that already exists on the "source")

2.2 Backup all admins

cd /migration
zmprov gaaa > admins.txt

2.3 Backup all accounts

cd /migration
zmprov -l gaa > accounts.txt

(open the file accounts.txt and remove the accounts starting with: spam,ham,virus,galsync)

2.4. Add the calendar resources to the accounts file

zmprov gacr >> accounts.txt

2.5 Backup all distribution lists

cd /migration
zmprov gadl > dlist.txt

2.6 Backup all members of the distribution lists

cd /migration
mkdir dl
for i in `cat /migration/dlist.txt`; do zmprov gdlm $i > /migration/dl/$i.txt ;echo "$i"; done

4. Backup all account passwords

cd /migration
mkdir userpass
for i in `cat accounts.txt`; do zmprov  -l ga $i userPassword | grep userPassword: | awk '{ print $2}' > userpass/$i.shadow; done

5. Backup all user,display and given names

cd /migration
mkdir userdata
for i in `cat accounts.txt`; do zmprov ga $i  | grep -i Name: > userdata/$i.txt ; done

6. Backup all accounts data

cd /migration
mkdir accounts
for account in `cat /migration/accounts.txt`; do /opt/zimbra/bin/zmmailbox -z -m $account -t 0 getRestURL "//?fmt=tgz" > accounts/$account.tgz ;  echo $account done; done

7. Backup aliases

cd /migration
mkdir alias
for i in `cat accounts.txt`; do zmprov ga  $i | grep zimbraMailAlias |awk '{print $2}' > alias/$i.txt ;echo $i ;done

Remove the ones that are empty files:

find /migration/alias/* -size 0 | xargs rm -f

8. Backup COS (if different from the "source" server)

If you have only the default COS on the "client", but its with different settings from the COS on the "source" server, you have to rename it and then import it under different name.

Create new entry as follows:

Host: "client" server IP
Port 389 ; Protocol v3
Base DN: cn=zimbra

Authentication:

Level: User + Password
User DN: cn=config
Password is the output from: zmlocalconfig -s ldap_root_password

Click OK.

  • From the left Tree View, expand "cos"
  • Select the COS that you want to export&move to the "source" server
  • From the File menu line above, click on LDIF > Export Subtree > Save on the local disk.
  • Do that with all COS you want to move.

(leave the JXplorer open for the import)

9. Create files for each COS, and their respective user members.

mkdir /migration/cos_users

SCRIPT:


#!/bin/bash

# The result is a file/s for each COS, and the users within this specific COS
# Note: If there are no custom created COS, there will be no users listed in the default and default_external files. That 
# means all users are in the "default" COS.
# Since the default and the default_external files will contain no users, their respective files are deleted at 
# the end. Zimbra only detect users that are not in the default COS. Thus, every non detected user is in the default COS. 

source ~/bin/zmshutil; zmsetvars

for i in `zmprov gac`
   do
       id=`zmprov gc $i | grep -e "\bzimbraId\b"|cut -d " " -f 2`

       ldapsearch -x -H $ldap_master_url -D $zimbra_ldap_userdn -w $zimbra_ldap_password| grep -e $id -e "dn: uid=" -e mail:|grep -v Email| grep -e zimbraCOSId -B1 -A1|grep mail|awk '{print $2}' > /migration/cos_users/$i

/bin/rm -rf /migration/cos_users/default*

done


10. Backup Resource/s

The resource backup is the same as the COS explained in point 8 above. The one difference is that you need to specify different "Base DN:" when you login. The Resource is type of a user, and to see the users, we need to use the following bind dn:

dc=domain,dc=com

As you substitute the above "domain" and "com" with your specific information. The rest is the same as in point 8.

11. RSYNC the backed up data to the new server.

rsync -e ssh -axvzKHS  /migration/ root@10.0.0.1:/migration/

The 10.0.0.1 is the IP of the "source" server.


On the new server (SOURCE)

0. Restore COS

To import to the "source" server:

  • Create new entry in the Softerra LDAP Admin browser as in the import, but this time specify the "source" server IP.
  • Click on, and expand "cn=cos"
  • File > Import Data > Select the LDIF to import > Import

Do that with all COS you want to import.

1. Restore Resource/s

Same as restoring COS, except the difference in the binding base dn, explained above in point 8.

2. Recreate all domains

for i in `cat /migration/domains.txt`; do  zmprov cd $i zimbraAuthMech zimbra ;echo $i ;done

3. Create the accounts and set the original passwords

3.1. Remove the resources from the accounts.txt file

SCRIPT:


#!/bin/bash
# Creates the users and sets the old passwords. 
# Use dummy password "zimbra" during the user creation

if [ $(whoami) != 'zimbra' ]; then
       echo "Must be zimbra to run $0"
       exit 1;
fi		
USERPASS="/migration/userpass"
USERDDATA="/migration/userdata"
USERS="/migration/accounts.txt"
for i in `cat $USERS`
do
givenName=$(grep givenName: $USERDDATA/$i.txt | cut -d ":" -f2)
displayName=$(grep displayName: $USERDDATA/$i.txt | cut -d ":" -f2)
shadowpass=$(cat $USERPASS/$i.shadow)
zmprov ca $i zimbra cn "$givenName" displayName "$displayName" givenName "$givenName" 
zmprov ma $i userPassword "$shadowpass"
done

4. Restore accounts data from the tgz files.

for i in `cat /migration/accounts.txt`; do zmmailbox -z -m $i postRestURL "/?fmt=tgz&resolve=skip" /migration/accounts/$i.tgz ; echo "$i done "; done

5. Add the moved users to their respective COS

SCRIPT:


#!/bin/bash
# The script first list the exported cos names, then iterate through all members within this cos and finally assigns the 
# user to the COS. 

for cos in `/bin/ls -1 /migration/cos_users | tr " " "\n"`
do
	for i in `cat /migration/cos_users/$cos`
		do
		zmprov sac $i $cos
		done
done


6. Recreate the distribution lists

for i in `cat dlist.txt`; do zmprov cdl $i ; echo "$i done " ; done

7. Add members to the distribution lists

SCRIPT:


#!/bin/bash
# add all memebers to each of these distribution lists

if [ $(whoami) != 'zimbra' ]; then
       echo "Must be zimbra to run $0"
       exit 1;
fi		
 for i in `cat /migration/dlist.txt`
 do
	for j in `grep -v '#' dl/$i.txt |grep '@'` 
	do
	zmprov adlm $i $j
	echo " $j member has been added to list $i"
	done

done

8. Restore the aliases

SCRIPT:


#!/bin/bash

if [ $(whoami) != 'zimbra' ]; then
       echo "Must be zimbra to run $0"
       exit 1;
fi		
for i in `cat /migration/accounts.txt`
do
	if [ -f "alias/$i.txt" ]; then
	for j in `grep '@' /migration/alias/$i.txt`
	do
	zmprov aaa $i $j
	echo "$i HAS ALIAS $j --- Restored"
	done
	fi
done


9. Post steps

Shut down the "client" server Fix the DNS entries


Additional Content










Verified Against: Zimbra Collaboration 8.7, 8.6, 8.0 Date Created: 28/09/2016
Article ID: https://wiki.zimbra.com/index.php?title=How_to_merge_two_independent_ZCS_servers_into_one Date Modified: 2016-12-24



Try Zimbra

Try Zimbra Collaboration with a 60-day free trial.
Get it now »

Want to get involved?

You can contribute in the Community, Wiki, Code, or development of Zimlets.
Find out more. »

Looking for a Video?

Visit our YouTube channel to get the latest webinars, technology news, product overviews, and so much more.
Go to the YouTube channel »


Wiki/KB reviewed by SME1 SME2 COPY EDITOR Last edit by Teodor Vizirov
Jump to: navigation, search