Difference between revisions of "User Migration"

m (fixed a little bug in the code to sync kerio to zimbra)
(Zimbra To Zimbra Server Migration)
Line 1,345: Line 1,345:
  
 
Here is a script that is a modification of the script above for using curl to migrate users between zimbra boxes using imap protocol.  It also transfers calendars and contacts from zimbra server to zimbra server.  I just started working on it and here is what i have so far.  It worked on two servers i just migrated last week, i hope it helps :)   
 
Here is a script that is a modification of the script above for using curl to migrate users between zimbra boxes using imap protocol.  It also transfers calendars and contacts from zimbra server to zimbra server.  I just started working on it and here is what i have so far.  It worked on two servers i just migrated last week, i hope it helps :)   
 +
 +
NOTE: Modified perl below: was not using IO::Scalar or File::Path so no need to use; now supports user names with "@" via wget; support for command line arguments; support for SSL imap; support for various input file separators; other samll tweaks - scottp@dd.com.au
  
 
<pre>
 
<pre>
Line 1,350: Line 1,352:
 
use strict;
 
use strict;
 
use warnings;
 
use warnings;
use POSIX;
 
use IO::Scalar; # for building up output on a string
 
use File::Path; # for removing directories
 
  
 
#############################################################################
 
#############################################################################
 
# Please make changes below to suit your system and requirements
 
# Please make changes below to suit your system and requirements
  
my $host1="192.168.55.14";
+
# NOTE: hard code host1 & host2 - or pass in as first two parameters to script
  #host1 is Source
+
my $host1=shift; #host1 is Source
   
+
my $host2=shift; #host2 is Dest
my $host2="192.168.55.12";
+
 
    #host2 is Dest
+
# NOTE: enable ssl for imapsync
   
+
my $imapsync = " -ssl1 -ssl2";
my $domain="xxxxx.com";
+
 
my $user="";
 
my $pass="";
 
 
##############################
 
##############################
 +
# NOTE: Do not set these here - they come from userlist.txt
 +
my $user;
 +
my $pass;
 +
 
my $data_file="userlist.txt";
 
my $data_file="userlist.txt";
 
open(DAT, $data_file) || die("Could not open file!");
 
open(DAT, $data_file) || die("Could not open file!");
my @raw_data=<DAT>;
+
while (<DAT>) {
close(DAT);
+
        chomp;
 +
        # NOTE: userlist.txt can have username@domain password (or comma, tab or |)
 +
        ($user,$pass)=split(/[\|\s\t,]+/,$_);
 +
 
 +
        ##############
 +
        open (TESTOUT, "imapsync --buffersize 18192000 --nosyncacls --subscribe --syncinternaldates --noauthmd5 --host1 $host1 --user1 $user --password1 $pass --host2 $host2 --user2 $user --password2 $pass $imapsync|");
 +
        while (<TESTOUT>){
 +
                print $_;
 +
        }
 +
        close TESTOUT;
 +
 
 +
        ### get contacts
 +
        open (TESTOUT, "wget --user $user --password $pass https://$host1/zimbra/user/$user/contacts.csv --no-check-certificate|");
 +
        while (<TESTOUT>){
 +
                print $_;
 +
        }
 +
        close TESTOUT;
 +
 
 +
        ### get calendar
 +
        open (TESTOUT, "wget --user $user --password $pass https://$host1/zimbra/user/$user/calendar.ics --no-check-certificate |");
 +
        while (<TESTOUT>){
 +
                print $_;
 +
        }
 +
        close TESTOUT;
  
foreach my $linein (@raw_data)
+
        ### import calendars
{
+
        open (TESTOUT, "curl -u $user:$pass --data-binary \@calendar.ics https://$host2/service/home/$user/calendar?fmt=ics --insecure|");
chop($linein);
+
        while (<TESTOUT>){
($user,$pass)=split(/\|/,$linein);
+
                print $_;
##############
+
        }
open (TESTOUT, "imapsync --buffersize 18192000 --nosyncacls --subscribe --syncinternaldates --noauthmd5 --host1 $host1 --user1 $user --password1 $pass --host2 $host2 --user2 $user --password2 $pass|");
+
        close TESTOUT;
while (<TESTOUT>){
 
print $_;
 
}
 
close TESTOUT;
 
 
 
### get contacts
 
open (TESTOUT, "wget https://".$user.":".$pass."@".$host1."/zimbra/user/".$user."/contacts.csv --no-check-certificate|");
 
while (<TESTOUT>){
 
print $_;
 
}
 
close TESTOUT;
 
  
### get calendar
+
        ### import contacts
open (TESTOUT, "wget https://".$user.":".$pass."@".$host1."/zimbra/user/".$user."/calendar.ics --no-check-certificate |");
+
        open (TESTOUT, "curl -u $user:$pass --data-binary \@contacts.csv https://$host2/service/home/$user/contacts?fmt=csv --insecure|");
while (<TESTOUT>){
+
        while (<TESTOUT>){
print $_;
+
                print $_;
}
+
        }
close TESTOUT;
+
        close TESTOUT;
### import calendars
 
open (TESTOUT, "curl -u ".$user.":".$pass." --data-binary \@calendar.ics https://".$host2."/service/home/".$user."/calendar?fmt=ics --insecure|");
 
while (<TESTOUT>){
 
print $_;
 
}
 
close TESTOUT;
 
### import contacts
 
open (TESTOUT, "curl -u ".$user.":".$pass." --data-binary \@contacts.csv https://".$host2."/service/home/".$user."/contacts?fmt=csv --insecure|");
 
while (<TESTOUT>){
 
print $_;
 
}
 
close TESTOUT;
 
### remove files
 
open (TESTOUT, "rm calendar.ics|");
 
while (<TESTOUT>){
 
print $_;
 
}
 
close TESTOUT;
 
##
 
open (TESTOUT, "rm contacts.csv|");
 
while (<TESTOUT>){
 
print $_;
 
}
 
close TESTOUT;
 
###############
 
  
 +
        ### remove files
 +
        unlink 'calendar.ics';
 +
        unlink 'contacts.csv';
 +
        ###############
 
}
 
}
 
</pre>
 
</pre>

Revision as of 06:54, 17 September 2008


It has been suggested to break out this page into seperate pages in the migration category. Thoughts? Talk page[1]

Migration from one mail system to another is often painful. It can result in a user having to look in two different places to find mail, or in the worst case mail disappearing.

Copying the contents of one mail system to another often seems like a good idea, that way if anything goes wrong at least the old mail is still available. Ideally you know exactly what you are going to do at the start, you move everything across, and then you kill the old mail system (or at least hide it from mail clients). That way users are not confused by having two sets of identical folders.

In reality, you may have a period of overlapping mail servers. After user data has been migrated to Zimbra, users can access their mail via the web client without any additional setup until their mail client has been configured to access the Zimbra server.

Migrating Mail

Bulk Creating the Accounts

The first step in migrating users is to create the accounts, this is a topic of its own at Bulk Create

Migrating from an existing IMAP server (Recommended Method)

Currently, the recommended method for migrating users to Zimbra from an existing IMAP server is with the imapsync tool written by Gilles Lamiral. When in doubt, please refer to the documentation for your version of imapsync for up-to-date options and settings.

The following guide to imapsync was originally posted to the forums by GertThiel:

Before you can use imapsync you must have both the source IMAP message store and ZCS up running and accessible to user accounts via IMAP. You can check that using an email client before starting the migration. You will need the login names (i.e. email addresses) and passwords for the users to be migrated. In addition, imapsync will not be able to authenticate to the Zimbra server until you enable clear text login for the IMAP service. You can set that option under the IMAP tab of the Global Settings or individual Server settings in the Zimbra Admin Console UI. There are also a few Perl module dependances, including; Digest::MD5, IO::Socket::SSL,Term::ReadKey, Digest:HMAC libmail:imapclient. You can install these with your favorite package manager, from RPM's or with cpan. Finally, consider that imapsync will be a heavy load on your CPU and memory; the system running the migration will be less responsive.

Imapsync will run faster and consume less CPU with larger I/O buffers. The default is only 4KB; increasing it by one or two orders of magnitude is recommended for typical Zimbra-ready configurations. Examples below use 8MB buffer size.

For this example my existing IMAP server is running on server.gtds.lan and I set up Zimbra on a new machine named zimbra.gtds.lan.

   imapsync --buffersize 8192000 --nosyncacls --subscribe --syncinternaldates \
   --host1 server.gtds.lan --user1 yourAccount --password1 yourPassword \
   --host2 zimbra.gtds.lan --user2 yourZimbraAccount --password2 yourZimbraPassword

Of course the complete command belongs on one line (signified by the backslashes: \).

A slightly more secure method is to write each password into a separate file, and then use the --passfile{1|2} options intead of the --password{1|2} options:

   imapsync --buffersize 8192000 --nosyncacls --subscribe --syncinternaldates \
   --host1 server.gtds.lan --user1 yourAccount --passfile1 yourPasswordFile \
   --host2 zimbra.gtds.lan --user2 yourZimbraAccount --passfile2 yourZimbraPasswordFile

If your old IMAP server doesen't support NAMESPACE you have to also add appropriate options for example on iMail 6.0 --sep1 . --prefix1 INBOX. are required.

If you receive this error (and using the above command line you are pretty much guaranteed to): NO CAPABILITY for AUTHENTICATE LOGIN from your Zimbra server please add the switch --noauthmd5 to the end of your very long command line.

Your destination account on the Zimbra server should also have Clear Text Login enabled. You can enable this under the administration GUI in the Configuration menu, under Servers, in the IMAP tab of your server. Click the checkbox. It is assumed your source server also has Clear Text login enabled, but to do that please read the documentation of your respective server.

You may interrupt imapsync at any time with CTRL-C. Simply restart the same command again to resume the migration. You can also run it more than once to sync changes for a staged migration. Imapsync will not copy the same data twice, so you do not have to be concerned about duplicate messages.

I used imapsync to migrate from a Cyrus IMAPd to Zimbra. The ReadMe lists a number of other IMAP servers compatible with imapsync.

script to download and build imapsync and all of the required packages

#!/bin/bash
#
#CopyLeft 2006 Steve Fink
#stevef-at-ublug.org
#
#This script will get all of
#the necessary packages to
#build imapsync 1.219 except OpenSSL
#OpenSSL & it's development libraries are
#required and OpenSSL is usually installed
#already so check and get the matching libraries
#
#you have to run this script as root
#or under sudo so it can do all the installs
#
#I don't recommend doing the extended tests
#they mostly fail anyway but imapsync still works


mkdir imapsync
cd imapsync
#
wget http://search.cpan.org/CPAN/authors/id/D/DC/DCONWAY/Parse-RecDescent-1.94.tar.gz
tar xvfpz Parse-RecDescent-1.94.tar.gz
rm Parse-RecDescent-1.94.tar.gz
cd Parse-RecDescent-1.94/
perl Makefile.PL
make
make install
#
cd ..
wget http://search.cpan.org/CPAN/authors/id/G/GA/GAAS/Digest-MD5-2.33.tar.gz
tar xvfpz Digest-MD5-2.33.tar.gz
rm Digest-MD5-2.33.tar.gz
cd Digest-MD5-2.33/
perl Makefile.PL
make
make install
#
cd ..
wget http://search.cpan.org/CPAN/authors/id/J/JS/JSTOWE/TermReadKey-2.30.tar.gz
tar xvfpz TermReadKey-2.30.tar.gz
rm TermReadKey-2.30.tar.gz
cd TermReadKey-2.30/
perl Makefile.PL
make
make install
#
cd ..
wget http://search.cpan.org/CPAN/authors/id/A/AS/ASPA/IO-Socket-SSL-0.81.tar.gz
tar xvfpz IO-Socket-SSL-0.81.tar.gz
rm IO-Socket-SSL-0.81.tar.gz
cd IO-Socket-SSL-0.81/
perl Makefile.PL
make
make install
#
cd ..
wget http://search.cpan.org/CPAN/authors/id/F/FL/FLORA/Net_SSLeay.pm-1.30.tar.gz
tar xvfpz Net_SSLeay.pm-1.30.tar.gz
rm Net_SSLeay.pm-1.30.tar.gz
cd Net_SSLeay.pm-1.30/
./Makefile.PL -t
make install
#
cd ..
wget http://search.cpan.org/CPAN/authors/id/D/DJ/DJKERNEN/Mail-IMAPClient-2.2.9.tar.gz
tar xvfpz Mail-IMAPClient-2.2.9.tar.gz
rm Mail-IMAPClient-2.2.9.tar.gz
cd Mail-IMAPClient-2.2.9/
perl Makefile.PL
make
make test
make install
#
cd ..
wget http://www.linux-france.org/prj/imapsync/dist/imapsync-1.219.tgz
tar xvfpz imapsync-1.219.tgz
rm imapsync-1.219.tgz
cd imapsync-1.219/
perl -c imapsync
make install
#
cd ..
cd ..
rm -Rf imapsync/

Save this script as buildimapsync, chmod 755 buildimapsync, sudo buildimapsync. Note that common build tools, such as make and a C compiler must be available.

Note: see Talk:User_Migration if you're having problems with the imapsync source.

Note: The current version (as of 2007 05 22) of imapsync is 1.219. If that last wget command returns 404 Not Found, check the parent folder for a newer version.

Note: The linux-france.org server has been unreachable for a very long time. Try a mirror site (v1.217) or (v1.223) if the link above does not work.

If you use version 1.223, once installed change line 1163 of /usr/bin/imapsync to this:

$d = UnixDate(ParseDate($d), "%d-%b-%Y %H:%M:%S %z");

(Thanks to Spremeau in the message boards for this tip)

Enabling Emails With Large File Attachments (Zimbra IMAP Max Size is 10mb)

The default Zimbra IMAP maximum email size is 10mb. This will cause errors to be raised during the imapsync for emails which are larger than 10mb. Errors such as the following will be displayed in the imapsync log.

+ Copying msg #131:11690588 to folder Folders/Projects/Consulting
flags from : [\Seen]["16-Jul-2007 14:57:35 +1000"]
Couldn't append msg #131 (Subject:[RE: Consultancy Approval]) to folder Folders/Projects/Consulting: 5160 BAD parse error: request too long

OR

+ Copying msg #87598:54935762 to folder INBOX
flags from : [\Seen]["08-Feb-2008 14:09:05 -0500"]
Couldn't append msg #87598 (Subject:[Re: photos]) to folder INBOX: 894 NO [TOOBIG] request too long

The Zimbra IMAP server uses zimbraFileUploadMaxSize to set the max size. The default is 10mb if it's not set. At this time the only way to set it is via the command line.

Use the following command as the zimbra user to increase to 50mb:

zmprov mcf zimbraFileUploadMaxSize 50000000

Then restart the mailboxd server process.

For more details see this thread, http://www.zimbra.com/forums/developers/1461-max-imap-message-size.html

UPDATE: Since Zimbra 5.0.6 the parameter has changed to zimbraMtaMaxMessageSize:

zmprov mcf zimbraMtaMaxMessageSize 50000000

See this bug report http://bugzilla.zimbra.com/show_bug.cgi?id=27610

imapsync over SSL

You can use SSL connections with imapsync providing "--ssl1" and/or "--ssl2" (and the "--port1 and/or --port2" if defaults are different) for each of the accounts you wish to migrate.

For example:

[root@freebsd ~]# imapsync \
--buffersize 8192000 --nosyncacls --subscribe --syncinternaldates \
--host1 oldmail.mydomain.com --user1 user1 --password1 secret1 --ssl1 \
--port1 993 \
--host2 zimbra.mydomain.com --user2 user2 --password2 secret2 --ssl2 \
--port2 993 \
--noauthmd5

Installing imapsync on FreeBSD 6.2:

1. Installing the binary:

[root@freebsd ~]# pkg_add -r imapsync

2. Installing the port (from latest sources):

[root@freebsd ~]# cd /usr/ports/mail/imapsync
[root@freebsd ~]# make install clean

Make sure you have updated your ports with portsnap. If you are updating your ports for the first time:

[root@freebsd ~]# portsnap fetch
[root@freebsd ~]# portsnap extract

More details on how to use portsnap:

http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/ports-using.html)

The following version worked for me using option 2 above:

Mail::IMAPClient version is 2.2.9 and imapsync version 1.223 2007/10/30


Installing imapsync on Ubuntu 7.10:

[user1@ubuntu:~]$ sudo apt-get install imapsync

The command above will automatically build imapsync. You should be able to use the command above to accomplish an imapsync over SSL migration.

The following version worked for me on Ubuntu 7.10:

Mail::IMAPClient version is 2.2.9 and imapsync version 1.219 2007/04/04


NOTE: imapsync will not be able to authenticate to ZCS until you "Enable clear text login" for the IMAP service. You can set that option under Global Settings or under Servers under IMAP in the ZCS Administration Console.

Batch Proccessing

You can automate multiple users by using this script.

    #!/bin/bash
    
    logfile="sinklog.txt"
    
    host1=123.123.123.123
    #host1 is Source
    
    host2=321.321.321.321
    #host2 is Dest
    
    domain=xyz.com
    #domain is where email account is
    #everything after @ symbol
    
    ###### Do not modify past here
    #######################################
    
    date=`date +%X_-_%x`
    
    echo "" >> $logfile
    echo "------------------------------------" >> $logfile
    echo "IMAPSync started..  $date" >> $logfile
    echo "" >> $logfile
    
    { while IFS=';' read  u1 p1; do 
            user=$u1"@"$domain    
    	     echo "Syncing User $user"
            date=`date +%X_-_%x`
            echo "Start Syncing User $u1"
            echo "Starting $u1 $date" >> $logfile
    imapsync --nosyncacls --syncinternaldates --host1 $host1 --user1 "$user" --password1 \ 
    "$p1"--host2 $host2 --user2 "$user" --password2 "$p1"
            date=`date +%X_-_%x`
            echo "User $user done"
            echo "Finished $user $date" >> $logfile
            echo "" >> $logfile
    
            done ; } < userlist.txt
    
    date=`date +%X_-_%x`
    
    echo "" >> $logfile
    echo "IMAPSync Finished..  $date" >> $logfile
    echo "------------------------------------" >> $logfile

Now create a CSV file with the user names and passwords in the following format.

    user;password
    user2;password2


Here is a second batch script that will allow different usernames on the Source and Destination servers. Name this file imapsyncbatch then chmod to 755 execute with ./imapsyncbatch can be used with the --dry option for a dry run.

        #!/bin/bash
        ##Modified by Steve Fink stevef-at-ublug.org
	##This IMAPSync Batch Script is used when you have different
	##usernames on the Source and Destination servers
	##(kinda what IMAPSync was intended for)
	##the format for the user-list.csv file is
	##sourceusername,sourcepassword,destinationusername,destinationpassword 
	
	## Get the info
		while [ -z $infile ]; do
			echo "What is the path to the input file?"
			read infile
		done
		while [ -z $host1 ]; do
			echo "What is the Source Host? (mail1.domain.com)"
			read host1
		done
		while [ -z $host2 ]; do
			echo "What is the Destination Host? (mail2.domain.com)"
			read host2
		done
		while [ -z $domain ]; do
			echo "What is the Domain? (domain.com)"
			read domain
		done
		while [ -z $logfile ]; do
			echo "Where would you like the log? (synclog.txt)"
			read logfile
		done

		if [ ! -f $infile ]
        	then
                	{
				echo "The input file does not exist!"
				echo ""
				echo "What is the path to the input file?"
				read infile
			}
		fi
		
		INPUTFILE=$infile
		
		clear
		echo ""
		echo ""
		echo "IMAPSync is about to begin using:"
		echo "Input File $INPUTFILE"
		echo "Source Host $host1"
		echo "Destination Host $host2"
		echo "Domain $domain"
		echo "Log File $logfile"
		echo ""
		echo ""
		echo "Is this information correct?"
                echo "Press Enter to continue or"
		echo "Hit CTRL+C to start over"
		read wait

	## Begin IMAPSync
		date=`date +%X_-_%x`
		echo "IMAPSync Logfile started @ $logfile"
		echo "" >> $logfile
		echo "------------------------------------" >> $logfile
    		echo "IMAPSync started..  $date" >> $logfile
    		echo "" >> $logfile

	#Get rid of the commas
		tr "," " " <$INPUTFILE | while read u1 p1 u2 p2
			do

        			user=$u1"@"$domain
           			user2=$u2"@"$domain
             			echo "Syncing User $user to $user2"
            			date=`date +%X_-_%x`
            			echo "Start Syncing User $user to $user2"
            			echo "Starting $u1 $date" >> $logfile
    				imapsync $1 --nosyncacls --syncinternaldates \
				--exclude "#KnownSpam" --exclude "#FalsePositives" \
				--exclude "Trash" --exclude "Deleted Items" \
				--exclude "Deleted Messages" --exclude "Deleted" \
				--exclude "Sent" --exclude "Sent Items" \
				--exclude "Sent Messages" \
				--host1 $host1 --user1 "$user" --password1 "$p1" \
				--host2 $host2 --user2 "$user2" --password2 "$p2"
            			date=`date +%X_-_%x`
            			echo "User $user to $user2 done"
            			echo "Finished $user to $user2 $date" >> $logfile
            			echo "" >> $logfile

            		done

    		date=`date +%X_-_%x`

    		echo "" >> $logfile
    		echo "IMAPSync Finished..  $date" >> $logfile
    		echo "------------------------------------" >> $logfile

The format for the .csv file for this script is: sourceusername,sourcepassword,destinationusername,destinationpassword

Batch processing without knowing the passwords

In the case your authentication backend stores user password in encrypted format and you don't know them, you may have the possibility to add a second authentication backend with the same users but a password of your choice.

During a cyrus imap to zimbra migration, here's how it worked :

  • temporary openldap installed on third server, provisioned with the list of accounts to be migrated
  • zimbra's external authentication configured with this ldap server (you can use 2 different ldap urls, only bind account must be the same)
  • cyrus imap was already set up to use pam_ldap, which did not manage well the presence of 2 different ldap servers for authentication (if the official ldap server denied authentication, the second wasn't used)
  • if cyrus imap relies on pam_ldap, you can switch to pam_mysql for example. Create a database, table with login and password, and feed...

With that in place, you can keep your cyrus and zimbra server running with no change for the users, but synchronize with the scripts showed above.

Migrating from POP3

pop2imap is a tool used to get a pop inbox and syncronise it to an IMAP folder. You can get it from [2]

It is very similar to imapsync above.

   pop2imap  \
   --host1 some.pop.com.au --user1 yourAccount --passfile1 yourPasswordFile \
   --host2 zimbra.imap.com.au --user2 yourZimbraAccount --passfile2 yourZimbraPasswordFile \
   --folder MyOldPOPMail

Will copy the INBOX on your pop account to MyOldPOPMail folder.

There are some problems with pop2imap though.

  • It will not create the folder, you must have that manually created
  • It does not support SSL, however this is easy to do with "stunnel" - providing the SSL tunnel to the server
  • It downloads all of the messages first to get headers, although some pop servers do this well, many do not and this is a very time consuming operation. Modifications to the script are simple to skip this step and force a copy of the mail (technically no longer a sync, but a straight copy).

Also check this out... http://www.athensfbc.com/imap_tools/ it has batch mode with lists built in for all activities...POP3-IMAP, IMAP-IMAP...

Migrating from Dovecot

See User_Migration_From_Dovecot_With_External_LDAP

Migrating from iMail

The first step is to export the users information including the password. I used this utility for CommuniGate Pro which creates a fixed length file. I then coverted it to a Tab-Delimited file with this utility so that I could use the Bulk Create process. After you create your Domains and provision the accounts, turn on IMAP4 for iMail and use the IMAP migration process.

Migrating from Exchange

There are a least two ways to migrate the content of the individual MS Exchange user. First, is to use the Zimbra Migration Wizard. Second, is to export the contents of the MS Exchange folders to a .pst file and import them using the Zimbra PST Import Wizard. A third option is to export the MS Exchange data to a .pst file and restore it once the Zimbra profile is created for a corresponding Zimbra user either through the Zimbra Connector for Outlook or through IMAP.

See forums: Migration from Exchange

Information on using the Zimbra Exchange Migration Wizard can be found in the Migration Wizard for Exchange Installation Guide.

Migrating from Kerio Mail Server

Only mail can be migrated at the moment. Refer to "Migrating from an existing IMAP server" for instructions on how to setup imapsync.

Using the following imapsync command to migrate emails from a single user to zimbra

   perl imapsync --buffersize 8192000 --nosyncacls --subscribe --syncinternaldates \
   --host1 kerio.host.com --user1 user@host.com --password1 keriopasswd --sep1 "/" --prefix1 "" \
   --host2 zimbra.host.com --user2 user@host.com --password2 zimbrapasswd

Migrating from Outlook

If you are using the Outlook email client without Exchange, you can import the local mail folders directly into Zimbra with the PST Import Wizard. Information on using the PST Import Wizard can be found in the Import Wizard for Outlook Guide.

Migrating from MBOX files

The MBOX file format is commonly used by many programs, most notably sendmail. The advantage of migrating from MBOX files is that no passwords or special accounts are needed. The following perl script will forward from an MBOX file to the designated address, while preserving attachments:

    #!/usr/bin/perl

    use strict;
    use Email::Folder;
    use Mail::Mailer;
    use MIME::Parser;
    use Net::SMTP;

    my $mbox = $ARGV[0];
    my $email = $ARGV[1];
    my $server = $ARGV[2];

    $server = 'smtp' if(!defined($server));

    die "Usage: $0 mbox dest_address [smtp server]" if(!defined($mbox) || !-f $mbox);
    die "Usage: $0 mbox dest_address [smtp server]" if(!defined($email) || $email !~ m/\@/);

    my $folder = Email::Folder->new($mbox ||
        die "Usage: $0 mbox dest_address [smtp server]
        Forward all mail found in mail file mbox to address.
    ");

    my $count=0;
    my @messages=$folder->messages;
    my $total=@messages;

    foreach (@messages){
        $count++;
        my $parser = new MIME::Parser;
        $parser->output_under("/tmp");
        $parser->decode_headers(0);
        $parser->ignore_errors(1);
        my $entity = $parser->parse_data($_->as_string);
        my $header = $entity->head;
        my $sender = $entity->head->get('From');
        next if $header->get("subject") =~ m/FOLDER INTERNAL/;
        $header->replace('To', $email);
        $header->delete('Received');
        $header->delete('MIME-Version');
        $header->delete('Return-Path');
        $header->delete('User-Agent');
        $header->delete('Message-ID');
        $header->delete('X-Mailer');
        $header->delete('X-Security');
        $header->delete('X-Spam-Checker-Version');
        $entity->head($header);
        $entity->sync_headers;
        print "Message $count / $total\n";
        print "Sending message with subject: " . $entity->head->get("subject");
        print " to $email via $server\n";

        my $smtp = new Net::SMTP($server) or die "No mailserver";
        $smtp->mail($sender) or die "unable to set sender";
        $smtp->to($email) or die "unable to address message";
        $smtp->data() or die "unable to start data send";
        $smtp->datasend($entity->as_string()) or die "Message send failed";
        $smtp->dataend() or die "Message end failed";
        $smtp->quit();
        print "Done\n\n";
    }

A python solution that I found to be easier to install (required no extra installation on Centos 4.4) was the mbox2imap script found at: http://people.cs.uchicago.edu/~brendan/scripts/mbox2imap . Here is a modified version that adds the ability to pass the imap password on the command line:

#!/usr/bin/env python

# Upload mbox format email to an IMAP server

########################################################################
# Libraries
########################################################################

import email, email.Errors, mailbox, imaplib, getpass, sys, getopt
import os.path, StringIO, re

########################################################################
# Configuration defaults
########################################################################

# Store configuration in a dictionary so that we need only one argument
# to "global" in funtions to reference all the values.
#
config = {}

# Set defaults
#
config['self'] = os.path.basename(sys.argv[0])
config['verbose'] = 5 # "notice"; Syslog-style priority level
config['user'] = getpass.getuser()
config['imapmailbox'] = 'INBOX'
config['imapserver'] = 'laime.cs.uchicago.edu'
config['recursive_mode'] = 0
config['passwd'] = ''

########################################################################
# Functions
########################################################################

def main():
    output('info', 'main(): starting')

    global config

    process_options()

    if config['recursive_mode'] == 1:
        output('debug', 'main(): about to run recursive_upload()')
        recursive_upload()
    else:
        output('debug', 'main(): about to run single_upload()')
        single_upload()

    output('info', 'main(): completed')

########################################################################

# Log in to the IMAP server and return an object representing the
# authenticated session
#
def login():
    output('info', 'login(): starting')

    global config

    print "%s: Authenticating to IMAP server" % config['self']
    server = imaplib.IMAP4_SSL(config['imapserver'])
    if len(config['passwd']) == 0:
        check_response(server.login(config['user'], getpass.getpass()))
    else:
        check_response(server.login(config['user'], config['passwd']))
    return server

########################################################################

# TODO test mbox files with . in the name
# TODO test both absolute and relative paths in upload
# Given an mbox file path, translate it to an IMAP-style mailbox path
def source2target(source):
    output('info', 'source2target(%s): starting' % source)

    target = source

    if re.search(r'\.', source) != None:
        output('warning', 'mbox file "%s" contains "." character' % source)
        output('warning', 'Replacing "." with "_"')
        target = re.sub(r'\.', '_', target)

    # Translate to IMAP-style path separator (replace "/" with ".")
    target = re.sub(r'/', '.', target)

    # Strip off the containing directory (up to the first "." character)
    target = re.sub(r'^[^\.]+.', '', target)

    output('info', 'source2target(%s): returning %s' % (source, target) )

    return target

########################################################################

# Upload all files in a hierarchy to an IMAP server
def recursive_upload():
    output('info', 'recursive_upload(): starting')

    global config

    if not os.path.isdir(config['source']):
        output('crit', 'Argument must be a directory when using -r')
        output('crit', 'Given "%s")' % config['source'])
        sys.exit(1)

    source_list = build_file_list(config['source'])
    output('debug', 'recursive_upload(): source_list = %s' % source_list)

    # Need to know what to strip off when creating targets on the
    # IMAP server
    #
    base = os.path.dirname(config['source'])
    base = base + '/'

    target_list = source_list

    # If the source directory was specified as a path with more that
    # one component, we need to strip it down to the last componend(the
    # containing directory) since that is what the hierarchy on the IMAP
    # server will be created relative to.
    #
    target_list = map(lambda x: re.sub('^' + base, '', x), target_list)

    target_list = map(source2target, target_list)
    output('debug', 'recursive_upload(): target_list = %s' % target_list)

    # Want to be able to explain what is going to happen to the user
    # before they have to type in their password. That is why login()
    # is called here.
    #
    server = login()

    create_imap_mailboxes(target_list, server)

    for source in source_list:
        output('debug', 'recursive_upload(): source = %s' % source)

        target = source
        output('debug', 'recursive_upload(): target = %s' % target)

        output('debug', 'recursive_upload(): base = %s' % base)
        target = re.sub('^' + base, '', target)
        output('debug', 'recursive_upload(): target = %s' % target)

        target = source2target(target)
        output('debug', 'recursive_upload(): target = %s' % target)

        output('debug', 'recursive_upload(): source = %s' % source)
        output('debug', 'recursive_upload(): target = %s' % target)

        output('notice', 'Starting upload of %s to %s' % (source, target) )
        upload(source, target, server)
        output('notice', 'Finished upload of %s to %s' % (source, target) )

    output('info', 'recursive_upload(): completed')

########################################################################

def single_upload():
    output('info', 'single_upload(): starting')
    global config

    source = config['source']
    target = config['imapmailbox']

    # Want to be able to explain what is going to happen to the user
    # before they have to type in their password. That is why login()
    # is called here.
    #
    server = login()

    output('notice', 'Starting upload of %s to %s' % (source, target) )
    upload(source, target, server)
    output('notice', 'Finished upload of %s to %s' % (source, target) )

    output('info', 'single_upload(): completed')

########################################################################

# Process command line options
def process_options():
    output('info', 'process_options(): starting')

    global config

    try:
        opts, args = getopt.getopt(sys.argv[1:], "i:rs:u:v:p:")

    except getopt.GetoptError:
        usage()
        sys.exit(1)
    
    for option, argument in opts:
        if option == '-i':
            config['imapmailbox'] = argument
        if option == '-r':
            config['recursive_mode'] = 1
        if option == "-s":
            config['imapserver'] = argument
        if option == "-u":
            config['user'] = argument
        if option == "-v":
            config['verbose'] = argument
        if option == '-p':
            config['passwd'] = argument

    # Make sure desired log level is stored as an integer
    config['verbose'] = numeric_log_level(config['verbose'])

    output('debug', 'process_options(): opts = %s' % opts)
    output('debug', 'process_options(): args = %s' % args)

    # Summarize config
    output('debug', "process_options(): config['imapmailbox'] = %s" %
        config['imapmailbox'] )
    output('debug', "process_options(): config['imapserver'] = %s" %
        config['imapserver'] )
    output('debug', "process_options(): config['user'] = %s" %
        config['user'] )
    output('debug', "process_options(): config['verbose'] = %s" %
        config['verbose'] )
    output('debug', "process_options(): config['recursive_mode'] = %s" %
        config['recursive_mode'] )

    if len(args) == 0:
        usage()
        sys.exit()

    if len(args) != 1:
        output('crit', 'Too many file arguments: %s' % ' '.join(args))
        output('crit', 'Expecting only one; aborting')
        sys.exit(1)

    config['source'] = args[0]

    output('info', 'process_options(): completed')

########################################################################

# Return true if file is in mbox format
def is_mbox_file(file):
    output('info', 'is_mbox_file(%s): starting' % file)

    return open(file).readline()[0:5] == 'From '

########################################################################

# Given a directory, return a list of contained mbox files
def build_file_list(node):
    output('info', 'build_file_list(%s): starting' % node)

    file_children = []
    directory_children = []

    for entry in os.listdir(node):
        if os.path.isfile(node + '/' + entry):
            if is_mbox_file(node + '/' + entry):
                file_children.append(entry)
        elif os.path.isdir(node + '/' + entry):
            directory_children.append(entry)

    # Add containing directory to each entry
    flat = map(lambda x: node + '/' + x, file_children)

    # Recursively process directory children
    for entry in directory_children:
        flat.extend(build_file_list(node + '/' + entry))

    return flat

########################################################################

def create_imap_mailboxes(imap_mailboxes, server):
    output('info', 'create_imap_mailboxes(%s): starting' % imap_mailboxes)

    # Attempting to create a mailbox that already exists produces an
    # IMAP protocol error, so we only want to attempt to create a
    # mailbox that does not exist. To do this, we need a list of the
    # current mailboxes. We can get that with the list() method of the
    # IMAP4_SSL object, but the output it returns is formatted in a
    # strange way:
    #
    #    (\Noinferiors) "." "INBOX"
    #
    # We need to extract the string in the INBOX location. Use map() to
    # iterate over the list and pull out the folder name using a regular
    # expression.
    #
    extract = lambda x: re.search(r'^.*"\." "(.*)"', x).group(1)
    current_mailboxes = server.list()[1]
    current_mailboxes = map(extract, current_mailboxes)

    for mailbox in imap_mailboxes:
        if not current_mailboxes.__contains__(mailbox):
            output('notice', 'Creating mailbox: ' + mailbox)
            check_response(server.create(mailbox))

########################################################################

# Take an integer or string log level and return an integer log level
#
def numeric_log_level(level):
    # If level is an integer between 0 and 7, pass it back
    if range(8).__contains__(level):
        return(level)
    if level == 'debug':
        return(7)
    if level == 'info':
        return(6)
    if level == 'notice':
        return(5)
    if level == 'warning':
        return(4)
    if level == 'err':
        return(3)
    if level == 'crit':
        return(2)
    if level == 'alert':
        return(1)
    if level == 'emerg':
        return(0)
    # crit, alert, emerg: critical error, immediate termination
    # err: non-fatal problem
    # warning: possibly negative informational message
    # notice: neutral informational... TODO
    # info: function calls, arguments
    # debug: protocol, data details

    output('warning', 'Unknown log level "%s", assuming "emerg"' % level)
    return(0)

########################################################################

# Take an integer or string log level and return a string log level
#
def string_log_level(level):
    string_levels = ['emerg', 'alert', 'crit', 'err', 'warning',
                     'notice', 'info', 'debug']

    # If level is already a valid string, pass it back
    if string_levels.__contains__(level):
        return(level)

    # If level is a string between 0 and 7, return appropriate string
    if range(8).__contains__(level):
        return(string_levels[level])

    output('warning', 'Unknown log level "%s", assuming "emerg"' % level)
    return('emerg')
    
########################################################################

def output(level, message):
    global config

    if numeric_log_level(level) <= config['verbose']:
        print "%s: (%s) %s" % (config['self'],
                               string_log_level(level),
                               message)

########################################################################

# TODO
def usage():
    global config

    print '''Usage: %s [OPTION]... FILE
Upload contents of mbox FILE to an SSL IMAP server.

  -i MAILBOX     when not using -r, upload to MAILBOX (default: %s)
  -r             recursively upload mbox files (FILE must be a directory)
  -s SERVER      connect to SERVER (default: %s)
  -u USER        authenticate as USER
  -v LEVEL       set verbosity to LEVEL (syslog priority style)
  -p PASSWORD    password for USER

Note: "." characters are not allowed in IMAP mailbox names or directory
names. Such characters will be converted to "_" on the server.

When using -r, IMAP mailbox names will be derived from mbox file
hierarchy structure.

Warning: Please do not delete source mail until you have verified that
it has been uploaded successfully. This tool has been written with
safety in mind, but there are no guarantees.
''' % (config['self'], config['imapmailbox'], config['imapserver'])

########################################################################

def msgfactory(fp):
    try:
        return email.message_from_file(fp)
    except email.Errors.MessageParseError:
        # Don't return None since that will stop the mailbox iterator
        return ''

########################################################################

def check_response(response):
    output('info', 'check_response(): starting')

    r = response[0]
    data = response[1]

    output('debug', 'IMAP protocol response: %s' % str(r))
    output('debug', 'IMAP protocol data: %s' % str(data))

    if r != 'OK':
        output(1, "IMAP protocol error")
        output(1, "Protocol response: " + str(r))
        output(1, "Diagnostic message: " + str(data))

########################################################################

# Extract the subject from a string representing an email message
def get_subject(msg_txt):
    output('info', 'get_subject(): starting')

    buffer = StringIO.StringIO(msg_txt)

    for line in buffer:
        if re.search(r'^Subject:', line):
            return line.rstrip()
        if line == '\n':
            # End of headers. If we reached here, there is no subject.
            output('warning', 'Message does not have a subject')
            return ''

########################################################################
########################################################################

def check_response(response):
    output('info', 'check_response(): starting')

    r = response[0]
    data = response[1]

    output('debug', 'IMAP protocol response: %s' % str(r))
    output('debug', 'IMAP protocol data: %s' % str(data))

    if r != 'OK':
        output(1, "IMAP protocol error")
        output(1, "Protocol response: " + str(r))
        output(1, "Diagnostic message: " + str(data))

########################################################################

# Extract the subject from a string representing an email message
def get_subject(msg_txt):
    output('info', 'get_subject(): starting')

    buffer = StringIO.StringIO(msg_txt)

    for line in buffer:
        if re.search(r'^Subject:', line):
            return line.rstrip()
        if line == '\n':
            # End of headers. If we reached here, there is no subject.
            output('warning', 'Message does not have a subject')
            return ''

########################################################################

def upload(from_file, to_mailbox, server):
    output('info', 'upload(%s, %s): starting' % (from_file, to_mailbox) )

    fp = open(from_file, 'r')
    mbox = mailbox.UnixMailbox(fp, msgfactory)
    
    for msg_obj in mbox:
        msg_txt = msg_obj.as_string(unixfrom=False)

        subject = get_subject(msg_txt)
        output('notice', 'Uploading message: %s' % subject)

        # Regarding third argument to append,
        # see RFC 3501 sections 2.3.3, 6.3.11
        check_response(server.append(to_mailbox, "", "", msg_txt))

########################################################################

main()

I wrote a script that iterated over the mbox names, got the password from a csv file and uploaded the emails automatically using this tool. Hope this helps someone.

A third option, if the mbox file can be converted to maildir format, is to use the Zimbra zmmailbox utility to add messages to a user's mailbox. This will preserve the original time and date information, and will allow administrators to sort the mail into folders giong through IMAP.

Migrating from Lotus Notes/Domino

Information on migrating from Lotus Notes/Domino to ZCS can be found in the Migration Wizard for Domino Installation Guide

Migrating User Password from Shadow Password to Zimbra

Basically we will use the hash password from /etc/shadow. Get the hash values in between the first and second colon only. Eg:

user1:$1$MWAOIgPB$skcX.nKYV9JSctTR60uat/:13780:0:99999:7:::

Then run this as user zimbra.

zmprov ma user1@domain userPassword '{crypt}$1$MWAOIgPB$skcX.nKYV9JSctTR60uat/'

A simple batch script to migrate bulk shadow password into zimbra password as below:

#!/usr/bin/perl
# Usage: as root   # ./shadow2zm.pl /etc/shadow > shadow.zm
#        as zimbra # zmprov < shadow.zm

$domain="my.domain.com";

while(<>) {
    chomp;
    my ($uname,$pass) = split(/:/);

    print qq{zmprov ma $uname\@$domain userPassword '{crypt}$pass'\n};
    print qq{\n};
}

Credit to bewley from this forum post

./scalper

Migrating Contacts and Calendar

If your mail migration strategy doesn't cover contacts and calendar, you can import via the REST (REpresentational State Transfer) [3] interface. (Currently, contacts and calendar are only imported with the Exchange migration tool and the PST Import Wizard, so this applies to most IMAP migrations).

Procedure Overview

The basic procedure is this:

  1. Export calendar or contact data from your existing server into a csv or ics file
  2. Migrate that data file to a host that can access the zimbra server
  3. Use the REST interface to insert the data into Zimbra


REST overview

Today, within the Zimbra Collaboration Suite we have a number of different server-side URLs that our client accesses to download an attachment, export contacts as CSV, export a calendar as an ICS, file etc. We are also adding sharing (what would collaboration be without sharing, after all) of calendars, contacts, etc. Not only within a particular Zimbra community, but between Zimbra communities and the public at large.

In order to facilitate this, we are coming up with a clean, consistent URL interface to all our resources. The best way to describe this is with some examples.

Lets say I want access to my calendar folder from within iCal. The URL would look like:

 http://server/zimbra/user/roland/calendar

The default format on calendar folders is ICS, so no need to specify the format.

Lets say I want to export my contacts folder so I can import them into another account:

 http://server/zimbra/user/roland/contacts

Contact folders have a default type of CSV, so like calendars, no need to specify the format.

How about an RSS feed of unread messages in my inbox:

 http://server/zimbra/user/roland/inbox.rss?query=is:unread

By specifying an extension of ".rss" on the inbox folder, the server will automatically generate an RSS feed on it. Adding the "query" parameter lets me further refine what gets returned. You can also specify "?fmt=rss" instead of using the ".rss" extension if you'd like. You can also use the encoding for double quotes, %22 (For example, inbox.rss?query=%22is:unread%22)

Lets do something a little more interesting. How about a zip file containing all messages in my talks/ajax folder:

 http://server/zimbra/user/roland/talks/ajax.zip

The server zips them all up and returns the zip file.

Another interesting example is say you have created a public calendar that you want to share with everyone. Once you have granted access to the calendar, it is up to the consumer to chose what format they want to view it in:

 http://server/zimbra/user/roland/calendar/talks.ics
 http://server/zimbra/user/roland/calendar/talks.html?view=month
 http://server/zimbra/user/roland/calendar/talks.atom
 http://server/zimbra/user/roland/calendar/talks.xml
 http://server/zimbra/user/roland/calendar/talks.txt

How about accessing another user's calendar/folder? Once they grant you access, you can use the same exact syntax:

 http://server/zimbra/user/janie/holidays.ics

One last interesting example to leave you with. Lets say you have a friend at widgets.com who you know is running Zimbra and who has shared their calendar with you, but you don't know the name of their public Zimbra server. As long as they publish some DNS SRV records for _zimbra._tcp.widgets.com, then you can access it directly from your Zimbra without needing to know his server's address:

 http://server/zimbra/user/friend@widgets.com/calendar

This last format (user@domain.com) is also necessary when accessing resources in a non-default domain on your local Zimbra server.

REST file formats

Format Types

We are supporting a number of different "formats" for returned data. The formats we currently have implemented are:

REST file formats
Format Description
atom For generating an ATOM feed of mail messages and calendar appts.
csv For exporting contacts.
ics For exporting calendar appointments. See RFC 2445.
ifb Calendar free/busy data. See RFC 2445.
native Default formatter used to output messages and attachments in their "native" format. Used by the web client to reference attachments for downloading.
rss For generating an RSS feed of mail messages and calendar appts.
sync Similar to the native formatter, used by sync clients to retrieve raw message data, along with some extra meta-data in the HTTP header (tags, flags, received date).
vcf For exporting vcard files (vcard 3.0)
zip For exporting a set of messages (folder or search result) as a ZIP file.

Time Range Specifications

The time range can be specified in a number of different formats:

  • (milliseconds UTC)
  • MM/DD/YYYY
  • YYYY/MM/DD
  • {relative dates}

relative dates are either in the future (p/+/{not-specified}) or the past (m/-), along with a numeric value, followed by the units.

For units, everything after the first character is ignored (except for the "mi" case):

  • m(onths)
  • mi(nutes)
  • d(ays)
  • w(eeks)
  • h(ours)
  • y(ears)
Examples
1day 1 day from now
+2days 2 days from now
p1day 1 day from now
-2days 2 days ago
+60mi 60 minutes from now
+1week 1 week from now
+6mon 6 months from now
1year 1 year from now

The start/end query parameters apply to the atom, ics, rss, and ifb formatters.

Format Examples

ATOM

 http://server/zimbra/home/schemers/inbox.atom  

Returns an Atom feed for all messages in the inbox. If you only want unread messages as part of the feed, you could include a search query:

 http://server/zimbra/home/schemers/inbox.atom?query="is:unread"

You can also request an Atom feed of a calendar folder as well:

 http://server/zimbra/home/schemers/calendar.atom  

Note that "calendar" is the pathname to the folder called "calendar", it isn't a special-cased pathname. If you had a calendar folder called "talks", then you can access that folder via:

 http://server/zimbra/home/schemers/talks.atom  

When used with a calendar folder, the Atom feed will (by default) generate a feed of all appointments -/+ 7 days from the current time. To specify a different time range, you can use the start/end query parameters:

 http://server/zimbra/home/schemers/calendar.atom?start=0days&end=30days  


CSV

 http://server/zimbra/home/schemers/contacts.csv  

Exports all contacts in the contacts folder. If you want to export only contacts tagged "zimbra", you can specify a query:

 http://server/zimbra/home/schemers/contacts.csv?query=tag:zimbra  

This will only export contacts that are tagged with "zimbra".

ICS

 http://server/zimbra/home/schemers/calendar.ics  

Will export all appointments in the calendar folder in the ICS format, suitable for import into a calendar client such as Apple's iCal, or Mozilla's Sunbird calendar client.

IFB

 http://server/zimbra/home/schemers/calendar.ifb?start=0d&end=60d  

Will export the free/busy data in the calendar folder for the next 60 days. Note that this is the free/busy data for the calendar folder only. If someone has multiple calendar folders then you'd want to use the following URL instead:

 http://server/zimbra/home/schemers/?fmt=ifb&start=0d&end=60d  

That URL will return free/busy data across all calendars that are configured to be included in free/busy data.

NATIVE

The native formatter is used to request raw RFC 822 messages (for example, to implement "show original") or to down an attachment. For example, lets assume that message id 376 in my inbox is a multipart/mixed MIME message, and part 2 is an image.

The following URL (which is normally generated automatically when you are viewing your mail and click on an attachment), would download that attachment and display it in your browser:

 http://server/zimbra/home/schemers/?id=376&part=2  

Lets say part 3 in that same message is a word doc, you could display that via:

 http://server/zimbra/home/schemers/?id=376&part=3  

Finally, lets say you have the Network edition installed and want to view that same word doc as HTML instead of downloading the word doc and opening it up as an attachment. Adding "view=html" to the end of the URL will convert the word doc to HTML:

 http://server/zimbra/home/schemers/?id=376&part=3&view=html  

An upcoming release will expose this functionality directly in the web UI.

RSS

 http://server/zimbra/home/schemers/inbox.rss  

Returns an RSS feed for all messages in the inbox. All the other examples given for the Atom formatter also apply to the RSS formatter.

One further note on both the Atom and RSS formatters is that the calendar feeds provided by them currently only include the title/notes data. As soon as a more complete server-side I18N infrastructure is in place (we have been focused on the web client) the Atom/RSS feeds for calendars will be much more detailed.

SYNC

The sync formatter is similar to the native formatter, with the only exception being it adds some extra meta-data to the output for sync clients.

VCF

The "vcf" formatter can export your contacts as series of vcards (vCard 3.0), or can be used to export a single contact as a vCard (to be used by the web client in a future version).

To get all your contacts as a series of vCard entries, specify "vcf" as the formatter on the GET url:

 http://server/zimbra/home/schemers/contacts.vcf  

ZIP

The Zip formatter allows you to request a set of messages (currently only mail messages, at some point will probably allow attachments/contacts/appts as well) returned to you as a Zip file.

For example, if you want a copy of all messages in your inbox, you could use the following URL:

 http://server/zimbra/home/schemers/inbox.zip  

If you wanted all messages across all folders that were from Ross, you could use a query that returned only those:

 http://server/zimbra/home/schemers/?fmt=zip&query="from:ross"  

The zip file will consist of a set of RFC822 files, with the name of the file based on the subject of the message.

Importing into Zimbra

We've added the ability to update/create content by POST'ing content to REST urls. GETs continue to remain read only, with no side-effects (as they should be).

For example, you can POST an RFC822 formatted message to your inbox REST url to append messages to the inbox folder. Using the popular curl program, this would look like the following:

 curl -u schemers:password --data-binary @/tmp/rfc822.txt https://server/service/home/schemers/inbox

Note, you currently have to use /service/home for POSTs instead of /zimbra/home, because /zimbra/home issues a redirect, which isn't allowed with POSTs. We'll be fixing that in an upcoming release.

Other items that can be updated this way are calendar appointments (ICS), and contacts (csv and the new vcf (vCard) format):

 curl -u schemers:password --data-binary @/tmp/new.csv http://server/service/home/schemers/contacts?fmt=csv
 curl -u schemers:password --data-binary @/tmp/new.ics http://server/service/home/schemers/calendar?fmt=ics
 curl -u schemers:password --data-binary @/tmp/new.vcf http://server/service/home/schemers/contacts?fmt=vcf

The full user name (user@domain.com) is required when importing to resources on a non-default domain on your local Zimbra server.

NOTE - this works for resource accounts as well.

The target "contacts" in the above command corresponds to the actual folder name on the Zimbra server, rather than a service. This means that you can import to user-defined folders by specifying the appropriate target name in the command. If you have created a folder named "SharedContacts", for example, the command to import into this folder would be

 curl -u schemers:password --data-binary @/tmp/new.csv http://server/service/home/schemers/sharedcontacts?fmt=csv

Migrating Contacts from Clients into Zimbra

From Eudora

1) In Eudora, select the Address Book that you wish to import to Zimbra. (Select Tools/Address Book) and then Click on the correct Book.

2) Select File/Save As, Save as type: CSV Files (*.csv) and type in an appropriate file name. Click Save

3) Open Notepad, Open the file.csv that you saved in step 2.

4) Insert the following line EXACTLY. (Must be the first line in the file.)

nickname,email,fullName,firstName,lastName,homeStreet,homeCity,homeState,homeCountry,homePostalCode,homePhone,homeFax,homePhone2,homeURL,company,jobTitle,workStreet,workCity,workState,workCountry,workPostalCode,workPhone,workFax,workPhone2,workURL,email2,otherPhone,otherURL,notes

5) (Make sure to press enter after the above line and save and close notepad.)

6) Login to Zimbra, and go to Options/Address Book, Click Browse and select the file. Click import.

7) Select the address book to import into or select New to create new address book. Wait until box in bottom left hand corner says import complete.

Click Address Book and make sure the contacts imported.

Zimbra To Zimbra Server Migration

Here is a script that is a modification of the script above for using curl to migrate users between zimbra boxes using imap protocol. It also transfers calendars and contacts from zimbra server to zimbra server. I just started working on it and here is what i have so far. It worked on two servers i just migrated last week, i hope it helps :)

NOTE: Modified perl below: was not using IO::Scalar or File::Path so no need to use; now supports user names with "@" via wget; support for command line arguments; support for SSL imap; support for various input file separators; other samll tweaks - scottp@dd.com.au

#!/usr/bin/perl
use strict;
use warnings;

#############################################################################
# Please make changes below to suit your system and requirements

# NOTE: hard code host1 & host2 - or pass in as first two parameters to script
my $host1=shift; #host1 is Source
my $host2=shift; #host2 is Dest

# NOTE: enable ssl for imapsync
my $imapsync = " -ssl1 -ssl2";

##############################
# NOTE: Do not set these here - they come from userlist.txt
my $user;
my $pass;

my $data_file="userlist.txt";
open(DAT, $data_file) || die("Could not open file!");
while (<DAT>) {
        chomp;
        # NOTE: userlist.txt can have username@domain password (or comma, tab or |)
        ($user,$pass)=split(/[\|\s\t,]+/,$_);

        ##############
        open (TESTOUT, "imapsync --buffersize 18192000 --nosyncacls --subscribe --syncinternaldates --noauthmd5 --host1 $host1 --user1 $user --password1 $pass --host2 $host2 --user2 $user --password2 $pass $imapsync|");
        while (<TESTOUT>){
                print $_;
        }
        close TESTOUT;

        ### get contacts
        open (TESTOUT, "wget --user $user --password $pass https://$host1/zimbra/user/$user/contacts.csv --no-check-certificate|");
        while (<TESTOUT>){
                print $_;
        }
        close TESTOUT;

        ### get calendar
        open (TESTOUT, "wget --user $user --password $pass https://$host1/zimbra/user/$user/calendar.ics --no-check-certificate |");
        while (<TESTOUT>){
                print $_;
        }
        close TESTOUT;

        ### import calendars
        open (TESTOUT, "curl -u $user:$pass --data-binary \@calendar.ics https://$host2/service/home/$user/calendar?fmt=ics --insecure|");
        while (<TESTOUT>){
                print $_;
        }
        close TESTOUT;

        ### import contacts
        open (TESTOUT, "curl -u $user:$pass --data-binary \@contacts.csv https://$host2/service/home/$user/contacts?fmt=csv --insecure|");
        while (<TESTOUT>){
                print $_;
        }
        close TESTOUT;

        ### remove files
        unlink 'calendar.ics';
        unlink 'contacts.csv';
        ###############
}

users and passwords go in a text file userlist.txt in the format: username|password.

With some versions of wget you may need to specify the username and password as command line arguements (ie --http-passwd=thepass --http-user=user@dom.com) instead of in the URL (ie https://usersname:pass@theurl). If you are getting the error Bad Port, then give this a try.


Copy Calendar From One Zimbra User to Another

In this example we will export a calendar called "LPO calendar" from one user to another user and name it "LPO" in the destination account.

  • Export the calendar "LPO calendar" from user SRC_USER:
[root@zimbra ~]# curl -u admin 'https://zimbra:7071/home/SRC_USER/LPO%20calendar?fmt=ics' > LPO.ics
  • Adjust all the meeting organizer addresses. If you don't do this, the destination user will see the appointments, but can not edit them.
[root@zimbra ~]# cat LPO.ics | sed 's/SRC_USER@/DST_USER@/g' > LPO-dest.ics
  • Create empty calendar in destination user account.
[root@zimbra ~]# su - zimbra -c 'zmmailbox -z -m DST_USER createFolder --view appointment /LPO'
  • Import modified ical data into new calendar.
[root@zebra ~]# curl -u "admin" --data-binary @LPO-dest.ics 'https://zimbra:7071/home/DST_USER/LPO?fmt=ics'


Copy All Messages, Contacts, Calendars, and Mail Filters

I have written a script, which can be run in batch, to copy user data between servers or between mailboxes. This will copy contacts and calendars from every folder no just /Contacts and /Calendar. I used this script two times to copy pretty much everything important when migrating from 4.5.8 to 5.0.2. It doesn't support shared folders, task lists, documents, briefcase, and some other things (I recreated shared folders manually with zmmailbox) but support could be added given enough motivation :)

ZCS Tools currently contains these Migration scripts written in Python/Bash. These scripts are packaged as zimbraMigration-0.x and require imapsync to be installed.

Migrating Sieve Filter Rules

You have a few options to migrate Sieve rules. Generally you can take existing rules and then just dump them into the account attribute zimbraMailSieveScript. The server will be able to process these rules. However, if you intend to enable the client to edit the rules you will need to make sure they conform to a ZCS-specific set of characteristics. There are two ways to do this.

Using zmmailbox addFilterRule

Within the zmmailbox command there is a "addFilterRule" command. This allows you to add a named filter to a particular user. Its usage defines the parameters that ZWC can understand for editing. Given that for any migration scenario you'll likely end up parsing the rules, this will let you reconstruct the rules in a syntax ZWC understands. You will need to find a "name" for each rule. It could be as simple as "rule1", "rule2", etc.

Here's the usage for addFilterRule [zimbra@qa03 ~]$ zmmailbox help filter

 addFilterRule(afrl)                    add filter rule
 deleteFilterRule(dfrl)                 add filter rule
 getFilterRules(gfrl)                   get filter rules
 modifyFilterRule(mfrl)                 add filter rule
 {conditions}:
   header "name" is|not_is|contains|not_contains|matches|not_matches "value"
   header "name" exists|not_exists
   date before|not_before|after|not_after "YYYYMMDD"
   size under|not_under|over|not_over "1|1K|1M"
   body contains|not_contains "text"
   addressbook in|not_in "header-name"
   attachment exists|not_exists
 {actions}:
   keep
   discard
   fileinto "/path"
   tag "/tag"
   mark read|flagged
   redirect "address"
   stop

Also, for add filter rule: mbox> afrl usage:

 addFilterRule(afrl)          [opts] {name}  [*active|inactive] [any|*all] {conditions}+ {actions}+
   -a/--after <arg>             add after filter-name
   -f/--first                   add as first filter rule
   -l/--last                    add as last filter rule
   -b/--before <arg>            add before filter-name


Each zmmailbox script invokes a Java VM (just like zmprov). So you'll want to create a script to be loaded into zmmailbox. That script would look something like
adminAuthenticate ...
selectMailbox user1 ...
addFilterRule rule1 header "from" contains "kevin" discard
addFilterRule rule2 ....
...
selectMailbox user2
addFilterRule ....
...

Then you can run it via zmmailbox < scriptname.

Processing a Sieve script and inserting it via zmprov

Alternately you can process the existing Sieve scripts to conform to ZCS expectations and then insert the whole script via zmprov ma account zimbraMailSieveScript ..... You'll need to restrict the entered rules to the Sieve commands as documented above (if ZWC or ZCO are expected to edit them). Each filter rule will need a comment above it; this comment is the filter's name as identified in addFilterRule. Creating one or two simple rules using the web client will give you a framework for the created rule.


Troubleshooting

Please see User Migration Troubleshooting if you are having problems migrating data to your Zimbra installation.

Jump to: navigation, search