User Migration: Difference between revisions

No edit summary
 
(79 intermediate revisions by 6 users not shown)
Line 1: Line 1:
{{Unsupported}}
{{BC|Community Sandbox}}
__FORCETOC__
<div class="col-md-12 ibox-content">
=User Migration=
{{KB|{{Unsupported}}|{{ZCS 8.0}}|{{ZCS 7.0}}|}}
{{WIP}}


=  Background =
{{Archive}}{{WIP}}


It has been suggested to break out this page into seperate pages in the migration category. Thoughts? Talk page[http://wiki.zimbra.com/index.php?title=Talk:User_Migration]
First, note that you'll want to create accounts in Zimbra first before you do the migration. If you want to do it in one shot, read [[Bulk Provisioning]].
[[Category:Migration]]


Migration from one mail system to another is often painful. It can result in a user having to look in two different places to find mail, or in the worst case mail disappearing.
Migrating your users means you're going to have to deal with [[Mail Migration|mail]],  [[Calendar and Contacts Migration|calendar, contacts]]  and [[Password Migration|passwords]]. This page used to be one monolithic monstrosity -- but has now been broken down into separate pages:


Copying the contents of one mail system to another often seems like a good idea, that way if anything goes wrong at least the old mail is still available. Ideally you know exactly what you are going to do at the start, you move everything across, and then you kill the old mail system (or at least hide it from mail clients).  That way users are not confused by having two sets of identical folders.
* [[Mail Migration]]
* [[Outlook NK2 Cache Rebuild]]
* [[Calendar and Contacts Migration]]
* [[Password Migration]]
* [[Email Rules Migration]]
* [[Aliases file Migration]]
* [[Migrating_from_Postfix_and_MySQL_with_bash|Postfix+MySQL aliases and accounts migration]]
* [[Migrating_from_Dovecot_passwd_with_bash|Migrating from Dovecot passwd]]
* [[Zimbra to Zimbra Server Migration]]
* [[Prevent duplicates messages for POP3 users post migration]]


In reality, you may have a period of overlapping mail servers.  After user data has been migrated to Zimbra, users can access their mail via the web client without any additional setup until their mail client has been configured to access the Zimbra server.
Please see [http://wiki.zimbra.com/index.php?title=User_Migration_Troubleshooting User Migration Troubleshooting] if you are having problems migrating data to your Zimbra installation.


=  Mail Migration =
[[Category:Migration]]
Note: the first step in migrating users is to create the accounts, this is a topic of its own at [[Bulk Create]]
{{Article_Footer|Zimbra Collaboration Suite 7.0|3/7/2006}}
 
== IMAP ==
 
=== from an existing IMAP server using imapsync (Recommended Method) ===
 
Currently, the recommended method for migrating users to Zimbra from an existing IMAP server is with the [http://freshmeat.net/projects/imapsync/ imapsync] tool written by Gilles Lamiral.  When in doubt, please refer to the documentation for your version of imapsync for up-to-date options and settings.
 
GertThiel originally posted [[Guide_to_imapsync|this guide]] to the zimbra forums.
 
==== from Kerio Mail Server (IMAP) ====
 
Only mail can be migrated at the moment. Refer to "Migrating from an existing IMAP server" for instructions on how to setup imapsync.
 
Using the following imapsync command to migrate emails from a single user to zimbra
 
    perl imapsync --buffersize 8192000 --nosyncacls --subscribe --syncinternaldates \
    --host1 kerio.host.com --user1 user@host.com --password1 keriopasswd --sep1 "/" --prefix1 "" \
    --host2 zimbra.host.com --user2 user@host.com --password2 zimbrapasswd
 
==== from Dovecot ====
See [[User_Migration_From_Dovecot_With_External_LDAP]]
 
=== from an existing IMAP server (Commercial Method) ===
 
Note: this was probably originally added by someone from YippieMove ([[User:Gettyless]])
 
[http://www.yippiemove.com YippieMove] is a SaaS for email migration. The service allows you move between any two email providers that supports IMAP, including Zimbra. No download is required and all transfers are performed on their servers. YippieMove charges $14.95 per account transfered (volume discount available).  A screencast is available [http://yippiemove.com/video/quick-introduction.html here].
 
== POP3 ==
 
=== pop2imap ===
 
pop2imap is a tool used to get a pop inbox and syncronise it to an IMAP folder. You can get it from [http://freshmeat.net/projects/pop2imap/]
 
It is very similar to imapsync above.
 
    pop2imap  \
    --host1 some.pop.com.au --user1 yourAccount --passfile1 yourPasswordFile \
    --host2 zimbra.imap.com.au --user2 yourZimbraAccount --passfile2 yourZimbraPasswordFile \
    --folder MyOldPOPMail
 
Will copy the INBOX on your pop account to MyOldPOPMail folder.
 
There are some problems with pop2imap though.
 
* It will not create the folder, you must have that manually created
* It does not support SSL, however this is easy to do with "stunnel" - providing the SSL tunnel to the server
* It downloads all of the messages first to get headers, although some pop servers do this well, many do not and this is a very time consuming operation. Modifications to the script are simple to skip this step and force a copy of the mail (technically no longer a sync, but a straight copy).
 
=== also check this out ===
 
Also check this out...
http://www.athensfbc.com/imap_tools/
it has batch mode with lists built in for all activities...POP3-IMAP, IMAP-IMAP...
 
 
 
== Migrating from iMail  ==
 
The first step is to export the users information including the password. I used [http://www.stalker.com/CGMigration/CGIMail.html this] utility
for CommuniGate Pro which creates a fixed length file. I then coverted it to a Tab-Delimited file with [http://files.twihd.com/EDC.exe this utility] so that I could use the [[Bulk Create]] process. After you create your Domains and provision the accounts, turn on IMAP4 for iMail and use the IMAP migration process.
 
== Migrating from Exchange ==
 
There are a least two ways to migrate the content of the individual MS Exchange user. First, is to use the Zimbra Migration Wizard. Second, is to export the contents of the MS Exchange folders to a .pst file and import them using the Zimbra PST Import Wizard.  A third option is to export the MS Exchange data to a .pst file and restore it once the Zimbra profile is created for a corresponding Zimbra user either through the Zimbra Connector for Outlook or through IMAP.
 
See forums: [http://www.zimbra.com/forums/showthread.php?t=2627 Migration from Exchange]
 
Information on using the Zimbra Exchange Migration Wizard can be found in the [http://www.zimbra.com/docs/ne/latest/migration_wizard_for_exchange_installation_guide/ Migration Wizard for Exchange Installation Guide].
 
 
 
== Migrating from Outlook ==
 
If you are using the Outlook email client without Exchange, you can import the local mail folders directly into Zimbra with the PST Import Wizard.  Information on using the PST Import Wizard can be found in the [http://www.zimbra.com/docs/ne/latest/import_wizard_for_outlook_guide/ Import Wizard for Outlook Guide].
 
== Migrating from MBOX files (i.e. the default sendmail format) ==
 
The MBOX file format is commonly used by many programs, most notably sendmail.  The advantage of migrating from MBOX files is that no passwords or special accounts are needed.  The following perl script will forward from an MBOX file to the designated address, while preserving attachments:
 
===Perl===
 
<pre>
    #!/usr/bin/perl
 
    use strict;
    use Email::Folder;
    use Mail::Mailer;
    use MIME::Parser;
    use Net::SMTP;
 
    my $mbox = $ARGV[0];
    my $email = $ARGV[1];
    my $server = $ARGV[2];
 
    $server = 'smtp' if(!defined($server));
 
    die "Usage: $0 mbox dest_address [smtp server]" if(!defined($mbox) || !-f $mbox);
    die "Usage: $0 mbox dest_address [smtp server]" if(!defined($email) || $email !~ m/\@/);
 
    my $folder = Email::Folder->new($mbox ||
        die "Usage: $0 mbox dest_address [smtp server]
        Forward all mail found in mail file mbox to address.
    ");
 
    my $count=0;
    my @messages=$folder->messages;
    my $total=@messages;
 
    foreach (@messages){
        $count++;
        my $parser = new MIME::Parser;
        $parser->output_under("/tmp");
        $parser->decode_headers(0);
        $parser->ignore_errors(1);
        my $entity = $parser->parse_data($_->as_string);
        my $header = $entity->head;
        my $sender = $entity->head->get('From');
        next if $header->get("subject") =~ m/FOLDER INTERNAL/;
        $header->replace('To', $email);
        $header->delete('Received');
        $header->delete('MIME-Version');
        $header->delete('Return-Path');
        $header->delete('User-Agent');
        $header->delete('Message-ID');
        $header->delete('X-Mailer');
        $header->delete('X-Security');
        $header->delete('X-Spam-Checker-Version');
        $entity->head($header);
        $entity->sync_headers;
        print "Message $count / $total\n";
        print "Sending message with subject: " . $entity->head->get("subject");
        print " to $email via $server\n";
 
        my $smtp = new Net::SMTP($server) or die "No mailserver";
        $smtp->mail($sender) or die "unable to set sender";
        $smtp->to($email) or die "unable to address message";
        $smtp->data() or die "unable to start data send";
        $smtp->datasend($entity->as_string()) or die "Message send failed";
        $smtp->dataend() or die "Message end failed";
        $smtp->quit();
        print "Done\n\n";
    }
</pre>
 
===Perl2===
<pre>
You will need the following CPAN packages:
 
    * Mail::IMAPClient to generate the folders and add the messages
    * Mail::Box::Mbox to read the messages from the mbox files
    * Mail::Message::Convert::MailInternet to get the Mail::Message objects into RFC 822 format
    * File::Find to make it easy to download the whole thing
</pre>
<pre>
#!/usr/bin/perl
#
# mbox2imap.pl
#
# Perl script to migrate a Kmail mbox store to IMAP. Change check_file for evolution
#
# Copyright 2005 Marco R. Gazzetta
 
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU General Public License for more details.
#
# contact the author directly for more information at:
# marcog AT pobox com
# http://diva.homelinux.org
#
use File::Find;
use Mail::IMAPClient;
use Mail::Box::Mbox;
use Mail::Message::Convert::MailInternet;
 
my $cache = "$ENV{HOME}/.mailboxes";
my %folders = ();
if (-e $cache) {
open CACHE, $cache;
while (<CACHE>) {
chomp;
@array = split /\|/;
$folders{$array[0]} = $array[1];
}
} else {
# First, get the local folders
find(\&check_file, "$ENV{HOME}/Mail");
open CACHE, "> $cache";
for (sort keys %folders) {
print CACHE "$_|$folders{$_}\n";
}
}
sub check_file {
# the regex checks for files whose name does NOT start with a dot
# notice that File::Find sets $_ to the file name, while $File::Find::name
# is with path
if (!/^\..*/) {
my $file = $File::Find::name;
my $name = $File::Find::name;
# delete non-kmail part of name
$name =~ s/\/[^.]+\//\//g;
# now normalize the directories
$name =~ s/\/\.([^\/]+)\.directory\//\/$1\//g; # replace the directory artifacts
$name =~ s/\/\.([^\/]+)\.directory\//\/$1\//g; # twice, because the first regex gets only half the line
$name =~ s/\//./g; # replace all slashes with dots
$name =~ s/\./_/g; # replace all dots with underscores
$folders{"INBOX$name"} = $file;
}
}
# open connection
my $imap = Mail::IMAPClient->new(
Server => 'server.domain.com',
User => 'marco',
Password => 'supersecret') || die "Could not connect to IMAP server.\n";
# push all messages in all folders (sorting is only so that a folder is created before the subfolders)
for $f (sort keys %folders) {
my $l = $folders{$f};
if (!$imap->exists($f)) {
print "WARNING: Creating folder " . $f . " since it does not exist on the server.\n";
$imap->create($f) or print "ERROR: Could not create folder.\n";
}
# Now copy all files from the local store to the IMAP store
my $box = Mail::Box::Mbox->new("folder" => $l);
my $conv = Mail::Message::Convert::MailInternet->new;
print $f . ": " . $imap->message_count($f) . "\n"; # debug output only
my $count = 1;
foreach ($box->messages()) {
$imap->append($f, $conv->export($_)->as_string()); # as_string ensures RFC 822 format
print $count++ . "\n";
}
print $f . ": " . $imap->message_count($f) . "\n";
}
$imap->close();
</pre>
 
===Python===
A python solution that I found to be easier to install (required no extra installation on Centos 4.4) was the mbox2imap script found at: http://people.cs.uchicago.edu/~brendan/scripts/mbox2imap . Here is a modified version that adds the ability to pass the imap password on the command line:
 
<pre>
#!/usr/bin/env python
 
# Upload mbox format email to an IMAP server
 
########################################################################
# Libraries
########################################################################
 
import email, email.Errors, mailbox, imaplib, getpass, sys, getopt
import os.path, StringIO, re
 
########################################################################
# Configuration defaults
########################################################################
 
# Store configuration in a dictionary so that we need only one argument
# to "global" in funtions to reference all the values.
#
config = {}
 
# Set defaults
#
config['self'] = os.path.basename(sys.argv[0])
config['verbose'] = 5 # "notice"; Syslog-style priority level
config['user'] = getpass.getuser()
config['imapmailbox'] = 'INBOX'
config['imapserver'] = 'laime.cs.uchicago.edu'
config['recursive_mode'] = 0
config['passwd'] = ''
 
########################################################################
# Functions
########################################################################
 
def main():
    output('info', 'main(): starting')
 
    global config
 
    process_options()
 
    if config['recursive_mode'] == 1:
        output('debug', 'main(): about to run recursive_upload()')
        recursive_upload()
    else:
        output('debug', 'main(): about to run single_upload()')
        single_upload()
 
    output('info', 'main(): completed')
 
########################################################################
 
# Log in to the IMAP server and return an object representing the
# authenticated session
#
def login():
    output('info', 'login(): starting')
 
    global config
 
    print "%s: Authenticating to IMAP server" % config['self']
    server = imaplib.IMAP4_SSL(config['imapserver'])
    if len(config['passwd']) == 0:
        check_response(server.login(config['user'], getpass.getpass()))
    else:
        check_response(server.login(config['user'], config['passwd']))
    return server
 
########################################################################
 
# TODO test mbox files with . in the name
# TODO test both absolute and relative paths in upload
# Given an mbox file path, translate it to an IMAP-style mailbox path
def source2target(source):
    output('info', 'source2target(%s): starting' % source)
 
    target = source
 
    if re.search(r'\.', source) != None:
        output('warning', 'mbox file "%s" contains "." character' % source)
        output('warning', 'Replacing "." with "_"')
        target = re.sub(r'\.', '_', target)
 
    # Translate to IMAP-style path separator (replace "/" with ".")
    target = re.sub(r'/', '.', target)
 
    # Strip off the containing directory (up to the first "." character)
    target = re.sub(r'^[^\.]+.', '', target)
 
    output('info', 'source2target(%s): returning %s' % (source, target) )
 
    return target
 
########################################################################
 
# Upload all files in a hierarchy to an IMAP server
def recursive_upload():
    output('info', 'recursive_upload(): starting')
 
    global config
 
    if not os.path.isdir(config['source']):
        output('crit', 'Argument must be a directory when using -r')
        output('crit', 'Given "%s")' % config['source'])
        sys.exit(1)
 
    source_list = build_file_list(config['source'])
    output('debug', 'recursive_upload(): source_list = %s' % source_list)
 
    # Need to know what to strip off when creating targets on the
    # IMAP server
    #
    base = os.path.dirname(config['source'])
    base = base + '/'
 
    target_list = source_list
 
    # If the source directory was specified as a path with more that
    # one component, we need to strip it down to the last componend(the
    # containing directory) since that is what the hierarchy on the IMAP
    # server will be created relative to.
    #
    target_list = map(lambda x: re.sub('^' + base, '', x), target_list)
 
    target_list = map(source2target, target_list)
    output('debug', 'recursive_upload(): target_list = %s' % target_list)
 
    # Want to be able to explain what is going to happen to the user
    # before they have to type in their password. That is why login()
    # is called here.
    #
    server = login()
 
    create_imap_mailboxes(target_list, server)
 
    for source in source_list:
        output('debug', 'recursive_upload(): source = %s' % source)
 
        target = source
        output('debug', 'recursive_upload(): target = %s' % target)
 
        output('debug', 'recursive_upload(): base = %s' % base)
        target = re.sub('^' + base, '', target)
        output('debug', 'recursive_upload(): target = %s' % target)
 
        target = source2target(target)
        output('debug', 'recursive_upload(): target = %s' % target)
 
        output('debug', 'recursive_upload(): source = %s' % source)
        output('debug', 'recursive_upload(): target = %s' % target)
 
        output('notice', 'Starting upload of %s to %s' % (source, target) )
        upload(source, target, server)
        output('notice', 'Finished upload of %s to %s' % (source, target) )
 
    output('info', 'recursive_upload(): completed')
 
########################################################################
 
def single_upload():
    output('info', 'single_upload(): starting')
    global config
 
    source = config['source']
    target = config['imapmailbox']
 
    # Want to be able to explain what is going to happen to the user
    # before they have to type in their password. That is why login()
    # is called here.
    #
    server = login()
 
    output('notice', 'Starting upload of %s to %s' % (source, target) )
    upload(source, target, server)
    output('notice', 'Finished upload of %s to %s' % (source, target) )
 
    output('info', 'single_upload(): completed')
 
########################################################################
 
# Process command line options
def process_options():
    output('info', 'process_options(): starting')
 
    global config
 
    try:
        opts, args = getopt.getopt(sys.argv[1:], "i:rs:u:v:p:")
 
    except getopt.GetoptError:
        usage()
        sys.exit(1)
   
    for option, argument in opts:
        if option == '-i':
            config['imapmailbox'] = argument
        if option == '-r':
            config['recursive_mode'] = 1
        if option == "-s":
            config['imapserver'] = argument
        if option == "-u":
            config['user'] = argument
        if option == "-v":
            config['verbose'] = argument
        if option == '-p':
            config['passwd'] = argument
 
    # Make sure desired log level is stored as an integer
    config['verbose'] = numeric_log_level(config['verbose'])
 
    output('debug', 'process_options(): opts = %s' % opts)
    output('debug', 'process_options(): args = %s' % args)
 
    # Summarize config
    output('debug', "process_options(): config['imapmailbox'] = %s" %
        config['imapmailbox'] )
    output('debug', "process_options(): config['imapserver'] = %s" %
        config['imapserver'] )
    output('debug', "process_options(): config['user'] = %s" %
        config['user'] )
    output('debug', "process_options(): config['verbose'] = %s" %
        config['verbose'] )
    output('debug', "process_options(): config['recursive_mode'] = %s" %
        config['recursive_mode'] )
 
    if len(args) == 0:
        usage()
        sys.exit()
 
    if len(args) != 1:
        output('crit', 'Too many file arguments: %s' % ' '.join(args))
        output('crit', 'Expecting only one; aborting')
        sys.exit(1)
 
    config['source'] = args[0]
 
    output('info', 'process_options(): completed')
 
########################################################################
 
# Return true if file is in mbox format
def is_mbox_file(file):
    output('info', 'is_mbox_file(%s): starting' % file)
 
    return open(file).readline()[0:5] == 'From '
 
########################################################################
 
# Given a directory, return a list of contained mbox files
def build_file_list(node):
    output('info', 'build_file_list(%s): starting' % node)
 
    file_children = []
    directory_children = []
 
    for entry in os.listdir(node):
        if os.path.isfile(node + '/' + entry):
            if is_mbox_file(node + '/' + entry):
                file_children.append(entry)
        elif os.path.isdir(node + '/' + entry):
            directory_children.append(entry)
 
    # Add containing directory to each entry
    flat = map(lambda x: node + '/' + x, file_children)
 
    # Recursively process directory children
    for entry in directory_children:
        flat.extend(build_file_list(node + '/' + entry))
 
    return flat
 
########################################################################
 
def create_imap_mailboxes(imap_mailboxes, server):
    output('info', 'create_imap_mailboxes(%s): starting' % imap_mailboxes)
 
    # Attempting to create a mailbox that already exists produces an
    # IMAP protocol error, so we only want to attempt to create a
    # mailbox that does not exist. To do this, we need a list of the
    # current mailboxes. We can get that with the list() method of the
    # IMAP4_SSL object, but the output it returns is formatted in a
    # strange way:
    #
    #    (\Noinferiors) "." "INBOX"
    #
    # We need to extract the string in the INBOX location. Use map() to
    # iterate over the list and pull out the folder name using a regular
    # expression.
    #
    extract = lambda x: re.search(r'^.*"\." "(.*)"', x).group(1)
    current_mailboxes = server.list()[1]
    current_mailboxes = map(extract, current_mailboxes)
 
    for mailbox in imap_mailboxes:
        if not current_mailboxes.__contains__(mailbox):
            output('notice', 'Creating mailbox: ' + mailbox)
            check_response(server.create(mailbox))
 
########################################################################
 
# Take an integer or string log level and return an integer log level
#
def numeric_log_level(level):
    # If level is an integer between 0 and 7, pass it back
    if range(8).__contains__(level):
        return(level)
    if level == 'debug':
        return(7)
    if level == 'info':
        return(6)
    if level == 'notice':
        return(5)
    if level == 'warning':
        return(4)
    if level == 'err':
        return(3)
    if level == 'crit':
        return(2)
    if level == 'alert':
        return(1)
    if level == 'emerg':
        return(0)
    # crit, alert, emerg: critical error, immediate termination
    # err: non-fatal problem
    # warning: possibly negative informational message
    # notice: neutral informational... TODO
    # info: function calls, arguments
    # debug: protocol, data details
 
    output('warning', 'Unknown log level "%s", assuming "emerg"' % level)
    return(0)
 
########################################################################
 
# Take an integer or string log level and return a string log level
#
def string_log_level(level):
    string_levels = ['emerg', 'alert', 'crit', 'err', 'warning',
                    'notice', 'info', 'debug']
 
    # If level is already a valid string, pass it back
    if string_levels.__contains__(level):
        return(level)
 
    # If level is a string between 0 and 7, return appropriate string
    if range(8).__contains__(level):
        return(string_levels[level])
 
    output('warning', 'Unknown log level "%s", assuming "emerg"' % level)
    return('emerg')
   
########################################################################
 
def output(level, message):
    global config
 
    if numeric_log_level(level) <= config['verbose']:
        print "%s: (%s) %s" % (config['self'],
                              string_log_level(level),
                              message)
 
########################################################################
 
# TODO
def usage():
    global config
 
    print '''Usage: %s [OPTION]... FILE
Upload contents of mbox FILE to an SSL IMAP server.
 
  -i MAILBOX    when not using -r, upload to MAILBOX (default: %s)
  -r            recursively upload mbox files (FILE must be a directory)
  -s SERVER      connect to SERVER (default: %s)
  -u USER        authenticate as USER
  -v LEVEL      set verbosity to LEVEL (syslog priority style)
  -p PASSWORD    password for USER
 
Note: "." characters are not allowed in IMAP mailbox names or directory
names. Such characters will be converted to "_" on the server.
 
When using -r, IMAP mailbox names will be derived from mbox file
hierarchy structure.
 
Warning: Please do not delete source mail until you have verified that
it has been uploaded successfully. This tool has been written with
safety in mind, but there are no guarantees.
''' % (config['self'], config['imapmailbox'], config['imapserver'])
 
########################################################################
 
def msgfactory(fp):
    try:
        return email.message_from_file(fp)
    except email.Errors.MessageParseError:
        # Don't return None since that will stop the mailbox iterator
        return ''
 
########################################################################
 
def check_response(response):
    output('info', 'check_response(): starting')
 
    r = response[0]
    data = response[1]
 
    output('debug', 'IMAP protocol response: %s' % str(r))
    output('debug', 'IMAP protocol data: %s' % str(data))
 
    if r != 'OK':
        output(1, "IMAP protocol error")
        output(1, "Protocol response: " + str(r))
        output(1, "Diagnostic message: " + str(data))
 
########################################################################
 
# Extract the subject from a string representing an email message
def get_subject(msg_txt):
    output('info', 'get_subject(): starting')
 
    buffer = StringIO.StringIO(msg_txt)
 
    for line in buffer:
        if re.search(r'^Subject:', line):
            return line.rstrip()
        if line == '\n':
            # End of headers. If we reached here, there is no subject.
            output('warning', 'Message does not have a subject')
            return ''
 
########################################################################
########################################################################
 
def check_response(response):
    output('info', 'check_response(): starting')
 
    r = response[0]
    data = response[1]
 
    output('debug', 'IMAP protocol response: %s' % str(r))
    output('debug', 'IMAP protocol data: %s' % str(data))
 
    if r != 'OK':
        output(1, "IMAP protocol error")
        output(1, "Protocol response: " + str(r))
        output(1, "Diagnostic message: " + str(data))
 
########################################################################
 
# Extract the subject from a string representing an email message
def get_subject(msg_txt):
    output('info', 'get_subject(): starting')
 
    buffer = StringIO.StringIO(msg_txt)
 
    for line in buffer:
        if re.search(r'^Subject:', line):
            return line.rstrip()
        if line == '\n':
            # End of headers. If we reached here, there is no subject.
            output('warning', 'Message does not have a subject')
            return ''
 
########################################################################
 
def upload(from_file, to_mailbox, server):
    output('info', 'upload(%s, %s): starting' % (from_file, to_mailbox) )
 
    fp = open(from_file, 'r')
    mbox = mailbox.UnixMailbox(fp, msgfactory)
   
    for msg_obj in mbox:
        msg_txt = msg_obj.as_string(unixfrom=False)
 
        subject = get_subject(msg_txt)
        output('notice', 'Uploading message: %s' % subject)
 
        # Regarding third argument to append,
        # see RFC 3501 sections 2.3.3, 6.3.11
        check_response(server.append(to_mailbox, "", "", msg_txt))
 
########################################################################
 
main()
 
</pre>
 
Someone wrote a script that iterated over the mbox names, got the password from a csv file and uploaded the emails automatically using this tool. (location?)
 
===MBOX > Maildir > Zimbra hash-dirs===
 
If the mbox file can be converted to maildir format, you can use the [http://wiki.zimbra.com/index.php?title=Zmmailbox zmmailbox] utility to add messages to a user's mailbox.  This will preserve the original time and date information, and will allow administrators to sort the mail into folders giong through IMAP.
 
OR zmlmtpinject
 
Usage for both outlined here: http://www.zimbra.com/forums/installation/12617-recover-data-store-folders.html#post64962
 
<pre>
#!/usr/local/bin/perl
#
# mbox2maildir: coverts mbox file to maildir directory - the reverse of
# maildir2mbox from the qmail distribution.
#
# Usage: mbox2maildir uses the same environment variables as maildir2mbox:
# MAILDIR is the name of your maildir directory; MAIL is the name of your
# mbox file; MAILTMP is ignored.  MAIL is deleted after the conversion.
#
# WARNING: there is no locking; don't run more than one of these!  you
# have been warned.
#
# based on convert-and-create by Russell Nelson <nelson@qmail.org>
# kludged into this by Ivan Kohler <ivan@voicenet.com> 97-sep-17
require 'stat.pl';
local $SIG{HUP} = 'IGNORE';
local $SIG{INT} = 'IGNORE';
local $SIG{QUIT} = 'IGNORE';
local $SIG{TERM} = 'IGNORE';
local $SIG{TSTP} = 'IGNORE';
($name, $passwd, $uid, $gid, $quota, $comment, $gcos, $dir, $shell) =
  getpwuid($<);
die "fatal: home dir $dir doesn't exist\n" unless -e $dir;
&Stat($dir);
die "fatal: $name is $uid, but $dir is owned by $st_uid\n" if $uid != $st_uid;
chdir($dir) or die "fatal: unable to chdir to $dir\n";
$spoolname = "$ENV{MAILDIR}";
-d $spoolname or mkdir $spoolname,0700
  or die("fatal: $spoolname doesn't exist and can't be created.\n");
chdir($spoolname) or die("fatal: unable to chdir to $spoolname.\n");
-d "tmp" or mkdir("tmp",0700) or die("fatal: unable to make tmp/ subdir\n");
-d "new" or mkdir("new",0700) or die("fatal: unable to make new/ subdir\n");
-d "cur" or mkdir("cur",0700) or die("fatal: unable to make cur/ subdir\n");
open(SPOOL, "<$ENV{MAIL}")
  or die "Unable to open $ENV{$MAIL}\n";
$i = time;
while(<SPOOL>) {
  if (/^From /) {
    $fn = sprintf("new/%d.$$.mbox", $i);
    open(OUT, ">$fn") or die("fatal: unable to create new message");
    $i++;
    next;
  }
  s/^>From /From /;
  print OUT or die("fatal: unable to write to new message");
}
close(SPOOL);
close(OUT);
unlink("$ENV{MAIL}");
</pre>
 
== Migrating from Lotus Notes/Domino ==
 
Information on migrating from Lotus Notes/Domino to ZCS can be found in the [http://www.zimbra.com/docs/ne/latest/migration_wizard_for_domino_installation_guide/ Migration Wizard for Domino Installation Guide]
 
 
 
 
 
== Migrating from Maildir ==
=== Using Bash-Skript directly on the Server ===
Place this script in your /home directory (/home NOT /home/USER/ !!!) and run. Script will import all emails from "cur" and "new" directories from /home/*/Maildir/ (and all subdirectories) into /Import IMAP folder.
 
<pre>
#!/bin/bash
 
#
# Maildir to Zimbra import
#
# Created by Jarosław Czarniak on 26-10-2008
#
 
clear
chown -R zimbra:zimbra *
 
domain="jarsat.pl" # change to your domain!
IMPORT="/Import"
 
 
for user in `ls -d1 */|sed s/\\\///`
do
    echo
    echo "User $user"
    echo
    echo "createFolder $IMPORT" > /tmp/tmp.txt
    /opt/zimbra/bin/zmmailbox -z -m $user@$domain < /tmp/tmp.txt
 
    #
    # CUR
    #
    for sciezka in `find $user -maxdepth 10 -type d -name cur`
    do
    echo
    echo "Directory $sciezka"
    echo
 
    nazwa=`echo $sciezka|cut -f3 -d"/"|sed s/\\\.//`
    if [ "$nazwa" = "cur" ]
        then
            echo "addMessage ${IMPORT} $PWD/$user/Maildir/cur" > /tmp/tmp.txt
            /opt/zimbra/bin/zmmailbox -z -m $user@$domain < /tmp/tmp.txt
        else
            echo "createFolder ${IMPORT}/$nazwa" > /tmp/tmp.txt
            /opt/zimbra/bin/zmmailbox -z -m $user@$domain < /tmp/tmp.txt
            echo "addMessage ${IMPORT}/$nazwa ${PWD}/${sciezka}" > /tmp/tmp.txt
            /opt/zimbra/bin/zmmailbox -z -m $user@$domain < /tmp/tmp.txt
    fi
    done
 
    #
    # NEW
    #
    for sciezka in `find $user -maxdepth 10 -type d -name new`
    do
    echo
    echo "Directory $sciezka"
    echo
    nazwa=`echo $sciezka|cut -f3 -d"/"|sed s/\\\.//`
    if [ "$nazwa" = "new" ]
        then
            echo "addMessage ${IMPORT} $PWD/$user/Maildir/new" > /tmp/tmp.txt
            /opt/zimbra/bin/zmmailbox -z -m $user@$domain < /tmp/tmp.txt
        else
            echo "createFolder ${IMPORT}/$nazwa" > /tmp/tmp.txt
            /opt/zimbra/bin/zmmailbox -z -m $user@$domain < /tmp/tmp.txt
            echo "addMessage ${IMPORT}/$nazwa ${PWD}/${sciezka}" > /tmp/tmp.txt
            /opt/zimbra/bin/zmmailbox -z -m $user@$domain < /tmp/tmp.txt
    fi
    done
done
</pre>
 
=== Using Perl-Script for Network-Transfer by IMAP ===
This script needs to be run directly from the main-maildir, so first change into it with "cd". For kmail you would need to execute "cd $HOME/.kde/share/apps/kmail/mail". This script works recursive, but as maildir seems not to handle subdirs, kmail creates subfolders in parallel with prefix and suffix. For example: The maildir "Work/Administration" is located in the path ".Work.directory/Administration". I don't know, if other mail programs work the same way, so you should check that before you begin.
 
The script will report some details about the message if the import fails. Major problems I encountered while importing (you should check too):
- The maximum message size accepted by the server was to small
- The folder could not be created due to some special characters used on filesystem entries
 
To be able to run this script, you also need to enable the imap clear text login with:
<pre>
su - zimbra -c "zmprov mcf zimbraImapCleartextLoginEnabled TRUE"
</pre>
You can/should disable this option after migration!
 
<pre>
#!/usr/bin/perl
# by Andreas Kammlott, thanks to perl and its community
 
use HTTP::Date qw(str2time);
use HTTP::Date qw(parse_date);
use Date::Format;
use Mail::IMAPClient;
use Mail::Box::Maildir;
use Mail::Message::Convert::MailInternet;
 
# Open imap connection, adjust values to your needs
my $imap = Mail::IMAPClient->new(
  Server  => 'zimbra.domain.com',
  User    => 'test',
  Password => 'test12') || die "Could not connect to IMAP server.\n";
 
# A maildir contains a subdir "cur"
foreach my $dir (`find ./ -type d -name cur|sort`) {
  my $imdir = ""; # to store the path on the server (imap-directory)
  my $fsdir = ""; # to store the path of the maildir (filesystem)
 
  # Bring the path to an array, each directory needs to be handled
  my @fspath = split(/\//, $dir);
 
  # Replace the leading point and the tailing .directory to acquire
  # the path, which has to be created in imap, the "cur" don't need
  # to be stored
  for (my $i = 1; $i < @fspath-1; $i++) {
      $fsdir = "$fsdir$fspath[$i]/";
      if ($fspath[$i] =~ /\..*\.directory/) {
        $fspath[$i] =~ s/^\.(.+)\.directory$/$1/;
      }
      $imdir = "$imdir$fspath[$i]/";
  }
 
  # Create the mdir object, this object contains all the messages from
  # the filesystem's maildir
  my $mdir = Mail::Box::Maildir->new(folder => $fsdir)
      or print "Failed to create mdir object from \"$fsdir\"\n";
 
  # Process this directory only if it contains messages
  if (!$mdir->messages() > 0) {
      #print "Skipping... folder has been detected as empty.\n";
      next;
  }
 
  # Check for the folder on the server (imap) and create if necessary
  if (!$imap->exists($imdir)) {
      # Jump to next folder if creation of this one fails
      if (!$imap->create($imdir)) {
        print "ERROR: Could not create folder \"$imdir\".\n";
        next;
      }
  }
 
  # Now begin to push all messages
  foreach my $msg ($mdir->messages()) {
      my $subject = $msg->subject;
      my $msgdate = $msg->head->get('Date');
      my $msgid  = $msg->messageId;
 
      # Required date as RFC2060  dd-Mon-yyyy hh:mm:ss +0000
      # Provided format is Tue, 20 Jul 2004 09:55:16 +0200
      my $time = str2time($msgdate);
      (undef, undef, undef, undef, undef, undef, $tz) = parse_date($msgdate);
      my $date = time2str("%d-%h-%Y %H:%M:%S %z",$time,$tz)
          or print "problem with time: $time and tz: $tz\n";
 
      # Create a converter object
      my $conv = Mail::Message::Convert::MailInternet->new;
 
      # Don't understand why the $conv->export($msg)->as_string() has to be called twice!
      my $exp = $conv->export($msg)->as_string();
      if (!$imap->append_string($imdir, $conv->export($msg)->as_string(), undef, $date)) {
        print "Failed to append message!\n"
              ."ID:$msgid\n"
              ."SUBJECT:$subject\n"
              ."IMAPDIR:$imdir\n"
              ."MESSAGEDATE:$msgdate\n"
              ."IMPORTDATE:$date\n\n";
      }
  }
}
$imap->close();</pre>
 
== Migrating from Thunderbird Local Folders ==
 
The easiest way I found for importing a lot of Thunderbird local folders is to set up a temporary dovecot server, convert the thunderbird mail stores to linux mbox mailstores and then imapsync it from the temp server to the zimbra server:
 
1) Install dovecot on a temporary server. On Debian etch: apt-get install dovecot-imapd, and then edit /etc/dovecot.conf and set the protocols to include imap. Restart dovecot. That is enough for our purposes.
 
2) Download the nstouwimap script here: http://www.cs.rice.edu/~dwallach/nstouwimap/. It will be used to convert Thunderbird mailboxes to mbox files that can be read by dovecot.
 
3) Copy All the local folders from all users to the server.
 
4) Create a transfer user on the dovecot server.
 
4) In the user's home directory, create a mail folder: /home/transferuser/mail.
 
5) For each mailbox, do the following (this can be scripted):
 
5.1) Clear the old mail folder on the server: rm -rf /home/transferuser/mail/*
 
5.2) Run nstouwimap /path/to/user/LocalFolders /home/transferuser/mail to convert the Local Folders to the transfer user's dovecot mailbox.
 
5.3) Run imapsync to transfer the account over to zimbra.
 
= Password Migration  =
 
== Migrating Users Password from Postfix Admin ==
 
Save this script as something.php, make <i>chmod +x something.php</i> and run. Script will export all email accounts into designated file.
 
<pre>
#!/usr/bin/php5
 
// Postfix.admin to Zimbra import
//
// Created by Jarosław Czarniak on 26-10-2008
// Trivial bug fixed by Luca G. on 18-01-2009
 
<?
/////////////////////////////////////////////////////////
 
$user="mysql_login";
$pass="mysql_pass";
$base="mysql_database";
$tabl="mailbox"; //table
$file="exported.sh";
 
/////////////////////////////////////////////////////////
echo "Usage: as \"zimbra\" user on destination server:\n";
echo "# sh ./exported.sh\n\n";
echo "";
 
$mydb = mysql_connect('localhost',$user, $pass) or die ('Błąd połączenia z serwerem');
mysql_select_db($base);
mysql_query("SET CHARACTER SET utf8");
mysql_query("SET NAMES utf8");
 
$query = "SELECT username,password,name,maildir,quota,domain FROM ".$tabl;
$dane = mysql_query($query) or die ('Błąd podczas zapytania do bazy1'.mysql_error());
 
$handle = fopen($file, "w");
 
while ($row = mysql_fetch_array($dane, MYSQL_NUM))
{
    $StringData = "zmprov ca ".$row[0]." dsfs123hsdyfgbsdgfbsd displayName '".$row[2]."'\n";
    fwrite($handle, $StringData);
    $StringData = "zmprov ma ".$row[0]." userPassword '{crypt}".$row[1]."'"."\n";
    fwrite($handle, $StringData);
}
 
?>
</pre>
 
== Migrating User Password from Shadow Password to Zimbra==
 
Basically we will use the hash password from /etc/shadow. Get the hash values in between the first and second colon only. Eg:
 
user1:'''$1$MWAOIgPB$skcX.nKYV9JSctTR60uat/''':13780:0:99999:7:::
 
Then run this as user zimbra.  
 
<pre>
zmprov ma user1@domain userPassword '{crypt}$1$MWAOIgPB$skcX.nKYV9JSctTR60uat/'
</pre>
 
A simple batch script to migrate bulk shadow password into zimbra password as below:
 
<pre>
#!/usr/bin/perl
# Usage: as root  # ./shadow2zm.pl /etc/shadow > shadow.zm
#        as zimbra # zmprov < shadow.zm
 
$domain="my.domain.com";
 
while(<>) {
    chomp;
    my ($uname,$pass) = split(/:/);
 
    print qq{zmprov ma $uname\@$domain userPassword '{crypt}$pass'\n};
    print qq{\n};
}
</pre>
 
Credit to bewley from this [http://www.zimbra.com/forums/administrators/231-migrating-accounts-users-passwd-shadow-file.html forum post]
 
./scalper
 
In some cases above perl script doesn't work very well. And if user doesn't already exist you will get en error. My script will import and create all users from shadow file (with exception of system accounts).
<pre>
#!/bin/bash
 
# Shadow to Zimbra import
#
# Created by Jarosław Czarniak on 26-10-2008
#
 
clear
echo "Usage: as \"zimbra\" user on destination server"
echo "# zmprov < shadow.file"
 
domain="jarsat.pl"  # change to your domain!
file="shadow.file"
x=0
echo ''>$file
 
for linia in `cat /etc/shadow`
do
    user=`echo $linia|cut -f1 -d":"`
    pass=`echo $linia|cut -f2 -d":"`
 
    if [ "$pass" != "*" ]
    then
        if [ "$pass" != "!" ]
        then
            echo "zmprov ca $user@$domain temppasswordQAZXSW displayName $user">>$file
            echo "zmprov ma $user@$domain userPassword '{crypt}$pass'">>$file
            x=$[x+1]
        fi
    fi
done
 
echo
echo
echo "$x accounts exported to \"$PWD/$file\""
 
sleep 5
</pre>
 
= Calendar and Contacts Migration =
 
If your mail migration strategy doesn't cover contacts and calendar, you can import via the REST (REpresentational State Transfer) [http://en.wikipedia.org/wiki/Representational_State_Transfer] interface.  (Currently, contacts and calendar are only imported with the Exchange migration tool and the PST Import Wizard, so this applies to most IMAP migrations).
 
== Overview ==
 
The basic procedure is this:
 
# Export calendar or contact data from your existing server into a csv or ics file
# Migrate that data file to a host that can access the zimbra server
# Use the REST interface to insert the data into Zimbra
 
== REST overview ==
 
Today, within the Zimbra Collaboration Suite we have a number of different server-side URLs that our client accesses to download an attachment, export contacts as CSV, export a calendar as an ICS, file etc. We are also adding sharing (what would collaboration be without sharing, after all) of calendars, contacts, etc. Not only within a particular Zimbra community, but between Zimbra communities and the public at large.
 
In order to facilitate this, we are coming up with a clean, consistent URL interface to all our resources. The best way to describe this is with some examples.
 
Lets say I want access to my calendar folder from within iCal. The URL would look like:
 
  <nowiki>http://server/zimbra/user/roland/calendar</nowiki>
 
The default format on calendar folders is ICS, so no need to specify the format.
 
Lets say I want to export my contacts folder so I can import them into another account:
 
  <nowiki>http://server/zimbra/user/roland/contacts</nowiki>
 
Contact folders have a default type of CSV, so like calendars, no need to specify the format.
 
How about an RSS feed of unread messages in my inbox:
 
  <nowiki>http://server/zimbra/user/roland/inbox.rss?query=is:unread</nowiki>
 
By specifying an extension of ".rss" on the inbox folder, the server will automatically generate an RSS feed on it. Adding the "query" parameter lets me further refine what gets returned. You can also specify "?fmt=rss" instead of using the ".rss" extension if you'd like. You can also use the encoding for double quotes, %22  (For example, '''inbox.rss?query=%22is:unread%22''')
 
Lets do something a little more interesting. How about a zip file containing all messages in my talks/ajax folder:
 
  <nowiki>http://server/zimbra/user/roland/talks/ajax.zip</nowiki>
 
The server zips them all up and returns the zip file.
 
Another interesting example is say you have created a public calendar that you want to share with everyone. Once you have granted access to the calendar, it is up to the consumer to chose what format they want to view it in:
 
  <nowiki>http://server/zimbra/user/roland/calendar/talks.ics</nowiki>
  <nowiki>http://server/zimbra/user/roland/calendar/talks.html?view=month</nowiki>
  <nowiki>http://server/zimbra/user/roland/calendar/talks.atom</nowiki>
  <nowiki>http://server/zimbra/user/roland/calendar/talks.xml</nowiki>
  <nowiki>http://server/zimbra/user/roland/calendar/talks.txt</nowiki>
 
How about accessing another user's calendar/folder? Once they grant you access, you can use the same exact syntax:
 
  <nowiki>http://server/zimbra/user/janie/holidays.ics</nowiki>
 
One last interesting example to leave you with. Lets say you have a friend at widgets.com who you know is running Zimbra and who has shared their calendar with you, but you don't know the name of their public Zimbra server. As long as they publish some DNS SRV records for _zimbra._tcp.widgets.com, then you can access it directly from your Zimbra without needing to know his server's address:
 
  <nowiki>http://server/zimbra/user/friend@widgets.com/calendar</nowiki>
 
This last format (user@domain.com) is also necessary when accessing resources in a non-default domain on your local Zimbra server.
 
== REST file formats ==
 
see [[REST_File_Formats]]
 
== Importing into Zimbra ==
 
We've added the ability to update/create content by POST'ing content to REST urls. GETs continue to remain read only, with no side-effects (as they should be).
 
For example, you can POST an RFC822 formatted message to your inbox REST url to append messages to the inbox folder. Using the popular curl program, this would look like the following:
 
  <nowiki>curl -u schemers:password --data-binary @/tmp/rfc822.txt https://server/service/home/schemers/inbox</nowiki>
 
Note, you currently have to use /service/home for POSTs instead of /zimbra/home, because /zimbra/home issues a redirect, which isn't allowed with POSTs. We'll be fixing that in an upcoming release.
 
Other items that can be updated this way are calendar appointments (ICS), and contacts (csv and the new vcf (vCard) format):
 
  <nowiki>curl -u username:password --data-binary @/tmp/new.csv http://server/service/home/username/contacts?fmt=csv</nowiki>
 
  <nowiki>curl -u username:password --data-binary @/tmp/new.ics http://server/service/home/username/calendar?fmt=ics</nowiki>
 
  <nowiki>curl -u username:password --data-binary @/tmp/new.vcf http://server/service/home/username/contacts?fmt=vcf</nowiki>
 
The full user name (user@domain.com) is required when importing to resources on a non-default domain on your local Zimbra server.
 
NOTE - this works for resource accounts as well.
 
The target "contacts" in the above command corresponds to the actual folder name on the Zimbra server, rather than a service.  This means that you can import to user-defined folders by specifying the appropriate target name in the command.  If you have created a folder named "SharedContacts", for example, the command to import into this folder would be
 
  <nowiki>curl -u username:password --data-binary @/tmp/new.csv http://server/service/home/username/sharedcontacts?fmt=csv</nowiki>
 
 
On later version's (such as 5.0.10+) instead of --data-binary, you should use --upload-file.
 
For example:
curl --insecure -u admin:password --upload-file /tmp/test.csv
https://server/service/home/test1@domain.com/contacts?fmt=csv
 
Alternatively, you can use zmmailbox pru:
zmmailbox -z -m test1@domain.com pru /Contacts /tmp/test.csv
 
== Migrating Contacts into Zimbra ==
 
=== Eudora ===
1) In Eudora, select the Address Book that you wish to import to Zimbra. (Select Tools/Address Book) and then Click on the correct Book.
 
2) Select File/Save As, Save as type: CSV Files (*.csv) and type in an appropriate file name. Click Save
 
3) Open Notepad, Open the file.csv that you saved in step 2.
 
4) Insert the following line EXACTLY. (Must be the first line in the file.)
 
nickname,email,fullName,firstName,lastName,homeStreet,homeCity,homeState,homeCountry,homePostalCode,homePhone,homeFax,homePhone2,homeURL,company,jobTitle,workStreet,workCity,workState,workCountry,workPostalCode,workPhone,workFax,workPhone2,workURL,email2,otherPhone,otherURL,notes
 
5) (Make sure to press enter after the above line and save and close notepad.)
 
6) Login to Zimbra, and go to Options/Address Book, Click Browse and select the file. Click import.
 
7) Select the address book to import into or select New to create new address book. Wait until box in bottom left hand corner says import complete.
 
Click Address Book and make sure the contacts imported.
 
=== Thunderbird ===
 
1) Download Dawn Here: http://mysite.verizon.net/zakharin/software/Dawn/
 
2) Use Dawn Wizard to import contacts into Outlook.
 
3) Run the Outlook Import Wizard to import into Zimbra.
 
= Zimbra To Zimbra Server Migration =
 
== as a whole method ==
 
Unofficial method (work with 5.0, may work with <5.0):
http://www.zimbra.com/forums/migration/22697-zimbra-zimbra-migration-script.html
 
Official method (only work > 5.0.9)
http://www.zimbrablog.com/blog/archives/2008/09/zcs-to-zcs-migrations.html
 
== item to item method ==
Here is a script that is a modification of the script above for using curl to migrate users between zimbra boxes using imap protocol.  It also transfers calendars and contacts from zimbra server to zimbra server.  I just started working on it and here is what i have so far.  It worked on two servers i just migrated last week, i hope it helps :) 
 
NOTE: Modified perl below: was not using IO::Scalar or File::Path so no need to use; now supports user names with "@" via wget; support for command line arguments; support for SSL imap; support for various input file separators; other samll tweaks - scottp@dd.com.au
 
<pre>
#!/usr/bin/perl
use strict;
use warnings;
 
#############################################################################
# Please make changes below to suit your system and requirements
 
# NOTE: hard code host1 & host2 - or pass in as first two parameters to script
my $host1=shift; #host1 is Source
my $host2=shift; #host2 is Dest
 
# NOTE: enable ssl for imapsync
my $imapsync = " -ssl1 -ssl2";
 
##############################
# NOTE: Do not set these here - they come from userlist.txt
my $user;
my $pass;
 
my $data_file="userlist.txt";
open(DAT, $data_file) || die("Could not open file!");
while (<DAT>) {
        chomp;
        # NOTE: userlist.txt can have username@domain password (or comma, tab or |)
        ($user,$pass)=split(/[\|\s\t,]+/,$_);
 
        ##############
        open (TESTOUT, "imapsync --buffersize 18192000 --nosyncacls --subscribe --syncinternaldates --noauthmd5 --host1 $host1 --user1 $user --password1 $pass --host2 $host2 --user2 $user --password2 $pass $imapsync|");
        while (<TESTOUT>){
                print $_;
        }
        close TESTOUT;
 
        ### get contacts
        open (TESTOUT, "wget --user $user --password $pass https://$host1/zimbra/user/$user/contacts.csv --no-check-certificate|");
        while (<TESTOUT>){
                print $_;
        }
        close TESTOUT;
 
        ### get calendar
        open (TESTOUT, "wget --user $user --password $pass https://$host1/zimbra/user/$user/calendar.ics --no-check-certificate |");
        while (<TESTOUT>){
                print $_;
        }
        close TESTOUT;
 
        ### import calendars
        open (TESTOUT, "curl -u $user:$pass --data-binary \@calendar.ics https://$host2/service/home/$user/calendar?fmt=ics --insecure|");
        while (<TESTOUT>){
                print $_;
        }
        close TESTOUT;
 
        ### import contacts
        open (TESTOUT, "curl -u $user:$pass --data-binary \@contacts.csv https://$host2/service/home/$user/contacts?fmt=csv --insecure|");
        while (<TESTOUT>){
                print $_;
        }
        close TESTOUT;
 
        ### remove files
        unlink 'calendar.ics';
        unlink 'contacts.csv';
        ###############
}
</pre>
 
users and passwords go in a text file userlist.txt in the format: username|password.
 
With some versions of wget you may need to specify the username and password as command line arguements (ie --http-passwd=thepass --http-user=user@dom.com) instead of in the URL (ie https://usersname:pass@theurl).  If you are getting the error Bad Port, then give this a try.
 
== Copy Calendar From One Zimbra User to Another ==
In this example we will export a calendar called "LPO calendar" from one user
to another user and name it "LPO" in the destination account.
 
* Export the calendar "LPO calendar" from user SRC_USER:
<nowiki>[root@zimbra ~]# curl -u admin 'https://zimbra:7071/home/SRC_USER/LPO%20calendar?fmt=ics' > LPO.ics</nowiki>
 
* Adjust all the meeting organizer addresses. If you don't do this, the destination user will see the appointments, but can not edit them.
<nowiki>[root@zimbra ~]# cat LPO.ics | sed 's/SRC_USER@/DST_USER@/g' > LPO-dest.ics</nowiki>
 
* Create empty calendar in destination user account.
<nowiki>[root@zimbra ~]# su - zimbra -c 'zmmailbox -z -m DST_USER createFolder --view appointment /LPO'</nowiki>
 
* Import modified ical data into new calendar.
<nowiki>[root@zebra ~]# curl -u "admin" --data-binary @LPO-dest.ics 'https://zimbra:7071/home/DST_USER/LPO?fmt=ics'</nowiki>
 
 
== Copy All Messages, Contacts, Calendars, and Mail Filters ==
 
I have written a script, which can be run in batch, to copy user data between servers or between mailboxes.  This will copy contacts and calendars from every folder no just /Contacts and /Calendar.  I used this script two times to copy pretty much everything important when migrating from 4.5.8 to 5.0.2.  It doesn't support shared folders, task lists, documents, briefcase, and some other things (I recreated shared folders manually with zmmailbox) but support could be added given enough motivation :)
 
[http://sourceforge.net/projects/zcstools/ ZCS Tools] currently contains these Migration scripts written in Python/Bash.  These scripts are packaged as zimbraMigration-0.x and require imapsync to be installed.
 
== Import distribution list ==
I wrote a simply bash script to migrate distribution lists from one Zimbra server
to another it creates a provisioning script for each list.
 
<pre>myPath=$(pwd)
/opt/zimbra/bin/zmprov gadl | while read listname;
do
  echo "/opt/zimbra/bin/zmprov cdl $listname" > $myPath/$listname
  /opt/zimbra/bin/zmprov gdl $listname | grep zimbraMailForwardingAddress >  $myPath/$listname.tmp
  cat $myPath/$listname.tmp | sed 's/zimbraMailForwardingAddress: //g' |
  while read member; do
    echo "/opt/zimbra/bin/zmprov adlm $listname $member" >> $myPath/$listname
  done
  /bin/rm $myPath/$listname.tmp
done</pre>
 
= Migrating Sieve Filter Rules =
 
You have a few options to migrate Sieve rules.  Generally you can take existing rules and then just dump them into the account attribute zimbraMailSieveScript.  The server will be able to process these rules.  However, if you intend to enable the client to edit the rules you will need to make sure they conform to a ZCS-specific set of characteristics.  There are two ways to do this.
 
== Using zmmailbox addFilterRule ==
 
Within the zmmailbox command there is a "addFilterRule" command.  This allows you to add a named filter to a particular user.  Its usage defines the parameters that ZWC can understand for editing.  Given that for any migration scenario you'll likely end up parsing the rules, this will let you reconstruct the rules in a syntax ZWC understands.  You will need to find a "name" for each rule.  It could be as simple as "rule1", "rule2", etc.
 
Here's the usage for addFilterRule
<code>
[zimbra@qa03 ~]$ zmmailbox help filter
 
  addFilterRule(afrl)                    add filter rule
 
  deleteFilterRule(dfrl)                add filter rule
 
  getFilterRules(gfrl)                  get filter rules
 
  modifyFilterRule(mfrl)                add filter rule
 
  {conditions}:
    header "name" is|not_is|contains|not_contains|matches|not_matches "value"
    header "name" exists|not_exists
    date before|not_before|after|not_after "YYYYMMDD"
    size under|not_under|over|not_over "1|1K|1M"
    body contains|not_contains "text"
    addressbook in|not_in "header-name"
    attachment exists|not_exists
 
  {actions}:
    keep
    discard
    fileinto "/path"
    tag "/tag"
    mark read|flagged
    redirect "address"
    stop
 
</code>
Also, for add filter rule:
mbox> afrl
usage:
 
  addFilterRule(afrl)          [opts] {name}  [*active|inactive] [any|*all] {conditions}+ {actions}+
    -a/--after <arg>            add after filter-name
    -f/--first                  add as first filter rule
    -l/--last                    add as last filter rule
    -b/--before <arg>            add before filter-name
 
 
Each zmmailbox script invokes a Java VM (just like zmprov).  So you'll want to create a script to be loaded into zmmailbox.  That script would look something like
<code>
<br>
adminAuthenticate ...<br>
selectMailbox user1 ...<br>
addFilterRule rule1 header "from" contains "kevin" discard<br>
addFilterRule rule2 ....<br>
...<br>
selectMailbox user2<br>
addFilterRule ....<br>
...<br>
 
</code>
Then you can run it via <code>zmmailbox < scriptname</code>.
 
== Processing a Sieve script and inserting it via zmprov ==
 
Alternately you can process the existing Sieve scripts to conform to ZCS expectations and then insert the whole script via <code>zmprov ma account zimbraMailSieveScript ....</code>.  You'll need to restrict the entered rules to the Sieve commands as documented above (if ZWC or ZCO are expected to edit them).  Each filter rule will need a comment above it; this comment is the filter's name as identified in addFilterRule.  Creating one or two simple rules using the web client will give you a framework for the created rule.
 
[[Category:Pending Certification]]
 
 
= Troubleshooting =
 
Please see [http://wiki.zimbra.com/index.php?title=User_Migration_Troubleshooting User Migration Troubleshooting] if you are having problems migrating data to your Zimbra installation.

Latest revision as of 03:11, 11 July 2015

User Migration

   KB 1340        Last updated on 2015-07-11  




0.00
(0 votes)



First, note that you'll want to create accounts in Zimbra first before you do the migration. If you want to do it in one shot, read Bulk Provisioning.

Migrating your users means you're going to have to deal with mail, calendar, contacts and passwords. This page used to be one monolithic monstrosity -- but has now been broken down into separate pages:

Please see User Migration Troubleshooting if you are having problems migrating data to your Zimbra installation.

Verified Against: Zimbra Collaboration Suite 7.0 Date Created: 3/7/2006
Article ID: https://wiki.zimbra.com/index.php?title=User_Migration Date Modified: 2015-07-11



Try Zimbra

Try Zimbra Collaboration with a 60-day free trial.
Get it now »

Want to get involved?

You can contribute in the Community, Wiki, Code, or development of Zimlets.
Find out more. »

Looking for a Video?

Visit our YouTube channel to get the latest webinars, technology news, product overviews, and so much more.
Go to the YouTube channel »


Jump to: navigation, search