Migrating from Dovecot with External LDAP

Migrating from Dovecot with External LDAP

   KB 1438        Last updated on 2015-07-12  




0.00
(0 votes)

Note: This tutorial is specific to Dovecot but is useful wherever the following issues exist:

  • You have a number of existing users, and obtaining the passwords for them is impractical.
  • You are authenticating against an external LDAP server
  • The old mailserver offers the ability to conduct IMAP auth as an administrator for all user accounts


Problem Definition

You're ready to migrate from your old IMAP/POP3 server to your new Zimbra machine. The only problem is, there are a few hundred users on the old server, and tracking them all down to give you their passwords is neither practical nor desirable - after all, one should try to encourage users to not give out their passwords. The Dovecot box does support a "manager" login method for IMAP login, but the Zimbra server does not. And since the Zimbra server is going to authenticate against an external LDAP server, it will be wanting valid passwords for IMAP.

Problem Solution

Doing everything in the right order makes this possible:

  1. Bulk Create the user accounts with identical passwords.
  2. Perform User Migration using a modification of the suggested script.
  3. Connect to the external LDAP server

Details

Step One: Create Users

First, get a dump from the LDAP server in CSV format. Open that CSV file with a spreadsheet program, and edit it so that only the following four columns remain:

  1. UID. This is the part that goes before the @ in the email address: john.smith@yourzimbraserver.example would have a UID of john.smith
  2. Password. Make this identical and reasonably hard to guess - this is going to be a temporary "administrative" password on your zimbra server. Hint: use the drag trick on your spreadsheet program to duplicate your password to all the rows in the user list.
  3. First Name. Self-explanatory.
  4. Last Name. Ditto.

Now save the csv file and open it up with vi^Wyour favorite text editor and use it to strip out all the quotation marks. The correct vim commands to do this (and save and exit vim) are as follows:

  :%s/\"//g
  :wq

Now run your recently-edited csv2mprov.pl script against your csv as indicated in the Bulk Provisioning article, and use the output of this script to provision your user accounts on the Zimbra server.

Step Two: User Migration

For the most part, this section tracks the previously-written User_Migration page - the process is to install imapsync and all its dependencies somewhere (for performance reasons, it should be a powerful machine that is neither the zimbra server nor the original mail server), test execute it against an account, and then run a script to do the full user migration. However, our case includes two significant differences: the usage of an administrative account format for usernames in Dovecot, and the fact that each machine may (or may not) use a different administrative password.

First off, configure Dovecot to enable manager authentication. This requires Dovecot 1.0.b4 or later, which may require backporting or other sorts of dependency battles to obtain. While a lenghthy discussion of the changes in configuration syntax are beyond the scope of this article, do take note of the fact that some changes will need to be done by hand if you are upgrading from 0.9x to >=1.0b4. Once you have a functioning dovecot server again, uncomment or create the following lines in dovecot.conf:

  auth_master_user_separator = *
  passdb passwd-file {
  args = /etc/dovecot/dovecot.master
  master=yes
  }

Next, create /etc/dovecot/dovecot.master by executing the following command:

  htpasswd -c /etc/dovecot/dovecot.master zimbra

This will result in a prompt to enter and verify the password for a user called zimbra on the Dovecot server, and to crypt that password into the indicated file. Once these steps have been taken, reload Dovecot.

Return to your CSV file from before. This time, trim it until only two columns exist:

  1. UID Same as before
    • zimbra This is the second half of the authentication piece being passed to Dovecot to identfy the IMAP authentication as an administrative login.

Save this CSV and again use a plaintext editor to strip out all quote marks, commas, and spaces. The result should be a list that resembles the following:

 john.smith*zimbra
 jane.smith*zimbra
 sugar.ray.leonard*zimbra

and so on. Save this file as, say, users.txt, and put it in an empty directory. In the same directory create a file named host1pass.txt that contains only the password you created for the Dovecot administrative account. Do the same thing for the password you used for all the user accounts on the Zimbra server, and call this one host2pass.txt. Now, you'll want to create a shell script that does the work. The following is based very heavily upon the one from the User_Migration page. Note that I've stripped out the logging features. Feel free to add them back in if you feel the need to have that. Also be sure to edit the hostnames I've given in the script to match your setup. You can use either DNS or IP address for this script.

   #!/bin/bash
   
   host1=dante
   #host1 is Source
   
   host2=alighieri
   #host2 is Dest
   
   cat users.txt | while read
       do
       username1=`echo $REPLY` # $REPLY is a bash builtin
       username2=`echo $REPLY | cut -d\* -f 1` # Strip the star etc
       
       echo "Syncing User $username1 to $username2"
       imapsync --nosyncacls --syncinternaldates \
               --host1 $host1 --user1 "$username1" --passfile1 host1pass.txt \
               --host2 $host2 --user2 "$username2" --passfile2 host2pass.txt
       done

Save this script as migrate-imap.sh in the same directory as users.txt, host1pass.txt, and host2pass.txt. Tar it all up and copy it over to the machine that you have installed imapsync on.

Now shut down the MTA on your old server - postfix, exim, sendmail, whatever. Just make sure it's not accepting any new inbound email for a while - the sending servers will defer the messages it has for your server for at least 24 hours before returning failure messages to the sending parties, so as long as this goes well, your users won't lose any mail.

Test the synchronization by running the commands from the script by hand against one user - you can probably use your own account if it is on the server being migrated, as you'll know whether the messages came across successfully.

Now run the migrate-imap.sh script - I strongly recommend running this under a screen session, as it can and will take several hours to run, and the last thing you want is a network hiccup to kill the execution of your script midway through. As you run this script, you may notice a number of error messages. My experience was that all the mails synchronized anyhow, but your mileage may vary. This is why you want to test the migration on one user first. At any rate, be patient, but also watch for a hung user account. This will often happen right as another user loops into the script - you'll see a number of plus signs and discussions about the sizes of the various INBOX folders, but then nothing at all. My best guess is that this occurs when Dovecot's index files get out of whack, but I'm not sure. At any rate, ctrl-c out of the script, and delete all the users up to the one where the script filed, then go to the old server and delete using rm -rf dovecot* from within that user's Maildir and restart Dovecot. Run the script again, and it should go well.

Unfortunately, imapsync is high-maintainance, so if you're on a strict timeline, plan to be stuck in front of the keyboard for several hours. You can shorten this time considerably by encouraging all your users to empty their IMAP trashes in the days leading up to the migration, but do expect this part to be an all-day operation. Be patient, and eventually it will complete.

Step Three: Connect Zimbra to the External LDAP Server

In the Zimbra admin console, navigate to Configuration/Domains/yourdomain.example. On the main pane at the top, click "Configure Authentication." A wizard will pop up. Select External LDAP from the Authentication mechanism drop-down menu, and click Next. Give the address of your LDAP server, and fill in the filter and search base fields. I suggest uid=%u for the LDAP filter, and whatever is correct for your domain with syntax like ou=People, dc=mydomain, dc=example for the search base. You may or may not need to use a password and bind DN to connect to the LDAP server. Ours doesn't require this. Next, test the authentication by first passing your username but rubbish for the password, then upon an error, click the back button and use the correct password. If the test is successful, you can click the finish button.

That's it - your users will now authenticate to Zimbra using their regular credentials from the LDAP server, and the mock-admin password you created back in Step One will be ignored (and will not work at all so long as LDAP is the auth source). The user mailboxes will be as the users left them.

The next steps would be to edit DNS, firewall, etc. but that's material for a different tutorial.

Verified Against: unknown Date Created: 4/23/2006
Article ID: https://wiki.zimbra.com/index.php?title=Migrating_from_Dovecot_with_External_LDAP Date Modified: 2015-07-12



Try Zimbra

Try Zimbra Collaboration with a 60-day free trial.
Get it now »

Want to get involved?

You can contribute in the Community, Wiki, Code, or development of Zimlets.
Find out more. »

Looking for a Video?

Visit our YouTube channel to get the latest webinars, technology news, product overviews, and so much more.
Go to the YouTube channel »

Jump to: navigation, search