Difference between revisions of "Bulk Provisioning"

(Zmprov Command Files)
 
(10 intermediate revisions by 6 users not shown)
Line 1: Line 1:
[[Category:Migration]]
+
{{BC|Community Sandbox}}
 +
__FORCETOC__
 +
<div class="col-md-12 ibox-content">
 +
=Bulk Provisioning=
 +
{{KB|{{Unsupported}}|{{ZCS 8.0}}|{{ZCS 7.0}}|}}
 +
{{WIP}}[[Category:Migration]]
  
= Zmprov Command Files =
+
= zmprov Command Files =
  
The [[Zmprov]] command will accept commands from a file (or stdin) as input. Create a text file ("commands.zmp" for example) with the zmprov subcommands each on a line.
+
The [[zmprov]] command will accept commands from a file (or stdin) as input. Create a text file ("commands.zmp" for example) with the zmprov subcommands each on a line.
  
 
  createDomain domain.com
 
  createDomain domain.com
Line 11: Line 16:
 
Then send the contents of the file to zmprov.
 
Then send the contents of the file to zmprov.
  
As Zimbra:
+
As zimbra:
   zmprov < commands.zmp
+
   zmprov -f commands.zmp
  
 
As Root:
 
As Root:
 
   cat commands.zmp | su - zimbra -c zmprov
 
   cat commands.zmp | su - zimbra -c zmprov
  
Note: You'll probably want to avoid doing a bunch of direct zmprov commands in a bash script if you have a lot to add, since each zmprov command starts up a new jvm for each one. In other words you'll want to avoid doing thousands of these (unless you have plenty of time):
+
=Useful examples of bulk provisioning with zmprov=
 +
 
 +
You'll probably want to avoid doing a bunch of direct zmprov commands in a bash script if you have a lot to add, since each zmprov command starts up a new jvm for each one. In other words you'll want to avoid doing thousands of these (unless you have plenty of time):
  
 
<pre>
 
<pre>
Line 27: Line 34:
 
</pre>
 
</pre>
  
Below are some ways to create a command file from your existing account data.
+
As shown in the first section, you can create a list of subcommands to send to zmprov all at once, which will save time because it's only necessary to start a single jvm. Below are some ways to create a command file from your existing account data.
  
 
= Create Accounts =
 
= Create Accounts =
Line 61: Line 68:
  
 
   Example provisioning ZCS accounts as 'zimbra' user:
 
   Example provisioning ZCS accounts as 'zimbra' user:
     zimbra$ zmprov < commands.zmp
+
     zimbra$ zmprov -f commands.zmp
  
 
=head1 DESCRIPTION:
 
=head1 DESCRIPTION:
Line 86: Line 93:
 
   or pod2usage( -verbose => 0 );
 
   or pod2usage( -verbose => 0 );
  
pod2usage( -verbose => 1 ) if ( $option{help} );
+
pod2usage( -verbose => 1 ) if $option{help};
pod2usage( -verbose => 2 ) if ( $option{man} );
+
pod2usage( -verbose => 2 ) if $option{man};
  
 
push( @err, "-domain <domain> is required" )
 
push( @err, "-domain <domain> is required" )
   unless ( $option{domain} );
+
   unless $option{domain};
  
 
pod2usage( -verbose => 0, -message => map( "$prog: $_\n", @err ) )
 
pod2usage( -verbose => 0, -message => map( "$prog: $_\n", @err ) )
   if (@err);
+
   if @err;
  
warn("$prog: using Default COS\n") unless ( $option{cos_id} );
+
warn("$prog: using Default COS\n") unless $option{cos_id};
warn("$prog: reading passwd like entries from STDIN\n") unless (@ARGV);
+
warn("$prog: reading passwd like entries from STDIN\n") unless @ARGV;
  
my $date    = localtime();
+
my $date    = localtime;
 
my $cosid    = $option{cosid};
 
my $cosid    = $option{cosid};
 
my $domain  = $option{domain};
 
my $domain  = $option{domain};
Line 110: Line 117:
 
while (<>) {
 
while (<>) {
 
     chomp;
 
     chomp;
     next if (/^\s*$/);    # skip empty lines
+
     next if /^\s*$/;    # skip empty lines
  
 
     my ( $uname, $x, $uid, $gid, $gecos, $dir, $shell ) = split( /:/, $_, 7 );
 
     my ( $uname, $x, $uid, $gid, $gecos, $dir, $shell ) = split( /:/, $_, 7 );
Line 158: Line 165:
 
   perl passwd2zmprov -domain example.moc < passwd > mydata.zmp
 
   perl passwd2zmprov -domain example.moc < passwd > mydata.zmp
  
== CSV File to Zmprov ==
+
== CSV File to zmprov ==
  
 
Following is a simple perl script to take a CSV file and turn it into the correct zmprov commands
 
Following is a simple perl script to take a CSV file and turn it into the correct zmprov commands
  
#!/usr/bin/perl
+
<pre>
 +
#!/usr/bin/perl
 +
 +
# Lookup the valid COS (Class of Service) ID in the interface or like this
 +
my $cosid = `su - zimbra -c 'zmprov gc Default |grep zimbraId:'`;
 +
$cosid =~ s/zimbraId:\s*|\s*$//g;
 
    
 
    
# Lookup the valid COS (Class of Service) ID in the interface or like this
+
while (<>) {
my $cosid = `su - zimbra -c 'zmprov gc Default |grep zimbraId:'`;
+
      chomp;
$cosid =~ s/zimbraId:\s*|\s*$//g;
+
 
+
      # CHANGE ME: To the actual fields you use in your CSV file
while (<>) {
+
      my ($email, $password, $first, $last) = split(/\,/, $_, 4);
        chomp;
+
       
 
+
      my ($uid, $domain) = split(/@/, $email, 2);
        # CHANGE ME: To the actual fields you use in your CSV file
+
        my ($email, $password, $first, $last) = split(/\,/, $_, 4);
+
      print qq{ca $uid\@$domain $password\n};
         
+
      print qq{ma $uid\@$domain zimbraCOSid "$cosid"\n};
        my ($uid, $domain) = split(/@/, $email, 2);
+
      print qq{ma $uid\@$domain givenName "$first"\n};
 
+
      print qq{ma $uid\@$domain sn "$last"\n};
        print qq{ca $uid\@$domain $password\n};
+
      print qq{ma $uid\@$domain cn "$uid"\n};
        print qq{ma $uid\@$domain zimbraCOSid "$cosid"\n};
+
      print qq{ma $uid\@$domain displayName "$first $last"\n};
        print qq{ma $uid\@$domain givenName "$first"\n};
+
      print qq{ma $uid\@$domain zimbraPasswordMustChange TRUE\n};
        print qq{ma $uid\@$domain sn "$last"\n};
+
      print qq{\n};
        print qq{ma $uid\@$domain cn "$uid"\n};
+
}
        print qq{ma $uid\@$domain displayName "$first $last"\n};
+
</pre>
        print qq{ma $uid\@$domain zimbraPasswordMustChange TRUE\n};
 
        print qq{\n};
 
}
 
  
 
The above is only a starting place, you will need to change other options (eg: the zimbraPasswordMustChange is an example only) and of course how you create and split the input data.
 
The above is only a starting place, you will need to change other options (eg: the zimbraPasswordMustChange is an example only) and of course how you create and split the input data.
  
Also, some CSV files may contain quotes you want to remove.
+
Also, some CSV files might contain quotes you want to remove.
  
 
== MacOSX Server mail users ==
 
== MacOSX Server mail users ==
Line 214: Line 223:
 
use Tie::File;
 
use Tie::File;
  
system `clear`;
+
system "clear";
  
 
print "This script is now going to connect to your existing mail server\n";
 
print "This script is now going to connect to your existing mail server\n";
Line 220: Line 229:
 
print "If this step fails your probably have not configure the vars in this script\n";
 
print "If this step fails your probably have not configure the vars in this script\n";
  
system `ssh $username1\@$hostname1 sudo ls /var/spool/imap/user > userlist.txt`;
+
system "ssh $username1\@$hostname1 sudo ls /var/spool/imap/user > userlist.txt";
system `cp userlist.txt imapuserlist.txt`;
+
system "cp userlist.txt imapuserlist.txt";
  
 
print "users retrieved:\n";
 
print "users retrieved:\n";
Line 233: Line 242:
 
$answer = <>;
 
$answer = <>;
  
if ($answer =~ m/Y/ )
+
if ($answer =~ /Y/ )
 
{
 
{
 
modtextfile();
 
modtextfile();
 
}
 
}
 
 
elsif ( $answer =~ m/N/ )  
+
elsif ( $answer =~ /N/ )  
 
{
 
{
 
die;
 
die;
Line 276: Line 285:
  
 
<pre>
 
<pre>
 
 
#!/usr/bin/perl
 
#!/usr/bin/perl
  
Line 282: Line 290:
 
=head1 NAME
 
=head1 NAME
  
ldap2zm - create zimbra accounts for LDAP users
+
ldap2zm  - create zimbra accounts for LDAP users
  
 
=head1 SYNOPSIS
 
=head1 SYNOPSIS
Line 298: Line 306:
 
   -f    LDAP search filter (default: '(objectclass=*)' )
 
   -f    LDAP search filter (default: '(objectclass=*)' )
 
   -l    list users found in LDAP search; take no other action
 
   -l    list users found in LDAP search; take no other action
 
+
  -n    new domain (default to first found domain)
 
=head1 DESCRIPTION
 
=head1 DESCRIPTION
  
Line 315: Line 323:
 
  ldap2zm -h adserver -u administrator -p s3cr3t -b "DC=example,DC=com" \  
 
  ldap2zm -h adserver -u administrator -p s3cr3t -b "DC=example,DC=com" \  
 
     -f '(memberOf=CN=Zimbra Users,OU=Employees,DC=example,DC=com)'
 
     -f '(memberOf=CN=Zimbra Users,OU=Employees,DC=example,DC=com)'
 +
 +
This one will also generate email as $user@example.com if email address is missing but it will extract all
 +
regular user from AD (sAMAccountType may differ for you)
 +
ldap2zm -h adserver -u administrator -p s3cr3t -b "DC=example,DC=com" \
 +
    -f "(sAMAccountType=805306368)" -n example.com
  
 
=head1 AUTHOR
 
=head1 AUTHOR
  
 
   Greg Boyington <greg@automagick.us>
 
   Greg Boyington <greg@automagick.us>
 +
  adapted to Centos/AD by dominix <dominix at mail.pf>
  
 
=head1 SEE ALSO
 
=head1 SEE ALSO
Line 328: Line 342:
 
use strict;
 
use strict;
 
use Data::Dumper;
 
use Data::Dumper;
use String::MkPasswd qw(mkpasswd);
+
use Text::Password::Pronounceable;
 
use IPC::Open3;
 
use IPC::Open3;
 
use IO::Select;
 
use IO::Select;
Line 350: Line 364:
 
# ensure the 'zimbra' user is running the show
 
# ensure the 'zimbra' user is running the show
 
unless ( ( getpwuid( $< ) )[0] eq 'zimbra' ) {
 
unless ( ( getpwuid( $< ) )[0] eq 'zimbra' ) {
print "You must run this script as the 'zimbra' user.\n";
+
        print "You must run this script as the 'zimbra' user.\n";
exit 1;
+
        exit 1;
 
}
 
}
  
 
# process command-line switches
 
# process command-line switches
getopts('vh:u:p:b:f:l',\%opt);
+
getopts('vh:u:p:b:f:ln:',\%opt);
$VERBOSE = $opt{'v'} ? 1 : 0;
+
$VERBOSE = $opt{v} ? 1 : 0;
 
die "You must specify your LDAP host with -h.\n"
 
die "You must specify your LDAP host with -h.\n"
unless $opt{'h'};
+
        unless $opt{h};
 
die "You must specify your LDAP base with -b.\n"
 
die "You must specify your LDAP base with -b.\n"
unless $opt{'b'};
+
        unless $opt{b};
 +
 
 +
unless ( $opt{f} ) {
 +
        $opt{f} = q/(objectclass='*')/;
 +
        warn qq/Warning: using default LDAP filter "$opt{f}"; override with -f.\n/;
 +
}
  
unless ( $opt{'f'} ) {
+
unless ( $opt{n} ) {
$opt{'f'} = q/(objectclass='*')/;
+
    $opt{n}= `$zmprov_cmd gad|head -1`;
warn qq/Warning: using default LDAP filter "$opt{'f'}"; override with -f.\n/;
+
    chomp($opt{n});
 
}
 
}
 +
 +
unless ($opt{v}) { print "using default domain = $opt{n}\n";}
  
 
# create the ldap object and bind to the LDAP server
 
# create the ldap object and bind to the LDAP server
my $ldap = Net::LDAP->new( $opt{'h'} ) or die $@;
+
my $ldap = Net::LDAP->new( $opt{h} ) or die $@;
my $msg = $ldap->bind( $opt{'u'} ? ( $opt{'u'}, password => $opt{'p'} ) : () );
+
my $msg = $ldap->bind( $opt{u} ? ( $opt{u}, password => $opt{p} ) : () );
$msg->code && die $msg->error;
+
$msg->code and die $msg->error;
  
 
# get a list of all users in the AD
 
# get a list of all users in the AD
 
print "Loading LDAP users...";
 
print "Loading LDAP users...";
my %users = &search( $ldap, base => $opt{'b'}, filter => $opt{'f'} );
+
my %users = search( $ldap, base => $opt{b}, filter => $opt{f} );
 
print "OK.\n";
 
print "OK.\n";
  
 
# no users? no work.
 
# no users? no work.
if ( ! scalar keys %users ) {
+
if ( ! %users ) {
warn "No LDAP users found; aborting.\n";
+
        warn "No LDAP users found; aborting.\n";
exit 1;
+
        exit 1;
 
}
 
}
  
Line 388: Line 409:
 
my @lines;
 
my @lines;
 
foreach ( sort keys %users ) {
 
foreach ( sort keys %users ) {
$users{ $_ } = &gen_zmprov_command( user => $users{ $_ } );
+
        $users{ $_ } = gen_zmprov_command( user => $users{ $_ } );
push @lines, "da '$users{ $_ }->{'_address'}'\n";
+
        push @lines, "da '$users{ $_ }->{_address}'\n";
 
}
 
}
  
Line 395: Line 416:
 
# after the passing the results through gen_zmprov_command so we have the _address  
 
# after the passing the results through gen_zmprov_command so we have the _address  
 
# to display.
 
# to display.
if ( $opt{'l'} || $VERBOSE ) {
+
if ( $opt{l} || $VERBOSE ) {
print "The following users will be (re)created in zimbra:\n";
+
        print "The following users will be (re)created in zimbra:\n";
printf '%-40s %s  %s'."\n", $users{ $_ }->{'cn'}[0], $users{ $_ }->{'_address'}, $users{ $_ }->{'_password'}
+
        printf '%-40s %s  %s'."\n", $users{ $_ }->{cn}[0], $users{ $_ }->{_address}, $users{ $_ }->{_password}
foreach sort keys %users;
+
                foreach sort keys %users;
 
}
 
}
exit if $opt{'l'};
+
exit if $opt{l};
  
 
# Now we modify zimbra...
 
# Now we modify zimbra...
Line 406: Line 427:
 
# deprovision existing accounts
 
# deprovision existing accounts
 
print "Deleting existing accounts...";
 
print "Deleting existing accounts...";
&zmprov(@lines);
+
zmprov(@lines);
 
print "OK.\n";
 
print "OK.\n";
  
Line 419: Line 440:
 
# create new accounts
 
# create new accounts
 
print "Creating new accounts...";
 
print "Creating new accounts...";
@lines = map { $users{ $_ }->{'_cmd'} } sort keys %users;
+
@lines = map { $users{ $_ }->{_cmd} } sort keys %users;
&zmprov(@lines);
+
zmprov(@lines);
 
print "OK.\nRecreated " . scalar( keys %users ) . " accounts.\n";
 
print "OK.\nRecreated " . scalar( keys %users ) . " accounts.\n";
  
Line 439: Line 460:
 
#  
 
#  
 
sub search {
 
sub search {
my $ldap = shift;
+
        my $ldap = shift;
my %args = @_;
+
        my %args = @_;
  
my $index_attr = delete $args{'index_attr'} || 'userprincipalname';
+
        my $index_attr = delete $args{index_attr} || 'samaccountname';
  
# establish some defaults
+
        # establish some defaults
$args{'attrs'} ||= [ 'cn', 'userPrincipalName', 'memberOf', 'givenName', 'sn' ];
+
        $args{attrs}   ||= [ 'cn', 'userPrincipalName', 'memberOf', 'givenName', 'sn', 'sAMAccountName' ];
$args{'scope'} ||= 'sub';
+
        $args{scope}   ||= 'sub';
  
# do the search
+
        # do the search
my $result = $ldap->search( %args );
+
        my $result = $ldap->search( %args );
  
my %users;
+
        my %users;
  
# rejigger the results into a useful format
+
        # rejigger the results into a useful format
my $href = $result->as_struct;
+
        my $href = $result->as_struct;
foreach ( keys %$href ) {
+
        foreach my $valref ( values %$href ) {
my $valref = $$href{$_};
+
                my $this;
my $this;
+
                foreach my $attr ( sort keys %$valref ) {
foreach my $attr ( sort keys %$valref ) {
+
                        next if $attr =~ /;binary$/; # ignore any binary data
next if $attr =~ /;binary$/; # ignore any binary data
+
                        $this->{ lc $attr } = $valref->{ $attr };
$this->{ lc $attr } = @$valref{ $attr };
+
                }
}
 
  
# add this user to the users hash
+
                # add this user to the users hash
$users{ $this->{ lc $index_attr }[0] } = $this;
+
                $users{ $this->{ lc $index_attr }[0] } = $this;
}
+
        }
+
 
return %users;
+
        return %users;
 
}
 
}
  
Line 478: Line 498:
 
#
 
#
 
sub gen_zmprov_command {
 
sub gen_zmprov_command {
my %args = @_;
+
        my %args = @_;
my $user = delete $args{'user'};
+
        my $user = delete $args{user};
  
# we authenticate against AD, so the local zimbra password is irrelevant;  
+
        # we authenticate against AD, so the local zimbra password is irrelevant;  
# we'll generate difficult passwords just to be on the safe side.
+
        # we'll generate difficult passwords just to be on the safe side.
$user->{'_password'} = mkpasswd( -length => 12 );
+
        #$user->{_password} = mkpasswd( -length => 12 );
 +
        $user->{_password} = Text::Password::Pronounceable->generate(6, 10);
  
# our AD server manages internal.example.com, but we want email addresses
+
        # our AD server manages internal.example.com, but we want email addresses
# to be in the example.com domain, so we fix up the address here.  Your  
+
        # to be in the example.com domain, so we fix up the address here.  Your  
# setup (and AD schema) may vary.
+
        # setup (and AD schema) may vary.
my $address = $user->{'userprincipalname'}[0];
+
        my $address = $user->{userprincipalname}[0] ? $user->{userprincipalname}[0] : "$user->{cn}[0]\@$opt{n}" ;
$address =~ s/internal\.//;
+
        $address =~ s/internal\.//;
+
 
$user->{'_address'} = $address;
+
        $user->{_address} = $address;
  
# build the createAccount command to be sent to zmprov.
+
        # build the createAccount command to be sent to zmprov.
$user->{'_cmd'} = qq(createAccount '$address' ) .  
+
        $user->{_cmd} = qq(createAccount '$address' ) .  
  qq('$user->{'_password'}' ) .  
+
                                          qq('$user->{_password}' ) .  
  qq(displayName '$user->{'cn'}[0]' ) .  
+
                                          qq(displayName '$user->{cn}[0]' ) .  
  qq(givenName '$user->{'givenname'}[0]' ) .  
+
                                          qq(givenName '$user->{givenname}[0]' ) .  
  qq(sn '$user->{'sn'}[0]'\n);
+
                                          qq(sn '$user->{sn}[0]'\n);
  
# return the modified hashref
+
        # return the modified hashref
return $user;
+
        return $user;
 
}
 
}
  
Line 510: Line 531:
 
sub zmprov {
 
sub zmprov {
  
# start the zmprov process and capture its filehandles
+
        # start the zmprov process and capture its filehandles
my $pid = open3(\*WRITE, \*READ, \*ERROR, $zmprov_cmd)
+
        my $pid = open3(\*WRITE, \*READ, \*ERROR, $zmprov_cmd)
or die "Couldn't open pipe to $zmprov_cmd: $!";
+
                or die "Couldn't open pipe to $zmprov_cmd: $!";
  
# use IO::Select to poll zmprov's STDERR
+
        # use IO::Select to poll zmprov's STDERR
my $sel = new IO::Select();
+
        my $sel = IO::Select->new;
$sel->add(\*READ);
+
        $sel->add(\*READ);
$sel->add(\*ERROR);
+
        $sel->add(\*ERROR);
  
# send every command we've been given to zmprov and watch for errors.
+
        # send every command we've been given to zmprov and watch for errors.
foreach ( @_ ) {
+
        foreach ( @_ ) {
  
# report what we're doing as we do it, if we're being verbose
+
                # report what we're doing as we do it, if we're being verbose
print $_ if $VERBOSE;
+
                print $_ if $VERBOSE;
  
# send the command to zmprov
+
                # send the command to zmprov
print WRITE $_;
+
                print WRITE $_;
  
# watch for a response on STDERR
+
                # watch for a response on STDERR
foreach my $h ( $sel->can_read ) {
+
                foreach my $h ( $sel->can_read ) {
my $buf='';
+
                        my $buf='';
if ( $h eq \*ERROR ) {
+
                        if ( $h eq \*ERROR ) {
  
# XXX: should we abort if we get an error?
+
                                # XXX: should we abort if we get an error?
sysread(ERROR,$buf,4096);
+
                                sysread(ERROR,$buf,4096);
warn "ERROR ( $_ ): $buf\n" if $buf;
+
                                warn "ERROR ( $_ ): $buf\n" if $buf;
}
+
                        }
}
+
                }
}
+
        }
  
# no zombies
+
        # no zombies
waitpid($pid,1);
+
        waitpid($pid,1);
 
}
 
}
  
Line 612: Line 633:
 
</pre>
 
</pre>
  
{{Article_Footer|unknown|4/3/2006}}
+
{{Article_Footer|Zimbra Collaboration 8.0, 7.0|4/3/2006}}
 
 
[[Category:OSX]]
 

Latest revision as of 14:58, 11 July 2015

Bulk Provisioning

   KB 1382        Last updated on 2015-07-11  




0.00
(0 votes)

zmprov Command Files

The zmprov command will accept commands from a file (or stdin) as input. Create a text file ("commands.zmp" for example) with the zmprov subcommands each on a line.

createDomain domain.com
createAccount andy@domain.com password displayName 'Andy Anderson' givenName Andy sn Anderson
createAccount betty@domain.com password displayName 'Betty Brown' givenName Betty sn Brown

Then send the contents of the file to zmprov.

As zimbra:

 zmprov -f commands.zmp

As Root:

 cat commands.zmp | su - zimbra -c zmprov

Useful examples of bulk provisioning with zmprov

You'll probably want to avoid doing a bunch of direct zmprov commands in a bash script if you have a lot to add, since each zmprov command starts up a new jvm for each one. In other words you'll want to avoid doing thousands of these (unless you have plenty of time):

zmprov createDomain domain.com
zmprov  createAccount andy@domain.com password displayName 'Andy Anderson' givenName Andy sn Anderson
zmprov  createAccount betty@domain.com password displayName 'Betty Brown' givenName Betty sn Brown
.. .. ..
.. .. ..

As shown in the first section, you can create a list of subcommands to send to zmprov all at once, which will save time because it's only necessary to start a single jvm. Below are some ways to create a command file from your existing account data.

Create Accounts

Passwd File to Zmprov

Following is a perl script to take a passwd file and turn it into a zmprov command file for account provisioning.

#!/usr/bin/perl
#
# $Id: passwd2zmprov,v 1.2 2008/03/05 05:01:29 phil Exp $

=head1 NAME

passwd2zmprov - create zmprov commands from a passwd file

=head1 SYNOPSIS

  usage: passwd2zmprov [options] [[passwd_file] ...] > commands.zmp
     -help   show a brief help message
     -man    show the full documentation

     -domain <domain>     [REQUIRED]
     -cosid <cos_id>      [default "Default COS"]
     -password <password> [default ""]

  Getting a COS id:
    zimbra$ zmprov gc <myCos> | grep ^zimbraId:

  Example converting CSV to zmprov commands:
    $ ./passwd2zmprov -domain example.moc /etc/passwd > commands.zmp

  Example provisioning ZCS accounts as 'zimbra' user:
    zimbra$ zmprov -f commands.zmp

=head1 DESCRIPTION:

Tool to create commands suitable for zmprov from a UNIX passwd file.
We don't use getpwent etc., because we are likely working on a copy
and not running as root.

See Also:
  http://wiki.zimbra.com/index.php?title=Bulk_Create

=cut

use strict;
use warnings;
use File::Basename qw(basename);
use Getopt::Long qw(GetOptions);
use Pod::Usage qw(pod2usage);

my $prog = basename($0);
my ( @err, %option );

GetOptions( \%option, 'help|?', 'man', 'domain=s', 'cosid=s', 'password=s' )
  or pod2usage( -verbose => 0 );

pod2usage( -verbose => 1 ) if $option{help};
pod2usage( -verbose => 2 ) if $option{man};

push( @err, "-domain <domain> is required" )
  unless $option{domain};

pod2usage( -verbose => 0, -message => map( "$prog: $_\n", @err ) )
  if @err;

warn("$prog: using Default COS\n") unless $option{cos_id};
warn("$prog: reading passwd like entries from STDIN\n") unless @ARGV;

my $date     = localtime;
my $cosid    = $option{cosid};
my $domain   = $option{domain};
my $password = defined $option{password} ? $option{password} : "";
my $MIN_UID = 500;      # skip system accounts like httpd
my $MAX_UID = 60000;    # skip other system accounts like nfsnobody

# sanitize password
$password =~ s/\"/\\\"/g;

while (<>) {
    chomp;
    next if /^\s*$/;    # skip empty lines

    my ( $uname, $x, $uid, $gid, $gecos, $dir, $shell ) = split( /:/, $_, 7 );

    if ( $uid < $MIN_UID or $uid > $MAX_UID ) {
        warn("$prog: skip $uname: $uid not between $MIN_UID and $MAX_UID\n");
        next;
    }

    # assuming gecos format is First [[MI] [Last]], sanitize a little
    $gecos =~ s/\"/\\\"/g;

    my ( $fullname, $description ) = split( /\s*,\s*/, $gecos, 2 );
    my ( $fname, $mname, $lname ) = split( " ", $fullname, 3 );
    unless ( defined($lname) ) {
        $lname = $mname;
        undef($mname);
    }
    my $displayname = $fname
      . ( defined($mname) ? " $mname" : "" )
      . ( defined($lname) ? " $lname" : "" );

    print(
        qq{ca "$uname\@$domain" "$password"},
        ( defined($cosid)       ? qq{ zimbraCOSid "$cosid"}       : () ),
        ( defined($fname)       ? qq{ givenName "$fname"}         : () ),
        ( defined($lname)       ? qq{ sn "$lname"}                : () ),
        ( defined($uname)       ? qq{ cn "$uname"}                : () ),
        ( defined($displayname) ? qq{ displayName "$displayname"} : () ),
        ( defined($description) ? qq{ description "$description"} : () ),
        qq{ zimbraNotes "Migrated $date"},
        qq{ zimbraPasswordMustChange TRUE},
        qq{\n},
    );
}

=head1 HISTORY

 2007/01/23, Version 1.0/1.1 Dlbewley
 2008/03/04, Version 1.2 Plobbes

=cut

To execute:

 perl passwd2zmprov -domain example.moc < passwd > mydata.zmp

CSV File to zmprov

Following is a simple perl script to take a CSV file and turn it into the correct zmprov commands

#!/usr/bin/perl
 
# Lookup the valid COS (Class of Service) ID in the interface or like this
my $cosid = `su - zimbra -c 'zmprov gc Default |grep zimbraId:'`;
$cosid =~ s/zimbraId:\s*|\s*$//g;
  
while (<>) {
       chomp;
 
       # CHANGE ME: To the actual fields you use in your CSV file
       my ($email, $password, $first, $last) = split(/\,/, $_, 4);
         
       my ($uid, $domain) = split(/@/, $email, 2);
 
       print qq{ca $uid\@$domain $password\n};
       print qq{ma $uid\@$domain zimbraCOSid "$cosid"\n};
       print qq{ma $uid\@$domain givenName "$first"\n};
       print qq{ma $uid\@$domain sn "$last"\n};
       print qq{ma $uid\@$domain cn "$uid"\n};
       print qq{ma $uid\@$domain displayName "$first $last"\n};
       print qq{ma $uid\@$domain zimbraPasswordMustChange TRUE\n};
       print qq{\n};
}

The above is only a starting place, you will need to change other options (eg: the zimbraPasswordMustChange is an example only) and of course how you create and split the input data.

Also, some CSV files might contain quotes you want to remove.

MacOSX Server mail users

This script will remotely login to your existing MacOSX mail server and retrieve the users from /var/spool/imap/user. You will need an sudo/ssh account on boxes, don't forget to configure the variables at the top of the script

#!/usr/bin/perl
#This file is going to provision all the users for your new Zimbra Mail Server from MacOSX
#Written by Jordan Eunson - jordan@spidernetworks.ca

#Enter the top level domain for your email
$domain = "foo.bar.com";
#Enter the a sudo/SSH username for your existing mail server
$username1 = "username";
#Enter the IP/FQDN for your existing mail server
$hostname1 = "macosx.foo.bar.com";
#Enter the a sudo/SSH username for your new Zimbra server
$username2 = "username";
#Enter the IP/FQDN for your Zimbra server
$hostname2 = "zimbra.foo.bar.com";

#DO NOT MODIFY BELOW THIS LINE
use warnings;
use Tie::File;

system "clear";

print "This script is now going to connect to your existing mail server\n";
print "It will ask you for your sudo password twice\n";
print "If this step fails your probably have not configure the vars in this script\n";

system "ssh $username1\@$hostname1 sudo ls /var/spool/imap/user > userlist.txt";
system "cp userlist.txt imapuserlist.txt";

print "users retrieved:\n";
open FILE, "userlist.txt" or die $!;
while (<FILE>) { print $_; }
close (FILE); 

confirmation:

print "Are these all your users?[Y/N]";
$answer = <>;

if ($answer =~ /Y/ )
	{
		modtextfile();
	}
	
elsif ( $answer =~ /N/ ) 
	{
		die;
	}
else 
	{ 
		print "you must answer Yes or No!\n";
        goto confirmation; 
    }
    
print "\n";
print "We are now ready to provision the users into your new Zimbra Server\n";
print "You will now be prompted thrice for your sudo password for your new Zimbra Server\n";
provusers();

print "provisioning complete\n";
exit;


sub modtextfile() {
	tie my @file, "Tie::File", "userlist.txt";
	foreach my $line (@file)
	{ 
	$line = "ca " . $line . "@" . $domain . ' ""';
	}
}

sub provusers() {
	system `ssh $username2\@$hostname2 sudo cd; ssh $username2\@$hostname2 sudo /opt/zimbra/bin/zmprov < userlist.txt > useroutput.txt`;
	}

LDAP Users to Zimbra Accounts

This script will generate zimbra accounts based on users found in your LDAP server. WARNING: EXISTING ACCOUNTS WILL BE DELETED FROM ZIMBRA, so use with care!

#!/usr/bin/perl

=pod
=head1 NAME

ldap2zm  - create zimbra accounts for LDAP users

=head1 SYNOPSIS

usage: ldap2zm -h host -b 'base' [ options ]

Switches:
  -h    LDAP hostname
  -b    LDAP search base

Options:
  -v    enable verbose output
  -u    username for LDAP bind
  -p    password for LDAP bind
  -f    LDAP search filter (default: '(objectclass=*)' )
  -l    list users found in LDAP search; take no other action
  -n    new domain (default to first found domain)
=head1 DESCRIPTION

*** WARNING! THIS SCRIPT WILL DESTROY EXISTING MAILBOXES!

ldap2zm will create accounts in zimbra for every user it finds in the specified LDAP server.
Existing accounts are purged from zimbra at the start of each run, so DO NOT RUN THIS ON A 
PRODUCTION ZIMBRA SERVER!  Don't say I didn't warn you. :)

=head1 EXAMPLE

The following command would create zimbra accounts for all employees of example.com who are members of the 
'Zimbra Users' group on the hypothetical Active Directory server 'adserver', using the administrator's 
credentials:

 ldap2zm -h adserver -u administrator -p s3cr3t -b "DC=example,DC=com" \ 
     -f '(memberOf=CN=Zimbra Users,OU=Employees,DC=example,DC=com)'

This one will also generate email as $user@example.com if email address is missing but it will extract all
regular user from AD (sAMAccountType may differ for you)
 ldap2zm -h adserver -u administrator -p s3cr3t -b "DC=example,DC=com" \ 
     -f "(sAMAccountType=805306368)" -n example.com

=head1 AUTHOR

  Greg Boyington <greg@automagick.us>
  adapted to Centos/AD by dominix <dominix at mail.pf>

=head1 SEE ALSO

  http://wiki.zimbra.com/index.php?title=Bulk_Provisioning

=cut

use strict;
use Data::Dumper;
use Text::Password::Pronounceable;
use IPC::Open3;
use IO::Select;
use Net::LDAP;
use Getopt::Std;

$|=1;

use vars qw/$VERSION $VERBOSE %opt $zmprov_cmd $zmcontrol_cmd/;

# set up usage info
$VERSION=0.5;
sub main::VERSION_MESSAGE { print $0.', version '.$main::VERSION."\n" }
sub main::HELP_MESSAGE    { print "For usage details please run:\n\tperldoc -F $0\n" }
$Getopt::Std::STANDARD_HELP_VERSION = 1;

# zimbra executables
$zmprov_cmd = '~zimbra/bin/zmprov';
$zmcontrol_cmd = '~zimbra/bin/zmmailboxdctl';

# ensure the 'zimbra' user is running the show
unless ( ( getpwuid( $< ) )[0] eq 'zimbra' ) {
        print "You must run this script as the 'zimbra' user.\n";
        exit 1;
}

# process command-line switches
getopts('vh:u:p:b:f:ln:',\%opt);
$VERBOSE = $opt{v} ? 1 : 0;
die "You must specify your LDAP host with -h.\n"
        unless $opt{h};
die "You must specify your LDAP base with -b.\n"
        unless $opt{b};

unless ( $opt{f} ) {
        $opt{f} = q/(objectclass='*')/;
        warn qq/Warning: using default LDAP filter "$opt{f}"; override with -f.\n/;
}

unless ( $opt{n} ) {
    $opt{n}= `$zmprov_cmd gad|head -1`;
    chomp($opt{n});
}

unless ($opt{v}) { print "using default domain = $opt{n}\n";}

# create the ldap object and bind to the LDAP server
my $ldap = Net::LDAP->new( $opt{h} ) or die $@;
my $msg = $ldap->bind( $opt{u} ? ( $opt{u}, password => $opt{p} ) : () );
$msg->code and die $msg->error;

# get a list of all users in the AD
print "Loading LDAP users...";
my %users = search( $ldap, base => $opt{b}, filter => $opt{f} );
print "OK.\n";

# no users? no work.
if ( ! %users ) {
        warn "No LDAP users found; aborting.\n";
        exit 1;
}

# pass every user to the gen_zmprov_command() routine to prepare the 
# user for a new zimbra account.  We also prepare the list of deleteAccount
# commands.
my @lines;
foreach ( sort keys %users ) {
        $users{ $_ } = gen_zmprov_command( user => $users{ $_ } );
        push @lines, "da '$users{ $_ }->{_address}'\n";
}

# List the users we found and bail, if the -l command-line switch is on.  We do this
# after the passing the results through gen_zmprov_command so we have the _address 
# to display.
if ( $opt{l} || $VERBOSE ) {
        print "The following users will be (re)created in zimbra:\n";
        printf '%-40s %s  %s'."\n", $users{ $_ }->{cn}[0], $users{ $_ }->{_address}, $users{ $_ }->{_password}
                foreach sort keys %users;
}
exit if $opt{l};

# Now we modify zimbra...

# deprovision existing accounts
print "Deleting existing accounts...";
zmprov(@lines);
print "OK.\n";

# restart mailboxd, to force the account cache to be cleared 
print "Restarting mailboxd...";
print `$zmcontrol_cmd restart`;

print "Pausing to allow server restart...";
sleep 3;
print "OK.\n";

# create new accounts
print "Creating new accounts...";
@lines = map { $users{ $_ }->{_cmd} } sort keys %users;
zmprov(@lines);
print "OK.\nRecreated " . scalar( keys %users ) . " accounts.\n";

# all done!
exit;

#
# SUB-ROUTINES 
#

# search( %param_hash )
# 
# Execute an LDAP search and return the results as a hash.
#
# Any args to Net::LDAP::search() may be passed as part of %param_hash; 
# if it exists, the 'index_attr' param specifies which LDAP attribute to 
# use as the key for the resulting %users hash.
# 
sub search {
        my $ldap = shift;
        my %args = @_;

        my $index_attr = delete $args{index_attr} || 'samaccountname';

        # establish some defaults
        $args{attrs}    ||= [ 'cn', 'userPrincipalName', 'memberOf', 'givenName', 'sn', 'sAMAccountName' ];
        $args{scope}    ||= 'sub';

        # do the search
        my $result = $ldap->search( %args );

        my %users;

        # rejigger the results into a useful format
        my $href = $result->as_struct;
        foreach my $valref ( values %$href ) {
                my $this;
                foreach my $attr ( sort keys %$valref ) {
                        next if $attr =~ /;binary$/; # ignore any binary data
                        $this->{ lc $attr } = $valref->{ $attr };
                }

                # add this user to the users hash
                $users{ $this->{ lc $index_attr }[0] } = $this;
        }

        return %users;
}

# gen_zmprov_command( user => $hashref )
#
# Determine the email address of the zimbra account to 
# create for the given LDAP user, and generate a createAccount
# command for zmprov.  These are added to the hashref as _address
# and _cmd, respectively, and the whole thing is returned.
#
sub gen_zmprov_command {
        my %args = @_;
        my $user = delete $args{user};

        # we authenticate against AD, so the local zimbra password is irrelevant; 
        # we'll generate difficult passwords just to be on the safe side.
        #$user->{_password} = mkpasswd( -length => 12 );
        $user->{_password} = Text::Password::Pronounceable->generate(6, 10);

        # our AD server manages internal.example.com, but we want email addresses
        # to be in the example.com domain, so we fix up the address here.  Your 
        # setup (and AD schema) may vary.
        my $address = $user->{userprincipalname}[0] ? $user->{userprincipalname}[0] : "$user->{cn}[0]\@$opt{n}" ;
        $address =~ s/internal\.//;

        $user->{_address} = $address;

        # build the createAccount command to be sent to zmprov.
        $user->{_cmd} = qq(createAccount '$address' ) . 
                                          qq('$user->{_password}' ) . 
                                          qq(displayName '$user->{cn}[0]' ) . 
                                          qq(givenName '$user->{givenname}[0]' ) . 
                                          qq(sn '$user->{sn}[0]'\n);

        # return the modified hashref
        return $user;
}

# zmprov( @commands )
# 
# execute zmprov and feed it a list of commands
# 
sub zmprov {

        # start the zmprov process and capture its filehandles
        my $pid = open3(\*WRITE, \*READ, \*ERROR, $zmprov_cmd)
                or die "Couldn't open pipe to $zmprov_cmd: $!";

        # use IO::Select to poll zmprov's STDERR
        my $sel = IO::Select->new;
        $sel->add(\*READ);
        $sel->add(\*ERROR);

        # send every command we've been given to zmprov and watch for errors.
        foreach ( @_ ) {

                # report what we're doing as we do it, if we're being verbose
                print $_ if $VERBOSE;

                # send the command to zmprov
                print WRITE $_;

                # watch for a response on STDERR
                foreach my $h ( $sel->can_read ) {
                        my $buf='';
                        if ( $h eq \*ERROR ) {

                                # XXX: should we abort if we get an error?
                                sysread(ERROR,$buf,4096);
                                warn "ERROR ( $_ ): $buf\n" if $buf;
                        }
                }
        }

        # no zombies
        waitpid($pid,1);
}


Remove Accounts

Here is a script that can be used to remove all accounts and distribution lists, save for accounts you specify. By default it leaves the admin account, the wiki account, and the ham and spam accounts.

#!/bin/bash
################################################################################
# $Id: deprovision,v 1.2 2007/02/18 20:41:43 dlbewley Exp $
#-------------------------------------------------------------------------------
# Description:
#   Script to remove all Zimbra accounts and distribution lists.
#   This may be useful for getting back to square one if you are testing bulk 
#   provisioning during a migration.
#
# See Also:
#   http://wiki.zimbra.com/index.php?title=Bulk_Create
#
# Usage:
#    su - zimbra
#    ./deprovision > deprovision.zmp
#    zmprov < deprovision.zmp
#
################################################################################

# this will be passed to grep -vE which acts as an exclude list
KEEP_ACCOUNTS='^(admin@|ham\.|spam\.|wiki@)'
# this will be passed to grep -vE which acts as an exclude list
KEEP_LISTS=''
# files to hold distlist and account lists
ACCOUNTS='accounts.zmprov'
LISTS='distlists.zmprov'

TMPA=`mktemp "$ACCOUNTS.XXXXXX"`
TMPL=`mktemp "$LISTS.XXXXXX"`

# get accounts lists
zmprov gaa > "$TMPA"
if [ -n "$KEEP_ACCOUNTS" ]; then
    grep -vE "$KEEP_ACCOUNTS" "$TMPA" > "$ACCOUNTS"
    rm "$TMPA"
else
    mv "$TMPA" "$ACCOUNTS"
fi

# get distribution lists
zmprov gadl > "$TMPL"
if [ -n "$KEEP_LISTS" ]; then
    grep -vE "$KEEP_LISTS" "$TMPL" > "$LISTS"
    rm "$TMPL"
else
    mv "$TMPL" "$LISTS"
fi

# remove accounts
cat "$ACCOUNTS" | while read account; do
    echo "da $account"
done

# remove lists
cat "$LISTS" | while read list; do
    echo "ddl $list"
done
Verified Against: Zimbra Collaboration 8.0, 7.0 Date Created: 4/3/2006
Article ID: https://wiki.zimbra.com/index.php?title=Bulk_Provisioning Date Modified: 2015-07-11



Try Zimbra

Try Zimbra Collaboration with a 60-day free trial.
Get it now »

Want to get involved?

You can contribute in the Community, Wiki, Code, or development of Zimlets.
Find out more. »

Looking for a Video?

Visit our YouTube channel to get the latest webinars, technology news, product overviews, and so much more.
Go to the YouTube channel »


Jump to: navigation, search