Mail Injection Stress Test

From Zimbra :: Wiki

Jump to: navigation, search
   Article-alert.png  - This article is a community contribution and may include unsupported customizations for network edition customers.


Contents

Zimbra version and platform

This script was developed and tested on Release 7.2 Network Edition.

Introduction

The purpose of this script is to load test a server by injecting email at a specified rate into a number of test accounts. Existing email found in the mail store is used as a source of email as this probably represents typical current and future email. Of course, this needs a server with something in the mail store already to work properly, not a fresh install.

Features:

  • Operates as "zimbra" user
  • Should works on both Open Source or Network edition
  • Email rate can be specified
  • Test accounts for email dump can be specified
  • Accounts can be locked to prevent anyone logging in and seeing everyone else's email
  • Accounts can be flushed when finished

Precautions:

  • Do not use on any account you need to keep, it will be flooded with junk
  • Ensure none of the accounts have forwarding of any kind set up or they will spout junk to somewhere else

Configuration

A file in the same directory as the script is run, lmtp_injector_conf, must exist to specify the user accounts and batch size. The batch size is the number of emails that are injected into an account per zmlmtpinject command. The smaller the number the more accounts will be injected in parallel. If it is too small the email rate won't be reached. If the batch size is too large then all the emails will be injected into one account at a time which may not exercise the system so well.

Here is a typical config file, change account batch size and accounts as per your requirements.

message_batch_size="100"

account_list="fred@example.com
harry@example.com
ned@example.com
mary@example.com
sue@example.com
larry@example.com
jane@example.com
liz@example.com"

Usage

First it is wise to lock all the test email accounts to prevent any chance of email being accidentally exposed to a user.

./lmtp_injector lock

To start injecting email at 10 emails per second just type,

./lmtp_injector 10/sec

The rate can be specified as <N>/sec, <N>/min, <N>/hour or <N>/day.

The script will now produce stats every 10 seconds until CTRL-C is pressed. As long as the actual message count keeps up with the target message count then all is well. If the actual count falls further and further behind then the process is running to full capacity. You may be able to increase the capacity by increasing the batch size in the configuration file.

To clean up after running the script type,

./lmtp_injector flush

The lmtp_injector script

#!/bin/bash
#    This program is free software: you can redistribute it and/or modify
#    it under the terms of the GNU General Public License as published by
#    the Free Software Foundation, either version 3 of the License, or
#    (at your option) any later version.

#    This program is distributed in the hope that it will be useful,
#    but WITHOUT ANY WARRANTY; without even the implied warranty of
#    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
#    GNU General Public License for more details.

#    You should have received a copy of the GNU General Public License
#    along with this program.  If not, see <http://www.gnu.org/licenses/>

##########################################################################
# Title      :  lmtp_injector
# Author     :  Simon Blandford <simon -at- onepointltd -dt- com>
# Date       :  2012-05-02
# Requires   :  zimbra
# Category   :  Administration
# Version    :  1.0.0
# Copyright  :  Simon Blandford, Onepoint Consulting Limited
# License    :  GPLv3 (see above)
##########################################################################
# Description
# Inject email found in store to a list of accounts at specified rate
##########################################################################


#Quit if not running as Zimbra
if [ "x$( whoami )" != "xzimbra" ]; then
  echo "You must be user zimbra to run this"
  exit 1
fi

message_store="/opt/zimbra/store/0"
tmp_folder="/tmp/lmtp_injector"
message_count_file="$tmp_folder"/"message_count"
stats_run_file="$tmp_folder"/"run_stats"
lmtp_injector_conf_file="lmtp_injector_conf"

source "$lmtp_injector_conf_file"

if ! echo "$account_list" | grep "@" >/dev/null; then
  echo "Account list not found in $lmtp_injector_conf_file" >&2
  exit 1
fi

if ! echo "$message_batch_size" | egrep "[0-9]+" >/dev/null; then
  echo "Batch size not set in $lmtp_injector_conf_file" >&2
  exit 1
fi

account_list_length="$( echo "$account_list" | wc -l )"

start_time=$( date "+%s" )

#Return message rate in messages per second
convert_message_rate () {
  local input_rate arr

  input_rate="$1"

  #Split at forward slash
  arr=(${input_rate//\// })
  input_number=$( echo ${arr[0]} | egrep "[0-9\.]+" )
  case ${arr[1]} in
    second)
      echo "scale=12; $input_number / 1.0" | bc
      ;;
    minute)
      echo "scale=12; $input_number / 60.0" | bc
      ;;
    hour)
      echo "scale=12; $input_number / 3600.0" | bc
      ;;
    day)
      echo "scale=12; $input_number / 43200.0" | bc
      ;;
    *)
      echo "Bad rate. Format should be numeric/(second/minute/hour/day)." >&2
      return 1
      ;;
  esac
}

#Return the time in seconds since the start
time_since_start () {
  echo $(( $( date "+%s" ) - start_time ))
}

#Return the numnber of messages that should have been sent by now
message_target () {
  local message_rate

  message_rate="$1"

  #Return integer
  printf "%1.0f\n" "$( echo "$( time_since_start ) * $message_rate" | bc )"
}

print_stats_line () {
  local message_rate count

  message_rate="$1"
  printf "%-40s%-30s%-30s%-30s\n" \
        "Message rate: $requested_rate" \
        "Elapsed : $( time_since_start ) seconds" \
        "Target sent: $( message_target "$message_rate " )" \
        "Actual sent: $( head "$message_count_file" )"
}

#Show statistics
show_stats () {
  local message_rate count

  touch "$stats_run_file"
  message_rate="$1"
 {
    while [ -f "$stats_run_file" ]; do
      if [ $(( i++  % 10 )) -eq 0 ]; then
        print_stats_line "$message_rate"
      fi
      sleep 1
    done
    print_stats_line "$message_rate"
  } &
}

#Set the list of accounts to given state given as parameter
set_accounts_state () {
  local zmcommand account state
  state=$1

  for account in $( echo $account_list ); do
    zmcommand="$zmcommand""
               ma $account  zimbraAccountStatus $state"
  done
  echo "$zmcommand" | /opt/zimbra/bin/zmprov
}

#Return a random account from the list
account_random () {
  while [ 1 ]; do
    account="$( echo "$account_list" | \
                sed -n "$(( (RANDOM % account_list_length) + 1 )){p;q;}" )"
    [ ${#account} -gt 5 ] && break
  done
  echo "$account"
}

#Return a random directory from the message store
drill_random () {
  local store item items

  store="$message_store"
  item="$store"
  (
    while [ -d "$item" ]; do
      cd "$item" || return 1
      items=$( ls -1 | wc -l )
      if [ $items -eq 0 ]; then
        item="$store"
        continue
      fi
      item=$( ls -1 | sed -n "$(( (RANDOM % items) + 1 )){p;q;}" )
    done
    echo "$( pwd )"
  )
}

#Find a random directory that has enough messages in it for a batch
get_full_directory () {
  local messages_in_dir selected_dir

  messages_in_dir=0
  while [ $messages_in_dir -lt $message_batch_size ]; do
    selected_dir=$( drill_random )
    messages_in_dir=$( ls -d "$selected_dir"/* | wc -l )
  done
  echo "$messages_in_dir $selected_dir"
}

#Get a batch of files
#Return the name of the directory on the first line
#and the list of message files on the next line
get_file_batch () {
  local message_dir_and_count message_count message_dir start_point

  message_dir_and_count=$( get_full_directory )
  message_count=$( echo "$message_dir_and_count" | awk '{print $1}' )
  message_dir=$( echo "$message_dir_and_count" | awk '{print $2}' )

  echo "$message_dir"
  start_point=$(( (RANDOM % message_count) + 1 ))
  while [ $(( start_point + message_batch_size )) -ge $message_count ]; do
    start_point=$(( (RANDOM % message_count) + 1 ))
  done
  echo $( ls -1 "$message_dir" | sed -n "$start_point,$(( start_point + message_batch_size - 1 ))p" )
}

#Flush the accounts
flush_accounts () {
  local account

  for account in $( echo $account_list ); do
    echo "Flushing account: $account"
    /opt/zimbra/bin/zmmailbox -z -m "$account" emptyFolder /inbox
  done
}

#Inject a batch of messages with From address and To addresss
#given as parameters
inject_batch () {
  local dir_and_file_list directory file_list

  from_account=$1
  to_account=$2

  dir_and_file_list=$( get_file_batch )
  directory=$( echo "$dir_and_file_list" | head -n 1 )
  file_list=$( echo "$dir_and_file_list" | tail -n 1 )

  #Create the lock directory or return fail
  mkdir "$tmp_folder""/""$to_account" 2>/dev/null || return 1
  (
    cd "$directory"
    #Run command in background and delete lock directory when done
    {
      /opt/zimbra/bin/zmlmtpinject -q -r "$from_account" -s "$to_account" $file_list
      rmdir "$tmp_folder""/""$to_account"
    } &
  )
  #Return success
  return 0
}

#Inject mail contained in a store directory into an email account
inject_mail () {
  from_account=$1
  to_account=$2
  mail_directory=$3
  [ $4 ] && flush="yes"
  #Silently quit if this account is already being injected
  mkdir "$tmp_folder""/""$to_account" 2>/dev/null || return
  echo "Injecting mail to account: $to_account from $mail_directory"
  stats=$( /opt/zimbra/bin/zmlmtpinject -N 9999 -r "$from_account" -s "$to_account" -d "$mail_directory" )
  [ $flush ] && echo "Flushing mail from account: $to_account from $mail_directory"
  [ $flush ] && /opt/zimbra/bin/zmmailbox -z -m $to_account emptyFolder /inbox
  echo "$stats"
  echo "Done"
  rmdir "$tmp_folder""/""$to_account"
}

quit_out () {
    local time_out_counter

    echo
    echo "Quitting..."
    #Wait for batches to finish
    while ls -d "$tmp_folder""/"*@* &>/dev/null; do
      if [ $(( time_out_counter++ )) -gt 60 ]; then
        echo "Batches not finished in a minute. Giving up." >&2
        echo "Please kill running processes manually." >&2
        break;
      fi
      sleep 1
    done
    rm -f "$stats_run_file"
    sleep 1
    rm -rf "$tmp_folder"
    echo "Done"
    exit 0
}


if [ "$1" ] && [ "$1" == "flush" ]; then
  flush_accounts
  exit
fi
if [ "$1" ] && [ "$1" == "lock" ]; then
  set_accounts_state locked
  exit
fi


#Trap Ctrl-C
trap quit_out SIGINT

#Make and clear temporary lock directory
mkdir -p "$tmp_folder" 2>/dev/null
rmdir "$tmp_folder"/* 2>/dev/null

requested_rate="$1"
message_rate=$( convert_message_rate "$requested_rate" ) || exit 1

echo "0" >"$message_count_file"
show_stats "$message_rate"

while [ 1 ]; do
  #Submit a new batch every time more messages are required to meet target
  if [ $( head "$message_count_file" ) -lt $( message_target "$message_rate " ) ] 2>/dev/null; then
    from_account=$( account_random )
    to_acccount=$( account_random )
    #If batch runs OK then add to total message count
    if inject_batch "$from_account" "$to_acccount"; then
      echo "$(( $( head "$message_count_file" ) + message_batch_size ))" >"$message_count_file"
    fi
  fi
done
Personal tools