MTA unable to start due to wrong /dev/null permissions
MTA unable to start due to wrong /dev/null permissions
Problem
MTA unable to start due to wrong /dev/null permissions. There are two symptoms that may indicate that:
- 1. Switching to a zimbra user results in the following output:
-bash: /dev/null: Permission denied -bash: /dev/null: Permission denied -bash: /dev/null: Permission denied -bash: /dev/null: Permission denied -bash: /dev/null: Permission denied -bash: /dev/null: Permission denied /opt/zimbra/libexec/get_plat_tag.sh: line 138: /dev/null: Permission denied /opt/zimbra/libexec/get_plat_tag.sh: line 143: /dev/null: Permission denied /opt/zimbra/libexec/get_plat_tag.sh: line 148: /dev/null: Permission denied /opt/zimbra/libexec/get_plat_tag.sh: line 153: /dev/null: Permission denied /opt/zimbra/libexec/get_plat_tag.sh: line 158: /dev/null: Permission denied /opt/zimbra/libexec/get_plat_tag.sh: line 163: /dev/null: Permission denied /opt/zimbra/libexec/get_plat_tag.sh: line 168: /dev/null: Permission denied /opt/zimbra/libexec/get_plat_tag.sh: line 173: /dev/null: Permission denied /opt/zimbra/libexec/get_plat_tag.sh: line 178: /dev/null: Permission denied /opt/zimbra/libexec/get_plat_tag.sh: line 220: dpkg: command not found /opt/zimbra/libexec/get_plat_tag.sh: line 226: /dev/null: Permission denied /opt/zimbra/libexec/get_plat_tag.sh: line 263: /dev/null: Permission denied Can't open perl script "/dev/null": Permission denied -bash: /dev/null: Permission denied /opt/zimbra/bin/zmswatchctl: line 45: /dev/null: Permission denied Starting swatch.../opt/zimbra/bin/zmswatchctl: line 69: cannot redirect standard input from /dev/null: Permission denied /opt/zimbra/bin/zmswatchctl: line 45: /dev/null: Permission denied failed.
- 2. The second symptom is that MTA will not start.
Cause
The problem might be caused by different things. One could be that the permissions are changed during boot time by a security software. Another cause might be that udev is missing a rule, or the rule granting rights to the /dev/null file are changed due to an upgrade of udev to systemd-tools.
Resolution
To fix the permissions, we can do the following:
- 1) As root, delete the currently existing /dev/null:
# rm /dev/null
- 2) Recreate it with the mknod command:
# mknod -m 0666 /dev/null c 1 3
Troubleshooting
1. We can check in /lib/udev/rules.d the 50-udev-default.rules file (CentOS release 6.7), and make sure there is a line like the following or similar if its a different distribution:
KERNEL=="null|zero|full|random|urandom", MODE="0666"
2. If that line is there, but the rights are still wrong, we can create persistent audit watch rule, in order to determine whether anything is changing /dev/null permissions after boot.
- a) Set up a persistent audit watch in /etc/audit/audit.rules which will capture any write or attribute changes that are made to /dev/null. Include a key such as null-watch in order to easily search later for audit messages related to this rule. Please note, this rule needs to be included at some point after the -D rule at the top of that file which removes rules initially before proceeding systematically through the file.
Ideally our rule should be after the line: # Feel free to add below this line. See auditctl man page.
add the following line:
-w /dev/null -p wa -k null-watch
w - write to the file a - change in the file’s attribute
- b) Once this is in place, restart the auditd service in order to query the rules file and initiate this watch:
service auditd restart
- c) Check that the rule is in place:
# auditctl -l LIST_RULES: exit,always watch=/dev/null perm=wa key=null-watch
- d) Then check that auditing is enabled:
# auditctl -s AUDIT_STATUS: enabled=1 flag=1 pid=1826 rate_limit=0 backlog_limit=320 lost=0 backlog=0
- e) If not, enable auditing and ensure that the auditd service is configured to run persistently, in the appropriate run level:
auditctl -e 1 chkconfig --list auditd
- f) Reboot the system
- g) After the reboot, if the /dev/null has wrong permissions, check for any audit messages related to the watch using the following command:
ls -al /dev/null ausearch -i -k null-watch
