Home Mail Server with TLS and non-Plaintext Authentication
Setting up your own mail server is quite involved, and you don’t want to go into that if you are happy with the privacy declaration of your favourite mail hosting provider. If not, then your own home mail server could take out most of your e-mail communication (probably not all) from the radar of those many interested third parties.
Please don’t expect the perfect privacy solution for all your communication needs. This does not exist, by the way. A proper Home Mail Server installation will only help to prevent that your favorite Secret Service reads all your e-mails, they would have access only to some of them, and in the course of fine tuning your setup this portion could become less every day.
The main advantage of a Home Mail Server is, that all e-mails once received reside in our Castle behind the Drawbridge, and for this the Castle Doctrine or similar legislation applies - in short, Sucking Secret Service Stalkers (SSSS) would have a hard time to get access to your e-mails on your home server without a judicial search decision. In addtion, at least in Germany, any unauthorized access to our IT systems (even by government entities without a JSD) is considered a criminal act - Computer-Sabotage:
Computer sabotage is in Germany according to § 303b of the Criminal Code (StGB) an offense which is punishable with imprisonment up to three years, in particularly serious cases up to ten years, or a fine.
The attempt is punishable.
So, the odds of our Home Mail Server of surviving in the IT-Security-Wild-West are not that bad. Two main parts constitute a mail server. The Mail Transport Agent (MTA) which is responsible for the transport of incoming and outgoing e-mail, and the Mail Storage System wich is responsible for mail storage and for retrieval by the clients.
The Dovecot Mail Storage System
Many people believe that plaintext authentication is secure once used together with TLS, and therefore they leave out the hassle of setting up an inherently more secure authentication mechanism. The Dovecot documentation wiki discusses more security considerations and in this respect it is specially important to understand the relationship between the Authentication Mechanism and the Password Storage scheme, one of which must be for technical reasons plaintext (or near to plaintext with respect to the security quality). This means, we either may have encrypted authentication with plaintext (or weakly hashed) passwords in the password store, or we may have plaintext authentication with strongly salted+hashed passwords in the password store on the server. The Dovecot conclusion seems to be to go with the latter, i.e. to have plaintext authentication secured by TLS, and to stay with the benefit of a highly secured password store.
This is not necessarily a bad choice, specially if the server admin sees a certain danger that the password store can be accessed by an unauthorized 3rd party. In this case, it would be really good to have the passwords salted+hashed as strongly as possible, and as god will, let TLS secure the plaintext authentication. Most probably this is the choice for small to medium companies where a considerable number of persons have access to the mail server, and where a separate password server, e.g. Kerberos, has not been setup. There are other points of views though:
-
I am talking about a home mail server, and if 3rd parties ever would get access to it and come even close to the password store, then the damage would be already that huge that access to the passwords would be of least concern.
-
Even in a small company, I would setup separate servers for mail, file, web, etc. and passwords.
-
There are advanced technics of TLS interception. Companies tend to install DPI firewalls, showing a valid certificate to internal clients, so the firewall can decrypt the packets, filter on it, and encrypt it again for the transport to the final destination. So, accessing your home mail server from behind a DPI firewall potentially shows your password in clear to the company’s admin(s).
-
in principal said kind of DPI would work also on a larger scale, a potential MITM for example sitting in Cheltenham would only need to offer a somehow valid certificate to the client and could decrypt the traffic. Another consideration is that TLS connections are still not guaranteed to provide Perfect Forward Secrecy (s. also SSL: Intercepted today, decrypted tomorrow). That means, government attackers may intercept the message, and obtain the private keys from the CA by the way of, for example a NSL accompanied with a gag order. With the private key, the whole traffic could be decrypted later on. In both cases any plaintext passwords are revealed, and could be used for secret logins in order to get access to all the mails stored on that IMAP mail server for the respective user.
These considerations let me not to follow the suggestion of the Dovecot documentation. I choose to setup my mail server with the CRAM-MD5 authentication mechanism, staying with the drawback that the passwords in the store may only be weakly hashed using the MD5 algorithm.
I choose CRAM-MD5 because all of my clients (macOS, iOS, Android and Windows) do support this for IMAP, POP3, and SMTP-Auth. Even, if MD5 is nowadays vulnerable to brute force attacks by its own, in the CRAM-MD5 scheme, it can be still considered safe against online attacks, i.e. cracking the scheme in the course of the connection. However, an offline dictionary or brute force attack to recover the password seems to be feasible after intercepting a successful CRAM-MD5 protocol exchange.
Yes, CRAM-MD5 and Digest-MD5 came to age, and I would love to see a Digest-SHA512 authentication mechanism, but it doesn't exit yet and we have to live with what we have. It is worth to note, that MD5 is not completely useless. The occasional brute force cracker would already have a hard time for passwords with more than 10 characters in length, and also super computers would take a reasonable amount of dedicated time for brute-forcing a 13 char password. Reasonable means, that it wouldn't finish until you rotated your password anyway, let's say once in a month. Dedicated means, that this computer wouldn't do anything else than trying to crack your password. The budget of the best equipped Secret Service would be rapidly exhausted, if they want to do this for millions of intercepted CRAM-MD5 sessions.
In addition to length (>11 characters), the usual advises for password strength keep-on being valid, like using the full character set, i.e. alphanumerics, punctation, accented chars, etc. In this respect it helps a lot to switch from short Passwords to long Passphrases, e.g. something like:
«Ach wie gut, daß niemand weiß, daß ich Rumpelstilzchen heiß.»
The Postfix MTA and the ISP must not block port 25
You need a FreeBSD server connected to the internet, and your ISP must not block the incoming SMTP port 25. A fixed IP address is not needed, but doesn't harm either. You don't want to go into the big effort of setting up your home mail server, only to find out that it is useless because port 25 is blocked, so it is better to check it before. Connect a *nix machine to the internet by the way of the public endpoint that you want to check. If you connect it by the way of a router, then make sure that port 25 is NAT-redirected to said machine. Also you need allow traffic on port 25 on your firewall(s) for that test machine.
-
On the command line of the test machine execute as user
root:
# nc -lk 25
-
Enter a web browser of a computer sitting in the same local network of said machine, and point it to: http://www.canyouseeme.org.
-
Verify that the web page shows the correct public IP, then enter 25 into the text field and press the button <Check Port>.
In case of Success, you may proceed, in case of Error, first troubleshoot your setup (firewall, NAT, router) and in case this doesn’t help, ask your ISP or perhaps another ISP for a different contract.
The Domain Name
We need our own domain hosted by a hosting provider having the following characteristics:
-
dynamic DNS configuration for that domain including MX records,
-
a basic mail service, so the provider would accept authenticated SMTP relaying of our outgoing messages - note, we won't need anything else of the mail services from the hosting provider.
Installations
We install Postfix and Dovecot with the default options from the FreeBSD package repository:
pkg install dovecot
pkg install postfix-sasl
Verify, that the file /etc/mail/mailer.conf contains the following:
# $FreeBSD: releng/11.2/etc/mail/mailer.conf 327765 2018-01-10 09:06:07Z delphij $
#
# Execute the Postfix sendmail program, named /usr/local/sbin/sendmail
#
sendmail /usr/local/sbin/sendmail
send-mail /usr/local/sbin/sendmail
mailq /usr/local/sbin/sendmail
newaliases /usr/local/sbin/sendmail
...
...
Configurations
Dovecot and Postfix offer a huge variety of configuration options which may serve for any imaginable mail server constellation. We are going to set up a simple Home Mail Server and want to stay with the default settings as far as possible. Anyway, some decisions must be taken:
-
All mail services shall be secured by TLS using a certificate from Let’s Encrypt, and the certificate store is at
/usr/local/etc/letsencrypt/live/example.com/
-
The mail services shall be offered to virtual users that are different from the system users.
-
The expected small number of users does not justify a special backend for the user database (like LDAP or xSQL), so we stay with a simple passwd-style file.
-
For the mail store we create the new directory tree at
/var/mail/users/.
For the following let us assume that the mail service, which we are going to set up, is for the domain example.com. The MX for this would be mail.example.com. In addition, both domains resolve by the way of dynamic DNS updates to the public IP of our FreeBSD machine, and we also have already the respective Let’s Encrypt certificate in place.
We need to generate a set of Diffie-Hellman parameters in order to harden the DH ephemeral key exchange which provides for Perfect Forward Secrecy:
mkdir -p ~/config/certs
cd ~/config/certs
openssl dhparam -out dh_2048.pem -2 2048
openssl dhparam -out dh_1024.pem -2 1024
chmod -R 0000 ~/config/certs
The Virtual Mail User Store
The mail users in this setup are virtual users, i.e. they are not linked to system users. Mail users are registered in a separate Virtual User Store, which may be a simple file or a LDAP or xSQL database. The dovecot/imap daemon accesses on behalf of the virtual user his mails, and the virtual user would have no access by other means.
Dovecot would authenticate users against the user names and passwords in the Virtual Mail User Store. A user entry may contain uid and gid values, and if present, dovecot/imap accesses the user's mails with that uid/gid pair. It is recommended to assign to each virtual user a unique uid. This one, let's call it v-uid, would be valid for the file system, i.e. dovecot/imap may use it for read/write access to the virtual user's mails, but this v-uid would ideally have no correspondent among the system users.
I recommend to choose 10000 as the first valid v-uid and v-gid each. In order to manifest that choice, let us create one system user and one system group having these id's. Technically, this won't have any effect, but it has the benefit that the Virtual Mail Users got a mark in the regular user database, which later on may make the admin remember that virtual users do exist on the system.
pw groupadd virtmail -g 10000
pw useradd virtuser -u 10000 -g 10000 -c "Virtual Mail User" -d /var/empty -s /usr/sbin/nologin
pw groupmod virtmail -m virtuser,dovecot,postfix
The login name of a mail user is his/her e-mail address, e.g. rolf@example.com. For the CRAM-MD5 authentication scheme, the hash of the password is stored, and the hash can be obtained using the command doveadm pw passing to it for example the incredible password “rolfheinrich":
doveadm pw -s CRAM-MD5 -p rolfheinrich
>>>>>
{CRAM-MD5}cf8527ae5867fc08068952261363c49f0bcc0d1d5f166bbd20d3783b3e97f828
For adding a new user to the mail users file, do the following:
cd /usr/local/etc/dovecot
printf "rolf@example.com:%s:10001:10000:\n" `doveadm pw -s CRAM-MD5 -p "rolfheinrich"` >> users
The format of the virtual users file /usr/local/etc/dovecot/users is:
<login name>:<password hash>:<v-uid>:<v-gid>::<home directory>::<extra fields>
In case this file doesn't yet exist, it would be created by the above command, otherwise the new user would be simply added on a new line at the end of the file. Assign a unique v-uid starting at 10001 to each user, and stay with the base v-gid of 10000 for everyone. Leave the field for the home directory empty, i.e. the trailing colon:
rolf@example.com:{CRAM-MD5}cf8527ae5867fc08068952261363c49f0bcc0d1d5f166bbd20d3783b3e97f828:10001:10000:
...
...
The Mail Directory
In /var/mail create the new directory users. By standard, mails that are send to system users, mainly root are stored in plain mbox files at the root level of /var/mail - one file for each user, and its file name is the user's name. The new sub-directory users will receive and store the mails for the virtual mail accounts.
Dovecot can store mails individually in directories or in mbox files. A mbox file contains all the mails appended one to another in the sequence of entrance. This format is adequate for mail stores that are not likely to change heavily. However, a IMAP server allows its users to organize the mails in sub-directories. The user may keep some mails, delete others, mark mails as junk that are deleted later, store drafts that are moved to the sent box later, etc. Therefore, it is better, to have the mails stored individually in respective sub-directories. Thereby, for example, moving mails around is basically a file system operation, which is much more effecient compared to extracting a chunk from the middle of a huge text file, and insert this chunk somewhere into another huge text file. Definitely, we want our mails to be stored in the Maildir format.
For now, adjust the correct access rights of the new directory /var/mail/users:
mkdir -p /var/mail/users
chown dovecot:virtmail /var/mail/users
chmod 770 /var/mail/users
Dovecot populates this mail directory with the mailboxes of the virtual mail users when appropriate.
The Dovecot Configuration File
In the previous chapters all pre-requisites were set up, and so Dovecot itself can be configured quickly. Create and edit the file /usr/local/etc/dovecot/dovecot.conf having the following content:
auth_mechanisms = cram-md5
first_valid_uid = 10000
first_valid_gid = 10000
mail_location = maildir:/var/mail/users/%u
userdb {
args = username_format=%u /usr/local/etc/dovecot/users
driver = passwd-file
}
passdb {
args = username_format=%u /usr/local/etc/dovecot/users
driver = passwd-file
}
namespace inbox {
inbox = yes
location =
mailbox Drafts {
special_use = \Drafts
}
mailbox Junk {
special_use = \Junk
}
mailbox "Sent Messages" {
special_use = \Sent
}
mailbox "Deleted Messages" {
special_use = \Trash
}
prefix =
}
service auth {
unix_listener /var/spool/postfix/private/auth {
user = postfix
group = postfix
mode = 0666
}
}
service lmtp {
unix_listener /var/spool/postfix/private/dovecot-lmtp {
user = postfix
group = postfix
mode = 0660
}
}
syslog_facility = local1
ssl_cipher_list = HIGH:!aNULL:!AES128:!SSLv2
ssl_cert = </usr/local/etc/letsencrypt/live/example.com/fullchain.pem
ssl_key = </usr/local/etc/letsencrypt/live/example.com/privkey.pem
ssl_dh = </root/config/certs/dh_2048.pem
-
Replace
example.com by your actual domain.
-
The only allowed authentication mechanism is
cram-md5.
-
Dovecot doesn't serve system users having
uid/gid values less than of that which is assigned to the parameters first_valid_uid/first_valid_gid.
-
The mail store is
/var/mail/users/, and each user owns a mailbox %u.
-
The virtual mail users are registered in the passwd-like file
/usr/local/etc/dovecot/users.
-
Each mailbox contains some special sub-directories, whose names shall match the respective names used by the mail client.
-
Dovecot provides user authentication and local transport for Postfix.
-
The Let’s Encrypt certificate chain is assigned, and strong SSL ciphers are preferred.
-
Use
syslog for logging. Choose a free facility, e.g. local1.
Configuration of the Postfix Mail Transfer Agent
The Postfix MTA is supposed to receive mails directly from the internet. For this, the first entry in the MX record of our domain shall point to the public IP address of the Postfix host. In addition, our ISP and our firewall must not block the incoming port 25. However, many peers would not accept outgoing mails directly from our mail server, as long as we don't run it on a fixed public IP address with an associated valid PTR record, allowing reverse resolution of that IP to our domain. Therefore, we setup Postfix to pass outgoing mails to a relay host, which then distributes it to the final recipients. Usually we would contract a simple mail service together with the domain hosting service provided by the hosting provider.
Relay Host Configuration
Our hosting provider informed to us a domain name of its outgoing SMTP server, e.g. smtp.maildomainhoster.net, and we received SMTP-Auth login credentials for passing our outgoing mails to this server. Postfix acts in this respect pretty much like a normal e-mail client, when sending out mail. Our hosting provider won't even see a difference.
Create and edit the file /usr/local/etc/postfix/relay-sasl-password and enter the login credentials for accessing the SMTP server of our hosting provider:
[smtp.maildomainhoster.net]:submission admin_user@example.com:plaintext-password
Replace the items in blue with the real SMTP host name and with your exact login credentials. You would indicate a admin user who is allowed to send e-mails on behalf of any user of your domain. The above assumes that the relay host accepts mail on the submission port 587. If this is not the case, then remove :submission after the closing square bracket. The file must be mapped in order Postfix can use it:
cd /usr/local/etc/postfix
postmap relay-sasl-password
Note: /usr/local/etc/postfix/relay-sasl-password must be mapped again each time when it has been edited.
Local Recipients and Mail Aliases
We want Postfix to accept incoming mails for local recipients only, and therefore we need to tell it somehow who the local recipients are. For this we can simply map the /usr/local/etc/dovecot/users file which has been created in the course of setting up Dovecot’s virtual mail users. Since the file contains information which is not needed for said purpose, we pass it through sed(1) which strips off the password hashes etc. before mapping:
cd /usr/local/etc/dovecot
sed 's/:.*/ exists/' users > tmp; postmap tmp; mv tmp.db users.db; rm tmp
Note: The file /usr/local/etc/dovecot/users must be re-mapped for Postfix each time it has been changed.
Postfix considers the users listed in the system file /etc/aliases also as local recipients. The default /etc/aliases file tells the system to redirect all diagnostic messages to the user root. For our convenience we add to aliases one line, which redirects system mails from root to a virtual mail user, and thereby, the results of the nightly security checks and of the other periodic scripts would be passed to a normal mail user. Depending on our preferences, we would add a separate admin user in the same way we would add any other user to the virtual mail users file, and redirect mails from root to that virtual admin user, or we would simply redirect it to our own mail account. I did the latter, and I added to /etc/aliases at the very end of the file the following line:
...
root: rolf
This file must also be mapped for Postfix, and in this case we use the sendmail compatible command:
/usr/local/bin/newaliases
The Postfix Configuration Files
All pre-requisites are discussed in the previous chapters, and finally Postfix can be setup by editing its two configuration files /usr/local/etc/postfix/main.cf and /usr/local/etc/postfix/master.cf. A new installation comes with example configuration files. Move main.cf out of the way, and create a copy of master.cf from the template master.cf.example:
cd /usr/local/etc/postfix
rm main.cf
Edit master.cf, for enabling Postfix listening on the submission ports 465 and 587. Remove the hash signs in front of the respective protocol lines and add the options as outlined below:
...
#tlsproxy unix - - n - 0 tlsproxy
submission inet n - n - - smtpd
-o syslog_name=postfix/submission
-o smtpd_tls_security_level=encrypt
-o smtpd_sasl_auth_enable=yes
-o smtpd_tls_auth_only=yes
-o smtpd_recipient_restrictions=permit_sasl_authenticated,reject
-o receive_override_options=no_header_body_checks
-o smtpd_client_auth_rate_limit=0
-o smtpd_client_connection_rate_limit=0
-o smtpd_soft_error_limit=99
-o smtpd_hard_error_limit=100
-o smtpd_error_sleep_time=0s
...
smtps inet n - n - - smtpd
-o syslog_name=postfix/submission
-o smtpd_tls_wrappermode=yes
-o smtpd_tls_security_level=encrypt
-o smtpd_sasl_auth_enable=yes
-o smtpd_tls_auth_only=yes
-o smtpd_recipient_restrictions=permit_sasl_authenticated,reject
-o receive_override_options=no_header_body_checks
-o smtpd_client_auth_rate_limit=0
-o smtpd_client_connection_rate_limit=0
-o smtpd_soft_error_limit=99
-o smtpd_hard_error_limit=100
-o smtpd_error_sleep_time=0s
Edit the replacement file main.cf for having the following content:
mydomain = example.com
myhostname = mail.$mydomain
myorigin = $mydomain
mydestination = $mydomain, $myhostname, localhost
# RECEIVE MAILS FOR KNOWN USERS AND ALIASES ONLY
local_recipient_maps = $alias_maps,
hash:/usr/local/etc/dovecot/users
virtual_alias_maps = pcre:/usr/local/etc/postfix/catchall_recipients.pcre
# TRUST AND RELAY CONTROL
mynetworks = 127.0.0.1/32
# DELIVERY TO MAILBOX
mailbox_transport = lmtp:unix:private/dovecot-lmtp
# INCOMING MAIL
disable_vrfy_command = yes
smtpd_client_restrictions = reject_unauth_pipelining,
permit
smtpd_helo_required = yes
unknown_hostname_reject_code = 550
smtpd_helo_restrictions = permit_mynetworks,
permit_sasl_authenticated,
reject_unknown_helo_hostname,
check_helo_access hash:/usr/local/etc/postfix/helo_reject_domains,
permit
smtpd_discard_ehlo_keywords = silent-discard, dsn
smtpd_sender_restrictions = reject_unknown_sender_domain,
reject_non_fqdn_sender,
check_sender_access hash:/usr/local/etc/postfix/sender_domain_reject,
check_sender_access pcre:/usr/local/etc/postfix/sender_access.pcre,
permit
smtpd_etrn_restrictions = permit_mynetworks,
reject
smtpd_sasl_auth_enable = no
smtpd_sasl_type = dovecot
smtpd_sasl_path = private/auth
smtpd_relay_restrictions = permit_mynetworks,
permit_sasl_authenticated,
reject_unauth_destination
smtpd_recipient_restrictions = permit_mynetworks,
permit_sasl_authenticated,
check_policy_service unix:private/greyfix,
check_recipient_access hash:/usr/local/etc/postfix/invalid_recipients,
permit
header_checks = pcre:/usr/local/etc/postfix/header_checks.pcre
message_size_limit = 104857600
mailbox_size_limit = 209715200
# RATE THROTTLING
smtpd_soft_error_limit = 3
smtpd_hard_error_limit = 5
smtpd_client_auth_rate_limit = 11
smtpd_client_connection_rate_limit = 13
smtpd_error_sleep_time = 17s
# TLS ADJUSTMENTS
tls_high_cipherlist = HIGH:!aNULL:!AES128:!SSLv2
tls_preempt_cipherlist = yes
smtpd_tls_security_level = encrypt
smtpd_tls_received_header = yes
smtpd_tls_mandatory_ciphers = high
smtpd_tls_mandatory_exclude_ciphers = aNULL
smtpd_tls_mandatory_protocols = !SSLv2, !SSLv3
smtpd_tls_eecdh_grade = strong
smtpd_tls_dh1024_param_file = /root/config/certs/dh_2048.pem
smtpd_tls_dh512_param_file = /root/config/certs/dh_1024.pem
smtpd_tls_cert_file = /usr/local/etc/letsencrypt/live/example.com/fullchain.pem
smtpd_tls_key_file = /usr/local/etc/letsencrypt/live/example.com/privkey.pem
# OUTGOING MAIL
queue_run_delay = 120s
minimal_backoff_time = 120s
maximal_backoff_time = 180s
maximal_queue_lifetime = 24h
bounce_queue_lifetime = 24h
relayhost = [smtp.maildomainhoster.net]:submission
smtp_sasl_auth_enable = yes
smtp_sasl_password_maps = hash:/usr/local/etc/postfix/relay-sasl-password
smtp_tls_security_level = encrypt
smtp_tls_mandatory_ciphers = high
smtp_tls_mandatory_exclude_ciphers = aNULL
smtp_tls_mandatory_protocols = !SSLv2, !SSLv3
# POSTFIX LOCATIONS, USER, GROUP, AND PROTOCOL
readme_directory = /usr/local/share/doc/postfix
sample_directory = /usr/local/etc/postfix
sendmail_path = /usr/local/sbin/sendmail
html_directory = /usr/local/share/doc/postfix
setgid_group = maildrop
command_directory = /usr/local/sbin
manpage_directory = /usr/local/man
daemon_directory = /usr/local/libexec/postfix
newaliases_path = /usr/local/bin/newaliases
mailq_path = /usr/local/bin/mailq
queue_directory = /var/spool/postfix
mail_owner = postfix
data_directory = /var/db/postfix
inet_protocols = ipv4
compatibility_level = 2
-
Replace
example.com by your actual domain.
-
Restrict mynetworks to only localhost, i.e. force also the LAN clients to use SMTP auth for mail submission.
-
Utilize Dovecot for the mailbox transport.
-
Utilize Dovecot-SASL for SMTP client authentication.
-
Pass outgoing mails to the relay host of our domain & mail hosting provider.
-
Replace
smtp.maildomainhoster.net by the domain of our domain & mail hosting provider.
-
The above settings do not allow non-TLS connections. If some of you peers do not understand TLS, then you may consider to change
smtpd_tls_security_level from encrypt to may.
-
Likewise, if the relay host does not understand TLS, set
smtp_tls_security_level = may.
-
In
virtual_alias_maps we inform our catchall addresses - see my BLog post Disposable E-Mail Addresses with Postfix.
File /usr/local/etc/postfix/catchall_recipients.pcre:
# PERSONALIZED CATHALL
/^.*me@example\.com$/ me@example.com
/^.*he@example\.com$/ he@example.com
...
# GENERAL CATCHALL
/^.+@example\.com$/ me@example.com
-
In
smtpd_helo_restrictions we inform domain names wich we don’t accept in the helo/ehlo phase of a connection request, e.g. 3rd party peers who pretend to reside in our domain are frauds, and we stop’em already at this stage. Domains of other idiots go into here as well.
File /usr/local/etc/postfix/helo_reject_domains:
example.com REJECT User unknown in local recipient table
server0.workflicker.com REJECT User unknown in local recipient table
server1.workflicker.com REJECT User unknown in local recipient table
server2.workflicker.com REJECT User unknown in local recipient table
server3.workflicker.com REJECT User unknown in local recipient table
server4.workflicker.com REJECT User unknown in local recipient table
server5.workflicker.com REJECT User unknown in local recipient table
server6.workflicker.com REJECT User unknown in local recipient table
server7.workflicker.com REJECT User unknown in local recipient table
server8.workflicker.com REJECT User unknown in local recipient table
server9.workflicker.com REJECT User unknown in local recipient table
...
-
In
smtpd_sender_restrictions we inform well known e-mail domains (hash map) and in another file regular expression mappings to e-mail-addresses of senders, from whom we do not want to receive anything.
File /usr/local/etc/postfix/sender_domain_reject:
gutemails.ga REJECT User unknown in local recipient table
officeemailinfo.net REJECT User unknown in local recipient table
organicstuffer.com REJECT User unknown in local recipient table
servsahost.store REJECT User unknown in local recipient table
servsbhost.store REJECT User unknown in local recipient table
servschost.store REJECT User unknown in local recipient table
servsdhost.store REJECT User unknown in local recipient table
servsehost.store REJECT User unknown in local recipient table
...
File /usr/local/etc/postfix/sender_access.pcre:
/^.*example\.(com|de)@(?!.*github.com)(.*)$/ 554 User unknown in local recipient table
/^.*gautamgovinda@outlook.com$/ 554 User unknown in local recipient table
/^.*vivofelizbr.com.br$/ 554 User unknown in local recipient table
/^.*laviena.com.br$/ 554 User unknown in local recipient table
/^.*vejanovidades.com.br$/ 554 User unknown in local recipient table
/^.*xltaffiliate.se$/ 554 User unknown in local recipient table
/^.*correiosebraesp.com.br$/ 554 User unknown in local recipient table
/^.*thecontenders.com.br$/ 554 User unknown in local recipient table
/^.*mpst.net.br$/ 554 User unknown in local recipient table
/^.*emktlw-[0-9]*.com$/ 554 User unknown in local recipient table
...
-
In
smtpd_recipient_restrictions we activate Greylisting utilizing the mail/greyfix facility - see my BLog post Why not use Spamhaus? Why not use another DNSBL?. In addition we inform our e-mail addresses which have been discarded, and to which we don’t want to receive e-mails anymore.
File /usr/local/etc/postfix/invalid_recipients:
adobe@example.com
adobe@example.com
alx@example.com
ebrats2015@example.com
ebrats2018@example.com
ama-rolf@example.com
daora-rolf@example.com
...
-
In
header_checks we inform regular expressions for sanitizing some of the e-mail headers, and here we place also our spam trap(s), and in addition tell extensions of attachments which we don’t want to see coming-in to our site.
File /usr/local/etc/postfix/header_checks.pcre:
### Removes the notification request from the mail headers
/^Return-Receipt-To:/ IGNORE
/^Disposition-Notification-To:/ IGNORE
### E-mails with the wrong date disturb our mail sorting
/^Date: .* 20[2-9][0-9]/ REJECT Your email got a date in the future.
/^Date: .* 2019/ REJECT Your email got a date in the future.
/^Date: .* 20[0-1][0-7]/ REJECT Your email got a date in the past.
/^Date: .* 19[0-9][0-9]/ REJECT Your email got a date in the past.
### Spam-Traps
/^Cc: .*xyz@example\.com/ REJECT User unknown in local recipient table.
/^Cc: .*adobe@example\.com/ REJECT User unknown in local recipient table.
/^Cc: .*ama.*@example\.com/ REJECT User unknown in local recipient table.
/^Cc: .*daora.*@example\.com/ REJECT User unknown in local recipient table.
### Emails containing attachments with the following endings will be rejected
/^Content-(Disposition|Type).*name\s*=\s*"?([^;]*(\.|=2E)(ade|adp|asp|bas|bat|chm|cmd|com|cpl|crt|dll|exe|hlp|ht[at]|inf|ins|isp|jse?|lnk|md[betw]|ms[cipt]|nws|\{[[:xdigit:]]{8}(?:-[[:xdigit:]]{4}){3}-[[:xdigit:]]{12}\}|ops|pcd|pif|prf|reg|sc[frt]|sh[bsm]|swf|vb[esx]?|vxd|ws[cfh]))(\?=)?"?\s*(;|$)/x REJECT Attachment name "$2" may not end with ".$4"
Syslog Configurations
By default, Postfix logs everything by the way of the syslog mail facility and Dovecot has been configured to use the syslog local1 facility for logging. Without this setting, Dovecot would intermix its log messages into the same log-file as Postfix.
Edit the file /etc/syslog.conf, add the local1.* entry and change the mail.info entry to mail.* as follows:
...
local1.* /var/log/dovecot.log
mail.* /var/log/postfix.log
...
For configuring log rotation for the new log files, edit the file /etc/newsyslog.conf, and remove the setting for /var/log/maillog and add 2 new settings:
...
/var/log/dovecot.log 640 7 * @T00 JC
/var/log/postfix.log 640 7 * @T00 JC
...
Disable Sendmail and Start-Up Dovecot/Postfix
In file /etc/rc.conf add the following section:
...
## Mail Services
# Disable sendmail
sendmail_enable="NONE"
# Enable Dovecot POP3/IMAP
dovecot_enable="YES"
# Enable Postfix SMTP
postfix_enable="YES"
Edit the file /etc/periodic.conf adding the following content:
...
daily_clean_hoststat_enable="NO"
daily_status_mail_rejects_enable="NO"
daily_status_include_submit_mailq="NO"
daily_submit_queuerun="NO"
Finally, take 3 times a deep breath - - - then:
service sendmail stop
>>>>>
Stopping sendmail.
Waiting for PIDS: 1107.
sendmail_submit not running? (check /var/run/sendmail.pid).
Stopping sendmail_clientmqueue.
Waiting for PIDS: 1110.
service postfix start
>>>>>
postfix/postfix-script: starting the Postfix mail system
service dovecot start
>>>>>
Starting dovecot.
ps -axj
>>>>>
...
root 6891 1 6891 6891 0 Is ?? 0:00.02 /usr/local/sbin/dovecot -c /usr/local/etc/dovecot/dovecot.conf
dovecot 6892 6891 6891 6891 0 I ?? 0:00.01 dovecot/anvil
root 6893 6891 6891 6891 0 I ?? 0:00.01 dovecot/log
root 6895 6891 6891 6891 0 I ?? 0:00.03 dovecot/config
...
root 7014 1 7014 7014 0 Ss ?? 0:00.05 /usr/local/libexec/postfix/master -w
postfix 7015 7014 7014 7014 0 S ?? 0:00.02 pickup -l -t unix -u
postfix 7016 7014 7014 7014 0 I ?? 0:00.02 qmgr -l -t unix -u
...
If you see more or less something like the above in your process listing, then your Home Mail Server is up and running. Give it another clean start by rebooting the FreeBSD machine.
In case this does not show the Dovecot and/or the Postfix Daemons, then look into the log files and start with trouble shooting.
tail -100 /var/log/dovecot.log
tail -100 /var/log/postfix.log
Web Mail
The typical usage scenario for a Web Mail Service on our Mail Server is to have access to our mails using foreign equipment and software, usually because our own equipment with its dedicated mail clients is not always with us, or does not work properly in all network environments.
For example:
-
at work you would use the companies computer to browse your mails using Web Mail.
-
on travel you use a terminal at the airport, hotel, LAN-house, etc. because it is not guarantteed that you can connect your equipment at any time you would need it.
Now, being concerned on security issues, of course, we want our Web Mail to run as a HTTPS service. So the web server needs to have TLS (SSL) enabled, and we need to provide it with our certificate chain. However, enabling TLS on the server side will not always prevent 3rd party eavesdropping, and we have to be specially concerned when we are using 3rd party equipment for connecting to our Web Mail Server. Deep Packet Inspection firewalls encrypt and re-encrypt SSL traffic in real-time, and can be configured to transfer a copy of the decrypted stream to anywhere. There are a lot of models and suppliers in the market. And we could even setup our own transparent TLS proxy using Squid.
The company you are working for, may have installed it, the hotel, the LAN-house, the public wireless provider may have it, perhaps the NSA, who knows? With this kind of equipment in the background, when using clients trusting the firewall’s certificate re-signing authority, your connection to your Web Mail Server at home appears non-encrypted to the operator of that transparent DPI-SSL firewall or proxy system.
We need to be aware of this, in order to evaluate the risks. My conclusion is, that I can live with somebody is occasionally eavesdropping me communicating to my Web Mail Account on my Home Mail Server. They would see the headers and the messages that I open. My main concern is about the user credentials of my IMAP account. The default setup of most Web Mail Clients (Roundcube among these) sends the credentials by a simple html form, and therefore these must appear non-encrypted for the operator of a DPI-SSL firewall. Having my credentials, anybody could enter my account and read and modify anything to his/her willing.
We have to go an extra mile for securing the credentials. So, below I show, how to install Roundcube Web Mail together with HTTP Digest Authentication. With that in place, an occasional eavesdropper would see some headers, and the messages that are opened and sent during a session, but the credentials are yet safe. For cases we need 100 % secrecy, we must not use Web Mail with foreign equipment in untrusted environments.
Installation of Roundcube Web Mail
Roundcube Web Mail requires a web server, PHP + a SQL database backend in operational state. I install Apache 2.4, PHP and the PHP module separately.
pkg install apache24
I configure PHP to build the CLI only and both, PHP and the PHP module with thread safety and without DTrace and IPv6.
cd /usr/ports/lang/php
make config
make install clean
cd /usr/ports/www/mod_php72
make config
make install clean
In addition a bunch of PHP extensions need to be installed:
cd `ls -1d /usr/ports/*/php72-mbstring` ; make clean ; make install clean
cd `ls -1d /usr/ports/*/php72-session` ; make clean ; make install clean
cd `ls -1d /usr/ports/*/php72-iconv` ; make clean ; make install clean
cd `ls -1d /usr/ports/*/php72-dom` ; make clean ; make install clean
cd `ls -1d /usr/ports/*/php72-xml` ; make clean ; make install clean
cd `ls -1d /usr/ports/*/php72-json` ; make clean ; make install clean
cd `ls -1d /usr/ports/*/php72-intl` ; make clean ; make install clean
cd `ls -1d /usr/ports/*/php72-zip` ; make clean ; make install clean
cd `ls -1d /usr/ports/*/php72-openssl` ; make clean ; make install clean
cd `ls -1d /usr/ports/*/php72-fileinfo` ; make clean ; make install clean
cd `ls -1d /usr/ports/*/php72-exif` ; make clean ; make install clean
cd `ls -1d /usr/ports/*/php72-pdo` ; make clean ; make install clean
cd `ls -1d /usr/ports/*/php72-pdo_sqlite` ; make clean ; make install clean
cd `ls -1d /usr/ports/*/php72-filter` ; make clean ; make install clean
Prepare the file /usr/local/etc/php.ini and here it is important to set the correct time zone of the web mail service - mine is America/Sao_Paulo:
cat /usr/local/etc/php.ini-production | sed "s|;date.timezone =|date.timezone = America/Sao_Paulo|" > /usr/local/etc/php.ini
Having Apache, the PHP module and the extensions in place, the Roundcube Web Mail package can be installed:
cd /usr/ports/mail/roundcube
make config
I choose the SQLite database backend, because as the name suggests it is lightweight and it is completely adequate with respect to the needed functionality.
make install clean
Configuration of the Apache Web Server
For the web mail facility, the following content must be placed into the virtual host configuration file /usr/local/etc/apache24/Includes/mail.example.com.conf:
LoadModule auth_digest_module libexec/apache24/mod_auth_digest.so
LoadModule deflate_module libexec/apache24/mod_deflate.so
LoadModule ssl_module libexec/apache24/mod_ssl.so
LoadModule socache_shmcb_module libexec/apache24/mod_socache_shmcb.so
SetOutputFilter DEFLATE
SetEnvIfNoCase Request_URI "\.(?:gif|jpe?g|png)$" no-gzip
Listen 443
SSLProtocol All -SSLv2 -SSLv3 -TLSv1
SSLCipherSuite HIGH:!aNULL:!RSA:!AES128:!SSLv2:!SSLv3:!TLSv1
SSLHonorCipherOrder on
SSLPassPhraseDialog builtin
SSLSessionCache "shmcb:/var/run/ssl_scache(512000)"
SSLSessionCacheTimeout 300
<VirtualHost *:80>
ServerName mail.example.com:80
RedirectPermanent / https://mail.example.com/
</VirtualHost>
<VirtualHost *:443>
ServerName mail.example.com:443
ServerAdmin admin@example.com
DocumentRoot "/usr/local/www/roundcube"
<Directory "/usr/local/www/roundcube">
AuthType Digest
AuthDigestProvider file
AuthUserFile "etc/apache24/MailUsers.passwd"
AuthName example.com
AuthDigestDomain /
Require valid-user
RedirectMatch 404 "/\.(_|ht|DS_])"
RedirectMatch 404 "^/(bin|logs|temp|config|vendor|SQL|installer)/"
RedirectMatch 404 "\.c$"
DirectoryIndex index.php index.html
AddType application/x-httpd-php .php
</Directory>
SSLEngine on
SSLCertificateFile "etc/letsencrypt/live/example.com/fullchain.pem"
SSLCertificateKeyFile "etc/letsencrypt/live/example.com/privkey.pem"
</VirtualHost>
Just another password file needs to be created. This one is needed for the Digest Authentication for the web mail application, which is controlled by the Apache web server. The following creates the file, because of the option -c, and adds the first user credentials:
htdigest -c /usr/local/etc/apache24/MailUsers.passwd example.com me@example.com
More users may be added with the same command without the option -c:
htdigest /usr/local/etc/apache24/MailUsers.passwd example.com she@example.com
Configuration of Roundcube Web Mail
For the sake of brevity here comes the content of the configuration file, which needs to be placed to /usr/local/www/roundcube/config/config.inc.php:
<?php
error_reporting(E_ALL & ~E_DEPRECATED & ~E_NOTICE);
$config['db_dsnw'] = 'sqlite:////usr/local/sqlite/roundcube.sql?mode=0646';
$config['log_driver'] = 'syslog';
$config['syslog_facility'] = LOG_LOCAL1;
$config['default_host'] = '127.0.0.1';
$config['smtp_server'] = '127.0.0.1';
$config['smtp_port'] = 10025;
$config['support_url'] = 'mailto:admin@example.com';
$config['skin'] = 'larry';
$config['skin_logo'] = array('*[favicon]' => 'favicon.ico', '*' => 'mail.example.com.png');
$config['ip_check'] = true;
$config['product_name'] = 'Example Web Mail';
$config['identities_level'] = 3;
$config['sent_mbox'] = 'Sent Messages';
$config['trash_mbox'] = 'Deleted Messages';
$config['default_folders'] = array('INBOX', 'Drafts', 'Sent Messages', 'Junk', 'Deleted Messages');
$config['enable_spellcheck'] = false;
$config['draft_autosave'] = 0;
$config['mime_param_folding'] = 0;
$config['mdn_requests'] = 2;
$config['login_lc'] = 0;
$config['default_charset'] = 'UTF-8';
$config['password_charset'] = 'UTF-8';
$config['mime_types'] = '/usr/local/etc/apache24/mime.types';
$config['plugins'] = array('digest_authentication');
$config['cipher_method'] = 'AES-256-CBC';
$config['des_key'] = '<A_RANDOM_KEY>';
As before, replace any occurence of example.com by your respective domain name. in addition generate a random key to be inserted into the config file:
/usr/bin/sed -e "s|\['des_key'\] = '.*';|\['des_key'\] = '`/usr/bin/openssl rand -base64 32`';|" -i "" /usr/local/www/roundcube/config/config.inc.php
I added the very same command to /etc/rc.local, and this would re-generate said key upon every restart of the server.
Initialize the SQLite database which would store the settings of the web mail users - on the sqlite3 prompt, enter .quit:
mkdir -p /usr/local/sqlite/
sqlite3 -init /usr/local/www/roundcube/SQL/sqlite.initial.sql /usr/local/sqlite/roundcube.sql
chown -R www:www /usr/local/sqlite
First Start and Testing of the Web Mail System
In file /etc/rc.conf enable Apache:
...
apache24_enable="YES"
Then start the web server:
service apache24 start
Point your browser to the configured Web Mail Domain, and enter the credentials, For now this needs to be done 2 times, one time for authentication with Apache and another time for authentication with Roundcube. Before going the extra mile of enabling HTTP Digest Authentication for Roundcube (s. below), test the installation. Try to browse the directories of the IMAP account, open e-mails, write a test e-mail and send it to an external address, etc.
Roundcube with HTTP Digest Authentication
Upon login on a standard Roundcube Web Mail service, the Roundcube server receives the users credentials, i.e. the login-id and password in clear text, and it needs the clear password for being able to locally login to the dovecot IMAP service.
Above we created four files with credentials for the different mailer subsystems:
-
a file with the credentials for CRAM-MD5-authentication into the IMAP/POP3/SMTP services, which is controlled by Dovecot,
-
a file for informing Postfix about its local recipients, which it would accept to receive mail for,
-
a file with regular expressions for letting Postfix collect e-mails to personalized and one general catchall e-mail addresses,
-
a file with credentials in Digest-MD5 format, for the purpose of HTTP Digest authentication controlled by Apache.
Now Roundcube needs somehow to lookup the non-encrypted password for a given user who logged-in by the HTTP Digest method. Let's remember, that the whole purpose of HTTP Digest Authentication is to prevent plaintext password exchange over the connection. This means the plaintext password must not come from the client side, but must be looked-up somehow by the server using its user/password store. So we would need just another store with plaintext passwords.
In case we want our Postfix relay e-mails for users who don’t have a mail account, we would need one more file with credentials for CRAM-MD5-authentication.
By all these requirements, the manual management of the Virtual Mail User Store would become complicated, and the solution presented here, is to maintain the Virtual User information in one master file, and a program derives from that one the files necessary for providing the user credentials in the required formats for authentication methods of the different subsystems:
-
credentials for CRAM-MD5 authentication to IMAP/POP3 accounts, i.e.
imap_virtual_users
-
credentials for CRAM-MD5 authentication to the outgoing SMTP service - the authentication is managed by Dovecot’s SASL service, hence:
sasl_virtual_passwords
-
the lookup table for
postfix knows about its virtual users, for whom it wants to accept incoming mails.: smtp_virtual_users.db
-
the regular expressions for collection of personalized catchall e-mail-addresses and one general catchall address:
smtp_virtual_catchall.pcre
-
credentials for Digest-MD5 authentication to the Web Mail service controlled by the Appache web server:
http_md5_digest
-
the lookup table where Roundcube may pick from the plaintext password for a user who has been logged in to Apache by HTTP Digest Authentication:
roundcube_pw_map.db
The master file looks like follows - replace the blue mail addresses and the purple passphrases by the actual credentials („...“ means, continue alike, the 3 dots must not be taken literally):
# login-id plain passphrase uid gid
user1@example.com synb c1z t3z 10001 10000
user2@example.com bb q2y yV7zl 10002 10000
...
# SASL authentication only, i.e. no local mail box (uid == gid)
relay1@example.com 62xLj dfq oh 10000 10000
...
The following command line utility produces automatically all required files mentioned above:
// vumap.c
// vumap
//
// Compilation:
// clang -g0 -O3 -march=native vumap.c -I/usr/local/include -L/usr/local/lib -Wno-parentheses -lcrypto -s -o ~/bin/vumap
//
// Created by Dr. Rolf Jansen 2013-10-27.
// Copyright © 2013-2018 obsigna.com. All rights reserved.
#include <stdlib.h>
#include <stdio.h>
#include <stdbool.h>
#include <stdint.h>
#include <string.h>
#include <errno.h>
#include <unistd.h>
#include <sys/stat.h>
#include <sys/wait.h>
#include <openssl/md5.h>
#define no_error 0
typedef uint8_t MD5_BinDigest[MD5_DIGEST_LENGTH];
typedef uint8_t MD5_HexDigest[MD5_DIGEST_LENGTH*2 + 1];
typedef uint8_t HMAC_iMD5_Hex[MD5_DIGEST_LENGTH*4 + 1];
void convert2Hex(uint8_t *bin, uint8_t *hex, uint16_t len)
{
uint8_t c;
uint16_t i, j;
for (i = 0, j = 0; i < len; i++)
{
c = (bin[i] >> 4) & 0xF;
hex[j++] = (c <= 9) ? (c + '0') : (c + 'a' - 10);
c = bin[i] & 0xF;
hex[j++] = (c <= 9) ? (c + '0') : (c + 'a' - 10);
}
hex[j] = '\0';
}
void md5(const char *data, MD5_HexDigest hex)
{
MD5_BinDigest bin;
MD5_CTX context;
MD5_Init(&context);
data = (data) ?: "";
MD5_Update(&context, (const uint8_t *)data, strlen(data));
MD5_Final(bin, &context);
convert2Hex(bin, hex, MD5_DIGEST_LENGTH);
}
void hmac_md5_intermediate(const char *data, HMAC_iMD5_Hex hex)
{
MD5_CTX ctx;
uint8_t opad[64] = {};
uint8_t ipad[64] = {};
size_t i, n = strlen(data);
if (n <= 64)
{
memcpy(opad, data, n);
memcpy(ipad, data, n);
}
else
{
MD5_Init(&ctx);
MD5_Update(&ctx, (const uint8_t *)data, n);
MD5_Final(opad, &ctx);
memcpy(ipad, opad, MD5_DIGEST_LENGTH);
}
for (i = 0; i < 64; i++)
{
opad[i] ^= 0x5c;
ipad[i] ^= 0x36;
}
MD5_Init(&ctx);
MD5_Update(&ctx, opad, 64);
convert2Hex((uint8_t *)&ctx, hex, MD5_DIGEST_LENGTH);
MD5_Init(&ctx);
MD5_Update(&ctx, ipad, 64);
convert2Hex((uint8_t *)&ctx, hex+MD5_DIGEST_LENGTH*2, MD5_DIGEST_LENGTH);
}
static inline int linelen(const char *line)
{
if (!line || !*line)
return 0;
int l;
for (l = 0; line[l] && line[l] != '\n'; l++)
;
return l;
}
int main(int argc, const char *argv[])
{
if (argc != 3)
{
printf("Usage:\n"\
" vumap <virtual users listing> <virtusers db location>\n\n"\
"Example:\n"\
" vumap ~/config/virtusers /var/mail/users/virtusers.d\n");
return 1;
}
bool mapOK = false;
size_t prefixLen = strlen(argv[2]);
struct stat st;
FILE *tb, *dg, *rc, *su, *ca, *iu, *sp;
char *dgname, *rcname = NULL, *suname = NULL, *caname, *iuname, *spname;
if ((stat(argv[2], &st) == no_error
|| mkdir(argv[2], 0750) == no_error
&& stat(argv[2], &st) == no_error)
&& S_ISDIR(st.st_mode))
{
umask(0127);
if (stat(argv[1], &st) == no_error
&& (tb = fopen(argv[1], "r")))
{
dgname = strcat(strcpy(alloca(prefixLen+sizeof("/http_md5_digest")), argv[2]), "/http_md5_digest");
if (dg = fopen(dgname, "w"))
{
rcname = strcat(strcpy(alloca(prefixLen+sizeof("/roundcube_pw_map")), argv[2]), "/roundcube_pw_map");
if (rc = fopen(rcname, "w"))
{
suname = strcat(strcpy(alloca(prefixLen+sizeof("/smtp_virtual_users")), argv[2]), "/smtp_virtual_users");
if (su = fopen(suname, "w"))
{
caname = strcat(strcpy(alloca(prefixLen+sizeof("/smtp_virtual_catchall.pcre")), argv[2]), "/smtp_virtual_catchall.pcre");
if (ca = fopen(caname, "w"))
{
iuname = strcat(strcpy(alloca(prefixLen+sizeof("/imap_virtual_users")), argv[2]), "/imap_virtual_users");
if (iu = fopen(iuname, "w"))
{
spname = strcat(strcpy(alloca(prefixLen+sizeof("/sasl_virtual_passwords")), argv[2]), "/sasl_virtual_passwords");
if (sp = fopen(spname, "w"))
{
char *inbuf;
if (inbuf = malloc(st.st_size+1))
{
if (fread(inbuf, st.st_size, 1, tb) == 1)
{
inbuf[st.st_size] = '\0';
MD5_HexDigest md5hex;
HMAC_iMD5_Hex itmhex;
char c, outbuf[4096];
char *line, *next = inbuf;
// for each domain we consider the first listed user to be the general catchall user
int k, gencnt = 0,
gencap = 16;
char **genall = malloc(gencap*2*sizeof(char *));
fprintf(ca, "# SMTP Personalized Catchall regular expressions\n");
while (*(line = next))
{
size_t i, n = linelen(line); next = line+n+1;
if (*line != '#' && *line != '\r' && *line != '\n')
{
// extract the fields, the format is:
// login-id plain passphrase uid gid
// strip trailing white space and cr/lf from gid
for (i = n - 1; i > 0 && (uint8_t)line[i] <= ' '; i--);
line[i+1] = '\0';
// extract gid
for (; i > 0 && (uint8_t)line[i] > ' '; i--);
char *gid = line + i + 1;
// strip trailing white space from uid
for (; i > 0 && (uint8_t)line[i] <= ' '; i--);
line[i+1] = '\0';
// extract uid
for (; i > 0 && (uint8_t)line[i] > ' '; i--);
char *uid = line + i + 1;
// strip trailing white space from the pass phrase
for (; i > 0 && (uint8_t)line[i] <= ' '; i--);
line[n = i+1] = '\0';
// strip initial white space of the line
for (i = 0; i < n && (uint8_t)line[i] <= ' '; i++);
// extract login-id & realm
char *loginid = line+i, *realm = loginid;
for (; i < n && (uint8_t)(c = line[i]) > ' '; i++)
if (c == '@')
realm = loginid+i+1;
line[i++] = '\0';
for (k = 0; k < gencnt; k += 2)
if (strcmp(realm, genall[k]) == 0)
goto found;
if (gencnt == gencap)
genall = realloc(genall, (gencap += 16)*2*sizeof(char *));
genall[gencnt++] = realm;
genall[gencnt++] = loginid;
found:
// strip trailing white space from the login-id
for (; i < n && (uint8_t)line[i] <= ' '; i++);
// extract the pass phrase
char *passphrase = line + i;
// write out the fields, by ommiting and converting as necessary
// for the IMAP/POP3 SASL tables, generate the joined outer+inner intermediate MD5 states as the CRAM-MD5 password hash
hmac_md5_intermediate(passphrase, itmhex);
// IMAP/POP3 virtual passwords file
fprintf(sp, "%s:{CRAM-MD5}%s:%s:%s:\n", loginid, itmhex, uid, gid);
if (strtol(uid, NULL, 10) > strtol(gid, NULL, 10))
{
// IMAP/POP3 virtual users file
fprintf(iu, "%s:{CRAM-MD5}%s:%s:%s:\n", loginid, itmhex, uid, gid);
// SMTP Personalized Catchall regular expressions
fprintf(ca, "/^.*%s$/\t%s\n", loginid, loginid);
// input file for the SMTP Virtual User map
fprintf(su, "%s %s\n", loginid, loginid);
// input file for the Roundcube Password Map (includes clear text passwords)
fprintf(rc, "%s %s\n", loginid, passphrase);
// HTTP MD5-Digest
snprintf(outbuf, 4095, "%s:%s:%s", loginid, realm, passphrase);
md5(outbuf, md5hex);
fprintf(dg, "%s:%s:%s\n", loginid, realm, md5hex);
// add the short user name, i.e. with out the @domain (presumably the realm) part
char *r;
if ((r = strstr(loginid, realm)) && r[-1] == '@')
{
r[-1] = '\0';
snprintf(outbuf, 4095, "%s:%s:%s", loginid, realm, passphrase);
md5(outbuf, md5hex);
fprintf(dg, "%s:%s:%s\n", loginid, realm, md5hex);
r[-1] = '@';
}
}
}
}
fprintf(ca, "\n# SMTP General Catchall regular expressions\n");
for (k = 0; k < gencnt; k += 2)
fprintf(ca, "/^.+@%s$/\t%s\n", genall[k], genall[k+1]);
free(genall);
}
else
printf("Error: Could not read the Virtual Users Master file input file. %d\n", errno);
free(inbuf);
}
else
printf("Error: Could not allocate the memory for reading the Virtual Users Master file input file. %d\n", errno);
fclose(sp);
}
else
printf("Error: The IMAP/POP3 Virtual Passords table could not be opened for writing. %d\n", errno);
fclose(iu);
}
else
printf("Error: The IMAP/POP3 Virtual Users table could not be opened for writing. %d\n", errno);
fclose(ca);
}
else
printf("Error: The SMTP Virtual Catchall Regular Expressions could not be opened for writing. %d\n", errno);
fclose(su);
}
else
printf("Error: The SMTP Virtual User table could not be opened for writing. %d\n", errno);
fclose(rc);
mapOK = true;
}
else
printf("Error: The Plaintext Password Map could not be opened for writing. %d\n", errno);
fclose(dg);
}
else
printf("Error: The HTTP MD5 Digest file could not be opened for writing. %d\n", errno);
fclose(tb);
}
else
printf("Error: The Canonical User table could not be opened for reading. %d\n", errno);
}
else
printf("Error: The target directory does not exist and could not be created. %d\n", errno);
if (mapOK && rcname && suname)
{
if (fork() == 0)
execl("/usr/local/sbin/postmap", "/usr/local/sbin/postmap", rcname, NULL);
else
{
int status;
wait(&status);
unlink(rcname);
if (fork() == 0)
execl("/usr/local/sbin/postmap", "/usr/local/sbin/postmap", suname, NULL);
else
{
int status;
wait(&status);
unlink(suname);
}
}
}
return 0;
}
The source code of the above vumap utility is part of the Digest Authentication plugin for Roundcube, which may be downloaded from my site at https://obsigna.com/Downloads/digest_authentication.zip.
On the home mail server, login as the root user, and then do the following:
fetch https://obsigna.com/Downloads/digest_authentication.zip
unzip -d /usr/local/www/roundcube/plugins digest_authentication.zip
rm digest_authentication.zip
Now compile the source code of vumap:
clang -g0 -O3 -march=native -I/usr/local/include -L/usr/local/lib -Wno-parentheses -lcrypto /usr/local/www/roundcube/plugins/digest_authentication/vumap.c -s -o /usr/local/bin/vumap
Create the virtual users store for your mail service, and map the virtual users which have been informed by the way of the above master file into the various files with credentials for the mailer subsystems:
mkdir -m 0550 -p /var/mail/users/virtusers.d
vumap /path/to/the/virtuser/master /var/mail/users/virtusers.d
chown -R dovecot /var/mail/users/virtusers.d
chmod -R u-w /var/mail/users/virtusers.d
pw groupmod virtmail -m www
ls -l /var/mail/users/virtusers.d
>>>>>
total 208
-r--r----- 1 dovecot virtmail 219 Oct 15 00:37 http_md5_digest
-r--r----- 1 dovecot virtmail 207 Oct 15 00:37 imap_virtual_users
-r--r----- 1 dovecot virtmail 131072 Oct 15 00:37 roundcube_pw_map.db
-r--r----- 1 dovecot virtmail 313 Oct 15 00:37 sasl_virtual_passwords
-r--r----- 1 dovecot virtmail 110 Oct 15 00:37 smtp_virtual_catchall.pcre
-r--r----- 1 dovecot virtmail 131072 Oct 15 00:37 smtp_virtual_users.db
Final Steps
The last thing which needs to be done, is to replace in the various configuration files the old locations of the virtual mail user credentials by the new ones:
1. Dovecot configuration /usr/local/etc/dovecot/dovecot.conf:
auth_mechanisms = cram-md5
first_valid_uid = 10000
first_valid_gid = 10000
mail_location = maildir:/var/mail/users/%u
userdb {
args = username_format=%u /var/mail/users/virtusers.d/imap_virtual_users
driver = passwd-file
}
passdb {
args = username_format=%u /var/mail/users/virtusers.d/sasl_virtual_passwords
driver = passwd-file
}
namespace inbox {
inbox = yes
location =
mailbox Drafts {
special_use = \Drafts
}
mailbox Junk {
special_use = \Junk
}
mailbox "Sent Messages" {
special_use = \Sent
}
mailbox "Deleted Messages" {
special_use = \Trash
}
prefix =
}
service auth {
unix_listener /var/spool/postfix/private/auth {
user = postfix
group = postfix
mode = 0666
}
}
service lmtp {
unix_listener /var/spool/postfix/private/dovecot-lmtp {
user = postfix
group = postfix
mode = 0660
}
}
syslog_facility = local1
ssl_cipher_list = HIGH:!aNULL:!AES128:!SSLv2
ssl_cert = </usr/local/etc/letsencrypt/live/example.com/fullchain.pem
ssl_key = </usr/local/etc/letsencrypt/live/example.com/privkey.pem
ssl_dh = </usr/local/etc/dovecot/dh.pem
2. Postfix configuration /usr/local/etc/postfix/main.cf:
# INTERNET HOST AND DOMAIN NAMES
mydomain = example.com
myhostname = mail.$mydomain
myorigin = $mydomain
mydestination = $mydomain, $myhostname, localhost
# RECEIVE MAILS FOR KNOWN USERS AND ALIASES ONLY
virtual_alias_maps = hash:/var/mail/users/virtusers.d/smtp_virtual_users
pcre:/var/mail/users/virtusers.d/smtp_virtual_catchall.pcre
local_recipient_maps =
# TRUST AND RELAY CONTROL
mynetworks = 127.0.0.1/32
# DELIVERY TO MAILBOX
mailbox_transport = lmtp:unix:private/dovecot-lmtp
# INCOMING MAIL
disable_vrfy_command = yes
smtpd_client_restrictions = reject_unauth_pipelining,
permit
smtpd_helo_required = yes
unknown_hostname_reject_code = 550
smtpd_helo_restrictions = permit_mynetworks,
reject_unknown_helo_hostname,
check_helo_access hash:/usr/local/etc/postfix/helo_reject_domains,
permit
smtpd_discard_ehlo_keywords = silent-discard, dsn
smtpd_sender_restrictions = reject_unknown_sender_domain,
reject_non_fqdn_sender,
check_sender_access hash:/usr/local/etc/postfix/sender_domain_reject,
check_sender_access pcre:/usr/local/etc/postfix/sender_access.pcre,
permit
smtpd_etrn_restrictions = permit_mynetworks,
reject
smtpd_sasl_auth_enable = no
smtpd_sasl_type = dovecot
smtpd_sasl_path = private/auth
smtpd_relay_restrictions = permit_mynetworks,
permit_sasl_authenticated,
reject_unauth_destination
smtpd_recipient_restrictions = permit_mynetworks,
permit_sasl_authenticated,
check_policy_service unix:private/greyfix,
check_recipient_access hash:/usr/local/etc/postfix/invalid_recipients,
permit
header_checks = pcre:/usr/local/etc/postfix/header_checks.pcre
message_size_limit = 104857600
mailbox_size_limit = 209715200
# RATE THROTTLING
smtpd_soft_error_limit = 3
smtpd_hard_error_limit = 5
smtpd_client_auth_rate_limit = 11
smtpd_client_connection_rate_limit = 13
smtpd_error_sleep_time = 17s
# TLS ADJUSTMENTS
tls_high_cipherlist = HIGH:!aNULL:!AES128:!SSLv2
tls_preempt_cipherlist = yes
smtpd_tls_security_level = may
smtpd_tls_received_header = yes
smtpd_tls_mandatory_ciphers = high
smtpd_tls_mandatory_exclude_ciphers = aNULL
smtpd_tls_mandatory_protocols = !SSLv2, !SSLv3
smtpd_tls_eecdh_grade = strong
smtpd_tls_dh1024_param_file = /root/certdir/dh2048.pem
smtpd_tls_dh512_param_file = /root/certdir/dh1024.pem
smtpd_tls_cert_file = /usr/local/etc/letsencrypt/live/example.com/fullchain.pem
smtpd_tls_key_file = /usr/local/etc/letsencrypt/live/example.com/privkey.pem
# OUTGOING MAIL
queue_run_delay = 120s
minimal_backoff_time = 120s
maximal_backoff_time = 180s
maximal_queue_lifetime = 24h
bounce_queue_lifetime = 24h
relayhost = [smtp.maildomainhoster.net]:submission
smtp_sasl_auth_enable = yes
smtp_sasl_password_maps = hash:/usr/local/etc/postfix/relay-sasl-password
smtp_tls_security_level = encrypt
smtp_tls_mandatory_ciphers = high
smtp_tls_mandatory_exclude_ciphers = aNULL
smtp_tls_mandatory_protocols = !SSLv2, !SSLv3
# POSTFIX LOCATIONS, USER, GROUP, AND PROTOCOL
readme_directory = /usr/local/share/doc/postfix
sample_directory = /usr/local/etc/postfix
sendmail_path = /usr/local/sbin/sendmail
html_directory = /usr/local/share/doc/postfix
setgid_group = maildrop
command_directory = /usr/local/sbin
manpage_directory = /usr/local/man
daemon_directory = /usr/local/libexec/postfix
newaliases_path = /usr/local/bin/newaliases
mailq_path = /usr/local/bin/mailq
queue_directory = /var/spool/postfix
mail_owner = postfix
data_directory = /var/db/postfix
inet_protocols = ipv4
compatibility_level = 2
3. Postfix daemons configuration /usr/local/etc/postfix/master.cf:
Insert entries for the web mail’s submission (webmission) and the greyfix services:
# CUSTOM ADDITIONS
127.0.0.1:10025 inet n - n - - smtpd
-o syslog_name=postfix/webmission
-o smtpd_sasl_auth_enable=yes
-o receive_override_options=no_header_body_checks
#
greyfix unix - n n - - spawn
user=nobody argv=/usr/local/sbin/greyfix -/ 16 -g 60 -r {DEFER_IF_PERMIT Please try again in %d s} -G {PREPEND X-Greylisting: deferred for %d s}
4. Apache virtual host file /usr/local/etc/apache24/Includes/mail.example.com.conf:
LoadModule auth_digest_module libexec/apache24/mod_auth_digest.so
LoadModule deflate_module libexec/apache24/mod_deflate.so
LoadModule ssl_module libexec/apache24/mod_ssl.so
LoadModule socache_shmcb_module libexec/apache24/mod_socache_shmcb.so
SetOutputFilter DEFLATE
SetEnvIfNoCase Request_URI "\.(?:gif|jpe?g|png)$" no-gzip
Listen 443
SSLProtocol All -SSLv2 -SSLv3 -TLSv1
SSLCipherSuite HIGH:!aNULL:!RSA:!AES128:!SSLv2:!SSLv3:!TLSv1
SSLHonorCipherOrder on
SSLPassPhraseDialog builtin
SSLSessionCache "shmcb:/var/run/ssl_scache(512000)"
SSLSessionCacheTimeout 300
<VirtualHost *:80>
ServerName mail.example.com:80
RedirectPermanent / https://mail.example.com/
</VirtualHost>
<VirtualHost *:443>
ServerName mail.example.com:443
ServerAdmin admin@example.com
DocumentRoot "/usr/local/www/roundcube"
<Directory "/usr/local/www/roundcube">
AuthType Digest
AuthDigestProvider file
AuthUserFile "/var/mail/users/virtusers.d/http_md5_digest"
AuthName example.com
AuthDigestDomain /
Require valid-user
RedirectMatch 404 "/\.(_|ht|DS_])"
RedirectMatch 404 "^/(bin|logs|temp|config|vendor|SQL|installer)/"
RedirectMatch 404 "\.c$"
DirectoryIndex index.php index.html
AddType application/x-httpd-php .php
</Directory>
SSLEngine on
SSLCertificateFile "etc/letsencrypt/live/example.com/fullchain.pem"
SSLCertificateKeyFile "etc/letsencrypt/live/example.com/privkey.pem"
</VirtualHost>
Restart the services:
service dovecot restart
service postfix restart
service apache24 restart
That’s it.
Copyright © Dr. Rolf Jansen - 2018-10-16 18:49:58
Discussion on Twitter: 1082831774319824896
|
