22. Install Mailman

by Cover Tower - Updated March 14, 2021

There are two types of workflows associated with mailing lists:

1) that in which you want any list subscriber to be able to send an email to the list’s email address, and that email to be automatically sent to all the other list subscribers;

2) that in which you want to send the same email message directly to thousands of subscribers and receive their responses (if any) on a ‘Reply-To’ email address.

For the first situation, the free and open source application of choice is Mailman, for the second, it’s phpList.

Mailman, officially ‘GNU Mailman’, is a web-based mailing lists manager that provides a web administration interface, while relying on a SMTP server such as Postfix to send and receive emails.

You can install Mailman from the Debian repositories by running:

apt-get install mailman

During the installation, you will be asked to select languages for Mailman. Use the arrow keys to move up and down. Use the space bar to select your language. A star character indicates that the language is selected. Press Enter to use the selected language. Immediately after that, a new window will inform you that you will have to create a new list called mailman. Just press Enter to close the window. You don’t need to create the mailman list at this moment.

Edit the /usr/lib/mailman/Mailman/mm_cfg.py file:

nano /usr/lib/mailman/Mailman/mm_cfg.py

Change/add the following settings, to make them look like this:

DEFAULT_URL_PATTERN = ‘https://%s/mailman/’

DEFAULT_EMAIL_HOST = ‘mailman.example.com’

DEFAULT_URL_HOST = ‘mailman.example.com’

add_virtualhost(DEFAULT_URL_HOST, DEFAULT_EMAIL_HOST)

POSTFIX_STYLE_VIRTUAL_DOMAINS = [‘mailman.example.com’]

MTA=’Postfix’

OWNERS_CAN_DELETE_THEIR_OWN_LISTS = yes

OWNERS_CAN_ENABLE_PERSONALIZATION = 1

Replace example.com with the main domain hosted on your server.

22.1. Configure Postfix

Open the /etc/postfix/main.cf file:

nano /etc/postfix/main.cf

Enable the following lines, by uncommenting them:

relay_domains = mailman.example.com

relay_recipient_maps = hash:/var/lib/mailman/data/virtual-mailman

transport_maps = hash:/etc/postfix/transport

mailman_destination_recipient_limit = 1

virtual_alias_maps = proxy:mysql:/etc/postfix/mysql-virtual-alias-maps.cf,

hash:/var/lib/mailman/data/virtual-mailman

Please note that you have to remove this line:

virtual_alias_maps = proxy:mysql:/etc/postfix/mysql-virtual-alias-maps.cf

since you already added the virtual_alias_maps parameter.

Also open the /etc/postfix/master.cf file:

nano /etc/postfix/master.cf

Uncomment to last two lines of the file, to make them look like this:

mailman unix – n n – – pipe

flags=FR user=list:list argv=/var/lib/mailman/bin/postfix-to-mailman.py ${nexthop} ${recipient}

Create the /etc/postfix/transport file:

nano /etc/postfix/transport

Add the following line inside this file:

mail.example.com mailman:

Replace example.com with the main domain hosted on your server. Please note that the domain is not mailman.example.com, but mail.example.com .

Turn the /etc/postfix/transport file into a Postfix lookup table:

postmap /etc/postfix/transport

Add user postfix to the list group:

adduser postfix list

Restart Postfix:

systemctl restart postfix

22.2. Install FCGIWrap

To properly display the Mailman web interface, we need to install the fcgiwrap package which allows us to run CGI applications with FastCGI and Nginx.

apt-get install fcgiwrap

Once installed, fcgiwrap will start automatically, as can be seen by running:

systemctl status fcgiwrap

Output:

● fcgiwrap.service – Simple CGI Server

Loaded: loaded (/lib/systemd/system/fcgiwrap.service; indirect; vendor preset: enabled)

Active: active (running) since Mon 2020-10-12 01:25:20 EDT; 4min 27s ago

Main PID: 27780 (fcgiwrap)

CGroup: /system.slice/fcgiwrap.service

└─27780 /usr/sbin/fcgiwrap -f

The fcgiwrap service listens on the /var/run/fcigwrap.socket by default.

22.3. Configure Nginx for Mailman

To access Mailman from a sub-directory of the mailman.example.com domain, first create the mailman.example.com directory:

cd /var/www
mkdir mailman.example.com

To obtain a Let’s Encrypt SSL Certificate for mailman.example.com, first edit the Nginx server blocks configuration file:

nano /etc/nginx/sites-enabled/0-conf

Add a temporary server block for mailman.example.com, at the bottom of the file:

server {

listen 80;

listen [::]:80;

server_name mailman.example.com;

location /.well-known/acme-challenge {

root /var/www;

}

}

Restart Nginx:

systemctl restart nginx

Next edit your DNS settings. Add an A and an AAAA entry for mailman.example.com. These entries will be identical to those for mail.example.com. It’s just that instead of mail, you’ll enter mailman . Wait a few minutes until the DNS changes propagate.

You will need to add an MX entry for mailman.example.com, like this:

mailman IN MX 10 mail.example.com.

You will also need to add a SPF record for mailman.example.com like this:

mailman IN TXT “v=spf1 a mx ip4:123.123.123.123 ip6:4c02:7bd0:3:6::8473 -all”

Where 123.123.123.123 is the IPv4 of your server and 4c02:7bd0:3:6::8473 is its IPv6.

To obtain the Let’s Encrypt certificate for mailman.example.com, run:

certbot certonly –agree-tos –webroot -w /var/www/ -d mailman.example.com

Then open the /etc/nginx/sites-enabled/0-conf file:

nano /etc/nginx/sites-enabled/0-conf

Replace the temporary server block for mailman.example.com with the following server blocks:

server {

listen 80;

listen [::]:80;

server_name mailman.example.com;

return 301 https://mailman.example.com$request_uri;

}

server {

listen 443 ssl http2;

listen [::]:443 ssl http2;

server_name mailman.example.com;

root /var/www/mailman.example.com;

index index.html;

ssl_certificate /etc/letsencrypt/live/mailman.example.com/fullchain.pem;

ssl_certificate_key /etc/letsencrypt/live/mailman.example.com/privkey.pem;

ssl_trusted_certificate /etc/letsencrypt/live/mailman.example.com/chain.pem;

ssl_dhparam /etc/nginx/ssl/dhparam.pem;

ssl_session_timeout 4h;

ssl_session_cache shared:SSL:40m;

ssl_protocols TLSv1.2 TLSv1.3;

ssl_prefer_server_ciphers on;

ssl_ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384;

ssl_stapling on;

ssl_stapling_verify on;

add_header Strict-Transport-Security “max-age=63072000” always;

add_header X-Frame-Options SAMEORIGIN;

add_header X-Content-Type-Options nosniff;

location = /robots.txt {

allow all;

}

location / {

try_files $uri $uri/ /index.php?$args;

}

location /.well-known/acme-challenge {

root /var/www;

}

location /mailman {

root /usr/lib/cgi-bin;

fastcgi_pass unix:/var/run/fcgiwrap.socket;

fastcgi_param HTTPS on;

fastcgi_split_path_info (^/mailman/[^/]*)(.*)$;

fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;

fastcgi_param PATH_INFO $fastcgi_path_info;

include /etc/nginx/fastcgi_params;

}

location /mailman/admin {

auth_basic ‘Restricted’;

auth_basic_user_file /etc/nginx/htpass/mailman.example.com;

root /usr/lib/cgi-bin;

fastcgi_pass unix:/var/run/fcgiwrap.socket;

fastcgi_param HTTPS on;

fastcgi_split_path_info (^/mailman/[^/]*)(.*)$;

fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;

fastcgi_param PATH_INFO $fastcgi_path_info;

include /etc/nginx/fastcgi_params;

}

location /images/mailman {

alias /usr/share/images/mailman;

}

location /pipermail {

alias /var/lib/mailman/archives/public;

autoindex on;

}

access_log /var/log/sites/mailman.example.com/access.log;

error_log /var/log/nginx/mailman.example.com.error.log notice;

}

Replace example.com with the main domain hosted on your server.

Create the access log directory:

mkdir /var/log/sites/mailman.example.com

Create the robots.txt file.

cd /var/www/mailman.example.com
nano robots.txt

Add the following lines inside this file:

User-agent: *

Disallow: /mailman

Disallow: /images

In this way all the search engine bots will not crowl and index the mailman.example.com/mailman directory, the mailman.example.com/images directory and their respective subdirectories, but they will crowl and index the mailman.example.com/pipermail directory, containing all the public mailing list archives and their messages.

22.4. Configure logrotate to rotate the Mailman access logs

nano /etc/logrotate.d/nginx

Add the following section at the bottom of this file:

/var/log/sites/mailman.example.com/access.log {

missingok

rotate 10

compress

delaycompress

notifempty

create 0640 www-data adm

size 2M

sharedscripts

prerotate

if [ -d /etc/logrotate.d/httpd-prerotate ]; then \

run-parts /etc/logrotate.d/httpd-prerotate; \

fi; \

endscript

postrotate

[ ! -f /var/run/nginx.pid ] || kill -USR1 `cat /var/run/nginx.pid`

endscript

}

Replace example.com with the main domain hosted on your server.

22.5. Initial configuration

Please note that we have set up HTTP authentication for the ‘mailman/admin’ directory so as to prevent unauthorized access to the Mailman administration page. You’ll have to create the /etc/nginx/htpass/mailman.example.com password file, to allow specific users to access the Mailman administration page. If you want to give access to the user gerald, run:

htpasswd -c /etc/nginx/htpass/mailman.example.com gerald

To add the user verner to the same password file run:

htpasswd /etc/nginx/htpass/mailman.example.com verner

Change ownership and permissions for the password file:

chown www-data:root /etc/nginx/htpass/mailman.example.com
chmod 400 /etc/nginx/htpass/mailman.example.com

Restart Nginx:

systemctl restart nginx

Next, run the following command to create a site list creator password and avoid a future error when creating new lists:

mmsitepass

New site password: strongpassword

Please note that you cannot start Mailman until you create the mailman list. To create it run:

newlist mailman

The output of this command and your answers should be:

Enter the email of the person running the list: admin@example.com

Initial mailman password: strongpassword

Hit enter to notify mailman owner… (Press Enter)

Restart Mailman:

systemctl restart mailman

Now you can access the administration web page of Mailman at:

https://mailman.example.com/mailman/admin

On the administration page you will see a list with the names of all the mailing lists available. Since you have created only one mailing list, ‘mailman’, you will see its name on the list:


If you click on ‘Mailman’ under List, you will be asked to enter the password that you configured earlier when you created the ‘mailman’ list, then you will see the administration page for this list:

Here you can click on an entry in the ‘Configuration Categories’ or ‘Other Administrative Activities’ sections and you can change any setting available for that category or activity.

22.6. Create a new mailing list

To create a new mailing list go to https://mailman.example.com/mailman/admin and click on the ‘create a new mailing list’ link. The new screen will look like this:

In the ‘Name of list’ field enter the name of the new list, let’s say test1. In the ‘Initial list owner address’ field enter the email address of the list owner, let’s say admin@example.com (it can be any email address that you like), in the ‘Initial list password’ field enter a password for this list, confirm the password, then, check a language from the list of languages, in the ‘List creator’s (authentication) password’ field at the bottom of the form, enter the password that you configured earlier, when you created the ‘mailman’ list in command line, then click ‘Create List’. In the next window you will be informed that the new list has been created.

You can change the settings of a list by going to the administration page: https://mailman.example.com/mailman/admin and clicking on the name of the list that you want to configure. Then you can click on an entry in the ‘Configuration Categories’ or ‘Other Administrative Activities’ sections and you can change any setting available for that category or activity.

22.7. Add subscribers to a mailing list

To manually add multiple subscribers to the test1 mailing list that you have just created, first go to the list’s administration page: https://mailman.example.com /mailman/admin/test1 . Here click on ‘Membership Management’ > ‘Mass Subscription’ , then enter one email address per line in the first text box, then click ‘Submit Your Changes’ at the bottom of the page.

However, the typical way in which subscribers are added to a mailing list is by directing them to the list’s subscription page: https://mailman.example.com/mailman/listinfo/test1 :




On this page they can enter their email address and a password. However, it’s better to leave the password field empty, because Mailman will generate a random password and send it in the ‘Welcome’ message. The subscriber can then change the generated password with the password that they desire.

After subscribers click ‘Subscribe’, they receive an email in which they have to click a link to confirm their subscription. Once they confirm their subscription, they receive the ‘Welcome’ message containing the URL of their subscription page:

https://mailman.example.com/mailman/options/test1/username@domain.com

where username@domain.com is the email address that they used to subscribe. That message also contains the email address of the mailing list (name-of-list@mailman.example.com), the URL with general information about the mailing list (https://mailman.example.com/mailman/listinfo/name-of-list) and the subscriber’s password.


On their subscription page, the subscribers can change their email address, they can change their password, they can unsubscribe and they can make other configurations related to the way they want to interact with the mailing list.

The next step for new subscribers should be to go to their subscription page and change their password, in the ‘Change Your Password’ section. Also, under ‘Get password reminder email for this list?‘ they should select ‘No’, then click ‘Submit My Changes’, so that they can avoid receiving an email with a password reminder every month.

22.8. Send an email to a mailing list

Once a person subscribes to a mailing list, they can send messages to all the subscribers of that list by sending the email to the list’s email address: name-of-list@mailman.example.com .

When trying to send an email to a list address, if you ever encounter Postfix errors such as: ‘User not found in local recipients table’ you can simply delete the aliases and virtual-mailman files:

cd /var/lib/mailman/data

rm aliases aliases.db virtual-mailman virtual-mailman.db

then regenerate these files by running:

/usr/lib/mailman/bin/genaliases

Then restart Postfix and Mailman:

systemctl restart postfix

systemctl restart mailman

The administration page is:

https://mailman.example.com/mailman/admin

The page where a user can subscribe to a list is:

https://mailman.example.com/mailman/listinfo/name-of-list

If the archive page of a list is public, the URL of the archive page will be:

https://mailman.example.com/pipermail/name-of-list/

If you want the archive pages to be publicly accessible for a particular list, so that they can be viewed by anyone and indexed by search engines, on the list’s administration page go to ‘Archiving Options’ and verify that ‘Is archive file source for public or private archival?’ is set to ‘public’. Also, as mentioned before, to let search engines index the archive pages, you should make sure that the content of the /var/www/mailman.example.com/robots.txt file looks like this:

User-agent: *
Disallow: /mailman
Disallow: /images

In this way, all the search engine bots will be instructed not to crowl and index the mailman.example.com/mailman directory, the mailman.example.com/images directory and their respective subdirectories, but to crowl and index the mailman.example.com/pipermail directory, containing all the public mailing list archives, with their respective messages.

If on the contrary, you don’t want the archive pages to be publicly accessible, on the list’s administration page, under ‘Archiving Options’, you should set ‘Is archive file source for public or private archival?’ to ‘private’ and click ‘Submit Your Changes’; from that moment on, the archive pages for that respective list will be accessible only after entering the email address and password of a list’s subscriber. For a private archive, the URL will be:

https://mailman.example.com/mailman/private/name-of-list/

If all the mailing lists on your server are private lists, you should also include the following line in the server block for mailman.example.com, in the /etc/nginx/sites-enabled/0-conf file, right above the location = /robots.txt { line:

add_header X-Robots-Tag “noindex, nofollow, nosnippet, noarchive”;

Please note that you will have to send emails in plain textor in both plain text and HTML format to mailing lists, if you want to be sure that all the receivers will be able to view them properly. If you send messages to mailing lists in HTML format, by default, the archived messages will be displayed as ‘scrubbed’ attachments and will contain all the escaped HTML tags, thus becoming impossible to read. The best method is to send emails to mailing lists in both plain text and HTML format. In this way, the receivers that can view them as HTML will view them as such, those who can view plain text only emails will view them as plain text, and the archived messages will contain the proper plain text version, with the HTML version attached as scrubbed content.

If you use Thunderbird to send emails to mailing lists, to make sure that you send them in plain text, you should go to Edit > Preferences > Composition > General tab > click on the ‘Send Options …’ button; uncheck the ‘Send messages as plain text if possible’ box and from the drop-down list under ‘When sending messages in HTML format and one or more recipients are not listed as being able to receive HTML’, choose ‘Send the message in both plain text and HTML’ > click OK.

When replying to messages coming from a mailing list, the subscribers have to click on the ‘Reply List’ button in their email client. If they click on ‘Reply’, they will send their reply just to the subscriber who sent the initial message and not to the other subscribers.

If a subscriber has a badly configured auto-responder that sends “I’m on vacation” replies to all incoming messages, this can trigger a mailing list loop: each automatic reply will be sent back to the mailing list, Mailman will then send it to all the subscribers, the auto-responder will receive another email and will reply again, etc. This can clutter the mailing list very rapidly. Well configured auto-responders will identify that the email comes from a mailing list and will not trigger this loop. However, if it ever happens, the first thing to do is to go to the list’s administration page (https://mailman.example.com/mailman/admin/name-of-list) > ‘Membership Management’ > ‘Membership List’ and there check the ‘mod’ and ‘nomail’ checkboxes next to the email address of the user causing the loop, in order to remove her/his ability to send emails to the list and receive emails from the list respectively, then click on the ‘Submit Your Changes’ button. If you cannot identify the user causing the loop quickly, you can go to the ‘General Options’ and at ‘Emergency moderation of all list traffic’ select ‘Yes’, then click on ‘Submit Your Changes’. This will suspend all email traffic to/from the mailing list. The next step is to inform the user that caused the loop that (s)he has a badly configured auto-responder that needs to be fixed.

22.9. Configure Fail2ban to protect Mailman against brute-force attacks

Navigate to the /etc/fail2ban/filter.d directory:

cd /etc/fail2ban/filter.d

Create a file called mailman.conf:

nano mailman.conf

Add the following content in this file:

[Definition]

failregex = ^<HOST> .* \”POST /mailman/admin/.* HTTP/2.0\” 401 2.*$

^<HOST> .* \”POST /mailman/private/.* HTTP/2.0\” 401 2.*$

^<HOST> .* \”GET /mailman/admin.* HTTP/2.0\” 401 195 .*$

^<HOST> .* \”GET /mailman/private.* HTTP/2.0\” 401 195 .*$

ignoreregex =

Please note that the third and the fourth line in the ‘failregex’ section are used to identify failed log in attempts against the HTTP authentication login window, which can be also subject to brute-force attacks.

Next, open the /etc/fail2ban/jail.local file:

nano /etc/fail2ban/jail.local

Add the following section below the [lighttpd-auth] section:

[mailman]

enabled = true

filter = mailman

logpath = /var/log/sites/mailman.example.com/access.log

port = 80,443

maxretry = 4

bantime = 259200

Replace example.com with the main domain hosted on your server.

Reload Fail2ban:

systemctl reload fail2ban

22.10. Upgrading Mailman

Before upgrading Mailman to a new version, it’s recommended to check if the new version has been tested and confirmed to function well within the suite by visiting this page.

Since the mailman package has been installed from the Debian repositories and it can be included automatically among many other upgrades when executing apt-get dist-upgrade, it is recommended to exclude it from automatic upgrades so as to prevent an upgrade before making a backup of the most important files. So, first exclude mailman from automatic upgrades by running:

apt-mark hold mailman

Then, you can upgrade Mailman manually, after making all the necessary backups.

It’s recommended to run apt-get dist-upgrade periodically, at least once every four weeks. When you run this command, if you see:

The following packages have been kept back:

mailman

The following packages will be upgraded:

this means that there is an upgrade to Mailman that has been held back. So, after upgrading all the other packages, you can upgrade Mailman manually, by following the next steps:

Stop Mailman:

systemctl stop mailman

Go to the backups directory, /var/bm_archives, and make a backup copy of the /var/lib/mailman directory:

cd /var/bm_archives

tar czf var-lib-mailman-2020-10-20.tgz /var/lib/mailman

Replace 2020-10-20 with the date of the backup. Also make a backup copy of the /etc/mailman directory:

tar czf etc-mailman-2020-10-20.tgz /etc/mailman

Upgrade Mailman to the latest version by running:

apt-get install mailman

Start Mailman:

systemctl start mailman

If you want to see all the packages that are excluded from upgrading at a certain moment, run:

dpkg –get-selections | grep hold

If you want to unhold a package run:

apt-mark unhold packagename

You can also join the Mailman-announce mailing list to receive a notification when a new version of mailman has been released. However, if a new version has been officially released, this doesn’t mean that it will be immediately included in the Debian repositories to install it from there. Usually any new package is included in the Debian repositories after a certain period of time necessary for testing.