Get rid of hard coded IPs (#441)

Replace them with:
* containers' hostnames
* gateway's IP address for default SMTP_HOST
* Core: when SMTP_HOST undef, echo the default value
* docker-compose: add port mapping
* docker-compose: drop network driver conf
* Exim macro: use localhost as LMTP host
* Update README.md
* docker-compose: Limit port mapping to loopback
* Update other docker-compose example files

README.md

@@ -214,12 +214,12 @@ To configure the mailman-web container to send emails, add this to your
 
 ```
 EMAIL_BACKEND = 'django.core.mail.backends.smtp.EmailBackend'
-EMAIL_HOST = '172.19.199.1'
+EMAIL_HOST = 'smtp.example.com'
 EMAIL_PORT = 25
 ```
 
 Alternatively, you can use the environment variables `SMTP_HOST` (defaults to
-`172.19.199.1`), `SMTP_PORT` (defaults to `25`), `SMTP_HOST_USER` (defaults to
+the container's gateway), `SMTP_PORT` (defaults to `25`), `SMTP_HOST_USER` (defaults to
 an empty string), `SMTP_HOST_PASSWORD` (defaults to an empty string) and
 `SMTP_USE_TLS` (defaults to `False`).
 
@@ -288,7 +288,7 @@ See [the nginx configuration][17] as an example.
 This command will do several things, most importantly:
 
 - Run a wsgi server using [`uwsgi`][7] for the Mailman's Django-based web
-  frontend listening on http://172.19.199.3:8000/. It will run 2 worker
+  frontend listening on port 8000. It will run 2 worker
   processes with 4 threads each. You may want to change the setting
   `ALLOWED_HOSTS` in the settings before deploying the application in
   production.
@@ -297,9 +297,9 @@ This command will do several things, most importantly:
   mentioned in the `docker-compose.yaml`. You will have to change configuration
   files too if you change any of these.
 
-- Run mailman-core listening an LMTP server at http://172.19.199.2:8024/ for
-  messages from your MTA. You will have to configure your MTA to send messages at
-  this address.
+- Run mailman-core listening on port 8001 for REST API and port 8024 (LMTP
+  server) for messages from your MTA. You will have to configure your MTA to
+  send messages at this address.
 
 Some more details about what the above system achieves is mentioned below. If you
 are only going to deploy a simple configuration, you don't need to read
@@ -307,13 +307,12 @@ this. However, these are very easy to understand if you know how docker works.
 
 - First create a bridge network called `mailman` in the
   `docker-compose.yaml`. It will probably be named something else in your
-  machine, but it will use the `172.19.199.0/24` as subnet. All the containers
+  machine. All the containers
   mentioned (mailman-core, mailman-web, database) will join this network and are
-  assigned static IPs. The host operating system is available at `172.19.199.1`
+  assigned static IPs. The host operating system is the default gateway
   from within these containers.
 
-- Spin off a mailman-core container which has a static IP address of
-  `172.19.199.2` in the mailman bridge network created above. It has
+- Spin off a mailman-core container attached to the mailman bridge network created above. It has
   GNU Mailman 3 core running inside it. Mailman core's REST API is available at
   port 8001 and LMTP server listens at port 8024.
 
@@ -354,8 +353,8 @@ The provided docker containers do not have an MTA in-built. You can either run
 your own MTA inside a container and have them relay emails to the mailman-core
 container or just install an MTA on the host and have them relay emails.
 
-To use [Exim4][8], it should be setup to relay emails from `172.19.199.3` and
-`172.19.199.2`. The mailman specific configuration is provided in the
+To use [Exim4][8], it should be setup to relay emails from mailman-core and
+mailman-web. The mailman specific configuration is provided in the
 repository at `core/assets/exim`. There are three files
 
 - [25_mm_macros](core/assets/exim/25_mm3_macros) to be placed at
@@ -387,7 +386,7 @@ configuration: python:mailman.config.exim4
 
 To use [Postfix][12], edit the `main.cf` configuration file, which is typically
 at `/etc/postfix/main.cf` on Debian-based operating systems.  Add
-`172.19.199.2` and `172.19.199.3` to `mynetworks` so it will relay emails from
+mailman-core and mailman-web to `mynetworks` so it will relay emails from
 the containers and add the following configuration lines:
 
 ```
@@ -415,9 +414,11 @@ at `/opt/mailman/core/mailman-extra.cfg`.
 [mta]
 incoming: mailman.mta.postfix.LMTP
 outgoing: mailman.mta.deliver.deliver
-lmtp_host: 172.19.199.2
+# mailman-core hostname or IP from the Postfix server
+lmtp_host: localhost
 lmtp_port: 8024
-smtp_host: 172.19.199.1
+# Postfix server's hostname or IP from mailman-core
+smtp_host: smtp.example.com
 smtp_port: 25
 configuration: /etc/postfix-mailman.cfg
 ```
@@ -472,7 +473,7 @@ It is advisable to run your Django (interfaced through WSGI server) through an
 _actual_ webserver in production for better performance.
 
 If you are using v0.1.0, the uwsgi server is configured to listen to requests at
-`172.19.199.3:8000` using the `HTTP` protocol. Make sure that you preserve the `HOST`
+port `8000` using the `HTTP` protocol. Make sure that you preserve the `HOST`
 header when you proxy the requests from your Web Server. In Nginx you can do
 that by adding the following to your configuration:
 
@@ -485,7 +486,7 @@ that by adding the following to your configuration:
 
 
     location / {
-		  proxy_pass http://172.19.199.3:8000;
+		  proxy_pass http://localhost:8000;
 		  include uwsgi_params;
 		  uwsgi_read_timeout 300;
 		  proxy_set_header Host $host;
@@ -500,7 +501,7 @@ uwsgi
 -----
 
 Starting from v0.1.1, the uwsgi server is configured to listen to requests at
-`172.19.199.3:8000` with the http protocol and `172.19.199.3:8080` for the uwsgi
+port `8000` with the http protocol and port `8080` for the uwsgi
 protocol.
 
 **Please make sure that you are using port 8080 for uwsgi protocol.**
@@ -519,7 +520,7 @@ To move to uwsgi protocol in the above nginx configuration use this
     }
 
     location / {
-		  uwsgi_pass 172.19.199.3:8080;
+		  uwsgi_pass localhost:8080;
 		  include uwsgi_params;
 		  uwsgi_read_timeout 300;
     }

core/README.md

@@ -40,8 +40,8 @@ These are the variables that you MUST change before deploying:
 These are the variables that you don't need to change if you are using a
 standard version of docker-compose.yaml from this repository.
 
-- `MM_HOSTNAME`: Which IP should Core bind to for REST API and LMTP. If not
-  defined output for `hostname -i` command is used.
+- `MM_HOSTNAME`: Which hostname or IP should Core bind to for REST API and
+  LMTP. If not defined output from `hostname` command is used.
 
 - `MAILMAN_REST_PORT`: Which port should Core use for the REST API. If not defined
   the default is `8001`.
@@ -55,7 +55,8 @@ standard version of docker-compose.yaml from this repository.
 - `MTA`: Mail Transfer Agent to use. Either `exim` or `postfix`. Default value is `exim`.
 
 - `SMTP_HOST`: IP Address/hostname from which you will be sending
-  emails. Default value is `172.19.199.1`, which is the address of the Host OS.
+  emails. Default value is the container's gateway retrieved from:
+    /sbin/ip route | awk '/default/ { print $3 }'
 
 - `SMTP_PORT`: Port used for SMTP. Default is `25`.
 

core/assets/exim/25_mm3_macros

@@ -2,7 +2,9 @@
 # /etc/exim4/conf.d/main/25_mm3_macros
 
 domainlist mm3_domains=MY_DOMAIN_NAME
-MM3_LMTP_HOST=172.19.199.2
+# Depending on your network configuration
+#MM3_LMTP_HOST=mailman-core
+MM3_LMTP_HOST=localhost
 MM3_LMTP_PORT=8024
 MM3_HOME=/opt/mailman/core/var
 

core/docker-entrypoint.sh

@@ -27,14 +27,16 @@ function wait_for_mysql () {
 echo "# This file is autogenerated at container startup." > /etc/mailman.cfg
 
 # Check if $MM_HOSTNAME is set, if not, set it to the value returned by
-# `hostname -i` command to set it to whatever IP address is assigned to the
+# `hostname` command to set it to whatever hostname is assigned to the
 # container.
 if [[ ! -v MM_HOSTNAME ]]; then
-	export MM_HOSTNAME=`hostname -i`
+	export MM_HOSTNAME=`hostname`
 fi
 
+# SMTP_HOST defaults to the gateway
 if [[ ! -v SMTP_HOST ]]; then
-	export SMTP_HOST='172.19.199.1'
+	export SMTP_HOST=$(/sbin/ip route | awk '/default/ { print $3 }')
+	echo "SMPT_HOST not specified, using the gateway ($SMTP_HOST) as default"
 fi
 
 if [[ ! -v SMTP_PORT ]]; then

docker-compose-mysql.yaml

@@ -16,9 +16,11 @@ services:
     - DATABASE_TYPE=mysql
     - DATABASE_CLASS=mailman.database.mysql.MySQLDatabase
     - HYPERKITTY_API_KEY=someapikey
+    ports:
+    - "127.0.0.1:8001:8001" # API
+    - "127.0.0.1:8024:8024" # LMTP - incoming emails
     networks:
       mailman:
-        ipv4_address: 172.19.199.2
 
   mailman-web:
     image: maxking/mailman-web:0.3
@@ -37,9 +39,11 @@ services:
     - HYPERKITTY_API_KEY=someapikey
     - SECRET_KEY=thisisaverysecretkey
     - DYLD_LIBRARY_PATH=/usr/local/mysql/lib/
+    ports:
+    - "127.0.0.1:8000:8000" # HTTP
+    - "127.0.0.1:8080:8080" # uwsgi
     networks:
       mailman:
-        ipv4_address: 172.19.199.3
 
   database:
     environment:
@@ -54,13 +58,6 @@ services:
     - /opt/mailman/database:/var/lib/mysql
     networks:
       mailman:
-        ipv4_address: 172.19.199.4
 
 networks:
    mailman:
-     driver: bridge
-     ipam:
-       driver: default
-       config:
-       -
-         subnet: 172.19.199.0/24

docker-compose-postorius.yaml

@@ -16,9 +16,11 @@ services:
     - DATABASE_URL=postgres://mailman:mailmanpass@database/mailmandb
     - DATABASE_TYPE=postgres
     - DATABASE_CLASS=mailman.database.postgresql.PostgreSQLDatabase
+    ports:
+    - "127.0.0.1:8001:8001" # API
+    - "127.0.0.1:8024:8024" # LMTP - incoming emails
     networks:
       mailman:
-        ipv4_address: 172.19.199.2
 
   mailman-web:
     image: maxking/postorius:0.3
@@ -36,9 +38,11 @@ services:
     - DATABASE_URL=postgres://mailman:mailmanpass@database/mailmandb
     - SECRET_KEY=ksjdbaksdba
     - UWSGI_STATIC_MAP=/static=/opt/mailman-web-data/static 
+    ports:
+    - "127.0.0.1:8000:8000" # HTTP
+    - "127.0.0.1:8080:8080" # uwsgi
     networks:
       mailman:
-        ipv4_address: 172.19.199.3
 
   database:
     environment:
@@ -51,13 +55,6 @@ services:
     - /opt/mailman/database:/var/lib/postgresql/data
     networks:
       mailman:
-        ipv4_address: 172.19.199.4
 
 networks:
    mailman:
-     driver: bridge
-     ipam:
-       driver: default
-       config:
-       -
-         subnet: 172.19.199.0/24

docker-compose.yaml

@@ -17,9 +17,11 @@ services:
     - DATABASE_TYPE=postgres
     - DATABASE_CLASS=mailman.database.postgresql.PostgreSQLDatabase
     - HYPERKITTY_API_KEY=someapikey
+    ports:
+    - "127.0.0.1:8001:8001" # API
+    - "127.0.0.1:8024:8024" # LMTP - incoming emails
     networks:
       mailman:
-        ipv4_address: 172.19.199.2
 
   mailman-web:
     image: maxking/mailman-web:0.3
@@ -36,9 +38,11 @@ services:
     - DATABASE_TYPE=postgres
     - DATABASE_URL=postgres://mailman:mailmanpass@database/mailmandb
     - HYPERKITTY_API_KEY=someapikey
+    ports:
+    - "127.0.0.1:8000:8000" # HTTP
+    - "127.0.0.1:8080:8080" # uwsgi
     networks:
       mailman:
-        ipv4_address: 172.19.199.3
 
   database:
     environment:
@@ -50,13 +54,6 @@ services:
     - /opt/mailman/database:/var/lib/postgresql/data
     networks:
       mailman:
-        ipv4_address: 172.19.199.4
 
 networks:
    mailman:
-     driver: bridge
-     ipam:
-       driver: default
-       config:
-       -
-         subnet: 172.19.199.0/24

postorius/docker-entrypoint.sh

@@ -53,6 +53,11 @@ function check_or_create () {
 # END
 # }
 
+# SMTP_HOST defaults to the gateway
+if [[ ! -v SMTP_HOST ]]; then
+	export SMTP_HOST=$(/sbin/ip route | awk '/default/ { print $3 }')
+fi
+
 # Check if $SECRET_KEY is defined, if not, bail out.
 if [[ ! -v SECRET_KEY ]]; then
 	echo "SECRET_KEY is not defined. Aborting."

postorius/mailman-web/settings.py

@@ -29,6 +29,7 @@ https://docs.djangoproject.com/en/1.8/ref/settings/
 import os
 import dj_database_url
 import sys
+from socket import gethostbyname
 
 BASE_DIR = os.path.dirname(os.path.abspath(__file__))
 
@@ -51,7 +52,7 @@ ALLOWED_HOSTS = [
     # "lists.your-domain.org",
     # Add here all production URLs you may have.
     "mailman-web",
-    "172.19.199.3",
+    gethostbyname("mailman-web"),
     os.environ.get('SERVE_FROM_DOMAIN'),
     os.environ.get('DJANGO_ALLOWED_HOSTS'),
 ]
@@ -60,7 +61,7 @@ ALLOWED_HOSTS = [
 MAILMAN_REST_API_URL = os.environ.get('MAILMAN_REST_URL', 'http://mailman-core:8001')
 MAILMAN_REST_API_USER = os.environ.get('MAILMAN_REST_USER', 'restadmin')
 MAILMAN_REST_API_PASS = os.environ.get('MAILMAN_REST_PASSWORD', 'restpass')
-MAILMAN_ARCHIVER_FROM = (os.environ.get('MAILMAN_HOST_IP', '172.19.199.2'),)
+MAILMAN_ARCHIVER_FROM = (os.environ.get('MAILMAN_HOST_IP', gethostbyname(os.environ.get('MAILMAN_HOSTNAME', 'mailman-core'))),)
 
 # Application definition
 
@@ -208,7 +209,7 @@ SERVER_EMAIL = 'root@{}'.format(hostname)
 
 # Change this when you have a real email backend
 EMAIL_BACKEND = 'django.core.mail.backends.smtp.EmailBackend'
-EMAIL_HOST = os.environ.get('SMTP_HOST', '172.19.199.1')
+EMAIL_HOST = os.environ.get('SMTP_HOST', '')
 EMAIL_PORT = os.environ.get('SMTP_PORT', 25)
 EMAIL_HOST_USER = os.environ.get('SMTP_HOST_USER', '')
 EMAIL_HOST_PASSWORD = os.environ.get('SMTP_HOST_PASSWORD', '')

tests/test.sh

@@ -27,10 +27,10 @@ docker logs mailman-core
 
 
 # Check to see if the core is working as expected.
-docker exec mailman-core curl -u restadmin:restpass http://172.19.199.2:8001/3.1/system | grep "GNU Mailman"
+docker exec mailman-core curl -u restadmin:restpass http://mailman-core:8001/3.1/system | grep "GNU Mailman"
 
 # Check to see if postorius is working.
-docker exec mailman-web curl -L http://172.19.199.3:8000/postorius/lists | grep "Mailing List"
+docker exec mailman-web curl -L http://mailman-web:8000/postorius/lists | grep "Mailing List"
 
 # Check to see if hyperkitty is working.
-docker exec  mailman-web  curl -L http://172.19.199.3:8000/hyperkitty/ | grep "Available lists"
+docker exec  mailman-web  curl -L http://mailman-web:8000/hyperkitty/ | grep "Available lists"

web/README.md

@@ -43,11 +43,12 @@ change them unless you know what you want.
 - `MAILMAN_REST_PASSWORD`: Mailman's REST API user's password. Default value is
   `restpass`
 
-- `MAILMAN_HOST_IP`: IP of the Container from which Mailman will send emails to
-  hyperkitty (django). Set to `172.19.199.2` by default.
+- `MAILMAN_HOSTNAME`: IP of the Container from which Mailman will send emails to
+  hyperkitty (django). Set to `mailman-core` by default.
 
 - `SMTP_HOST`: IP Address/hostname from which you will be sending
-  emails. Default value is `172.19.199.1`, which is the address of the Host OS.
+  emails. Default value is the container's gateway retrieved from:
+    /sbin/ip route | awk '/default/ { print $3 }'
 
 - `SMTP_PORT`: Port used for SMTP. Default is `25`.
 

web/docker-entrypoint.sh

@@ -53,6 +53,11 @@ function check_or_create () {
 # END
 # }
 
+# SMTP_HOST defaults to the gateway
+if [[ ! -v SMTP_HOST ]]; then
+	export SMTP_HOST=$(/sbin/ip route | awk '/default/ { print $3 }')
+fi
+
 # Check if $SECRET_KEY is defined, if not, bail out.
 if [[ ! -v SECRET_KEY ]]; then
 	echo "SECRET_KEY is not defined. Aborting."

web/mailman-web/settings.py

@@ -29,6 +29,7 @@ https://docs.djangoproject.com/en/1.8/ref/settings/
 import os
 import dj_database_url
 import sys
+from socket import gethostbyname
 
 BASE_DIR = os.path.dirname(os.path.abspath(__file__))
 
@@ -51,7 +52,7 @@ ALLOWED_HOSTS = [
     # "lists.your-domain.org",
     # Add here all production URLs you may have.
     "mailman-web",
-    "172.19.199.3",
+    gethostbyname("mailman-web"),
     os.environ.get('SERVE_FROM_DOMAIN'),
     os.environ.get('DJANGO_ALLOWED_HOSTS'),
 ]
@@ -61,7 +62,7 @@ MAILMAN_REST_API_URL = os.environ.get('MAILMAN_REST_URL', 'http://mailman-core:8
 MAILMAN_REST_API_USER = os.environ.get('MAILMAN_REST_USER', 'restadmin')
 MAILMAN_REST_API_PASS = os.environ.get('MAILMAN_REST_PASSWORD', 'restpass')
 MAILMAN_ARCHIVER_KEY = os.environ.get('HYPERKITTY_API_KEY')
-MAILMAN_ARCHIVER_FROM = (os.environ.get('MAILMAN_HOST_IP', '172.19.199.2'),)
+MAILMAN_ARCHIVER_FROM = (os.environ.get('MAILMAN_HOST_IP', gethostbyname(os.environ.get('MAILMAN_HOSTNAME', 'mailman-core'))),)
 
 # Application definition
 
@@ -227,7 +228,7 @@ SERVER_EMAIL = 'root@{}'.format(hostname)
 
 # Change this when you have a real email backend
 EMAIL_BACKEND = 'django.core.mail.backends.smtp.EmailBackend'
-EMAIL_HOST = os.environ.get('SMTP_HOST', '172.19.199.1')
+EMAIL_HOST = os.environ.get('SMTP_HOST', '')
 EMAIL_PORT = os.environ.get('SMTP_PORT', 25)
 EMAIL_HOST_USER = os.environ.get('SMTP_HOST_USER', '')
 EMAIL_HOST_PASSWORD = os.environ.get('SMTP_HOST_PASSWORD', '')