learning-website-django1-2  Exercise 3: Behind Apache and Gunicorn
=================================================================

In production, Apache ends the HTTPS connection and passes the request to
Gunicorn (the Python application server), which talks to Django over a plain
local connection. Django only sees what the proxy tells it, so three things
must be right: the host name, whether the request was HTTPS, and which headers
to trust. First test Django's side with code, then write the proxy
configuration.

PART A: Test what Django does with proxy headers. Save as tests/test_proxy.py:

from django.test import Client, RequestFactory, SimpleTestCase, override_settings

PROXY_HEADER = ("HTTP_X_FORWARDED_PROTO", "https")


class BehindAProxyTests(SimpleTestCase):
    """Apache ends the HTTPS connection and talks plain HTTP to Django. These tests show
    what Django does with the headers the proxy adds."""

    def test_https_is_only_believed_when_the_header_is_trusted(self):
        request = RequestFactory().get("/", HTTP_HOST="languages.localhost", HTTP_X_FORWARDED_PROTO="https")
        self.assertFalse(request.is_secure())                      # default: the header is ignored
        with override_settings(SECURE_PROXY_SSL_HEADER=PROXY_HEADER):
            self.assertTrue(request.is_secure())                   # trusted: now https

    def test_without_the_header_a_plain_request_is_not_secure(self):
        request = RequestFactory().get("/", HTTP_HOST="languages.localhost")
        with override_settings(SECURE_PROXY_SSL_HEADER=PROXY_HEADER):
            self.assertFalse(request.is_secure())

    @override_settings(SECURE_SSL_REDIRECT=True, SECURE_PROXY_SSL_HEADER=PROXY_HEADER)
    def test_ssl_redirect_does_not_loop_behind_the_proxy(self):
        plain = Client().get("/", HTTP_HOST="languages.localhost")
        self.assertEqual(plain.status_code, 301)
        self.assertEqual(plain["Location"], "https://languages.localhost/")
        proxied = Client().get("/", HTTP_HOST="languages.localhost", HTTP_X_FORWARDED_PROTO="https")
        self.assertEqual(proxied.status_code, 200)                 # no redirect loop

    def test_x_forwarded_host_is_ignored_by_default(self):
        response = Client().get("/", HTTP_HOST="languages.localhost", HTTP_X_FORWARDED_HOST="systems.localhost")
        self.assertIn("site: languages", response.content.decode())

    @override_settings(USE_X_FORWARDED_HOST=True)
    def test_trusting_x_forwarded_host_lets_a_visitor_pick_the_site(self):
        # This is why the setting stays off: anyone can send that header.
        response = Client().get("/", HTTP_HOST="languages.localhost", HTTP_X_FORWARDED_HOST="systems.localhost")
        self.assertIn("site: systems", response.content.decode())

  python manage.py test tests

Output (checked by running it; this is the full test run, with all the tests so far):

Found 24 test(s).
System check identified no issues (0 silenced).
........................
----------------------------------------------------------------------
Ran 24 tests in 0.043s
OK

What the tests show:
- With no SECURE_PROXY_SSL_HEADER, Django ignores X-Forwarded-Proto: the request
  is not "secure", even if the proxy says https. With the setting, it believes
  the header.
- SECURE_SSL_REDIRECT=True sends plain HTTP requests to https, and does NOT loop
  when the proxy header says https. Without SECURE_PROXY_SSL_HEADER it would
  redirect forever, because Django never sees HTTPS.
- X-Forwarded-Host is ignored by default. With USE_X_FORWARDED_HOST=True, a
  visitor can choose the site (and the host Django believes it is serving) by
  sending that header. Leave it off, and make the proxy pass the real host
  instead (ProxyPreserveHost On).

PART B: The proxy configuration. THIS PART WAS NOT RUN: Apache and Gunicorn
are not installed on the machine used to write the course. Test it on the
server with apache2ctl configtest, and with curl -I against each site.

Gunicorn, as a systemd service (/etc/systemd/system/learning-site.service):

  [Unit]
  Description=Learning site (Django, all sites)
  After=network.target

  [Service]
  User=www-data
  WorkingDirectory=/var/www/learning-site
  Environment=DJANGO_SETTINGS_MODULE=config.settings.prod
  EnvironmentFile=/etc/learning-site.env        # holds DJANGO_SECRET_KEY, mode 600, NOT in git
  ExecStart=/var/www/learning-site/venv/bin/gunicorn config.wsgi:application \
            --bind unix:/run/learning-site.sock --workers 3
  Restart=on-failure

  [Install]
  WantedBy=multi-user.target

Apache, one virtual host per site (the same text with ServerName changed). Enable the
modules once: a2enmod proxy proxy_http headers

  <VirtualHost *:80>
      ServerName languages.osztromok.com
      # Certbot adds the matching *:443 virtual host
  </VirtualHost>

  <VirtualHost *:443>
      ServerName languages.osztromok.com
      ProxyPreserveHost On                         # Django sees languages.osztromok.com
      RequestHeader set X-Forwarded-Proto "https"  # matches SECURE_PROXY_SSL_HEADER
      Alias /static/ /var/www/learning-site/staticfiles/
      <Directory /var/www/learning-site/staticfiles>
          Require all granted
      </Directory>
      ProxyPass /static/ !                         # Apache serves static files itself
      ProxyPass / unix:/run/learning-site.sock|http://localhost/
      ProxyPassReverse / unix:/run/learning-site.sock|http://localhost/
  </VirtualHost>

Checklist for the server:
1. apache2ctl configtest, then systemctl reload apache2
2. curl -I https://languages.osztromok.com/       expect 200
3. curl -I http://languages.osztromok.com/        expect a redirect to https, once
4. python manage.py check --deploy with the production settings
5. a request for the wrong site's folder gets 404 (languages with /linux/...)

WHY THIS WORKS AS AN ANSWER
---------------------------
The risky part of a proxied setup is the three things Django cannot see for
itself. Writing each one as a test (HTTPS, redirect loop, forwarded host) turns
the configuration from a hope into something you can check, and being clear
about which half was not run keeps the server test on your list.