learning-website-django1-8  Exercise 3: Static Files and the Copy Button
=====================================================================

PART A. The theme's own files (the three stylesheets and copy-code.js) are static
files. Serve them with WhiteNoise, which lets the Django process do it with no web
server configuration, and gives each file a name that contains its hash so browsers
can keep it for years. WhiteNoise version 6.12.0.

pip install whitenoise      (and add  whitenoise>=6.12,<7  to requirements/base.txt)

Settings:

  MIDDLEWARE = [
      "django.middleware.security.SecurityMiddleware",
      "whitenoise.middleware.WhiteNoiseMiddleware",    # directly after SecurityMiddleware
      "apps.core.middleware.SiteMiddleware",
      ...
  ]
  # config/settings/prod.py
  STORAGES = {
      "default": {"BACKEND": "django.core.files.storage.FileSystemStorage"},
      "staticfiles": {"BACKEND": "whitenoise.storage.CompressedManifestStaticFilesStorage"},
  }

Then, before deploying:  python manage.py collectstatic

WhiteNoise's documentation says all static files must be generated in advance, and
that it is not suitable for user-uploaded files. The PDFs and solutions are not
static files in this sense (they are big and belong to one site), which is why they
are mirrored and served separately (Exercise 2).

Save as tests/test_static.py:

import re
import shutil
import tempfile
from pathlib import Path

from django.core.management import call_command
from django.test import Client, TestCase, override_settings

from apps.content.models import Page

PROD_STORAGES = {
    "default": {"BACKEND": "django.core.files.storage.FileSystemStorage"},
    "staticfiles": {"BACKEND": "whitenoise.storage.CompressedManifestStaticFilesStorage"},
}


class HashedStaticFilesTests(TestCase):
    """What production does: collectstatic gives every file a name that contains its hash."""

    def setUp(self):
        self.root = Path(tempfile.mkdtemp())
        self.addCleanup(shutil.rmtree, self.root, True)
        self.settings = override_settings(STATIC_ROOT=self.root, STORAGES=PROD_STORAGES, DEBUG=False,
                                          WHITENOISE_AUTOREFRESH=False, WHITENOISE_USE_FINDERS=False)
        self.settings.enable()
        self.addCleanup(self.settings.disable)
        call_command("collectstatic", interactive=False, verbosity=0)
        Page.objects.create(path="hungary/x/a.html", site="languages", kind="lesson", title="A", fragment="<p>x</p>")

    def test_every_stylesheet_and_the_script_get_a_hashed_name(self):
        names = sorted(p.name for p in (self.root / "theme").iterdir() if not p.name.endswith((".gz", ".br")))
        hashed = [n for n in names if re.search(r"\.[0-9a-f]{12}\.(css|js)$", n)]
        self.assertEqual(len(hashed), 4)                                  # tokens, base, components, copy-code

    def test_compressed_copies_are_made_for_the_text_files(self):
        self.assertTrue(list((self.root / "theme").glob("*.css.gz")))

    def test_the_manifest_lists_the_files(self):
        self.assertTrue((self.root / "staticfiles.json").is_file())

    def test_pages_link_to_the_hashed_files(self):
        html = Client().get("/hungary/x/a/", HTTP_HOST="languages.localhost").content.decode()
        for name in ("tokens", "base", "components"):
            self.assertRegex(html, rf"/static/theme/{name}\.[0-9a-f]{{12}}\.css")
        self.assertRegex(html, r"/static/theme/copy-code\.[0-9a-f]{12}\.js")

    def fetch(self, url):
        """Ask WhiteNoise for a file the way a browser would. Its file list is read when it starts, so build a new one."""
        from django.test import RequestFactory
        from whitenoise.middleware import WhiteNoiseMiddleware
        middleware = WhiteNoiseMiddleware(lambda request: None)
        response = middleware(RequestFactory().get(url, HTTP_HOST="languages.localhost"))
        self.addCleanup(response.close)
        return response

    def test_a_hashed_file_may_be_cached_for_ten_years_and_never_revalidated(self):
        html = Client().get("/hungary/x/a/", HTTP_HOST="languages.localhost").content.decode()
        url = re.search(r'href="(/static/theme/tokens\.[0-9a-f]{12}\.css)"', html)[1]
        response = self.fetch(url)
        self.assertEqual(response.status_code, 200)
        self.assertEqual(response["Cache-Control"], "max-age=315360000, public, immutable")
        self.assertTrue(response["Content-Type"].startswith("text/css"))

    def test_the_unhashed_name_is_cached_for_a_minute_only(self):
        response = self.fetch("/static/theme/tokens.css")
        self.assertEqual(response["Cache-Control"], "max-age=60, public")

    def test_static_files_are_sent_with_a_cross_origin_header(self):
        # handy: a font or stylesheet shared by several sites may be loaded from another one (Framework 1)
        self.assertEqual(self.fetch("/static/theme/tokens.css")["Access-Control-Allow-Origin"], "*")

(The full test run is shown in Exercise 2.)

What the tests found out about WhiteNoise:
- After collectstatic every stylesheet and script has a name with a 12-character
  hash (tokens.5f6ee8c98574.css) and a .gz copy; a manifest records them; and the
  pages link to the hashed names.
- A hashed file is sent with Cache-Control: max-age=315360000, public, immutable
  (ten years, never revalidated). A first guess in the test, one year, was wrong,
  and the test said so. The unhashed name is cached for 60 seconds only.
- Static files are sent with Access-Control-Allow-Origin: *, which is handy for
  a stylesheet or font shared between the sites (Learning Website: Framework &
  Architecture 1), and means anything you put under /static/ is readable by any web
  page.
- Referencing a file that is not in the manifest is an error in production, so a
  typo in a template shows up at once, not as a missing stylesheet.

PART B. The copy button (apps/theme/static/theme/copy-code.js). The first version
used the modern clipboard and did nothing if it was unavailable. It is available
only on HTTPS and on localhost, so an HTTP address (for example the development
server opened from another machine) would fail without a word. The new version
falls back to a hidden text box and the older copy command, and shows "Copy failed"
if both fail:

/* copy-code.js: loaded once by the base template.

   Page bodies are inserted as HTML text, and the browser does not run a <script> that arrives that way.
   So the copy function lives here, once, and every code block calls it (onclick="copyCodeBlock(this)").
   The code block must be the element right AFTER the button. */
function copyCodeBlock(btn) {
  const code = btn.nextElementSibling.innerText;
  const show = (message) => {
    const original = btn.dataset.label || btn.textContent;
    btn.dataset.label = original;
    btn.textContent = message;
    setTimeout(() => { btn.textContent = original; }, 1500);
  };

  // The modern clipboard only works on HTTPS (or localhost). Anywhere else, fall back to the old way
  // (a hidden text box and the "copy" command) instead of failing without a word.
  const fallback = () => {
    const box = document.createElement('textarea');
    box.value = code;
    box.setAttribute('readonly', '');
    box.style.position = 'fixed';
    box.style.opacity = '0';
    document.body.appendChild(box);
    box.select();
    let ok = false;
    try { ok = document.execCommand('copy'); } catch (e) { ok = false; }
    document.body.removeChild(box);
    show(ok ? 'Copied!' : 'Copy failed');
  };

  if (navigator.clipboard && window.isSecureContext) {
    navigator.clipboard.writeText(code).then(() => show('Copied!'), fallback);
  } else {
    fallback();
  }
}
window.copyCodeBlock = copyCodeBlock;

/* A <script> inside a page body (an interactive tool) never runs by itself. Re-creating it does. */
document.addEventListener('DOMContentLoaded', () => {
  document.querySelectorAll('.page-body script').forEach((old) => {
    const fresh = document.createElement('script');
    for (const attr of old.attributes) fresh.setAttribute(attr.name, attr.value);
    fresh.textContent = old.textContent;
    old.replaceWith(fresh);
  });
});

A browser test page, loaded with headless Chrome (tests/browser/copy_test.html).
It inserts a page body as TEXT, as the site does, with a script in it, and then
clicks the copy button:

<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="utf-8">
  <title>copy-code.js browser test</title>
  <!-- the real script, loaded the way the base template loads it -->
  <script src="../../apps/theme/static/theme/copy-code.js" defer></script>
</head>
<body>
  <div class="page-body"></div>
  <pre id="result">waiting</pre>

  <script>
    // 1. Insert a page body as TEXT, the way the site does. A script that arrives this way does not run by itself.
    document.querySelector('.page-body').innerHTML =
      '<script>window.ranFlag = "yes";<\/script>' +
      '<div class="code-block-wrap"><button class="copy-btn" onclick="copyCodeBlock(this)">Copy</button>' +
      '<div class="code-block">x = 1\ny = 2</div></div>';
    window.ranBeforeLoad = window.ranFlag || "no";

    // 2. After the page has loaded (and copy-code.js has re-created the script), try the button.
    document.addEventListener('DOMContentLoaded', () => {
      const params = new URLSearchParams(location.search);
      if (params.get('insecure') === '1') {
        Object.defineProperty(window, 'isSecureContext', { value: false });   // force the fallback path
      }
      const button = document.querySelector('.copy-btn');
      button.click();
      setTimeout(() => {
        document.getElementById('result').textContent = JSON.stringify({
          scriptRanBeforeLoad: window.ranBeforeLoad,
          scriptRanAfterLoad: window.ranFlag || 'no',
          buttonText: button.textContent,
          secureContext: window.isSecureContext,
          clipboardApi: !!navigator.clipboard,
        });
      }, 400);
    });
  </script>
</body>
</html>

Run it twice, the second time forcing the "insecure" path:

  chrome --headless=new --virtual-time-budget=4000 --dump-dom "file:///.../copy_test.html"
  chrome --headless=new --virtual-time-budget=4000 --dump-dom "file:///.../copy_test.html?insecure=1"

Output (checked by running it):

  {"scriptRanBeforeLoad":"no","scriptRanAfterLoad":"yes","buttonText":"Copy failed","secureContext":true,"clipboardApi":true}
  {"scriptRanBeforeLoad":"no","scriptRanAfterLoad":"yes","buttonText":"Copy failed","secureContext":false,"clipboardApi":true}

What the browser test shows, and what it cannot
-----------------------------------------------
- THE SCRIPT RE-CREATION WORKS: the script inserted as text had not run before the
  page loaded ("no") and had run afterwards ("yes"). That is what makes interactive
  tools work on the live site.
- THE COPY ITSELF COULD NOT BE VERIFIED. In headless Chrome the clipboard is not
  permitted and the older copy command also fails, so the button reported "Copy
  failed" on both paths. What this does prove: the fallback is reached when the
  clipboard is unavailable, and the button SAYS it failed instead of staying
  silent. Whether copying works must be tested by hand in a real browser, on both
  an HTTPS address and a plain HTTP one. It is in the review notes.

WHY THIS WORKS AS AN ANSWER
---------------------------
Hashed static files turn caching from a risk into a feature, and the tests read the
real headers instead of assuming them (the assumption was wrong). For the browser
code, saying exactly which part was checked and which was not is the honest result.