learning-website-nextjs1-4  Exercise 3: Scripts Inside a Fragment, Tested in a Real Browser
=======================================================================================

1,844 of the 4,415 pages have a <script> in their fragment (the copy buttons and the interactive tools; on the languages
site only 2). They have to work. Putting HTML into a page with dangerouslySetInnerHTML does NOT run scripts that arrive
later, so there is a trap, and it can only be seen in a browser.

The trap: a browser runs the scripts in the page it first downloads. But when a visitor CLICKS A LINK, Next fetches the new
page's data and inserts it into the page, and scripts inserted that way never run. The cure is to replace each script with a
fresh copy of itself, which the browser does run. A tiny client component does it:

Save as apps/languages/components/RunScripts.tsx:

"use client";

import { useEffect } from "react";

declare global {
  interface Window {
    /** Set once the first page has been hydrated. */
    __lwHydrated?: boolean;
  }
}

/**
 * Make the <script> elements inside a fragment run, when the browser has not already run them.
 *
 * A browser runs scripts that are in the page it first downloads. But when the visitor moves to another page by
 * clicking a link, Next fetches the new page's data and inserts it into the page, and scripts inserted that way never
 * run. The fix is to replace each such script with a new copy of itself, which the browser does run.
 *
 * It must NOT do this for the page that was downloaded directly: the browser has already run those scripts, and a
 * second run would add every listener twice. The first test of this component did exactly that (a script ran twice).
 * HydrationMark, in the layout, sets window.__lwHydrated after the first page is hydrated. Child effects run before
 * their parent's, so on the first page this component sees the flag unset, and on every later page it sees it set.
 *
 * Two markers keep React's development mode (which runs every effect twice) from running a script twice:
 * data-lw-initial on the container (this page came from the download) and data-lw-ran on each script copy.
 */
export function RunScripts({ targetId }: { targetId: string }) {
  useEffect(() => {
    const container = document.getElementById(targetId);
    if (!container) return;
    if (container.dataset["lwInitial"] === "1") return;
    if (!window.__lwHydrated) {
      container.dataset["lwInitial"] = "1";                    // downloaded with the page: already run by the browser
      return;
    }
    for (const old of Array.from(container.querySelectorAll("script"))) {
      if (old.dataset["lwRan"] === "1") continue;
      const copy = document.createElement("script");
      for (const { name, value } of Array.from(old.attributes)) copy.setAttribute(name, value);
      copy.text = old.text;
      copy.dataset["lwRan"] = "1";
      old.replaceWith(copy);
    }
  }, [targetId]);
  return null;
}

Save as apps/languages/components/HydrationMark.tsx:

"use client";

import { useEffect } from "react";

/** Tells RunScripts that the first page has been hydrated, so later pages came from navigation. Place it in the layout. */
export function HydrationMark() {
  useEffect(() => {
    window.__lwHydrated = true;
  }, []);
  return null;
}

(HydrationMark is placed in the layout after {children}.)

A test bench exists only in a build made with LW_LAB=1 (a normal build gives 404): two pages with the same script, one with
the component and one without, and an index page that links to both.

Save as apps/languages/app/lab/[name]/page.tsx:

import { notFound } from "next/navigation";

import { Fragment } from "../../../components/Fragment";

export const dynamicParams = false;

export function generateStaticParams() {
  return [{ name: "plain" }, { name: "rerun" }];
}

// a fragment with a script in it, like the interactive tools elsewhere on the site
const HTML = `<p id="out">waiting</p><script>
  window.__runs = (window.__runs || 0) + 1;
  document.getElementById("out").textContent = "script ran " + window.__runs + " time(s) on this page";
</script>`;

export default async function LabPage({ params }: { params: Promise<{ name: string }> }) {
  if (process.env["LW_LAB"] !== "1") notFound();
  const { name } = await params;
  return (
    <main>
      <h1>Lab: {name}</h1>
      <Fragment html={HTML} id="lab-body" rerunScripts={name === "rerun"} />
    </main>
  );
}

Save as apps/languages/app/lab/page.tsx:

import Link from "next/link";
import { notFound } from "next/navigation";

/** A test bench for scripts inside fragments. It exists only in a build made with LW_LAB=1. */
export default function Lab() {
  if (process.env["LW_LAB"] !== "1") notFound();
  return (
    <main>
      <h1>Lab</h1>
      <p><Link id="to-plain" href="/lab/plain">plain (scripts not re-run)</Link></p>
      <p><Link id="to-rerun" href="/lab/rerun">rerun (scripts re-run)</Link></p>
    </main>
  );
}

It is driven by a real headless Chrome through the DevTools protocol, using only Node's own WebSocket:

Save as lab-test.mjs:

// Drive a real (headless) Chrome through the DevTools protocol: do scripts inside fragments run
// (a) when the page is opened directly, and (b) when the visitor arrives by clicking a link?
//   node lab-test.mjs <base url>      (the languages app built with LW_LAB=1 and started)
import { spawn } from "node:child_process";
import { mkdtempSync } from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";

const base = process.argv[2] ?? "http://localhost:3001";
const chrome = spawn("C:/Program Files/Google/Chrome/Application/chrome.exe", [
  "--headless=new", "--disable-gpu", "--remote-debugging-port=9333", `--user-data-dir=${mkdtempSync(join(tmpdir(), "lw-chrome-"))}`, "about:blank",
], { stdio: "ignore" });
const sleep = (ms) => new Promise((resolve) => setTimeout(resolve, ms));

let targets;
for (let i = 0; i < 40; i++) {
  try { targets = await (await fetch("http://127.0.0.1:9333/json")).json(); if (targets.length) break; } catch { /* not up yet */ }
  await sleep(250);
}
const ws = new WebSocket(targets.find((t) => t.type === "page").webSocketDebuggerUrl);
await new Promise((resolve) => ws.addEventListener("open", resolve));
let id = 0;
const pending = new Map();
ws.addEventListener("message", (event) => {
  const message = JSON.parse(event.data);
  if (message.id && pending.has(message.id)) { pending.get(message.id)(message); pending.delete(message.id); }
});
const send = (method, params = {}) => new Promise((resolve) => { const n = ++id; pending.set(n, resolve); ws.send(JSON.stringify({ id: n, method, params })); });
const evaluate = async (expression) => (await send("Runtime.evaluate", { expression, returnByValue: true })).result?.result?.value;

async function open(path) {
  await send("Page.enable");
  await send("Page.navigate", { url: base + path });
  await sleep(2000);
}
const text = () => evaluate('document.getElementById("out") ? document.getElementById("out").textContent : "(no #out)"');

const rows = [];
await open("/lab/plain");  rows.push(["open /lab/plain directly", await text()]);
await open("/lab/rerun");  rows.push(["open /lab/rerun directly", await text()]);
await open("/lab");        await evaluate('document.getElementById("to-plain").click()');  await sleep(2500);
rows.push(["click a link to /lab/plain", `${await text()}   [address now ${await evaluate("location.pathname")}]`]);
await open("/lab");        await evaluate('document.getElementById("to-rerun").click()');  await sleep(2500);
rows.push(["click a link to /lab/rerun", `${await text()}   [address now ${await evaluate("location.pathname")}]`]);
// the same link twice in one window, going back in between: the script runs once per visit
await open("/lab");        await evaluate('document.getElementById("to-rerun").click()');  await sleep(2000);
await evaluate("history.back()");  await sleep(1500);
await evaluate('document.getElementById("to-rerun").click()');  await sleep(2000);
rows.push(["click, back, click again", `${await text()}   (the count is per window, so 2 is right)`]);
for (const [what, result] of rows) console.log(what.padEnd(30), result);

ws.close();
chrome.kill();
process.exit(0);

What happened, in order.

FIRST VERSION (RunScripts re-created every script it found):

  open /lab/plain directly       script ran 1 time(s) on this page
  open /lab/rerun directly       script ran 2 time(s) on this page      <-- a bug
  click a link to /lab/plain     waiting                                <-- the trap: the script never ran
  click a link to /lab/rerun     script ran 1 time(s) on this page

What went wrong: the component also re-ran the scripts on the page that was downloaded directly, where the browser had
already run them, so the script ran TWICE. A script that adds a button's click handler would have added it twice. I would
not have found this without opening the page in a browser and counting.
How to avoid it: re-run scripts ONLY for pages that arrived by navigation, and test both ways of arriving, with a script that
counts how often it ran.

THE FIX: HydrationMark (in the layout) sets window.__lwHydrated after the first page is hydrated. A child's effect runs
before its parent's, so on the first page RunScripts sees the flag unset (the page was downloaded: skip) and on every later
page it sees it set (the page came by navigation: re-run). Two markers (data-lw-initial, data-lw-ran) stop React's
development mode, which runs every effect twice, from running a script twice.

SECOND VERSION:

  open /lab/plain directly       script ran 1 time(s) on this page
  open /lab/rerun directly       script ran 1 time(s) on this page
  click a link to /lab/plain     waiting   [address now /lab/plain]       (still the trap: no component, no run)
  click a link to /lab/rerun     script ran 1 time(s) on this page   [address now /lab/rerun]
  click, back, click again       script ran 2 time(s) on this page   (the count is per window, so 2 is right: once per visit)

What was NOT tested: React's development mode (its double effects are handled by the markers but "next dev" was not run);
scripts that load other scripts with src= (the copy keeps the attributes, but none was tried); and other browsers (only
Chrome).

WHY THIS WORKS AS AN ANSWER
---------------------------
The behaviour that matters (does it run, and how many times, depending on how the visitor arrived) was measured in a real
browser, a bug was found and fixed that reading the code would not have shown, and both outcomes are in the record.