Search and SEO

A site can be well organised and still be hard to find. Visitors need to search it, and search engines need to know what is on it and which address is the real one. When the content moves to several sites, three things become more important: each site must describe itself, the old addresses must lead to the new ones without losing ground, and no page should be listed under two addresses. This chapter looks at what the live site has today, then designs search, sitemaps, canonical addresses and redirects for the family of sites.

What Exists Today

That is a clean slate rather than a problem: the new sites can include all of this from the start, and none of it has to be migrated.

Search

Search comes in three levels, and each one builds on the one before:

A static site search tool such as Pagefind fits the second level: it indexes the built HTML files after the build and ships the index as static files, so the sites need no search server. For the third level its documentation describes a multisite feature: a search page loads other sites' indexes with mergeIndex, given each index's bundlePath, and an indexWeight option ranks one index above another. Two caveats from the documentation matter here. Indexes on separate domains must be served with CORS headers, which is the origin issue from Learning Website: Framework & Architecture 1 again, and a merged index uses the main instance's language support, which matters for a site with Hungarian, Japanese, German and French text.

Sitemaps

A sitemap is a list of the URLs you want search engines to know about, optionally with the date each page was last changed. The rules that matter, from Google's documentation:

• A single sitemap is limited to 50,000 URLs or 50 MB uncompressed. Larger sites use several files and a sitemap index.
• A sitemap's scope depends on where it lives: unless you submit it through Search Console, it affects only the directory it sits in and below. Put each one at the site root.
• Sitemaps can span several hosts, or a site's sitemap can be hosted centrally with each site's robots.txt pointing to it. A simple one sitemap per site avoids needing either.

Exercise 1 builds the sitemaps from the real content tree. The result for the proposed sites:

Every site is far below the limit. The page counts add to slightly more than the 4,925 distinct pages, because each of the six sites that receives sidebar pages has its own /sidebar/ index page.

robots.txt, Canonical Links and Descriptions

robots.txt belongs to a host

Google's documentation states that a robots.txt file applies only to the host, protocol and port that serve it, and that it must sit at the root. So languages.osztromok.com needs its own, and it is also the natural place to name that site's sitemap:

A new site that is not ready to be public should disallow crawling, and the day it launches that line must change. It is the kind of thing that is forgotten, so it goes on the launch checklist (Learning Website: Framework & Architecture 12).

One address per page

A canonical link tells search engines which address is the real one when a page can be reached in several ways (with and without www, over HTTP and HTTPS, with and without a trailing slash, or from an old and a new site). Google recommends an absolute URL, and recommends that a page include a canonical link to itself. Treat it as a strong signal, not a command. To keep the signals consistent, make three things agree on the same string: the canonical link, the sitemap entry and the internal links. That includes the decision about www from Learning Website: Framework & Architecture 5, and the trailing slash that the site's configuration already requires.

The meta description does not need new writing: the banner's Topic: line is already a written summary of each chapter, and can be shortened for the tag.

Moving Without Losing Ground

The part of SEO that can really hurt is the move itself. Google's site-move guidance gives a few rules that fit this migration well:

• Use permanent redirects, a 301 or 308, from each old URL to its new URL.
• Keep them a long time: generally at least a year, and indefinitely is better for visitors who have old links.
• A section-by-section move is acceptable for large sites, which matches moving languages first and the rest later.
• Expect weeks, not days: a small or medium site can take a few weeks for most pages to move.
• The Change of Address tool in Search Console applies to domain or subdomain changes (for example one subdomain to another), and is not needed for path-only changes. Whether it applies when a section moves from a path to a new subdomain is a question to check in Search Console at the time.

Redirect each old URL to its exact new URL in a single hop, not to the new site's home page, and not through a chain. Exercise 3 builds the rule for the languages area from the site map and tests it against every old page URL.

What Splitting Does to Search Visibility

Honest expectations help. Google has said it can handle subdomains and paths, but there is no guarantee that a moved section ranks the same afterwards, and for a while the old and new addresses are both known. Reduce the risk instead of trusting it: redirect every old URL, keep internal links consistent, submit each new site's sitemap, watch the Search Console reports for crawl and coverage errors, and move one area at a time so that a problem is found while it is small.

Hands-On Exercises