Skip to content

Latest commit

 

History

History
75 lines (54 loc) · 3.92 KB

seo.md

File metadata and controls

75 lines (54 loc) · 3.92 KB

Search Engine Optimisation (SEO)

Head Start is pre-configured to provide an XML sitemap, inject SEO meta, canonical and alternate links.

SEO meta data

Head Start utilises the SEO Preferences and SEO fields in DatoCMS. The global SEO settings can therefore be configured via SEO Preferences: /editor/settings in your CMS instance. For each individual page, its SEO settings can be extended using its SEO field.

To use the SEO data from DatoCMS you must query it, and pass it on to the default layout, which in turn uses our <SeoHead> component.

For example:

query Page($locale: SiteLocale!, $slug: String!) {
  page(locale: $locale, filter: { slug: { eq: $slug } }) {
    _seoMetaTags {
      attributes
      content
      tag
    }
  # ...

The _seoMetaTags contains the merged values of the global SEO Preferences and a page's SEO field. The layout adds them all to the head of the HTML:

---
import Layout from '@layouts/Default.astro';
const { page } = // ...
---
<Layout
  seoMetaTags={ page._seoMetaTags }
  { ...otherProps }
>

Canonical and alternate links

For SEO it's important that a page has a preferred canonical URL and links to pages in alternate locales. Head Start makes it easy to set these, by providing pageUrls to the default layout:

---
import Layout from '@layouts/Default.astro';
---
<Layout 
  pageUrls={[
    { locale: 'en', pathname: '/en/some/path/' },
    { locale: 'nl', pathname: '/nl/ander/pad/' },
    { locale: '..', pathname: '...' },
  ]}
  { ...otherProps }
>

The page URL matching the current page locale is used as link[rel=canonical]. The other URLs are used as link[rel=alternate][hreflang={locale}] (also see I18n Routing). If an empty list of pageUrls is provided, Head Start defaults to /{locale}/ for all links.

Sitemap

Head Start automatically generates an XML sitemap, using the official @astro/sitemap. The XML sitemap automatically includes all static pages, based on the getStaticPages() of all routes. If you need to include dynamic pages, you can configure these using the customPages option in @astro/sitemap.

The default layout and a robots.txt both link to the generated XML sitemap (/sitemap-index.xml), so it's picked up and indexed by search engines.

Note: Head Start does not set changefreq, lastmod and priority values in the XML Sitemap. See decision log.

(Dis)allow AI bots

Editors can toggle if they want to (dis)allow AI bots access to their content via the CMS (under App). When AI bots are disallowed, a snippet is injected into the robots.txt to Disallow: / a list of known AI bots (such as GPTBot and ClaudeBot).

Note: the list of known AI bots is saved to lib/seo/ai.robots.txt as vendor code. To download a newer version from github.com/ai-robots-txt/ai.robots.txt you can run:

npx jiti scripts/download-ai-robots-txt.ts

Tip: if the domain is also managed on Cloudflare, you can Block AI bots from Cloudlare domain security settings (also see background info). And in addition you can run an audit for AI bot insights on your domain.