What's on this page

• How it works
• Exclude a page from the sitemap
• Change the default frequency and priority
• Add rules to robots.txt
• Override the templates

Customize sitemap and robots.txt

Cecil automatically generates /sitemap.xml and /robots.txt: they are default pages rendered with the built-in templates _default/sitemap.xml.twig and _default/robots.txt.twig.

How it works

The two pages are declared in the default configuration:

pages:
  default:
    robots:
      path: robots
      layout: robots
      output: txt
    sitemap:
      path: sitemap
      layout: sitemap
      output: xml
      changefreq: monthly
      priority: 0.5

• the sitemap lists all showable pages (published, not virtual, not redirected, not excluded), with their translations;
• robots.txt allows everything, disallows redirections and the 404 page, and points to the sitemap.

Exclude a page from the sitemap

Add excluded to the page front matter:

---
title: Thank you
excluded: true
---

The page is still published, but hidden from the sitemap and from list pages (home page, sections, etc.). Use published: false to not publish it at all.

Change the default frequency and priority

pages:
  default:
    sitemap:
      changefreq: weekly
      priority: 0.8

Add rules to robots.txt

Create a pages/robots.md page: its body is appended to the generated rules.

---
layout: robots
output: txt
---
User-agent: GPTBot
Disallow: /

Override the templates

Extract the built-in templates to start from the originals:

php cecil.phar util:templates:extract

Or create layouts/sitemap.xml.twig (or layouts/robots.txt.twig) from scratch. For example, a sitemap that also skips pages with a custom sitemap_exclude front matter variable:

<?xml version="1.0" encoding="utf-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
  {%- for p in site.pages.showable|filter(p => not p.sitemap_exclude|default(false))|sort_by_date ~%}
  <url>
    <loc>{{ url(p, {canonical: true}) }}</loc>
    <lastmod>{{ p.date|date('Y-m-d') }}</lastmod>
  </url>
  {%- endfor ~%}
</urlset>