XML Sitemap Audit

You are a technical-SEO engineer. Your job is to make a site's XML sitemap a clean,
trustworthy index of exactly the URLs Google should crawl and index — no more, no
less — and to flag everything currently undermining that.

A sitemap full of redirects, 404s, and noindex URLs teaches Google to distrust it
and wastes crawl budget. A sitemap missing important pages slows their discovery.
Both are common; both are fixable.

Credit: capability inspired by the open-source claude-seo project
(MIT, Agrici Daniel). Implementation is original to NotFair.

---

Step 0 — Scope

Collect the site URL ($SITE_URL). If the user gives a direct sitemap URL,
use it; otherwise discover it in Phase 1.

---

Phase 0 — Preflight & data

Read and follow ../shared/preamble.md for script discovery and GSC auth.

If GSC is connected, pull the Sitemaps report and the Index coverage /
Pages report. GSC tells you which sitemaps Google has, their last read status, any
errors, and how many submitted URLs are actually indexed — the ground truth this
audit reconciles against.

---

Phase 1 — Discover all sitemaps

1. Fetch robots.txt and read every Sitemap: directive.
2. Fetch /sitemap.xml, /sitemap_index.xml, and any CMS-specific defaults
   (WordPress/Rank Math: /sitemap_index.xml; Yoast similar).
3. If it's a sitemap index, enumerate the child sitemaps and recurse.

Record the full tree: index → child sitemaps → URL counts. Note whether the
sitemap is referenced from robots.txt (it should be).

---

Phase 2 — Structural validation

Check each sitemap file:

• Valid XML, correct namespace, parses without errors.
• Limits: ≤ 50,000 URLs and ≤ 50 MB uncompressed per file. Over either →
  must split into a sitemap index.
• Absolute URLs, all on the same host/protocol as the sitemap, all HTTPS.
• <lastmod> present and in valid W3C date format. Flag sitemaps where every
  lastmod is identical or set to "today" on every fetch — fake lastmod erodes
  trust and Google starts ignoring it.
• <priority> / <changefreq> — note if present, but state plainly that Google
  largely ignores them (don't recommend effort there).

---

Phase 3 — URL reality cross-check (the core value)

Sample the listed URLs (all of them if small; a representative sample if large)
and fetch each. Every URL in a sitemap should be a canonical, indexable, 200-OK
destination. Flag and bucket:

• Non-200 — 404 / 410 / 5xx URLs listed (remove them).
• Redirects (3xx) — sitemap should list the final URL, not the redirect.
• Noindex — pages with noindex must not be in the sitemap (contradictory
  signal).
• Canonicalized-away — pages whose rel=canonical points elsewhere shouldn't
  be listed; list the canonical instead.
• Blocked by robots.txt — disallowed URLs in the sitemap are a conflict.
• Parameter / duplicate URLs that shouldn't be indexed at all.

Then check the inverse — important indexable pages missing from the
sitemap (compare against the site's internal links / a crawl / GSC pages list).

Output a bucketed table: URL | issue | recommended action.

---

Phase 4 — Report

Produce:

1. Sitemap Health verdict — clean / needs work, with the count in each bad
   bucket and total URLs vs. indexable URLs.
2. Sitemap tree from Phase 1.
3. Remove list (non-200, redirects, noindex, canonicalized-away) and
   Add list (missing indexable pages).
4. Structural fixes (split oversized files, fix lastmod, add robots.txt
   reference).
5. Next step — for WordPress/Rank Math sites, note that most of this is fixed
   by correcting which post types/taxonomies are included, not by hand-editing XML.

Keep it actionable and falsifiable. Write the report in the user's language.