1. Next.js Sitemap Evolution (App Router vs Pages)
In legacy Next.js Pages Router architectures, generating a dynamic XML sitemap required hacky server-side solutions: custom API routes with manual XML string concatenation, third-party packages like next-sitemap writing to the public directory during build time, or heavy getServerSideProps middleware wrappers. In the modern Next.js App Router, sitemaps are treated as first-class citizens of the Metadata Route API, unlocking streaming, native TypeScript typings, and automatic edge routing without external build plugins.
⚡ Why the Native App Router Metadata Route Wins
No post-build scripts or external npm packages required
Natively integrates with Incremental Static Regeneration (ISR)
Native generateSitemaps splits >50k URLs into sitemap indexes
2. The Native app/sitemap.ts File Convention
Creating a sitemap is as simple as placing a sitemap.ts (or sitemap.js) file inside your root app/ directory. Next.js automatically maps this route to https://yourdomain.com/sitemap.xml with the appropriate application/xml Content-Type header:
// app/sitemap.ts - Production TypeScript Pattern
import { MetadataRoute } from 'next';
export const revalidate = 86400; // Cache at CDN edge for 24 hours (ISR)
export default async function sitemap(): Promise<MetadataRoute.Sitemap> {
const baseUrl = 'https://dailytool.net';
// Static marketing routes
const staticRoutes: MetadataRoute.Sitemap = [
{
url: `${baseUrl}/`,
lastModified: new Date(),
changeFrequency: 'daily',
priority: 1.0,
},
{
url: `${baseUrl}/tools/seo/sitemap-generator/`,
lastModified: new Date(),
changeFrequency: 'weekly',
priority: 0.9,
},
];
return staticRoutes;
}3. Streaming CMS & Database Records at Scale
Real applications query headless CMS endpoints (Sanity, Strapi, Contentful) or PostgreSQL/Prisma databases. Here is how to query and transform dynamic entities asynchronously with resilient error boundaries:
// app/sitemap.ts - Dynamic CMS Query with Fallback Handling
import { MetadataRoute } from 'next';
import { db } from '@/lib/db';
export const revalidate = 43200; // Revalidate every 12 hours
export default async function sitemap(): Promise<MetadataRoute.Sitemap> {
const baseUrl = process.env.NEXT_PUBLIC_SITE_URL || 'https://dailytool.net';
try {
const publishedArticles = await db.article.findMany({
where: { status: 'PUBLISHED' },
select: { slug: true, updatedAt: true },
orderBy: { updatedAt: 'desc' },
take: 20000,
});
const dynamicEntries: MetadataRoute.Sitemap = publishedArticles.map((article) => ({
url: `${baseUrl}/blog/${article.slug}/`,
lastModified: article.updatedAt,
changeFrequency: 'weekly',
priority: 0.8,
}));
return [
{ url: `${baseUrl}/`, lastModified: new Date(), changeFrequency: 'daily', priority: 1.0 },
...dynamicEntries,
];
} catch (error) {
console.error('Failed to generate dynamic sitemap entries:', error);
// Return critical static core pages so Googlebot never encounters a 500 error
return [{ url: `${baseUrl}/`, lastModified: new Date(), priority: 1.0 }];
}
}4. Sitemap Index & generateSitemaps for 50,000+ URLs
Google Search Console enforces strict limits on individual XML sitemaps: maximum 50,000 URLs and uncompressed file size < 50 MB. For e-commerce catalogs or programmatic SEO hubs with hundreds of thousands of pages, Next.js provides the native generateSitemaps function:
// app/sitemap.ts - Native Sitemap Index Partitioning
import { MetadataRoute } from 'next';
const URLS_PER_SITEMAP = 25000;
// Next.js automatically creates a Sitemap Index at /sitemap.xml
// and routes individual chunks to /sitemap/0.xml, /sitemap/1.xml, etc.
export async function generateSitemaps() {
const totalProducts = await fetchProductCount();
const numberOfSitemaps = Math.ceil(totalProducts / URLS_PER_SITEMAP);
return Array.from({ length: numberOfSitemaps }, (_, index) => ({
id: index,
}));
}
export default async function sitemap({ id }: { id: number }): Promise<MetadataRoute.Sitemap> {
const skip = id * URLS_PER_SITEMAP;
const products = await fetchProductsChunk({ skip, take: URLS_PER_SITEMAP });
return products.map((product) => ({
url: `https://dailytool.net/products/${product.slug}/`,
lastModified: product.updatedAt,
changeFrequency: 'daily',
priority: 0.7,
}));
}5. Edge Caching & On-Demand ISR Revalidation
Generating a sitemap dynamically on every single HTTP GET request will saturate database connection pools when Googlebot, Bingbot, and Ahrefs crawl your domain concurrently. Configure edge caching using route segment configurations:
Time-Based ISR Revalidation
Add export const revalidate = 86400; at the top of sitemap.ts. The initial crawler request is cached at the Vercel or Cloudflare CDN edge for 24 hours. Subsequent crawl hits return in under 25 milliseconds with 0 DB load.
On-Demand Webhook Purging
When a marketing editor publishes a breakthrough post in your CMS, trigger an instant cache bust via Next.js server actions or API routes: revalidatePath('/sitemap.xml');. Googlebot will receive fresh URLs within seconds.
6. The Trailing Slash & Canonical Alignment Trap
The single most widespread technical SEO flaw causing 'Crawled - currently not indexed' in Google Search Console is URL syntax divergence between your sitemap, your canonical tags, and your next.config.js settings:
⚠️ The 308 Permanent Redirect Loop
If your next.config.js specifies trailingSlash: true, but your sitemap generates non-trailing-slash URLs like https://dailytool.net/tools/seo, Googlebot crawls the sitemap, hits an immediate HTTP 308 redirect to /tools/seo/, and flags your sitemap for containing redirected non-canonical URLs.
Rule: Always ensure sitemap URLs, page canonical tags, and next.config.js trailingSlash configuration are 100% identical.
7. Multilingual Hreflang Sitemaps in App Router
Next.js App Router natively supports encoding alternate language variations directly into sitemap objects using the alternates.languages property, eliminating the need for bulky page-level HTML meta tags:
// Multi-Region Hreflang Encoding
return [
{
url: 'https://dailytool.net/tools/seo/sitemap-generator/',
lastModified: new Date(),
alternates: {
languages: {
es: 'https://dailytool.net/es/tools/seo/sitemap-generator/',
de: 'https://dailytool.net/de/tools/seo/sitemap-generator/',
'x-default': 'https://dailytool.net/tools/seo/sitemap-generator/',
},
},
},
];8. Static HTML Export (output: 'export') Workarounds
When compiling Next.js as a completely static site (e.g., deploying to GitHub Pages or AWS S3 with output: 'export'), dynamic server-side route handlers are disabled at runtime. Next.js solves this by pre-rendering sitemap.ts into a static out/sitemap.xml file at build time, provided all data fetching inside sitemap.ts executes synchronously or resolves during next build.
9. Frequently Asked Technical Questions
Should I include noindex pages or 301 redirects in my sitemap?
Never. Sitemaps should strictly contain 200 OK, indexable, canonical URLs. Including noindex or 301 redirect URLs wastes crawl budget and triggers GSC validation errors.
Where does Next.js serve the generated sitemap?
Next.js serves app/sitemap.ts at the standard /sitemap.xml endpoint automatically.
How frequently should Googlebot re-crawl my sitemap?
Googlebot polls sitemaps based on publishing frequency. Submitting your sitemap URL once inside Google Search Console allows Google's crawler to check your XML feed periodically on its own schedule.
10. Production Audit Checklist
Before deploying your Next.js application, verify these 5 production items:
// Next.js Sitemap Pre-Flight Checklist
[✓] File located at app/sitemap.ts (returns MetadataRoute.Sitemap)
[✓] Configured export const revalidate to avoid cold-boot DB timeouts
[✓] Trailing slashes strictly aligned with next.config.js
[✓] Partitioned with generateSitemaps if total URLs exceed 25,000
[✓] Validated XML syntax using DailyTool XML Sitemap Generator
Audit & Validate Your XML Sitemaps Instantly
Generate, validate, and preview standards-compliant XML sitemaps with Googlebot crawl budget optimization.