mirror of
https://github.com/terribleplan/next.js.git
synced 2024-01-19 02:48:18 +00:00
3f6834dfec
This example app shows you how to set up sitemap.xml and robots.txt files for proper indexing by search engine bots.
10 lines
160 B
Plaintext
10 lines
160 B
Plaintext
User-agent: *
|
|
Allow: /a
|
|
Allow: /b
|
|
Allow: /posts/*
|
|
Disallow: /
|
|
Disallow: /a/*
|
|
Disallow: /b/*
|
|
Disallow: /posts
|
|
|
|
Sitemap: https://sitemap-robots.now.sh/sitemap.xml |