# All search and AI crawlers welcome (GPTBot, OAI-SearchBot, ClaudeBot, Claude-SearchBot, PerplexityBot, Google-Extended, ...).
# /x/ share pages stay private via X-Robots-Tag noindex — do NOT Disallow them here, crawlers must see that header.
# /api/, /mcp, /lang- are machine endpoints (POST-only / token-gated / lang-switch 303), not pages — search renderers kept fetching them as URLs.
User-agent: *
Disallow: /api/
Disallow: /mcp
Disallow: /lang-

Sitemap: https://walalook.me/sitemap.xml