# Plstka: the public site (landing, city pages, materials, guides, privacy policy) is open to search engines and AI crawlers. # The staff dashboard answers `X-Robots-Tag: noindex` instead of a Disallow, so crawlers can read it and drop it. User-agent: * Allow: / Disallow: /api/ User-agent: GPTBot Allow: / Disallow: /api/ User-agent: OAI-SearchBot Allow: / Disallow: /api/ User-agent: ChatGPT-User Allow: / Disallow: /api/ User-agent: ClaudeBot Allow: / Disallow: /api/ User-agent: anthropic-ai Allow: / Disallow: /api/ User-agent: PerplexityBot Allow: / Disallow: /api/ User-agent: Google-Extended Allow: / Disallow: /api/ User-agent: Bytespider Allow: / Disallow: /api/ User-agent: CCBot Allow: / Disallow: /api/ Sitemap: https://plstka.com/sitemap.xml