# Дарим - privacy-first crawl policy.
# Public surfaces (landing, create-flow, legal, guides) are open.
# Private surfaces (reveal player, organizer cabinet, contributor form,
# auth, "my cards") MUST NOT be crawled - they contain or lead to
# personal data subject to 152-ФЗ.
#
# Single wildcard policy by design. Per the robots.txt protocol,
# named user-agent groups do NOT inherit rules from the wildcard -
# a block like
#     User-agent: GPTBot
#     Allow: /
# would silently grant that bot access to /r/, /c/, /m/, etc., because
# the named group has no Disallow rules and shadows the wildcard.
#
# AI crawlers (GPTBot, OAI-SearchBot, ChatGPT-User, PerplexityBot,
# ClaudeBot, Claude-Web, Google-Extended, Bingbot, YandexBot, …) are
# intentionally NOT given named blocks here - they fall back to the
# wildcard and stay on the same private/public boundary as everyone
# else. We want them to read our public guides; we don't want them to
# touch private user data.
#
# When adding a new private path, add it under the wildcard Disallow
# block AND drop a <meta name="robots" content="noindex, nofollow">
# in the matching HTML head (belt-and-braces).

User-agent: *

# Backend + admin
Disallow: /api/
Disallow: /admin/

# Private user surfaces
Disallow: /r/
Disallow: /k/
Disallow: /c/
Disallow: /m/
Disallow: /me/
Disallow: /e/
Disallow: /h/
Disallow: /b/

# Error page
Disallow: /404.html

Allow: /

Sitemap: https://www.darim.ai/sitemap.xml

# The same public pages as plain text, for anything that would rather read
# prose than parse HTML. /llms.txt is the index and links to the full file.
# Not a standard directive - a parser that does not know it skips the line,
# which is the point: nothing here depends on it being honoured.
#
# Measured 2026-09-28: no AI crawler has ever requested either file. The only
# fetches in a week came from Dataprovider.com and BuiltWith. These lines are
# insurance against the convention catching on, not a lever; do not count them
# as GEO work.
Llms: https://www.darim.ai/llms.txt
Llms-full: https://www.darim.ai/llms-full.txt