User-agent: * # Content Signals (https://contentsignals.org/) — how this content may be used # once it has been fetched. `yes` grants permission, `no` withholds it; these # are declarations of preference, not access control, and say nothing about # whether a crawler may fetch the page (that is the Allow/Disallow rules). # We want to be found in search and cited in AI answers, so both are yes; # we do not grant use of this content as training data. # search — build a search index and link back to the source # ai-input — retrieve at answer time to ground a generated response (RAG) # ai-train — train or fine-tune a generative model Content-Signal: search=yes, ai-input=yes, ai-train=no Allow: / User-agent: Googlebot Allow: / User-agent: Bingbot Allow: / User-agent: Slurp Allow: / User-agent: DuckDuckBot Allow: / User-agent: Baiduspider Allow: / User-agent: YandexBot Allow: / User-agent: GPTBot Allow: / User-agent: ChatGPT-User Allow: / User-agent: OAI-SearchBot Allow: / User-agent: ClaudeBot Allow: / User-agent: anthropic-ai Allow: / User-agent: claude-web Allow: / User-agent: PerplexityBot Allow: / User-agent: Google-Extended Allow: / Disallow: /_next/ Disallow: /api/ Disallow: /*.json$ Disallow: /*_buildManifest.js$ Disallow: /*_middlewareManifest.js$ Disallow: /*_ssgManifest.js$ Disallow: /*.js$ Disallow: /jsconfig.json Disallow: /next.config.mjs Disallow: /package.json Disallow: /package-lock.json Disallow: /tailwind.config.js Disallow: /postcss.config.mjs Allow: /assets/ Allow: /favicon.ico Allow: /*.webp$ Allow: /*.jpg$ Allow: /*.jpeg$ Allow: /*.png$ Allow: /*.svg$ Allow: /*.gif$ Allow: /*.css$ Allow: / Allow: /create/ Allow: /privacy Allow: /terms Allow: /refund-policy User-agent: MJ12bot Disallow: / User-agent: DotBot Disallow: / User-agent: SemrushBot Disallow: / User-agent: MegaIndex Disallow: / # Sitemap — the apex 308s to this host, so declare it directly and save # crawlers the redirect. Sitemap: https://www.artificialstudio.ai/sitemap.xml # Agent capability manifest (ARD) — what this origin exposes to agents: the # MCP server, the REST API catalog, the installable skills, and how to # authenticate for each. Agentmap: https://www.artificialstudio.ai/.well-known/ai-catalog.json Crawl-delay: 1