AI Crawler Check Skill
Your agent shows which AI bots visit your site, what they read, what is blocked by mistake, and where crawling stops turning into citations.
- Bots grouped by purpose
- Mistaken blocks found
- Crawl to citation to visit
Skill
seomatic-ai-crawlers v1.0.0
Needs
SEOmatic's MCP server (free account)
Works In
Claude Code, Claude, Cursor, Codex, Gemini CLI, OpenClaw, Hermes Agent, Grok Bot and any agent that reads Agent Skills
Source
SKILL.md on GitHub, MIT license
Last Checked
October 2026
Inside the AI Crawler Check Skill
The Short Answer
The SEOmatic AI crawler skill teaches an AI agent to check which AI bots, such as GPTBot, ChatGPT-User, OAI-SearchBot, PerplexityBot and ClaudeBot, visit your site, grouped by purpose: answering a user's question live, building an AI search index, or training models. It checks robots.txt and firewall rules for mistaken blocks, then joins crawls with AI citations and assistant-referred visits per page to show where the chain breaks. SEOmatic can stage a robots.txt fix or content changes, and nothing goes live until you click approve.
What You Get
Which bots reach the site and which are blocked, on purpose or not; the pages AI reads most; and the three fixes with the biggest effect.
Ask Your Agent
Once the skill is installed, ask in your own words. For example:
- "Can ChatGPT and Perplexity see my site?"
- "Which AI bots crawl my site?"
- "Am I blocking AI search by mistake?"
How the Skill Works
- Who visits
- Is anything blocked by mistake
- Crawl to citation
- Report
Every number comes from a tool result with its exact date range. The skill never estimates a figure it could read.
Tools It Uses
| Tool | What it reads or does | Cost |
|---|---|---|
get_ai_crawler_activity | AI Crawler Activity | Free |
get_ai_funnel | AI Referral Funnel | Free |
ai_citations | Observed AI citations & brand facts | Free |
site_audit | Site-Wide Technical Audit | Free |
site_pages | Site & Page Analysis | Free |
task_manage | SEO Tasks | Free to stage; paid plan, your approval |
All from SEOmatic's MCP server. Parameters and full cost rules: MCP tool reference.
Install the Skill
One command installs it in Claude Code, Cursor, Codex, Gemini CLI, OpenCode and other agents that read Agent Skills. Then connect SEOmatic's MCP server at https://app.seomatic.ai/api/mcp and sign in (free, no card). In Claude on the web or desktop, upload the skill folder as a zip under Settings, Capabilities, Skills. Setup for every agent: AI assistants.
npx skills add Minh42/seomatic-mcp --full-depth --skill seomatic-ai-crawlersNothing Goes Live Without You
By default, nothing changes on your site until you approve it. A fix is staged as a proposal that says what will change and why. Approved fixes run within seconds, the page's previous state is stored, and any edit to an existing page can be rolled back. A rollback refuses to overwrite edits someone made since. Changes that could take a page out of Google, like a noindex, need an extra confirmation.
Run It on a Schedule
This skill is part of the AI Visibility Watch Grok Bot template, which run it on a schedule and only speak when something changed. All templates: SEO skills and templates.
More SEO Skills
Frequently Asked Questions
Should I block GPTBot?
That is a licensing choice about model training, not a ranking tactic. Blocking training bots does not remove you from AI search, but blocking search and retrieval bots does. The skill shows the difference.
Do I need an llms.txt file?
No major AI engine has said it uses llms.txt. It is cheap to add but it is not a fix, and the skill says so.
Is the AI Crawler Check skill free?
The skill is free and open source (MIT). Connecting is free with no card. Search Console reports, page checks and 30-page site audits are free, as often as you like. Live market data (keyword volumes, competitors, backlinks) uses free questions: 5 a month, plus a one-time 10 when you connect Search Console, and one question covers up to 5 lookups. Staging and publishing fixes is part of paid plans, which start with a 14-day trial.
Which AI agents can use it?
Any agent that reads Agent Skills (SKILL.md): Claude Code, Claude on the web and desktop, Cursor, Codex, Gemini CLI, OpenCode, Kiro, OpenClaw, Hermes Agent, Grok Bot and more. It installs with one command: npx skills add.
Do I need SEOmatic's MCP server?
Yes. The skill tells your agent how to do the job; SEOmatic's MCP server gives it your real data and the tools. Connecting is free: add https://app.seomatic.ai/api/mcp to your agent and sign in.