Meta-ExternalAgent
Meta-ExternalAgent is Meta's dedicated crawler for ingesting public web data to train Meta Llama foundation models and improve Meta AI across Instagram, WhatsApp, and Facebook.
Technical Specifications
meta-externalagentFrequently Asked Questions
How do I block Meta AI training crawler?
User-agent: meta-externalagent Disallow: /
Ensure this bot can index your public pages for citations.
# Allow Meta-ExternalAgent User-agent: meta-externalagent Allow: /
Prevent this bot from accessing any content on your domain.
# Block Meta-ExternalAgent User-agent: meta-externalagent Disallow: /
Related Crawlers
Suggested Tools for Your Stack
Explore related utilities in this workflow or discover cross-category tools.
Related in AI & GEO Visibility
AI Crawler & GEO Readiness Hub
Instant 0–100 GEO Visibility Score across 26+ AI bots (OpenAI, Perplexity, Gemini) and Cloudflare WAF.
llms.txt & Knowledge Context Generator
Generate and validate standard llms.txt & llms-full.txt files to provide structured context to AI search agents.
Recommended from Other Categories
SaaS UX & Friction Audit
0–100 Product Experience Score across onboarding, IA, visual hierarchy, and navigation.
SaaS Pricing Page & Psychology Analyzer
Audit tier differentiation, annual toggle psychology, feature comparison, and objection handling.