Bots
facebookexternalhit: allow it or block it?
facebookexternalhit is Meta's link preview crawler: it fetches your page when someone shares the link on Facebook, Instagram or Messenger, and builds the card with your title and image. Allow it. Blocking it turns every share into a bare link. Meta says it may bypass robots.txt for security checks, and publishes no way to verify it.
7 days free · card required
0AI-view requests from facebookexternalhit · 30 days
filed as conventional traffic (SEO tools, uptime checks, link unfurlers)
| Agent | AI requests, 30 days | attensira.com robots.txt |
|---|---|---|
| facebookexternalhit (this page) | not counted | Allowed |
| Slackbot-LinkExpanding | not counted | Wildcard |
None of the 2 had a request filed in the window. "Not counted" is off the board's AI view, not a zero.
Analyst · reads the AI traffic boardAI traffic board, read over the Attensira MCP; it files facebookexternalhit as conventional traffic, off its AI view · · n=143,601 AI requests
Karl-Gustav Kallasmaa, Founder & CEOLast updated How do I allow or block facebookexternalhit in robots.txt?
Paste this into the robots.txt at the root of your domain. The operator says the agent may not read robots.txt, so the group records your preference; enforce anything sensitive on the server. Beside it, the group attensira.com serves facebookexternalhit today.
# Allowing the preview crawler is usually the right call:# it is what renders your share card. Block it only if you# mean to lose that.User-agent: facebookexternalhitAllow: / # To block it instead:# User-agent: facebookexternalhit# Disallow: /User-agent: facebookexternalhitAllow: /Disallow: /api/We name facebookexternalhit and allow it everywhere except /api/, the same as every AI and search crawler we allow by name. Its operator states robots.txt may not apply to it, so this group is a preference at most.
Read the whole file, or write yours with the robots.txt generator.
Who runs facebookexternalhit, and what is it for?
Meta runs it, and we file it under user-triggered fetch. Fetches a page when someone shares its link, to build the preview card. One request per share: not a crawl, not training, and not a citation path.
From Meta’s documentation, read . Why a training fetch and a search fetch are different decisions: the AI crawler guide.
What breaks when you block facebookexternalhit?
Your share cards. Meta documents the crawler's primary purpose as crawling the content of an app or website that was shared on one of Meta's family of apps, such as Facebook, Instagram or Messenger1. It fetches your page because a person, seconds earlier, pasted your link. It reads your Open Graph tags, builds the card with title, description and image, and leaves.
With it disallowed, a URL posted to Facebook, Instagram or Messenger renders as a bare link: no title, no description, no image. Sites get there almost entirely by accident. Someone writes a wildcard Disallow to keep AI crawlers out and takes every link unfurler with it, this one and Slack's included. Nothing errors. The first symptom is a colleague saying links to the site "look broken" when they post them.
The robots.txt block above allows it explicitly, so an AI-crawler policy written later does not sweep it up.
Is facebookexternalhit one of Meta's AI crawlers?
No. Its job is the preview card. Meta documents separate agents with separate tokens for its other work; Meta-ExternalAgent and Meta-ExternalFetcher have their own pages. If your goal is to keep Meta's AI crawling out, name those tokens. A rule naming facebookexternalhit costs you share cards and leaves the AI question untouched.
The scale difference is visible in our own data. On attensira.com, Attensira's AI traffic measurement recorded 131,105 requests from Meta-ExternalAgent across 873 pages in the 30 days to 4 October 2026, 91% of the 143,601 AI-crawler requests it filed. A preview fetch is filed apart from those, as conventional traffic, so the board gives no facebookexternalhit count at all. A preview crawler that fires when someone shares a link is a different animal from that. Counts are user agents as sent, from one site.
Does facebookexternalhit obey robots.txt?
Meta says two things that need to be read together. Its crawlers can be blocked by adding a disallow for the relevant crawler to robots.txt, and the facebookexternalhit crawler might bypass robots.txt when performing security or integrity checks, such as checking for malware or malicious content3.
So this page records its position as not bound rather than respecting. That is not an accusation: a malware check that honoured a Disallow would be one any malicious site could switch off. It does mean your rule is a strong preference, not a guarantee, and a request on a disallowed path is not by itself evidence of anything going wrong.
Meta also asks for up to 24 hours for a robots.txt change to take effect, because its crawlers may cache the file that long6. Do not judge a rule on the afternoon you wrote it.
How do I verify that a visit really came from facebookexternalhit?
A user-agent header is free text, so a log line naming facebookexternalhit is a claim until you check it.
Meta's crawler documentation publishes no IP range file and no reverse DNS suffix for facebookexternalhit. It advises allowlisting either the user-agent strings or the IP addresses, without publishing the addresses on that page.
Why is the card still empty when the crawler is allowed?
A blocked crawler is only one cause, and Meta documents the others. The server must support gzip or deflate. Open Graph properties must appear before the first 1 MB of the response or they are cut off. And the content must be crawlable within a few seconds, or Facebook cannot display it4.
The 1 MB rule catches modern sites. A framework that inlines a large block of critical CSS or a script payload above the meta tags pushes the Open Graph tags past the cutoff, and the result, a bare link, looks exactly like a robots.txt problem. Check the order of your <head> before you go looking for a rule. Meta's Sharing Debugger forces a re-crawl once the page is fixed.
Why is this token risky to allowlist at the CDN?
Because it cannot be verified and sites routinely let it through so share cards work. Meta's crawler page advises allowlisting either the user-agent strings or the IP addresses, and does not publish the addresses there: no range file and no reverse DNS suffix5.
facebookexternalhit is a token sites often allowlist by name at the CDN, because that exception is the usual fix for an empty card. A token that is both unverifiable and broadly trusted is exactly the one worth forging, and the exception you granted at the CDN is what it buys. Keep any exception narrow: let it fetch pages and their images, not your admin paths or APIs.
Meta publishes no reverse-DNS suffix and no IP range file for facebookexternalhit, so a visit can be matched only by its user agent.
ResearcherTranscribed from Meta documentation · Agents filed beside facebookexternalhit
In this entry
- What breaks when you block facebookexternalhit?
- Is facebookexternalhit one of Meta's AI crawlers?
- Does facebookexternalhit obey robots.txt?
- Why is the card still empty when the crawler is allowed?
- Why is this token risky to allowlist at the CDN?
- How should you match it in logs?
- What can Attensira show you about facebookexternalhit?
How should you match it in logs?
Match the bare token facebookexternalhit, case-insensitively. Meta publishes two strings, one with a link to its crawler help page and one that is just facebookexternalhit/1.1, and says the one in your logs will be similar to one of them2. Both carry /1.1, a version Meta can increment. A filter pinned to the long form already misses the short one.
What can Attensira show you about facebookexternalhit?
AI traffic, on Growth and Business, groups fetches by operator and declared purpose at the edge, so a preview fetch is not counted as AI crawling and Meta's AI crawler is not hidden among share cards. It reports user agents as sent, which for this token is all anyone can report. See crawler logs and the robots.txt generator.
facebookexternalhit/1.1 (+http:/Should I allow facebookexternalhit in my situation?
Concrete cases, and what each one implies for this agent.
Somebody shared your page on Facebook and the post came through as a bare link with no image.
Check for a facebookexternalhit Disallow first, then check the three documented crawler requirements.
Meta requires gzip or deflate support, Open Graph properties within the first 1 MB of the response, and content crawlable within a few seconds. A blocked crawler and a 1.2 MB head section produce the same symptom, and only one of them is fixed in robots.txt.
You wrote a blanket rule to keep AI crawlers out and preview cards stopped working across every platform.
Replace the wildcard with per-token groups and put the preview fetchers back.
Meta documents five separate crawler tokens, of which facebookexternalhit is the preview one and the others do advertising, indexing and model training. A wildcard Disallow catches all five, plus Slack's and every other unfurler. The preview crawlers are the ones whose fetch a person just asked for by pasting your link.
You need a definitive answer on whether a burst of facebookexternalhit traffic is really Meta.
You will not get one from Meta's crawler documentation; respond to the load instead.
The page publishes no IP list and no reverse DNS suffix for this agent, and this token is widely allowlisted at CDNs, which makes it worth forging. Rate-limit on behaviour rather than granting an unconditional bypass to a name you cannot check.
What else do people ask?
What is facebookexternalhit?
Does facebookexternalhit respect robots.txt?
Why does my link share with no image even though the crawler is allowed?
Where do these facts come from?
6 claims on this page, each with the page it was read from; last checked . The numbers in the text point here.
Meta documents the primary purpose of FacebookExternalHit as crawling the content of an app or website that was shared on one of Meta's family of apps, such as Facebook, Instagram or Messenger.
developers.facebook.com · retrieved
“The primary purpose of FacebookExternalHit is to crawl the content of an app or website that was shared on one of Meta's family of apps”
Meta publishes two user-agent strings for the crawler and states the string seen in log files will be similar to one of them — "facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php)" and "facebookexternalhit/1.1".
developers.facebook.com · retrieved · changes often, check the source
Meta states that its crawlers can be blocked by adding a disallow for the relevant crawler to robots.txt, and that the FacebookExternalHit crawler might bypass robots.txt when performing security or integrity checks, such as checking for malware or malicious content.
developers.facebook.com · retrieved
“the FacebookExternalHit crawler might bypass robots.txt when performing security or integrity checks, such as checking for malware or malicious content”
Meta states that a server must use gzip or deflate, that Open Graph properties must be listed before the first 1 MB of a website or app or they will be cut off, and that content must be crawlable within a few seconds or Facebook will be unable to display it.
developers.facebook.com · retrieved
“Any Open Graph properties need to be listed before the first 1 MB of your website or app, or it will be cutoff.”
Meta's web crawlers documentation publishes no IP range file and no reverse DNS suffix for facebookexternalhit, advising instead that site owners allowlist either the user-agent strings or the IP addresses.
developers.facebook.com · retrieved
Meta asks site owners to allow up to 24 hours for changes to robots.txt to take effect, because its crawlers may cache the contents of robots.txt for up to 24 hours.
developers.facebook.com · retrieved
Want whatever it takes to win the answer?
7 days free · card required