For the complete documentation index, see llms.txt. Every page on this site is also served as Markdown: append `.md` to any URL, or send `Accept: text/markdown`.
Skip to content
Attensira LogoAttensira
Book a demo

Bots

Link preview bots

Preview bots fetch a page when someone shares its link, to build the card that appears in a chat or feed. facebookexternalhit does it for Facebook, Instagram and Messenger, and Slackbot-LinkExpanding does it for Slack. Meta says robots.txt may not apply, Slack says it does not honor it, and blocking either breaks your share cards.

Start freeBook a demo

7 days free · card required

Link preview bots2 agents
agents documented
2
publish IP ranges or reverse DNS
0 of 2

The rule each entry recommends, from the facebookexternalhit and Slackbot-LinkExpanding entries, per the operator's documentation.

User-agent: facebookexternalhitAllow: / User-agent: Slackbot-LinkExpandingAllow: /
Researcher agentResearcherOperator documentation and our AI traffic board (Attensira MCP) ·

Which preview bots are documented?

Fetches a page when someone shares its link, to build the preview card. One request per share: not a crawl, not training, and not a citation path.

How do the link preview bots compare?

Each row is transcribed from the operator's own documentation. The attensira.com column is ours: what our live robots.txt says to the agent.

  • facebookexternalhit

    Operator
    Meta
    robots.txt
    May not apply
    Verify by
    Nothing published
    attensira.com
    Allowed by name

  • Slackbot-LinkExpanding

    Operator
    Slack
    robots.txt
    Not honored
    Verify by
    Nothing published
    attensira.com
    No named group

Each row read from the operator's documentation, 3 Sep 2026.

What does blocking a preview bot cost?

Your share card. When a preview fetch fails, the link is posted without the title, description and image the card is built from. Meta documents a robots.txt bypass for facebookexternalhit, and Slack states it does not honor robots.txt, so a Disallow mostly records intent; Slack's documented route to exclusion is contacting Slack for a blocklist entry.

attensira.com allows facebookexternalhit by name. A shared link with a proper card is worth more than the one request it costs.

User-Agentfacebookexternalhit
facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php)

The version number in this string changes; match the token, not the whole string. The facebookexternalhit entry, and reading user-agent strings.

Researcher agentResearcherTranscribed from Meta documentation ·

Why can't I verify a preview bot?

Neither operator publishes anything to verify against. Meta publishes no IP range file or reverse DNS suffix for facebookexternalhit and advises allowlisting the user-agent strings or addresses without listing the addresses. Slack publishes neither for any of its three agents. A user-agent string is free text, so treat unexpected volume under these names with suspicion.

Preview fetches are not AI crawls. Meta's training, search and agentic fetches run under meta-externalagent, meta-webindexer and meta-externalfetcher, which have their own pages.

What else do people ask?

What are preview bots?
Agents that fetch a page the moment someone shares its link, to build the preview card: title, description and image. facebookexternalhit and Slackbot-LinkExpanding are the documented examples here.
Should I block facebookexternalhit?
Usually not. It is what renders your share card on Facebook, Instagram and Messenger, Meta documents that it may bypass robots.txt anyway, and blocking it at the network layer means every share of your page posts without a card.
Does Slack honor robots.txt?
Slack states that it does not honor robots.txt for its link unfurler. Its documented route to being excluded is contacting Slack to be added to a blocklist.
Is facebookexternalhit used for Meta AI training?
Meta documents link previews as its job. Meta's training crawler is meta-externalagent and Meta AI's search crawler is meta-webindexer, both separate tokens with their own robots.txt groups.

Where do these facts come from?

2 claims on this page, each with the page it was read from; last checked .

  1. Meta documents the primary purpose of FacebookExternalHit as crawling the content of an app or website that was shared on one of Meta's family of apps, such as Facebook, Instagram or Messenger.

    developers.facebook.com · retrieved

    “The primary purpose of FacebookExternalHit is to crawl the content of an app or website that was shared on one of Meta's family of apps”

  2. Slack documents Slackbot-LinkExpanding as the robot that responds to links Slack users post into their channels, fetching as little of the page as it can using HTTP Range headers to extract oEmbed and Twitter Card or Open Graph meta tags.

    api.slack.com · retrieved

    “This robot responds to links that Slack users post into their channels.”

Want whatever it takes to win the answer?

Start freeBook a demo

7 days free · card required