AI crawler
DuckAssistBot: what it does and how to control it.
DuckDuckGo's crawler for AI-assisted answers. What it does with your pages, how to spot it in your logs and what a robots.txt block costs you.
Short answer
What is this DuckDuckGo crawler?
DuckAssistBot is a web crawler for DuckDuckGo Search. It crawls pages in real time to power DuckDuckGo's AI-assisted answers, which cite their sources. DuckDuckGo says the data is not used to train AI models.
In brief
Four things to know first.
Answers, not training
DuckDuckGo says this crawler's data is not used in any way to train AI models.
Easy to recognise
It uses a user agent ending in /1.2 and crawls from IP addresses DuckDuckGo publishes.
One robots.txt line controls it
Disallow this crawler's user agent. The change takes effect after 72 hours.
Search rankings stay put
DuckDuckGo says opting out does not affect organic rankings or whether you appear in its results.
What it does
DuckDuckGo's crawler for AI answers.
According to DuckDuckGo's help page, this crawler fetches pages in real time for its AI-assisted answers. Those answers prominently cite their sources, so a crawled page can appear as a cited source.
| Item | What DuckDuckGo says |
|---|---|
| Operator | DuckDuckGo Search |
| Purpose | Real-time crawling for AI-assisted answers |
| Model training | Data is not used to train AI models |
| Citations | Answers prominently cite their sources |
| Search rankings | Unaffected by opting out |
Recognise it
Spot DuckAssistBot in your logs.
According to DuckDuckGo, the crawler's user agent shows the bot's name with version 1.2, followed by a link to its DuckDuckGo help page.
Anyone can fake a user agent. DuckDuckGo publishes the crawler's IP addresses, also as JSON, so you can check a request against the list.
Find the user agent
Search your access logs for the user agent string DuckDuckGo publishes for this crawler, including its version number 1.2.Fetch the IP list
Download the JSON file DuckDuckGo publishes for this crawler's IP addresses.Match the address
Treat requests from addresses not on the list as impostors.
Control it
Allow or block it in robots.txt.
DuckDuckGo says you can opt out of being a source for AI-assisted answers by editing your domain's robots.txt. Disallow this crawler's user agent.
The change takes effect after 72 hours, and the crawler then stops visiting your site. For opt-out problems, DuckDuckGo lists crawling@duckduckgo.com.
| Allow it | Block it | |
|---|---|---|
| robots.txt | No rule needed; leave it out | User-agent line naming the bot, then Disallow: / |
| AI answers | Pages can be a cited source | You opt out of being a source |
| Search results | No change | No change, per DuckDuckGo |
| Timing | Immediate | Takes effect after 72 hours |
Not sure which AI crawlers reach you?
We check crawler access, rendering and schema so the pages you want cited can be read.
Your decision
Should you block it?
DuckDuckGo says this crawler's data does not train AI models. Blocking it in your robots.txt file stops your pages from being sources in DuckDuckGo's AI-assisted answers.
Allow the crawler if you want your pages to be possible sources in DuckDuckGo's AI-assisted answers. Check each AI crawler in our robots.txt guide. Then confirm your pages are readable with a technical GEO review.
Allow if you want citations
Crawled pages can be cited in DuckDuckGo's AI-assisted answers.
Block if you want out
One robots.txt rule removes you as a source for those answers.
Rankings are separate
Per DuckDuckGo, the opt-out leaves search results untouched.
FAQ
DuckAssistBot: common questions.
What is this DuckDuckGo crawler?
DuckAssistBot is a web crawler for DuckDuckGo Search. The bot crawls pages in real time to power DuckDuckGo's AI-assisted answers, and those answers prominently cite their sources. DuckDuckGo describes the crawler on its own help page.
Does this crawler train AI models?
DuckDuckGo says no. Its help page states that the data this crawler collects is not used in any way to train AI models. The crawler serves AI-assisted answers, which cite the pages they draw on.
How do I block this crawler in robots.txt?
Add a robots.txt group that names the DuckDuckGo AI crawler as its user agent and sets Disallow: /. DuckDuckGo says the change takes effect after 72 hours, after which the crawler stops visiting your site. Contact crawling@duckduckgo.com if the opt-out fails.
Does blocking this crawler hurt my DuckDuckGo rankings?
DuckDuckGo says no. Opting out does not affect organic search rankings and does not change whether your site appears in its search results. It only removes you as a potential source for AI-assisted answers.
How can I verify a request really comes from this crawler?
Compare the request's IP address with the list DuckDuckGo publishes, which is also available as a JSON file on its site. The user agent string alone proves nothing, because anyone can send it.
Find out which AI answers cite your pages.
We run your buyers' questions through ChatGPT, Claude, Perplexity and Gemini and show you the gap.