The Zuellni crawler (ZuellniBot)

If you found ZuellniBot in your server’s logs: it belongs to Zuellni, a search for official sources. It is used only on sites that are listed, or are being considered for listing, to check that an address is still the one it was and to read the titles of a site’s own documentation and news pages.

How it behaves

• It identifies itself: ZuellniBot/0.0.1 (+https://zuellni.com/bot; index of official sources; honours robots.txt).
• It reads /robots.txt first and follows it, including Crawl-delay. It reads only the sitemaps your robots.txt names (or /sitemap.xml) and the pages they list.
• At most one request per second to your site, only https addresses, never your login-protected areas.
• It stores the address, the page title, a short description, the headings and the date of a page. It does not store page bodies, and it removes email addresses from what it stores.

Stop it

Add to your robots.txt:

User-agent: ZuellniBot
Disallow: /

It will not fetch your pages again. That is all it takes. A contact address for removal requests and problems with the crawler does not exist yet; it will be published on this page before the project is announced.