FactoryFoxBot, our crawler

FactoryFoxBot is the web crawler of FactoryFox, a directory that helps buyers and their AI assistants find industrial suppliers. It visits company websites to keep each supplier's profile accurate: what the company makes, which services it offers, where it is and how to reach its sales team. It is operated by Monemetrics OÜ, Sepapaja 6, 15551 Tallinn, Estonia.

How to recognise it

Every request identifies itself with this User-Agent:

FactoryFoxBot/0.1 (+https://getfactoryfox.com/bot)

Requests are also signed with Web Bot Auth HTTP message signatures (Ed25519). They carry Signature-Agent: "https://getfactoryfox.com", and you can check the signature against our public keys at https://getfactoryfox.com/.well-known/http-message-signatures-directory. A request that claims to be FactoryFoxBot without a valid signature is not from us.

What it does

It starts at a company's home page and follows links on the same website to pages such as products, services, about, contact and news. It visits at most 12 pages and makes at most 100 requests per website per visit, and stops after 3 minutes.
It reads public pages only. It doesn't log in, and it doesn't fill in contact, search, order or sign-up forms. The one exception: on sites that switch language through a form, it may send that language switch to read the English edition.
Some pages need JavaScript, so it may render them in a headless browser and press buttons such as "read more" or a cookie banner's "reject" to reveal text. It doesn't load images, video or fonts there, and it doesn't contact third-party sites.
It collects business information: company name, addresses, general phone numbers and email addresses, products, services, certifications and news. It doesn't collect named people's contact details for publication (see our privacy policy).
FactoryFox shows that information on supplier profiles with a link back to the source website. It isn't used to train AI models.

robots.txt and crawl rate

FactoryFoxBot obeys robots.txt. It follows a group for FactoryFoxBot if there is one, otherwise the * group. If your robots.txt can't be read because of a server error or rate limit, it doesn't crawl the site.
It sends one request at a time to a website, at least one second apart, and honours Crawl-delay when it is longer.
Each website is revisited from time to time to keep its profile current, not continuously.

To keep FactoryFoxBot out of your whole site:

User-agent: FactoryFoxBot
Disallow: /

AI-visibility audits

When someone orders an AI-visibility audit of a website, FactoryFoxBot fetches a few of that site's pages in the same way. To report whether AI assistants can reach the site, the audit also requests its home page once each with the User-Agents of common AI crawlers and assistants, without a signature. Those few requests are not FactoryFoxBot crawling and are not made outside an audit.

Contact

To report a problem, ask us to slow down or stop, or have information about your company corrected or removed, email privacy@getfactoryfox.com. Companies can also claim their profile on FactoryFox and edit it themselves.