Blueperry Ltd web crawler
This page describes how webmasters can control how PartBot interacts with their site. Changes to your robots.txt can take up to 24 hours to take effect.
PartBot
PartBot collects publicly listed product information: prices, stock availability, delivery estimates and manufacturer part numbers. Blueperry Ltd uses this information for price comparison and parts sourcing. PartBot only visits public pages. It doesn't log in, submit forms or place orders, and it doesn't crawl content for generative AI model training.
- User Agent String:
Mozilla/5.0 (compatible; PartBot/1.0; +https://bpws.co.uk/bot) - Request Signing Keys: https://bpws.co.uk/.well-known/http-message-signatures-directory
Our Approach to Robots.txt
PartBot follows the Robots Exclusion Protocol. It honours
the user-agent, allow, disallow and crawl-delay directives. Use
User-agent: PartBot to give it its own rules. PartBot fetches robots.txt before crawling and uses a cached
copy for no longer than 24 hours. If robots.txt returns a server error or can't be reached, PartBot doesn't crawl the
site. If the file doesn't exist, PartBot treats the site as open to crawling.
PartBot reads robots.txt separately for each host. For example.com it reads
example.com/robots.txt, and for shop.example.com it reads shop.example.com/robots.txt.
It checks robots.txt again whenever it follows a redirect.
On each page, PartBot honours the link-level rel=nofollow attribute. It also honours the
noindex, nofollow and none directives in robots meta tags and the
X-Robots-Tag header. It sends one request at a time to each site, at least 2 seconds apart.
It slows down when it receives 429 or 503 responses.
Verifying PartBot
Every PartBot request is signed with HTTP Message Signatures (Web Bot Auth), using the public keys listed above. If a request says it's PartBot but doesn't carry a valid signature, it didn't come from us.
Contact Us
If you're a content owner or publisher and have questions, please contact us at [email protected]. Please include the relevant domain names in your message.