We compare agencies using their own public websites: live job adverts and pages about their services. If your site's security blocks our crawler, we can't include what's on your site when we compare agencies. It's free to fix.
HealthAgenciesBot. It only comes from 13.42.169.219 (current list). One request at a time, with a pause between requests; it obeys robots.txt and never tries to get around a bot check. More about it.Where your tool lets you, allow by IP address and user agent together. Anyone can copy a user agent, but genuine requests from us only ever come from our IP address. If your tool only takes IP addresses, that alone is fine.
Cloudflare
Pro, Business or Enterprise plan: a Skip rule
- Go to Security → WAF → Custom rules → Create rule (on the newer dashboard: Security → Security rules → Create rule → Custom rules).
- Name it Allow HealthAgenciesBot, choose Edit expression and paste:
(http.user_agent contains "HealthAgenciesBot" and ip.src in {13.42.169.219}) - Action: Skip. Tick All remaining custom rules, All rate limiting rules, All Super Bot Fight Mode rules, and under More components to skip: Browser Integrity Check, Security Level, User Agent Blocking. Tick All managed rules only if those were blocking us.
- Place it first, then Deploy.
Free plan
On the free plan, Bot Fight Mode can't be skipped by a rule. Either:
- Go to Security → WAF → Tools → IP Access Rules (newer dashboard: Security → Security rules → IP access rules), enter our IP address (
13.42.169.219), action Allow, This website; or - turn off Bot Fight Mode under Security → Bots (newer dashboard: Security → Settings).
Also check: "Block AI bots"
Under Security → Bots (or Security → Settings on the newer dashboard), Cloudflare can block known AI crawlers. Ours isn't on that list and doesn't train AI models, so you can leave this setting on.
WordPress
Wordfence
Wordfence → Firewall → All Firewall Options → Advanced Firewall Options → Allowlisted IP addresses that bypass all rules: add 13.42.169.219, then Save Changes. Wordfence rarely blocks us (we stay well under its rate limits), so first check Wordfence → Tools → Live Traffic for our IP address; only add it if you see it blocked.
Sucuri
Sucuri dashboard → your site → Firewall → Access Control → Allow IP Addresses: add 13.42.169.219.
Solid Security (iThemes), All In One WP Security
Add 13.42.169.219 to the plugin's list of allowed (sometimes called whitelisted) IP addresses.
Other hosts and firewalls
Your host or web developer
Send them this: "Please allowlist IP 13.42.169.219 for user agent HealthAgenciesBot (details: healthagencies.co.uk/about/bot)."This is all they need, whether you are with SiteGround, WP Engine, Kinsta, 20i or a recruitment website platform such as Volcanic or a Bullhorn portal.
Vercel, AWS WAF, Akamai, Imperva, DataDome
Add a custom rule that matches IP address 13.42.169.219 and user agent contains HealthAgenciesBot, with the action Bypass or Allow, placed before your bot-protection rules.
robots.txt
We obey robots.txt. If it blocks us, we can't read your site. To allow us but keep the load low:
User-agent: HealthAgenciesBot Crawl-delay: 5 Disallow: /admin/ Allow: /
If your robots.txt has User-agent: * with Disallow: /, add the lines above as well; the group that names us takes precedence.
Still blocked?
Tell us and we'll read your site again: bot@healthagencies.co.uk, report an error, or use your agency account. You can also check your firewall log for requests from 13.42.169.219 with HealthAgenciesBot in the user agent: they should show as allowed or skipped.