About ContactGraphBot

ContactGraphBot is the web crawler operated by ContactGraph, an Australian business-data company. We index publicly accessible business websites in Australia.

What the crawler does

• Fetches the homepage and a small number of public pages (typically < 10 per site) such as /about, /team, /privacy and /terms.
• Reads robots.txt before any other request and honours the rules declared for ContactGraphBot and for *.
• Observes any Crawl-delay directive; defaults to one request every ~500–1000 ms when none is specified.
• Requests HTML only; we do not download images, video, fonts, or stylesheets.
• Revisits each site at most a few times per month unless the site is known to update frequently.

What the crawler does not do

• It does not log in, submit forms, or access anything behind authentication.
• It does not bypass CAPTCHAs, WAF blocks, or rate limits. A 403, 429, or challenge page ends the visit.
• It does not impersonate a browser or any other crawler. The User-Agent above is always sent.
• It does not collect personal information beyond what a site publishes on public pages for business contact purposes.

How to block us

The preferred way is a robots.txt rule. To block ContactGraphBot from your entire site, add:

User-agent: ContactGraphBot
Disallow: /

To block a specific section, use a path prefix:

User-agent: ContactGraphBot
Disallow: /private/
Disallow: /members/

We check robots.txt before every site visit, so the rule takes effect on the next scheduled crawl.

Permanent exclusion by request

If you would like your domain added to our permanent exclusion list — so that we will not crawl it again even if robots.txt later changes — email abuse@contactgraph.io from an address at the domain in question (or that can be clearly linked to the domain owner) with:

• The domain(s) you want excluded.
• Whether you also want any previously collected data about the domain removed from our database.

We will action the request within 10 business days and confirm by reply.

Data removal and privacy enquiries

Enquiries about personal information held by ContactGraph — including access and correction requests under the Privacy Act 1988 (Cth) — should be sent to privacy@contactgraph.io. We respond within 30 days as required by Australian Privacy Principle 12.

How to verify a request really came from us

Anyone can put ContactGraphBot in a User-Agent string. If you want to confirm that a request in your access logs is genuinely ours, check the source IP. Our crawler operates from a fixed set of IP ranges; if you need the current list for allow/deny-listing, email abuse@contactgraph.io. Requests from IPs outside those ranges that claim to be ContactGraphBot are not ours.