TagwatchBot

TagWatch customers ask us to watch one or more pages of their own website, or a website they manage for a client, and to tell them when a tag stops loading: Google Analytics, Google Tag Manager, the Meta pixel and the like. TagwatchBot opens each of those pages in a real browser, accepts the cookie banner the way a visitor would, scrolls down the page and records which tags load.

It visits only the pages a customer configured. It does not follow links, crawl a site, submit forms or collect content. Pages are normally checked once a day, and once more when a customer sets up a monitor or asks for a check by hand.

Every request TagwatchBot sends to the website it checks is signed with Web Bot Auth (HTTP Message Signatures, RFC 9421). The request carries a Signature-Agent header with the value "https://tagwatch.grip-analytics.com", and the signature verifies against the public key published at https://tagwatch.grip-analytics.com/.well-known/http-message-signatures-directory.

It runs from cloud servers without fixed IP addresses, so the signature, not the address, is how to tell it apart. Requests to third parties the page loads (Google, Meta and other vendors) are not signed.

TagwatchBot obeys robots.txt rules that name it. To keep it out of your site, or part of it, add a group for its product token:

User-agent: TagwatchBot
Disallow: /

A generic User-agent: * group is not applied. Like an uptime monitor, TagwatchBot visits a page because the site's owner, or someone they asked, told it to, and a rule written to keep crawlers out of a whole site is not meant for that. A page that robots.txt closes to TagwatchBot is not visited, and the customer is told that the site does not allow the check.

TagwatchBot signs its requests so that Cloudflare can recognise it as a verified bot, and bot settings that allow verified bots let it through. A custom rule that challenges every visitor, or Under Attack mode, stops it like any other visitor; add an exception for verified bots, or for the signature agent above, to keep the checks running.

Questions about TagwatchBot, or traffic from it you did not expect? Email [email protected].