If you run a mailing list or send automated emails with links, you may have noticed something strange: Googlebot is following links sent to Gmail subscribers, which can trigger unsubscribe actions or inflate download counts.
Why Common Solutions Don't Work
The typical approach—using robots.txt or adding rel="nofollow" to links—doesn't work for email links. These methods only apply to web pages, not email messages. When Gmail displays emails in the web interface, Googlebot can still crawl those links.
The Solution: User Agent Filtering
The best approach is to detect and handle Googlebot requests at the server level. Check the user agent string in incoming requests and filter out bot traffic before processing actions like unsubscribes or incrementing download counters.