Twitter.com/robots.txt # Crawl-delay: 10 -- Googlebot ignores crawl-delay ftltwitter.com2 pointsguynamedloren15 years ago