Improving on Robots Exclusion Protocol (from Google Webmaster Blog)googlewebmastercentral.blogspot.com1 pointandreyf18 years ago