# All robots will spider the domain User-agent: * Disallow: # Disallow directories Disallow: /P006/ Disallow: /P01/ Disallow: /P012/ Disallow: /i/ Disallow: /lists/ Disallow: /mt/ Disallow: /opendb/ Disallow: /phpMyAdmin/ Disallow: /print/ Disallow: /stats/ Disallow: /vservers/ Disallow: /tw/ Disallow: /awstatsicons/ Disallow: /geoip/ Disallow: /include/ Disallow: /P103/ Disallow: /selector/ Disallow: /webstats/