# Robots.txt file from http://www.2shequ.com # All robots will spider the domain User-agent: * Disallow: /log/ Disallow: /bin/ Disallow: /config/ Disallow: /ajaxpage/