# web robot/crawler/spider/ant file

User-agent: *
Disallow: /cgi-bin
Disallow: /test
Disallow: /yd
Disallow: /tap