BANDWIDTH
EXCESSES:
Googlebot is a main offender.
You can slow this robot by placing a robot.txt file with the
following:
User-agent: googlebot
Crawl-delay: 180
or
in the Header of your pages place the following tags:
<META NAME="ROBOTS" CONTENT="index, follow">
<META NAME="REVISIT-AFTER" CONTENT="1 Week">
