# robots.txt to slow down RavenCrawler User-agent: RavenCrawler Crawl-delay: 5