No Contracts You can stay providing you want or terminate at any time. You get to produce that selection, providing you are not funding a tool. txt file is then parsed and may instruct the robot as to which web pages will not be for being crawled. As being a https://www.youtube.com/watch?v=hgyQy18ZcIs