Anemone

Anemone is a Ruby library that makes it quick and painless to write programs that spider a website. It provides a simple DSL for performing actions on every PaGE of a site, skipping certain URLs, and calculating the shortest path to a given page on a site.The multi-threaded design makes Anemone fast. The API makes it simple. And the expressiveness of Ruby makes it powerful.