creative web cam software safari web browser creative web cam vf0050 web accelerator outlook web access pokemon ruby rom pokemon ruby
Description - Ruby web crawler
This Ruby script reads in a list of links from links.dat, it then picks out the ones it can easily spider and gets a list of URLs from each PaGE listed in links.dat. Every new URL it finds will be added to newlinks.dat for later spidering by another bot running along side this one.