URL spider and domain extractor script

You are going to love this script!  A simple, yet powerful cgi script with mysql database interface that will allow you to spider the internet and collect unique urls and domain names at the speed of light!  5 minutes to configure, set your starting url and the spider will start probing the internet from the starting url collecting every single link and unique domain name it finds along the way.  A great way to construct targeted marketing databases or start your own search engine.  Script allows you to select spidering depth or just leave it until your hard drive explodes with data.  This script is a masterpiece. 

System Requirements

• Perl 5
• Telnet access
• Mysql Database Software
• Msql-Mysql Module
• DBI Module
• Unix Style Operating System

Features

• Sorts out all non unique urls and domain names automatically
• Able to stop and restart spider from where you left off
• Mysql data storage for maximum efficiency and speed
• Customizable starting (seed) url target
• Customizable spidering depth

Get More Fantastic Scripts Here