We have developed a very fast Caching engine. It Crawls / Spiders sites that you specify. It can take keyword parameter, and search for specific key words. For example you can search government websites, business directories, Ebay, yahoo Shops, other search engines like yahoo, google altavista etc.
A powerful web crawler utility to Extract
2) meta tag (title, description, keyword),plain text between to tag
3) page size
4) last modified date value
from , Web Sites, Web Directories, Search Engines!!
High speed, multi-threaded, accurate extraction - directly saves data to Database. Uses MS SQL at this point. You can use it with Access, MySQL etc..
Program has numerous filters to restrict session, like - URL filter, text filter, data filter, domain filter, date modified, etc. It allows user-selectable recursion levels, retrieval threads, timeout, proxy support and many other options.
A must have tool for webmasters who build Search Directories.
Contact me back, if you wish to buy it. Price is negotiable.