I am looking for someone to write a highly efficient, multi threaded web crawler that will create an index of MP3 files, complete with audio file information such as Artist, Track Title, and Length gathered from the file's metadata. This crawler would be identical to the crawler that is behind the [url removed, login to view] website.
The project must be written in C#.
The crawler must be multi-threaded.
The crawler must be efficient and not waste time re-indexing pages recently indexed, ect.
The index must be stored in a MS SQL 2005 database.
The crawler must follow [url removed, login to view] files.
The crawler must read these MP3ID fields from the files if present:
- Track Title
- Artist Name
- Album Name
- Track Length
- Track Number