First, this is NOT a WEB CRAWLER!
To briefly explain what I need (of course there are more details):
I need a script to get ALL links from a given URL/page regardless of where/how they are generated from, not just static hyperlinks. For example, the links could be generated by a javascript of some feed. It must get ALL links. After we get the links, we will need to manipulate them in some ways, I will explain this later.
Thank you for looking.