Sync a local file store by screen scraping a website

We are a data integration company based in Melbourne Australia and we are interested in running our first Freelancer task. The developer should be familiar with making Http requests from C#.Net 1. Overview A C#.Net application is required to run on MS Azure. It can run as a service or some other style of application, and it will call a set of URLs and screen-scrape the html responses in order to sync the data files on the website with a local store. Note that the client has full legal access to those folders and files, and that this is simply automating a manual process. The application will be called by a scheduled task every n hours, and the objective is to keep the local file/folder store in sync with the remote store. 2. Authorisation The application should navigate to the URL, which will be provided in the detailed specifications. Normally when you manually navigate to that page you would be prompted for a username/password. Instead, these credentials will be read from a config file and passed via the http request header (or similar). You will be assigned these credentials as part of the detailed specification. 3. Screen scraping The URL above points to a tree structure. Each folder in that tree should be opened recursively. The tree nodes will contain either further tree nodes or a list of files. This tree structure of folders and files should be replicated on the local machine, however only csv files need to be downloaded. If new folders or csv files are encountered, or existing ones are updated, these should be downloaded. You do not need to worry about deletions. 4. Technical constraints As mentioned, the service should run on MS Azure. It can be run as a scheduled task, meaning that you do not need to build timers/scheduling into the system (this will be done by the scheduler). The application should be written in C# .Net. An SQL database will be available if required to store the tracking of files and folders, or this could be done by simply using a JSON file or XML file stored on the system. To be completed in the next couple of weeks.