Huddle API integration

Project Outline We need a program to automatically copy files and folders stored in Huddle ([login to view URL]) workspaces to a local filesystem. There are around 500 workspaces with an unknown number of files totalling around 19Tb in size. The program will be run on a Windows computer with sufficient file storage space for the full contents of Huddle to be copied to it, and it will use the NTFS filesystem. The program to be developed must: • copy, in a single run of the program and at the user’s option: o a single Huddle workspace, o a list of Huddle workspaces, or o all workspaces in our Huddle account • copy in a single direction, from Huddle to a designated folder on the local filesystem • each file and folder must be given the same name and file path within the designated local filesystem folder as it has in Huddle • each workspace in Huddle must be copied to a subfolder with the same name within the designated local filesystem folder, creating the subfolder if it is absent Detailed Requirements In addition to the Key Requirements listed above, the program to be developed must: • be able to authenticate to Huddle (with manual assistance to provide login details, if required) in order to obtain any required authentication tokens such as API Keys • maintain the validity of authentication tokens while it is running (for example, by utilising any OAuth2 Refresh Tokens received to acquire new Access Tokens), in order that the program does not require further manual intervention during a single run • manual assistance to retrieve authentication tokens must occur at most once per program run and ideally should not be required again until long-lived tokens issued by Huddle expire • work as fast as possible while managing copying workload to conform with API rate limits, bandwidth limits and any other limits enforced by Huddle • deal gracefully with errors wherever practical so that execution of the program is not halted and processing steps are not missed • be re-startable in the event of an error or any other failure or interruption; i.e. that the program, when run again with the same inputs, is able to resume from the point at which previous execution halted without repeating a significant amount of previous processing and/or copying – we suggest this should be done by comparing file modification times and file sizes in the local filesystem with those in Huddle o If a local file has a different length from equivalent file in Huddle, overwrite it with the file contents from Huddle o If a local file has the same length as the equivalent file in Huddle, check its modification time  If the local file was modified before the file in Huddle, consider it to be out-of-date and overwrite it with the file contents from Huddle  Otherwise consider the local file to be up-to-date and do not copy from Huddle • must check for file and folder names that are not valid in the local filesystem (NTFS) and must translate any disallowed characters into allowed ones – exact translation scheme to be agreed in writing with us • if file/folder name or full path length restrictions in the local filesystem prevent any files being copied from Huddle, agree in writing with us and implement a scheme to truncate file/folder names and/or file paths in a consistent, repeatable way that avoids creating conflicting or ambiguous file/folder names and avoids overwriting the contents of one file with another • if copying a file fails because of a temporary condition, retry the copy 3 times, resuming an interrupted, partial copy (see [login to view URL]) You may find the following documentation useful: • [login to view URL] • [login to view URL] • [login to view URL]