Automated PDF Downloader for Website

This project is an automated PDF downloader for a logged-in website. The goal is to let a user log in to a website manually, including OTP or 2FA if needed, then use a tool to automatically navigate the website’s content tree and download thousands of PDF files using the website’s own PDF download button. The tool is designed so the user does not need to work in Terminal. It includes a double-click Mac launcher that opens a local dashboard in the browser. From that dashboard, the user can control the full workflow: login, record the website structure, discover available files, download PDFs, and generate reports. **Main Requirement** The website has a content/sidebar tree. When a section is clicked, it expands and shows more pages or products. When a final page/product is clicked, its information loads on the right side of the website. That page has a dedicated PDF download button. The tool must go through all relevant pages/products one by one and download the PDF for each one. It should preserve the same folder structure as shown on the website and use the filename provided by the website for each downloaded PDF. **User Workflow** 1. The user double-clicks `Launch IP [login to view URL]`. 2. A local dashboard opens in the browser. 3. The user clicks the login button and logs into the website manually. 4. After login, the tool saves the browser session locally so credentials do not need to be stored. 5. The user records the website layout by clicking: - the content tree container, - one expandable folder, - one final page/product item, - the PDF download button. 6. The tool scans the full content tree and creates a manifest of all downloadable items. 7. The user starts the download job. 8. The tool downloads all PDFs, tracks progress, retries failures, and creates a report. **Important Features** - Manual login support for OTP/2FA. - No credentials are stored. - Double-click launcher for non-technical usage. - Browser dashboard instead of command-line operation. - Website structure discovery. - Automatic PDF downloading using the website’s own download button. - Preserves website folder hierarchy locally. - Keeps the original filename provided by the website. - Does not overwrite existing files; creates timestamped versions instead. - Maintains a detailed manifest and report. - Supports retrying failed downloads. - Supports resume, so completed files do not need to be downloaded again. - Uses adaptive concurrency to improve speed while reducing risk of website throttling. **Technical Stack** The project is built with Node.js and Playwright. Playwright controls Chromium to automate browser actions such as opening the website, clicking items in the content tree, and capturing file downloads. A local Node.js server powers the dashboard UI. The Mac launcher starts this local server and opens the dashboard automatically. **Generated Data** The tool creates: - `.ipdownloader/` for local session and site configuration. - `data/[login to view URL]`, `[login to view URL]`, and `[login to view URL]` for discovered content and download status. - `downloads/` for the downloaded PDFs. - `logs/` for local dashboard logs. **Success Criteria** The project is successful when a user can: - double-click the launcher, - log in manually, - teach the tool the website structure, - discover all downloadable pages/products, - download thousands of PDFs automatically, - preserve the website folder structure, - retain original PDF filenames, - resume interrupted downloads, - and review a report of completed and failed files.

Project ID: 40552056