BeautifulSoup Jobs

I am building a real estate “market radar” for Uruguay that identifies undervalued properties and potentially motivated sellers. The goal of this project is to create a data workflow that automatically collects property listings from real estate portals and organizes the data so it can be analyzed in Excel. Primary data sources: • MercadoLibre Inmuebles (primary source) • InfoCasas Uruguay (secondary source, optional) No public records or auction data are required at this stage. PROJECT SCOPE The system should automatically collect property listings and store the data in a structured dataset. The scraper should run once per day and collect the following fields: • listing ID or unique identifier • property price • property location or neighborhoo...

I have around 7,000 Aliexpress products that I need fully harvested for content-creation purposes. From each listing I only require the official product photos and any product videos—no customer review photos. Everything should come back to me in ready-to-use JPEG for images and MP4 for videos, preserved at the highest resolution Aliexpress serves. Please pull all files directly from the product gallery and video carousel, avoiding watermarks or compression wherever possible. A light file-name convention that ties each asset to its product URL or SKU will make downstream editing much easier for me. Deliverables: • Folder structure or archive segmented by product (one folder per listing). • Inside each folder: all JPEG images and any MP4 videos found. • A simple C...

I need a reliable, one-time scrape of roughly 300 public education websites that all share a similar page structure. Each site lists between 5 and 200 staff contacts (average ≈50). For every person you find, capture four fields: • full name • job title • email address • exact source URL where the data appears All pages are publicly accessible—no authentication hurdles—so the script can run headless without session handling. Please deliver: 1. A consolidated Excel file (.xlsx) containing every contact, with clear column headers and either a “Site” column or separate tabs—whichever keeps the data easiest to filter. 2. The scraper’s source code (Python with Scrapy, BeautifulSoup, or similar is fine) plus a brief REA...

I need a small, reliable program that can scan Google Search results, spot which ads are running, and export the findings to a CSV. The workflow is simple: I type a niche and a city, hit run, and the tool crawls Google for professional-service businesses in major cities only. For every advertiser it detects, the CSV must list business name, Country, city, street address, email, phone, primary contact, and an estimated PPC budget. On top of that, the scrape should extract key campaign insights: • Budget • Keywords used • Ad placements A straightforward command-line script in Python is fine—Selenium, Scrapy, BeautifulSoup, SerpAPI, or the Google Ads API can all be leveraged so long as the solution stays within Google’s terms and reliably handles captchas...

No worries, that's way more workable. Here's the full post trimmed to fit under 10,000 characters: Nationwide Property Auction Web Scraping & Intelligent Alert System (Ongoing) About Us We're a commercial real estate investment firm that acquires distressed properties nationwide. We have the capital to close on any deal in the U.S. — our bottleneck is finding opportunities before competitors. We're building an automated system that monitors every property auction source in the country, filters against our criteria, and alerts us only on qualified deals. This is not a data dump project. We don't want spreadsheets with thousands of rows. We want a smart radar system that scans everything, filters ruthlessly, and only pings us when something matches. Long-t...

I need a lightweight, repeatable scraper that gathers every publicly visible customer review talking about Bayer from social-media sources—right now the focus is on Goole. The crawler should pull the full review text, star rating (or reaction score, if available), reviewer name or handle, date, and the direct URL to each post. Please build it so I can run it on demand, ideally from a simple command line or Jupyter notebook. Python with requests / BeautifulSoup, Selenium, or Scrapy is fine; if you prefer another stack, let me know why it would be a better fit. Deliverables • Clean, well-commented source code • One sample export in CSV or JSON showing at least 100 live reviews • A short README explaining environment setup, run instructions, and how to alter s...

I need a small, reliable tool that watches a few specific product URLs on the Hermes Luxembourg site and lets me know the moment the button changes from “View more” to “Add to cart.” Everything I want to track is public and this type of monitoring is fully permitted, so no concerns on that front. My ideal workflow is simple: • I enter or update the list of product links I care about. • The script pings those pages at a reasonable interval, detects the switch in stock status, and immediately triggers a push notification to my smartphone. A lightweight web-scraper with a clear, maintainable rule for that button-text change should be enough. If you prefer to use Python with requests/BeautifulSoup —or Playwright, Puppeteer, or another headless ap...

I need a Python-based scraper that pulls complete car-listing information from every day. At a minimum the script has to capture make, model, price, and mileage but, in practice, I want every publicly visible field on each listing so that nothing useful is missed. Here’s what matters to me: • Reliability – the code must navigate pagination, work around basic anti-bot measures (rotating user-agents / respectful delays), and throw clear errors if the site layout changes. • Clean output – save to CSV or an SQLite database with consistent column names, ready for later analysis. You’re free to choose libraries you trust (requests, BeautifulSoup, Selenium, Scrapy, Playwright, etc.); just document any setup steps and keep third-party dependencies to a mi...

No worries, that's way more workable. Here's the full post trimmed to fit under 10,000 characters: Nationwide Property Auction Web Scraping & Intelligent Alert System (Ongoing) About Us We're a commercial real estate investment firm that acquires distressed properties nationwide. We have the capital to close on any deal in the U.S. — our bottleneck is finding opportunities before competitors. We're building an automated system that monitors every property auction source in the country, filters against our criteria, and alerts us only on qualified deals. This is not a data dump project. We don't want spreadsheets with thousands of rows. We want a smart radar system that scans everything, filters ruthlessly, and only pings us when something matches. Long-t...

I need a Python-based solution that automatically gathers companies and shareholders data, pulls supplementary details via external APIs, and outputs a clean, unified dataset I can query at any time. Scope of the scrape • Sources: company websites, financial databases and relevant public records. • Website focus: company profiles, turnover figures and any available Demat / share-holding particulars. What the tool should do 1. Crawl or call the above sources, respecting and rate limits. 2. Parse the required fields, normalise names and IDs, then enrich each record through one or more APIs (for example OpenCorporates, Clearbit or any better suggestion you have). 3. Store results in a structured format (CSV plus an SQLite or Postgres option). 4. Offer a simple comma...

I need a reliable script or windows-application that automatically gathers text content from specified websites and online databases, then saves everything into a clean, well-structured CSV file. A Windows-software would be preferred. The crawler should be able to crawl the website and spider a list of urls for approval or automatically go through the website Or just scrape a given list of urls (from a txt-file) Key details • Sources: public-facing websites and shops (also with login using username:password) • Data type: text only—no images or binary files. • Output: one CSV per run, UTF-8 encoded, with a header row • should be able to read/exrtract data from !! various shops & websites !! -> generally i need a basic software + "plugins" fo...

I have to confirm whether specific street addresses qualify for the government-funded home-insulation programme. The Energy department website holds an “address eligibility” checker, and I need that information pulled automatically rather than re-typing each location by hand. Your task is to build and run a scraper that goes through the same steps the public tool requires, captures the eligibility result for every address I supply, and returns the full set in a clean Spreadsheet (Excel or CSV is fine). A repeatable script—Python with requests / BeautifulSoup or Selenium, or any language you are comfortable with—is preferred so I can rerun it later when the list of addresses grows. Handle captchas or session cookies if the site uses them, and respect polite crawlin...

We are looking for an experienced developer who can build an automated system to extract daily newly incorporated company data from the MCA (Ministry of Corporate Affairs) website – https://www.mca.gov.in. The system should automatically collect and deliver the list of companies incorporated each day in structured format (Excel / CSV / API / Database). Scope of Work: Develop a web scraping or API-based solution to extract daily incorporated company data from the MCA portal. The tool should automatically fetch newly incorporated companies every day. Data should include the following fields (minimum): CIN Company Name Date of Incorporation ROC (Registrar of Companies) State Company Type (Private Limited / LLP / OPC / Public Limited) Authorized Capital (if available) Regist...

We are looking for an experienced developer to build a robust web scraping solution capable of extracting structured data from a login-protected medical/drug repository website. The platform contains a large database of drug information (potentially hundreds of thousands to over a million pages). The scraper should be able to navigate through the website after login, systematically extract relevant drug data, and store it in a structured format. Scope of Work: Develop a scraper that can log into a protected website. Navigate through the drug repository pages. Extract structured information from each drug page. Handle pagination and large-scale crawling. Implement mechanisms to prevent crashes or interruptions during long scraping runs. Store extracted data in a structured format such as ...

I need a reliable, repeatable script that automatically pulls historical and fresh match-result data for the Premier League, La Liga, Serie A, the English Championship and the Bundesliga 1. The workflow should: • visit publicly available sources you identify (official league sites, APIs, or reputable statistics portals), • extract the full-time score, date, home/away sides, venue and any metadata you can pick up (round, referee, attendance), • extract data on goals including the exact time and goalscorer • additional data extracted from match Commentary would be helpful, i.e. substitutions, shots on goal, shots off target, etc. with times will help • normalise club names so they are consistent across all leagues, and • write everything into a single, tidy...

I have a curated list of specific company websites and I need an automated solution that extracts complete contact information from each one. The goal is to turn every URL into a clean, ready-to-use lead. WEBSITE : The scraper should capture: • Email addresses • Phone numbers • Mailing addresses • LinkedIn profile link • Location (city / state / country) • First and last name • Occupation / job title • Company name • Company website A well-structured CSV or Excel file is the preferred output, with each field in its own column. I am comfortable with your choice of tech—Python with BeautifulSoup, Scrapy, or Selenium are all fine—as long as the script runs reliably and respects and rate limits where required. Ac...