Extend Open-Source Australian Grocery Price Scraper for Grocery (Python)

@Microlent

Hello I have reviewed the existing repository and understand exactly what needs to be done here. This is a well-scoped project and the open-source base gives us a solid starting point to extend rather than rebuild from scratch. The core challenge is not the scraping itself but the product normalisation and canonical matching across stores. Getting "Dairy Farmers Full Cream Milk 2L" to reliably resolve to the same product ID whether it comes from Woolworths or Coles requires a thoughtful cleaning pipeline, and I will build that with care. Here is how I will approach each phase: - Data Collection: Extend the existing spiders for Woolworths and Coles with improved pagination, category traversal, and rotation handling to reduce block rates. Scheduler-ready from day one, designed for weekly runs extending to daily. - Normalisation Pipeline: Standardised product names, brand spelling, size and unit formats all resolved before matching. Canonical product records created with fuzzy matching to handle minor naming differences across stores. - Storage: Versioned compressed JSON datasets with the exact schema you specified, including promo pricing and unit price fields. Tech will be Python only using requests, BeautifulSoup4, and pandas as specified. No heavy frameworks. Happy to start with a pull request on the normalisation pipeline so you can review the approach before full build begins. Best, Jenifer

@ZohaibRoy

⭐⭐⭐⭐⭐ Enhance Your Grocery App for Price Comparison and Data Management ❇️ Hi My Friend, I hope you are doing well. I've reviewed your project requirements and see you are looking for a developer to improve your grocery price comparison app. Look no further; Zohaib is here to assist you! My team has successfully completed 50+ similar projects for data scraping and management. I will enhance the existing project to ensure reliable data collection, cleaning, and structuring for easy product comparison. ➡️ Why Me? I can easily improve your grocery app as I have 5 years of experience in Python development, specializing in web scraping, data management, and normalization. My expertise includes data collection, automation, and working with JSON datasets. I also have a strong grip on relevant technologies, ensuring a comprehensive and efficient approach to your project. ➡️ Let's have a quick chat to discuss your project in detail and let me show you samples of my previous work. Looking forward to discussing this with you! ➡️ Skills & Experience: ✅ Python Development ✅ Web Scraping ✅ Data Normalization ✅ JSON Data Handling ✅ Automation with Cron ✅ Data Cleaning Pipelines ✅ Error Logging ✅ API Integration ✅ BeautifulSoup ✅ Requests Library ✅ Data Structuring ✅ Documentation Writing Waiting for your response! Best Regards, Zohaib

@Rekhathakur

As an organization with a stellar reputation for delivering on-time projects, including complex Python development like the one at hand, I am uniquely positioned to tackle your project. Our expertise in web scraping, data cleaning, and structured data management ensures that we are the best suited to extend and improve the existing open-source grocery price scraper. Further, our technical mastery in Python, BeautifulSoup4, and Pandas guarantees a seamless integration of this solution into your ecosystem. We understand the significance of consistently reliable product comparison data for your platform. Building on our experience with creating scalable B2B and B2C solutions in diverse niches such as Fintech and eCommerce, we'll ensure that the scraper runs automatically on a schedule you prefer – whether weekly or even daily – and handles pagination, category traversal, and anti-bot protections where necessary. Moreover, our approach aligns perfectly with yours in maintaining a detailed record of changes, new products, and removed products between runs. This stems from our keen dedication to quality assurance and comprehensive project management. thanks.....

@hireageek

Hi there, I'm excited about the opportunity to extend the Australian grocery price scraper for your grocery comparison app. With extensive experience in data mining and web scraping, particularly using Python, I’m confident in my ability to enhance the existing repository. My goal will be to ensure reliable and efficient data collection from Woolworths and Coles, implement robust data normalization processes, and establish a seamless workflow automation for your project. I understand that the key requirements include improving the data collection methods, creating a standardized dataset for price comparison, and developing a user-friendly command interface. I will ensure that the dataset is structured in a way that allows for easy comparisons, delivering a clean JSON output as specified. Please message me to discuss further details or any specific features you'd like to prioritize in the initial stages. What specific features of the scraper would you like to prioritize in the initial implementation?

@Friends4it

I have strong Python scraping experience and have reviewed the aus_grocery_price_database GitHub repo - I can extend it effectively. My approach: - Fork the existing repo, review all existing Woolworths + Coles spiders - Fix broken selectors, add missing category traversal + pagination - Add anti-bot handling (rotating user agents, request delays, optional proxy support) - Build a normalisation pipeline: standardise product names, brands, units (g/kg/ml/L), sizes - Create canonical product matching so same item across stores shares one product ID - Weekly scheduler (cron/APScheduler) to run automatically - Clean output: CSV + SQLite or PostgreSQL as needed A$500 / 14 days. Can start immediately.

@devendrathakur12

Hello, I specialize in data scraping systems and built & customized large scale price comparison datasets. The main challenge here is matching the same product across stores while keeping scrapers stable. I am certified in Python development and I will solve this using requests, BeautifulSoup, and pandas with a clean normalization pipeline and cron workflow. Should product matching rely on brand + size rules or fuzzy NLP similarity? Do you plan to add Aldi later? Should price change reports also generate a simple API for the app? Best regards, Devendra S.

@DeveloperStation

Hi there, We’ve built a similar product called PriceHistory, where we scraped data from multiple sources and normalized it for price comparison. We also implemented a CI/CD pipeline to automate daily scraping and product matching. For your project, we can leverage our existing code and expertise to quickly deliver a robust solution. We’ve worked extensively with libraries like Scrapy, BeautifulSoup, Selenium, and Playwright, and we’re well-versed in handling anti-bot measures. We can also integrate additional features like product similarity matching and barcode extraction using NLP. Let’s schedule a 10-minute call to discuss your project in more detail and ensure I fully understand your requirements. I usually respond within 10 minutes. I’m eager to learn more about your exciting project. Best, Adil

@denisSolution

Hello client, I’ve carefully reviewed your job description and have strong experience in these Python, Data Mining, Web Scraping and Software Architecture. I can build a reliable web scraping solution tailored specifically to your needs. Whether using Node.js with Puppeteer/Cheerio or Python with Selenium/BeautifulSoup, I will extract, clean, and organize your data efficiently. I also handle anti-bot protections, pagination, and full automation as required. As you can see from my profile, my web scraping reviews are excellent, reflecting my commitment to quality work. I focus on writing clean, maintainable, and scalable code because I know the difference between 99% and 100%. If you hire me, I’ll do my best until you’re completely satisfied with the result. Let’s discuss your target website and preferred data format. Thanks, Denis

@mobimubasir

I am well-equipped to tackle the challenge of extending and enhancing the existing Australian grocery price scraper. Over the course of my decade-long career, I have successfully built and optimized web applications, including data collection and cleaning pipelines, which directly aligns with your project goals. I am used to working with tools like requests and beautifulsoup4, ensuring that I'm not only comfortable with the allowed libraries but also adept at keeping dependencies lightweight. I will ensure your vision of a reliable grocery price dataset comes to fruition. Collaborating with you throughout the process is something I deeply value – be it quick feedback on interim builds or review of pull requests. Together, we'll produce a clean JSON dataset with a bonus implementation of change detection functionality that will fuel your grocery comparison app enticingly perfect for Australian users’.

@sf264

Hello! I am a senior full stack developer having 5+ years of professional experience. After going through your project requirements in detail, I see that you need the open source project extended and improved so that it will reliably collect, clean, and structure grocery price data suitable for the product comparison application. I would love to chat with you to know more details about your project. Let's get started, Fahad.

@FaizalShaik80

Hi, I would love to help. I went through your project details and found that I worked on almost the exact same task about two months ago. I am an experienced and specialized freelancer with 6+ years of practical experience in Python, Web Scraping and I’m able to complete and deliver this project promptly. Feel free to visit my profile to check latest work and feedback from clients. Connect in chat to discuss details and next steps. Regards.

@amgadmourad18

Hi there, This project is very interesting — especially the product normalization and matching part, which is the core of any price comparison system. I can extend the existing repository and build a reliable scraping + data pipeline that collects prices from Woolworths and Coles, cleans the data, and produces a structured dataset where identical products share a single canonical ID. What I’ll focus on: Improving the scrapers for stable weekly/daily runs Building a clean normalization pipeline (brand, size, units) Matching products across stores into canonical records Exporting versioned JSON datasets for each run Adding a simple CLI workflow to run the entire pipeline Generating price change reports between runs The result will be a clean, structured grocery dataset ready to power your comparison app. I’m comfortable working with Python scraping pipelines and data normalization, and I’ll keep the solution lightweight and aligned with the existing repo structure. Happy to collaborate via PRs and iterate quickly as we build it.

@hotanloc

I have rich experienced in Web Scraping, Python programming. Firstly, I will extract product names and price via year from woolworths and save that with excel sheet. After that I will import that sheet using python and I will match these data and analysis with comparing prices via year and product name. I will display result some GUI. I have full experience to develop python code for extracting the specific data from web site with python, HTML, CSS and JavaScript. I am sure for your project and i can complete your project perfectly on time and with high quality. Please send me your message to discuss more about your project. I am waiting your reply now. Thanks.

@johnallenvillan1

Hi, I've gone through your goal of extending the aus_grocery_price_database repo and the need for both reliable scraping and robust product normalisation across Woolworths and Coles. I’ve delivered similar pipelines before, including a supermarket scraper where I built canonical product mapping that reduced mismatches by 40% and a versioned JSON dataset system for weekly ingestion. The main complexity here is aligning inconsistent product metadata between stores, brand fragments, size units, and naming noise. If this step isn’t engineered carefully, comparison logic becomes unreliable no matter how good the scraper is. I’ll reinforce the current spiders, implement pagination‑safe traversal, and add automated scheduling compatibility. Then I’ll build a normalisation layer that standardises names, brands, and units, followed by a fuzzy‑matching routine to generate stable canonical IDs. I’ll output compressed, versioned JSON snapshots and implement a diff step for price movements. Finally, I’ll add a clean CLI entry point and update the documentation. Before starting, I need to confirm preferred storage structure and whether promo detection should include multi‑buy logic. Thanks, John allen.

@Azursol

Dear , I am a seasoned Python developer with a strong background in web scraping and data processing. I have carefully reviewed your project requirements for extending the open-source Australian grocery price scraper and am confident in my ability to enhance the existing solution to meet your needs effectively. My approach involves improving data collection by enhancing spiders/modules, implementing a robust data cleaning pipeline for product normalization, ensuring efficient data storage, developing a CLI for workflow automation, and implementing change detection functionality. I will also update the documentation for seamless usability. I prioritize reliable delivery and transparent communication throughout the project. I am eager to discuss how we can collaborate to achieve your objectives efficiently. Thank you for considering my proposal. Warm regards,

@Techroj

Dear Client, Greetings!! Your grocery comparison project is exciting, and I’d loved to help enhance the existing repository into a reliable, automated dataa pipeline. I have strong experience in Python, web scraping, data normalisation, and building clean, maintainable workflows. I can improve the scrapers, implement canonical product matching, structure versioned JSON outputs, add price-change reporting, and deliver a simple CLI pipeline ready for cron scheduling all wiht lightweight dependencies. I truly enjoy building structured, data-driven systems and am confident I can create a solid foundation for your grocery app. Hope to hear from you soon!! Regards, Rojan .U

@kchoudhary95

As a skilled Python developer with a specific focus on automation and data manipulation, I believe I am the perfect candidate to extend and enhance your grocery price scraper. With over 7 years of experience in full-stack web development, I have developed a deep understanding of the intricacies involved in large-scale data collection, cleaning, and structuring. I am well-versed in developing robust spiders/modules that are capable of handling various complexities that may arise during scraping, such as pagination and anti-bot protections. Furthermore, deploying cron schedules to ensure the scraper runs reliably is second nature to me. My expertise in data manipulation and normalization is another strong asset for your project. I have previously worked on projects similar to yours where I successfully implemented pipelines that clean, standardize, and match product information across different sources. This will translate into a comprehensive grocery dataset that can power your upcoming comparison application effectively. To iterate, my propensity for clean coding, optimized performance, strong problem-solving abilities along with my experience in data-driven web development make me an ideal fit for this project. With my help, you can not only expect a reliable grocery price dataset but also a seamless integration with your existing repository structure. Looking forward to discussing your project details further!

@zhens2

Having worked as a Senior Full-Stack Developer for the past 8 years, I'm confident in my ability to take on the challenge of extending the open-source Australian grocery price scraper for your grocery app. Throughout my career, I've consistently built end-to-end web and software solutions for enterprises that required reliable, structured datasets comparable to your project. In regards to web scraping, I'm highly skilled in utilizing Python's requests and beautifulsoup4 libraries, which precisely aligns with the technical requirements of your project. More importantly, I've had extensive experience in handling pagination, category traversal, and anti-bot protections – all of which will be essential to ensure the scraper runs automatically on a regular schedule without any hiccups. Given my breadth of experience, technical prowess and deep understanding of your goals for this project; I believe we have a great opportunity here! Let's collaborate to build a reliable grocery price dataset together which will power your Grocery comparison app impeccably!

@HiraMahmood4072

Hello, With over 7 years of experience in Web Scraping, Python, and Data Mining, I have carefully reviewed your project requirements. I propose to extend the existing open-source Australian grocery price scraper by improving data collection, implementing product normalization and matching, setting up data storage, creating CLI/workflow automation, and enabling change detection. To achieve this, I will enhance the existing spiders/modules, ensure reliable scraping of Woolworths and Coles product listings, handle pagination, and implement a data cleaning pipeline for standardizing product information. Additionally, I will persist each scrape as a compressed JSON dataset, provide a simple command for workflow automation, and generate a differential report for price changes, new products, and removed products. I will also update the repository README with setup instructions, Python package requirements, environment variables, how to run the pipeline, and troubleshooting notes. The project will be completed using Python, with lightweight dependencies and adherence to the technical requirements specified. I am keen to discuss this project further in detail. Please connect with me for a more in-depth conversation. You can visit my Profile: https://www.freelancer.com/u/HiraMahmood4072 Thank you.