Southeast Asia Daily AQI Dataset -- 2

@manoval09

Hi, A clean, research-ready daily AQI dataset covering India and Indonesia means reconciling at least three fragmented source ecosystems — CPCB for India, BMKG and regional APIs for Indonesia — where station coverage, pollutant reporting (PM2.5 vs PM10 vs AQI composite), and update cadence vary significantly by region. My approach: pull from each country's official monitoring APIs plus OpenAQ as a cross-validation layer, normalize all readings to a unified schema (date, station ID, lat/lon, pollutant type, raw value, AQI index, data source), and flag gaps or outlier readings rather than silently interpolating them. I'd store the pipeline in Python using `requests` + `pandas`, with a SQLite or Parquet output depending on your downstream tooling — Parquet is the better default for research workflows given columnar query performance. One clarifying question before I start: is this a one-time historical pull (e.g., last 2–5 years) or are you also expecting a recurring daily update pipeline? That distinction shapes the delivery significantly. Either way, within 24 hours I can share a sample schema and a data coverage map showing which stations and date ranges are reliably available for both countries. Best regards, Val

@sinchana0123

Hello, Yes, I can definitely help with this project. I have experience with Python and can work with data scraping, cleaning, preprocessing, and CSV generation workflows. I understand that you need a research-ready air-quality dataset for India, Indonesia, and the Philippines from 2019–2025 using official government monitoring sources only, including station-level pollutant and meteorological variables with computed PM2.5-based AQI values. I can assist with: • Collecting data directly from official government portals and monitoring stations • Building Python scripts for scraping and cleaning the data • Standardizing all variables into consistent units • Maintaining station-level details with latitude and longitude • Removing duplicates and validating daily continuity • Generating structured CSV datasets and a clear README with source references and methodology I also understand the importance of reproducibility, traceable source documentation, and maintaining clean research-quality datasets. Before starting, I would confirm the preferred AQI calculation standard and finalize the exact source scope to ensure the output fully matches your research requirements.

@qianq5

Hi, I can build this as a reproducible Python data pipeline rather than a one-off spreadsheet. I would pull daily AQI / PM2.5 / PM10 records from documented public sources, normalize India/Indonesia/Philippines into one schema, flag gaps/outliers, and deliver both the clean Excel/CSV dataset and the scripts used to rebuild it. I will include source URLs, date coverage notes, and a short QA summary so the dataset is research-ready and repeatable.