Skip to content

Team · Team Project | Data Collection and Analysis

Amazon Web Scraping and Product Intelligence

An end-to-end data engineering and product analytics pipeline that extracts publicly visible e-commerce listings and transforms them into structured, analysis-ready market intelligence.

My contribution

I contributed to collection design, cleaning and structuring product fields, and shaping outputs stakeholders could actually analyse — trustworthy structure over flashy charts.

Python · Web scraping · HTML parsing · Data cleaning · Pandas · SQL · Data analysis · ETL

Problem

E-commerce pricing, rating distributions, and catalog attributes are locked inside unstructured web pages, preventing systematic competitive benchmarking.

Solution & architecture

Engineered automated extraction scripts with rate-limit compliance, HTML parsing, schema normalisation, and data cleaning workflows to output structured tabular datasets for pricing and sentiment analysis.

ETL workflow & analytics output

Request orchestration → DOM parsing and attribute extraction → regex/currency/rating normalisation → structured storage. Outputs include price-to-rating correlation tables and category-level competitive summaries.

My contribution

Team contribution across pipeline design, cleaning logic, and delivering structured outputs suitable for downstream analysis.

Impact

Enables rapid market comparison, pricing distribution tracking, and review-volume insights across product categories — without inventing commercial KPIs.