Experience: collecting geo-diverse image datasets via scraping for training — seeking feedback on dedup/diversity

Hi everyone, I’ve been building a training set for a region-sensitive classification task (retail product images across 4 markets), and since no public dataset covers the regional variants I need, I ended up collecting one myself. Wanted to share where I landed and get feedback from people who’ve done dataset construction at scale. Setup: scraping product image URLs from regional e-commerce listings, collecting each market’s data from in-country IP vantage points (residential proxies with coun...