Ali's Tech Tales

Join Ali on a fun journey through the world of programming and web scraping. Discover cool tips, quirky projects, and tech tidbits that spark curiosity. Perfect for tech enthusiasts who love to explore and learn in a laid-back way!

The hottest Substack posts of Ali's Tech Tales

And their main takeaways

Discovering Shopify Domains: A Journey Through Common Crawl Data

7 HN points • 17 Jun 24
🕹 Technology Data processing Web scraping
  1. Utilizing object storage like MinIO can streamline processes and reduce the amount of code needed for handling large data sets efficiently.
  2. Efficiently processing large volumes of data using multiprocessing in Python can significantly speed up tasks like parsing vast numbers of URLs in parallel.
  3. By merging dictionaries containing hostnames and then splitting them into manageable chunks, it's possible to handle huge amounts of data effectively, such as discovering over 140 million unique website hostnames.