The hottest Substack posts of Monthly Python Data Engineering

And their main takeaways
179 implied HN points 25 Jul 24
  1. The Python Data Engineering newsletter focuses on key updates and tools for building data engineering projects, rather than just data science.
  2. This month showcased rapid development in projects like Narwhals and Polars, with Narwhals making 26 releases and Polars reaching version 1.0.0.
  3. Several other libraries, such as Great Tables and Dask, also had important updates, making it a busy month for Python data engineering tools.
59 implied HN points 19 Aug 24
  1. Datafusion Comet was released, making it easier and faster to use Apache Spark for data processing, which is great for improving performance.
  2. Several major data tools like Datafusion, Arrow, and Dask updated their versions, showing ongoing improvements in speed, efficiency, and new features.
  3. New dashboard solutions like Panel and updates in libraries such as CUDF reflect the growing interest in making data access and visualization easier for users.
2 HN points 26 Sep 24
  1. A new free book called 'How Data Platforms Work' is being created for Python developers. It will explain the inner workings of data platforms in simple terms, with one chapter released each month.
  2. The Ibis library has removed the Pandas backend and now uses DuckDB, which is faster and has fewer dependencies. This change is expected to improve performance and usability.
  3. Several popular libraries in Python, such as GreatTables and Shiny, have released updates with new features and improvements, focusing on better usability and integration with modern technologies.