Top posts of the year

And their main takeaways

Building Identity Resolution on Snowflake Using Snowpark

58 implied HN points • 19 Jun 23
🕹 Technology Data processing Machine Learning Software Development Open Source Cloud Computing
  1. Building ML pipelines in Snowpark requires using third-party libraries like scikit-learn for machine learning.
  2. Integrating specialized functionalities like graph processing in Snowpark may require additional support or custom solutions.
  3. Adapting a codebase from Apache Spark to Snowpark requires careful consideration and potential restructuring to maintain efficiency and avoid technical debt.

Performance Tuning Snowpark For Identity Resolution On Snowflake

19 implied HN points • 29 Jul 23
🕹 Technology Data processing Database Management
  1. Performance tuning Snowpark on Snowflake can significantly reduce processing time, from half a day to half an hour.
  2. Utilizing the query profiler by Snowflake and making targeted optimizations can have a high impact on performance.
  3. Optimizations like converting UDTFs to UDFs, caching Dataframes, and using batch size annotations can further optimize Snowpark workflows.