The hottest Data Transformation Substack posts right now

Data teams face challenges when separated from product engineering, leading to loss of metadata and concerns about data quality. Data contracts can help address these issues by defining the nature, completeness, and format of shared data.
Integrating data professionals within product teams can enhance understanding and usage of data, reducing the need for separate contracts. This approach allows for direct-to-consumer, organic data processes.
Centralized data platform teams can establish common standards and infrastructure, enabling embedded data personnel in product teams to work efficiently. This collaborative model streamlines data transformation and enhances data accessibility.

The best predictions come from deep analysis of today's data challenges and trends.
Data oracles provide valuable insights for the future by understanding present data trends.
Data writers like Davenport, Moses, Madsen, and Thomas offer grounded observations and advice on data topics.

Malloy CLI simplifies data transformation by offering reusable calculations.
Malloy enables defining named measures for easy aggregation across dimensions.
Malloy CLI makes code concise and maintainable compared to traditional SQL views.

The concept of 'embedding' in machine learning has evolved and become widely used, replacing terms like vectors and representations.
Embeddings can be applied to various types of data, come from different layers in a neural network, and are not always about reducing dimensions.
Defining 'embedding' has become challenging due to its widespread use, but the essence is about learned transformations that make data more useful.

Get a weekly roundup of the best Substack posts, by hacker news affinity:

Developing key systems is crucial for a successful data transformation.
Role creep in CDO/CDAO responsibilities is an ongoing concern.
Consider joining a data strategy book club for learning and community engagement.

Data space is growing exponentially with new trends and transformations.
In a complex data environment, continuous probing and response is crucial.
Consider large-scale transformations to change how your company works with data.

Consider ditching dbt for SQLMesh, a tool designed for data engineers.
Explore alternatives to dbt for modern data transformation tools to suit your workflow better.
Invest in understanding geometric deep learning for its business potential, even though it might not be practical yet.