davidj.substack

davidj.substack focuses on the exploration and application of data strategies, tools, and practices for organizational impact. It covers the evolution of the Modern Data Stack, the importance of semantic layers, data team dynamics, and tooling efficiencies to enhance data handling, analytics, and management.

Data Strategy and Analysis Data Team Management Modern Data Stack Technologies Semantic Layers in Data Analytics Data Quality and Governance Tooling and Automation in Data Management Productivity and Efficiency in Data Operations Real-time Data Processing

The hottest Substack posts of davidj.substack

And their main takeaways

We need to talk about Excel

95 implied HN points • 10 May 23

Excel is still widely used in the data space for its ease of use and versatility
Data teams aim to reduce Excel use due to limitations such as scalability and version control issues
New tools like Count and Equals are emerging to address Excel limitations and improve collaboration in data analysis

Metricalypse Now

107 implied HN points • 15 Feb 23

🕹 Technology Data Management Analytics Software Integration BI Tools Commercialization

Two approaches to metrics layers: wide datasets without defined data models vs. defined data model for more powerful metrics.
Importance of new semantic layer by dbt Labs acquiring Transform for a universal standalone analytics solution.
Opportunity for data consumption vendors to integrate with new dbt semantic layer for a ubiquitous solution.

Semantic Superiority - Part 3

83 implied HN points • 05 Apr 23

🕹 Technology Semantic Layers BI Tools Open Source APIs

Semantic layers are crucial for governance, security, accessibility, and developer experience benefits in data analytics.
Standalone semantic layers offer more flexibility and serve multiple use cases compared to semantic layers built into BI tools.
Different standalone semantic layer options like Cube, AtScale, dbt/MetricFlow, and Looker Modeller provide unique features and cater to varying needs in data modeling and analytics.

Excelibur

71 implied HN points • 17 May 23

🕹 Technology Data Analysis Software Development User Interface Data Management BI Tools

Excel scalability can be improved by integrating technologies like DuckDB for handling larger datasets.
Enhancing data cleanliness through exposing hidden issues to the user for resolution.
Implementing a full semantic layer in Excel could make data pulling easier and more secure.

Semantic Stream

23 implied HN points • 29 Feb 24

🕹 Technology Querying Data Storage

Consider how to use a semantic layer with streaming data to enhance efficiency and data processing.
Streaming data warehouses handle storage differently than batch data warehouses, keeping fresh data in-memory and reducing compute cost.
The semantic layer abstracts entities, attributes, and metrics, aiding in managing and optimizing queries on streaming data.

Get a weekly roundup of the best Substack posts, by hacker news affinity:

Once more unto the breach...

71 implied HN points • 01 Mar 23

🕹 Technology Data Engineering Business Intelligence Automation AI Analytics

Automating simple data requests can save time and effort in data analysis workflows.
Effective self-serve BI tools require reliable data pipelines, transformation processes, and a semantic layer.
Text-to-SQL tools can improve automation but may require caution due to potential errors and lack of trust in results.

A Darker Truth

2 HN points • 07 Mar 24

🕹 Technology Artificial Intelligence Data Management Startups Workplace dynamics

Text-to-semantic layer systems can work in enterprise but text-to-SQL ones won't due to technical deficiencies.
Even with infinite resources, achieving a perfect text-to-SQL system may not be enough due to the importance of how data is perceived by stakeholders.
Blame and humiliation dynamics in human interactions make text-to-semantic layer systems more viable than text-to-SQL systems in corporate settings.

sqlmesh cube_generate build part 1

0 implied HN points • 17 Dec 24

🕹 Technology Software Development Data Analysis Programming Machine Learning Database Management

There's a new command called `sqlmesh cube_generate` that helps build models for data analysis. It's designed to make working with data easier for users.
The tool outputs useful information in a structured format, which includes joins and fields for data analysis. This makes it simple to understand how the data connects.
Even if there are challenges with complex data types, the output is still effective and can be enhanced using AI, showing there's room for creativity in data modeling.