The hottest Data Modeling Substack posts right now

And their main takeaways
Category
Top Technology Topics

Beyond YAML

Making Things • 1 implied HN point • 06 Nov 23
🕹 Technology Data Modeling
  1. Many semantic layers are built with YAML for its readability and quick setup, but it can lead to a poor developer experience.
  2. YAML lacks immediate feedback for complex expressions, forcing users into a guessing game when writing configurations.
  3. Implementing a real programming language instead of just a configuration DSL can provide instant feedback and support complex data modeling.

What is Surgical Fine-Tuning and Why You Should Care

Three Data Point Thursday • 0 implied HN points • 13 Jul 23
🕹 Technology Data Modeling
  1. Surgical fine-tuning in ML makes algorithms better suited for specific business contexts through precise changes, an advancement over regular fine-tuning.
  2. Entity-centric data modeling marries ML feature engineering with data engineering, improving data operations for companies.
  3. Estimating efforts for ML projects can be simplified by considering the cost of delay and the real-time requirement of the algorithm.
Get a weekly roundup of the best Substack posts, by hacker news affinity:

Ghosts of Diffusion

Cybernetic Forests • 0 implied HN points • 21 Aug 22
🕹 Technology Data Modeling
  1. AI-generated images are similar to spirit photography from the 19th century, evoking a mystical connection to new technologies
  2. Diffusion models like DALLE2 differ from GANs by stripping images to noise and then reconstructing them, learning how images become noise and reverting them back
  3. DALLE2 creates images by finding patterns in noise, showing that the foundation of every image is arbitrary, like a dream, and that the AI is not really creating art but tracing possibilities in decay

Understanding your Big Data problem: Transforming Big Data with Incremental Models

The Orchestra Data Leadership Newsletter • 0 implied HN points • 31 Oct 23
🕹 Technology Data Modeling
  1. Understanding the importance of incremental models for managing big data is crucial to efficiently running complex queries and maintaining data quality.
  2. Design patterns in data modeling, such as Star Schema and Data Vault, play a significant role in how dbt models are structured and managed.
  3. Using Jinja templating and implementing continuous data integration processes are key elements in handling big models effectively and ensuring data reliability.

Why Google App Engine

realkinetic • 0 implied HN points • 23 Jul 18
🕹 Technology Data Modeling
  1. Google App Engine provides automated operations that manage scalability, fault-tolerance, and traffic splitting, freeing you to focus on your application and business logic.
  2. Designing applications on Google App Engine requires embracing statelessness, optimizing data models, and minimizing request latency to ensure efficient scaling and performance.
  3. Utilize App Engine's features like task queues and services, understand the limitations of Memcache, and plan for modular design to maximize the platform's capabilities and scalability.

GroupBy #21: How to design resilient and large scale data systems, What Data Modeling is NOT

VuTrinh. • 0 implied HN points • 06 Feb 24
🕹 Technology Data Modeling
  1. Designing data systems requires resilience and scalability, which means they should handle growth and failures efficiently.
  2. Data modeling is more than just making diagrams; it's about understanding the entire system and how data flows within it.
  3. Using tools like DuckDB in the browser can open up new possibilities for data processing, making it more accessible and flexible.

GroupBy #8: Demystifying the Parquet File, the future of the data engineer, intro to data modeling.

VuTrinh. • 0 implied HN points • 06 Nov 23
🕹 Technology Data Modeling
  1. The Parquet file format is becoming popular for data storage because it is efficient and works well with big data tools. Understanding how to use it can help data engineers be more effective.
  2. Data engineering is evolving, and new trends like data mesh are changing how data platforms are built. Keeping up with these changes is important for anyone in the field.
  3. Starting a small data engineering project can be a great way to learn new skills. Even a quick project can teach you important techniques, like web scraping and using cloud storage.

Data Modeling for Data Engineering

DataSketch’s Substack • 0 implied HN points • 18 Mar 24
🕹 Technology Data Modeling
  1. Data modeling is like creating a map for organizing and finding data easily. It helps keep everything tidy and accessible.
  2. There are three types of data models: conceptual, logical, and physical, each serving different levels of detail in planning data structure.
  3. A practical example is organizing a library, where the models help define books, authors, and loans, ensuring everything links and works smoothly.

A model of everything

Simplicity is SOTA • 0 implied HN points • 17 Jul 23
🕹 Technology Data Modeling
  1. A model of everything predicts final and intermediate goals of a company, is causal, and covers significant inputs.
  2. Foundational choices in building a model of everything include deciding the scope, complexity of relationships, and optimization strategy.
  3. Financial forecasting often involves models of everything, built in spreadsheets, but may not work well for machine learning models.