The hottest Data Analysis Substack posts right now

And their main takeaways
Category
Top Technology Topics

Must Learn KQL Part 20: Building Your First Microsoft Sentinel Analytics Rule

Rod’s Blog • 19 implied HN points • 31 May 23
🕹 Technology Data Analysis Security
  1. Understanding the Kusto Query Language (KQL) is essential for utilizing tools like Microsoft Sentinel to monitor security and detect threats.
  2. Building your first Microsoft Sentinel Analytics Rule involves filtering data, summarizing information, and assigning entities for investigations.
  3. Creating a Watchlist in Microsoft Sentinel can enhance the intelligence of your KQL query by filtering out trusted users and capturing potential threats more accurately.

Must Learn KQL Part 19: The Join Operator

Rod’s Blog • 19 implied HN points • 31 May 23
🕹 Technology Data Analysis
  1. The Join operator in KQL is used to merge rows from two tables by matching values of specified columns.
  2. Using different flavors of Join, like inner, leftouter, rightouter, and fullouter, can change how data is displayed in the results.
  3. To practice and understand the Join operator better, examples can be tested in the KQL Playground or explored in advanced tutorials like 'Addicted to KQL'.

Must Learn KQL Part 18: The Union Operator

Rod’s Blog • 19 implied HN points • 31 May 23
🕹 Technology Data Analysis Query Language Data Integration
  1. The Union operator in KQL allows you to combine data from multiple tables to display all rows together, while the Join operator is used for more specific results by matching column values of two tables.
  2. Union in KQL supports wildcard usage to merge multiple tables and can be used to combine tables from different data sources like Log Analytics Workspaces.
  3. In Microsoft security tools like Microsoft Sentinel and Defender, the Join operator is commonly used for creating Analytics Rules for specific results, while Union is useful for advanced hunting tasks.

Must Learn KQL Part 17: The Let Statement

Rod’s Blog • 19 implied HN points • 31 May 23
🕹 Technology Data Analysis Programming Cloud Computing
  1. The Let statement in KQL allows you to create variables that can be used throughout the query, aiding in better query performance.
  2. Let statements can be used to create variables either from scratch, from existing data, or from Microsoft Sentinel Watchlists.
  3. It's important to properly finalize the Let statement with a semicolon to ensure the variable is stored correctly for query execution.
Get a weekly roundup of the best Substack posts, by hacker news affinity:

Debunking Dunning-Kruger

The Grey Matter • 19 implied HN points • 01 Aug 23
🔬 Science Psychology Research Data Analysis Simulation
  1. The Dunning-Kruger effect is likely a statistical artifact, not a genuine psychological phenomenon
  2. The popular interpretation of the Dunning-Kruger effect as 'the dumbest people think they're the smartest' is a distortion
  3. Replication of the Dunning-Kruger effect through simulation suggests it may not be a real psychological finding

Must Learn KQL Part 14: The Project Operator

Rod’s Blog • 19 implied HN points • 31 May 23
🕹 Technology Data Analysis
  1. Using the Project operator in KQL allows selecting specific data columns to display, providing efficiency for security analysis.
  2. The Project operator variants like Project-away, Project-keep, Project-rename, and Project-reorder offer additional functionalities like excluding columns, renaming headers, and reordering columns in query results.
  3. Understanding and utilizing these Project operator variants can enhance data visualization and streamline data analysis processes in Kusto Query Language.

Must Learn KQL Part 13: The Extend Operator

Rod’s Blog • 19 implied HN points • 31 May 23
🕹 Technology Data Analysis Programming Data Visualization
  1. Custom data views in KQL are crucial for tailoring information to each environment's unique requirements for security and operations.
  2. The Extend operator in KQL allows users to create custom columns in real-time for query results, enhancing data analysis and presentation.
  3. By using the Extend operator, it's possible to generate calculated columns, append them to results, and combine existing data to display meaningful information in KQL queries.

Must Learn KQL Part 12: The Render Operator

Rod’s Blog • 19 implied HN points • 31 May 23
🕹 Technology Data Analysis Visualization Learning
  1. The Render operator in KQL allows you to turn data into visualizations like area graphs, bar charts, and pie charts among others.
  2. Using KQL to create visualizations is crucial for tasks like developing dashboards in Microsoft Sentinel, providing real-time insights to security teams.
  3. Learning to transform data into graphs and charts can make information more engaging and appealing, especially for visual or hands-on learners.

Flashback Friday: Taxi!

Conspirador Norteño • 20 implied HN points • 09 Feb 24
🕹 Technology Social media Automation Data Analysis
  1. A network of taxi and real estate-themed social media accounts were used to boost political content on Twitter through automation.
  2. The botnet consisted of at least 98 Twitter accounts with automated posting schedules that operated 24/7.
  3. The botnet retweeted content based on hashtags, focusing on small accounts and political tweets rather than popular ones.

COVID Vaccines: In Search of All-Cause Mortality

Harnessing the Power of Nutrients • 79 implied HN points • 19 Feb 22
🏥 Health & Wellness COVID vaccines Mortality Data Analysis Vaccine Safety Public Health
  1. Understanding the impact of COVID vaccines on all-cause mortality is crucial for assessing their risk versus reward.
  2. Manipulation of data definitions can lead to misinterpretation of findings, emphasizing the importance of transparent reporting.
  3. All-cause mortality is a key metric to evaluate, but other factors like long-term complications and individual risk profiles should also be considered.

The New CDC Study on Natural Immunity

Harnessing the Power of Nutrients • 79 implied HN points • 19 Feb 22
🏥 Health Politics Vaccination Natural Immunity COVID-19 Data Analysis
  1. Some studies suggest natural immunity from past infection can be as good or even better than full vaccination at protecting against COVID-19 infection.
  2. The new CDC study does not directly compare infection risk between vaccinated and naturally immune populations, but instead looks at hospitalized individuals with COVID-like symptoms.
  3. The study raises questions about the effectiveness of vaccines in preventing hospitalization for COVID-like illness and emphasizes the importance of examining data carefully to draw meaningful conclusions.

Santa Clara County's wastewater shows the COVID vaccine increased infections

Steve Kirsch's newsletter • 7 implied HN points • 04 Nov 24
🏥 Health Politics Public Health Vaccination Data Analysis Health Policy
  1. In Santa Clara County, the amount of COVID in wastewater is higher than the national average. This suggests that vaccination may not have helped reduce infections.
  2. The data shows that after vaccinations were rolled out, infection rates actually went up. This raises questions about the effectiveness of the vaccines.
  3. There hasn't been much discussion from health officials about these findings, which seems strange given the serious implications for public health.

Want To Know How To Calculate Your Enterprise AI Usage? You Can't!

Theology • 3 implied HN points • 26 Jan 25
🕹 Technology AI Pricing Business Models Data Analysis Procurement
  1. Different AI services have complicated pricing models that make it hard to budget. This can lead to unexpected costs every month.
  2. It's tough to compare different AI vendors since their pricing isn't standardized. You might not even know if you're paying for the same features with different companies.
  3. Trying to manage multiple AI platforms can be a headache. In the end, the savings you expect might vanish due to the effort needed to track everything.

Building a data tool: Python vs. JavaScript

The Open-Source Blueprint • 5 HN points • 04 Apr 24
🕹 Technology Programming Web Development Data Analysis
  1. Python has a strong ecosystem for data-related libraries and first-party clients for databases, making it a good choice for data tools.
  2. Javascript also has a large ecosystem of data libraries, first-party clients for major databases, and excellent support for building frontend experiences.
  3. Choosing between Python and JavaScript for building data tools depends on the project requirements and the potential need for incorporating web frameworks.

Under the Hood

Sarah's Newsletter • 59 implied HN points • 29 Mar 22
🕹 Technology Programming Data Analysis Data Management No-Code Tools Cloud Computing
  1. Python's popularity is due to its ease of use and readability, making it one of the top 5 most popular languages.
  2. Abstractions like AWS Lambda can be efficient but may become harmful if not managed properly, leading to issues like security and cost concerns.
  3. Using SQL GUI tools for data aggregation can speed up the process but may lead to inaccurate results and wrong decisions due to lack of testing and QA processes.

Analyzing media bias with AI

serious web3 analysis • 5 HN points • 20 Aug 24
🇺🇸 U.S. Politics Media bias Political Analysis Artificial Intelligence Data Analysis
  1. AI can quickly analyze news articles for bias, saving time compared to human assessment. It rates articles on a scale to determine if they lean left or right.
  2. Mainstream outlets like CNN and NYT tend to show moderate left-wing bias, while Fox News has a stronger right-wing bias. Some sources like AP and Reuters are closer to neutral.
  3. Bias in media can change over time. For example, CNN has become more left-wing recently, especially since the rise of Donald Trump, while Fox News has consistently maintained a right-wing stance.

Data Quality

Data Thoughts • 39 implied HN points • 21 Jan 23
🕹 Technology Data Management Data Quality Data Analysis Data Integrity Information Systems
  1. Data quality is all about how useful the data is for the specific task at hand. What is considered high quality in one situation might not be in another.
  2. There are several key aspects of data quality, including accuracy, completeness, consistency, and uniqueness. Each of these factors helps to determine how reliable the data is.
  3. Improving data quality involves preventing errors, detecting them when they occur, and repairing them. It's about making sure the data is accurate and useful over time.

A Walk Through of the New Analyst Dashboard

The Security Industry • 15 implied HN points • 04 Mar 24
🕹 Technology Cybersecurity Data Analysis Database Management
  1. Version 6 of the Analyst Dashboard for cybersecurity industry research brings a dramatic update to user interface and introduces useful new tools.
  2. Knowing all cybersecurity product vendors is crucial for creating a comprehensive data tool, and manual categorization of vendors is currently necessary.
  3. By collecting data on vendors, answering specific questions about the cybersecurity industry becomes possible, like listing vendors in a certain city or sorting them by year founded.

Reducing selection bias / popularity bias in ranking

Recommender systems • 26 implied HN points • 20 Jan 24
🕹 Technology AI Machine Learning Data Analysis Algorithms Coding
  1. Reducing selection bias and popularity bias in ranking is important for recommender systems.
  2. An advocated approach is to factorize user interaction signals to account for biases originating from power users and power items.
  3. The proposals for causal/debiased ranking involve factorization, mutual information, and mixture of logits to improve the ranking model.

America's Economy Was Bigger Than We Thought

Apricitas Economics • 26 implied HN points • 03 Oct 23
💰 Finance Economy Investment Data Analysis
  1. The US economy was actually larger than previously believed due to comprehensive revisions in GDP data.
  2. America's investment boom was stronger than initially reported, with notable upgrades in real fixed investment across sectors like housing and manufacturing.
  3. Revisions to US GDP data included improved methodologies, extensive data integration, and new data series to enhance the accuracy of measuring economic growth.

From Monte Carlo to Stockholm

HackBoyFly • 1 HN point • 17 Jul 24
💰 Finance Investments Simulation Real Estate Data Analysis
  1. Using a Monte Carlo Simulation can help estimate a wide range of potential outcomes when making investment decisions like buying an apartment in Stockholm
  2. Historical data, mean annual returns, and standard deviations are crucial inputs for simulations to introduce randomness and variability to financial projections
  3. Visualizing simulations through charts can provide insights on possible outcomes, such as optimistic and pessimistic scenarios, aiding in making informed decisions about investments

What are the best "gold standard" data sources for determining COVID safety and efficacy?

Steve Kirsch's newsletter • 5 implied HN points • 01 Nov 24
🏥 Health Politics Public Health Vaccination COVID-19 Data Analysis Epidemiology
  1. It’s important to find reliable data sources to understand the COVID vaccine's impact on safety and effectiveness. Good data helps answer important questions about health.
  2. Key questions include how vaccines affect infection risk, death rates from COVID, and overall mortality rates. These questions guide the research on vaccine impact.
  3. Some of the best data sources for these questions include worldwide COVID case numbers, nursing home COVID data in the US, and detailed records from the Czech Republic.

What is Sundar Pichai thinking?

Technology Made Simple • 39 implied HN points • 26 Mar 22
🕹 Technology AI Machine Learning Data Analysis Search Algorithms Consulting
  1. Google invests significantly in AI and Machine Learning research to enhance their business model - focusing on data-driven ads and boosting operational efficiency.
  2. Google's AI projects often revolve around solving complex search problems, which aligns with their goal of improving search algorithms for hyper-specific advertising.
  3. By mastering core skills like math, theoretical knowledge, problem-solving, and coding, individuals can prepare themselves to tackle challenges at scale similar to what Google does.

Black Friday, Hosting Providers and IPv6

Dataplane.org Newsletter • 19 implied HN points • 07 Nov 22
🕹 Technology Data Analysis Cybersecurity
  1. Black Friday is a good time to look for discounted server hosting plans, but this year's deals might be limited due to economic factors.
  2. IPv6 availability from hosting providers is widespread, but there is inconsistency in how it is provisioned and managed, affecting operational practices.
  3. Dataplane.org is expanding its network of sensor systems and vantage points, exploring active measurement probes with a focus on both IPv4 and IPv6 connectivity.