The hottest Statistics Substack posts right now

And their main takeaways
Category
Top Technology Topics

Where does the mean squared error come from?

The Palindrome • 0 implied HN points • 21 Dec 23
🔬 Science Statistics
  1. Mean squared error is a common loss function for machine learning models due to its mathematical simplicity and alignment with statistical principles.
  2. Absolute value functions are not commonly chosen for loss function in machine learning due to issues with differentiability at zero.
  3. The linear model and mean squared error naturally arise when approaching machine learning with a statistical mindset.

How to interview people that are more competent than you

Boris Again • 0 implied HN points • 27 Sep 23
💼 Business Statistics
  1. When interviewing people more competent than you, have them explain complex concepts to test communication and understanding.
  2. Include both open-ended and technical questions in the interview process to assess problem-solving and technical skills.
  3. Ensure candidates ask questions, articulate needs, and provide appropriate solutions to evaluate their potential in the position.

Descriptive Statistics with Orange

rtnF • 0 implied HN points • 01 Apr 23
🕹 Technology Statistics
  1. Descriptive statistics with Orange allows for easy data analysis without needing spreadsheet equations or code.
  2. The mean and median provide insight into average building height, helping to understand outlier influence on data.
  3. Understanding dispersion, like the coefficient of variation, reveals how data points spread out relative to the mean.

Where Bayes Falls Short

As Clay Awakens • 0 implied HN points • 30 May 23
🔬 Science Statistics
  1. Deep learning algorithms are powerful for intelligence and learning, especially in contexts where Bayes' theorem falls short.
  2. Simpson's paradox shows how data separation can change conclusions based on initial beliefs.
  3. Deep learning approaches in regression tasks offer solutions without the need for ad-hoc choices, allowing for better predictions and generalization.
Get a weekly roundup of the best Substack posts, by hacker news affinity:

Data Week: How to collect data

Joshua Gans' Newsletter • 0 implied HN points • 16 Oct 20
🔬 Science Statistics
  1. Data collected at a manhole level can help detect outbreaks more rapidly and support targeted interventions.
  2. Sophisticated statistical techniques can provide a deeper understanding of outbreaks by leveraging sewage system data.
  3. Bayesian framework can convert sewage flows into probability flows to identify hot spot neighborhoods with just a few samples.

Power tools for confusion

Meaningness • 0 implied HN points • 25 Oct 20
🔬 Science Statistics
  1. The post discusses how rationalist misunderstanding of statistics contributed to the replication crisis in some sciences, revealing that much of what was believed to be true was false.
  2. The section focuses on probabilistic rationalism and sheds light on the importance of statistics in research and decision-making processes.
  3. The post is geared towards paid subscribers, providing exclusive content on challenging topics related to confusion and rationalism.

New Study Shows that Lying About Your Hamburger Intake Prevents Disease and Death When You Eat a Low-Carb Diet High in Carbohydrates

Harnessing the Power of Nutrients • 0 implied HN points • 09 Sep 10
🏥 Health & Wellness Statistics
  1. Drawing conclusions about diet and health from observational studies can be misleading and should be approached with caution.
  2. The way 'low-carbohydrate' was defined in the study was unconventional, making interpretations challenging.
  3. Epidemiological studies can be influenced by participants' biases and inaccuracies, leading to potential misinterpretations of data.

Baseball and Carbon Accounting, Part 1

Solar Powered Data • 0 implied HN points • 21 Aug 23
🎾 Sports Statistics
  1. Measuring carbon emissions is challenging and involves various frameworks like the GreenHouse Gas Protocol and Science-Based Targets.
  2. Just like baseball teams aim to score more runs by balancing offense and defense, individuals in carbon accounting also strive to reduce emissions while enhancing carbon removal.
  3. In both baseball and carbon accounting, accurately attributing individual contributions is complex, and there is a need for improved methods to credit and analyze performance.

The First Post

Solar Powered Data • 0 implied HN points • 08 Jun 23
🌞 Climate & Environment Statistics
  1. Climate tech is a significant solution for a big problem and a great opportunity.
  2. Data is a powerful tool to explore climate tech and understand the impact of climate change.
  3. Sharing knowledge and insights about climate data can contribute positively to addressing climate change.

MSSQL query plan optimisation advantages

Conserving CPU's cycles ... • 0 implied HN points • 21 May 24
🕹 Technology Statistics
  1. In MSSQL to PostgreSQL migrations, challenges like query slowdowns may arise, with some queries taking significantly longer to execute in PostgreSQL compared to MSSQL.
  2. Join algorithm selection and parallelism are two key advantages contributing to MSSQL's impressive query execution speed.
  3. Multi-clause selectivity estimation in MSSQL allows for more precise cardinality estimation in complex join queries, giving it an edge over PostgreSQL in certain scenarios.

On Humans And Our Love of Data

The Digital Anthropologist • 0 implied HN points • 16 Sep 23
🔬 Science Statistics
  1. Our brains love patterns, math, and language to comprehend the world and shape realities.
  2. Humans have a deep-rooted history of creating, analyzing, and utilizing data for various purposes throughout civilizations.
  3. Data, when transformed into information and knowledge, holds significant value and potential for enhancing human evolution and species advancement.

Newsletter #4: Probabilities with Python

Decoding Coding • 0 implied HN points • 02 Mar 23
🕹 Technology Statistics
  1. NumPy is a powerful tool for working with probability distributions in Python. You can easily generate data and calculate probabilities using its features.
  2. Common probability distributions like Normal, Binomial, and Poisson can be modeled using NumPy. Each distribution has its own formula to calculate probabilities.
  3. De Morgan's Laws help in calculating probabilities of complements in events. They show how to relate the union and intersection of events, which can be useful in probability theory.

Super Bowl Coda: Twelve Ideas Circulating in Manhattan

Matt’s Five Points • 0 implied HN points • 10 Feb 12
🎾 Sports Statistics
  1. The Giants didn't plan to have 12 players on the field during the game. It was a mistake, not a strategy.
  2. Eli Manning is seen as an elite quarterback and his ability in tough situations gives the Giants an edge in close games.
  3. Having a first-round bye in the playoffs might not be as beneficial as it seems, as recent statistics show that teams with byes have struggled to win their games.

I love you Tough Tom, but you were wrong on 1st and goal

Matt’s Five Points • 0 implied HN points • 06 Feb 12
🎾 Sports Statistics
  1. In a key moment of the game, the strategy of when to score is really important. The Giants didn't play it smart by scoring a touchdown instead of just getting in a position to kick a field goal.
  2. If the Giants had chosen to kneel at the one-yard line, they would have had a much better chance of winning. It's all about reducing risk and thinking strategically.
  3. Coaches often stick to traditional tactics instead of trying new strategies, even if those could lead to better outcomes. Changing how they think could really improve their chances of winning in the future.

Sports Fan, Stats Geek?

Matt’s Five Points • 0 implied HN points • 24 Sep 11
🎾 Sports Statistics
  1. Ben Morris is a talented sports statistician with a fun writing style.
  2. He won the 2011 ESPN Stat Geek Smackdown, showcasing his expertise.
  3. He will be doing a live blog of NFL games, which should be exciting to follow.

On youthful talent

Matt’s Five Points • 0 implied HN points • 20 Jul 11
🎾 Sports Statistics
  1. Young talent can achieve impressive things at a very young age, like Bob Feller striking out 17 batters as a rookie at 17.
  2. Many people might not know that other young players have also reached great accomplishments at a young age.
  3. Youthful talent often gets overlooked, but their achievements deserve recognition just like the famous legends.

Posterior probabilities: A Bayesian update of the Miami Heat

Matt’s Five Points • 0 implied HN points • 27 Oct 10
🎾 Sports Statistics
  1. The updated probability of the Miami Heat being a top team, after their first game loss, is still high at 76%.
  2. The terms 'juggernaut' and 'disappointment' need clearer definitions when analyzing the team's performance.
  3. Revising the probabilities based on better historical data showed that even with a lower win rate, the Miami Heat's potential can still be seen positively.

How valuable is a great kicker?

Matt’s Five Points • 0 implied HN points • 01 Sep 10
🎾 Sports Statistics
  1. A great kicker can significantly change the dynamics of a high school football team, especially if they can consistently make long field goals.
  2. In college and NFL, the value of a kicker varies, but even a guaranteed 3 points can make a big impact on a team's competitiveness.
  3. Kickers don't get paid like other star players because there isn’t a huge difference between the top and average kickers, yet having a top performer can still be crucial to winning games.

On Jim Joyce and this generation's Harvey Haddix...

Matt’s Five Points • 0 implied HN points • 03 Jun 10
🎾 Sports Statistics
  1. Perfect games are really exciting to watch and can turn a boring game into something thrilling. People will often go out of their way to watch every pitch of these games.
  2. The missed call by umpire Jim Joyce during Galarraga's near-perfect game is one of the most tragic moments in baseball history. Many think it marks the worst missed call in a regular season game.
  3. People might remember Galarraga's name for years, but it may not carry the same weight as Harvey Haddix's legendary 1959 performance. There’s a unique story about Haddix that adds to its memory.

How to use Google’s CausalImpact

Logos • 0 implied HN points • 23 Dec 21
🕹 Technology Statistics
  1. Google's CausalImpact helps you see how actions, like a marketing campaign, affect outcomes like sales. It predicts what would have happened without that action, making it easier to understand its impact.
  2. Using CausalImpact requires some basic coding in R, but even beginners can follow along. You'll collect data in a simple format, run the analysis, and see results visually and in tables.
  3. When using CausalImpact, it's crucial to choose the right control variables. They should correlate with your main outcomes but not be influenced by the actions you're analyzing.

Mean Reversion: Gravitational Super Force or Dangerous Delusion?

Musings on Markets • 0 implied HN points • 31 Aug 16
💰 Finance Statistics
  1. Mean reversion is the idea that extreme results will return to the average over time. This is seen in sports and investing, but it can lead us to make wrong assumptions about future performance.
  2. There are two types of mean reversion: time series mean reversion, which looks at past average values over time, and cross-sectional mean reversion, which compares values against the average of similar items. Both have their own risks and assumptions.
  3. Structural changes in the economy or companies can disrupt mean reversion, meaning trusting it too much could lead to poor investment decisions. It's important to stay aware of these changes and not just rely on historical data.

My Valuation Class: The Fall 2015 Model Preview

Musings on Markets • 0 implied HN points • 24 Aug 15
🚌 Education Statistics
  1. Valuation is a skill, not just numbers or theory. It's like cooking or building things, where you get better by doing it rather than just studying the details.
  2. There's a big difference between valuing an asset and pricing it. Valuation looks deeper into the intrinsic value, while pricing is often about what the market will pay.
  3. You can value almost any asset, even if it seems tricky. By the end of a valuation class, you'll have the tools to value different types of assets confidently.

The unemployment rate: A look at the sampling question.

Musings on Markets • 0 implied HN points • 06 Feb 11
🇺🇸 U.S. Politics Statistics
  1. The unemployment rate is calculated using a survey of about 60,000 households, while payroll numbers come from a survey of 140,000 businesses. These different sources can lead to different results.
  2. Sampling bias can affect results if the survey doesn't represent the whole population well. It's important to trust that statisticians are working to avoid these biases.
  3. Data can have noise or errors, especially when the job market is changing a lot. Seasonal adjustments and revisions to previous data can impact how we understand the unemployment rate.

[in case you missed it] Data Science Weekly - Issue 448

Data Science Weekly Newsletter • 0 implied HN points • 26 Jun 22
🕹 Technology Statistics
  1. Machine learning can help the IRS by better analyzing the large amount of tax data they collect, making tax enforcement more effective.
  2. New models like Denoising Diffusion Probabilistic Models are showing great promise in generating high-quality images and audio from simpler inputs.
  3. There is a focus on improving machine learning practices, such as being careful with training data and understanding how to boost model performance through proper methods.

[in case you missed It] Data Science Weekly - Issue 440

Data Science Weekly Newsletter • 0 implied HN points • 01 May 22
🕹 Technology Statistics
  1. AI is getting smarter, but we need better ways to ask it questions about its decisions to understand it better.
  2. Synthetic data can help when there's not enough real data for training, allowing us to create more examples for our models.
  3. Data accessibility is really important because unlocking the data can help solve big problems and improve society as a whole.

[in case you missed it] Data Science Weekly - Issue 410

Data Science Weekly Newsletter • 0 implied HN points • 03 Oct 21
🕹 Technology Statistics
  1. Data science is growing quickly, and the best companies to work for vary depending on your career stage. It's important to find a workplace that helps you grow in your data science career.
  2. Recent research is improving weather prediction by looking at short-term changes, like predicting rain in the next hour. This can be really useful for planning daily activities.
  3. Using statistics can help us understand large groups by studying small samples. It simplifies the data and gives us insights without needing to look at everything.

[in case you missed it] Data Science Weekly - Issue 382

Data Science Weekly Newsletter • 0 implied HN points • 21 Mar 21
🕹 Technology Statistics
  1. Computers can't write good stories. It's a big claim, but they really don't understand literature like humans do.
  2. Using color scales is important when showing data visually. Choosing the right colors can make your data easier to understand.
  3. Data science can help fight illegal fishing with satellite data. By tracking boats, experts can prevent unlawful activities in our oceans.

[in case you missed it] Data Science Weekly - Issue 376

Data Science Weekly Newsletter • 0 implied HN points • 07 Feb 21
🕹 Technology Statistics
  1. Data quality is really important in high-stakes AI because it can greatly affect results in areas like health and finance. Many people focus on building models instead of ensuring good data quality.
  2. DanNet was a game-changer in computer vision when it was released ten years ago. It showed that deep learning models could even surpass human performance in certain tasks.
  3. Cohort analysis helps businesses understand their customers better by tracking different groups over time. It's useful for figuring out things like customer engagement and product performance.

[in case you missed it] Data Science Weekly - Issue 337

Data Science Weekly Newsletter • 0 implied HN points • 10 May 20
🕹 Technology Statistics
  1. There are online seminars available that cover math topics related to data science and machine learning. These can help you understand the foundations better.
  2. Deep reinforcement learning has made big advances recently, but there's still room for improvement and new ideas in the field.
  3. If you're looking for a data science job, there are resources and guides that can help you improve your resume, build a project portfolio, and get started in the field.

[in case you missed it] Data Science Weekly - Issue 324

Data Science Weekly Newsletter • 0 implied HN points • 08 Feb 20
🕹 Technology Statistics
  1. Experimentation is key in product development. Good experiments help in understanding customer needs better and making informed decisions.
  2. AI technology can have a real-world impact, as seen with early warnings about health crises. Tools like AI can gather and analyze data faster than traditional methods.
  3. Improving AI means making it more human-like for better performance. Understanding the limits and potential of AI can help us use it more effectively.

[in case you missed it] Data Science Weekly - Issue 323

Data Science Weekly Newsletter • 0 implied HN points • 01 Feb 20
🕹 Technology Statistics
  1. Cleaning and organizing data takes a lot of time for data scientists, and lack of access to good data can stop many projects from happening.
  2. Using a checklist can help data scientists keep track of all the necessary steps in their projects, making their work less overwhelming.
  3. Learning progressively with simpler concepts first can help both humans and machine learning models tackle more complex problems effectively.

[in case you missed it] Data Science Weekly - Issue 312

Data Science Weekly Newsletter • 0 implied HN points • 16 Nov 19
🕹 Technology Statistics
  1. Researchers are discovering ways to turn brain signals into speech, which could change how people communicate.
  2. There's a growing concern about bias in AI systems, and finding solutions is important to ensure fairness.
  3. Data scientists are highly sought after in the job market, highlighting the importance of skills in data analysis and machine learning.

[in case you missed it] Data Science Weekly - Issue 310

Data Science Weekly Newsletter • 0 implied HN points • 02 Nov 19
🕹 Technology Statistics
  1. Rising sea levels are going to affect more cities than we thought, and scientists are using AI to improve predictions about future coastlines.
  2. A new neural network can solve a difficult math problem much faster than before, showing how machine learning can change traditional math approaches.
  3. DeepMind's AI has learned to play StarCraft II better than most human players, using self-cooperation to develop new strategies in the game.

[in case you missed it] Data Science Weekly - Issue 253

Data Science Weekly Newsletter • 0 implied HN points • 29 Sep 18
🕹 Technology Statistics
  1. Uber uses machine learning and deep learning to make better forecasts for their products and services. They focus on combining traditional statistical methods with advanced techniques for accurate predictions.
  2. There's a shift in software development where deep learning is automating much of the coding process. Developers now create a basic outline, allowing the computer to generate the code from past examples.
  3. Tiny computers are increasingly replacing larger controllers in technology. This trend highlights the importance of smaller, more efficient computing solutions in the embedded world.

Simultaneous Localization and Mapping

The Halfway Point • 0 implied HN points • 26 Apr 24
🕹 Technology Statistics
  1. Self-driving cars need to know their exact location to avoid accidents. GPS and sensors like RADAR have errors, so it's tricky to get precise positioning.
  2. The Kalman filter helps improve the accuracy of measurements by combining noisy data over time. It has two main steps: updating measurements and predicting motion.
  3. For complex situations, there are advanced versions of the Kalman filter, like the Extended and Unscented Kalman filters, which can handle non-linear data better for more accurate tracking.

Statistical Concepts Useful in Life

Kartick’s Blog • 0 implied HN points • 21 Jan 25
🚌 Education Statistics
  1. Variance helps us understand risk in different jobs. A steady job is low risk, while a startup can be very unpredictable.
  2. The median is a strong way to find a typical value because it's not easily affected by extreme numbers. So, when data is messy, the median usually gives a better answer than the mean.
  3. To get better estimates, look at a lot of data over time. More data usually means less error, helping you make smarter decisions.

Not All Zeros Are Equal

Nano Thoughts • 0 implied HN points • 20 Jan 25
🔬 Science Statistics
  1. Not all zeros in data mean the same thing. Sometimes, they can indicate something was never there, or other times, they mean something was just missed.
  2. Zero inflation happens when there's lots of data and many readings come back as zero. This can make it hard to understand what's really going on behind those zeros.
  3. There are different methods to deal with zeros in data, like checking if they are real or just unnoticed signals. Choosing the right method is important to get accurate insights.