Machine Learning Diaries

We help to discover and learn rare Machine learning insights in a simple and engaging way so you can impress your boss.

The hottest Substack posts of Machine Learning Diaries

And their main takeaways

Learn from Experiences of Experts - Running Trustworthy A/B Test

7 implied HN points • 27 Nov 24
🕹 Technology Machine Learning Data science Statistical Analysis Experimental Design
  1. A/B tests are important for businesses because they help test ideas and make informed decisions. Many companies have seen significant revenue increases by using A/B tests.
  2. It's crucial to define the right performance metrics for A/B tests to ensure long-term success. Focus on metrics that show real customer engagement, not just short-term results.
  3. Pay close attention to statistical principles when running A/B tests. Misunderstanding p-values and making hasty conclusions can lead to incorrect results and poor decisions.

Super Weights in LLMs - How Pruning Them Destroys a LLM's Ability to Generate Text ?

3 implied HN points • 18 Nov 24
🕹 Technology Machine Learning Artificial Intelligence Data science Model optimization Software Development
  1. Super weights are very important for how well large language models (LLMs) perform. Even though they're a tiny part of the model, they can greatly affect the results.
  2. If a super weight is removed, it can ruin the model's ability to generate clear text and make predictions. Just taking out one of these weights can cause a huge drop in performance.
  3. Removing regular outlier weights doesn't harm performance much, but losing just one super weight is much worse than taking out a lot of other weights combined.

Introducing Etalon: How we choose a LLM with optimal Runtime Performance ?

3 implied HN points • 11 Nov 24
🕹 Technology Machine Learning AI Models Performance Metrics User Experience
  1. Evaluating large language models (LLMs) is important for ensuring a good user experience. Existing metrics like Time to First Token (TTFT) and Time Between Tokens (TBT) don't fully capture how these models perform in real-time applications.
  2. The proposed 'Etalon' framework offers a new way to measure LLMs using a 'fluidity-index' that helps track how well the model meets deadlines. This ensures smoother and more responsive interactions.
  3. Current metrics can hide issues like delays and jitters during token generation. The new approach aims to provide a clearer picture of performance by considering these factors, leading to better user satisfaction.
Get a weekly roundup of the best Substack posts, by hacker news affinity:

How to apply Boosting when the Data Labels are Noisy and Uncertain

0 implied HN points • 28 Feb 24
🔬 Science Machine Learning Model Evaluation
  1. Boosting algorithms can struggle when dealing with noisy and uncertain data labels.
  2. Weakly supervised learning (WSL) is gaining attention as a way to handle noisy and weak data labels more effectively than fully-supervised methods.
  3. The LocalBoost approach aims to address challenges by iteratively and adaptively enhancing boosting in a weakly supervised setting.