Conrado Miranda

Welcome to my Substack, where I share insights on AI, leadership, and strategy. Join me as we explore the evolving world of artificial intelligence.

The hottest Substack posts of Conrado Miranda

And their main takeaways

The problem with how we evaluate LLMs

2 HN points • 28 May 24
🕹 Technology LLMs Evaluation Language Models Research
  1. Evaluating Large Language Models (LLMs) can be challenging, especially with traditional off-the-shelf metrics not always being suitable for broader LLM applications.
  2. Using an LLM-as-a-judge method for evaluation can provide insights, but there's a risk of over-reliance on the black-box model, leading to potential lack of understanding on improvements.
  3. Creating clear, specific evaluation criteria and considering use cases are crucial. Auto-criteria, like auto-prompting, may be future tools to enhance LLM evaluations.