Product Evals in Three Simple Steps
Label some data, align LLM-evaluators, and run the eval harness with each change.
324 articles · Fresh posts aggregated from independent blogs across the directory.
Label some data, align LLM-evaluators, and run the eval harness with each change.


As AI advice becomes more important, we are going to need to get better at assessing it

Based on what I've learned from role models and mentors in Amazon
An LLM that can converse in English & item IDs, and make recommendations w/o retrieval or tools.
Evaluation metrics, how to build eval datasets, eval methodology, and a review of several benchmarks.
01. 基础RAG(Simple RAG) 方法简介 基础RAG(Retrieval-Augmented Generation)是最简单的检索增强生成方法。它通过向量化检索获取与用户查询最相关的文档片段,并将这些片段作为上下文输入给大语言模型进行答案生成。 核心代码 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45…
02. 语义分块RAG(Semantic Chunking RAG) 方法简介 语义分块RAG通过计算句子间的语义相似度来智能分块,而不是简单的固定长度分块。它使用百分位数、标准差或四分位距等方法找到语义断点,将文本分割成语义连贯的块,提升检索精度。 核心代码 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46…
Recsys & search are converging with LLMs via semantic IDs, data augmentation, and unified foundation models.
What makes a good leader? What do good leaders do? And commando, soldier, and police leadership.
Learning to automate simple agentic workflows with Amazon Q CLI, Anthropic MCP, and tmux.
Special thanks to John Schulman for a lot of super valuable feedback and direct edits on this post. Test time compute (Graves et al. 2016, Ling, et al. 2017, Cobbe et al. 2021) and Chain-of-thought (CoT) (Wei et al. 2022, Nye et al. 2021), have led to signifi…
Applying the scientific method, building via eval-driven development, and monitoring AI output.
How I started, why I write, who I write for, how I write, and more.
I’m freezing this blog and starting to post on my Substack instead. The authoring experience is much more convenient for me there. Please follow me there, and check out The Illustrated DeepSeek R-1 if you haven’t yet. And check out our How Transformer LLMs Wo…
Chip Huyen and I share what we've learned, best practices, and insights at NVIDIA GTC 2025.
Model architectures, data generation, training paradigms, and unified frameworks inspired by LLMs.