Tag: model-evaluation

Showing 1-2 of 2

Jan 29, 20266 min readFounder NotesIntermediate5 min build

Anthropic’s 1.5M-chat analysis identifies reality, belief and action disempowerment in Claude

Anthropic analyzed 1.5M Claude conversations and defines three disempowerment patterns—reality, belief, action. Rare by percent but meaningful at scale; includes monitoring guidance.

ai-safety chatbots Anthropic user-harm model-evaluation

+2 more

product founder-notes

Model BreakdownsFrance

Open

Dec 09, 20257 min readTooling Deep DiveIntermediate5 min build

DeepMind's FACTS Benchmark Suite: a claim-level framework and quick-start checklist for evaluating LLM factuality

DeepMind's FACTS Benchmark Suite evaluates LLM factuality with claim-level tests, error taxonomies and provenance checks. Includes a 5-item quick-start checklist and decision framework.

FACTS factuality benchmarks LLMs model-evaluation

+3 more

DeepMind tooling metrics