Notes on evaluating AI

Announcements, hands-on expertise and honest comparisons from the team building EvaliQA. One problem per post, from the work we do with clients.

2 posts tagged LLM judges. Clear filters

Expertise24 September 202617 min read

What is AI evaluation and how it differs from testing

Traditional tests assume the same input always gives the same, known-correct output, and AI systems break both assumptions. We explain what AI evaluation measures instead and how it fits next to the tests you already run.

Read the post

Expertise8 September 20263 min read

What an LLM judge score actually means

A 0.82 from a judge is not 82% correct. Here is what the number is, what it is not, and how to calibrate one before you gate a release on it.

LLM judges Metrics Method

Read the post