How we evaluate AI systems, and what the metrics actually measure.
4 posts in Expertise tagged Method. Clear filters
Traditional tests assume the same input always gives the same, known-correct output, and AI systems break both assumptions. We explain what AI evaluation measures instead and how it fits next to the tests you already run.
Read the postA 0.82 from a judge is not 82% correct. Here is what the number is, what it is not, and how to calibrate one before you gate a release on it.
Read the postRetrieval failures and generation failures look identical from the outside, and averaging them into one score hides the bug. Measure the stages separately.
Read the postTwenty clever prompts in a spreadsheet is not a red team. What separates an exercise you can repeat from an afternoon of anecdotes.
Read the post
