What's new in EvaliQA, August 2026

EvaliQA is live. Connect your AI system, build test plans and datasets, run evaluations and red-team attacks, and keep scoring it once it is in production, all in one workspace.

August 25, 2026 to August 31, 2026

Connect any AI system over HTTP

Add your chatbot, agent or RAG app as a connector, or import its request from a cURL command, a HAR file or a Postman collection. The Playground lets you send test requests and check your model credentials before you run anything.

Test plans with generated datasets

Describe what your system should do, pick metrics, and let EvaliQA generate single-turn or multi-turn test cases. You can edit every row by hand before you run.

Eval runs with per-row scores

Run a dataset against your system and see each answer scored by every metric, with pass rates, reasons and costs per run.

Also shipped

  • Red teaming for your AI system. Red-team plans send attack prompts to your system and judge its replies, so you can find unsafe or leaking answers before your users do.
  • Voice bot evaluation. EvaliQA calls your voice agent over the phone, holds the test conversation and scores the call like any other run.
  • Production tracing. Send traces from your app and browse them by trace, session and end user, with a timeline of every step in a session.
  • Online evaluation and alerts. Score live production traces with your metrics and get notified in Slack, Discord, Microsoft Teams, a webhook or by email when a score crosses your threshold.
  • Run evaluations from your CI/CD pipeline. Issue a workspace API token on the Integrations page and trigger eval runs from GitHub Actions, GitLab CI, Jenkins and other tools with a ready-to-paste snippet.
  • Reusable custom metrics. Save your own judging criteria once and pick them in any test plan instead of retyping them.
  • AI-written reports. Turn eval runs into a written report with the results, failures and what to fix next, ready to read in the app.
  • Compare runs side by side. Select two or more runs of a test plan to compare their scores, duration and cost, with the best and worst values highlighted.
  • Step-by-step tracing setup. A setup guide for OpenTelemetry, OpenAI, the Anthropic SDK, LangChain, LlamaIndex, AutoGen, CrewAI and custom code, with a ready-to-paste prompt for your coding assistant. It is always one click away on the project's Tracing tab.
  • Plans, billing and easy sign-in. Choose a plan and upgrade from inside the app, sign in with Google or GitHub, and reach support without leaving your workspace.

All changelogs