I'm working on an open-source project dokimos, because every LLM eval framework I found was Python and TypeScript-only, but a lot of companies will be building LLM apps and AI agents with Java.
Key features: - JUnit 5 integration for test-driven evals - Works with LangChain4j - Framework-agnostic - Supports custom evaluators and datasets
GitHub: https://github.com/dokimos-dev/dokimos
Would love contributions or to team up with anyone who has Java experience and wants to work on this together.