Built for Scale
EDDI runs on Java 25 with Quarkus, leveraging Virtual Threads (Project Loom) for massive I/O-bound concurrency. Unlike Node.js event loops, Virtual Threads provide genuine multi-threaded parallelism with minimal overhead.
Why Virtual Threads Matter for AI Agents
Modern AI agents are fundamentally I/O-bound: they spend most of their time waiting for LLM API responses, vector store queries, external tool calls, and database operations. Traditional platform threads (and Node.js event loops) struggle with this pattern β each blocking call either occupies an expensive OS thread or forces complex callback chains.
Java Virtual Threads solve this elegantly: when an agent awaits an LLM response, its virtual thread yields seamlessly, freeing the underlying carrier thread to process other agent operations. This means EDDI can serve millions of concurrent agent conversations with a fraction of the hardware required by traditional architectures.
Performance Highlights
- Virtual Threads β Millions of lightweight threads for concurrent LLM calls, each yielding during I/O waits
- Quarkus Runtime β Cloud-native Java with dev-mode hot reload, optimized for containers. Quarkus handles 2.7Γ more transactions per second than legacy Spring Boot while starting 2.3Γ faster and using half the memory
- Loom-Friendly Connection Pools β Agroal connection pooling avoids the ThreadLocal bottlenecks that can affect traditional connection pools under virtual thread workloads
- NATS JetStream β Horizontal scaling with event-driven architecture for multi-instance coordination
- Dual Database β MongoDB or PostgreSQL, switch with one env var. Single Docker image for both
- SSE Streaming β Real-time chat responses, group discussion feeds, and live log streaming via Server-Sent Events
- One-Command Install β Interactive wizard deploys EDDI + database + starter agent via Docker Compose
- Red Hat Certified β Container certification with automated preflight checks in CI/CD
Performance in Context
No runtime wins in every scenario. Industry benchmarks show that Node.js can outperform Java in highly specific, pure-I/O routing scenarios. However, Java consistently outperforms Node.js by 30β68% across scenarios involving CPU-intensive tasks β which is exactly what AI agents require. Modern agents perform a complex mix of massive I/O (LLM API calls, vector queries) and intensive computation (data transformation, routing logic, embedding processing, confidence evaluation). This mixed-workload profile is where virtual threads on Quarkus deliver their strongest advantage.
EDDI's architecture is intentionally optimized for this reality: Quarkus with virtual threads for the I/O-heavy agent orchestration layer, combined with Loom-friendly connection pools (Agroal) that avoid the ThreadLocal contention issues documented by the community with traditional pooling libraries under high virtual thread concurrency.