A fully local LLM agent platform. It traces model calls, caches repeated requests, and runs automated quality and safety evaluations on each code push.
Ollama / FastAPI / Redis / Celery / Grafana
Implementation notes
- Asynchronous task processing with Redis and Celery.
- Grafana dashboards for latency, cache hit rate, and request history.
- CI blocks deployment when safety-refusal evaluations fail.