Client BenchmarkBenchmark Design & Evaluation
Galahad vs. vLLM Inference Benchmark
Designed and ran an A/B benchmark on an NVIDIA A40. With the cache sized for the workload, Galahad cut median revisit latency from 3.29 s to 0.91 s and used 19.5% less GPU energy.
Designed the workload, ran both configurations, and published the methodology, raw logs, and results so others could reproduce the comparison. The measured speedup applies to repeated long-context requests in this benchmark.
Validated Metrics & Results
0.91 s vs. 3.29 s median revisit latency (3.6× faster)
19.5% less measured GPU energy on the benchmark workload
36/36 correct answers across three Galahad runs
Tech Stack
PythonvLLMNVIDIA A40RunPod