The Hidden Network: Ray Control-Plane Exposure in Distributed LLM Inference on Kubernetes (Empirical Study, EKS)
Source Reddit NetSec
Mentions 1 CVE
Mentioned vulnerabilities
Discussion
An empirical study evaluated the network exposure of Ray control-plane components in distributed large language model (LLM) inference deployments on Kubernetes using Amazon EKS 1.35. The study found that multiple Ray control-plane sockets were not declared in pod metadata, allowing an unprivileged pod in a different namespace to access Ray GCS and raylet RPC services over unauthenticated cleartext gRPC. Default security scanners did not detect this exposure. A default-deny NetworkPolicy blocked access in steady state, and WireGuard encryption added modest overhead. The Ray Job API responded to unauthenticated requests on CPU Ray images but was not active in GPU deployments. The findings reflect vendor default configurations rather than new vulnerabilities, and a related CVE affecting vLLM was already fixed upstream.
r/netsec (opens in a new tab)u/No-Peanut-69880 points0 comments
We evaluated the runtime network surface of distributed LLM serving (KubeRay + vLLM pipeline parallelism) on an Amazon EKS 1.35 cluster from the perspective of an unprivileged neighbour pod in an unrelated namespace.
Summary of observations:
- 15 of 17 observed Ray listening sockets were omitted from pod declared containerPort metadata.
- An unprivileged neighbour pod in an unrelated namespace reached Ray GCS (6379) and raylet RPC (10002 to 10006) over unauthenticated cleartext gRPC.
- Four default security scanners (Trivy, Checkov, Kubescape, kube-linter) did not inspect the RayCluster custom resource, failing to represent runtime exposure.
- An ingress default-deny NetworkPolicy blocked all probed Ray ports from the neighbour pod in steady state.
- WireGuard encryption via Cilium chained with AWS VPC CNI carried the workload with an observed 3.4% to 6.5% throughput drop across a bracketed single-run test.
- The Ray Job API (8265) answered unauthenticated requests on default CPU Ray images, but was not listening in the GPU vLLM deployment (RCE against the GPU inference stack was not demonstrated).
All test manifests, scanner outputs, network logs, and repro scripts are open source:
https://github.com/Sorami-Consulting-AU/distributed-llm-inference-hidden-network