# Scaling to 100,000 RPS: Deploying Distributed Load Testing on Kubernetes with Helm and Docker URL: https://hakangul.lovable.app/blog/locustpilot-part-4-scaling-100000-rps-kubernetes-helm-docker Author: Hakan Gül — Team Lead Senior Test Automation Engineer Published: 2026-08-29 Updated: 2026-08-29 Category: devops Tags: kubernetes, helm, docker, devops, performance-testing, sdet > How to scale LocustPilot to 100,000+ RPS using Kubernetes Helm charts and distributed Master-Worker container orchestration. ## English **TL;DR:** Single-machine load tests inevitably hit CPU and socket descriptor limits when pushing beyond a few thousand requests per second. In LocustPilot, we packaged the entire control plane and distributed worker architecture into Kubernetes Helm charts (`helm/locust`), allowing engineering teams to scale to 100,000+ RPS across cloud nodes in minutes with automated resource isolation. ## When a Single Machine Hits the Wall A single modern laptop or cloud VM running Locust can comfortably generate between 2,000 and 5,000 requests per second (RPS). However, when preparing for enterprise-scale traffic events (e.g. streaming sports events, flash sales, or banking cutovers), you need **tens of thousands of concurrent users**. Attempting this on one machine leads to misleading test results: 1. **CPU Saturation:** Python processes hit 100% CPU on single cores, artificially increasing response times on the client side. 2. **Socket Exhaustion (TIME_WAIT):** The operating system runs out of ephemeral ports, causing false `Connection Refused` errors that have nothing to do with your backend. 3. **Bandwidth Throttling:** Cloud VM network interfaces bottleneck before your target servers are even stressed. To scale reliably, you must distribute load generation across a cluster of worker nodes. --- ## The Distributed Master-Worker Architecture In distributed mode, Locust separates responsibilities across two roles: * **Master Node:** Runs the LocustPilot web control center, coordinates worker connections, aggregates incoming metrics, and streams telemetry to ReportPortal. It generates zero HTTP traffic itself. * **Worker Nodes:** Connect to the master, spawn virtual users, and bombard the target backend with requests. ``` ┌───────────────────────────────┐ │ LocustPilot Master (UI) │ │ (Streamlit + ReportPortal) │ └──────────────┬────────────────┘ │ Port 5557 (ZMQ) ┌────────────┼────────────┐ │ │ │ ┌─────▼────┐ ┌─────▼────┐ ┌─────▼────┐ │ Worker 1 │ │ Worker 2 │ │ Worker N │ └─────┬────┘ └─────┬────┘ └─────┬────┘ │ │ │ └────────────┼────────────┘ ▼ [ Target Microservices ] ``` --- ## Packaging for Kubernetes with Helm (`helm/locust`) LocustPilot provides a production-ready Helm chart located in `helm/locust/`. ### 1. The Optimized Container (`k8s.Dockerfile`) ```dockerfile FROM python:3.10-slim WORKDIR /app RUN apt-get update && apt-get install -y --no-install-recommends \ curl build-essential && \ rm -rf /var/lib/apt/lists/* COPY requirements.txt . RUN pip install --no-cache-dir -r requirements.txt COPY . . EXPOSE 8501 5557 CMD ["streamlit", "run", "app.py", "--server.port=8501", "--server.address=0.0.0.0"] ``` --- ![DevOps Pipeline Configuration](https://images.unsplash.com/photo-1618401471353-b98aedd04e11?auto=format&fit=crop&w=1200&q=80) *Photo by [Fotis Fotopoulos](https://unsplash.com/@ffstop) on Unsplash* --- ### 2. Deploying the Cluster with One Command Using the pre-built Helm chart, deploying the entire distributed load testing cluster with ReportPortal telemetry requires a single command: ```bash helm upgrade --install locust ./helm/locust \ --namespace loadtest \ --create-namespace \ --set image.repository=my-registry.io/locust-pilot \ --set image.tag=latest \ --set replicaCount=10 \ --set extraEnv[0].name=RP_TOKEN --set extraEnv[0].value=$RP_TOKEN \ --set extraEnv[1].name=RP_ENDPOINT --set extraEnv[1].value=$RP_ENDPOINT \ --set extraEnv[2].name=RP_PROJECT --set extraEnv[2].value=$RP_PROJECT ``` ### 3. Tuning OS Limits for Massive Scale When running high-volume workers inside Kubernetes, ensure your Pod security contexts allow high file descriptors: ```yaml # helm/locust/values.yaml worker: resources: limits: cpu: 2000m memory: 2Gi requests: cpu: 1000m memory: 1Gi sysctls: - name: net.ipv4.tcp_tw_reuse value: "1" - name: net.core.somaxconn value: "65535" ``` --- ## What Is Next? In the **final installment (Part 5)**, we will connect all the pieces into your continuous delivery pipeline: **Shift-Left Performance Testing with Automated Quality Gates in GitHub Actions and Bitbucket Pipelines**. 👉 **[Explore the Helm Charts on GitHub](https://github.com/hakanngul/LocustPilot)** --- ## FAQ ### How many workers are needed for 50,000 RPS? Depending on endpoint payload size and SSL overhead, each CPU core running `FastHttpUser` can typically generate 1,500–3,000 RPS. A cluster of 20–25 worker pods can comfortably sustain 50,000+ RPS. ### Can workers be dynamically auto-scaled? Yes. Kubernetes Horizontal Pod Autoscaler (HPA) can scale worker pods based on CPU utilization or custom test duration parameters. ## Türkçe **TL;DR:** Single-machine load tests inevitably hit CPU and socket descriptor limits when pushing beyond a few thousand requests per second. In LocustPilot, we packaged the entire control plane and distributed worker architecture into Kubernetes Helm charts (`helm/locust`), allowing engineering teams to scale to 100,000+ RPS across cloud nodes in minutes with automated resource isolation. ## When a Single Machine Hits the Wall A single modern laptop or cloud VM running Locust can comfortably generate between 2,000 and 5,000 requests per second (RPS). However, when preparing for enterprise-scale traffic events (e.g. streaming sports events, flash sales, or banking cutovers), you need **tens of thousands of concurrent users**. Attempting this on one machine leads to misleading test results: 1. **CPU Saturation:** Python processes hit 100% CPU on single cores, artificially increasing response times on the client side. 2. **Socket Exhaustion (TIME_WAIT):** The operating system runs out of ephemeral ports, causing false `Connection Refused` errors that have nothing to do with your backend. 3. **Bandwidth Throttling:** Cloud VM network interfaces bottleneck before your target servers are even stressed. To scale reliably, you must distribute load generation across a cluster of worker nodes. --- ## The Distributed Master-Worker Architecture In distributed mode, Locust separates responsibilities across two roles: * **Master Node:** Runs the LocustPilot web control center, coordinates worker connections, aggregates incoming metrics, and streams telemetry to ReportPortal. It generates zero HTTP traffic itself. * **Worker Nodes:** Connect to the master, spawn virtual users, and bombard the target backend with requests. ``` ┌───────────────────────────────┐ │ LocustPilot Master (UI) │ │ (Streamlit + ReportPortal) │ └──────────────┬────────────────┘ │ Port 5557 (ZMQ) ┌────────────┼────────────┐ │ │ │ ┌─────▼────┐ ┌─────▼────┐ ┌─────▼────┐ │ Worker 1 │ │ Worker 2 │ │ Worker N │ └─────┬────┘ └─────┬────┘ └─────┬────┘ │ │ │ └────────────┼────────────┘ ▼ [ Target Microservices ] ``` --- ## Packaging for Kubernetes with Helm (`helm/locust`) LocustPilot provides a production-ready Helm chart located in `helm/locust/`. ### 1. The Optimized Container (`k8s.Dockerfile`) ```dockerfile FROM python:3.10-slim WORKDIR /app RUN apt-get update && apt-get install -y --no-install-recommends \ curl build-essential && \ rm -rf /var/lib/apt/lists/* COPY requirements.txt . RUN pip install --no-cache-dir -r requirements.txt COPY . . EXPOSE 8501 5557 CMD ["streamlit", "run", "app.py", "--server.port=8501", "--server.address=0.0.0.0"] ``` --- ![DevOps Pipeline Configuration](https://images.unsplash.com/photo-1618401471353-b98aedd04e11?auto=format&fit=crop&w=1200&q=80) *Photo by [Fotis Fotopoulos](https://unsplash.com/@ffstop) on Unsplash* --- ### 2. Deploying the Cluster with One Command Using the pre-built Helm chart, deploying the entire distributed load testing cluster with ReportPortal telemetry requires a single command: ```bash helm upgrade --install locust ./helm/locust \ --namespace loadtest \ --create-namespace \ --set image.repository=my-registry.io/locust-pilot \ --set image.tag=latest \ --set replicaCount=10 \ --set extraEnv[0].name=RP_TOKEN --set extraEnv[0].value=$RP_TOKEN \ --set extraEnv[1].name=RP_ENDPOINT --set extraEnv[1].value=$RP_ENDPOINT \ --set extraEnv[2].name=RP_PROJECT --set extraEnv[2].value=$RP_PROJECT ``` ### 3. Tuning OS Limits for Massive Scale When running high-volume workers inside Kubernetes, ensure your Pod security contexts allow high file descriptors: ```yaml # helm/locust/values.yaml worker: resources: limits: cpu: 2000m memory: 2Gi requests: cpu: 1000m memory: 1Gi sysctls: - name: net.ipv4.tcp_tw_reuse value: "1" - name: net.core.somaxconn value: "65535" ``` --- ## What Is Next? In the **final installment (Part 5)**, we will connect all the pieces into your continuous delivery pipeline: **Shift-Left Performance Testing with Automated Quality Gates in GitHub Actions and Bitbucket Pipelines**. 👉 **[Explore the Helm Charts on GitHub](https://github.com/hakanngul/LocustPilot)** --- ## FAQ ### How many workers are needed for 50,000 RPS? Depending on endpoint payload size and SSL overhead, each CPU core running `FastHttpUser` can typically generate 1,500–3,000 RPS. A cluster of 20–25 worker pods can comfortably sustain 50,000+ RPS. ### Can workers be dynamically auto-scaled? Yes. Kubernetes Horizontal Pod Autoscaler (HPA) can scale worker pods based on CPU utilization or custom test duration parameters.