NodeBalancers are supposed to handle 10,000 Maximum Concurrent Connections

Apache2 defaults to only allowing 150 concurrent connections. NGINX can handle a maximum of 512 concurrent connections. In newer versions, NGINX supports up to 1024 concurrent connections, by default.

The wrk tool (https://github.com/wg/wrk) can be used to generate client requests. While it provides an option --connections described as the total number of HTTP connections to keep open. The assumption cannot be made that a single client and the network bandwidth between your remote location and your NodeBalancer would be sufficient to saturate your NodeBalancer's connection limit.

Here is a Test configuration (to be scaled as necessary):

A NodeBalancer

3 backend 2GB Shared CPU Linodes running Ubuntu 20.04 LTS with added private IP.

Apache2 is installed with defaults on each backend. Optionally, the default Apache2 web page can be customized to show which server is handling the request.

apt-get update && apt-get upgrade
apt-get install apache2
nano /var/www/http/index.html (to individualize the pages)
systemctl enable apache2.service
systemctl restart apache2.service

I used the default settings for the NodeBalancer except for changing session stickiness to none. I kept the configurations as close to the defaults as possible. Backends were added in HTTP mode. No SSL or HTTPS was configured.

For a client, I created an Ubuntu 20.04 LTS Nanode and ran the following: (see https://github.com/wg/wrk for testing options)

apt-get update && apt-get upgrade
apt-get install build-essential
apt-get install libssl-dev
apt-get install git
apt-get install unzip
git clone https://github.com/wg/wrk.git
cd wrk
make
./wrk --threads 1 --connections 3333 --duration 3m http://<NODEBALANCER.IP>/ --latency

I observed the NodeBalancer's graphs in the CloudManager

I added 2 additional clients to see concurrent connections increase.

To increase the connection limits on each backend server, I ran the following:

a2dismod mpm_event
a2enmod mpm_worker
nano /etc/apache2/mods-available/mpm_worker.conf (to increase MaxRequestWorkers)
systemctl restart apache2.service

mpm_worker.conf was updated as follows:

<IfModule mpm_worker_module>
        StartServers             10
        MinSpareThreads          75
        MaxSpareThreads          250
        ThreadLimit              64
        ThreadsPerChild          32
        MaxRequestWorkers        8000
        MaxConnectionsPerChild   0
</IfModule>

Additional backend servers can be added as needed to further observe more concurrent connections on the NodeBalancer.

References:

https://github.com/wg/wrk

https://ubiq.co/tech-blog/how-to-fix-nginx-worker-connections-are-not-enough/#:~:text=How%20many%20connections%20can%20NGINX,1024%20concurrent%20connections%2C%20by%20default.

https://ubiq.co/tech-blog/increase-max-connections-apache/