Configuring WAPT Server for large deployments¶
The default operating system, Nginx and Postgresql settings are adapted for around 400 WAPT agents. If you have more than 400 clients it is necessary to modify a few system level parameters along with PostgreSQL database, Nginx web and WAPT Server python server.
In the future the postconf.sh script might take charge of this configuration depending on the expected number of client computers.
With the following parameters, one WAPT Server should scale up to around 5000 concurrent active clients. You may have more clients in the database if they are not all running at the same time. If you have more than 5000 clients it is recommended to have more than one WAPT Server.
The limit in the number of end point clients is due to the bottleneck in the python code and the PostgreSQL backend. WAPT performance gets better with time and in the future WAPT Server might support a large base on a single server. However the Nginx part scales very well and it can takes full advantage of a 10Gbps connection for high load package deployments.
Configuration changes for better performance¶
Note
The parameters to be modified below are linked together and should be modified globally and not individually.
Configuring Nginx¶
In the /etc/nginx/nginx.conf file (for Windows C:\wapt\waptserver\nginx\conf\nginx.conf), modify worker_connections
parameter. The value should be around 2.5 times the number of WAPT clients
(n connections for websockets and n connections for package downloads
and inventory upload + some margin).
events {
worker_connections 4096;
}
Then upgrade the number of filedescriptors
in the /etc/nginx/nginx.conf file (for Windows C:\wapt\waptserver\nginx\conf\nginx.conf):
worker_rlimit_nofile 32768;
Configuring the Linux System¶
Increase the number of filedescriptors. The system unit file asks for an increase in the allowed number of filedescriptors (LimitNOFILE=32768). We should have the same thing for Nginx. There are a few limits to modify.
First we modify system wide the number of filedescriptors allowed for Nginx and WAPT.
create the
/etc/security/limits.d/wapt.conf:cat > /etc/security/limits.d/wapt.conf <<EOF wapt hard nofile 32768 wapt soft nofile 32768 www-data hard nofile 32768 www-data soft nofile 32768 EOF
Nginx serves as a reverse proxy and makes quite a lot of connections. Each WAPT client keeps a websocket connection up all the time in order to respond to actions from the WAPT Server.
The Linux kernel has a protection against having too many TCP connections opened at the same time and one may get the SYN flooding on port message in the Nginx log. In order to avoid these messages, it is necessary to modify the two following parameters. It must around 1.5 times the number of WAPT clients.
cat > /etc/sysctl.d/wapt.conf <<EOF
net.ipv4.tcp_max_syn_backlog=4096
net.core.somaxconn=4096
EOF
sysctl --system
Configuring the PostgreSQL database¶
A higher number of clients need a higher number of connections to the PostgreSQL
database. In the postgresql.conf
file (file:/etc/postgresql/11/main/postgresql.conf on debian 10 for example or for Windows C:\wapt\waptserver\pgsql9.6_data\postgresql.conf),
you need to increase the following parameter to approximately 1/4
the number of active WAPT agents.
max_connections = 1000
Configuring the WAPT Server¶
In /opt/wapt/conf/waptserver.ini file (for Windows C:\wapt\conf\waptserver.ini, db_max_connections
should be equal to PostgreSQL max_connections minus 10 (PostgreSQL needs
to keep some connections for its housekeeping stuff). The max_clients
parameter should be set around 1.2 times the number of WAPT agents:
[options]
...
max_clients = 4096
db_max_connections = 990
Configuration for large package upload¶
Depending on the partitioning of your WAPT server you might have to be careful with the Nginx temporary file upload directory. Nginx acts as a reverse proxy for the WAPTServer Python engine and its does a caching of packages uploaded when uploading a new package from the console.
The packages are stored in the /var/lib/nginx/proxy directory.
You have to make sure that the partition hosting this directory is large enough.
You may change this directory location using the following Nginx
configuration parameter.
$client_body_temp_path