.. Reminder for header structure: Parts (H1) : #################### with overline Chapters (H2) : ******************** with overline Sections (H3) : ==================== Subsections (H4) : -------------------- Subsubsections (H5) : ^^^^^^^^^^^^^^^^^^^^ Paragraphs (H6) : """"""""""""""""""""" .. meta:: :description: Configuring WAPT repositories :keywords: multi-repo, multi-repositories, replicate, replication, bandwidth, Edge Computing .. |enterprise_feature| image:: wapt-resources/icon_wapt_enterprise.png :scale: 3% :alt: WAPT Enterprise feature only ***************************** Configuring WAPT repositories ***************************** .. _repository_location: Repository location on the WAPT Server ====================================== .. list-table:: :header-rows: 1 :widths: auto :align: center * - Operating System - Value * - Debian and derivatives - :file:`/var/www/wapt/` * - RedHat and derivatives - :file:`/var/www/html/wapt/` * - Windows - :file:`C:\\wapt\\waptserver\\repository` .. _replication_usage: Replicating a repository |enterprise_feature| ============================================= Functional overview ------------------- .. hint:: **The method explained below is for the Enterprise version only**. The deprecated and **unsupported** `Syncthing `_ method may be used for the **Discovery** version of WAPT. WAPT Agent replication role ^^^^^^^^^^^^^^^^^^^^^^^^^^^ Repository replication can be enabled using a WAPT Agent installed on an existing host, a dedicated appliance or a Virtual Host. The replication role is deployed through a WAPT package that enables the :program:`Nginx web server` and configures scheduling, packages types, packages sync, and much more. This feature allows WAPT Agents to find dynamically their closest available WAPT repository from a list of rules stored on the WAPT Server. Replication behavior ^^^^^^^^^^^^^^^^^^^^ Repository replication in WAPT is handled by WAPT Agents natively. It is based on a :file:`sync.json` file which indexes every files present in these folders: * :file:`wapt`; * :file:`waptwua`; * :file:`wapt-host`; * :file:`wads`. Enabling replication has the following effects: * Once :code:`enable_remote_repo` is enabled on a WAPT Agent, it will sync packages locally inside the :file:`local_repo_path` folder. * It adds the WAPT Agent in the :guilabel:`Repositories` tab as a Remote repository, enabling new actions such as :guilabel:`Force Sync` or :guilabel:`Check files`. * By default, only the :guilabel:`wapt` folder is synchronized, you can select which folder to sync by adding up elements in :code:`remote_repo_dirs` parameters. * Synchronization period can be configured with :code:`local_repo_time_for_sync_start` and :code:`local_repo_time_for_sync_stop` parameters. * Bandwidth allocated to sync can be configured with :code:`local_repo_limit_bandwidth`. Every parameters of WAPT repository sync **MUST** be set in the ``[repo-sync]`` section of the WAPT Agent :file:`wapt-get.ini` configuration file. .. figure:: wapt-resources/wapt_concept_replication_flow-diagram.png :align: center :alt: Flow diagram of the replication behavior of the WAPT Agent Flow diagram of the replication behavior of the WAPT Agent WAPT Agent configuration ------------------------ To enable replication on an :ref:`existing WAPT Agent ` (Linux / Windows), you need to set in the ``[repo-sync]`` section in the :file:`wapt-get.ini` configuration file of the WAPT Agent. .. hint:: If you use DNS, please remind to create a DNS entry for your WAPT agent. .. list-table:: WAPT Agent replication configuration :header-rows: 1 :widths: auto :align: center * - Options (Default Value) - Definition - Example * - :code:`enable_remote_repo` (default ``False``) - Enables remote repository to synchronize with the main repository. - :code:`enable_remote_repo` (default ``True``) * - :code:`local_repo_path` (default ``WAPT root dir/repository``) - Sets the path to the root directory of the local repository for WAPT packages. - :code:`local_repo_path` = ``/var/www/`` * - :code:`local_repo_time_for_sync_start` (default ``None``) - Sets synchronization start time (HH:MM / 24h format). - :code:`local_repo_time_for_sync_start` = ``22:30`` * - :code:`local_repo_time_for_sync_end` (default ``None``) - Sets synchronization stop time (HH:MM / 24h format). - :code:`local_repo_time_for_sync_end` = ``05:30`` * - :code:`local_repo_sync_task_period` (default ``None``) - Sets synchronization periodicity (minutes). - :code:`local_repo_sync_task_period` = ``25`` * - :code:`local_repo_limit_bandwidth` (default ``None``) - Sets synchronization allowed bandwidth (Mbits/s). - :code:`local_repo_limit_bandwidth` = ``2`` * - :code:`remote_repo_dirs` (default ``wapt,waptwua``) - Defines folders to synchronize. - :code:`remote_repo_dirs` = ``wapt,waptwua`` * - :code:`use_repo_rules` (default ``False``) - Enables for use :ref:`repository rules `. - :code:`use_repo_rules` = ``True`` * - :code:`sync_only_forced` (default ``False``) - Synchronizes the repository only if forced. - :code:`sync_only_forced` = ``True`` .. warning:: If you modify manually :file:`wapt-get.ini` on the remote repository, you need to restart the **WAPT service**. .. note:: A ready-to-use `WAPT package `_ is available in **Tranquil IT public store** to enable repository replication on Windows or Linux based WAPT Agents. This way, the desktop of the welcome desk in a remote office of any organization may become a WAPT repository to distribute WAPT packages to the fleet of computers in the remote office. This special package: * Installs and enables the :program:`Nginx web server` on the remote repository. * Configures :program:`Nginx` virtualhost environment. * Enables remote repository configuration in :file:`wapt-get.ini`. It is possible to automatically configure repositories with your own preferred values by editing this package. Below is an example :file:`wapt-get.ini` file for a WAPT Agent. .. code-block:: ini [global] ... use_repo_rules = True [repo-sync] enable_remote_repo = True local_repo_path = D:\WAPT\ local_repo_time_for_sync_start = 20:30 local_repo_time_for_sync_end = 05:30 local_repo_sync_task_period = 25 local_repo_limit_bandwidth = 4 remote_repo_dirs = wapt, waptwua WAPT Server configuration ------------------------- By default, the WAPT Server will know which WAPT Agents are configured as remote repositories and it will list them in the WAPT Console. .. _repository_rules: Repository rules ---------------- When a WAPT Agent has been configured as a repository, it will automatically retrieve its :file:`rules.json` file from the WAPT Server. The :file:`rules.json` file is a signed :mimetype:`.json` file that contains a list of sorted rules to apply to the remote WAPT Agents, so they may connect to their most appropriate repositories. If no rules can be matched, the WAPT Agent will fallback to the ``repo_url`` attribute defined in its :file:`wapt-get.ini` configuration file. .. figure:: wapt-resources/wapt_concept_repository_rules_flow-diagram.png :align: center :alt: Flow diagram for the replication behavior of the WAPT Agent WAPT Agent ^^^^^^^^^^ .. warning:: **If you have configured GeoIP redirects on Nginx, you should disable it as it might conflict with repository rules**. To enable WAPT Agent repository rules, you **MUST** enable this setting in the ``[global]`` section of the :file:`wapt-get.ini` configuration file of the WAPT Agent. .. list-table:: :header-rows: 1 :widths: auto :align: center * - Options (Default Value) - Description - Example * - :code:`use_repo_rules` (default ``False``) - For using :ref:`replicating repository `. - :code:`use_repo_rules` = ``True`` Below is an example :file:`wapt-get.ini` file for a WAPT Agent. .. code-block:: ini [global] ... use_repo_rules = True .. note:: It is possible to enable this option when :ref:`generating a WAPT Agent `. WAPT Server ^^^^^^^^^^^ On the WAPT Server, remote repositories functionality is automatically enabled. To verify, edit :file:`waptserver.ini` and read :code:`remote_repo_support` value. .. list-table:: :header-rows: 1 :widths: auto :align: center * - Options (Default Value) - Example value - Definition * - :code:`remote_repo_support` - ``True`` - Enables the WAPT Server to serve as a repository. WAPT Console ^^^^^^^^^^^^ Repository rules can be managed from the WAPT Console and are based on several parameters: .. list-table:: Available parameters for repository rules :header-rows: 1 :widths: auto * - Options - Example value - Description * - :guilabel:`Agent IP` - ``192.168.85.0/24`` - Defines a repository rule based on Agent IP sub-network. * - :guilabel:`Domain` - ``ad.mydomain.lan`` - Defines a repository rule based on Active Directory domain name. * - :guilabel:`Hostname` - ``desktop-04feb1`` - Defines a repository rule based on the hostname of the WAPT Agent. * - :guilabel:`Public IP` - ``256.89.299.22/32`` - Defines a repository rule based on the public IP address (NATed hosts). * - :guilabel:`Site` - ``Paris-HQ`` - Defines a repository rule based on Active Directory Sites and Services. Adding a repository rule """""""""""""""""""""""" To add a new repository rule, go to the :guilabel:`Repositories` tab in the WAPT Console and click on the :guilabel:`Add rule` button. .. list-table:: Options for repository rules :header-rows: 1 :widths: auto :align: center * - Options - Example value - Description * - :guilabel:`Name` - repo25 - Defines the name for the repository rule. * - :guilabel:`Condition` - AGENT IP - Defines the condition to match for the repository rule to apply (see above). * - :guilabel:`Value` - 192.168.25.0/24 - Defines the value when the condition applies. If :guilabel:`NOT` is checked, the value applies to the reverse of the condition. * - :guilabel:`Repository URL` - https://repo25.mydomain.lan - Defines the list of available remote repositories. The list includes http://download.windowsupdate.com/microsoftupdate/v6/wsusscan/ to allow directly downloading of Windows Updates by the remote repositories to preserve WAPT Server bandwidth. * - :guilabel:`Package type` - WAPT - Defines what :ref:`types of packages ` are replicated. * - :guilabel:`Other` - No fallbacks - The option :guilabel:`No Fallback` will prevent from falling back to the main WAPT Server and will avoid undesired network congestion if the remote repository becomes temporarily unavailable. * The option :guilabel:`Proxy` will need to be set if the remote repository is required to connect via a proxy. .. figure:: wapt-resources/wapt_console_repository-rule_container-window.png :align: center :alt: Window for setting repository rules in the WAPT Console Window for setting repository rules in the WAPT Console You can then choose from the different above parameters and affect values to a specific secondary WAPT repository. .. warning:: **The rules are applied from top to bottom**. **The first rule that matches the conditions overrides all the other rules placed under.** .. danger:: Do not forget to save the replication rules. .. _wapt_multi-repository: Multiple repositories ===================== .. _wapt_agent_multi-repository: Similar to Debian repositories, it is possible for the WAPT Agent to use multiple repositories for updating package. The WAPT Agents will check all repositories. .. danger:: If you use this functionality, **KNOW WHAT YOU ARE DOING**. When using repositories with different signers, the additional signer's public certificates **MUST** be added to :file:`C:\\Program Files (x86)\\wapt\\ssl` on Windows or :file:`/opt/wapt/ssl` on Linux and macOS, therefore, you **MUST** trust their work and their signature. You then **MUST** deploy the WAPT Agents with both keys. Please refer to the documentation on :ref:`creating the WAPT Agent ` to add certificates. WAPT Agent configuration ------------------------ This parameters are modifiable on :file:`wapt-get.ini` file. Description of available parameters ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ .. list-table:: Options for defining multiple repositories :header-rows: 1 :widths: auto :align: center * - Options - Example value - Description * - ``[global]`` - repositories = wapt-templates,private - Defines the repositories, for example ``wapt-templates`` and ``private``, where their settings are set in additional ``[section]`` sections of the :file:`wapt-get.ini` file. * - ``[section]`` - [wapt-templates] repo_url=https://store.wapt.fr/wapt verify_cert = True [private] repo_url=https://srvwapt.mydomain.lan/wapt verify_cert = False - Defines the properties of each repository declared in the ``[global]`` section of the :file:`wapt-get.ini` file. .. list-table:: Options for repository properties :header-rows: 1 :widths: auto :align: center * - Options (Default Value) - Description - Example * - :code:`http_proxy` (default ``None``) - Defines the HTTP proxy address. - http_proxy = http://user:pwd@host_fqdn:port * - :code:`repo_url` (default ``None``) - Defines the address of the main WAPT repository. - repo_url = https://srvwapt.mydomain.lan/wapt * - :code:`timeout` (default ``None``) - Defines the timeout when connecting to remote repositories (in miliseconds). - timeout = 5000 * - :code:`use_http_proxy_for_repo` (default ``False``) - Defines whether a proxy needs to be set to access the repositories. - use_http_proxy_for_repo = True * - :code:`verify_cert` (default ``None``) - Defines whether :ref:`HTTPS certificates of the repository needs to be verified `, and if so defines the path to the certificate bundle. - verify_cert = True .. note:: The WAPT Agent will look for updates in all repositories defined in its :file:`wapt-get.ini` configuration file when doing a :command:`wapt-get search`. More info on :ref:`using WAPT with the command line interface `. .. _wapt_console_multi-repository: Configuring the WAPT Console for using multiple repositories ------------------------------------------------------------ After having configured the WAPT Agent for using multiple repositories, we can make the repositories show up in the WAPT Console. To do so, modify the :file:`%appdata%\\local\\waptconsole\\waptconsole.ini` file. Example: .. code-block:: ini [wapt-template] repo_url = https://wapt.tranquil.it/wapt http_proxy = verify_cert = True public_certs_dir = client_certificate = client_private_key = timeout = 5 [private] repo_url = https://srvwapt.mydomain.lan/wapt http_proxy = verify_cert = False public_certs_dir = client_certificate = client_private_key = timeout = 5 .. list-table:: Options for external repositories in the WAPT Console :header-rows: 1 :widths: auto :align: center * - Options (Default Value) - Description - Example * - :code:`client_certificate` (default ``None``) - Defines the folder that contains the certificates used to authenticate downloaded external packages. - client_certificate = :file:`C:\\Program Files (x86)\\wapt\\ssl\\server\\srvwapt.mydomain.lan.crt (on Windows)` * - :code:`client_private_key` = None - Defines the folder that contains the private key. - client_private_key = C:\\Program Files (x86)\\wapt\\ssl\\server\\srvwapt.mydomain.lan.pem (on Windows) * - :code:`http_proxy` (default ``None``) - Defines the HTTP proxy address. - http_proxy = http://user:pwd@srvproxy.mydomain.lan:port * - :code:`public_certs_dir` = - Defines the folder that contains the certificates used to authenticate downloaded external packages. - public_certs_dir = C:\\private * - :code:`repo_url` (default ``None``) - Defines the address of the main WAPT repository. - repo_url = https://srvwapt.mydomain.lan/wapt * - :code:`timeout` (default ``None``) - Defines the timeout when connecting to remote repositories (in miliseconds). - timeout = 5000 * - :code:`verify_cert` (default ``None``) - Defines whether :ref:`HTTPS certificates of the repository needs to be verified `, and if so defines the path to the certificate bundle. - verify_cert = True