.. Reminder for header structure: Parts (H1) : #################### with overline Chapters (H2) : ******************** with overline Sections (H3) : ==================== Subsections (H4) : -------------------- Subsubsections (H5) : ^^^^^^^^^^^^^^^^^^^^ Paragraphs (H6) : """"""""""""""""""""" .. meta:: :description: Configuring WAPT repositories :keywords: multi-repo, multi-repositories, replicate, replication, bandwidth, Edge Computing .. |enterprise_feature| image:: wapt-resources/wapt_enterprise.png :scale: 3% :alt: WAPT Enterprise feature only ***************************** Configuring WAPT repositories ***************************** .. _repository_location: Repository location on server ============================= .. list-table:: :header-rows: 1 :widths: auto :align: center * - Operating System - Value * - Debian / Ubuntu - /var/www/wapt/ * - Centos / RedHat - /var/www/html/wapt/ * - Windows - C:\\wapt\\waptserver\\repository .. _replication_usage: Replicating a repository |enterprise_feature| ============================================= Functional overview ------------------- .. hint:: **The method explained below is for the Enterprise version only**. The deprecated and **unsupported** `Syncthing `_ method may be used for the Discovery versions of WAPT. WAPT agent replication role ^^^^^^^^^^^^^^^^^^^^^^^^^^^ Repository replication can be enabled using a WAPT agent installed on an existing machine, a dedicated appliance or a Virtual Machine. The replication role is deployed through a WAPT package that enables the :program:`Nginx web server` and configures scheduling, packages types, packages sync, and much more. This feature allows WAPT agents to find dynamically their closest available WAPT repository from a list of rules stored on the WAPT server. Replication behavior ^^^^^^^^^^^^^^^^^^^^ Repository replication in WAPT is handled by WAPT agents natively. It is based on a :file:`sync.json` file which indexes every files present in these folders: * wapt; * waptwua; * wapt-host. Enabling replication has the following effects: * Once ``enable_remote_repo`` is enabled on a WAPT agent, it will sync packages locally inside the :file:`local_repo_path` folder. * It adds the WAPT agent in the :guilabel:`Repositories` tab as a Remote repository, enabling new actions such as :guilabel:`Force Sync` or :guilabel:`Check files`. * By default, only the :guilabel:`wapt` folder is synchronized, you can select which folder to sync by adding up elements in ``remote_repo_dirs`` parameters. * Synchronization period can be configured with ``local_repo_time_for_sync_start`` and ``local_repo_time_for_sync_stop`` parameters. * Bandwidth allocated to sync can be configured with ``local_repo_limit_bandwidth``. Every parameters of WAPT repository sync must be set in the ``[repo-sync]`` section of the WAPT agent's :file:`wapt-get.ini` configuration file. .. figure:: wapt-resources/replication_behavior.png :align: center :alt: WAPT agent replication behavior WAPT Agent configuration ------------------------ To enable replication on an :ref:`existing WAPT Agent ` (Linux / Windows) you need to set in the ``[repo-sync]`` section in the :file:`wapt-get.ini` configuration file. .. list-table:: WAPT Agent replication configuration :header-rows: 1 :widths: auto :align: center * - Options / Default value - Definition - Example * - ``enable_remote_repo`` = False - Enables remote repository to synchronize with the main repository. - ``enable_remote_repo`` = True * - ``local_repo_path`` = WAPT root dir/repository - Sets the path to the root directory of the local repository for WAPT packages. - ``local_repo_path`` = /var/www/ * - ``local_repo_time_for_sync_start`` = None - Sets synchronization start time (HH:MM / 24h format) - ``local_repo_time_for_sync_start`` = 22:30 * - ``local_repo_time_for_sync_end`` = None - Sets synchronization stop time (HH:MM / 24h format) - ``local_repo_time_for_sync_end`` = 05:30 * - ``local_repo_sync_task_period`` = None - Sets synchronization periodicity (minutes) - ``local_repo_sync_task_period`` = 25 * - ``local_repo_limit_bandwidth`` = None - Sets synchronization allowed bandwidth (Mbits/s) - ``local_repo_limit_bandwidth`` = 2 * - ``remote_repo_dirs`` = wapt,waptwua - Defines folders to synchronize - ``remote_repo_dirs`` = wapt,waptwua,wapt-host * - ``use_repo_rules`` = False - Enable for use :ref:`repository rules ` - ``use_repo_rules`` = True .. warning:: If you modify manually :file:`wapt-get.ini` on the remote repository, you need to restart the **waptservice**. .. note:: A ready-to-use `WAPT package `_ is available in **Tranquil IT public store** to enable repository replication on Windows or Linux based WAPT agents. This special package: * Installs and enables the :program:`Nginx web server` on the remote repository. * Configures :program:`Nginx` virtualhost environment. * Enables remote repository configuration in :file:`wapt-get.ini`. It is possible to automatically configure repositories with your own preferred values by editing this package. Below is an example of :file:`wapt-get.ini`. .. code-block:: ini [global] ... use_repo_rules = True [repo-sync] enable_remote_repo = True local_repo_path = D:\WAPT\ local_repo_time_for_sync_start = 20:30 local_repo_time_for_sync_end = 05:30 local_repo_sync_task_period = 25 local_repo_limit_bandwidth = 4 remote_repo_dirs = wapt,waptwua,wapt-host WAPT Server configuration ------------------------- By default, the WAPT server will know which WAPT agents are configured as remote repositories and it will list them in the WAPT console. .. _repository_rules: Repository rules ---------------- When a WAPT agent has been configured as a repository, it will automatically retrieve its :file:`rules.json` file from the WAPT server. The :file:`rules.json` file is a signed :mimetype:`.json` file that contains a list of sorted rules to apply to the remote WAPT agents, so they may connect to their most appropriate repositories. If no rules can be matched, the WAPT agent will fallback to the ``repo_url`` setting of the WAPT server defined in the :file:`wapt-get.ini` configuration file. .. figure:: wapt-resources/repository_rules.png :align: center :alt: WAPT agent replication behavior WAPT agent ^^^^^^^^^^ .. warning:: **If you have configured GeoIP redirects on Nginx, you should disable it as it might conflict with repository rules**. To enable WAPT Agent repository rules, you must enable this setting in the ``[global]`` section of the :file:`wapt-get.ini` configuration file of the WAPT agent. .. list-table:: :header-rows: 1 :widths: auto :align: center * - Options / Default value - Description - Example * - ``use_repo_rules`` = 0 - For using :ref:`replicating repository `. - ``use_repo_rules`` = True Below is an example of :file:`wapt-get.ini`. .. code-block:: ini [global] ... use_repo_rules = True .. note:: It is possible to enable this option when :ref:`generating a WAPT agent `. WAPT Server ^^^^^^^^^^^ On the WAPT Server, remote repositories functionality are automatically enabled. For control, edit :file:`waptserver.ini` and read ``remote_repo_support`` value. .. list-table:: :header-rows: 1 :widths: auto :align: center * - Options / Default value - Example value - Definition * - ``remote_repo_support`` - ``True`` - Enables repository usage WAPT Console ^^^^^^^^^^^^ Repository rules can be managed from the WAPT console and are based on several parameters: .. list-table:: Available parameters for repository rules :header-rows: 1 :widths: auto * - Options - Example value - Description * - Agent IP - ``192.168.85.0/24`` - Rule based on Agent IP sub-network. * - Domain - ``ad.mydomain.lan`` - Rule based on Active Directory domain name. * - Hostname - ``desktop-04feb1`` - Rule based on the hostname of the WAPT agent. * - Public IP - ``256.89.299.22/32`` - Rule based on the public IP address (NATed hosts). * - Site - ``Paris-HQ`` - Rule based on Active Directory Sites and Services. Adding a rule """"""""""""" In :guilabel:`Repositories`, click on the :guilabel:`Add rule` button. The following window appears. .. figure:: wapt-resources/create_new_rule.png :align: center :alt: Creating a new repository rule Creating a new repository rule .. list-table:: Detail of values :header-rows: 1 :widths: auto :align: center * - Options - Example value - Description * - :guilabel:`Name` - repo25 - Defines the name for the rule. * - :guilabel:`Condition` - AGENT IP - Defines the condition to match for the rule to apply (see above). * - :guilabel:`Value` - 192.168.25.0/24 - Defines the value when the condition applies. If ``NOT`` checked, the value applies to the reverse of the condition. * - :guilabel:`Repository URL` - https://repo25.domain.lan - Defines the list of available remote repositories. The list includes ``http://download.windowsupdate.com/microsoftupdate/v6/wsusscan/`` to allow directly downloading of Windows Updates by the remote repositories to preserve bandwidth. * - :guilabel:`Package type` - WAPT - Defines what :ref:`types of packages ` are replicated. * - :guilabel:`Other` - No fallbacks - See below * The option :guilabel:`No Fallback` will prevent from falling back to the main WAPT server and will avoid undesired network congestion if the remote repository becomes temporarily unavailable. * The option :guilabel:`Proxy` will need to be set if the remote repository is required to connect via a proxy. .. figure:: wapt-resources/create_new_rule_proxy.png :align: center :alt: Using proxy on rule You can then choose from the different above parameters and affect values to a specific secondary WAPT repository. .. warning:: **The rules are applied from top to bottom. The first rule that matches the conditions overrides all the other rules placed under.** .. danger:: Do not forget to save your replication rules. .. figure:: wapt-resources/save_rules.png :align: center :alt: Saving the configuration .. _wapt_multi-repository: Multiple repositories ===================== Similar to Debian repositories, it is possible for the WAPT agent to use multiple repositories for updating package. The WAPT agents will check all repositories. .. danger:: If you use this functionality, **KNOW WHAT YOU ARE DOING**. When using repositories with different signers, the additional signer's public certificates must be added to :file:`C:\\Program Files (x86)\\wapt\\ssl` on Windows or :file:`/opt/wapt/ssl` on Linux and MacOS, therefore, you **MUST** trust their work and their signature. You then must deploy WAPT agent with both keys. Please refer to the documentation on :ref:`creating the WAPT agent ` to add more trusted certificates. .. _wapt_agent_multi-repository: WAPT Agent configuration ------------------------ This parameters are modifiable on :file:`wapt-get.ini` file. Description of available parameters ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ * [global] *repositories* parameter: The parameter *repositories* allows to set several options for package repositories, for example *wapt-templates* and *private*, where their settings are set in additional ``[section]`` of the file. .. code-block:: bash repositories=wapt-templates,private * [section] settings of secondary repositories .. code-block:: ini [wapt-templates] repo_url=https://store.wapt.fr/wapt verify_cert = 1 [private] repo_url=https://srvwapt.mydomain.lan/wapt verify_cert = 0 With that configuration, WAPT clients will now see packages from the main repository and from the secondary repository. Description of available parameters ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ .. list-table:: Description of available options for using multiple repositories :header-rows: 1 :widths: auto :align: center * - Options / Default value - Description - Example * - ``http_proxy`` = None - Defines the HTTP proxy address. - ``http_proxy`` = http://user:pwd@host_fqdn:port * - ``repo_url`` = None - Defines the address of the main WAPT repository. - ``repo_url`` = https://srvwapt.mydomain.lan/wapt * - ``timeout`` = None - Defines the timeout when connecting to remote repositories. Value in miliseconds. - ``timeout`` = 5000 * - ``use_http_proxy_for_repo`` = False - Defines whether a proxy needs to be set to access the repositories. - ``use_http_proxy_for_repo`` = 1 * - ``verify_cert`` = None - Defines whether :ref:`HTTPS certificates of the repository needs to be verified `, and if so defines the path to the certificate bundle. - ``verify_cert`` = :file:`C:\\Program Files (x86)\\wapt\\ssl\\server\\srvwapt.mydomain.lan.crt` (on Windows) .. note:: The WAPT agents will look for updates on all repositories. .. code-block:: bash wapt-get search More info on :ref:`using WAPT with the command line interface `. .. hint:: Accessible packages from the all defined repositories will also be visible using the web interface http://127.0.0.1:8088 on WAPT equipped devices. .. _wapt_console_multi-repository: WAPT Console configuration -------------------------- After having configured the WAPT agent for using multiple repositories, we can make the repositories show up in the WAPT console. To do so, modify the :file:`%appdata%\\local\\waptconsole\\waptconsole.ini` file. Example: .. code-block:: ini [wapt-template] repo_url=https://wapt.tranquil.it/wapt http_proxy= verify_cert=1 public_certs_dir= client_certificate= client_private_key= timeout=5 [private] repo_url=https://srvwapt.mydomain.lan/wapt http_proxy= verify_cert=0 public_certs_dir= client_certificate= client_private_key= timeout=5 .. list-table:: Description of available options for external repositories :header-rows: 1 :widths: auto :align: center * - Options / Default value - Description - Example * - ``client_certificate`` = None - Defines the folder that contains the certificates used to authenticate downloaded external packages. - ``client_certificate`` = :file:`C:\\Program Files (x86)\\wapt\\ssl\\server\\srvwapt.mydomain.lan.crt` (on Windows) * - ``client_private_key`` = None - Defines the folder that contains the private key. - ``client_private_key`` = :file:`C:\\Program Files (x86)\\wapt\\ssl\\server\\srvwapt.mydomain.lan.pem` (on Windows) * - ``http_proxy`` = None - Defines the address of the proxy to use to access the external repository referenced in the ``[section]``. - ``http_proxy`` = http://proxy.mydomain.lan:8080 * - ``public_certs_dir`` = - Defines the folder that contains the certificates used to authenticate downloaded external packages. - ``public_certs_dir`` = :file:`C:\\private` * - ``repo_url`` = None - Defines the address of the external WAPT repository. - ``repo_url`` = http://srvwapt.mydomain.lan/wapt * - ``timeout`` = None - Defines the timeout for the external repository referenced in the ``[section]``. If left empty, the connection will not timeout. - ``timeout`` = 2 * - ``verify_cert`` = None - For :ref:`verifying HTTPS certificates `. - ``verify_cert`` = 1