Configuring Execution Servers to Support Python Virtual Environments
This article describes how to configure the Execution Server to support Python virtual environments, manage driver dependencies, and implement manual processes (when required) to run Python drivers in CloudShell. It also describes how to remove unneeded virtual environments to free up system space.
In this article:
Each Python driver has different Python models and dependencies. Therefore, CloudShell installs the driver dependencies specified in the requirements.txt file on separate virtual environments in each TestShell Execution Server. This operation creates a unique environment for each driver, which enables running different drivers with different dependencies simultaneously.
Whenever you upload a new version of a Python driver, a new virtual environment is required in the Execution Server. The new virtual environment will be created only when the new version of the driver is used in the Execution Server for the first time. The dependencies of the driver will also be installed. The installation path of the virtual environment is: %PROGRAMDATA%\QualiSystems\venv.
Note: Using the driver for the first time on the Execution Server may take some setup time to create the new virtual environment and install the dependencies. The next time the same Python driver is used, any new dependencies uploaded to the Python Package Index (PyPI) repository for this driver will be installed. Any existing dependencies will be reused without any changes (and will not require additional setup time).
The following configuration modes are available for Python driver dependencies:
Automatic online mode
This is the default mode. CloudShell automatically creates the new virtual environment and installs the driver dependencies online from the PyPI Python repository on the Internet. The Execution Server must have access to the Internet in order for this to work.
Manual offline mode
This mode is used when access to the Internet is unavailable for the Execution Server(s). In offline mode, the admin needs to download the out-of-the-box driver dependencies, as well as dependencies for Python drivers you load into CloudShell, and place them in a local repository.
Note: If you are using a shared folder to store the driver dependencies offline, make sure your execution server services have permissions to access this folder.
To place driver dependencies in a local repository:
Create the folder that will contain all driver dependencies for CloudShell and the execution servers.
It is recommended to use a single shared repository for all the site's execution servers but you can have a local repository on each execution server.
Use the configuration key
<PythonOfflineRepositoryPath>to specify the path to the required repository in the the CloudShell\Server\customer.config file. To specify a different path for a specific execution server, add the key with the new path to the customer.config file of the required execution server's TestShell\Execution Server\customer.config file. See Setting the Python dependencies repository.
- For out-of-the-box Python-based functionality such as vCenter and AWS EC2 App deployments, do the following:
- For each Python driver loaded into CloudShell:
Connect to the Internet and download each dependency specified in the requirements.txt file with the following command:
pip install --download=/path/to/packages/downloaded -r requirements.txt
- Place the file in the path defined in the above configuration key.
- Restart the relevant services (Quali Server service on the CloudShell Server machine, and TestShell Execution Server service on each updated Execution Server machine).
As new drivers and new versions of drivers are being uploaded, new virtual environments are created in the installation path on the Execution Servers: %PROGRAMDATA%\QualiSystems\venv.
This causes the folder size to keep growing. You can remove unneeded virtual environments to free up system space.
To remove unneeded virtual environments:
Restart the Execution Server.
CloudShell will automatically clean up the unneeded virtual environments that were asssociated with previous driver versions.