Configuring Query Engine Segmentation
This documentation is for Kyvos 2026.8.x. To view documentation for other Kyvos versions, click here. |
|---|
Overview
This section explains how to configure segmentation in Kyvos Manager to create and manage multiple segments and allocate cluster resources (such as Query Engines or disk space) across these segments.
You can see the last performed Segmentation Configuration operation details on the Operations page
You can create multiple segments and divide the system resources (such as Query Engines or disk space) among these segments.
For Query Engine-based segments (Certified from Kyvos 2026.8.3), Query Engines can be grouped into dedicated segments. The available Semantic Models can then be mapped to these segments.
Note
This feature is applicable only to Azure Enterprise deployments with multiple segments.
For a resource-based segment (Not Certified), each node can have multiple instances of Query Engines. Also, each instance can have multiple disks attached to it. In this case, all the Query Engines participate in querying. You can only configure resources like disks.
All the available Query Engines must be linked to a specific segment under the cluster name > Kyvos and Ecosystem> Query Engine Segments.
Query Engine Segmentation for Azure Enterprise
This feature allows you to group Query Engines into dedicated segments and map semantic models to those segments, helping isolate workloads and improve query performance consistency.
The sections below describe how to create and delete Query Engine segments, as well as how to map Semantic Models to these segments.
Before you begin
Segment names must be unique across the deployment.
The VMSS instances associated with segments must be deployed in the same resource group as the Kyvos deployment.
If you have existing Query Engine segments, delete them before creating a new Query Engine segment from Kyvos Manager. This applies only to upgrades to Kyvos version 2026.8.3.
If you use an existing VMSS, perform the following steps to make it suitable for the Kyvos environment:
Add the ROLE tag with the value QE_Cluster.
Download the PrereqScriptAllClouds.sh and follow the steps mentioned in the Common Prerequisites for Kyvos Enterprise Deployment section.
Run the following Azure CLI command to encode the script:
PrereqScriptAllClouds.sh >> Encoded_BootstrapRun the following Azure CLI command to add the Lapextension script:
az vmss extension set --resource-group <KYVOS-RG> --vmss-name <KyvosDedicatedCompute> --name CustomScript --publisher Microsoft.Azure.Extensions --version 2.1 --settings '{}' --protected-settings "{\"commandToExecute\": \"echo <ENCODED_SCRIPT_TEXT> | base64 -d > /tmp/init.sh && chmod +x /tmp/init.sh && /tmp/init.sh\"}"In the above command, replace the following placeholders:
<KYVOS-RG>with the name of your resource group.<KyvosDedicatedCompute>with the name of your VMSS.<ENCODED_SCRIPT_TEXT>with the encoded text generated in step c.
Go to the VMSS and verify that the information specified in step d is updated in the Lapextension script.
To create new VMSS, see the Creating External Resources for Kyvos on Azure section.
A Segment (except Default Segment) must have at least one Query Engine.
A Default Segment exists as a system-level construct and cannot be deleted.
All Semantic Models not explicitly mapped to any named Segment are routed to the Default Segment.
Unprocessed Semantic Models can also be mapped to Segments.
After remapping a segment or semantic model to another segment, the first query may fail due to metadata replication issues. To avoid this failure, wait for the Range Balancer thread to complete.
Create a Query Engine Segment
To create and configure a Query Engine segment, perform the following steps.
Note
Only Kyvos admins can configure segments.
From the hamburger menu, navigate to Kyvos and Ecosystem > Query Engine Segments.
On the Query Engine Segments page, click + Add Segment.
In Segment Name, enter a name for the segment.
In Segment Type, select Query Engine Based.
In Scale Set, select the required segment.
To configure memory settings, do one of the following:
To use the global memory configuration for Query Engines, disable the Configure Memory toggle.
To define custom memory settings for the segment, enable the toggle and configure settings such as Max Heap Memory and Analytical Store Max Memory for the segment. You can also specify Kyvos Java options in the Additional Java Options field. If no options are specified, the default Kyvos Java options are applied. All Query Engines mapped to the segment use the configured memory settings.
Note: Segment-level memory settings override the global memory configuration for that segment.
In Memory Configuration Mode, select one of the following:
Auto: Allows Kyvos to configure the memory settings automatically.
Manual: Allows you to specify the memory settings manually. Enter values for the following settings:
Max Heap Memory
Analytical Store Max Memory
Review the memory information displayed on the page and ensure that the configured memory is within the limits defined by the license and the available node memory.
In Additional Java Options, enter Kyvos Java Options if needed. If not configured, the default Kyvos Java Options will be applied. All Query Engines mapped to the segment will use the configured memory settings.
To create additional segments, click + Add Segment and repeat the preceding steps.
Click Save. This will save the segment configuration.
Enter your Kyvos Manager password.
Restart Kyvos application services for the segment to become active.
Now you can map semantic models to Query Engine segments from Kyvos.
Note
All Query Engines and resources that are not mapped to a segment are part of the Default Segment.
After creating a segment, add the required Query Engine nodes by clicking Add Node on the Node and Service page in Kyvos Manager.
Delete a Query Engine Segment
On Segment deletion, mapped semantic models are remapped to the Default Segment.
To delete a segment, perform the following steps.
Hover over the segment name.
Click the three-dot icon on the right side of the segment.
Select Delete.
Click Save and restart all Kyvos application services.
Note
Deleting a segment moves its mapped semantic models to the Default Segment. Nodes from its Scale Set that are part of the Kyvos Query Engine cluster are also moved to the Default Segment.
Resource-Based Segmentation (not certified)
To create resource-based segments, from the Segment Type, select the Resource Based Segments option.
Local Disk Path: You can select any of the following options.
Custom: Select this option to define custom resource allocation on each query engine.
Unique: Select this option to configure unique resource allocation on each query engine
All: Select this option to define common resource allocation for all the query engines.
Used For: Select the purpose for which the resources will be used from:
Cuboid Replication
Extended Cuboid Block Cache
Metadata
Select the percentage of space allowed for the purpose.
Click Apply.
Important Notes
By default, all the query engines are assigned to the Default Segment until Analytical Server is restarted.
You can also create a Resource based segment by defining the resource allocation for all the Query Engines in that segment. Through this, you can allocate resources to each query engine in the segment for Cuboid Replication, Extended Cuboid Block Cache, and Metadata.
For Resource based segments, there must be at least one Query Engine assigned to each segment, and the same query engine CAN be assigned to multiple segments.
Deleting a segment will move all the Query Engines under it to the Default segment.
Migrating the role of any Query Engine will remove it from the segment(s) under which it is allocated.