Configuring an LLM connection
This documentation is for Kyvos 2025.11.x. To view documentation for other Kyvos versions, click here. |
|---|
LLM Connection is the configuration that enables a system to communicate with a Large Language Model (LLM), such as those developed by OpenAI, Azure OpenAI, AWS Bedrock, and so on. An LLM connection is used to:
Translate natural language into analytical queries.
Power chat-based or voice-based interfaces.
Enhance user experience through intelligent, language-based interaction with data.
An LLM connection allows:
Sending natural language inputs (e.g., user queries) to the LLM.
Receiving generated outputs like MDX/SQL queries, summaries, or explanations.
Enabling features such as conversational analytics, natural language querying, or smart recommendations.
From Kyvos 2025.10 onwards, you can add AWS Bedrock as a supported provider for LLM and embedding connections. You can create and manage Bedrock-based connections, as with other supported providers such as Azure OpenAI and OpenAI.
To configure the GenAI LLM connection, perform the following steps:
From the Toolbox, click Connections.
From the Actions menu ( ⋮ ) click Add Connection.
Select the name of the GenAI provider from the Provide list. The system will use the provider to generate output.
To create an LLM connection, enter details as:
After you finish configuring the settings using the table shown below the screenshot, click Save.
Parameter/Field | Description |
|---|---|
Name | A unique name that identifies your GenAI connections. |
Category | Select the required category from the list. |
Provider | Select the OpenAI provider from the list. The system will this provider to generate output. |
URL | The LLM Service URL of the provider-specific endpoint for generating output. |
API EndPoint | Specify which endpoint to be used to generate AI-powered conversational responses. |
Authentication Type | Select Authentication Type as Authentication Key or O Auth 2.0.
|
Model | The name of the GenAI LLM model used to generate the output. |
Is Model Fine Tuned | Select one of the following: |
Usage | Select one of the following for which this connection can generate output:
Note: When you select the required usage, you can make the selected usage as a default connection. The checkbox will be displayed to set the selected usage as a default connection. |
Allow Sending Data for LLM | Select Yes or No to specify whether the generated questions should include values or not. |
Max Data Points for Summary | Enter the value to configure maximum number of data points involved in generating the summary. NOTE: The default value is 5000. |
Input Prompt Token Limit | Specify the maximum tokens allowed for a prompt in a single request for the current provider. NOTE: The default value is 16000. The minimum value is 0. |
Output Prompt Token Limit | Specify the maximum number of tokens shared between the prompt and output, which varies by model. One token is approximately four characters for English text. NOTE: The default value is 2048. The minimum value is 0. |
Max Retry Count | Specify maximum retry count attempted so that we get correct query. NOTE: The default value is 2. |
Summary Records Threshold | Specify similarity threshold for query autocorrection. NOTE: The default value is 0.1 The minimum value is 2. |
LLM Temperature | Specify the LLM temperature, which controls the level of randomness in the output. Lowering the temperature results in less random completions. The responses of the model become increasingly deterministic and repetitive as it approaches zero. It is recommended to adjust either the temperature or top-p, but not both simultaneously. |
Is Reasoning Model | Specify whether model used is reasoning model. |
Properties | Click Properties to view or set properties. |
Parameter/Field | Description |
|---|---|
Connection Name | A unique name that identifies your GenAI connections. |
Category | Select the required category from the list. |
Provider | Select the Azure OpenAI provider from the list. The system will this provider to generate output. |
URL | The LLM Service URL of the provider-specific endpoint for generating output. |
API EndPoint | Specify which endpoint to be used to generate AI-powered conversational responses. |
Authentication Type | You can select Authentication key, Client ID, Bearer Token, and OAuth 2.0. Based on your selection, the following will be displayed.
|
Model | The name of the GenAI LLM model used to generate the output. |
Is Model Fine Tuned | Select one of the following: |
Embedding Connection | Specify the name of the GenAI embedding provider that the system will use to generate embeddings. |
Usage | Select one of the following for which this connection can generate output:
Note: When you select the required usage, you can make the selected usage as a default connection. The checkbox will be displayed to set the selected usage as a default connection. |
Allow Sending Data for LLM | Select Yes or No to specify whether the generated questions should include values or not. |
Max Data Points for Summary | Enter the value to configure maximum number of data points involved in generating the summary. NOTE: The default value is 5000. |
Input Prompt Token Limit | Specify the maximum tokens allowed for a prompt in a single request for the current provider. NOTE: The default value is 16000. The minimum value is 0. |
Output Prompt Token Limit | Specify the maximum number of tokens shared between the prompt and output, which varies by model. One token is approximately four characters for English text. NOTE: The default value is 2048. The minimum value is 0. |
Max Retry Count | Specify maximum retry count attempted so that we get correct query. NOTE: The default value is 2. |
Summary Records Threshold | Specify similarity threshold for query autocorrection. NOTE: The default value is 0.1 The minimum value is 2. |
LLM Temperature | Specify the LLM temperature, which controls the level of randomness in the output. Lowering the temperature results in less random completions. The responses of the model become increasingly deterministic and repetitive as it approaches zero. It is recommended to adjust either the temperature or top-p, but not both simultaneously. |
Top P | This property manages through diversity through nucleus sampling. Setting it to 0.5indicates that half of all likelihood-weighted option will be considered. It is recommended to adjust either this parameter or the temperature parameter, but not both simultaneously. |
Frequency Penalty | This property specifies a number between -2.0 and 2.0. Positive values penalize new tokens based on their frequency in the existing text. This reduces the likelihood of the model repeating the same line verbatim. |
Presence Penalty | This property specifies a number between -2.0 and 2.0. Positive values penalize new tokens based on their appearance in the text so far, thereby increasing the model's likelihood of discussing new topics. |
Is Reasoning Model | Specify whether model used is reasoning model. |
Properties | Click Properties to view or set properties. |
Parameter/Field | Description |
|---|---|
Connection Name | A unique name that identifies your GenAI connections. |
Category | Select the LLM from the list. |
Provider | Select the AWS Bedrock provider from the list. The system will this provider to generate output. |
Access ID | Enter the access ID for accessing AWS services. |
Secret ID | Enter the secret ID for accessing AWS services. |
AWS Region | Enter the AWS region where the model is present. |
Model | The name of the GenAI LLM model used to generate the output. |
Is Model Fine Tuned | Select one of the following: |
Usage | Select one of the following for which this connection can generate output:
Note: When you select the required usage, you can make the selected usage as a default connection. The checkbox will be displayed to set the selected usage as a default connection. |
Allow Sending Data for LLM | Select Yes or No to specify whether the generated questions should include values or not. |
Max Data Points for Summary | Enter the value to configure maximum number of data points involved in generating the summary. NOTE: The default value is 5000. |
Input Prompt Token Limit | Specify the maximum tokens allowed for a prompt in a single request for the current provider. NOTE: The default value is 16000. The minimum value is 0. |
Output Prompt Token Limit | Specify the maximum number of tokens shared between the prompt and output, which varies by model. One token is approximately four characters for English text. NOTE: The default value is 2048. The minimum value is 0. |
Max Retry Count | Specify maximum retry count attempted so that we get correct query. NOTE: The default value is 2. |
Summary Records Threshold | Specify similarity threshold for query autocorrection. NOTE: The default value is 0.1 The minimum value is 2. |
LLM Temperature | Specify the LLM temperature, which controls the level of randomness in the output. Lowering the temperature results in less random completions. The responses of the model become increasingly deterministic and repetitive as it approaches zero. It is recommended to adjust either the temperature or top-p, but not both simultaneously. |
Enter Unique name identifier for the connection.
Select the required LLM Category.
Select one of the following Authentication Type:
Service Account JSON: Uses a service account JSON file.
Note: Service Account JSON is only supported for on prem clusters whereas ADC and Service account JSON. Both are supported on GCP Environment.ADC: Uses Application Default Credentials available in the environment.
Enter details as:
Parameter | Description |
|---|---|
Service Account File Name | This property specifies the exact file name of the Service Account key file, including the .json extension (for example, my-project-sa-key.json), placed in the olapengine/bin directory. |
Location | Specifies the Google Cloud region (e.g., us-central1) where Vertex AI resources are hosted and processed. |
Project ID | Specifies the unique identifier for your Google Cloud Project where Vertex AI is enabled. |
Candidate count | Specifies the number of independent, alternative responses (or "candidates") the model should generate for a single prompt. The default candidate count is 1. |
Model | Select the model name. For example, Gemini-2.5-pro. |
Thinking budget | Specifies the maximum number of tokens the model can use for its internal reasoning process to improve answer quality and complexity. If the selected model is 'gemini-2.5-pro', the thinking budget must be set to a minimum of 128. |
Is model fine tuned | Select one of the following: |
Usage | Select one of the following for which this connection can generate output:
Note: When you select the required usage, you can make the selected usage as a default connection. The checkbox will be displayed to set the selected usage as a default connection. |
Default Connection for Natural Language Querying | Select this checkbox to set the default connection for natural language queries. |
Allow sending data to LLM | Select Yes or No to specify whether the generated questions should include values or not. |
Max data points for summary | Enter the value to configure maximum number of data points involved in generating the summary. NOTE: The default value is 5000. |
Input prompt token limit | Specify the maximum tokens allowed for a prompt in a single request for the current provider. NOTE: The default value is 16000. The minimum value is 0. |
Output prompt token limit | Specify the maximum number of tokens shared between the prompt and output, which varies by model. One token is approximately four characters for English text. NOTE: The default value is 2048. The minimum value is 0. |
Max retry count | Specifies maximum retry count attempted so that we get correct query. NOTE: The default value is 2. |
Summary records threshold | Specifies the similarity threshold for autocorrecting queries. NOTE: The default value is 0.1 |
LLM Temperature | Specify the LLM temperature, which controls the level of randomness in the output. Lowering the temperature results in less random completions. The responses of the model become increasingly deterministic and repetitive as it approaches zero. It is recommended to adjust either the temperature or top-p, but not both simultaneously. |
Top P | Manages diversity through nucleus sampling. Setting it to 0.95 indicates that half of all likelihood-weighted options will be considered. It is recommended to adjust either this parameter or the temperature parameter, but not both simultaneously. |
Top K | Specifies an integer parameter that limits the model's word choices at each step to a fixed number of the most probable next tokens, discarding all others. |
Frequency penalty | Specifies a number between -2.0 and 2.0. Positive values penalize new tokens based on their frequency in the existing text. This reduces the likelihood of the model repeating the same line verbatim. |
Presence penalty | Specifies a number between -2.0 and 2.0. Positive values penalize new tokens based on their appearance in the text so far, thereby increasing the model's likelihood of discussing new topics. |
Authentication Types
1. JSON Authentication
Use when you have a service account key file.
Specify the exact file name of the Service Account key file, including the .json extension (for example, my-project-sa-key.json), placed in the olapengine/bin directory.
Specify Location and Project Id explicitly.
Ensure the service account has the following roles:
Vertex AI User
Service Account Token Creator
Storage Object Viewer (for model storage access)
For more details, see Google documentation.
Creating a Service Account Key (For service-to-service or other environments)
ADC can also use a Service Account Key JSON file if the path to the file is set in the GOOGLE_APPLICATION_CREDENTIALS environment variable.
How to create and download the JSON key file:
Go to the Service Accounts page in the Google Cloud Console.
Select your project and click on the desired Service Account email.
Go to the Keys tab.
Click ADD KEY > Create new key.
Select JSON as the Key type and click CREATE.
Note
The key will immediately download to your computer. Store this file securely, as it is a private key.
2. ADC (Application Default Credentials)
Use when credentials are configured at environment or instance level.
Recommended for production setups using workload identity, GCE/Dataproc instance credentials, or gcloud auth application-default login.
Find Your Project ID
Go to the Google Cloud Home Page.
On the top navigation bar, click the Project dropdown (it usually shows your current project name).
You’ll see a list of available projects.
Check Location or Region
Open the Vertex AI Dashboard. To access dashboard, see https://console.cloud.google/vertex-ai/dashboard.
The region selector appears at the top of the page. For example, us-central1.
This value is used in the Location field of your connection.
If you are using a Service Account JSON file for authentication:
Specify the exact file name of the Service Account key file, including the .json extension (for example, my-project-sa-key.json), placed in the olapengine/bin directory.
You must manually place the JSON file in the OE/bin directory on all the BI nodes.
Restart OLAP engine.
Comments
Generate content which includes Key Inisghts Summary and Title is missing
In AWS bedrock Reasononing Models parameter is missing
Also we need to add parameter of reasoning