Installing Cluster using Resources Created on AWS

Cluster Name

Provide a unique cluster name.

License

Upload your Kyvos license file.

Installation Path

Provide the installation path for Kyvos.

User Name

Enter your username.

Authentication Type

• Password: Provide your credentials. Ensure that the permissions listed here are available on all the EC2 instances for Linux user deploying the cluster.

• Private Key: Choose and Upload the Private Key file.

Configure Kyvos Services on Cloud

• Select Instance By:

  • IP Address: Select this option for IP Address-based deployment.

  • Hostname: Select this option for hostname-based deployment. In this case, you can deploy the cluster using custom hostnames for nodes to be used for Kyvos services. 
    Click the Use custom hostname for selected nodes link to specify or map a custom hostname for the selected analytical server, Query Engine, and Kyvos Manager/repository node.
    NOTE: Ensure that the instances have been configured for using Custom hostnames.

• Resource By:

  • Select Stack or Account as needed.

  • For Stack:

    1. Select the stack from the list.

    2. Click Refresh Stack resources to view and modify any resources. The resources configured in the stack, such as EC2 instance(s) for Analytical Server, EC2 instance(s) for Query Engine, EC2 instance(s) for Web Portal, and so on are displayed. Click the Modify link corresponding to the Identified Resources to edit them. You can also select the Repository here.
       NOTE: If you want to use your own resources, select the Account option, and provide the details in the displayed fields. Here, if you have the requisite permissions, your AWS resources will be displayed in the lists. Otherwise, you will have to manually provide the details.

  • For Account:

    1. Select the Deploy BI Server, Query Engine, and Web Portal services on ip-<>.ec2 internal instance checkbox. This checkbox is displayed only when you select the Account (node mapping Manually) option. Selecting this checkbox will deploy Kyvos services on the same instance. 
       NOTE: By default, the option to deploy EC2 instance(s) for BI Server, EC2 instance(s) for Query Engine, and EC2 instance(s) for Web Portal will be disabled.

• Virtual IP/Hostname: This is enabled only when multiple Web Portal based deployment is opted.

• Enable Kyvos Manager for High Availability: Select this checkbox to deploy Kyvos Manager High Availability (HA) enabled cluster. This is enabled only when multiple Web Portal based deployment is opted.

  • Load Balancer Configuration: Select the Configuration for Web Portal and Configuration for Kyvos Manager checkboxes.

    • Configuration for Web Portal: Select the ARN or Target Group Name from the Resource Identified by list. The options will vary based on your selection.

      • Enter the ARN or Target Group Name.

    • Configuration for Kyvos Manager: Select the ARN or Target Group Name from the Resource Identified by list. The options will vary based on your selection.

      • Enter the ARN or Target Group Name.

Kyvos Repository

Select the required external Kyvos Repository.

In case you are using an externally managed repository, the following options are displayed.

• Endpoint: Enter the repository endpoint.

• Port: Enter the repository port.

• User Name: Enter the user name.

• Password Source: Select the required option for Password Source:

  1. Enter a new password and save

  2. Use an existing password from configured Secret

     Note: When selecting either of these options, you can choose to allow Kyvos to change the repository’s password by selecting the checkbox Enable the option to change repository password in Kyvos Manager after deployment.

Repository Password

• Read from Secret: Select this option to read the password from existing Secret Manager.
  If you select this option, the following option to change the repository password will be displayed.

  • Enable the option to change repository passwords in Kyvos Manager after deployment: Select this option to change the Kyvos repository password from Kyvos Manager after cluster deployment.

• Enter Manually: Enter your Kyvos external repository password.

Output S3 Storage > Bucket Name

Select the output S3 bucket name.

Secrets Manager

Enter the name of the Secrets Manager that will be used for storing passwords. This is a mandatory field.

Note: This field is displayed if you selected the Deploy with Enhanced Security option at the time of creating resources.

Compute Cluster

Select the Kubernetes, Shared Query Engines, or Dedicated Compute Cluster option from the Compute Cluster Type list.

NOTE:

• For Kubernetes: When you select the Kubernetes option from the Compute Cluster list, the Kubernetes Details section is displayed.

• For Dedicated Compute: Selecting the Dedicated Compute option from the list will display the Auto Scaling Group, Enable Compute Server Scaling and Min Node Count, and Max Node Count fields. To create a dedicated compute cluster, see the Creating a dedicated compute cluster section.

  • Auto Scaling Group: Select the Auto Scaling Group that you created externally.

  • Enable Compute Server Scaling: Select this checkbox to enable compute server scaling. The Min Node Count and Max Node Count fields are displayed.
    Note: If you do not select this checkbox, only the max node count field is displayed.

    • Min Node Count: Select the minimum node count that will be started initially.

    • Max Node Count: Select the maximum node count up to which you want to scale.

Kubernetes Details

Enter the following Kubernetes details:

• Kubernetes Cluster Name: Select the Kubernetes cluster name from the list.
  If the list is not populated, you need to enter the Kubernetes cluster name manually, and then click Fetch Details. The Kubernetes cluster details will be fetched.

• Shared Resources with Multiple Applications: Select this checkbox to use Kubernetes or User node pool as shared. If this checkbox is selected, the Maximum Compute Server Count field will be disabled.

• Node Pool Name: Select the Node Pool name from the list. If the node pool name list is not populated, you need to enter the Node Pool Name manually, and then click Validate Kubernetes Cluster Details to validate Kubernetes cluster details.

• K8S Addon Auth Role Name: The K8S Addon Auth Role Name is displayed by default.

• Enable Compute Server Scaling: Select this checkbox to enable maximum or minimum compute server for scaling. When you select this checkbox, the following options will be displayed.

  • Minimum Compute Server Count: Select the minimum compute Server count that will be started initially.

  • Maximum Compute Server Count: Select the maximum compute Server count up to which you want to scale.

  Note: If you do not select this checkbox, the Compute Server Count field is displayed. Enter the compute server count that you want to set for minimum or maximum compute server scaling.

• Dedicated Compute Namespace: Enter the namespace for compute pod.

  Note: If parameter ‘KubernetesClusterType’ is Shared then the provided namespace must exist. If parameter ‘KubernetesClusterType’ is Dedicated, Kyvos will create the specified namespace.

Lambda Function

Select the required function from the list:

• Using Lambda Function Name: The Cluster Scaling Schedule, Cluster Status, and Analytical server Start fields are displayed.

  • Cluster Scaling Schedule

  • Cluster Status

  • BI Server Start

• Lambda Function Stack: The Function Deployment field is displayed.

  • Function Deployment: Provide the name of the deployment where your Google Deployment Functions are deployed.

• None: Selecting this option will display an error message. You must specify the Function Deployment name as this is mandatory for Kyvos scaling feature.

Common

Kyvos Setup Package

Select the Kyvos bundle to be installed. Kyvos Manager contains compatible Kyvos bundles. To use any other Kyvos version, click the Upload button

Semantic Model Local Path

Enter the semantic model local path where cuboids will be copied. For multiple paths, enter a semicolon-separated list.

Note: The deployment user must have full permission on the parent of the local semantic model path.

Automatically restart Kyvos services when down

Select this option to enable high availability mode for the Kyvos service. This will ensure Kyvos services are always up and running. The Kyvos Manager will automatically restart the , query engine, repository, and web portal services, in case any of these services go down

Listener Port

Enter the listener port for analytical server.

JMX Port

Enter the JMX port for analytical server health monitoring.

Work Directory

Enter the directory location on HDFS, where analytical server has access permissions.

Memory Configuration Mode

By default, the Auto mode is selected. In Auto Mode, calculations are performed automatically by the system internally.
In Manual Mode, you can manually provide memory values as needed.

Note: The Maximum Heap Memory and Analytical Store Max Memory options are enabled only when you set the memory configuration in the Manual mode.

Maximum Heap Memory

Enter the maximum memory available for analytical server.

Analytical Store Max Memory

Specify analytical store max memory for querying in GB.

Additional Java Options

Use this to define custom Java properties for your analytical server.

Query Engine

Thrift Listener Port

Enter the thrift listener port for Query Engine.

JMX Port

Enter the port for Query Engine health monitoring.

Memory Configuration Mode

By default, the Auto mode is selected. In Auto Mode, calculations are performed automatically by the system internally. In the Manual Mode, you can manually provide memory values as needed.

Note:

• The Maximum Heap Memory, Analytical Store Max Memory for Querying, Analytical Store Max Memory for Processing, and Configure Concurrent Processing Task options are enabled only when you set the memory configuration in the Manual mode.

• If you set the memory configuration in the Manual mode, the Configure Concurrent Processing Task option can be enabled as Auto or Manual as needed.

Maximum Heap Memory

Enter the maximum memory for Query Engine.

Analytical Store Max Memory for Querying

Specify analytical store max memory for querying in GB.

Analytical Store Max Memory for Processing

Specify analytical store max memory for processing in GB.

Configure Concurrent Processing Task Count

By default, the Auto mode is selected. If you select the Manual mode, specify the number of nodes to be used for concurrent processing task count.

Additional Java Options

Use this to define custom Java properties for your Query Engine.

Compute Server

Listener Port

Enter the listener port for compute server configuration.

Maximum Heap Memory

Enter the maximum memory available for compute server configuration.

Additional Java options

Use this to define custom Java properties for your compute server configuration.

Analytical Store Maximum Memory

Enter the maximum Analytical Store memory available for compute server configuration.

Web Portal

UI Port

Enter the port on which Kyvos Web client will run.

JMX Port 

Enter the port for Web portal health monitoring.

Shut-down Port 

Enter the port to be used for shutting the Tomcat server.

AJP Port 

Enter the port on which the AJP connector creates a socket and awaits an incoming connection.

Maximum Heap Memory

Enter the maximum memory for the Web portal.

Additional Java Options

Use this to define custom Java properties for the Web portal.

Network Protocol Configuration 

Enable TLS 

Select the checkbox to enable TLS for the cluster.

Note: Before enabling TLS, if you are using your own script for resource creation instead of the Kyvos shared scripts, you must rename the OpenSSL configuration file on the Kyvos Manager and the node where the Postgres Service is running. The OpenSSL configuration file is available in the /etc/pki/tls/openssl.cnf folder.

To accomplish this, you will need to have sudo access.

HTTP Protocol Version 

Select the HTTP Protocol Version for use with TLS.

Kyvos Web Portal on HTTPS 

Select the checkbox to use HTTPS for connecting to Kyvos Web Portal.

Kyvos Web Portal Configuration

NOTE: This is displayed only if Kyvos Web Portal on HTTPS checkbox is selected

• Certification Mode: Select the Use Path or Upload File option.

• Keystore: Provide the path or upload the Keystore file for the certificate.

• Keystore Private Key: Provide the Keystore Private Key for the certificate.

• Custom Attributes for Connectors: If needed, provide attributes to be used for the HTTPS connector.

Enable GPU

Upload Drutdb Jar

For a GPU enabled cluster, upload the drutdb.jar file to enable GPU querying.