Creating a dataset

This documentation is for Kyvos 2026.8.x. To view documentation for other Kyvos versions, click here.

This documentation is for Kyvos 2026.8.x. To view documentation for other Kyvos versions, click here.

✅ Enterprise: AWS, Azure, GCP, and On-Premises ✅ Marketplace: AWS, Azure, and GCP ✅ SaaS: AWS, Azure, and GCP ✅ Single Node Installation (SNI)

Overview

This section details how to create a dataset in Kyvos by selecting a data connection, defining the input type (such as HDFS, table, or SQL), and preparing the data for use in semantic models and analysis.

You can create a dataset for use with Kyvos. When you create a dataset, choose the connection from which the data needs to be fetched. As you apply settings to sort and filter the columns, you can preview the results to make sure you get the results you are seeking.

The total number of columns is shown above the list of their names. If the column count has changed, for example, if any columns are hidden, the column total includes additional information and a link. For example, "Total 29 columns out of which 5 columns are hidden." When you click the link, the information dialog informs you that column details have been updated and lists the number of columns added and deleted, and recommends that you review the updated file. Click Dismiss to close the dialog.

On this page:

1 Overview
2 CSV Support for Kyvos Compute
3 Creating a dataset
4 Excel Support for creating dataset
5 Previewing dataset

Points to know:

From Kyvos 2026.3 onwards, a search option is now available when selecting a connection during dataset creation. The search box is displayed only when the number of connections exceeds seven.
From Kyvos 2026.2 onwards, Kyvos Lakehouse now allows direct reading of Parquet and Apache Iceberg data stored in S3 bucket. The system supports storage in Amazon S3, which eliminates the necessity for external catalogs or SQL engines.
Kyvos supports global parameters for datasets, making it easier to manage parameter values (like dbname and tablename) during environment migrations (e.g., from UAT to Production). You can set these parameters in the connection settings, and Kyvos will automatically use the correct values based on the target environment. This reduces manual work, improves deployment flexibility, and simplifies environment management. When users define parameters as connection properties, the parameter names must be prefixed with kyvos. param..
Example:
Suppose the query is:
SELECT * FROM <dbname>.EMPLOYEE
- On the dataset, you can define a parameter named dbname and assign the required value.
- On the connection, users can add a property named kyvos.param.dbname and set its corresponding value.
If Kyvos resolves the parameter value as devdb, the final query becomes:
SELECT * FROM devdb.EMPLOYEE
You can preview the entire dataset, replacing the previous limitation of a partial dataset preview.
NOTE: Opting to preview the full dataset may result in higher-than-expected costs. Additionally, the execution time required to generate the dataset preview could increase, depending on the size of the data.
You can view a column marked as "Modified" when changes are made to it in a dataset. Additionally, you can view the details of those changes. This feature helps users quickly identify modified columns and review their specifics.
You can mark a file as a Fact to use it as a fact table in relationships. And you can hide columns not required for analysis.
Use the Actions menu (...) to validate the dataset, share it, add a note, or show related entities.
If your instance of Kyvos has been configured via the portal.properties file to support it, you can register a file with a Presto connection. You can format columns and preview the result.
To register a file with a Presto connection, see Creating a file with a Presto connection. You can also create register files by Writing SQL queries using data in Hive.
To learn more about the effects of some of the settings you can use while registering the file, see Logic for creating relationships and semantic model.

CSV Support for Kyvos Compute

Starting from Kyvos 2026.7, the CSV properties available while creating a CSV dataset depend on the configured Global Compute Type (Kyvos Compute or External Compute).

Based on the selected compute type, Kyvos displays only the CSV properties that are applicable to that environment.

Option 1: External Compute

When Global Compute Type is set to External Compute:

Both Spark-supported and Datastore-supported CSV properties are displayed.
Each property indicates its supported execution environment:
- Spark-supported
- Datastore-supported
- Supported by both Spark and Datastore

The following properties are available when using External Compute:

Property	Supported Values	Default Value
Encoding	ASCII (External), ISO-8859-1 (External), UTF-8 (Both), UTF-16 (Kyvos), latin-1 (Kyvos)	ascii
Compression	None (Both), LZO (External), auto (Kyvos), gzip (Kyvos), zstd (Kyvos)	None
Line Separator	\r\n (Both), \n (Both)	\r\n
Escape	Any character (Both)	\
Enclosed By	Any character (Both)	"
Field Separator	Any character (Both)	,
Skip Top Lines	Numeric value (Both)	0
Contains header	True/false (Both)	true
Ignore Empty Rows	true/false (External only)	true
Key as Column	true/false (External only)	false
File Name as Column	True/false (Both)	false

Option 2: Kyvos Compute

When Global Compute Type is set to Kyvos Compute:

Only Datastore-supported CSV properties are displayed.

The following properties are available:

Property	Supported Values	Default Value
Encoding	utf-8, utf-16, latin-1	utf-8
Compression	auto, none, gzip, zstd	auto
Line Separator	\n, \r\n	\n
Escape	Double Quote (")	Double Quote (")
Enclosed By	Double Quote (")	Double Quote (")
Field Separator	Any character	,
Contains header	true/false	true
Skip Top Lines	Numeric value	0
File Name as Column	true/flase	false

Backward Compatibility

If Global Compute Type is set to External Compute, all existing Spark-supported CSV properties and execution behavior will continue to work unchanged.
If Global Compute Type is set to Kyvos Compute, then existing datasets created with Spark-supported properties should be updated if model build gives any error related to properties.

Collaborative editing support

From Kyvos 2026.4 onwards, Kyvos supports collaborative editing by allowing multiple users to work on the same entity (dataset, dataset relationship, semantic model, etc.) simultaneously without enforcing edit locks. It provides visibility into active users, detects concurrent modifications, and offers conflict resolution and merge capabilities to help users manage and reconcile changes efficiently.

Creating a dataset

To create a file, perform the following steps.

From the Toolbox, click Datasets.
Click the Actions menu ( ⋮ ) and click Add Dataset.
Select the Connection from the list.
Select any one of the following input types:
- HDFS
- Table
- SQL

Compatibility with Existing Dialog Upload Datasets

The SQL for Excel datasets created through the new Dataset workflow will remain consistent with the SQL generated for Dialog Upload datasets to ensure compatibility and consistent behavior. The following are some notable points for this-

Users will continue to edit existing Dialog Upload Excel datasets through the SQL editor. Users can only modify the underlying SQL through the SQL editor.
Excel datasets created through the Dataset creation workflow, and those created through Dialog Upload should generate equivalent SQL for the same file properties and configuration.

Excel Support for creating dataset

From Kyvos 2026.7 onwards, there’s support for reading Excel (.xlsx) files through the Datastore via Kyvos Lakehouse framework. Users can now create datasets directly from Excel files stored in Azure-ADLS. The system can infer schema from the selected worksheet and allow users to configure Excel-specific file properties during dataset creation. Additionally, users can create Datasets directly using SQL.

To create a dataset using an existing Excel file, perform the following steps.

Go to Menu > Datasets. Select the required folder.

Click three dots Actions menu and select Add Dataset. The Add Dataset dialog box is displayed.
In the dialog box, a list of connection appears. Search for the required connection.
Select the required Input Type. If selecting the Input Type as SQL, you can use an example SQL like:

SELECT * 
FROM kyvos_read_xlsx ( 
    file='workbook Path', 
    sheet='Sheet1', 
    header=true, 
    ignore_errors=true, 
    stop_at_empty=false, 
    all_varchar=true 
);

Note: Details of supported properties in SQL are provided at the end of this section.

Select the File Type as Excel.

Select the required file. To select the file, click the Actions menu (...) to navigate to the file.
After specifying the file, select the required sheet from the Excel workbook from the Sheets menu.
Click File Properties to configure how data is read from the selected workbook sheet during dataset creation. The following options are available:
1. Range: Specifies the range of data to import. The values entered in Start Cell and End Cell determine the portion of the worksheet that will be used for dataset creation.
2. Contains Header Row: Uses the first row of the imported data as column names.
3. Read All Columns as Text: Reads and stores all column values as text, regardless of their original data types (for example, numeric or date values).
4. Skip Rows with Errors: Skips rows containing invalid or erroneous values during data import.
5. Stop at First Empty Row: Stops reading data when the first empty row is encountered. When a range is specified, this option controls whether processing stops at empty rows within the selected range. If an explicit range is provided, the default is false; otherwise, true.
Click Apply.
Click Refresh to view the columns data in the right pane.
Click Save. The Save as dialog is displayed.
In the dialog, select the folder where the dataset will be saved and provide the dataset’s name.

Click Save to save the dataset. A confirmation message is displayed once the dataset is created.

Create Dataset Using SQL

To create a dataset from an Excel file using SQL, perform the following steps.

Go to Menu > Datasets > Folder.

Open menu (three vertical dots icon) and select Add Dataset.
In the dialog box that opens, search for the required connection and select the Connection from the list.
Select the Input Type as SQL and enter the SQL query in the input box. Below is an example for reference.

SELECT * 
FROM kyvos_read_xlsx ( 
    file='workbook Path', 
    sheet='Sheet1', 
    header=true, 
    ignore_errors=true, 
    stop_at_empty=false, 
    all_varchar=true 
);

Note

The “workbook Path“ in the above example is the complete/absolute path of the Excel file.

Below are the details of supported fields as shown in the SQL example above.

Option	Type	Default	Description
file	VARCHAR	-	Complete/absolute path of the Excel file. Note: This value is mandatory.
header	BOOLEAN	Selected (True)	Whether to treat the first row as containing the names of the resulting columns.
sheet	VARCHAR	First sheet selected	The name of the worksheet to read from the Excel file.
all_varchar	BOOLEAN	Unselected (False)	Whether to read all cells as VARCHAR values.
ignore_errors	BOOLEAN	Unselected (False)	Whether to ignore conversion errors and replace values that cannot be cast to the inferred column type with NULL.
range	VARCHAR/Range selection e.g., in dialog	Blank	The range of cells to read in spreadsheet notation (e.g., A1:B100). If not specified, Datastore automatically determines the data region.
stop_at_empty	BOOLEAN	True by default. If a range is specified, the default changes to False. Users can modify the value afterward.	Stop reading when an empty row is encountered. When a range is specified, this option controls whether processing stops at empty rows within the selected range. If an explicit range is provided, the default is false; otherwise, true.

Click Refresh to view the columns on the right.
Click Save. The Save as dialog is displayed.
In the dialog box, select the folder where the dataset will be saved and provide the dataset’s name.
Click Save to save the dataset. A confirmation message is displayed once the dataset is created.

Previewing dataset

You can preview the entire dataset, replacing the previous limitation of a partial dataset preview. You can also opt to preview the full dataset which may result in higher-than-expected costs. Additionally, the execution time required to generate the dataset preview could increase, depending on the size of the data.

See Formatting, sorting, filtering file data while registering datasets to learn additional ways to customize the data while you register datasets.

Related topics

File parameters
Creating a file with a Presto connection
Writing SQL queries using data in Hive
Refining data in datasets
Also see the common actions to know more about validating, sharing, and adding notes.