Version:

Data

Ingests

An ingest is a means for getting data into Kinetica to be used in models.

  • Point to + Add New Ingest then click one of the options to begin creating an ingest.
  • If an ingest has an output table, click the output table name to view a preview of the table
  • Type into Filter to filter down the ingests
  • Click show to display any archived ingests
  • Click refresh to refresh the table
  • Click export to export the table's values as JSON or CSV
  • Click an existing ingest to display additional actions:
    • Click View Selection to open the Ingest Details page
    • Click Start to start the ingest job
    • Click Export Entity to export the ingest as a JSON object
    • Click Terminate to stop the ingest
    • Click Clone to clone the ingest
    • Click Archive to archive the ingest; it will be stopped (if not already) and removed from the list of ingests. The ingest will no longer be useable
    • Click Description / Config to review summary information for the ingest
../_images/aaw_ui_ingests.png

Details

The Ingest Details page provides a detailed look at a given ingest, including configuration information, state, and a data preview (if the ingest has a destination table configured).

  • Click Back to return to the Ingests page
  • Click Start to start the ingest job (if it's not running currently)
  • Click Export Entity to export the ingest as a JSON object
  • Click Terminate to remove the ingest
  • Click Clone to clone the ingest
  • Click Archive to archive the ingest; it will be stopped (if not already) and removed from the list of ingests. The ingest will no longer be useable.
../_images/aaw_ui_ingest_details.png

Batch

A batch ingest uses the Kinetica Input/Output (KIO) tool to ingest a set of data at once.

To create a batch ingest:

  1. Provide a name for the ingest

  2. Optionally, provide a description for the ingest, then click Next.

  3. Select a Source Type.

  4. Fill the Source and Destination fields, then click Next.

    Important

    All sources require a credential; consult Security for more information.

  5. Review the summary, then click Create.

Streaming

A streaming ingest uses Kafka to continuously stream data into Kinetica for use in AAW.

To create a streaming ingest:

  1. Provide a name for the ingest

  2. Optionally, provide a description for the ingest, then click Next.

  3. Fill the Source and Destination fields, then click Next.

    Important

    All sources require a credential; consult Security for more information.

  4. Review the summary, then click Create.

Import BYOC

A BYOC (bring your own container) ingest uses a Docker container to continuously ingest data into Kinetica for use in AAW.

To create a BYOC ingest:

  1. Provide a Docker container source URI, e.g.,

    <repo-name>/<image-name>:<tag-name>
    

    Tip

    Optionally, click Inspect to preview container metadata

  2. Provide a name for the ingest

  3. Optionally, provide a description for the ingest

  4. Add environment variable key value pairs:

    1. Click Add to create as many environment variables as necessary
    2. Provide a key name for each environment variable
    3. Provide a value for each environment variable
  5. Provide a destination table name for the ingest

  6. Click Import

Datasets

Datasets are used for providing the training and test data for TensorFlow models.

  • Point to + Add New Dataset to begin the dataset setup process.
  • Click the Source Table name to display the table data
  • Type into Filter to filter down the datasets
  • Click show to display any archived datasets.
  • Click refresh to refresh the table
  • Click export to export the table's values as JSON or CSV
  • Click an existing dataset to display additional actions:
    • Click View Selection to open the Dataset Details page
    • Click Preview Data to open a dataset preview
    • Click Description / Config to review summary information for the dataset
    • Click Clone to clone the dataset
    • Click Export Entity to export the dataset as a JSON object
    • Click Archive to archive the dataset; it will be removed from the list of datasets and will be no longer useable
  • Click Description / Config to review summary information for the dataset
../_images/aaw_ui_datasets.png

Details

The Dataset Details page provides a detailed look at a given dataset, including configuration information and state.

  • Click Back to return to the Datasets page
  • Click Preview Data to open a dataset preview
  • Click Clone to clone the dataset
  • Click Export Entity to export the dataset as a JSON object
  • Click Archive to archive the dataset; it will be removed from the list of datasets and will be no longer useable
../_images/aaw_ui_dataset_details.png

New Dataset

To create a new dataset:

  1. Provide a Name for the dataset.

  2. Optionally, provide a Description.

  3. Select a Source Table.

    Note

    The list of source tables is populated with tables in the Kinetica installation associated with this instance of AAW.

  4. Select one or more Columns from the table, or click Select All to select all columns.

  5. Optionally, provide a filter expression for the columns.

  6. Click Create.

Feature Sets

Feature Sets transform columns (features) from datasets using functions and lambda functions.

  • Point to + Add New Feature Set to begin the feature set setup process.
  • Click the Dataset name to display the Dataset Details for the selected dataset
  • Type into Filter to filter down the feature sets
  • Click show to display any archived feature sets.
  • Click refresh to refresh the table
  • Click export to export the table's values as JSON or CSV
  • Click an existing feature set to display additional actions:
    • Click View Selection to open the Feature Set Details page
    • Click Clone to clone the feature set
    • Click Export Entity to export the feature set as a JSON object
    • Click Archive to archive the feature set; it will be removed from the list of feature set and no longer useable
../_images/aaw_ui_featuresets.png

Details

The Feature Set Details page provides a detailed look at a given feature set, including state, features, function, and lambda function information.

  • Click Back to return to the Feature Sets page
  • Click Clone to clone the feature set
  • Click Export Entity to export the feature set as a JSON object
  • Click Archive to archive the feature set; it will be removed from the list of feature set and no longer useable
../_images/aaw_ui_featureset_details.png

New Feature Set

To create a new feature set:

  1. Provide a Name for the feature set.
  2. Optionally, provide a Description.
  3. Select a Dataset from AAW.
  4. Click Transform Feature to create new features.
    1. Provide a Feature Name.
    2. Select a Column from the selected dataset.
    3. Select one or more Function(s) to transform the column.
  5. Click Create.