Version:

Data

Data comprises three types:

Ingests

An ingest is a means for getting data into Kinetica to be used in models.

  • Point to + Add New Ingest then click either New Batch or New Streaming to begin creating an ingest.
  • If an ingest has an output table, click the output table name to view a preview of the table
  • Type into Filter to filter down the ingests
  • Click show to display any archived ingests.
  • Click refresh to refresh the table
  • Click export to export the table's values as JSON or CSV
  • Click an existing ingest to display additional actions:
    • Click View Selection to open the Ingest Details page
    • Click Start to start the ingest job
    • Click Export Entity to export the ingest as a JSON object
    • Click Terminate to stop the ingest
    • Click Archive to archive the ingest; it will be stopped (if not already) and removed from the list of ingests. The ingest will no longer be useable.
    • Click Description / Config to review summary information for the ingest
../_images/aaw_ui_ingests.png

Ingest Details

The Ingest Details page provides a detailed look at a given ingest, including configuration information, state, and a data preview (if the ingest has a destination table configured).

  • Click Back to return to the Ingests page
  • Click Start to start the ingest job (if it's not running currently)
  • Click Export Entity to export the ingest as a JSON object
  • Click Terminate to remove the ingest
  • Click Archive to archive the ingest; it will be stopped (if not already) and removed from the list of ingests. The ingest will no longer be useable.
../_images/aaw_ui_ingest_details.png

Batch

A batch ingest uses the Kinetica Input/Output (KIO) tool to ingest a set of data at once.

To create a batch ingest:

  1. Provide a name for the ingest

  2. Optionally, provide a description for the ingest, then click Next.

  3. Select a Source Type. Kinetica, PostgreSQL, and AWS S3 are currently supported.

  4. Fill the Source and Destination fields, then click Next.

    Important

    All sources require a credential; consult Security for more information.

  5. Review the summary, then click Create.

Streaming

A streaming ingest uses Kafka to continuously stream data into Kinetica for use in AAW.

To create a streaming ingest:

  1. Provide a name for the ingest

  2. Optionally, provide a description for the ingest, then click Next.

  3. Fill the Source and Destination fields, then click Next.

    Important

    All sources require a credential; consult Security for more information.

  4. Review the summary, then click Create.

Datasets

Datasets are used for providing the training and test data for TensorFlow models.

  • Point to + Add New Dataset to begin the dataset setup process.
  • Click the Source Table name to display the table data
  • Type into Filter to filter down the datasets
  • Click show to display any archived datasets.
  • Click refresh to refresh the table
  • Click export to export the table's values as JSON or CSV
  • Click an existing dataset to display additional actions:
    • Click View Selection to open the Dataset Details page
    • Click Preview Data to open a dataset preview
    • Click Description / Config to review summary information for the dataset
    • Click Export Entity to export the dataset as a JSON object
    • Click Archive to archive the dataset; it will be removed from the list of datasets and will be no longer useable
  • Click Description / Config to review summary information for the dataset

Details

The Dataset Details page provides a detailed look at a given dataset, including configuration information and state.

  • Click Back to return to the Datasets page
  • Click Preview Data to open a dataset preview
  • Click Export Entity to export the dataset as a JSON object
  • Click Archive to archive the dataset; it will be removed from the list of datasets and will be no longer useable
../_images/aaw_ui_dataset_details.png

New Dataset

To create a new dataset:

  1. Provide a Name for the dataset.

  2. Optionally, provide a Description.

  3. Select a Source Table.

    Note

    The list of source tables is populated with tables in the Kinetica installation associated with this instance of AAW.

  4. Select one or more Columns from the table, or click Select All to select all columns.

  5. Optionally, provide a filter expression for the columns.

  6. Click Create.

Featuresets

Featuresets transform columns (features) from datasets using functions and lambda functions.

  • Point to + Add New Featureset to begin the featureset setup process.
  • Click the Dataset name to display the Dataset Details for the selected dataset
  • Type into Filter to filter down the featuresets
  • Click show to display any archived featuresets.
  • Click refresh to refresh the table
  • Click export to export the table's values as JSON or CSV
  • Click an existing featureset to display additional actions:
    • Click View Selection to open the Featureset Details page
    • Click Export Entity to export the featureset as a JSON object
    • Click Archive to archive the featureset; it will be removed from the list of featureset and no longer useabled

Details

The Featureset Details page provides a detailed look at a given featureset, including state, features, function, and lambda function information.

  • Click Back to return to the Featuresets page

  • Click Export Entity to export the featureset as a JSON object

  • Click Archive to archive the featureset; it will be removed from the list of featureset and no longer useable

  • If a lambda function is present in the Features table, click Lambda to view the lambda function code

    ../_images/aaw_ui_featureset_lambda_func.png
../_images/aaw_ui_featureset_details.png

New Featureset

To create a new featureset:

  1. Provide a Name for the featureset.
  2. Optionally, provide a Description.
  3. Select a Dataset from AAW.
  4. Click Transform Feature to create new features.
    1. Provide a Feature Name.
    2. Select a Column from the selected dataset.
    3. Select one or more Function(s) to transform the column.
  5. Click Lambda Feature to create new lambda features.
    1. Provide a Feature Name.
    2. Provide a Column name.
    3. Click Add Lambda Function and place the lambda code inside the text box, then click Update.
  6. Click Create.