Version:

7.0 Release Notes

Publish Date: 01/31/2019

Features

Active Analytics Workbench

  • The Active Analytics Workbench (AAW) platform is now available. A brand new API and UI are available to improve existing machine learning workflows. For machine learning, Tensorflow and Blackbox (using Docker containers) models are supported. AAW also supports continuous deployments, on-demand deployments (via an inferencing REST endpoint), and audits of both TensorFlow models trained internally and imported models. AAW has a new ingestion interface that allows ingesting data from variety of sources and methods, including Kinetica, PostgreSQL, and Kafka.

Core

  • Kinetica now supports Resource Management. Resource management involves the three following resources:
    • Storage Tiers: Data containment layers within the database (VRAM, RAM, Disk Cache, Cold Storage)
    • Tier Strategies: Data object eviction priorities within each storage tier to help define memory usage and data priorities
    • Resource Groups: Resource fencing -- process scheduling priorities and other limits imposed on specific groups of users
  • The ability to alter multiple columns in a single call is possible via the new /alter/table/columns endpoint.

Geospatial / Network Graph

  • Kinetica now includes a network graph solver server. The graph server provides a generic and extensible design of networks with the aim of being tailored or used for various real-life applications, including transportation, utility, social, and geospatial.
  • Kinetica now offers a Vector Tile Service (VTS) to generate Vector Tiles and support client-side visualization of geospatial data contained within the tiles. Generating Vector Tiles using Kinetica involves passing in the VTS URL to the client-side renderer.
  • The /wms endpoint now supports contour plot visualization functionality

KAgent

  • The Kinetica Agent (KAgent) UI is now available to automate Kinetica installation and configuration. KAgent can automatically install Nvidia drivers for CUDA installations, the Active Analytics Workbench (AAW) and Kubernetes (required for AAW), configure the cluster for SSL and/or external authentication and high availability. KAgent is also cloud ready and able to provision and/or deploy to instances in Amazon Web Services (AWS), Google Cloud Platform (GCP), or Microsoft Azure. KAgent also allows one to configure the location of the head node, AAW, and the graph server within the cluster. Upgrades will now be managed using KAgent.

SQL/OLAP

  • Kinetica is now packaged with a new and improved SQL interpreter.
  • A new /execute/sql endpoint is available. You can now send SQL commands to the native API without an ODBC driver.
  • The SQL engine uses a query planner that analyzes an entire query for the many different ways it can be solved (e.g., performing a filter before a join, performing a filter after a join, etc.) and selecting the plan that is the most efficient. Once this plan is utilized, it’s cached so the plan can be used again without having to re-compute it when the same query is received again.
  • Complex SQL queries that involve multiple operations are now analyzed for interdependency. Any operations that have no dependencies on others are executed in parallel.
  • Distributed operations, such as UNIONs between sharded and replicated tables or distributed JOINs, are now possible using SQL syntax or /execute/sql. The database will automatically re-shard or replicate tables temporarily as necessary to help process the query. Note that the distributed query can be slower and use more memory than a traditional non-distributed operation. The sql.distributed_joins setting in /opt/gpudb/core/etc/gpudb.conf controls the ability to use distributed operations.
  • Correlated sub-queries are now supported.
  • New SQL support for:
    • EXPLAIN
    • Partitions
    • Tier Strategy Definitions
    • Logging Levels
  • Partitioning is now available. Table data that is sharded or replicated can be partitioned to aid storage tiering and data skipping. The two types of supported partitioning schemes are:
    • Range
    • Interval

UI

  • The Kinetica Administration Application (GAdmin) has undergone a visual refresh and now supports functionality for managing resource groups, interfacing with tier strategies and partitions, reviewing graph node and edge counts, and deleting graphs.
  • Kinetica Reveal service has undergone a visual refresh as well as received some usability improvements.

Version 7.0.3

Publish Date: 05/08/2019

AAW

  • Added support for NVIDIA RAPIDS to all three deployment modes
  • Improved configurations for all container assets (for inside-firewall use)
  • Bundled dependencies in the RPM

Core

  • Various performance improvements and bug fixes

SQL/OLAP

  • Added support for RANGE and NULLS FIRST/LAST in window functions
  • Added support for using NOT IN within an UPDATE/DELETE query
  • Added support for managing resource groups
  • Added support for CSV & Multi-head ingestion via ODBC
  • Tuning improvements, including:
    • Faster STXY_* functions, geo-joins, & non-geospatial equi-joins
    • Avoiding creating projections when paging through result sets, to reduce memory

Geospatial / Network Graph

  • Added support for Adaptive LOD Vector Tiles (Client side VTS)
  • Added options to image/chart requests for applying log scaling for min/max request parameters
  • Added ST_ConcaveHull geo function
  • Added solution type support to geo functions:
    • ST_DFullyWithin
    • ST_LongestLine
    • ST_MaxDistance
    • ST_Segmentize (Euclidean)
  • Graphs
    • Improved interface for applying restrictions when querying
    • Added support for constant expressions in all endpoints

APIs/Tools/Connectors

  • Added HA support to multi-head ingest & egress via Java API
  • Added primary cluster identification for the C++ API

DevOps

  • Added support for Docker EGL rendering
  • Included S3 Cold storage in backups

UI

AAW

  • Added Blackbox container creation wizard
  • Added RAPIDS model creation, training, and deployment

KAgent

  • More detailed logging and easier to use logging console
  • Added support for upgrading clusters already in KAgent from previous releases of Kinetica 7.0

GAdmin

  • Added support for new KIO options (Spark timeout & multi-head, etc)
  • Added support for Drag & Drop import of CSV files with Kinetica schema headers
  • Added global cluster system status
  • Added support for init_with_now attribute in table configuration editor
  • Added support for CSV data export from WMS preview
  • Improved application of all stats optimization recommendations

Reveal

  • Added global filter on map slice extent
  • Added data export from data browser slice with authentication
  • Added support for dashboard import/export
  • Added dark theme CSS template

Version 7.0.2

Publish Date: 04/17/2019

AAW

  • Added support for NVIDIA RAPIDS Model Training (Regression) and Deployment/Inference
  • Added facility: BlackBox Container Wizard
  • Added auto-launcher for State Manager
  • Upgraded all endpoints to be capable of asynchronous operation

Core

  • Added support for assigning resource groups to roles; users will inherit resource groups from roles if they have none set explicitly

SQL/OLAP

  • Added support for standard SQL views
  • Added support for SQL UPDATE and DELETE statements that reference other tables via JOIN
  • Improved efficiency of geospatial filters
  • Added JDBC support for multi-head ingest, including from CSV files
  • Various optimizations and bug fixes

Geospatial / Network Graph

  • Added Map Matching feature, which can map a given set of points to an underlying graph
  • Added geodesic distance option to ST_buffer geospatial function

APIs/Tools/Connectors

  • R ODBC connector support for row.names and bind & query parameters
  • Added primary cluster identification for Java & Python APIs
  • KIO
    • Added support for connecting to public S3 buckets
    • Improved transfer time between Kinetica instances via Spark
    • Added support for multi-head ingestion into Kinetica
    • Added support for ingesting into a table within a collection
    • Added ability to tune the JVM parameters for Spark memory limits
    • Improved logging

UI

AAW

  • Added image inference testing support
  • Added support for managing credentials for private docker registries

KAgent

  • Added detailed progress reporting during install and other long running tasks
  • More flexible backup and restore operations

GAdmin

  • Added Drag & Drop inferencing of WKT fields within CSV files
  • Added support for tab-delimited file import
  • Added support for role-based resource groups
  • Added support for list partition configuration

Reveal

  • Improved dashboard global filtering performance
  • Added SDK command for removing slices from build
  • Added support for custom base map server URL for map slice
  • Added info pop-up and auto-fit data feature for scatter plot slice

Version 7.0.1

Publish Date: 03/08/2019

AAW

  • Introduced Batch Model deployments
  • Added Credentials concept to handle reuseable connection profiles
  • Added support on Containerized Ingests to utilize saved Credentials concept
  • Introduced second process (state manager) to manage long-running processes
  • Added more filtering options for audit searches
  • Added Config-Driven K8s Cluster Selection
  • Added capture & searchability of stack traces on individual inferences
  • Faster inventory/entity dashboard operations

Core

  • Added ability to bypass Disk Cache tier for persistent objects when Disk Cache & Persist tiers are similarly performant (tuning option)

KAgent

  • Improved handling of upgrading httpd 2.2 configurations (when upgrading from GPUdb 6.2)
  • Initial support for KAgent creation of Nvidia-enabled Kubernetes clusters

SQL/OLAP

  • Improved Partitioning
    • Added List partitions
    • Added support for Date columns
    • Improved partition syntax
  • Added ability to apply the current date/time as a default for date/time columns
  • Added support for Alteryx
  • Improved support for SQL with HA
  • Various optimizations and bug fixes

Geospatial / Network Graph

  • Added ability to query graphs by edge/node ID constant value
  • Added Probability Rank Solver (Hidden Markov Chain)
  • Added new edge/weight configurations (identifier combos)
  • Added ST_partition geospatial function

APIs/Tools/Connectors

  • New R ODBC connector
  • Added high availability support to the basic client class for C++, Java, & Python and to the multi-head operations for Java & Python
  • KIO
    • Added support for Oracle
    • Added support for Teradata
    • Added support for SybaseIQ
    • Added support for SQLServer

UI

AAW

  • Faster loading dashboard
  • Batch model deployment
  • Entity archiving

KAgent

  • Added configuration of non-core/gpudb cluster nodes for dedicated AAW or Graph server
  • Added display of cluster admin service statuses (Running, Stopped, etc)
  • Added AAW administration commands
  • Improved robustness authentication/login

GAdmin

  • KIO UI support for Oracle, Teradata, Sybase IQ, and SQL Server

Reveal

  • Pivot Table heatmap coloring
  • Map auto-zoom and class-break legend
  • New improved Scatter Plot with pan/zoom