Kinetica Release Notes

Build version:

  • 209 Resolved Issues


KDEVOPS-586 Add option to pass docs tarball rather than running docs build
KECO-1255 Added collection name to kinetica-to-kinetica with TYPE_VALIDATION
KECO-1435 Added --s-null-value parameter to specify a custom null token in csv files
KECO-1465 Added csv_fmt_options
KECO-1465 Added test cases for various combinations of CSV formatting
KECO-1465 Added test data and test cases
KECO-1465 Added test harness support for multiple matching csv fmt option values
KECO-1486 Added regression and unit tests for PostgreSQL URI parsing
KECO-1486 Added schema support to PostgreSQL cli URI format
KECO-1515 Update removing quoting around sql statement parameterized with sql.Identifier since quotes are added naturally if needed, and removing conn.close() and cursor.close() for memoized connections
KECO-1516 Added fully qualified reference to SQL in comparison
KECO-1524 Added new Postgresql cli arg --[s|d]-postgis-schema
KECO-1525 Added PostGIS query examples to docstring
KECO-1525 Added postgis_schema comment explanation
KECO-1525 Added URI source and destination property
KGEO-631 Added graph cancellation test
KGEO-631 Added graph jobs cancellation
KGEO-644 Graph table monitor test update
KGEO-649 st_area spheroid metric supports large polygons; added PostGIS ground truth to the test; ensuring results are within 5%
KGEO-654 Handle case of using float/double ranges in classbreak when the data is int/long

Added a new show/graph/grammar no-doc api for returning all the graph grammar for end-points and options with its defaults so that these could be used to generate the graph UI widgets without async with the native graph code

KGEO-658 Added the new match_supply_demand choice option to the /match/graph endpoint as well as the option partial_loading. The endpoint now has the capability to schedule multiple supply demand optimizations
KGEO-661 Removed unsafe asynchronous access of internal lod data structure in shape-processor's memory check
KGEO-662 Fixed a crash in WKT ordered class-break in double precision
KGEO-667 Fixed a crash happening with a hash-line rendering WMS request for an area without a line when double-precision rendering path is used
KGEO-668 Avoid memory leak in st_multipleringbuffers by not processing multi-geometries
KGEO-668 Fix leaks in st_envdwithin and st_buffer (spherical mode)
KGEO-668 Fix memory leak in xyEnvDWithin
KGEO-668 Fix potential leak in GeoUtils::multipleRingBuffers and bufferWithOutDissolve
KGEO-668 Make mercTransformPoly() not leak mem if it throws
KINFRA-1075 Add ability to create standalone thread pool
KINFRA-1085 Add gpudb_cluster_* --show-networks and --show-gpus args to help debug hardware issues w/o needing to start the db
KINFRA-1085 Closing the ZmqProxy context during normal shutdown is not actually an error, don't log the zmq error
KINFRA-1085 Fix Datetime.cpp concat string + raw int, need to call std::to_string() on int
KINFRA-1091 Add worker registry
KINFRA-1095 serial parts runs clear test first and last, don't fill drive for --repeat testing
KINFRA-1095 Update CMakeLists.txt vector[] check for gcc 4.8.2 (centos7)
KINFRA-1255 Add profile points for chunk compaction
KINFRA-1255 When rebalancing, grab the data to send over from column vectors vice IndexedDb, provided all the columns are already in ram
KINFRA-1281 Fix for incorrect shard version in RebalanceCache when rebalancing large number of records. Was causing rollback failures on sharded records
KINFRA-1302 /admin/remove/ranks endpoint now waits for notification from HostManager that persist has been cleared, before returning
KINFRA-1318 Account for ShapeProcessor memory
KINFRA-1320 Account for PinnedMemoryPool memory Misc code/include cleanup
KINFRA-1318 Remove extraneous shape processor memory usage set that could cause race
KINFRA-1318 Tracking shape processor memory usage for tiering even when shape processor memory limit is zero (unlimited) or lod rendering is disabled
KINFRA-1349 Add user admin permission to grant and revoke system endpoints
KINFRA-1349 Create USER ADMIN permission
KINFRA-1423 Make ITER table readable by public role Misc code cleanup
KINFRA-1349 Fix some user admin bugs, add tests
KINFRA-1363 Compilation fix for db_benchmarks.cpp to add args to call rocksdb::WriteBatch(0,0) for rocks 6.1.2
KINFRA-1363 Compilation fix for db_benchmarks.cpp to NOT add args to call rocksdb::WriteBatch(0,0) for rocks 4.x
KINFRA-1369 In Endpoint_get_records_by_column_internal, add force_refresh option for testing purposes
KINFRA-1369 Rebalancing tests use GRBC vice /get/records when necessary for record comparisons before/during/after rebalance
KINFRA-1370 Fix WorkerGRBC releasing join objects in the wrong order
KINFRA-1371 Check for job cancellation during projection reshard
KINFRA-1372  add logs in SetIndexedDbFile::rename_setdata_id
KINFRA-1375 When compacting a base table and inserting new chunks, insert stencils in all dependent views
KINFRA-1377 Update insert/records to use read vice write tables
KINFRA-1378 Add fifo lock on alter table for protection from inserts on multi head
KINFRA-1380 Ensure that the Sigterm handler successfully shuts down the crashing rank
KINFRA-1382 Add "run_tag" option to UDF execution endpoints
KINFRA-1385 Don't refresh replicated tables during or after rebalancing
KINFRA-1385 Fixed add ranks test to not refresh view/join tables in verify db call
KINFRA-1403 Catch error in SetManager if missing types
KINFRA-1403 JsonWrapper use buffered save by default, can now set buffer size in load and save
KINFRA-1407 Carry over is_automatic_partition flag to tables that are recreated on added ranks
KINFRA-1410 Add 'system limited' to error on missing persist directory
KINFRA-1410 Add "system-limited-fatal" message if persist folder is missing
KINFRA-1410 Add system-limited-fatal to all CommsUtils errors about missing responses Also fixed bug preventing restarted ranks from syncing tables/types due to system-limited-fatal blacklist
KINFRA-1410 Change so that we now update indexedDB first on inserts and updates, and verify persist directory exists before adding/updating records
KINFRA-1414 Promote a couple add ranks endpoint logging statements to info to help diagnose hangs when adding ranks
KINFRA-1419 Fail out of rebalancing if we can't get table sizes but the table still exists
KINFRA-1424 Defer reading of SetIndexedDb object files on startup until needed uses conf load_vectors_on_start to detect on-demand loading
KINFRA-1430 Add signature to zmq comms to detect bad tcp messages
KINFRA-1433 Add conf param: init_tables_num_threads_per_tom for loading tables metadata on startup
KINFRA-1433 Add omp threads to indexedDb init
KINFRA-1433 Adjust number of threads for init chunks for lowest from num chunks vs num threads from conf
KINFRA-1433 Change limit of number of threads to 1024
KINFRA-1433 Improve indexedDb init
KINFRA-1433 Parallel chunks loading
KINFRA-1439 Change against system resource group to have no memory limits
KINFRA-1441 Fixes for condition where worker rank had shard version of 0 causing ingest to fail
KINFRA-1445 Fix 'failed to release' WARN at shutdown
KINFRA-1449 Fix deadlock in multihead insert
KINFRA-1449 Prevent /alter/table and multihead /insert/records from running at the same time
KINFRA-1450 Set gaia_info.start_time in GM::reinitForRestart() to reflect ranks starting, not hostmanager
KINFRA-1453 GaiaSet sync (GaiaSetManager) pins the stencil before saving now
KINFRA-1453 Set stencil file out of sync
KINFRA-1454 Add auto rebuild table in GaiaSetManager
KINFRA-1459 Remove m_last from AllocatorLinkedList to eliminate memory leak/segfault; add validation logger and validator
KINFRA-1462 Prevent /alter/table and /alter/table/columns from timing out on the server
KINFRA-1463 Fix int overflow in SetIndexedDb for large records
KINFRA-524 NewAllocator new macros and env vars to control logging
KINFRA-575 Improvements to a couple rebalancing error messages
KINFRA-724 Fix gpudb_batch wms handling and threading
KOLAP-1461 Update aggregate_unique_internal_response.json file first before merge to master
KOLAP-1227 Add support for distributed update and delete,ki_record_id function
KOLAP-1305 Make EglTest even more verbose on error
KOLAP-1324 Increase sleep time for view refresh test there were intermittent errors when running under docker also moved the test to serial so fewer things will be running in parallel
KOLAP-1324 Use full UUID because name collision happening when running parallel
KOLAP-1399 Added chunk skipping mechanism for remaining visualize chart and labels Endpoints
KOLAP-1399 Added chunk skipping mechanism for visualize classbreak and contour Endpoints. Reorganized some previously merged code
KOLAP-1399 Added chunk skipping mechanism for visualize_image_heatmap Endpoint
KOLAP-1399 Added few more tests related to DECIMAL Data Type - Removed unnecessary include - Handled the DECIMAL Data Type for X/Y columns properly
KOLAP-1399 Added new option chunk_skipping to the visualise_image Endpoint. Worker is enhanced to do chunk skipping while processing the chunks by looking at the Min/Max or Bounding Box of a chunk. Chunk skipped information i.e Processed chunks to Total chunks, from every Rank and Tom is returned in the Response message
KOLAP-1399 Disabling chunk skipping of visualize Endpoints by DEFAULT
KOLAP-1399 Fix minor issue with the tests for visualize image chart
KOLAP-1399 Fixed a wms issue for visualize Endpoints by adding conversion logic from meters to lon/lat - Handled syncing between workers and GL renderer even when the chunk is skipped in image, image_chart and classbreak Endpoints - Added logic to check if the Min/Max or BoundingBox is set in the chunk. If not set, chunk skipping is disabled
KOLAP-1430 Explain analyze to show explain output even if the query fails. The succeeded endpoints will have all the info and failed endpoint to show the error message. inflight endpoints, unscheduled endpoints will show the default request data
KOLAP-1448 Add most tests for security functions, and fix issues they found
KOLAP-1448 Added support for security functions with non-constant parameters
KOLAP-1448 Added tests for MASK, and fixed issues it found.  Removed (commented out) USER_ID() and USER_NAME() functions.  Removed user_id returned in create_user_internal
KOLAP-1448 Adding CURRENT_USER and other new security-related functions
KOLAP-1448 Change missgin from previous check-in
KOLAP-1448 Fix for IS_MEMBER where first parameter is non-constant, and second parameter is non-constant
KOLAP-1448 More NULL tests for MASK function
KOLAP-1448 Temporary fix for tests failing to compile due to TierExecutionContex constructor now needing the user id
KOLAP-1452 Added columns option to Grant/RevokePermissionTable
KOLAP-1452 Added Revoke support for columns
KOLAP-1452 Granting column filters works
KOLAP-1452 Hook up columns parameter to Grant/RevokePermissionTable
KOLAP-1452 Save column & row security settings in GRANT table permissions
KOLAP-1452 Started adding new security functions
KOLAP-1452 Started implementation of new functions.  Need to pass authUserId around to get this to work
KOLAP-1461 Add abs() function tests on ulong datatype
KOLAP-1461 Add aggregate/unique sort_order tests
KOLAP-1461 Add copy_if_stencil_true template specialization on ulong
KOLAP-1461 Add filter test on ulong
KOLAP-1461 Add filter/bybox tests on ulong
KOLAP-1461 Add get/records tests on ulong
KOLAP-1461 Add index test on ulong with very large values
KOLAP-1461 Add more tests in test_ulong_update() function, and fix some low level mistakes
KOLAP-1461 Add primary key tests
KOLAP-1461 Add some missing cast functions on ulong
KOLAP-1461 Add test_ulong to all_test, and use gaia::remove_quotes() to handle ulong datatype in encode_value functions
KOLAP-1461 Add the missing bit xor/or/and left/right shift functions
KOLAP-1461 Add update/records tests, and fix some low level mistake
KOLAP-1461 aggregate/unique sort on ulong correctly
KOLAP-1461 Cast string constant to ulong correctly when parsing expressions
KOLAP-1461 Fix a bug in convert_constant_to_trpn1 when handling ulong datatype
KOLAP-1461 Fix bugs when inserting random records on ulong columns, and add tests
KOLAP-1461 Fix for wrong result in some scenarios when getting difference of unsigned values with std::abs()
KOLAP-1461 Fix handling of ulong vector constants (i.e. the 'in' clause); add in __host__ __device__ specifiers for the ulmod function
KOLAP-1461 Fix missing case for groupby on ulong
KOLAP-1461 in/not in operator support ulong datatype
KOLAP-1461 Optimize parseUlongStr function to handle more complicated input patterns
KOLAP-1461 Remove ulong_minus_op
KOLAP-1461 set TYPE_FIELD_ANNOTATION_ULONG a valid integer number
KOLAP-1461 Simplify parseUlongStr and have it return false for strings with a decimal point
KOLAP-1461 Small optimization, and put back filter/by_range tests
KOLAP-1461 Update ../EndPointParamsDefs.h/cpp, add create/type tests, and put back some assertions in get_records_by_column
KOLAP-1461 Update CMakeList.txt file when adding file
KOLAP-1461 Fix error message for ingest of too-long store_only charN values
KOLAP-1461 Use boost::try_lexical_cast rather than sscanf to parse unsigned longs
KOLAP-1490 Add expressions to window function range frames add general expressions to window function frame beginning and ending add interval expressions for date types check intervals for overlap add tests for lots of variations of expressions and date intervals allow plural interval values like HOURS and MINUTES for now we don't allow intervals for TIME and DATE because TIMESTAMPDIFF does not support them, but as soon as we modify TIMESTAMPDIFF to support them, support can be turned on

SQL showview statement will show the SQL definition if the materialized views are created through SQL path

KOLAP-1569 Fix StencilTierFixture.PKLookupTable_SpaceEfficientVector test failure

Do not publish paging table when KI_HINT_KEEP_TEMP_TABLES hint is on

KOLAP-1585 /aggregate/unique just calls /aggregate/groupby. This gets all the improvements that groupby has (such as using dict encoding) without reimplementation. Since unique did support operating on collections and groupby does not now unique does not as well tests have been updated to reflect that
KOLAP-1589 Fix error message for multihead key lookup with a sort specified (which isn't supported)
KOLAP-1594 Add execute_as option to /execute/sql
KOLAP-1594  Add support for impersonating another user
KOLAP-1594 Improve error message for bad impersonations
KOLAP-1594 Return Impersonation errors in a way more friendly for the API
KOLAP-1594 Support execute_as in /execute/sql
KOLAP-1602 Fix error string in parse_timestamp_string(), use to_string on int
KOLAP-1602  Fix problem where error is detected in parsing bad timestamp month value but we don't throw
KOLAP-1602 is_date_string, is_time_string and is_datetime_string no longer use try/catch to determine return value
KOLAP-1608 Add left_join filter-key decorator to primary-key joins and equi-joins moved functions to do this from Worker_predicate_join_filter to FilterPlanner eliminated primary-key attribute-index path from Worker_primary_key_join_filter
KOLAP-1608 Add some more extend_chunk_filter_key functions to the join filters #ifdef out all new chunk-filter-key functions while they are under construction to avoid compiler warnings and unused function calls
KOLAP-1608 Added more extend_chunk_filter_key calls. create_join_table create_explain and show_filters options added
KOLAP-1608 added more tests begin dealing with outer joins between join filter save all-true filter key for chunk correctly
KOLAP-1608 All FilterPlans that take expressions new extend the chunk-filter-cache.  Added new test that tests each filter-plan type. Added new create_join_table option, 'show_filters_in_explain', to show filter keys between filter-plan steps.  Removed ifdefs to comment out new filter cache key creation and instead use the chunk_cache_enabled gpudb.conf parameter to disable
KOLAP-1608 Begin creation of m_chunk_filter_key_map to store keys for joined stencils
KOLAP-1608 Change conf parameter enabling new filter-keys to shadow_cube_enabled.   Modify chunk function used in key generation to include set_indices and adjust the set_indies parameter during concatenation adding the set_index_offset
KOLAP-1608 Filter endpoint called on GaiaSet now updates chunk_filter_keys, Fixed problem with forwarding filter keys between joins, added more tests
KOLAP-1610 Fix groupby-on-wkt sinking for 5+ ranks/toms
KOLAP-1611 Enable parallel execution of chunking loop in make_index_stencil_left_join enable parallelism - already tested in 6.2
KOLAP-1616 Fix tests failing when planner not enabled
KOLAP-1616 Make FilterPlanner note set_nullability of left join minimize or maximize plan add tests to
KOLAP-1618 Fix asof join where data types of expressions being compared are not the same
KOLAP-1619 Change calls to ghp->get_copied_mem_block to ghp->get_mem_block in GaiaSet::update_records_with_map.  There was no need to copy the memblock as the ownership of the geometry pointers was not getting transferred to the updated table
KOLAP-1625 Add support for UNSIGNED BIGINT in show sql statement
KOLAP-1627 Fixed error Minimize filter plan when in using hardwired values of 0 and 1 for foreign_key_set_index and primary_key_set_index
KOLAP-1629 Add chunk skipping for != operator fix problem with unary filter not doing null-filtering for all-true nullable columns add additional tests to test nullable columns, indexed columns, and != operator
KOLAP-1629 Added support for chunk skipping on unary filters
KOLAP-1629 Backout inadvertent checkin of SqlDriver.cpp
KOLAP-1629 Finished unary filter chunk skipping. Unary filter done on skip-simplified expression.  Filter-keys formed based on skip-simplified expression
KOLAP-1629 Improved method of determining if expression is skippable that as byproduct produces a simplified version of the expression that can be used in filter caching
KOLAP-1629 Initial commit supporting chunk skipping for unary filters
KOLAP-1633 Added and tested new chunk skipping framework.   Tested on JoinedSet constructor between filter-plan chunk skipping.   Commented out. Can be used as a starting point for the unary filter chunk-skipping.  Already handles the 'or' condition but needs to be extended to handle in clauses
KOLAP-1633 Corrected conditions for GE_OP and EQ_OP in new skip_chunk function
KOLAP-1639 Add support for multiple keys in LIST partitions parse nested parentheses to support a LIST of multiple values print nested parens for show table generate filter of multiple keys when adding partition to move from default added tests for multikey list expressions add support for INCREMENT keyword for INTERVAL partitions to avoid redundancy
KOLAP-1640 Disable group-by chunk caching for JoinedSets until new filter-keys are fully implemented.  Disable test_chunk_cache tests for join_tables
KOLAP-1645 Add test for new hint and access privilege
KOLAP-1649 groupby-on-wkt support 'sort_by':'key' option
KOLAP-1650 Fix window function test to handle UTC time test was initialized for eastern time zone, didn't handle when the machine was set to UTC. changed the method of determining time since epoch and reinitialized the test to be timezone-independent
KOLAP-1656 Add PKLookupTable::isMultiChunk function that doesn't require lookup table to be pinned use this in place of lookup->getChunkIndices().size() != 0
KOLAP-1656 Remove TierExecutionContext member  from JoinedSetChunk, pass authUserId to its get_datatype() func instead
KOLAP-1659 Accelerate chunk skipping for between plans by using original method to determine if chunk is all_false
KOLAP-1659 Add chunk-skipping tests for charN types and indexed column * New skip algorithm for in clause where we preprocess in operators before looping through chunks   convert in operator vector constant to typed vector and record info in RpnElement m_skip_in_dt, m_skip_in_vector Use this information in skip_chunk_for_in_clause to more efficiently process in clauses for chunks   Add more tracers to * Parser.h added RpnElement m_skip_in_dt, m_skip_in_vector and handle in copy and assignment operators * Parser.cpp fast return from evaluate_skip if we determine apply_skip_to_rpn tree returns with single constant value do not go through simplifier and evaluator
KOLAP-1659 Add lock to protect the FilterPlan skip info map.  Move lock in Type::get_column_name_in_base_set to more correctly protect concurrent access to the m_name_to_base_set_name_map
KOLAP-1659 Cancel future chunks when get-records-by-column has filled the requested number of records test unary filter incremental updates with chunk skipping fix bug in constructing 'not is_null' expression in Worker_unary_join_filter
KOLAP-1659 Do not chunk skip unary filters part of outer join
KOLAP-1659 Fix problem with using create_join_table filtering when get_records_by_column has derived columns
KOLAP-1659 Get GaiaSet count without pinning stencils if possible improves JoinedSet constructor performance for many chunks - Split out chunk skipping in JoinedSet constructor into its own parallized loop required plumbing Worker into JoinedSet constructor - Modified Worker_create_join_table, Worker_join_filter and Worker_incremental_join to pass in Worker to JoinedSet constructor and extend function - WorkerChunker now has Tracer for onProcessChunkImpl
KOLAP-1659 Get_records_by_column now calls create_join_table rather than filter to implement filtering dictated by the expression option
KOLAP-1659 Handle exception in preprocessing_in_clause_vectors correctly. Remove stray debug print statement
KOLAP-1684 Use row_indices size to make host_info for db_attrs as this is clipped to the permutation size
KOLAP-316 Fix InitLock in all workers that didn't scope them properly


None currently


Should you have any questions or concerns, please visit our support page, official documentation page or email us at

Was this article helpful?
0 out of 0 found this helpful
Have more questions? Submit a request


Article is closed for comments.