Kinetica Release Notes

Build version:


KDEVOPS-374 Moved tomcat-restart to posttrans scriptlet
KDEVOPS-505 Add basic UDF API dependencies
KDEVOPS-505 Add UDF API to distributed python environment
KDEVOPS-505 Fix to UDF API instead of double install regular API
KDEVOPS-558 Removed unnecessary pycairo dependency from rpm
KDEVOPS-563 Added logging modules to httpd
KECO-1435 Added --s-null-value parameter
KECO-1435 Renamed null_value to custom_null_value
KECO-1459 Update add logging for cases where the Spark ingest errors out, and raise odo.KIOValueError instead of py4j.protocol.Py4JJavaError
KECO-1459 Update spark/ add SPARK_OPTION_INGESTER_FAIL_ON_ERRORS constant, and update add option to pass into the Spark Connector
KGEO-609 Added /show/graph endpoint that shows basic properties of one or all graph
KGEO-609 Added missing graph server internal show graph schemas
KGEO-612 Made the 'rings' as primary parameter of the query/graph endpoint + revised all tests and ppt files
KGEO-629 Flagging the conflicting weight-edge combinations from the interface now. Fixed the tests accordingly
KGEO-636 Fixed missing class issue in ordered class break; it was incurred by allocating a frame buffer outside of valid extent
KGEO-636 Fixed ordered class-break deadlock happening when the number of classes exceeds the number of frame buffers available
KGEO-636 Hooking up worker-renderer sync with AbortableConditionSet; make sure rendering tasks are cleaned up when a job is canceled
KGEO-638 hash/sha256 functions now support geo columns
KGEO-639 Fixed /show/graph not showing the original create request
KGEO-642 Fixed a frame buffer allocation issue with a wms request of large width/height

Improved GL frame buffer allocation fix by changing allocateFB() to return NULL ptr when allocation fails; additional check for frame buffer count and rendering request map

KGEO-643 Prevent invalid frame buffer objects from deallocating valid ones
KGEO-645 Fixed excessive memory usage by CPUDataBuffer with many toms and a stream of wms requests
KGEO-647 Hold a pin for a geo-vector until rendering is completed
KGEO-648 New geospatial aggregate: 'st_aggregate_intersection' (also st_intersection_aggregate) to be able to compute the intersection of WKTs within a group
KGEO-649 First cut at a 'spheroid'-based are metric for st_area. Modelled on PostGIS's pt_array_spheroid method. Not yet implemented: iterative loop for polygons with large deltas between longitude values, support for polygons in the Southern Hemisphere, or those straddling the dateline or the equator
KGEO-649 st_area spheroid metric supports large polygons; added PostGIS ground truth to the test; ensuring results are within 5%
KGEO-650 st_area improvement for the case of sphere option: It appears that smaller polygon area computations are very sensitive to the sub-corner angle oriented area signs. The near zero tolerance selection was observed to be too low. Once this is changed to 1.0E-10 from 1.0E-12 the results started to appear acceptable/comparable to PostGIS version for the spherical option
KGEO-651 Fixed ordered class break crash happening with many concurrent requests
KGEO-651 Rescheduling GL frame buffers to avoid a dead-lock
KGEO-651 Update GL rendering request cancellation logic to reflect frame buffer rescheduling change
KGEO-652 New 'st_perimeter' geospatial function to return the perimeter of a polygon/multipolygon/geometrycollection. Support our 3 distance metrics: Euclidean, Spherical, Spheroid
KINFRA-1001 Fixed issue where GRBC on replicated table could throw when a rank is being removed
KINFRA-1036 Add UDF execute permission support and tests; remove dead code; add GAIA_DESTRUCTOR logic to ScopedNamedSystemSemaphoreLock (in case release throws)
KINFRA-1085 Added timeout to and capture more disk info
KINFRA-1099 Avoid potential deadlock between materialized view refresh and solo jobs
KINFRA-1123 Fix issue of missing replicated table data on new rank with no shards
KINFRA-1123 Script to reproduce bug where you add a rank while a projection is making a replicated table, but the replicated table doesn't get sent to new ranks b/c it doesn't have shards yet
KINFRA-1142 In add ranks endpoint, if add fails then wait for new ranks to remove themselves from cluster before exiting endpoint
KINFRA-1142 Several improvements to add ranks timeouts
KINFRA-1171 Added test stencil after restart
KINFRA-1216 Fixed fuzzer generating invalid expressions on joins
KINFRA-1216 Fix against fuzzer update errors
KINFRA-1216 Updated fuzzer with numerous enhancements
KINFRA-1219 Limit rebalancing data messages to 10k records per message
KINFRA-1237 Fix Endpoint verify db to refresh views and joins (if requested by user) prior to checking shard routing correctness
KINFRA-1247 Make internal rebalance jobs (per round) not timeout
KINFRA-1254 Add option to show completed async jobs (which are hidden by default)
KINFRA-1254 Hide completed async jobs unless requested by an option
KINFRA-1255 Expose rebalance aggressiveness option to users
KINFRA-1255 In GaiaSet add rebalancing tuning flag that makes sender call decode_objects and send bulk_add_internal requests
KINFRA-1261 Added test to verify rebuild db runs on startup
KINFRA-1269 Fixed show table sizes on a collection
KINFRA-1271 Fix to get client IP address instead of host
KINFRA-1271 Include source ip address in the response for /show/jobs
KINFRA-1273 Fix log messages that write out the shard lists printing size instead of elements - Fix several compiler warnings
KINFRA-1273 Set rollback record count based on number of inserted records - Was using the delete stencil count which caused the rebalance job to wait indefinitely for rollback to complete
KINFRA-1273 Truncate long log messages when printing shard indices list - Restore option to log container size only
KINFRA-1279 Remove unused rq/rp objects from workers
KINFRA-1283 In WMSRequestHandler when the request is WMS_isochrone, invoke the solo job blocker before getting the job id and invoking Endpoint_visualize_isochrone internally
KINFRA-1286 Fix segfault in onEndWI
KINFRA-1287 In class RebalanceCache track inserts due to rebalance within a rebalance round
KINFRA-1287 In rebalance rollback, check that rebalanced records were added to the chunk with the requested shard version before pinning the stencil
KINFRA-1287 Use rebalance cache to clamp the chunk scan range during rollback
KINFRA-1290 Fixed case where a rebalance could potentially clear the shard controller's knowledge of active toms
KINFRA-1290 In GaiaSet added logging in rebalancing processing to log table name and tom when IndexedDb throws exceptions
KINFRA-1294 Bail out if attempting to write geometry vector that's not in memory to disk
KINFRA-1294 Use actual vector size when clearing geometry vector
KINFRA-1295 Add DEBUG timer for disk tier reads
KINFRA-1295 Add job cancelled check to cold tier disk reads
KINFRA-1295 Add TierMgr tweaks
KINFRA-1295 Remove call to GaiaSetMapFileSync -> syncMemTofile when calling syncMemToFile on vectors. add tests for rename_table with restart to verify persist
KINFRA-1295 Update DiskTierMgr
KINFRA-1295 Update sync_chunks to keep going if map sync throws
KINFRA-1296 Fix potential race condition during simultaneous access if allocation fails
KINFRA-1296 Update persistent objects to skip eviction to disk if unallocated
KINFRA-1298 Fix hung WIExecutor of work scheduled after exception
KINFRA-1308 idnexeddb::get_objects: dont throw errors when a request is cancelled
KINFRA-1308 idnexeddb::get_objects: move check for cancellation after rollback memory changes
KINFRA-1308 Revert changes in GaiaSet.cpp - committed by mistake
KINFRA-1310 Fix segfault in show_table when table deleted during call
KINFRA-1311 In JobManager make syncing replicated tables to newly added ranks not time out
KINFRA-1317 clean code in GaiaSet::AllocateInHost - load vectors on startup
KINFRA-1321 Fix for mismatching job_id between start and completion logs for WMS, WFS & VTS requests
KINFRA-1321 Fixed for mismatching job_id between start and completion logs for WMS, WFS & VTS requests
KINFRA-1327 On startup: do not try to load a table that is already marked for rebuild
KINFRA-1327 Rebuild from indexedDB: skip bad records instead of cancelling rebuild
KINFRA-1327 StringManager::decode_objects - allow throw/not throw on bad records
KINFRA-1328 Fix vector loading on startup with old file version
KINFRA-1328 Fix version check in vectors
KINFRA-1328 Remove second call to loadMetadata() put by mistake in perv commit
KINFRA-1330 Update show/table to be cancellable
KINFRA-1330 Updated show/table to be cancellable
KINFRA-1332 Fix persist directory discrepancy between tier and global persist path - Don't set vector files out of sync on startup or after reading from cold - Remove deprecated loadGaiaSetData function that no longer works - Enforce single persist tier support in TierManagerStorage - Clarify metadata error and warning messages - Update metadata on restart if the persist path has changed - Fixed indexeddb version upgrade bugs
KINFRA-1332 Limit scope of IndexedDb lock (restores prior 6.2 behavior) - Code cleanup and minor refactoring post review
KINFRA-1337 Fixed race condition in cold storage driver - Remove unnecessary pre/post-processing code
KINFRA-1342 Fix stencil size check in rebalance rollback
KINFRA-1342 Lock stencil during rollback - Remove excessive INFO logs during rebalancing
KINFRA-1354 Don't lock stencils while writing out to persist - Leads to a deadlock if stencil is already write-locked
KINFRA-1355 Return error from /show/system/properties if user is unauthenticated and asking for properties other than require_authentication
KINFRA-1356 Correctly capture the user_id in the job info
KINFRA-1358 Block UDF execution during rebalancing (temporary fix to prevent corruption)
KINFRA-1370 Fix WorkerGRBC releasing join objects in the wrong order
KINFRA-1371 Check for job cancellation during projection reshard
KINFRA-1372 Add logs in SetIndexedDbFile::rename_setdata_id
KINFRA-1378 Add fifo lock on alter table for protection from inserts on multi-head
KINFRA-199 Add tuning option to rebalancing test
KOLAP-1043 Only call the internal_hash_lookup endpoint with non-null hashes
KOLAP-1076 Add extra simplification test. Clear rpn result of simplification in sequenced simplifications to produce valid results
KOLAP-1076 FilterPlanner now postpones simplifications requiring is_nullable function to after outer joins have been done. TypePtr in FilterPlanner is now a copy that is updated with set-nullable info after each step - problem tested and fixed
KOLAP-1108 A bug was exposed during devel. Invalidating the chunk cache when updating pks on insert records
KOLAP-1227 Update tests based on ASOF revised rules
KOLAP-1272 Add comment about mutating the request in other table-creating endpoints
KOLAP-1324 Add tests for restart, insert update and delete into views - handle all different types of views, logical and materialized - don't add restart test to all_test for now, for obvious reasons - also make watermark changes to execute_sql_cancel test - cosmetic changes to other tests
KOLAP-1390 Added m_last_insert_time to GaiaSetBase.h and get_last_modified_time() function. Added m_last_modified_time to fk_constraint_t. When doing get_primary_key_lookup_table for chunk do full refresh if last_full_refresh_time of fk table is less than last_modified_time of pk_set
KOLAP-1401 Filter plans are now by default printed to the log with GAIA_LOG_INFO. Setting FilterPlannerT0 logger to DEBUG reverts to old stdout printing
KOLAP-1409 Between 2d and minimize/maximize filter-plans now handle both both left and right join correctly. Added tests for this to Added m_reportable_join_type and m_reportable_dim_set_indices to FilterPlan to support reporting original values of these when internal sense must be flipped
KOLAP-1409 Presence of between function in join forces what would be a primary key join to be done as an equijoin
KOLAP-1409 Revise test output based new right outer join ASOF fix
KOLAP-1479 Create_union with option mode = merge_views can be refreshed when part of a materialized view
KOLAP-1487 Removed 'INFO' logs from Worker_create_projection_finalize.cpp
KOLAP-1505 Outer predicate-equi-join now correctly sets the set_indices of both sides of the equi-join to cover the full set of set-indices required by the outer-join. Problem previously occurred when some of those indices were cross-joined
KOLAP-1505 Add more tests - sanity and full mode - added to all_test
KOLAP-1505 Outer predicate-equi-join now correctly sets the set_indices of both sides of the equi-join to cover the full set of set-indices required by the outer-join. Problem previously occurred when some of those indices were cross-joined
KOLAP-1505 Set enable_predicate_equi_join default to true in gpudb_conf.json
KOLAP-1509 Allow copying from larger CharN fields to smaller, when truncate_strings is specified
KOLAP-1509 Support more source string types when copying to CharN fields
KOLAP-1513 Enhanced test showing 3 workers gives correct results and same number and size of chunks for base and projected set
KOLAP-1513 Fix test_tiering.test_proj_multichunk_copy failure with 3 ranks
KOLAP-1519 Better error checking that non-unary operators have at least two parameters
KOLAP-1521 Add support for truncate_strings option in insert_records
KOLAP-1523 Change auth_id from STRING to INT
KOLAP-1524 Add support for show <object_name> using SQL
KOLAP-1531 Fixed left primary key joins no longer attempt to update the fk set
KOLAP-1532 Join tables produced by filter-by-list and filter-by-table can now be queried. filter-by-list and filter-by-table no longer work on collections
KOLAP-1532 Update filter_by_list collection check to use the existing EP mechanism
KOLAP-1533 Fix test checking dict vs non-dict column memory usage to account for a high number of toms; if the number of toms is too high the per-tom dict overhead will make the dict column memory usage exceed the non-dict memory usage in this case. Change the test to only check the relative memory usage if the number of toms is at most 8
KOLAP-1535 Add a test for case sensitivity in list and range partitions - during testing I noticed that when a partition is added and the MIN and MAX are the same value, the MAX was treated as exclusive (as usual), but this meant that records were not moved into the new partition. So in this case we needed to modify the generated filter to make MAX inclusive
KOLAP-1535 Make partition values case-sensitive for LIST and RANGE partitions - required separating out handling of keywords from other tokens - in the past we lowered all tokens, now we only lower when we are trying to match keyword tokens
KOLAP-1535 Minor style changes for partition parsing - minor changes from the review, such as using boost::iequals rather than lower casing
KOLAP-1535 Thus, partition tokens are not lowered, matching 'interval' needs to be case-insensitive
KOLAP-1543 Simplified ParserUtils::isa_constant_rpn and ParserUtils::isa_constant_rpn_tree
KOLAP-1543 Updated Simplified ParserUtils::isa_constant_rpn and ParserUtils::isa_constant_rpn_tree
KOLAP-1545 Initialized fp.m_total_num_chunks for BETWEEN_2D_JOIN filter plan type
KOLAP-1552 Add support for SHOW * to return SQL DDL for all objects
KOLAP-1557 Better handling of geohash_encode precision values
KOLAP-1565 Now allows outer joins with bitmask-1d filters included to be done on joins with more than 2 dimensions
KOLAP-1567 Ensure that we create chunks when doing a projection on a join input table - added a test for this and also for the case when a window function is included, both when the partition key matches the sharding and when it doesn't
KOLAP-1569 SpaceEfficientVector no long needs to store the NULL_INDEX. Can now store full range of values for each type (e.g for 8 bit type valid chunk indexes are 0..255). Some error checking added to insure NULL_INDEX is never added to the SpaceEfficientVector (check commented out in low level setAt function)
KOLAP-1572 Fix proj storing mutated request avro
KOLAP-1575 Added m_mutex_ptr to JoinedSetChunk and use to lock GPUManager.cpp:get_all_indices
KOLAP-1576 Update all_test and join_test to include
KOLAP-1576 Worker_one_dimensional_join_filter.cpp only does left join step on last plan in left join and restores the original last_sequence_js bitmask for the foreign key set
KOLAP-1577 Fix error message for ingest of too-long store_only charN values
KOLAP-1586 Fix GaiaSetDictVector::copy_to function to not add offset to destination pointer (offset should only be applied to source dict vector)
KOLAP-1610 Fix groupby-on-wkt sinking for 5+ ranks/toms

Change calls to ghp->get_copied_mem_block to ghp->get_mem_block in GaiaSet::update_records_with_map. There was no need to copy the memblock as the ownership of the geometry pointers was not getting transferred to the updated table

KOLAP-186 Add case expression tests after adding charN_date/time_cast functions

Add charN_ts_cast functions

KOLAP-186 Fix the score calculation bug when checking if charN constant string can be converted to time/date/datetime
KOLAP-186 Modified test that previously expected an error when trying to do an implicit cast between a datetime and a char128 to expect ok
KOLAP-186 Added back charN_datetime_cast functions
KOLAP-316 Add tracers for group-by sink, bulk insert, JoinedSet construction add avro_json info to top-level worker tracers
KOLAP-316 Move all calls to appendArgsToThreadLocalTracer within the initLock. Temporarily disable appendArgsToThreadLocalTracer until better tested
KOLAP-316 Only append request json to tracer in first tom to decode message - make thread_local m_tracer a shared pointer
KOLAP-316 Update worker_create_join_table to properly scope InitLock
KOLAP-378 Add decorator to to skip the test if the sql-server is not enabled
KOLAP-378 Added tests - all passed without modifying gpudb
KOLAP-453 GroupBy supports grouping on WKT columns. Internally we group on hash(<wkt_column>) and we keep the hashes and the wkts around. Once we are done all processing and are ready to either return data to the user or write to a result table we replace the hash value with the WKT value
KUI-866 Update histogram layout
KUI-889 UI Import doesn't handle extra copies of files ("x (1).*" or "x (copy).*")
KUI-898 Skip Error in Advanced CSV Import should drop extra columns in records
KUI-899 Handle null strings in all column types for Advanced CSV Import
KUI-902 Add write failed rows to CSV file feature to advanced CSV import tool
KUI-903 Allow server filesystem path to file for advanced CSV import


None currently


Should you have any questions or concerns, please visit our support page, official documentation page or email us at

Was this article helpful?
0 out of 0 found this helpful
Have more questions? Submit a request


Article is closed for comments.