rank_fallback_allocator alerts

Incident Synopsis:  

rank_fallback_allocator alerts



Warnings are shown in GAdmin Alerts and gpudb.log as follows:


rank_fallback_allocator:avg_failure_size: 1240804420, failure_count: 28, fallback_type: pinned_to_host, max_failure_size: 1292013184, min_failure_size: 1149354176, rank: 19


Problem Detail:

char256 columns with the default chunk size (8,000,000) are failing to be allocated space in VRAM as they take 2GB. 


Environment :

Kinetica On-prem


Cause :

There is not enough VRAM to accommodate the level of concurrency currently set with the chunks being the size they are.


Solution/Answer :

  • Dictionary encode your char256 and char128 columns. 
    • Table Stats on the table page in GAdmin will show you which columns have low to medium cardinality and should be dict encoded.
    • note: don't do this with primary key or shard key columns, and the encoding may take a while depending on table size. 
  • Reduction of chunk sizes when using char128 and larger columns.
    • You'll have to create a new table with a custom chunk size and then insert into​ that table from the original table. 
  • Resize char256 columns to char64 and char128​.
    • Recommend this as a last resort. 


Special Considerations :

Was this article helpful?
0 out of 0 found this helpful
Have more questions? Submit a request


Article is closed for comments.