Using HybridCache

[sriramsrao]

Hello...

I modified the simple server example code to use the HybridCache. I set it up for being a Block cache. My test code inserts 5 large objects (256MB in size apiece), and then tries to retrieve them back. I have two questions:

1. The cache pool is configured to 1G in size. While the inserts succeed, retrieval fails. After the 5 inserts, when I start reading the keys back, only the last one is in cache. What is the relationship between the cache pool size and the HybridCache?
2. What triggers data migration from RAM to SSD? I don't see any writes being sent to the SSD.

Can you please help me?

Here is my updated initialization:

void initializeNVMCache(uint64_t cacheSize_gb, const std::string& navy_file_name) {
    CacheConfig lruConfig;
    NVMCacheConfig nvmConfig;
    // assuming caching 20 million items
    AccessConfig accessConfig({25 /* bucket power */, 10 /* lock power */});

    nvmConfig.navyConfig.setBlockSize(4096);
    
   //readerThreads, writerThreads, maxNumReads, maxNumWrites, stackSizeKB);
    nvmConfig.navyConfig.setReaderAndWriterThreads(4, 4, 64, 32, 64); 

    nvmConfig.navyConfig.setMaxParcelMemoryMB(5 * 1024); // 5G
    nvmConfig.navyConfig.setDeviceMaxWriteSize(1 << 20);
    nvmConfig.navyConfig.setSimpleFile(navy_file_name,
                                   cacheSize_gb * (1 << 30),  /*fileSize*/
                                   false /*truncateFile*/);
    nvmConfig.navyConfig.blockCache().setRegionSize(32 << 20); 
    lruConfig.enableNvmCache(nvmConfig);
   
    lruConfig.setAccessConfig(accessConfig);
    lruConfig.configureChainedItems(accessConfig);
    lruConfig.validate();
    gCache_ = std::make_unique<Cache>(lruConfig);
    XLOG(INFO) << "Using the following navy config"
          << folly::toPrettyJson(
                       folly::toDynamic(nvmConfig.navyConfig.serialize()));
    std::cout << "RAM cache size: " << gCache_->getCacheMemoryStats().ramCacheSize << std::endl;
    defaultPool_ =
        gCache_->addPool("default", gCache_->getCacheMemoryStats().ramCacheSize);
  }

The cache config:

Using the following navy config{
   "navyConfig::QDepth": "16",
   "navyConfig::admissionPolicy": "random",
   "navyConfig::admissionProbBaseSize": "0",
   "navyConfig::admissionProbFactorLowerBound": "0",
   "navyConfig::admissionProbFactorUpperBound": "0",
   "navyConfig::admissionProbability": "0.9",
   "navyConfig::admissionSuffixLen": "0",
   "navyConfig::admissionWriteRate": "0",
   "navyConfig::bigHashBucketBfSize": "8",
   "navyConfig::bigHashBucketSize": "4096",
   "navyConfig::bigHashSizePct": "0",
   "navyConfig::bigHashSmallItemMaxSize": "0",
   "navyConfig::blockCacheCleanRegionThreads": "1",
   "navyConfig::blockCacheCleanRegions": "1",
   "navyConfig::blockCacheDataChecksum": "true",
   "navyConfig::blockCacheLru": "true",
   "navyConfig::blockCacheNumInMemBuffers": "2",
   "navyConfig::blockCacheRegionSize": "33554432",
   "navyConfig::blockCacheReinsertionHitsThreshold": "0",
   "navyConfig::blockCacheReinsertionPctThreshold": "0",
   "navyConfig::blockCacheSegmentedFifoSegmentRatio": "",
   "navyConfig::blockSize": "4096",
   "navyConfig::deviceMaxWriteSize": "1048576",
   "navyConfig::deviceMetadataSize": "0",
   "navyConfig::enableFDP": "0",
   "navyConfig::fileName": "/nvme/Testing/cache-file",
   "navyConfig::fileSize": "53687091200",
   "navyConfig::ioEngine": "io_uring",
   "navyConfig::maxConcurrentInserts": "1000000",
   "navyConfig::maxNumReads": "64",
   "navyConfig::maxNumWrites": "32",
   "navyConfig::maxParcelMemoryMB": "5120",
   "navyConfig::maxWriteRate": "0",
   "navyConfig::navyReqOrderingShards": "20",
   "navyConfig::raidPaths": "",
   "navyConfig::readerThreads": "4",
   "navyConfig::stackSize": "65536",
   "navyConfig::truncateFile": "false",
   "navyConfig::writerThreads": "4"
 }

[sriramsrao]

More output:

5 inserts:
Successfully stored item with key: test_key_0
Successfully stored item with key: test_key_1
Successfully stored item with key: test_key_2
Successfully stored item with key: test_key_3
Successfully stored item with key: test_key_4

5 retrieves that follow:

For key: test_key_0 handle is: nullptr
Unable to find key: test_key_0
For key: test_key_1 handle is: nullptr
Unable to find key: test_key_1
For key: test_key_2 handle is: nullptr
Unable to find key: test_key_2
For key: test_key_3 handle is: nullptr
Unable to find key: test_key_3
For key: test_key_4 handle is: item: memory=0x77c097c00000:raw-ref=38535169:size=0:key=test_key_4:hex-key=746573745f6b65795f34:isInMMContainer=true:isAccessible=true:isMarkedForEviction=false:isMoving=false:references=1:ctime=1736835299:expTime=0:updateTime=1736835299:isNvmClean=false:isNvmEvicted=false:hasChainedItem=true

[pbhandar2]

Hello @sriramsrao

Could you clarify more what you are trying to do? Do you want to only use NVM cache without any DRAM cache? Are you trying to split NVM cache into pools? Which pool are you inserting to?

To answer your questions,

What is the relationship between the cache pool size and the HybridCache? There is no relationship. The pools are a RAM cache concept and does not apply to NVM cache.

What triggers data migration from RAM to SSD? Evictions from RAM are admitted to SSD. You can configure the cache to be selective about what is admitted to SSD.

[sriramsrao]

Hello @pbhandar2

I want to use the NVM with DRAM cache. I only have a default pool and inserting into it. The default pool is sized to 1G. What I am expecting is that as I keep inserting into the cache, data starts in RAM and then migrates over to SSD. What is the configuration I need to have for this case?

For instance, I'd like to have 50G NVM cache. What should I set the size of the DRAM cache? When/how do I trigger migration? Lastly, what should size of the default pool be? (In my config above, I wonder if the pool is exhausted and that is causing cache entries to be evicted).

For now, it'd be great if the cache to put everything into the SSD. Just makes testing easier.

Thanks.

Sriram

[sriramsrao]

Hello @pbhandar2 ,

I pushed on this a bit more. Here is what I am doing:

1. Insert keys/values into the Cache, where the key is a string and the value is 1MB (of binary blob).
2. Insert keys/values into the Cache, where the key is a string and the value is a 256MB (of binary blob). I am inserting this using chained allocation.

My config is below.

When I try to retrieve the key/values that I previously inserted, the retrieve fails: the lookup using the find() API fails, and I am unable to retrieve the value. For example, I inserted 3 256MB blobs (with keys: test_key_0, test_key_1, and test_key_2; the insert is successful. Then, when I try to retrieve the inserted blobs, the first 2 keys fail (i.e., test_key_0 and test_key_1 result in cache misses); for the 3rd key (test_key_2), I am able to retrieve the blob.

I tried to follow the code using gdb and found that items were being evicted from RAM and being pushed to NVM, but nothing seemed to be written to the SSD. This likely explains why the subsequent reads fail.

Any pointers would be greatly appreciated.

AccessConfig accessConfig({25 /* bucket power */, 10 /* lock power */});

    nvmConfig.navyConfig.setBlockSize(4096);
    nvmConfig.navyConfig.setSimpleFile(navy_file_name,
                                   cacheSize_gb * (1 << 30),  /*fileSize*/
                                   false /*truncateFile*/);
    nvmConfig.navyConfig.blockCache().setRegionSize(32 * 1024 * 1024);
    // readerThreads, writerThreads, maxNumReads, maxNumWrites, stackSizeKB)
    nvmConfig.navyConfig.setReaderAndWriterThreads(4, 4, 64, 32, 64); 

    nvmConfig.navyConfig.setMaxParcelMemoryMB(5 * 1024); 
    nvmConfig.navyConfig.setDeviceMaxWriteSize(1 << 20);
    
    nvmConfig.navyConfig.enableRandomAdmPolicy().setAdmProbability(1.0);
    lruConfig.enableNvmCache(nvmConfig);

    lruConfig
        .setCacheSize(1<< 30)
        .setCacheName("Test NVM Cache")
        .setAccessConfig(accessConfig)
        .configureChainedItems(accessConfig)
        .validate();
    gCache_ = std::make_unique<Cache>(lruConfig);

[pbhandar2]

So when items are being evicted from DRAM to NVM, they are not being stored in NVM. You could check a few things.

• You can verify NVM inserts are succeeded. If they are succeeding, they are being written somewhere. You can print the NVM stats to verify. It might be interesting to see all the stats.
• Is a file created in the file path of NVM?

[sriramsrao]

Yes, a file is created in the NVM.

A bit of progress. I am able to read/write from the SSD. Here are some counters:

 # of NVM puts: 5205
 # of NVM put errors: 0
 # of items in cache: 10095
 # of cache evictions: 205
 # of NVM evictions: 0
 # of NVM gets: 5127
# of NVM get miss: 127
 # of NVM rejects by AP: 0
 NVM (MB) bytes written: 22848
 NVM (MB) bytes read: 20011.5
NVM device write errors: 0
NVM device read errors: 0

The issue is:

1. When the size of the object being inserted is 1MB or 2MB, the writes/reads to SSD work fine.
2. When the size of the object being inserted is 128MB, the writes/reads to SSD fails. I am using a chainedItem to store these large objects (based on the doc). Is there an issue with writing out chained objects?