Regular object of DAOS is stored in hierarchical format, it can support multi-version and arbitrary overwrites, DAOS has to keep tree indexes for these objects to support high efficient key and value insert and lookup. It means that DAOS has to do multi-allocation for even a very small object (object, tree node, key, tree node, value…), giving these allocations are very small, most of them will stay in the MD-blob forever: there is no way for DAOS to migrate them to DT-blob. It is worth noting that everything in MD-blob will be pinned in DRAM forever.
This is hard to change because data model of DAOS can support snapshot, overwrites, distributed transaction, MVCC. However, for workloads like AI/ML, the dataset will never be changed again after data ingestion. It means that keeping indexes and logs for objects is not always necessary, especially for those objects are relatively small. The terminology to describe this kind of AI/ML datasets is Write-Once-Read-Many (WORM).
NB: DAOS can still keep indexes for large object even it is WORM, because DAOS should provide fast search and read.