Regular object of DAOS is stored in hierarchical format, it can support multi-version and arbitrary overwrites, DAOS has to keep tree indexes for these objects to support high efficient key and value insert and lookup. It means that DAOS has to do multi-allocation for even a very small object every I/O (object, tree node, key, tree node, value…), giving if these allocations are very small enough, most of them will stay in the MD-blob forever: it is impossible for DAOS to migrate all small pieces to DT-blob because write amplification is unacceptable. It is worth noting that everything in MD-blob will be pinned in DRAM.

WORM object

Because data model of DAOS can support snapshot, overwritesoverwrite, distributed transaction, MVCC…, so it is hard to change the metadata format. However, for workloads like AI/ML, the dataset for ingestion will not not modified by AI/ML, It means that keeping indexes and logs for objects is not always necessary, especially for objects are relatively small. The terminology to describe this kind of AI/ML datasets is Write-Once-Read-Many (WORM).

...