Write handling of WORM object

All data are stored as regular(hierarchical) objects initially, because it is very challenging to handle write in flattened format (MVCC, ilog, DTX…). In addition, even for WORM object, application can still submit writes in multiple RPCs, e.g., appends, in this case, it is complex to add new keys and values to flattened buffer. To avoid these complexities, this design does not change the write handler, which creates trees to index keys and values.

Flattening service

DAOS can provide API to allow user to indicate immutability of WORM objects:

• Application/middleware calls flatten(obj).

• Set container property and convert all objects within a container to WORM.

When any of these cases happened, hierarchical object can be converted to WORM object by a new service which is called flattening service. The flattening service is similar to aggregation service, it can be activated periodically or on demand. This service traverses index trees of WORM object, appends keys and values to a contiguous buffer (if the object is small enough), then writes the buffer to DT-blob and releases the hierarchical object in MD-blob (and DRAM).

As flattened object cannot support snapshot, it is required for user to destroy existed container snapshot before calling flatten for container or object. When setting container property for flattening, DAOS internally associate an epoch with the flattening service (flatten epoch), any further modification with higher epoch will be rejected.

In order to avoid store unnecessary metadata in flattened buffer, the flattening service should run after aggregation, for example, after aggregation service merged extents generated by “append”. The flatten service can wait until the local VOS container’s HAE (Highest Aggregated Epoch) exceed the flatten epoch. At that time both EC aggregation and VOS aggregation finished before the flatten epoch and any modification higher than flatten epoch is forbidden, DAOS can shutdown the aggregation service and active flatten service.

The flattening service only serializes objects with small number of keys and values, because flattened format only supports leaner search which is inefficient if there are too many keys and values. It means that even for WORM object, if it has a lot of keys and values then it stays in hierarchical format to support efficient read.

Because DAOS only flattens small objects, so it should be able to store multiple flattened objects in the same SSD extent to avoid fragmentation and reduce write amplification.

WORM object eviction

After flattening service serialized WORM objects and wrote them to DT-blob, these objects can be evicted from DRAM. DAOS stores the BIO address of evicted object in the object index tree, which always stay in DRAM and MD-blob.

Read handling of flattened WORM object

DAOS will have a new read handler for flattened object, it can locate the requested key and value from the flattened object format by leaner search. Because flattened object is stored in DT-blob and it can be evicted from DRAM, if storage engine receives a read request against an evicted object, it can bring the entire object back to DRAM by one SSD read.