Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Version History

« Previous Version 2 Next »

All data are stored as regular(hierarchical) objects initially, because it is very challenging to handle write in flattened format (MVCC, ilog, DTX…). In addition, even for WORM object, application can still submit writes in multiple RPCs, e.g., appends, in this case, it is complex to add new keys and values to flattened buffer. To avoid these complexities, this project will not change the write handling, DAOS still creates trees to index keys and values for write RPC.

DAOS can provide API to allow user to indicate write completion of WORM objects:

  • Application/middleware calls write(obj) with “DONE” flag.

  • Application/middleware calls flatten(obj).

  • Set container property and convert all objects within a container to WORM.

When any of these cases happened, hierarchical object can be converted to WORM object by a new flattening service. The flattening service is similar to aggregation service, it is activated periodically and scans objects in the background, it can traverse object trees and append keys and values of the object to a contiguous buffer (if the object is small enough), then writes the buffer to DT-blob and release the original hierarchical object.

In order to avoid store unnecessary metadata in flattened buffer, the flattening service should run after aggregation, for example, after aggregation service merged extents generated by “append”. However, in this case, the index changes made by aggregation service is not meaningful anymore because they will be freed by flattening service later. Therefore, the ideal case is flattening service can also aggregate adjustment extents in EVTree. However, this means that functionalities of aggregation and flattening services are overlapped, so this can be considered as an improvement in the future.

WORM object

Mostly for small object: linear search is inefficient for large object

Multiple WORM objects can be stored in the same SSD extent

  • No labels