Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

In the current VOS implementation, each metadata of an object consumes 104 bytes (vos_obj_df), which includes object ID, index tree root, incarnation log header, etc. After flattening, most of these information are not necessary anymore: keeping the object ID and BIO address of the flattened buffer are sufficient, these information roughly consume 48 bytes based on the current estimate. During I/O processing, VOS can look up the object index tree and check if the object is stored in flattened or hierarchical format, it shall load the object its keys and values from DT-blob back to DRAM in the former case.

...

As shown in the diagram above, hierarchical objects and all the tree indexes remain in MD-blob and DRAM, whereas flattened objects only keeps a small header in DRAM and saves all keys and values in DT-blob. In this model, assuming each object consumes 48 bytes, VOS uses a few more bytes to index object and allocator also has its own overhead, overall bytes per object can be about around 100 bytes. Based on the previous assumption, a storage server has 800GB DRAM for metadata, then a server can store up to 8 billion 4K files.