Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

In the current VOS implementation, each metadata of an object consumes 104 bytes (vos_obj_df), which includes object ID, index tree root, incarnation log header, etc. After flattening, most of these information are not necessary anymore: keeping the object ID and BIO address of the flattened buffer are is sufficient, these information may metadata roughly consume 48 bytes based on the current estimate. During I/O processing, VOS can check look up the object index tree and see check if the object is stored in flattened or hierarchical format, it has to load the object shall load its keys and values from DT-blob back to DRAM in the former case.

...

As shown in the diagram above, hierarchical objects and all the tree indexes remain in MD-blob and DRAM, whereas flattened objects only keeps a small header little metadata in DRAM and saves all its keys and values in DT-blob. In this model, assuming each object consumes 48 bytes for metadata, VOS requires uses a few extra more bytes to index object and , blob allocator also has its own overhead, so the actual overall bytes per object can be about around 100 bytes. Based on the previous assumption, a storage server has 900GB DRAM for metadata, then a server can store up to 9 billion 4K files.