BIO && SMD Detailed Design

Architecture:

SMD changes:

 

New smd_type will be introduced:

1 2 3 4 5 6 enum smd_type { SMD_TYPE_DATA = 0, /* for data blob */ SMD_TYPE_META, /* metatadata blob */ SMD_TYPE_WAL, /* wal blob */ SMD_TYPE_MAX = 3, };

Now one target might be bonded with more than one device, besides original “target“ table to map data

devices to target, two new tables will be introduced to map vol/wal device and target information:

1 2 3 4 5 const char TABLE_TGTS[SMD_TYPE_MAX] = { "target", / compatible with old version */ "meta_target", "wal_target", };

And two new pools table are introduced to map pool target vol/wal blob information.

1 2 3 4 5 const char *TABLE_POOLS[SMD_TYPE_MAX] = { "pool", "meta_pool", "wal_pool", };

SMD public APIs will be extended:

int smd_pool_add_tgt(uuid_t pool_id, uint32_t tgt_id, uint64_t blob_id,

enum smd_type smd_type, uint64_t blob_sz);

int smd_pool_get_blob(uuid_t pool_id, uint32_t tgt_id, enum smd_type smd_type, uint64_t *blob_id);

 

int smd_dev_add_tgt(uuid_t dev_id, uint32_t tgt_id, enum smd_type smd_type);

int smd_dev_del_tgt(uuid_t dev_id, uint32_t tgt_id, enum smd_type smd_type);

int smd_dev_get_by_tgt(uint32_t tgt_id, enum smd_type smd_type, struct smd_dev_info **dev_info);

 

struct sm_pool_info need be changed to return more information for smd_pool_list()

struct smd_pool_info {
d_list_t spi_link;
uuid_t spi_id;
uint64_t spi_blobs_sz[SMD_TYPE_MAX];
uint32_t spi_tgt_cnt;
int *spi_tgts[SMD_TYPE_MAX];
uint64_t *spi_blobs[SMD_TYPE_MAX];
};

SMD interoperability

When DAOS was upgraded to newer version(PMEM for metadata), SMD should be able to read existed target and pool table.

BIO changes:

some of members from struct bio_xs_context will be moved to struct bio_blobstore

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 /* * SPDK blobstore isn't thread safe and there can be only one SPDK * blobstore for certain NVMe device. */ struct bio_blobstore { ..... /* inflight blob read/write */ unsigned int bb_blob_rw; /* spdk io channel */ struct spdk_io_channel *bb_io_channel; /* all I/O contexts for this blobstore */ d_list_t bb_io_ctxts; ...... };

struct bio_xs_context will be changed and a blobstore pointer will be added in struct bio_io_context:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 /* Per-xstream NVMe context / struct bio_xs_context { int bxc_tgt_id; struct spdk_thread *bxc_thread; struct bio_blobstore *bxc_blobstores[SMD_TYPE_MAX]; struct bio_dma_buffer bxc_dma_buf; unsigned int bxc_ready:1, /* xstream setup finished */ bxc_self_polling:1; /* for standalone VOS */ }; /* Per VOS instance I/O context / struct bio_io_context { .... struct bio_blobstore *bic_blobstore; .... };

vos/vos_internal.h:

1 2 3 4 struct vos_pool { …. struct bio_meta_instance *vp_io_metai; };

Public API:

New BIO APIs will be added (refer to details https://daosio.atlassian.net/wiki/spaces/DC/pages/11215339529 )

SYSDB changes

  1. To make SYSDB independent of SMD, bio_meta_xxx API above will be extended for SYSDB blob operations.

2. SMD might be duplicated on all Metadata SSDs in the future, for now we only scan and use first metadata device found for SYSDB device, SYSDB blob will be stored

as “root“ blob, spdk_bs_set_super/spdk_bs_get_super helper could be used to set/get it.

3. vos_db_init() will be moved from first main XS into system XS init, bxc_tgt_id will be assigned as -2 if this is for system XS.

4. blob cluster size shall be shrunk for META/WAL blob.