Architecture:
SMD changes:
New smd_type will be introduced:
enum smd_type { SMD_TYPE_DATA = 0, /* for data blob */ SMD_TYPE_META, /* metatadata blob */ SMD_TYPE_WAL, /* wal blob */ SMD_TYPE_MAX = 3, };
Now one target might be bonded with more than one device, besides original “target“ table to map data
devices to target, two new tables will be introduced to map vol/wal device and target information:
const char TABLE_TGTS[SMD_TYPE_MAX] = { "target", / compatible with old version */ "meta_target", "wal_target", };
And two new pools table are introduced to map pool target vol/wal blob information.
const char *TABLE_POOLS[SMD_TYPE_MAX] = { "pool", "meta_pool", "wal_pool", };
SMD public APIs will be extended:
int smd_pool_add_tgt(uuid_t pool_id, uint32_t tgt_id, uint64_t blob_id,
enum smd_type smd_type, uint64_t blob_sz);
int smd_pool_get_blob(uuid_t pool_id, uint32_t tgt_id, enum smd_type smd_type, uint64_t *blob_id);
int smd_dev_add_tgt(uuid_t dev_id, uint32_t tgt_id, enum smd_type smd_type);
int smd_dev_del_tgt(uuid_t dev_id, uint32_t tgt_id, enum smd_type smd_type);
int smd_dev_get_by_tgt(uint32_t tgt_id, enum smd_type smd_type, struct smd_dev_info **dev_info);
struct sm_pool_info need be changed to return more information for smd_pool_list()
struct smd_pool_info {
d_list_t spi_link;
uuid_t spi_id;
uint64_t spi_blobs_sz[SMD_TYPE_MAX];
uint32_t spi_tgt_cnt;
int *spi_tgts[SMD_TYPE_MAX];
uint64_t *spi_blobs[SMD_TYPE_MAX];
};
SMD interoperability
When DAOS was upgraded to newer version(PMEM for metadata), SMD should be able to read existed target and pool table.
BIO changes:
some of members from struct bio_xs_context will be moved to struct bio_blobstore
/* * SPDK blobstore isn't thread safe and there can be only one SPDK * blobstore for certain NVMe device. */ struct bio_blobstore { ..... /* inflight blob read/write */ unsigned int bb_blob_rw; /* spdk io channel */ struct spdk_io_channel *bb_io_channel; /* all I/O contexts for this blobstore */ d_list_t bb_io_ctxts; ...... };
struct bio_xs_context will be changed and a blobstore pointer will be added in struct bio_io_context:
/* Per-xstream NVMe context / struct bio_xs_context { int bxc_tgt_id; struct spdk_thread *bxc_thread; struct bio_blobstore *bxc_blobstores[SMD_TYPE_MAX]; struct bio_dma_buffer bxc_dma_buf; unsigned int bxc_ready:1, /* xstream setup finished */ bxc_self_polling:1; /* for standalone VOS */ }; /* Per VOS instance I/O context / struct bio_io_context { .... struct bio_blobstore *bic_blobstore; .... };
vos/vos_internal.h:
struct vos_pool { …. struct bio_meta_instance *vp_io_metai; };
Public API:
New BIO APIs will be added (refer to details WAL Detailed Design )
SYSDB changes
To make SYSDB independent of SMD, following new API will be added:
int bio_sys_create(struct bio_xs_context *xs_ctxt, uint64_t blob_sz); int bio_sys_open(struct bio_io_context *ctxt, bool async) int bio_sys_close(struct bio_io_context *ctxt, bool async)
struct bio_xs_context
need be extended:
bxc_tgt_id will be assigned as -2 if this is for system XS, To duplicate SMD on all Metadata SSDs in the future, number of bio_blobstores
will be different between main XS(SMD_TYPE_MAX=3 for now) and system XS(number of metadata SSDs).
struct bio_xs_context { int bxc_tgt_id; struct spdk_thread *bxc_thread; struct bio_blobstore **bxc_blobstores; unsigned int bxc_blobstore_num; struct bio_dma_buffer bxc_dma_buf; unsigned int bxc_ready:1, /* xstream setup finished */ bxc_self_polling:1; /* for standalone VOS */ };
SMD Blobstores will be init during init_bio_bdevs() which will scan spdk dev list and attach all meta device to system context.