Architecture:

SMD changes:

New smd_type will be introduced:

enum smd_type {
        SMD_TYPE_DATA = 0, /* for data blob */
        SMD_TYPE_META,     /* metatadata blob */
        SMD_TYPE_WAL,      /* wal blob */
        SMD_TYPE_MAX = 3,
};

Now one target might be bonded with more than one device, besides original “target“ table to map data

devices to target, two new tables will be introduced to map vol/wal device and target information:

const char TABLE_TGTS[SMD_TYPE_MAX] = {
        "target", / compatible with old version */
        "meta_target",
        "wal_target",
};

And two new pools table are introduced to map pool target vol/wal blob information.

const char *TABLE_POOLS[SMD_TYPE_MAX] = {
        "pool",
        "meta_pool",
        "wal_pool",
};

SMD public APIs will be extended:

int smd_pool_add_tgt(uuid_t pool_id, uint32_t tgt_id, uint64_t blob_id,

enum smd_type smd_type, uint64_t blob_sz);

int smd_pool_get_blob(uuid_t pool_id, uint32_t tgt_id, enum smd_type smd_type, uint64_t *blob_id);

int smd_dev_add_tgt(uuid_t dev_id, uint32_t tgt_id, enum smd_type smd_type);

int smd_dev_del_tgt(uuid_t dev_id, uint32_t tgt_id, enum smd_type smd_type);

int smd_dev_get_by_tgt(uint32_t tgt_id, enum smd_type smd_type, struct smd_dev_info **dev_info);

struct sm_pool_info need be changed to return more information for smd_pool_list()

struct smd_pool_info {
d_list_t spi_link;
uuid_t spi_id;
uint64_t spi_blobs_sz[SMD_TYPE_MAX];
uint32_t spi_tgt_cnt;
int *spi_tgts[SMD_TYPE_MAX];
uint64_t *spi_blobs[SMD_TYPE_MAX];
};

SMD interoperability

When DAOS was upgraded to newer version(PMEM for metadata), SMD should be able to read existed target and pool table.

BIO changes:

some of members from struct bio_xs_context will be moved to struct bio_blobstore

/*
 * SPDK blobstore isn't thread safe and there can be only one SPDK
 * blobstore for certain NVMe device.
 */
struct bio_blobstore {
        .....
        /* inflight blob read/write */
        unsigned int             bb_blob_rw;
        /* spdk io channel */
        struct spdk_io_channel  *bb_io_channel;
        /* all I/O contexts for this blobstore */
        d_list_t                 bb_io_ctxts;
        ......

};

struct bio_xs_context will be changed and a blobstore pointer will be added in struct bio_io_context:

/* Per-xstream NVMe context /
struct bio_xs_context {
            int                     bxc_tgt_id;
            struct spdk_thread     *bxc_thread;
            struct bio_blobstore   *bxc_blobstores[SMD_TYPE_MAX];
            struct bio_dma_buffer   bxc_dma_buf;
            unsigned int            bxc_ready:1,           /* xstream setup finished */
                                    bxc_self_polling:1;   /* for standalone VOS */
};

/* Per VOS instance I/O context /
struct bio_io_context {
            ....
            struct bio_blobstore   *bic_blobstore;
            ....
};

vos/vos_internal.h:

struct vos_pool {
            ….
            struct bio_meta_instance *vp_io_metai;
   };

Public API:

New BIO APIs will be added (refer to details WAL Detailed Design )

Other changes:

XS0 will need a NVME context to access NVME device(for RDB, meta-data service), a special target id -2 will be used to differ it from main XS NVME context.

Sysdb changes

todo.