DAOS Tools

Proposal for DAOS tools consolidation.

Syntax: dmg  [resource] [action] [args]
              daos [resource] [action] [args]

Proposal: High Level Characteristics

Proposal: Characteristics: Resource Names Summary

Specifying DAOS system name (formerly known as server group):

• --sys-name=SYSNAME ; or --sys=SYSNAME (example: --sys=daos_server)

Specifying storage server ranks (e.g., pool create/add-storage/del-storage, and system drain/reintegrate/kill/exclude)

• --ranks=SRVRANKLIST (example: --ranks=0,1,2)

Specifying added or removed pool service replica ranks (for pool add-svc/del-svc):

• --ranks=SRVRANKLIST (example: --ranks=0,1,2)

Specifying number of pool service replicas (for pool create):

• --nsvc=NUM

Specifying pool service replica ranks (legacy - currently required but eventually will not be needed) specify replica ranks:

• --svc=SRVRANKLIST (example: --svc=1,2,3)

Specifying a fault domain / entire rack of servers (e.g., for pool create/add-storage/del-storage and system drain/reintegrate/kill/exclude)

• --fdomains=FDRANKLIST (often a single item, but keeping a list for flexibility)

• --fd=FDRANKLIST (shorter option name for convenience)

Specifying targets (e.g., for system drain/reintegrate):

• List of Rank:Target pairs (--targets=SRVRANK:TGTRANK LIST ; or --tgt=SRVRANK:TGTRANK LIST)

  • server 0 targets 0 and 1 (0:0,0:1)

  • Server 1 targets 2 and 4 (1:2,1:4)

  • Server 2 targets 0 and 1 (2:0,2:1)

  • (whole list all together) --tgt=0:0,0:1,1:2,1:4,2:0,2:1

Specifying container snapshots

• named snapshot: --snap=NAME

• snapshot identified by a single epoch number: --epc=NUM

• snapshots that site within a specified range of epoch numbers: --epcrange=M-N

Proposal: Highlighted Operations and Tool Command Lines

Proposal: Highlighted Operations: System

• List all pools in a DAOS system

  •  dmg system list-pools

Proposal: Highlighted Operations: Pool

• Create (dmg pool create)

  • by server rank list

    • Specify only SCM storage

      •  dmg pool create --sys=SYSNAME --uid=UID --gid=GID --mode=MODE --nsvc=NREP --ranks=SRVRANKLIST --scm-size=SIZE

    • Specify SCM + NVMe storage

      • dmg pool create --sys=SYSNAME --uid=UID --gid=GID --mode=MODE --nsvc=NREP --ranks=SRVRANKLIST --scm-size=SIZE--nvme-size=SIZE

  • by fault domain (rack) rank list (using --fd shorthand for --fdomains)

    •  dmg pool create --sys=SYSNAME --uid=UID --gid=GID --mode=MODE --nsvc=NREP --fd=FDRANKLIST --scm-size=SIZE--nvme-size=SIZE

• Add pool service replicas (dmg pool add-svc)

  • Usage: dmg pool add-svc --pool=UUID --sys=SYSNAME --svc=SRVRANKLIST  --ranks=MORESRVRANKSADDLIST

• Remove pool replicas (dmg pool del-svc)

  • Usage: dmg pool del-svc --pool=UUID --sys=SYSNAME --svc=SRVRANKLIST  --ranks=OLDSRVRANKSDELLIST

• Destroy pool in a DAOS system (dmg pool destroy)

  • Usage: dmg pool destroy --pool=UUID --sys=SYSNAME --svc=SRVRANKLIST [--force]

• List all containers in a pool (dmg pool list-containers ; or shorter command equivalent dmg pool list-cont)

  • Usage: daos pool list-containers -pool=UUID --sys=SYSNAME --svc=SRVRANKLIST 

  • Usage: daos pool list-cont -pool=UUID --sys=SYSNAME --svc=SRVRANKLIST 

• Add storage (dmg pool add-storage aka extend)

  • Rack (all servers in the rack, and all targets in all of the servers; using --fd shorthand for --fdomains)

    • dmg pool add-storage --pool=UUID --sys=SYSNAME --svc=SRVRANKLIST  --fd=FDRANKLIST

  • Servers (all targets on the servers)

    • dmg pool add-storage --pool=UUID --sys=SYSNAME --svc=SRVRANKLIST  --ranks=SRVRANKLIST

• Remove storage (dmg pool del-storage aka exclude)

  • Rack (all servers in the rack, and all targets in all of the servers; using --fd shorthand for --fdomains)

    •  dmg pool del-storage --pool=UUID --sys=SYSNAME --svc=SRVRANKLIST  --fd=FDRANKLIST

  • Servers (all targets on the servers)

    • dmg pool del-storage --pool=UUID --sys=SYSNAME --svc=SRVRANKLIST --ranks=SRVRANKLIST

Proposal: Highlighted Operations: Container

Notes:

• command for all container operations is daos container (shown in the examples below). However, as a convenience, a shorter command equivalent may be used daos cont

• daos container list-objects command (shown in the examples below) has a shorter command equivalent daos container list-obj.

• resource for object commands is daos object. However, as a convenience, a shorter equivalent may be used daos obj

Container Create - by UUID and/or unified namespace path

• Create a container in a pool (daos container create)

  • User-specified container UUID

    • Usage: daos container create -pool=UUID --sys=SYSNAME --svc=SRVRANKLIST --cont=UUID

  • No container UUID specified (implementation generates a random UUID as a convenience)

    • Usage: daos container create --pool=UUID --sys=SYSNAME --svc=SRVRANKLIST

  • User-specified container UUID and user-specified unified namespace path to link the container to

    • Usage: daos container create --pool=UUID --sys=SYSNAME --svc=SRVRANKLIST --cont=UUID --path=/path/to/create_and_link --type=POSIX|HDF5 --oclass=tiny|small|large|R2|R2S|repl_max --chunk_size=BYTES

      • path is a directory for type=POSIX, and is a file for type=HDF5

      • oclass is DAOS object class

      • chunk_size is the chunk_size in bytes to use with files created in the container.

  • No container UUID specified, and user-specified unified namespace path to link the container to (implementation will generate a random UUID)

    • Usage:: daos container create --pool=UUID --sys=SYSNAME --svc=SRVRANKLIST -path=/path/to/create_and_link --type=POSIX|HDF5 --oclass=tiny|small|large|R2|R2S|repl_max --chunk_size=BYTES

Container "Lookup" (All Other Commands) - by UUID or unified namespace path

There are 2 variants of the commands: 1) where the user provides the pool and container UUIDs ; and a 2) where the user provides only the unified namespace path to which the container is linked. In the second format, the implementation will resolve the pool and container UUIDs by getting extended filesystem attributes of the specified entity in the path (i.e., the user does not provide the pool UUID and does not provide the container UUID).

• Destroy a container in a pool (daos container destroy)

  • Destroy by container UUID

    • Usage: daos container destroy -pool=UUID --sys=SYSNAME --svc=SRVRANKLIST --cont=UUID

  • Destroy by path that the container is linked to

    • Usage: daos container destroy -sys=SYSNAME --svc=SRVRANKLIST --path=/path/to/destroy_cont_and_unlink

The remaining container commands use the --cont=UUID form (the --path= option is available, but is not shown)

• List all objects in a container (daos-container list-objects)

  • Usage: daos container list-objects -pool=UUID --sys=SYSNAME --svc=SRVRANKLIST --cont=UUID 

• Create a snapshot on container based on the latest committed epoch

  • Unnamed

    • Usage: daos container create-snap -pool=UUID --sys=SYSNAME --svc=SRVRANKLIST --cont=UUID 

  • Named

    • Usage: daos container create-snap -pool=UUID --sys=SYSNAME --svc=SRVRANKLIST --cont=UUID --snap=mysnapname

• List all snapshots in a container

  • Usage: daos container list-snaps -pool=UUID --sys=SYSNAME --svc=SRVRANKLIST --cont=UUID

• Destroy container snapshot(s)

  • Single epoch snapshot

    • Usage: daos container destroy-snap  -pool=UUID --sys=SYSNAME --svc=SRVRANKLIST --cont=UUID --epc=B

  • Multiple snapshots within an epoch range

    • Usage: daos container destroy-snap -pool=UUID --sys=SYSNAME --svc=SRVRANKLIST --cont=UUID  --epcrange=B-D

• Rollback container to specified snapshot

  • Rollback to a named snapshot

    • Usage: daos container rollback  -pool=UUID --sys=SYSNAME --svc=SRVRANKLIST --cont=UUID --snap=mysnapname

  • Rollback to a snapshot at an epoch number

    • Usage: daos container rollback  -pool=UUID --sys=SYSNAME --svc=SRVRANKLIST --cont=UUID --epc=A

Proposal: daosctl test considerations

• Change --server-group to --sys=

• Change --size to --scm-size?

• Change --replicas=NUM_METADATA_REPLICAS to --nsvc=

• Change --servers=SRVRANKLIST to --svc= (for pool replica ranks)

• Change exclude-target --targets= to take a list of pairs (instead of current approach that makes pairs from 2 lists: --rank=ra,rb,rc and --targets=ta,tb,tc)

• Change --server= to --rank=

• Change --rank= to --ranks= (or svc= ???) for kill-leader - (what will we do for dmg kill? Probably --ranks=. Choose same)

• Change --server=SERVER-LIST to  --ranks= (or svc= ???) for kill-server (what will we do for dmg kill? Choose same)

• Change -c-uuid to --cont=CUUID

• Change -p-uuid to --pool=PUUID

Proposal: Commands, Resources, Operations and Arguments