NOTE THESE ARE NOT TO BE APPLIED TO 2.0 TESTING, USE THE QUICKSTARTS IN THE 2.0 ON-LINE DOCUMENTATION
Table of Contents
...
This documentation provides a general tour to DAOS management commands (dmg) for daos_admin, and DAOS tools (daos) for daos_client users. Provides help with pool and container create, list, query and destroy on DAOS server for daos_admin and daos_client users. Some frequent common errors and workarounds for new users when using the dmg and daos tools. Example runs of data transfer between DAOS file systems, by setting up of DAOS dfuse mount point and run traffic with dfuse fio and mpirun mdtest. Example of basic dmg and daos tools runs on 2 hosts DAOS server and 1 host client, runs of DAOS rebuild over dfuse fio and mpirun mdtest on a 4 hosts DAOS server.
...
Set environment variables for list of servers, client and admin node.
| Code Block |
|---|
|
# Example of 2 hosts server
# For 1 host server, export SERVER_NODES=node-1
export SERVER_NODES=node-1,node-2
# Example to use admin and client on the same node
export ADMIN_NODE=node-3
export CLIENT_NODE=node-3
export ALL_NODES=$SERVER_NODES,$CLIENT_NODE |
...
dmg system query
| Code Block |
|---|
|
# system query output for a 2 hosts DAOS server
$ dmg system query
Rank State
---- -----
[0-1] Joined |
...
dmg storage query usage
| Code Block |
|---|
|
# system storage query usage output for a 2 hosts DAOS server
$ dmg storage query usage
Hosts SCM-Total SCM-Free SCM-Used NVMe-Total NVMe-Free NVMe-Used
----- --------- -------- -------- ---------- --------- ---------
boro-35 17 GB 17 GB 0 % 0 B 0 B N/A
boro-8 17 GB 17 GB 0 % 0 B 0 B N/A |
dmg pool create help
| Code Block |
|---|
|
$ dmg pool create --help
Usage:
dmg [OPTIONS] pool create [create-OPTIONS]
Application Options:
--allow-proxy Allow proxy configuration via environment
-l, --host-list= comma separated list of addresses <ipv4addr/hostname>
-i, --insecure have dmg attempt to connect without certificates
-d, --debug enable debug output
-j, --json Enable JSON output
-J, --json-logging Enable JSON-formatted log output
-o, --config-path= Client config file path
Help Options:
-h, --help Show this help message
[create command options]
-g, --group= DAOS pool to be owned by given group, format name@domain
-u, --user= DAOS pool to be owned by given user, format name@domain
-p, --name= Unique name for pool (set as label)
-a, --acl-file= Access Control List file path for DAOS pool
-z, --size= Total size of DAOS pool (auto)
-t, --scm-ratio= Percentage of SCM:NVMe for pool storage (auto) (default: 6)
-k, --nranks= Number of ranks to use (auto)
-v, --nsvc= Number of pool service replicas
-s, --scm-size= Per-server SCM allocation for DAOS pool (manual)
-n, --nvme-size= Per-server NVMe allocation for DAOS pool (manual)
-r, --ranks= Storage server unique identifiers (ranks) for DAOS pool
-S, --sys= DAOS system that pool is to be a part of (default: daos_server) |
dmg pool create
| Code Block |
|---|
|
# Create a 10GB pool
$ dmg pool create --size=10G
Creating DAOS pool with automatic storage allocation: 10 GB NVMe + 6.00% SCM
Pool created with 100.00% SCM/NVMe ratio
-----------------------------------------
UUID : 0a6003c6-23a7-4cb5-8895-c004ca2b75f5
Service Ranks : 0
Storage Ranks : [0-1]
Total Size : 10 GB
SCM : 10 GB (5.0 GB / rank)
NVMe : 0 B (0 B / rank)
$ dmg storage query usage
Hosts SCM-Total SCM-Free SCM-Used NVMe-Total NVMe-Free NVMe-Used
----- --------- -------- -------- ---------- --------- ---------
boro-35 17 GB 12 GB 29 % 0 B 0 B N/A
boro-8 17 GB 11 GB 36 % 0 B 0 B N/A |
...
| Code Block |
|---|
|
$ daos cont query --pool=$DAOS_POOL --cont=$DAOS_CONT
Pool UUID: 528f4710-7eb8-4850-b6aa-09e4b3c8f532
Container UUID: bc4fe707-7470-4b7d-83bf-face75cc98fc
Number of snapshots: 0
Latest Persistent Snapshot: 0
Highest Aggregated Epoch: 172477977191481344
Container redundancy factor: 1
|
daos container snapshot help/create/list/destroy
| Code Block |
|---|
|
$ daos help cont create-snap
daos command (v1.2), libdaos 1.2.0
container options (snapshot and rollback-related):
--snap=NAME container snapshot (create/destroy-snap, rollback)
--epc=EPOCHNUM container epoch (destroy-snap, rollback)
--epcrange=B-E container epoch range (destroy-snap)
container options (query, and all commands except create):
<pool options> with --cont use: (--pool, --sys-name)
<pool options> with --path use: (--sys-name)
--cont=UUID (mandatory, or use --path)
--path=PATHSTR (mandatory,
or
use --cont) |
daos container snapshot create/list/destroy
| Code Block |
|---|
|
$ daos cont create-snap --pool=$DAOS_POOL --cont=$DAOS_CONT
snapshot/epoch 172646116775952384 has been created
$ daos container list-snaps --pool=$DAOS_POOL --cont=$DAOS_CONT
Container's snapshots :
172478166024060928
172646116775952384
$ daos container destroy-snap --pool=$DAOS_POOL --cont=$DAOS_CONT --epc=172646116775952384
$ daos container list-snaps --pool=$DAOS_POOL --cont=$DAOS_CONT
Container's snapshots :
172478166024060928
|
Frequent errors user might see and workaround
use dmg command without daos_admin
...
privilege
| Code Block |
|---|
|
# Error message or timeout after dmg system query
$ dmg system query
ERROR: dmg: Unable to load Certificate Data: could not load cert: stat /etc/daos/certs/admin.crt: no such file or directory
#
or
Node-hang after dmg system query command issued
# Workaround
# 1. Make sure the admin-host /etc/daos/daos_control.yml is correctly configured.
# including:
# hostlist: <daos_server_lists>
# port: <port_num>
# transport_config:
# allow_insecure: <true/false>
# ca_cert: /etc/daos/certs/daosCA.crt
# cert: /etc/daos/certs/admin.crt
# key: /etc/daos/certs/admin.key
#
# 2. Make sure the admin-host allow_insecure mode match with the servers'. |
...
dmg pool destroy timeout
| Code Block |
|---|
|
# Timeout whendmg pool destroy Timeout or failed due to pool has active container(s) inside the pool
# Workaround pool destroy --force option
$ dmg pool destroy --pool=$DAOS_POOL --force
Pool-destroy command succeeded |
...
unmount
| Code Block |
|---|
|
$ /usr/bin/fusermount -u /tmp/daos_test1/
$ /usr/bin/df -h -t fuse.daos
df: no file systems processed |
...