Table of Contents

...

• sudo access configured
• password-less ssh configured
• pdsh installed (or some other means of running multiple remote commands in parallel)

In addition the server nodes should also have:

...

1.    Prepare the pmem devices on Server nodes

   Code Block
   language bash
   pdsh -w $SERVER_NODES -x $SERVER_NODES daos_server storage prepare --scm-only Preparing locally-attached SCM... Memory allocation goals for SCM will be changed and namespaces modified, this will be a destructive operation. Please ensure namespaces are unmounted and locally attached SCM & NVMe devices are not in use. Please be patient as it may take several minutes and subsequent reboot maybe required. Are you sure you want to continue? (yes/no) yes A reboot is required to process new SCM memory allocation goals.

2. Reboot the server node

3. Re run the prepare cmdline again

   Code Block
   language bash
   pdsh -w $SERVER_NODES -x $SERVER_NODES daos_server storage prepare --scm-only Preparing locally-attached SCM... SCM namespaces: SCM Namespace Socket ID Capacity ------------- --------- -------- pmem0 0 3.2 TB pmem1 0 3.2 TB

4. Prepare the NVME devices on Server nodes

   Code Block
   language bash
   pdsh -w $SERVER_NODES -x $SERVER_NODES daos_server storage prepare --nvme-only -u root Preparing locally-attached NVMe storage...

5. Scan the available storage on the Server nodes

   Code Block
   language bash
   pdsh -w $SERVER_NODES -x $SERVER_NODES daos_server storage scan Scanning locally-attached storage... NVMe PCI Model FW Revision Socket ID Capacity -------- ----- ----------- --------- -------- 0000:5e:00.0 INTEL SSDPE2KE016T8 VDV10170 0 1.6 TB 0000:5f:00.0 INTEL SSDPE2KE016T8 VDV10170 0 1.6 TB 0000:81:00.0 INTEL SSDPED1K750GA E2010475 1 750 GB 0000:da:00.0 INTEL SSDPED1K750GA E2010475 1 750 GB SCM Namespace Socket ID Capacity ------------- --------- -------- pmem0 0 3.2 TB pmem1 1 3.2 TB

...

In this section certificates will be generated and installed for encrypting DAOS control plane communications.

Administrative nodes require the following certificate files:

...

1. Generate a new set of certificates.
   

   Code Block
   language bash
   cd /tmp /usr/lib64/daos/certgen/gen_certificates.sh

   Note
   These files should be protected from unauthorized access and preserved for future use.

   
   

2. Copy the certificates to a common location on each node in order to be able to move them to the final location
   

   Code Block
   language bash
   pdsh -S -w $ALL_NODES -x $(hostname -s) scp -r $(hostname -s):/tmp/daosCA /tmp

3. Copy the certificates to their default location (/etc/daos) on each admin node
   

   Code Block
   language bash
   pdsh -S -w $ADMIN_NODE sudo cp /tmp/daosCA/certs/daosCA.crt /etc/daos/certs/. pdsh -S -w $ADMIN_NODE sudo cp /tmp/daosCA/certs/admin.crt /etc/daos/certs/. pdsh -S -w $ADMIN_NODE sudo cp /tmp/daosCA/certs/admin.key /etc/daos/certs/.

   Note
   If the /etc/daos/certs directory does not exist on the admin nodes then use the following command to create it: pdsh -S -w $ADMIN_NODES sudo mkdir /etc/daos/certs

4. Copy the certificates to their default location (/etc/daos) on each client node
   

   Code Block
   language bash
   pdsh -S -w $CLIENT_NODES sudo cp /tmp/daosCA/certs/daosCA.crt /etc/daos/certs/. pdsh -S -w $CLIENT_NODES sudo cp /tmp/daosCA/certs/agent.crt /etc/daos/certs/. pdsh -S -w $CLIENT_NODES sudo cp /tmp/daosCA/certs/agent.key /etc/daos/certs/.

   Note
   If the /etc/daos/certs directory does not exist on the client nodes then use the following command to create it: pdsh -S -w $CLIENT_NODES sudo mkdir /etc/daos/certs

   
   

5. Copy the certificates to their default location (/etc/daos) on each server node
   

   Code Block
   language bash
   pdsh -S -w $SERVER_NODES sudo cp /tmp/daosCA/certs/daosCA.crt /etc/daos/certs/. pdsh -S -w $SERVER_NODES sudo cp /tmp/daosCA/certs/server.crt /etc/daos/certs/. pdsh -S -w $SERVER_NODES sudo cp /tmp/daosCA/certs/server.key /etc/daos/certs/. pdsh -S -w $SERVER_NODES sudo cp /tmp/daosCA/certs/agent.crt /etc/daos/certs/clients/agent.crt

   
   

6. Set the ownership of the admin certificates on each admin node
   

   Code Block
   language bash
   pdsh -S -w $ADMIN_NODE sudo chown $USER:$USER /etc/daos/certs/daosCA.crt pdsh -S -w $ADMIN_NODE sudo chown $USER:$USER /etc/daos/certs/admin.*

   
   

7. Set the ownership of the client certificates on each client node
   

   Code Block
   language bash
   pdsh -S -w $CLIENT_NODES sudo chown $USER:$USER /etc/daos/certs/daosCA.crt pdsh -S -w $CLIENT_NODES sudo chown daos_agent:daos_agent /etc/daos/certs/agent.*

   
   

8. Set the ownership of the server certificates on each server node
   

   Code Block
   language bash
   pdsh -S -w $SERVER_NODES sudo chown daos_server:daos_server /etc/daos/certs/daosCA.crt pdsh -S -w $SERVER_NODES sudo chown daos_server:daos_server /etc/daos/certs/server.* pdsh -S -w $SERVER_NODES sudo chown daos_server:daos_server /etc/daos/certs/clients/agent.crt pdsh -S -w $SERVER_NODES sudo chown daos_server:daos_server /etc/daos/certs/clients

   
   

...