...
To mark a container as WORM, we can add a new container property which can be changed only once using the daos tool:
daos cont set-prop worm:true
That operation that will enable all the different optimizations and layout changes that can be done knowing the container is a WORM container. There are two ways we need to consider how / when this property is set:
At container creation time, where data is being first added / generated in the container. This would mean for example that after a key is inserted, it cannot be modified; when an array object is opened and written to once, it cannot be written to again, etc. Enforcing that at the DAOS level might be challenging, so it might need to be the user responsibility to ensure that.
On an existing container that has already been ingested or generated, and will be read-only.
As for the different optimizations that can be done, those can be middleware dependent or internal to DAOS:
DFS Layout optimization:
The DFS inode entry for every file and directory is stored in the parent directory that this object belongs to. The POSIX mapping in DFS to DAOS object is described here:
https://github.com/daos-stack/daos/blob/master/src/client/dfs/README.md
...
To set the WORM property, users can use the daos tool or API on a container :
daos cont set-prop
daos_cont_set_prop()
The main issue here is (at create time or on an existing container to mark read-only)
When doing the layout optimization as the data is being generated / ingested, we need to consider how to enable the optimizations on the fly. For POSIX, we need to maintain the file size as we write to the file for example. We can possible also maintain the number of entries per directory or a list of entries in the directory for a faster readdir.
If we are doing the layout changes on an existing container, we need to consider how we implement the layout optimizations. Should the tool itself scan the container POSIX namespace or should we implement a different mechanism that does that. Usually the set-prop command and API only set a property value on the container and does not do any further work. So we probably need a new mechanism to implement the layout changes (query the file sizes and store them in the inode, any VOS layout optimizations, etc). This can be accomplished by adding a new option to the daos tool:
...