...

The initial implementation and testing can be done only for POSIX type containers, as the use cases are well know and testing can be straightforward. In a later phase, we can explore extending support for WORM to other types of containers. Generally the server side optimizations will apply to any type of container; however the client side and container layout optimizations that are middleware specific, apply to each container type and each middleware would need to be updated to see how it can leverage the WORM information to optimize its data access.

Design Overview

Describe the key architecture decisions, benefits & drawbacks.
Describe what software component(s) be modified and how. Diagrams are welcomed.

...

How is the user/admin expected to interact with the new feature? Describe the API/tool.
What are all the tunables provided to the user/admin?
Any extra statistics that should reported to the user/admin?
Explain how errors will be handled.

To set the WORM property, users can use the daos tool or API on a container:

• daos cont set-prop

• daos_cont_set_prop()

The main issue here is how we implement the layout optimizations. Should the tool itself scan the container POSIX namespace or should we implement a different mechanism that does that. Usually the set-prop command and API only set a property value on the container and does not do any further work. So we probably need a new mechanism to implement the layout changes (query the file sizes and store them in the inode, mpifileutils, API for propertyany VOS layout optimizations, etc). This can be accomplished by adding a new option to the daos tool:

• daos cont make-worm mypool mycont

We can also extent mpifileutils to support the scanning and layout changes in parallel (we would need to investigate what tool in MFU to extent to support this); or we will need to implement our own parallel tool.

Impacts

Any performance impact?
Any API changes? If so, internal or external API? Any changes required to middleware? Any interop requirements?Any VOS/config layout changes? How will migration will be supported?Any extra parameters required in the config file?Any wire protocol change? How will interop be supported?Any impact on the rebuild protocol?Any impact on aggregation?Any impact on security?

• New APIs to be added for setting the WORM property.

• New tools to be added for the layout changes.

• A container marked as WORM with optimized layout will not be accessible using older versions of DAOS that might not understand the layout changes that were made.

Quality

How the feature will be tested? Unit tests, functional tests and system tests need to be covered.
Describe the extra soak/performance tests that should be added.

...