feat(common): rework default spread and affinity to prevent RWX issues

[PrivatePuffin]

Is your feature request related to a problem?

• Currently we have certain charts that share storage between multiple pods.
• We also have a default spread constraint that prefers spreading all pods over multiple nodes

This creates issues where those pods all call for access to the same storage for many users, as RWX CSIs are definately not the default.

Describe the solution you'd like

Make some sane defaults:

• When multiple workloads access a persistence object thats PVC, deploy them on the same node
• When there are multiple replicas for a specific workload, prefer to spread them over multiple nodes
• When there are multiple workloads that do NOT share persistence objects, prefer to spread them over multiple nodes.

This should fix most RWX-RWO related issues

Describe alternatives you've considered

.

Additional context

Its important to note that "multi-workload" splits are not for High Availability, so it doesn't matter if they get deployed on the same pod.

However, multiple replicas áre intended for HA and hence should be split among nodes.

I've read and agree with the following

• I've checked all open and closed issues and my request is not there.
• I've checked all open and closed pull requests and my request is not there.