init container to decide allowing reconciliation of manager

[leelavg]

we want to run manager only when required, i.e, we are going to reconcile and manage CSI among other operands. we are achieving that by adding an init container which only get successful if client-operator is deployed standalone or running in provider configured odf cluster.

Depends on: red-hat-storage/ocs-operator#2717
ref: red-hat-storage/odf-operator#450

[leelavg]

/cherry-pick release-4.16

[openshift-cherrypick-robot]

@leelavg: once the present PR merges, I will cherry-pick it on top of release-4.16 in a new PR and assign it to you.

In response to this:

/cherry-pick release-4.16

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[leelavg]

/hold

[leelavg]

Testing:

- No StorageCluster CRD
I0729 14:00:24.636988 2124186 main.go:51] StorageCluster CRD not found and proceeding with reconciliation of operands

- CRD exists but not storagecluster
I0729 13:51:26.625930 2120304 main.go:77] Waiting for StorageCluster to be deployed
I0729 13:51:31.638621 2120304 main.go:77] Waiting for StorageCluster to be deployed
^Csignal: interrupt

- storagecluster exists but not in Provider mode
> k -nopenshift-storage get storagecluster/ocs-storagecluster -ojsonpath='{.metadata.annotations}' | jq 'del(."kubectl.kubernetes.io/last-applied-configuration")'
{
  "uninstall.ocs.openshift.io/cleanup-policy": "delete",
  "uninstall.ocs.openshift.io/mode": "graceful"
}
I0729 13:52:24.907032 2120630 main.go:79] Waiting for StorageCluster deployed mode to be 'Provider'
I0729 13:52:30.179057 2120630 main.go:79] Waiting for StorageCluster deployed mode to be 'Provider'
^Csignal: interrupt

> k -nopenshift-storage get storagecluster/ocs-storagecluster -ojsonpath='{.metadata.annotations}' | jq 'del(."kubectl.kubernetes.io/last-applied-configuration")'
{
  "mode": "provider",
  "uninstall.ocs.openshift.io/cleanup-policy": "delete",
  "uninstall.ocs.openshift.io/mode": "graceful"
}
I0729 13:56:41.060310 2123137 main.go:72] StorageCluster is configured to run in 'Provider' mode and proceeding with reconciliation of operands

update to the client-console deployment is yet to be tested.

[nb-ohad]

Why do we need a new service account an a new RBAC?
Why can't we just use the operators SA and RBAC?

[leelavg]

Why do we need a new service account an a new RBAC? Why can't we just use the operators SA and RBAC?

1. We have two deployments in CSV, client-op (custom sa) and client-console (default sa)
2. We need to inject sidecar into both of these deployments, however custom sa in client-op is enough for it's init containers but using the same which has more privilege for init container in client-console didn't seem good
3. Reusing client-op sa for client-console reduces good chunk of rbac in this PR which I could do based on your comment, wdyt?

---

I realized something when replying to the comments, CSV will only be successful if Pod is in Running state, with an init container, pod will only be in Running state after init which could potentially fail d/s CI. Overall we might need to rethink on this.

[nb-ohad]

Why do we need a new service account an a new RBAC? Why can't we just use the operators SA and RBAC?

1. We have two deployments in CSV, client-op (custom sa) and client-console (default sa)
2. We need to inject sidecar into both of these deployments, however custom sa in client-op is enough for it's init containers but using the same which has more privilege for init container in client-console didn't seem good
3. Reusing client-op sa for client-console reduces good chunk of rbac in this PR which I could do based on your comment, wdyt?

I realized something when replying to the comments, CSV will only be successful if Pod is in Running state, with an init container, pod will only be in Running state after init which could potentially fail d/s CI. Overall we might need to rethink on this.

CI should take into account that this pod is waiting on installation, not the other way around

[leelavg]

Tested:

1. client-op is running standalone : proceed
2. client-op is installed alongside odf with
   a. storagecluster doesn't exist : wait
   b. storagecluster not in provider mode: wait
   c. storagecluster in provider mode: proceed

[leelavg]

I have raised tracker issues for other u/s projects which tries to consume client-op as olm bundles, need to fix them if those projects require updated client-op bundles:
red-hat-storage/ocs-operator#2722
red-hat-storage/odf-operator#461

[nb-ohad]

/lgtm

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: leelavg, nb-ohad

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [leelavg,nb-ohad]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[leelavg]

/unhold

[openshift-cherrypick-robot]

@leelavg: #196 failed to apply on top of branch "release-4.16":

Applying: bundle: add a new init container to wait for reconcile condition
Using index info to reconstruct a base tree...
M	Dockerfile
M	Makefile
M	bundle.Dockerfile
M	bundle/manifests/ocs-client-operator.clusterserviceversion.yaml
M	bundle/metadata/annotations.yaml
M	hack/go-build.sh
Falling back to patching base and 3-way merge...
Auto-merging hack/go-build.sh
Auto-merging bundle/metadata/annotations.yaml
Auto-merging bundle/manifests/ocs-client-operator.clusterserviceversion.yaml
CONFLICT (content): Merge conflict in bundle/manifests/ocs-client-operator.clusterserviceversion.yaml
Auto-merging bundle.Dockerfile
Auto-merging Makefile
Auto-merging Dockerfile
error: Failed to merge in the changes.
hint: Use 'git am --show-current-patch=diff' to see the failed patch
Patch failed at 0001 bundle: add a new init container to wait for reconcile condition
When you have resolved this problem, run "git am --continue".
If you prefer to skip this patch, run "git am --skip" instead.
To restore the original branch and stop patching, run "git am --abort".

In response to this:

/cherry-pick release-4.16

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.