Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error when installing m3db operator in Docker for Desktop #310

Open
pat-wong-affirm opened this issue Jan 18, 2022 · 14 comments
Open

Error when installing m3db operator in Docker for Desktop #310

pat-wong-affirm opened this issue Jan 18, 2022 · 14 comments

Comments

@pat-wong-affirm
Copy link

Thanks for opening an issue for the M3DB Operator! We'd love to help you, but we need the following information included
with any issue:

  • What version of the operator are you running? Please include the docker tag. If using master, please include the git
    SHA logged when the operator first starts.

manifest = https://raw.githubusercontent.com/m3db/m3db-operator/master/bundle.yaml

docker image tag = quay.io/m3db/m3db-operator:latest

also tried with docker image tag being v0.13.0, same result

  • What version of Kubernetes are you running? Please include the output of kubectl version.
Client Version: version.Info{Major:"1", Minor:"23", GitVersion:"v1.23.1", GitCommit:"86ec240af8cbd1b60bcc4c03c20da9b98005b92e", GitTreeState:"clean", BuildDate:"2021-12-16T11:33:37Z", GoVersion:"go1.17.5", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"22", GitVersion:"v1.22.4", GitCommit:"b695d79d4f967c403a96986f1750a35eb75e75f1", GitTreeState:"clean", BuildDate:"2021-11-17T15:42:41Z", GoVersion:"go1.16.10", Compiler:"gc", Platform:"linux/amd64"}

MacOS - Docker For Desktop - built-in kubernetes

  • What are you trying to do?

Install the m3db operator

  • What did you expect to happen?

Operator pod would be in Running state

  • What happened?
{"level":"info","ts":1642550015.150743,"msg":"using InCluster k8s config"}
{"level":"info","ts":1642550015.1522524,"msg":"starting Operator controller","controller":"m3db-cluster-controller"}
E0118 23:53:35.163925       1 reflector.go:153] pkg/mod/k8s.io/[email protected]/tools/cache/reflector.go:105: Failed to list *v1alpha1.M3DBCluster: the server could not find the requested resource (get m3dbclusters.operator.m3db.io)
{"level":"fatal","ts":1642550015.1678174,"msg":"error running controller","error":"could not create or update CRD: error creating CRD 'm3dbclusters.operator.m3db.io': the server could not find the requested resource","errorVerbose":"the server could not find the requested resource\nerror creating CRD 'm3dbclusters.operator.m3db.io'\ncould not create or update CRD"}
@schallert
Copy link
Collaborator

Hey @pat-wong-affirm, thanks for reaching out. It looks like we haven't cut a release since merging #302, which makes the operator compatible with Kubernetes 1.21+.

Can you try using the master tag of the operator and letting me know if that fixes it? If so, we can likely solve this by cutting a new release. Thanks!

@pat-wong-affirm
Copy link
Author

unfortunately, got the same error when using image tag quay.io/m3db/m3db-operator:master

it's possible this error is specific to Docker for Desktop, as another team member was able to get the operator working in an EKS cluster

@kendrickclark
Copy link

I am having this issue as well

@larsf
Copy link

larsf commented Mar 23, 2022

I am also seeing this and I am also trying to deploy it on Docker for Desktop

@larsf
Copy link

larsf commented Mar 25, 2022

could this be because the api server in the desktop environment is not running as localhost? main.go sets masterURL to be 127.0.0:8001, but in the desktop environment you have:

kubectl cluster-info
Kubernetes control plane is running at https://kubernetes.docker.internal:6443
CoreDNS is running at https://kubernetes.docker.internal:6443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy

I tried adding
args: ["-debug", "-masterhost", "https://kubernetes.docker.internal:6443"]

to bundle.yaml
but it didn't help ...

@pat-wong-affirm pat-wong-affirm changed the title Error when installing m3db operator Error when installing m3db operator in Docker for Desktop Mar 25, 2022
@larsf
Copy link

larsf commented Mar 25, 2022

I was able to get the operator to start by passing in:
["-debug=true", "-manage-crd=false", "-masterhost", "https://kubernetes.docker.internal:6443"]

(I believe only -manage-crd=false is really needed to get it to start)

I do see the same errors from the controller as documented earlier periodically, but it does stay running.

Trying to stand up a m3db cluster though, fails with this error:
no matches for kind "M3DBCluster" in version "operator.m3db.io/v1alpha

@larsf
Copy link

larsf commented Mar 29, 2022

the m3db-operator seems to use interfaces that are no longer supported (k8s v1.22+):
46: crdClient := k.kubeExt.ApiextensionsV1beta1().CustomResourceDefin

see: storageos/cluster-operator#348

Server Version: version.Info{Major:"1", Minor:"22"

@kendrickclark
Copy link

Would a fix be to change that to ApiextensionsV1()

@larsf
Copy link

larsf commented Mar 30, 2022 via email

@larsf
Copy link

larsf commented Mar 30, 2022

Tried to do a build with my changes, but found that the .ci directory is empty, so the build fails because it cannot find go-build-ldflags.sh and common.mk

Any reason they are not in the source tree?

Answered my own question - found the scripts in the m3db/ci-scripts repo
Got further, now I see:
~/src/m3operator$ make
--- clean-all
build .: cannot find module for path .
make: *** [Makefile:194: clean-all] Error 1

@kendrickclark
Copy link

@larsf Any luck there?

@kendrickclark
Copy link

Hey @pat-wong-affirm, thanks for reaching out. It looks like we haven't cut a release since merging #302, which makes the operator compatible with Kubernetes 1.21+.

Can you try using the master tag of the operator and letting me know if that fixes it? If so, we can likely solve this by cutting a new release. Thanks!

schallert , Unfortunately this does not resolve the issue.

@niravpatel2008
Copy link

Kubernetes version 1.22

i am also facing similar issue on eks while installing operator using following command.

kubectl apply -f https://raw.githubusercontent.com/m3db/m3db-operator/master/bundle.yaml

error:

{"level":"info","ts":1657946530.7369597,"msg":"using InCluster k8s config"}
{"level":"info","ts":1657946530.7382486,"msg":"starting Operator controller","controller":"m3db-cluster-controller"}
{"level":"fatal","ts":1657946530.7782364,"msg":"error running controller","error":"could not create or update CRD: could not fetch CRD 'm3dbclusters.operator.m3db.io': customresourcedefinitions.apiextensions.k8s.io \"m3dbclusters.operator.m3db.io\" is forbidden: User \"system:serviceaccount:default:m3db-operator\" cannot get resource \"customresourcedefinitions\" in API group \"apiextensions.k8s.io\" at the cluster scope","errorVerbose":"customresourcedefinitions.apiextensions.k8s.io \"m3dbclusters.operator.m3db.io\" is forbidden: User \"system:serviceaccount:default:m3db-operator\" cannot get resource \"customresourcedefinitions\" in API group \"apiextensions.k8s.io\" at the cluster scope\ncould not fetch CRD 'm3dbclusters.operator.m3db.io'\ncould not create or update CRD"}

@niravpatel2008
Copy link

Kubernetes version 1.22

i am also facing similar issue on eks while installing operator using following command.

kubectl apply -f https://raw.githubusercontent.com/m3db/m3db-operator/master/bundle.yaml

error:

{"level":"info","ts":1657946530.7369597,"msg":"using InCluster k8s config"}
{"level":"info","ts":1657946530.7382486,"msg":"starting Operator controller","controller":"m3db-cluster-controller"}
{"level":"fatal","ts":1657946530.7782364,"msg":"error running controller","error":"could not create or update CRD: could not fetch CRD 'm3dbclusters.operator.m3db.io': customresourcedefinitions.apiextensions.k8s.io \"m3dbclusters.operator.m3db.io\" is forbidden: User \"system:serviceaccount:default:m3db-operator\" cannot get resource \"customresourcedefinitions\" in API group \"apiextensions.k8s.io\" at the cluster scope","errorVerbose":"customresourcedefinitions.apiextensions.k8s.io \"m3dbclusters.operator.m3db.io\" is forbidden: User \"system:serviceaccount:default:m3db-operator\" cannot get resource \"customresourcedefinitions\" in API group \"apiextensions.k8s.io\" at the cluster scope\ncould not fetch CRD 'm3dbclusters.operator.m3db.io'\ncould not create or update CRD"}

using operator image tag 0484dcc instead of 1.13.0 works for me. but it seems it has some security issues. wish it will gets resolved with 1.14.0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants