Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Framework cowers nested failures on openshift #1366

Open
mocenas opened this issue Oct 10, 2024 · 0 comments
Open

Framework cowers nested failures on openshift #1366

mocenas opened this issue Oct 10, 2024 · 0 comments

Comments

@mocenas
Copy link
Contributor

mocenas commented Oct 10, 2024

When framework deploys service to openshift which fails for some reason, FW then tries to collect evidence from OCP. But if that fails too, original exception is lost (happnes somewhere here).

Happened to me, when FW tried to collect logs from pod, but the pods was deleted in a meantime. In that case I got just error

... io.fabric8.kubernetes.client.KubernetesClientException ... Pod not found...
at io.fabric8.kubernetes.client.dsl.internal.OperationSupport.requestFailure(OperationSupport.java:660)
	at io.fabric8.kubernetes.client.dsl.internal.OperationSupport.requestFailure(OperationSupport.java:640)
	at io.fabric8.kubernetes.client.dsl.internal.OperationSupport.assertResponseCode(OperationSupport.java:589)
	at io.fabric8.kubernetes.client.dsl.internal.OperationSupport.handleRaw(OperationSupport.java:739)
	at io.fabric8.kubernetes.client.dsl.internal.OperationSupport.handleRawGet(OperationSupport.java:474)
	at io.fabric8.kubernetes.client.dsl.internal.core.v1.PodOperationsImpl.doGetLog(PodOperationsImpl.java:133)
	at io.fabric8.kubernetes.client.dsl.internal.core.v1.PodOperationsImpl.getLog(PodOperationsImpl.java:141)
	at io.quarkus.test.bootstrap.inject.OpenShiftClient.logs(OpenShiftClient.java:441)
	at io.quarkus.test.bootstrap.OpenShiftExtensionBootstrap.printPodLogs(OpenShiftExtensionBootstrap.java:89)
	at io.quarkus.test.bootstrap.OpenShiftExtensionBootstrap.onError(OpenShiftExtensionBootstrap.java:76)
	at io.quarkus.test.bootstrap.QuarkusScenarioBootstrap.lambda$scenarioOnError$11(QuarkusScenarioBootstrap.java:195)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)
	at io.quarkus.test.bootstrap.QuarkusScenarioBootstrap.scenarioOnError(QuarkusScenarioBootstrap.java:195)
	at io.quarkus.test.bootstrap.QuarkusScenarioBootstrap.launchService(QuarkusScenarioBootstrap.java:186)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)
	at io.quarkus.test.bootstrap.QuarkusScenarioBootstrap.beforeAll(QuarkusScenarioBootstrap.java:73)
	at io.quarkus.test.bootstrap.QuarkusScenarioBootstrap.beforeAll(QuarkusScenarioBootstrap.java:50)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)

Original failure that resource failed to start was completely lost. Problem in searching for pod was, that pod was app pods restarted (rescaled 1->0->1) in that time.

Kind of related to this - why is FW deploying pods to OCP is way that that it:

  1. Creates the deployment - which is doomed to fail, because no image is present yet
  2. Builds the image
  3. "Restarts" the deployment - scales it down to 0 and back up to 1.

On other places we do it in a way that it first builds the image and then deploys app on first try. That saves a lot of trouble with rescaling the pods.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant