Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[IAM]: Tests are flaky #1622

Open
amanda-tarafa opened this issue Feb 9, 2022 · 7 comments
Open

[IAM]: Tests are flaky #1622

amanda-tarafa opened this issue Feb 9, 2022 · 7 comments
Labels
api: iam Issues related to the Identity and Access Management API. priority: p2 Moderately-important priority. Fix may not be included in next release. samples Issues that are directly related to samples. type: process A process-related concern. May include testing, release, or the like.

Comments

@amanda-tarafa
Copy link
Member

In particular if run concurrently: https://source.cloud.google.com/results/invocations/fb9a32a6-b22a-4fe7-b1fd-fe7c137c462b/log
@amanda-tarafa amanda-tarafa added type: process A process-related concern. May include testing, release, or the like. priority: p2 Moderately-important priority. Fix may not be included in next release. api: iam Issues related to the Identity and Access Management API. labels Feb 9, 2022
@product-auto-label product-auto-label bot added the samples Issues that are directly related to samples. label Feb 9, 2022
@amanda-tarafa amanda-tarafa mentioned this issue Feb 9, 2022
Merged
1 task
@amanda-tarafa
Copy link
Member Author

Another one here: https://source.cloud.google.com/results/invocations/c8b62f40-423e-487d-a70b-46cded4c3f3b/log

@amanda-tarafa amanda-tarafa mentioned this issue Nov 8, 2022
Merged
1 task
@amanda-tarafa
Copy link
Member Author

And another one here: https://fusion2.corp.google.com/invocations/4adb6d71-fe0e-4cab-80f5-952b6526b951/log

@amanda-tarafa
Copy link
Member Author

And more here: https://fusion2.corp.google.com/invocations/7c0070b7-a921-4ba6-ada9-bd10f4fa6654/log, for service accounts. The IDs for the resources seem to be generated based on sequential numbers and that of course causes a lot of overlap.

@amanda-tarafa
Copy link
Member Author

Again, service account tests are flaky, or, service account creation takes a while to propagate, not sure which.

https://source.cloud.google.com/results/invocations/23208802-bd8f-4d8c-b4e3-feed40c83849/log

[xUnit.net 00:00:01.89]     GoogleCloudSamples.ServiceAccountTests.TestServiceAccounts [FAIL]
  Failed GoogleCloudSamples.ServiceAccountTests.TestServiceAccounts [1 s]
  Error Message:
   Google.GoogleApiException : The service iam has thrown an exception. HttpStatusCode is NotFound. Unknown service account
  Stack Trace:
     at Google.Apis.Requests.ClientServiceRequest`1.ParseResponse(HttpResponseMessage response)
   at Google.Apis.Requests.ClientServiceRequest`1.Execute()
   at ServiceAccounts.RenameServiceAccount(String email, String newDisplayName) in C:\tmpfs\src\github\dotnet-docs-samples\iam\api\ServiceAccounts\ServiceAccountsRename.cs:line 36
   at GoogleCloudSamples.ServiceAccountTests.TestServiceAccounts() in C:\tmpfs\src\github\dotnet-docs-samples\iam\api\ServiceAccountTests\ServiceAccountTests.cs:line 20

@amanda-tarafa amanda-tarafa mentioned this issue Jun 27, 2023
Merged
1 task
@amanda-tarafa amanda-tarafa mentioned this issue Jul 7, 2023
Merged
1 task
@amanda-tarafa
Copy link
Member Author

This is definetely because of tests overstepping each other, see here and here which executed roughly at the same time as part of #2281 checks:

Failed GoogleCloudSamples.ServiceAccountTests.TestServiceAccounts [933 ms]
  Error Message:
   Google.GoogleApiException : The service iam has thrown an exception. HttpStatusCode is NotFound. Unknown service account
  Stack Trace:
     at Google.Apis.Requests.ClientServiceRequest`1.ParseResponse(HttpResponseMessage response)
   at Google.Apis.Requests.ClientServiceRequest`1.Execute()
   at ServiceAccounts.RenameServiceAccount(String email, String newDisplayName) in /tmpfs/src/github/dotnet-docs-samples/iam/api/ServiceAccounts/ServiceAccountsRename.cs:line 36
   at GoogleCloudSamples.ServiceAccountTests.TestServiceAccounts() in /tmpfs/src/github/dotnet-docs-samples/iam/api/ServiceAccountTests/ServiceAccountTests.cs:line 20

Failed GoogleCloudSamples.ServiceAccountTests.TestServiceAccounts [484 ms]
  Error Message:
   Google.GoogleApiException : The service iam has thrown an exception. HttpStatusCode is Conflict. Service account dotnet-test-8 already exists within project projects/dotnet-docs-samples-tests.
  Stack Trace:
     at Google.Apis.Requests.ClientServiceRequest`1.ParseResponse(HttpResponseMessage response)
   at Google.Apis.Requests.ClientServiceRequest`1.Execute()
   at ServiceAccounts.CreateServiceAccount(String projectId, String name, String displayName) in C:\tmpfs\src\github\dotnet-docs-samples\iam\api\ServiceAccounts\ServiceAccountsCreate.cs:line 42
   at GoogleCloudSamples.ServiceAccountTests.TestServiceAccounts() in C:\tmpfs\src\github\dotnet-docs-samples\iam\api\ServiceAccountTests\ServiceAccountTests.cs:line 18

@amanda-tarafa amanda-tarafa mentioned this issue Jul 18, 2023
Merged
1 task
@amanda-tarafa amanda-tarafa mentioned this issue Aug 2, 2023
Merged
1 task
@Rishabh-V Rishabh-V mentioned this issue Oct 31, 2023
Merged
1 task
@amanda-tarafa amanda-tarafa mentioned this issue Dec 9, 2023
Merged
1 task
@amanda-tarafa
Copy link
Member Author

Another flake #2438

[xUnit.net 00:00:04.24]     GoogleCloudSamples.AccessTest.TestAccess [FAIL]
  Failed GoogleCloudSamples.AccessTest.TestAccess [2 s]
  Error Message:
   Google.GoogleApiException : The service cloudresourcemanager has thrown an exception. HttpStatusCode is Conflict. There were concurrent policy changes. Please retry the whole read-modify-write with exponential backoff. The request's ETag '\007\006\016\221\005\363y\335' did not match the current policy's ETag '\007\006\016\221\006\"\031\347'.
  Stack Trace:
     at Google.Apis.Requests.ClientServiceRequest`1.ParseResponse(HttpResponseMessage response)
   at Google.Apis.Requests.ClientServiceRequest`1.Execute()
   at AccessManager.SetPolicy(String projectId, Policy policy) in C:\tmpfs\src\github\dotnet-docs-samples\iam\api\Access\SetPolicy.cs:line 33
   at GoogleCloudSamples.AccessTest.TestAccess() in C:\tmpfs\src\github\dotnet-docs-samples\iam\api\AccessTest\AccessTest.cs:line 121

@amanda-tarafa
Copy link
Member Author

Again in #2596

[xUnit.net 00:00:01.81]     GoogleCloudSamples.ServiceAccountTests.TestServiceAccounts [FAIL]
  Failed GoogleCloudSamples.ServiceAccountTests.TestServiceAccounts [1 s]
  Error Message:
   Google.GoogleApiException : The service iam has thrown an exception. HttpStatusCode is Conflict. Service account dotnet-test-594 already exists within project projects/dotnet-docs-samples-tests.
  Stack Trace:
     at Google.Apis.Requests.ClientServiceRequest`1.ParseResponse(HttpResponseMessage response)
   at Google.Apis.Requests.ClientServiceRequest`1.Execute()
   at ServiceAccounts.CreateServiceAccount(String projectId, String name, String displayName) in C:\tmpfs\src\github\dotnet-docs-samples\iam\api\ServiceAccounts\ServiceAccountsCreate.cs:line 42
   at GoogleCloudSamples.ServiceAccountTests.TestServiceAccounts() in C:\tmpfs\src\github\dotnet-docs-samples\iam\api\ServiceAccountTests\ServiceAccountTests.cs:line 18

@amanda-tarafa amanda-tarafa mentioned this issue Apr 19, 2024
Merged
1 task
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
api: iam Issues related to the Identity and Access Management API. priority: p2 Moderately-important priority. Fix may not be included in next release. samples Issues that are directly related to samples. type: process A process-related concern. May include testing, release, or the like.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@amanda-tarafa and others