Reading of groups fails on pipeline #1206
Comments
|
Robert, good catch. I can confirm that I have seen the same issue periodically in 0-bootstrap. It looks to be an eventually consistent issue with the API. If the error persists then the tfvars may not be set correctly or came in unreferenced. in one of my last runs to 5-app-infra in the fork Q1) I have a question, is your example obfuscated with 12345 or did you run with the defaults - as you should be seeing a generated group id like " groups/035nkun24jo9ze2 " not "groups/12345" - this would be a tfvars issue - I ask this because the example in your comment is all defaults for example.org, the billing id, org etc... |
|
Sorry, I did not mention a few things. I did obfuscate the logs, domains and other numbers. Everything is already set up from the gcloud cli, both the seed and gh projects exist in gcloud and all groups were created automatically. On step 30 https://github.com/terraform-google-modules/terraform-example-foundation/blob/master/0-bootstrap/README-GitHub.md I have run the pipelines over 20 times trying different extra roles for the service accounts and activating could identity on the two projects but with no success. Locally, I do not have any issue running terraform plan |
|
I also set up a separate landing zone before this where I created the groups manually and did not encounter this issue Somewhere there is a permission issue because it does not make sense for it to be an eventual consistency one. Why would "plan" and even "apply" work locally but not in the pipeline? Something with the service account/group that the pipeline uses is not right. That is my assumption. Did you change any additional roles/enabled apis in your run in bootstrap? |
|
Rest api works fine too if I want to get the details of the groups manually |
|
I have a lot of errors for "google.apps.cloudidentity.groups.v1beta1.GroupsService.GetGroup" but I have no idea how to see logs for them |
|
FTR we had the same problem last week. What we found was that groups created by an organization member were not accessible to the bootstrap service account. But if the service account creates the groups itself then it's all fine. During local boostrap the groups were created by a real user (a member of the organization), but when the plan was run by the service account (in github for us), the service account did not have permissions to read the groups. The workaround is to let the service account create the groups. In details, from the state where the groups are already created and
in
And push this to version control. The bootstrap service account is now able to run Perhaps this should be mentioned in the docs? HTH |
|
Is anyone else using cloud build in 0-bootstrap step? I've got everything working up to #17 and then ran into an error with my plan on cloud build, whereas terraform plan locally works just fine. Here's the error that I get on cloud build in case anyone else has seen this: |
That fixed it for me. Would love to know the root cause here and I could look into it, but I'm also dealing with another issue (#1273 )... |
|
I think I've identified the root cause now, this comes from a strange overlap between GCP services and Cloud Identity / Workspace services and different permission models. If the terraform code is used to create groups with the bootstrap service account, this configuration includes WITH_INITIAL_OWNER so that the service account is granted the privilege to modify the group. If the groups are created manually, then the bootstrap service account as configured does not have any permission over cloud identity. It would need a workspace admin role like Groups Admin, which is configured through Workspace, not the GCP IAM policies. Now I'm considering 2 options on how to address this in the foundation blueprint:
|
|
@eeaton Your options are way too advanced for me right now so I was looking for another approach. As I understand it, specifying the SAs when creating the (required? need optional as well?) groups, creates a circular dependency (group <-- seed project <-- sa <-- group). Would it be possible to add to the groups after the fact? I put Testing that solution, I'm having some obscure issue. Here's an extract |
|
@eeaton Silly me. I used the wrong key from the But then it's complaining that I switched to |
|
This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 7 days |
TL;DR
Locally, terraform init/plan/apply works flawlessly.
When using GithubActions, the pipeline fails with:
Expected behavior
For the pipeline to finish successfully
Observed behavior
No response
Terraform Configuration
Terraform Version
Additional information
No response
The text was updated successfully, but these errors were encountered: