Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Modyfying Azure ad group members causes memory spike and crash #1507

Closed
jonashoye opened this issue Sep 30, 2024 · 7 comments
Closed

Modyfying Azure ad group members causes memory spike and crash #1507

jonashoye opened this issue Sep 30, 2024 · 7 comments

Comments

@jonashoye
Copy link

Community Note

  • Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritise this request
  • Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritise the request
  • If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform (and AzureAD Provider) Version

  • Terraform Version 1.9.6
  • provider registry.terraform.io/hashicorp/azuread v3.0.1 & 3.0.0

Affected Resource(s)

  • azuread_group.example

Terraform Configuration Files

# Copy-paste your Terraform configurations here - for large Terraform configs,
# please use a service like Dropbox and share a link to the ZIP file. For
# security, you can also encrypt the files using our GPG public key: https://keybase.io/hashicorp

Debug Output

Panic Output

Expected Behavior

We have several groups managed by terraform. However after upgrading to v.3 it's not able to add new members to the groups. We use the members block within the azuread_group resource, and plan shows up nicely. However, when trying to apply, it seems to stall and hug up an insane amount of memory (on my local machine i've seen it hug up 32 gigs), and is not able to carry out the changes, but is stuck on 'Still modyfying...'.--
I've looked in the API calls and it seems the http PATCH call gets a 204 No Content response, i don't know if this is intended, or helpful.

Actual Behavior

Steps to Reproduce

  1. terraform apply

Important Factoids

References

  • #0000
@jonashoye jonashoye changed the title Modyfying Azure ad group members causes memory spike Modyfying Azure ad group members causes memory spike and crash Sep 30, 2024
@manicminer
Copy link
Contributor

Thanks for reporting @jonashoye, I'll look into this and try to mitigate.

@nijin-spacege
Copy link

Having the similar issue with azuread_group terraform resource with multiple members.

Here is the difference we can see with the v2.53.1 vs v3.0.1.

With Provider Version: v3.0.1

# azuread_group.team_ad_group will be updated in-place
~ resource "azuread_group" "team_ad_group" {
      id                         = "/groups/d3be5fc3-5549-43b8-a483-cdd4fca7278c"
    ~ members                    = [
        + "683a2aac-642b-4dc9-b085-3c049debbe04",
          # (29 unchanged elements hidden)
      ]
      # (20 unchanged attributes hidden)
  }

With Provider Version: v2.53.1

# azuread_group.team_ad_group will be updated in-place
~ resource "azuread_group" "team_ad_group" {
      id                         = "d3be5fc3-5549-43b8-a483-cdd4fca7278c"
    ~ members                    = [
        + "683a2aac-642b-4dc9-b085-3c049debbe04",
          # (29 unchanged elements hidden)
      ]
      # (20 unchanged attributes hidden)
  }

With the latest v3.0.1, There is a /groups/ prefix added as part of the Azure group ID value. It might be causing this issue.

@alaskascooter
Copy link

Similar issue when simply creating an empty group in govcloud:

Stack trace from the terraform-provider-azuread_v3.0.1_x5.exe plugin:

panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xc0000005 code=0x0 addr=0x8 pc=0x1f3fe2e]

goroutine 25 [running]:
github.com/hashicorp/terraform-provider-azuread/internal/services/groups.groupResourceCreate({0x2dd0f18, 0xc00062a000}, 0xc000072080, {0x22e5980, 0xc0003d4908})
github.com/hashicorp/terraform-provider-azuread/internal/services/groups/group_resource.go:684 +0x37ce
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*Resource).create(0xc0004a0b60, {0x2dd0e70, 0xc00037fbf0}, 0xc000072080, {0x22e5980, 0xc0003d4908})
github.com/hashicorp/terraform-plugin-sdk/[email protected]/helper/schema/resource.go:806 +0x119
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*Resource).Apply(0xc0004a0b60, {0x2dd0e70, 0xc00037fbf0}, 0xc00037ca90, 0xc00035f100, {0x22e5980, 0xc0003d4908})
github.com/hashicorp/terraform-plugin-sdk/[email protected]/helper/schema/resource.go:937 +0xa89
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*GRPCProviderServer).ApplyResourceChange(0xc000456000, {0x2dd0e70?, 0xc00037fb30?}, 0xc000716050)
github.com/hashicorp/terraform-plugin-sdk/[email protected]/helper/schema/grpc_provider.go:1153 +0xd5c
github.com/hashicorp/terraform-plugin-go/tfprotov5/tf5server.(*server).ApplyResourceChange(0xc00045e000, {0x2dd0e70?, 0xc00037f170?}, 0xc0001c2310)
github.com/hashicorp/[email protected]/tfprotov5/tf5server/server.go:865 +0x3d0
github.com/hashicorp/terraform-plugin-go/tfprotov5/internal/tfplugin5._Provider_ApplyResourceChange_Handler({0x27eae60, 0xc00045e000}, {0x2dd0e70, 0xc00037f170}, 0xc00035e800, 0x0)
github.com/hashicorp/[email protected]/tfprotov5/internal/tfplugin5/tfplugin5_grpc.pb.go:545 +0x1a6
google.golang.org/grpc.(*Server).processUnaryRPC(0xc00014ae00, {0x2dd0e70, 0xc00037f0e0}, {0x2dd7320, 0xc000432340}, 0xc0004eeb40, 0xc000430ff0, 0x3ca2db8, 0x0)
google.golang.org/[email protected]/server.go:1394 +0xe49
google.golang.org/grpc.(*Server).handleStream(0xc00014ae00, {0x2dd7320, 0xc000432340}, 0xc0004eeb40)
google.golang.org/[email protected]/server.go:1805 +0xe8b
google.golang.org/grpc.(*Server).serveStreams.func2.1()
google.golang.org/[email protected]/server.go:1029 +0x8b
created by google.golang.org/grpc.(*Server).serveStreams.func2 in goroutine 46
google.golang.org/[email protected]/server.go:1040 +0x125

Error: The terraform-provider-azuread_v3.0.1_x5.exe plugin crashed!

@sunny10464
Copy link

Hi,
we are also getting the same issue on versions 3.0.0 and 3.0.1. we tried moving back to the older working version v2.53.1 but it's giving a different error now that all the old AAD/Entra ID groups which were created before are using the newer 3.0.x versions now, which is not correct, so we are completely blocked now, any idea when we can get a fix for this issue please ?

@anordby
Copy link

anordby commented Oct 3, 2024

We are having this issue also. For only a few groups members changes, provider uses up all memory (32 GB) on our Terraform runners before exploding (process getting terminated).
image

@stijndehaes
Copy link

I notice the same issue, the update actually never finishes even

@mbfrahry
Copy link
Member

mbfrahry commented Oct 4, 2024

Hey all! We found both the crash and the memory leak. It looks like owners had the same issue and that has been fixed and released with 3.0.2. Thanks for all the detailed information left here

@mbfrahry mbfrahry closed this as completed Oct 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

8 participants