Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix AC in T5 example #1273

Merged
merged 1 commit into from
Jun 29, 2024
Merged

Fix AC in T5 example #1273

merged 1 commit into from
Jun 29, 2024

Conversation

msaroufim
Copy link
Member

@msaroufim msaroufim commented Jun 29, 2024 •
edited
Loading

This fixes #1210

Enabling AC is enabling compile by default which is then failing to handle the cpu offloading flag for FSDP?

This is broken on 2.2, 2.3 and nightlies

cc @awgu @anijain2305

@netlify Netlify
Copy link

netlify bot commented Jun 29, 2024 •
edited
Loading

✅ Deploy Preview for pytorch-examples-preview canceled.

Name Link
🔨 Latest commit d16a5c2
🔍 Latest deploy log https://app.netlify.com/sites/pytorch-examples-preview/deploys/667fa9009bec590008d0a754

@msaroufim msaroufim mentioned this pull request Jun 29, 2024
Open
@msaroufim msaroufim merged commit 26de419 into main Jun 29, 2024
8 checks passed
@msaroufim msaroufim deleted the fix_checkpoint branch June 29, 2024 06:35
@awgu
Copy link
Contributor

awgu commented Jun 29, 2024

I think the arg signature changed for the checkpoint wrapper. We should remove this line:

examples/distributed/FSDP/policies/activation_checkpointing_functions.py

Line 16 in 26de419

offload_to_cpu=False,

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
cla signed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

FSDP T5 Example not working
3 participants
@msaroufim @awgu @facebook-github-bot