Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[I2V] implement image-to-video for open-sora-plan v1.2 #226

Open
oahzxl opened this issue Oct 4, 2024 · 3 comments
Open

[I2V] implement image-to-video for open-sora-plan v1.2 #226

oahzxl opened this issue Oct 4, 2024 · 3 comments
Assignees
Labels
enhancement New feature or request

Comments

@oahzxl
Copy link
Collaborator

oahzxl commented Oct 4, 2024

As requested in issue #217 and #179 , we want to implement image-to-video feature for open-sora-plan v1.2

Thanks @devtony10 for willing to contribute to this feature.

Open-Sora-Plan v1.2.0 provides a image-to-video feature here https://github.com/PKU-YuanGroup/Open-Sora-Plan/blob/main/opensora/sample/sample_inpaint.py .

The main difference between image-to-video and text-to-video is that the image-to-video introduce extra args for images here:
https://github.com/PKU-YuanGroup/Open-Sora-Plan/blob/294993ca78bf65dec1c3b6fb25541432c545eda9/opensora/sample/pipeline_inpaint.py#L621

I think we can add these new args and use if to determine whether to process images.

Can you help to add this feature to our Open-Sora-Plan pipeline? If so, can you first draft a simple plan (no need to write actual code) about code change?

@devtony10
Copy link

I'll get started on it pronto

@devtony10
Copy link

What of CogVideoX image to video?

@oahzxl
Copy link
Collaborator Author

oahzxl commented Oct 4, 2024

i think that should be same. maybe we can do open-sora-plan video first

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants