Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rust-commit2-worker #1487

Closed
marco-storswift opened this issue Mar 23, 2023 · 2 comments
Closed

rust-commit2-worker #1487

marco-storswift opened this issue Mar 23, 2023 · 2 comments
Assignees

Comments

@marco-storswift
Copy link

1. What is your project, and what problem does it solve? (max 100 words)

lotus-worker is a performance-sensitive application. The current version of lotus-worker is using cgo to call rust libraries.
Which is a huge performance burden. the commit2 phase is independent.
so we develop a rust-commit2-worker

2. Project links

Link to Github repo:https://github.com/storswiftlabs/rust-commit2-worker
Link to demo or website:
License: MIT

Here are the disadvantages of using cgo:

  • Cgo overhead, every time you are calling a cgo function, the go runtime needs to switch stack between go and rust, which will cause performance issues.
  • Calling cgo is unsafe, golang and c/rust have different memory models and object lifecycles, this can lead us to the risk of memory leaks.
  • As mentioned above, golang and c/rust have different memory models, we need more wrapper codes to pass complex data structures, and make the code structures more complex. Also, copying structures from go to c will increase memory usage.
  • Not all codes under rust-filecoin-ffi are required, but, by calling cgo, we need to bundle rust lib as an entirety, this will increase the build output size.

3. a) How is IPFS, Filecoin, or related technology used in this project?

We plan to write rust-commit2-worker as a rust implementation of lotus-worker to solve these problems, which will bring these features:

  • Fully compatible with the current go version of lotus-worker
  • Smaller size and clearer code structure since we don't need to bundle all code under filecoin-ffi and write wrappers
  • Better performance without cgo overhead
  • Memory safety

b) Is this project building with the current microgrants focus area (FVM)? (Yes or No)

NO

4. How will you improve your project with this grant? What steps will you take to meet this objective?

Number Grant Deliverable Briefly describe how you will meet deliverable objectives Timeframe (within 3 months)
1. commit2 flow chart Be familiar with the lotus-worker commit2 process 2 weeks
2. rust-commit2-worker develop rust-commit2-worker, test, doc 4 weeks
3.
4.

5. If accepted, do you agree to share monthly project updates in this Github issue for 3 months or until the project described here is complete?

Yes

6. Does your proposal comply with our Community Code of Conduct?

Yes

7. Links and submissions

  • If your project began at a hackathon, have you submitted it for the relevant Protocol Labs prizes? Include links here if available:

Additional questions:

  • For each team member(s), please list name, email, Github account, and role in the project.

Our team start to research and develop Web3 two years ago. We have implemented the optimized version of the PoREP/PoST code,
worker task scheduling code, DevOps management system, and so on. From the very beginning,
we have been focused on Filecoin security solutions. We research offline signature,
cold wallet and multi-sig solutions. We also develop a framework to ensure the security of DevOps for Filecoin clusters.

  • How did you learn about our microgrant program?
  • If your project was created as part of an event or hackathon:
    • What was the name of the event? (e.g. ETHGlobal NFTHack, Cal Hacks hello:world, Chainlink, CivHacks, GameDevJ, ETHGlobal Scaling Ethereum)
    • Please link to your hackathon submission
@vmx
Copy link
Contributor

vmx commented Mar 23, 2023

This sounds related to my work at filecoin-project/rust-fil-proofs#1678.

@eshon eshon added fvm and removed fvm labels Apr 18, 2023
@ErinOCon
Copy link
Collaborator

ErinOCon commented Jun 7, 2023

Hi @marco-storswift, thank you for your proposal and for your patience. At this time we are only considering microgrants that align with specific focus areas. If your project aligns with an assigned focus area in the future, please feel welcome to re-apply. Wishing you all the best as you continue to build!

@ErinOCon ErinOCon closed this as completed Jun 7, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants