Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

This pull request introduces library feature for leanify. #72

Open
wants to merge 12 commits into
base: master
Choose a base branch
from

Conversation

doterax
Copy link
Contributor

@doterax doterax commented May 12, 2021

Useful feature to increase optimization speed.

Use case one.

I have several projects that shares common resources. In my case five projects built for different platforms. That projects shares a lot of texture atlases.
And I need to leanify them all.
Leanifying one project takes 20 minutes.
For five projects it will take something around 100 minutes. This is bad.
With library feature, I need 20 minutes to fill library first time, following leanifying just reuse already compressed data and copy it.

Use case two.

I made pre-master build of web application and leanified it with library enabled. And during final test I find small bug and should rebuid application again.With library feature re-leanifying of my web application is fast as file copy.

Use case three.

I will be able to setup incremental leanifying of my projects just add leanify with enabled library in post build. First build will be slow, but next one will fast.

By default library feature is disabled.

To enable it use -l switch followed by path to library.

For example to leanify everything in current directory with custom library folder just use this command:

leanify -l c:\leanify_library .

or

leanify -l ~\leanify_library .

If you don't care about place for your library folder, just use * symbol

Example:

leanify -l * .

This will use global library folder in temp directory.

Full path to library will be printed in terminal.

WARNING! Library folder always grows, leanify never delete anything, you should clean it from time to time.

Bonus! Library feature also works in parallel mode.

Note:

Library use SHA1 hash of input file content to find already leanifyed data. So if you started leanifying with enabled library on folder that contains already leanifyed data, you did not noticed speed increase this is because library contains only hashes of source files. And this action doubles library size.

@doterax doterax changed the title This pull request introducing library feature for leanify. This pull request introduces library feature for leanify. May 12, 2021
@JayXon
Copy link
Owner

JayXon commented May 14, 2021

Hi, thanks for the PR, but I don't think this feature should be part of leanify, either whatever code calling leanify or a standalone wrapper is a more suitable place, something like ccache.

@doterax
Copy link
Contributor Author

doterax commented May 14, 2021

Thank you for answer. I also share same doubts about this feature in leanify. I played with self writed cache that use leanify to process. But after severall experiments I come up with idea that cache feature should be in leanify. This is best for overall processing speed, because no overhead for interprocess communications at all.

Also library feature is optional and won't break existing workflows.

So I will be happy to make it part of such great tool.

@JayXon
Copy link
Owner

JayXon commented May 16, 2021

I doubt the interprocess overhead matters much in this case.
Also this feature seems very specific to your use case, I doubt that this is useful to the general leanify user.
I don't think I've seen any precedence of such cache feature in any of the file optimization tools.

@doterax doterax force-pushed the feature/libraryOfProcessedFiles branch from 224b0ee to f52b850 Compare October 13, 2022 19:06
@doterax
Copy link
Contributor Author

doterax commented Oct 13, 2022

Hi @JayXon, I have merged master and fixed some conflicts. And it will be nice if you apply this PR. We have been used this feature in our company for a year+ and it greatly increased our productivity when we apply leanify to a built project. And no matter how huge is build, every next build and following leanifying is very fast.

Thank you in advance.

@doterax
Copy link
Contributor Author

doterax commented Dec 15, 2022

@JayXon can you reconsider your opinion on this PR?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants