Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use pickle to cache AST parse; ~25% speedup @ O(100) specs #92

Open
PhilHarnish opened this issue Dec 24, 2017 · 0 comments
Open

Use pickle to cache AST parse; ~25% speedup @ O(100) specs #92

PhilHarnish opened this issue Dec 24, 2017 · 0 comments

Comments

@PhilHarnish
Copy link

Consider adding a pkl cache to mamba's transformations.

My nasty, massive repo has the following speedup when using pkl:
Before: 853 examples ran (156 pending) in 1.5438 seconds
After: 853 examples ran (156 pending) in 1.1454 seconds
(~25% faster)

My diff looks like this:

    @pickle_cache.cache_from_file('mamba_spec')
    def _parse_and_transform_ast(self, path):
        with open(path) as f:
        # (...snip...)

Where pickle_cache.cache_from_file() is defined here:
https://github.com/PhilHarnish/forge/blob/master/src/data/pickle_cache.py
tl;dr: if path is unmodified then the AST is pulled from pkl.

Mamba's requirements are pretty straightforward (eg, function of only "path" and mamba's version number) and so it ought to be easy to accomplish in fewer lines.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant