Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Optim - added quantization code. #968

Open
wants to merge 70 commits into
base: main
Choose a base branch
from
Open

Optim - added quantization code. #968

wants to merge 70 commits into from

Commits on Dec 13, 2023

  1. Add inference benchmark, delete extra files

    @gtamer2
    gtamer2 committed Dec 13, 2023
    Configuration menu
    Copy the full SHA
    236b696 View commit details
    Browse the repository at this point in the history

  2. more deletions

    @gtamer2
    gtamer2 committed Dec 13, 2023
    Configuration menu
    Copy the full SHA
    ed186f5 View commit details
    Browse the repository at this point in the history

  3. Merge pull request #1 from gtamer2/add_benchmark 

    Add benchmark
    @gtamer2
    gtamer2 committed Dec 13, 2023
    Configuration menu
    Copy the full SHA
    64ab5a8 View commit details
    Browse the repository at this point in the history

  4. Changes

    @gtamer2
    gtamer2 committed Dec 13, 2023
    Configuration menu
    Copy the full SHA
    62f242d View commit details
    Browse the repository at this point in the history

  5. import fire

    @gtamer2
    gtamer2 committed Dec 13, 2023
    Configuration menu
    Copy the full SHA
    257962f View commit details
    Browse the repository at this point in the history

  6. git ignore

    @gtamer2
    gtamer2 committed Dec 13, 2023
    Configuration menu
    Copy the full SHA
    a89488a View commit details
    Browse the repository at this point in the history

  7. fixes

    @gtamer2
    gtamer2 committed Dec 13, 2023
    Configuration menu
    Copy the full SHA
    5a84145 View commit details
    Browse the repository at this point in the history

  8. Push changes

    @gtamer2
    gtamer2 committed Dec 13, 2023
    Configuration menu
    Copy the full SHA
    cf2b652 View commit details
    Browse the repository at this point in the history

  9. comment out omdel.to

    @gtamer2
    gtamer2 committed Dec 13, 2023
    Configuration menu
    Copy the full SHA
    47400da View commit details
    Browse the repository at this point in the history

  10. print x for debugging

    @gtamer2
    gtamer2 committed Dec 13, 2023
    Configuration menu
    Copy the full SHA
    4c4d439 View commit details
    Browse the repository at this point in the history

  11. inference_benchmamrk

    @gtamer2
    gtamer2 committed Dec 13, 2023
    Configuration menu
    Copy the full SHA
    2071539 View commit details
    Browse the repository at this point in the history

  12. inference_benchmamrk

    @gtamer2
    gtamer2 committed Dec 13, 2023
    Configuration menu
    Copy the full SHA
    58eee14 View commit details
    Browse the repository at this point in the history

  13. Try large batch

    @gtamer2
    gtamer2 committed Dec 13, 2023
    Configuration menu
    Copy the full SHA
    86ca1c7 View commit details
    Browse the repository at this point in the history

  14. Get workign for 1 batch

    @gtamer2
    gtamer2 committed Dec 13, 2023
    Configuration menu
    Copy the full SHA
    f146895 View commit details
    Browse the repository at this point in the history

  15. Get workign for 1 batch

    @gtamer2
    gtamer2 committed Dec 13, 2023
    Configuration menu
    Copy the full SHA
    fd8a6cf View commit details
    Browse the repository at this point in the history

Commits on Dec 14, 2023

  1. Add torch profiler

    @gtamer2
    gtamer2 committed Dec 14, 2023
    Configuration menu
    Copy the full SHA
    04cb363 View commit details
    Browse the repository at this point in the history

  2. Indent

    @gtamer2
    gtamer2 committed Dec 14, 2023
    Configuration menu
    Copy the full SHA
    40abad2 View commit details
    Browse the repository at this point in the history

  3. Indent

    @gtamer2
    gtamer2 committed Dec 14, 2023
    Configuration menu
    Copy the full SHA
    d6336fc View commit details
    Browse the repository at this point in the history

  4. Profiel cpu and cuda

    @gtamer2
    gtamer2 committed Dec 14, 2023
    Configuration menu
    Copy the full SHA
    f80cf22 View commit details
    Browse the repository at this point in the history

  5. move profiler down

    @gtamer2
    gtamer2 committed Dec 14, 2023
    Configuration menu
    Copy the full SHA
    8436d96 View commit details
    Browse the repository at this point in the history

  6. Try using simplified llama

    @gtamer2
    gtamer2 committed Dec 14, 2023
    Configuration menu
    Copy the full SHA
    60bc868 View commit details
    Browse the repository at this point in the history

  7. revert

    @gtamer2
    gtamer2 committed Dec 14, 2023
    Configuration menu
    Copy the full SHA
    84416a4 View commit details
    Browse the repository at this point in the history

  8. Move outside

    @gtamer2
    gtamer2 committed Dec 14, 2023
    Configuration menu
    Copy the full SHA
    7dfe9e9 View commit details
    Browse the repository at this point in the history

  9. one more yolo

    @gtamer2
    gtamer2 committed Dec 14, 2023
    Configuration menu
    Copy the full SHA
    e5f0408 View commit details
    Browse the repository at this point in the history

  10. rerecord mem

    @gtamer2
    gtamer2 committed Dec 14, 2023
    Configuration menu
    Copy the full SHA
    d7da6e1 View commit details
    Browse the repository at this point in the history

  11. Merge pull request #2 from gtamer2/benchmark2 

    Benchmark2
    @gtamer2
    gtamer2 committed Dec 14, 2023
    Configuration menu
    Copy the full SHA
    b988f0e View commit details
    Browse the repository at this point in the history

  12. added torch.jit.script to model in generation.py

    Guru Sharma committed Dec 14, 2023
    Configuration menu
    Copy the full SHA
    431ebab View commit details
    Browse the repository at this point in the history

  13. Script to run benchmarks

    @gtamer2
    gtamer2 committed Dec 14, 2023
    Configuration menu
    Copy the full SHA
    67d5a14 View commit details
    Browse the repository at this point in the history

  14. Fix missing param

    @gtamer2
    gtamer2 committed Dec 14, 2023
    Configuration menu
    Copy the full SHA
    dcda8a1 View commit details
    Browse the repository at this point in the history

  15. tried changing .jit.script to be around the llama object call in infe… 

    …rence benchmark
    Guru Sharma committed Dec 14, 2023
    Configuration menu
    Copy the full SHA
    816b895 View commit details
    Browse the repository at this point in the history

  16. Empty cuda cache

    @gtamer2
    gtamer2 committed Dec 14, 2023
    Configuration menu
    Copy the full SHA
    078aa9c View commit details
    Browse the repository at this point in the history

  17. changed to trace from script

    Guru Sharma committed Dec 14, 2023
    Configuration menu
    Copy the full SHA
    47aaa29 View commit details
    Browse the repository at this point in the history

  18. .trace on generation.py

    Guru Sharma committed Dec 14, 2023
    Configuration menu
    Copy the full SHA
    e8f2d14 View commit details
    Browse the repository at this point in the history

  19. get rid of trace entirely to see if it is causing issue

    Guru Sharma committed Dec 14, 2023
    Configuration menu
    Copy the full SHA
    c18ee45 View commit details
    Browse the repository at this point in the history

  20. Add torchscript python script

    @gtamer2
    gtamer2 committed Dec 14, 2023
    Configuration menu
    Copy the full SHA
    a79d31a View commit details
    Browse the repository at this point in the history

  21. reset

    @gtamer2
    gtamer2 committed Dec 14, 2023
    Configuration menu
    Copy the full SHA
    680d2a6 View commit details
    Browse the repository at this point in the history

  22. remove bard hallucination

    @gtamer2
    gtamer2 committed Dec 14, 2023
    Configuration menu
    Copy the full SHA
    aeab877 View commit details
    Browse the repository at this point in the history

  23. Merge pull request #3 from gtamer2/new_benchmarks 

    Script to run benchmarks
    @gtamer2
    gtamer2 committed Dec 14, 2023
    Configuration menu
    Copy the full SHA
    93d16e2 View commit details
    Browse the repository at this point in the history

  24. Merge branch 'main' of https://github.com/gtamer2/hpml_llama into tor… 

    …chscript
    @gtamer2
    gtamer2 committed Dec 14, 2023
    Configuration menu
    Copy the full SHA
    10188e4 View commit details
    Browse the repository at this point in the history

  25. revisions

    @gtamer2
    gtamer2 committed Dec 14, 2023
    Configuration menu
    Copy the full SHA
    c588e99 View commit details
    Browse the repository at this point in the history

  26. revisions

    @gtamer2
    gtamer2 committed Dec 14, 2023
    Configuration menu
    Copy the full SHA
    714dd19 View commit details
    Browse the repository at this point in the history

  27. revisions

    @gtamer2
    gtamer2 committed Dec 14, 2023
    Configuration menu
    Copy the full SHA
    0abe652 View commit details
    Browse the repository at this point in the history

  28. Add benchmarks

    @gtamer2
    gtamer2 committed Dec 14, 2023
    Configuration menu
    Copy the full SHA
    5534bbf View commit details
    Browse the repository at this point in the history

  29. get quantization working

    Guru Sharma committed Dec 14, 2023
    Configuration menu
    Copy the full SHA
    44feb11 View commit details
    Browse the repository at this point in the history

  30. Merge pull request #5 from gtamer2/torchscript 

    Torchscript
    @gtamer2
    gtamer2 committed Dec 14, 2023
    Configuration menu
    Copy the full SHA
    ae535a2 View commit details
    Browse the repository at this point in the history

  31. add some more quantization lines

    Guru Sharma committed Dec 14, 2023
    Configuration menu
    Copy the full SHA
    8e34266 View commit details
    Browse the repository at this point in the history

  32. got rid of fuse_model()

    Guru Sharma committed Dec 14, 2023
    Configuration menu
    Copy the full SHA
    94c70fb View commit details
    Browse the repository at this point in the history

  33. adding convert to quantization model

    Guru Sharma committed Dec 14, 2023
    Configuration menu
    Copy the full SHA
    3920ee5 View commit details
    Browse the repository at this point in the history

  34. quantize script

    @gtamer2
    gtamer2 committed Dec 14, 2023
    Configuration menu
    Copy the full SHA
    adb0190 View commit details
    Browse the repository at this point in the history

Commits on Dec 15, 2023

  1. fire launch

    @gtamer2
    gtamer2 committed Dec 15, 2023
    Configuration menu
    Copy the full SHA
    d71ffa0 View commit details
    Browse the repository at this point in the history

  2. Inplace

    @gtamer2
    gtamer2 committed Dec 15, 2023
    Configuration menu
    Copy the full SHA
    748bc32 View commit details
    Browse the repository at this point in the history

  3. access the transformer

    @gtamer2
    gtamer2 committed Dec 15, 2023
    Configuration menu
    Copy the full SHA
    775475a View commit details
    Browse the repository at this point in the history

  4. New quant logic

    @gtamer2
    gtamer2 committed Dec 15, 2023
    Configuration menu
    Copy the full SHA
    165cd54 View commit details
    Browse the repository at this point in the history

  5. rge branch 'main' of https://github.com/gtamer2/hpml_llama into optim

    @gtamer2
    gtamer2 committed Dec 15, 2023
    Configuration menu
    Copy the full SHA
    005d8c5 View commit details
    Browse the repository at this point in the history

  6. fix quant sample inf

    @gtamer2
    gtamer2 committed Dec 15, 2023
    Configuration menu
    Copy the full SHA
    c995e43 View commit details
    Browse the repository at this point in the history

  7. Fix args

    @gtamer2
    gtamer2 committed Dec 15, 2023
    Configuration menu
    Copy the full SHA
    575c197 View commit details
    Browse the repository at this point in the history

  8. move h=quant(h) to after firist layer. It was operating on tokens whi… 

    …cch are Half, not float
    @gtamer2
    gtamer2 committed Dec 15, 2023
    Configuration menu
    Copy the full SHA
    787efd5 View commit details
    Browse the repository at this point in the history

  9. added a sample for pruning model

    Guru Sharma committed Dec 15, 2023
    Configuration menu
    Copy the full SHA
    d0b8b39 View commit details
    Browse the repository at this point in the history

  10. maybe it prunes now?

    Guru Sharma committed Dec 15, 2023
    Configuration menu
    Copy the full SHA
    fab7b44 View commit details
    Browse the repository at this point in the history

  11. added attention

    Guru Sharma committed Dec 15, 2023
    Configuration menu
    Copy the full SHA
    1a85f36 View commit details
    Browse the repository at this point in the history

  12. add torch.nn.parameter to this

    Guru Sharma committed Dec 15, 2023
    Configuration menu
    Copy the full SHA
    0bf73c4 View commit details
    Browse the repository at this point in the history

  13. added some changes for including torch.nn.Parameter into the pruning,… 

    … created my own attribute
    Guru Sharma committed Dec 15, 2023
    Configuration menu
    Copy the full SHA
    ac4fb51 View commit details
    Browse the repository at this point in the history

  14. get rid of quantization modifications in model.py

    Guru Sharma committed Dec 15, 2023
    Configuration menu
    Copy the full SHA
    caf7482 View commit details
    Browse the repository at this point in the history

  15. checking sparsity

    Guru Sharma committed Dec 15, 2023
    Configuration menu
    Copy the full SHA
    b790b37 View commit details
    Browse the repository at this point in the history

  16. trying to find sparsity again

    Guru Sharma committed Dec 15, 2023
    Configuration menu
    Copy the full SHA
    1c13854 View commit details
    Browse the repository at this point in the history

  17. we got rid of sparsity

    Guru Sharma committed Dec 15, 2023
    Configuration menu
    Copy the full SHA
    bf0ebe0 View commit details
    Browse the repository at this point in the history

  18. giving sparsity another try

    Guru Sharma committed Dec 15, 2023
    Configuration menu
    Copy the full SHA
    cb43151 View commit details
    Browse the repository at this point in the history

  19. trying newer things with prune_model.py

    Guru Sharma committed Dec 15, 2023
    Configuration menu
    Copy the full SHA
    6779933 View commit details
    Browse the repository at this point in the history

  20. got rid of print statements, left them until after it was done

    Guru Sharma committed Dec 15, 2023
    Configuration menu
    Copy the full SHA
    660e231 View commit details
    Browse the repository at this point in the history

  21. trying to make sure layer is being modified in place

    Guru Sharma committed Dec 15, 2023
    Configuration menu
    Copy the full SHA
    651f8fb View commit details
    Browse the repository at this point in the history