Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update & Supplement with new custom optimizer #904

Open
wants to merge 14 commits into
base: main
Choose a base branch
from
Open

Update & Supplement with new custom optimizer #904

wants to merge 14 commits into from

Commits on Sep 29, 2024

  1. Add CAME Optimizer

    @tianshijing
    tianshijing authored Sep 29, 2024
    Configuration menu
    Copy the full SHA
    96dc67e View commit details
    Browse the repository at this point in the history

  2. Add Adafactor Optimizer

    @tianshijing
    tianshijing authored Sep 29, 2024
    Configuration menu
    Copy the full SHA
    9f75573 View commit details
    Browse the repository at this point in the history

  3. Add QHAdam Optimizer

    @tianshijing
    tianshijing authored Sep 29, 2024
    Configuration menu
    Copy the full SHA
    89afff8 View commit details
    Browse the repository at this point in the history

  4. Add QHM Optimizer

    @tianshijing
    tianshijing authored Sep 29, 2024
    Configuration menu
    Copy the full SHA
    5866ff8 View commit details
    Browse the repository at this point in the history

  5. Add RMSprop Optimizer

    @tianshijing
    tianshijing authored Sep 29, 2024
    Configuration menu
    Copy the full SHA
    21f5aeb View commit details
    Browse the repository at this point in the history

  6. Add SGD Optimizer

    @tianshijing
    tianshijing authored Sep 29, 2024
    Configuration menu
    Copy the full SHA
    f1e3974 View commit details
    Browse the repository at this point in the history

  7. Add SWATS Optimizer

    @tianshijing
    tianshijing authored Sep 29, 2024
    Configuration menu
    Copy the full SHA
    77b3619 View commit details
    Browse the repository at this point in the history

  8. Add Adahessian Optimizer

    @tianshijing
    tianshijing authored Sep 29, 2024
    Configuration menu
    Copy the full SHA
    5539993 View commit details
    Browse the repository at this point in the history

  9. Add AdamW Optimizer

    @tianshijing
    tianshijing authored Sep 29, 2024
    Configuration menu
    Copy the full SHA
    53c84e5 View commit details
    Browse the repository at this point in the history

  10. Add Lion Optimizer

    @tianshijing
    tianshijing authored Sep 29, 2024
    Configuration menu
    Copy the full SHA
    29f547b View commit details
    Browse the repository at this point in the history

  11. Update optimizers.py

    @tianshijing
    tianshijing authored Sep 29, 2024
    Configuration menu
    Copy the full SHA
    76d66a9 View commit details
    Browse the repository at this point in the history

  12. Update args.py

    @tianshijing
    tianshijing authored Sep 29, 2024
    Configuration menu
    Copy the full SHA
    27268f0 View commit details
    Browse the repository at this point in the history

  13. Update finetuner.py

    @tianshijing
    tianshijing authored Sep 29, 2024
    Configuration menu
    Copy the full SHA
    7f65cb7 View commit details
    Browse the repository at this point in the history

  14. Update run_finetune_with_custom_optim.sh

    @tianshijing
    tianshijing authored Sep 29, 2024
    Configuration menu
    Copy the full SHA
    577eb94 View commit details
    Browse the repository at this point in the history