adding GQA

[minowau]

Implementation by optimizing memory usage and performance for low-resource environments. Key updates include the integration of grouped query attention, modifications to the tokenizer for better encoding and decoding, and improvements to the text generation logic using nucleus sampling. Additionally, the code structure has been refined with comprehensive documentation, ensuring clarity and maintainability. Initial tests have been conducted to validate the overall functionality of the updated components.

Enhancements to Transformer Model Implementation

• Transformer Model (Transformer class):

  • Implemented grouped query attention to optimize memory usage.
  • Adjusted the forward method to handle dynamic token lengths.

• Transformer Block (TransformerBlock class):

  • Updated attention and feedforward layers for improved performance.

• Attention Module (Attention class):

  • Integrated grouped query attention and adjusted key/value caching mechanisms.

• Tokenizer (Tokenizer class):

  • Modified the encoding and decoding processes using SentencePiece.
  • Ensured proper handling of special tokens: beginning-of-sequence (BOS), end-of-sequence (EOS), and padding (PAD).

• Generation Method (generate function):

  • Enhanced logic to support dynamic input lengths.
  • Implemented nucleus sampling with adjustable temperature and top-p parameters for better control over text generation.
  • Improved handling of log probabilities and early stopping conditions based on EOS tokens.

• Documentation and Code Structure:

  • Added detailed docstrings and comments for clarity and maintainability.
  • Ensured consistent formatting throughout the codebase.

• Testing and Validation:

  • Conducted initial tests to validate the functionality of the model, tokenizer, and generation processes.

[minowau]

@msaroufim could you please check this.Just a request.This is a new architecture

[minowau]

@jspisak Just want you to review This