Sergey Penkovsky cfb4b6dfb1 feat(gemma): initial implementation of Gemma model and configs ...

- Add core Gemma model (architecture, attention, GeGLU, RoPE, RMSNorm, etc)
- Add configs for training and generation: gemma_train.json, gemma_generate.json
- Add Gemma notebook for exploratory analysis and demonstration
- Add __init__.py for Gemma submodule
- Update run_llm_experiment.py to support Gemma experiment configs

test(gemma): add comprehensive unit tests for Gemma

- Test forward pass (with/without cache)
- Test autoregressive generation (greedy, top-k, top-p)
- Test shape correctness and max sequence length errors
- Test multi-layer stack and token embeddings

docs: add documentation notebook for Gemma usage and analysis

Closes: #issue (if applicable)

2025-10-21 01:02:15 +03:00

..

configs

feat(gemma): initial implementation of Gemma model and configs

2025-10-21 01:02:15 +03:00

run_llm_experiment.py

feat(gemma): initial implementation of Gemma model and configs

2025-10-21 01:02:15 +03:00