llm-arch-research

mirror of https://github.com/pese-git/llm-arch-research.git synced 2026-01-24 13:32:08 +00:00

Files

Sergey Penkovsky 211adf574c refactor: extract LLaMA components to separate modules in core directory

- Moved GELU, RMSNorm, RoPE, SiLU, and SwiGLU implementations from llama.py to dedicated files in core/
- Updated feed_forward.py to use new modular components
- Modified llama.py to import components from core modules instead of local definitions
- Improved code organization and reusability of activation functions and normalization layers

This refactor enables better code reuse across different model architectures and follows the single responsibility principle.

2025-10-06 14:09:19 +03:00