penkovsky_sa/llm-arch-research
1
0
Fork 0
mirror of https://github.com/pese-git/llm-arch-research.git synced 2026-01-23 21:10:54 +00:00
Code Issues Packages Projects Releases Wiki Activity
65 Commits 8 Branches 0 Tags
master
Commit Graph

1 Commits

Author SHA1 Message Date
Sergey Penkovsky
ec0d2bd8d0 feat(mistral): add Mistral model implementation and configs 
- implement Mistral model in llm/models/mistral/mistral.py with GroupedQueryAttention, SwiGLU, RoPE, sliding window attention
- add __init__.py for module export
- add config files for mistral training and generation
- update universal experiment runner to support Mistral model
- add notebook for Mistral experiments
 2025-10-14 14:53:45 +03:00