mirror of
https://github.com/pese-git/llm-arch-research.git
synced 2026-01-23 21:10:54 +00:00
feat: add RoPE positional embeddings implementation in llama.ipynb
- Implement Rotary Positional Embeddings (RoPE) with separate cosine/sine components - Add vectorized computation of inverse frequencies for RoPE - Include tensor slicing utilities for even/odd column separation - Update dependencies in pyproject.toml and uv.lock
This commit is contained in:
@@ -12,6 +12,8 @@ dependencies = [
|
||||
"llm",
|
||||
"tqdm>=4,<5",
|
||||
"ipykernel",
|
||||
"torch==2.8.0",
|
||||
"matplotlib==3.10.6",
|
||||
]
|
||||
|
||||
[project.optional-dependencies]
|
||||
|
||||
Reference in New Issue
Block a user