feat: add RoPE positional embeddings implementation in llama.ipynb

Browse Source

- Implement Rotary Positional Embeddings (RoPE) with separate cosine/sine components
- Add vectorized computation of inverse frequencies for RoPE
- Include tensor slicing utilities for even/odd column separation
- Update dependencies in pyproject.toml and uv.lock

...

This commit is contained in:

Sergey Penkovsky

2025-10-06 12:52:59 +03:00

parent b6f56a2640

commit 9898e8ee83

3 changed files with 1561 additions and 72 deletions

Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL

Download Patch File Download Diff File Expand all files Collapse all files

1108

notebooks/llama.ipynb

File diff suppressed because one or more lines are too long