Pocket-science/ltx2-mps
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Textual

Browse Source
This commit is contained in:
Norbert Schmidt
2026-01-11 11:01:07 +01:00
parent c75e87b9be
commit 707f2447cf
3 changed files with 101 additions and 143 deletions
Download Patch File Download Diff File Expand all files Collapse all files

131
README.md
View File

@@ -1,134 +1,115 @@
# LTX-2 on Apple Silicon (MPS)
# ltx2-mps
Run [Lightricks LTX-2](https://huggingface.co/Lightricks/LTX-2) video generation on Mac with Apple Silicon using Metal Performance Shaders (MPS).
run [LTX-2](https://huggingface.co/Lightricks/LTX-2) video generation on mac using MPS (metal).
## The Problem
## what's this about
LTX-2 uses `float64` (double precision) for rotary position embeddings (RoPE), but Apple's MPS backend doesn't support float64 - only float32. This causes the error:
LTX-2 uses float64 for rotary position embeddings, but MPS doesn't support float64. you get this error:
```
TypeError: Cannot convert a MPS Tensor to float64 dtype as the MPS framework doesn't support float64
```
## The Solution
this repo patches diffusers to use float32 instead. works fine, no noticeable quality loss.
This repo provides a patch that forces `float32` for RoPE calculations. The quality difference is negligible, and it enables LTX-2 to run on Mac.
## requirements
## Requirements
- mac with apple silicon (m1/m2/m3/m4)
- python 3.11+
- 64GB+ ram recommended (model is ~40GB)
- **macOS** with Apple Silicon (M1, M2, M3, M4 - any variant)
- **Python 3.11+**
- **64GB+ RAM recommended** (model is ~40GB, 128GB ideal for max settings)
- **PyTorch 2.0+**
## Quick Start
## setup
```bash
# 1. Clone this repo
git clone https://github.com/YOUR_USERNAME/ltx2-mps.git
git clone https://github.com/Pocket-science/ltx2-mps.git
cd ltx2-mps
# 2. Create virtual environment
python3 -m venv venv
source venv/bin/activate
# 3. Install dependencies
pip install torch torchvision torchaudio
pip install git+https://github.com/huggingface/diffusers.git
pip install transformers accelerate safetensors sentencepiece
pip install imageio imageio-ffmpeg
# 4. Apply MPS patches
python patch_mps.py
# 5. Generate a video!
python generate.py "A cat walking through grass" -o output.mp4
```
## Usage
## usage
```bash
python generate.py "Your prompt here" -o output.mp4 [options]
python generate.py "a cat walking through grass" -o output.mp4
```
### Options
### options
| Option | Default | Description |
|--------|---------|-------------|
| `--width` | 512 | Video width (must be divisible by 32) |
| `--height` | 320 | Video height (must be divisible by 32) |
| `--frames` | 25 | Number of frames (must be 8n+1: 9, 17, 25, 33, 41, 49, 57, 65, 73, 81, 89, 97) |
| `--steps` | 20 | Inference steps (more = better quality, slower) |
| `--guidance` | 5.0 | Guidance scale |
| `--fps` | 24 | Output video FPS |
| `--seed` | random | Random seed for reproducibility |
| `-n` | "" | Negative prompt |
| flag | default | description |
|------|---------|-------------|
| `--width` | 512 | video width (divisible by 32) |
| `--height` | 320 | video height (divisible by 32) |
| `--frames` | 25 | frame count (must be 8n+1: 9, 17, 25, 33...) |
| `--steps` | 20 | inference steps |
| `--guidance` | 5.0 | guidance scale |
| `--fps` | 24 | output fps |
| `--seed` | random | seed for reproducibility |
| `-n` | "" | negative prompt |
### Examples
### examples
```bash
# Quick preview (fast)
python generate.py "A sunset over mountains" -o preview.mp4 --frames 25 --steps 10 --width 512 --height 320
# quick test
python generate.py "sunset over mountains" -o test.mp4 --steps 10
# Standard quality
python generate.py "A dog running on the beach" -o standard.mp4 --frames 49 --steps 20 --width 768 --height 448
# higher quality
python generate.py "dog running on beach" -o video.mp4 --frames 49 --steps 20 --width 768 --height 448
# High quality (slow, needs 128GB RAM)
python generate.py "Cinematic shot of a forest" -o hq.mp4 --frames 97 --steps 30 --width 1024 --height 576
# max quality (needs 128GB ram, takes ~30 min)
python generate.py "cinematic forest shot" -o hq.mp4 --frames 97 --steps 30 --width 1024 --height 576
```
## Performance
## performance
Tested on Mac with M-series chips:
tested on m3 ultra:
| Resolution | Frames | Steps | Time (approx) | RAM Usage |
|------------|--------|-------|---------------|-----------|
| 512x320 | 25 | 10 | ~1 min | ~45GB |
| 768x448 | 49 | 20 | ~10 min | ~60GB |
| 1024x576 | 97 | 30 | ~30 min | ~80GB |
| resolution | frames | steps | time |
|------------|--------|-------|------|
| 512x320 | 25 | 10 | ~1 min |
| 768x448 | 49 | 20 | ~10 min |
| 1024x576 | 97 | 30 | ~30 min |
## How the Patch Works
## how the patch works
Two files in diffusers are patched:
two files get patched in diffusers:
### 1. `diffusers/pipelines/ltx2/connectors.py`
**diffusers/pipelines/ltx2/connectors.py**
```python
# Before:
# before
freqs_dtype = torch.float64 if self.double_precision else torch.float32
# After:
freqs_dtype = torch.float32 # MPS fix
# after
freqs_dtype = torch.float32
```
### 2. `diffusers/models/transformers/transformer_ltx2.py`
**diffusers/models/transformers/transformer_ltx2.py**
```python
# Before:
freqs_dtype = torch.float64 if self.double_precision else torch.float32
# After:
freqs_dtype = torch.float32 # MPS fix
# same change
freqs_dtype = torch.float32
```
## Troubleshooting
## troubleshooting
### "MPS backend out of memory"
- Reduce resolution, frames, or close other apps
- Try `--width 512 --height 320 --frames 25`
**out of memory** - reduce resolution/frames or close other apps
### Model download fails
- Check your internet connection
- The model is ~40GB, first run takes a while to download
**model download fails** - it's ~40GB, first run takes a while
### Import errors
- Make sure you installed diffusers from git (dev version needed for LTX2Pipeline)
- Run `pip install git+https://github.com/huggingface/diffusers.git`
**import errors** - make sure you installed diffusers from git, not pip
## Credits
## credits
- [Lightricks](https://github.com/Lightricks) for LTX-2
- [Hugging Face](https://github.com/huggingface/diffusers) for diffusers
- MPS patch discovered while debugging with Claude
- [lightricks](https://github.com/Lightricks) for ltx-2
- [@ivanfioravanti](https://twitter.com/ivanfioravanti) for the mps fix approach
- [huggingface](https://github.com/huggingface/diffusers) for diffusers
## License
## license
MIT