n0p/mlx-video
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add LTX-2.3 model architecture with prompt-conditioned adaptive layer normalization (adaln) support. Introduce gating mechanisms in attention modules and update transformer configurations to accommodate new parameters. Refactor video and audio processing to utilize adaptive normalization, improving model flexibility and performance. Update weight loading and initialization logic to support dynamic block structures in the decoder.

Browse Source
This commit is contained in:
Prince Canuma
2026-03-10 16:47:36 +01:00
parent d028b239fb
commit 207c223354
8 changed files with 545 additions and 239 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
mlx_video/models/ltx/config.py
View File

@@ -131,6 +131,12 @@ class LTXModelConfig(BaseModelConfig):
# Attention type
attention_type: AttentionType = AttentionType.DEFAULT
# LTX-2.3: prompt-conditioned adaptive layer norm
# Controls: gate_logits in attention, 9-param scale_shift_table,
# prompt_adaln_single, per-block prompt_scale_shift_table,
# removal of caption_projection
has_prompt_adaln: bool = False
# VAE config
vae_config: Optional[VideoVAEConfig] = None