You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Refactor: Portable characters & one canvas for training (#57)
* feat: character scoring accuracy, encode progress, and an in-app Hugging Face token
* feat: train and apply characters per model, in a resizable editor panel
* moved models out of the app
* feat: apply characters on Krea 2 via their trained adapter
* merged training & studion tabs, new character training nodes, removed ui for character
* fix: canvas edits, wiring, node sizing and the models panel, complete ui refactor
* fix: exported graphs carry the models and settings needed to rebuild them
* fix: report the character encoders, and stop serving a frozen registry cache
* surface all models, export/import graph improvement, new connection for lora support
* fixed trainer adapter, run state per node, download fix, strength param for adapter
* model export fix, ti adapter
* fix: training adapter for krea2
> |[`inlineresearch/skin-lora-krea-2-raw`](https://huggingface.co/inlineresearch/skin-lora-krea-2-raw)| Skin-texture LoRA for Krea 2 RAW, trained in the Trainer tab|
33
-
> |[`inlineresearch/krea2-skin-lora`](https://huggingface.co/datasets/inlineresearch/krea2-skin-lora)| The 26 image + caption pairs that LoRA trained on |
> |[`inlineresearch/skin-lora-krea-2-raw`](https://huggingface.co/inlineresearch/skin-lora-krea-2-raw)| Skin-texture LoRA for Krea 2 RAW, trained on the canvas|
33
+
> |[`inlineresearch/krea2-skin-lora`](https://huggingface.co/datasets/inlineresearch/krea2-skin-lora)| The 26 image + caption pairs that LoRA trained on |
Copy file name to clipboardExpand all lines: TRAINING.md
+28-43Lines changed: 28 additions & 43 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -6,13 +6,14 @@ the canvas, see [LoRA training in the README](README.md#lora-training).
6
6
7
7
Inline Studio trains LoRAs for **Z-Image**, **Krea 2**, **FLUX.2**, **MiniMax H3** and **LTX-2.5**
8
8
on your own GPU, with no cloud step and nothing uploaded. Training is cheaper than generating: a
9
-
16GB card trains all three image models at 512px, and a LoRA trained at 512 applies at any
10
-
generation resolution. The two video models are the exception, and LTX-2.5 wants a 48GB card.
9
+
16GB card trains all three image models at 512px, and MiniMax H3 as well if it has 64GB of system
10
+
RAM behind it. A LoRA trained at 512 applies at any generation resolution. LTX-2.5 is the one
11
+
exception and wants a 48GB card.
11
12
12
13
**Contents:**[The graph](#the-graph) · [Datasets and outputs](#datasets-and-outputs) ·
13
14
[Stop and resume](#stop-and-resume) · [Trigger words](#trigger-words) ·
14
-
[Architecture and base model modes](#architecture-and-base-model-modes) ·[Install](#install) ·
15
-
[Training on clips](#training-on-clips) · [Control LoRAs](#control-loras) ·
15
+
[Architecture and base model modes](#architecture-and-base-model-modes) ·
16
+
[Training on clips](#training-on-clips) · [Install](#install) · [Control LoRAs](#control-loras) ·
16
17
[**Benchmark results**](#benchmark-results) ·
17
18
[Dataset and adapter options](#dataset-and-adapter-options) · [Base precision](#base-precision)
18
19
@@ -60,7 +61,7 @@ The Trainer's Adjust panel picks the **architecture** first (Z-Image, Krea 2, FL
60
61
-**FL2VA** is the only base, and it is undistilled, so there is no adapter and nothing to drift. Put `minimax_h3_fl2va_bf16.safetensors` in `models/diffusion_models/`, train on stills, then wire the LoRA into any of the four H3 nodes. **It has to be the bf16 file.** The smaller `pruned` and `pruned_fp8_scaled` builds generate but cannot train: they ship no timestep path for the modulation basis to be derived from, and they would save nothing anyway, because the base trains at 4-bit whichever file it starts from. The trainer says so rather than failing part way in. It loads on the Reference to Video node too, which uses a different checkpoint file: the two partitions are the same architecture.
61
62
-**Stills or short clips.** Drop images and it learns appearance: look, style, character, lighting. Drop video and it learns motion too. Sound is never learned either way, because the audio rows are empty. See [Training on clips](#training-on-clips).
62
63
-**The base is 4-bit, always.** H3 is 40GB after the AdaLN factorisation and 11.7GB after quantisation, so full precision is refused rather than offered and then failing. There is no base-precision control for H3 for the same reason.
63
-
-**A 24GB card is comfortable and a 16GB card works, slowly.** The run encodes latents and captions in two passes that never overlap, because H3's fp32 video VAE and its 32B conditioner cannot be resident together. On a card that holds the conditioner it peaks at 20.6GB; on one that does not, the conditioner runs on the CPU and the peak drops to 12.7GB while a step goes from 0.6s to 16s. Either way there is about seven minutes of startup, and 64GB of system RAM for the smaller card. See [Benchmark results](#benchmark-results) for the split. The download is about 124GB before any of that.
64
+
-**A 24GB card is comfortable and a 16GB card works, slowly.** The run encodes latents and captions in two passes that never overlap, because H3's fp32 video VAE and its 32B conditioner cannot be resident together. On a card that holds the conditioner it peaks at 20.6GB; on one that does not, the conditioner runs on the CPU and the peak drops to 12.7GB while a step goes from 0.6s to 16s. Either way there is about seven minutes of startup, and 64GB of system RAM for the smaller card. See [Benchmark results](#benchmark-results) for the split. The download is about 139GB before any of that.
64
65
65
66
**Z-Image** is distilled either way:
66
67
@@ -187,24 +188,11 @@ well as its own `0001.ref.mp4`, so a downloaded set can be trained on directly.
187
188
> guessing: LTX needs Ampere or newer for the same reason generation does (see the runbook below),
188
189
> and a 42GB bf16 base does not fit a 15GB card in any case.
189
190
190
-
### LTX-2.5 generation, measured on an L40S (44.4 GiB)
191
-
192
-
A 2 second clip at 960x576, distilled, same clip each time. The **cached** column is the one a
193
-
sequence actually pays, for every shot after the first.
Three fixes got from the first row to the third, and each needed the one before it. The full
202
-
write-up is in [docs/ltx-2-5.md](docs/ltx-2-5.md#performance-measured). The remaining cost is the
203
-
prompt encoder, reloaded on every render.
191
+
### Training VRAM by model
204
192
205
193
12 steps at rank 16, batch 1, gradient checkpointing on. The number is `torch.cuda.max_memory_allocated`, so leave headroom for the CUDA context and allocator slack.
206
194
207
-
| Model | Base mode | Res | Base precision | L40S (46GB) | T4 (15GB) |
195
+
| Model | Base mode | Res | Base precision | L40S (48GB) | T4 (16GB) |
@@ -259,11 +247,11 @@ A training adapter is free: it is fused into the base before training starts, so
259
247
260
248
**LTX-2.5 is the opposite case: the base is nearly the whole bill, and it barely fits.** The 22B dev
261
249
transformer lands at 38.0GB allocated before a step runs, and training peaks at 42.0GB allocated
262
-
against 43.4GB reserved on a 46GB card. That leaves under 3GB of headroom, so a 48GB card is the
250
+
against 43.4GB reserved on an L40S. That leaves under 3GB of headroom, so a 48GB card is the
263
251
floor rather than a comfort. There is no 4-bit rung to fall back on: the loader refuses to quantise
264
252
this architecture, and the trainer now says so instead of dropping the setting silently.
265
253
266
-
|| L40S (46GB) |
254
+
|| L40S (48GB) |
267
255
| -------------------------- | ----------- |
268
256
| Peak VRAM, 512px | 42.0GB |
269
257
| Seconds per step, 512px | 0.67 |
@@ -306,6 +294,22 @@ FLUX.2 is quicker than either. On an L40S, klein Base 4B trains at about 0.3s a
306
294
307
295
System RAM matters as well. Checkpoints are read tensor by tensor rather than mapped whole, so Krea 2 trains in about 3GB of host RAM. Without that, Linux refuses to map a file larger than physical RAM when there is no swap, and a 26GB checkpoint cannot be opened on a 16GB machine at all.
308
296
297
+
### LTX-2.5 generation, measured on an L40S
298
+
299
+
Generation rather than training, kept here because it is the same card and the same clip. A 2 second
300
+
clip at 960x576, distilled. The **cached** column is the one a sequence actually pays, for every shot
Three fixes got from the first row to the third, and each needed the one before it. The full
310
+
write-up is in [docs/ltx-2-5.md](docs/ltx-2-5.md#performance-measured). The remaining cost is the
311
+
prompt encoder, reloaded on every render.
312
+
309
313
## Dataset and adapter options
310
314
311
315
Three settings shape what the adapter learns rather than what it costs:
@@ -322,7 +326,7 @@ Krea 2's base is 26GB at bf16, which is what makes it expensive to fine-tune. Th
322
326
-**Full precision (bf16)** forces the unquantized base.
323
327
-**4-bit (NF4)** forces the quantized base.
324
328
325
-
The setting appears for Krea 2 and FLUX.2, but it only pays off on Krea 2. Z-Image has no 4-bit path and does not need one: it trains in about 15 GB at 1024, so bf16 already fits the cards people have. FLUX.2 has the path and gains nothing from it, because klein 4B is smaller than its own text encoder and the peak sits in the caching pass either way, so Auto leaves it at bf16. See [Benchmark results](#benchmark-results).
329
+
The setting appears for Krea 2 and FLUX.2, but it only pays off on Krea 2. Z-Image has no 4-bit path and does not need one: it trains in about 15GB at 1024, so bf16 already fits the cards people have. FLUX.2 has the path and gains nothing from it, because klein 4B is smaller than its own text encoder and the peak sits in the caching pass either way, so Auto leaves it at bf16. See [Benchmark results](#benchmark-results).
326
330
327
331
To keep the peak down, the VAE and text encoder are loaded first, used to cache latents and captions, then freed before the transformer loads, so the two never stack. Which half then owns the peak depends on the model: for Z-Image and Krea 2 it is the transformer, for FLUX.2 klein it is the caching pass. If you do hit an out-of-memory error, lower the training resolution before changing anything else.
0 commit comments