160 ms Nemotron streaming tier: the checkpoint's native [70,1] lookahead — conversion validated on your harness, plumbing patch ready

[nischith-memfold]

The three Nemotron streaming tiers you publish turn out to sit on a fourth one
the checkpoint already carries. nvidia/nemotron-speech-streaming-en-0.6b is a
multi-lookahead cache-aware export:

att_context_size_all = [[70,13], [70,6], [70,1], [70,0]]   (runtime-switchable, no retraining)

so the shipped tiers map to native modes — 1120 ms is [70,13], 560 ms is
[70,6], 2240 ms is 2× the [70,13] chunk — and [70,1] is an unshipped
160 ms tier: 16 mel frames per chunk, 2 encoder frames out, chunk_size [9,16], valid_out_len 2, cache shapes unchanged ([1,24,70,1024] /
[1,24,1024,8]). For live dictation (our use case — Alma, an open-source macOS
dictation app) that's partials at 3.5× the cadence of the current lowest tier,
from the same weights.

We've converted, validated, and shipped it behind a vendored patch, and we'd
like to upstream both halves: the tier plumbing here, and the conversion tweak
in mobius. Everything below was measured on an M-series Mac with your own
tooling.

Why it costs almost nothing

• Only the encoder bakes chunk geometry into its traced shapes. Decoder,
  joint, fused decoder_joint, preprocessor, and tokenizer are
  chunk-independent — the 160 ms artifact reuses them from the 560 ms tier
  byte-for-byte. One encoder re-trace + int8 quantize is the whole conversion.
• Your Swift runtime already handles it with zero changes. Geometry is
  metadata.json-driven and the decode loop reads encoded.shape[2]
  dynamically; a 160 ms artifact with a complete metadata
  (chunk_mel_frames 16, pre_encode_cache 9, total_mel_frames 25, chunk_ms 160, cache shapes as above) runs as-is via
  loadModels(from:) / --model-dir. The patch below is purely tier
  plumbing: an enum case, the Repo arms, and the two CLI parsers (whose help
  text already advertises 160 ms).

Validation (LibriSpeech test-clean, 100 files, fluidaudiocli nemotron-benchmark --model-dir for both rows)

Tier	WER	RTFx	int8 encoder / chunk (CPU_AND_NE)
560 ms (your published artifact)	2.42%	36.5	11.0 ms
160 ms ([70,1])	2.50%	12.7	10.5 ms

• Lookahead cost [70,6] → [70,1]: +0.08 WER points (2 extra word
  errors in 2,357). NeMo PyTorch at [70,1] anchors at 1.43% on the first 50
  files (pad_and_drop_preencoded=False), so nothing was lost in conversion.
• Trace parity vs PyTorch before quantization: cos = 1.000000 over chained
  chunks, carried caches included; ≥ 0.996 after per-channel int8.
• 10-file smoke through the Swift pipeline: WER 0%.

A conversion finding you probably want regardless of this tier

mobius's convert_nemotron_streaming.py defaults to
--precision FLOAT32, and an fp32 mlprogram never runs on the ANE — we
measured CPU_AND_NE ≡ CPU_ONLY at every shape we tried (37 ms vs 10.5 ms
per 160 ms chunk; 48 ms vs 11 ms at 560 ms shapes). The symptom looks exactly
like "ops fell off the ANE at this tensor shape", but shape is innocent:
weight-only int8 quantization preserves the program's compute precision, so
the FLOAT16 export is the one to quantize. Your published artifacts are
clearly fp16-derived already; the default just makes it easy for anyone
reproducing your pipeline to build a silently CPU-bound encoder. A default
flip or a README line in mobius would save the next person a profiling
session.

Composes with the decode-time biasing offer (#841)

Both offers serve the same integration. The bias engine from #841 hooks the
logits of the B1 fused decoder_joint (or bare decoder+joint), which the
160 ms artifact ships unchanged, and it carries match state across chunk
boundaries with no geometry assumptions. Measured together at 160 ms on the
rig from #841: baseline recall 2/8 → 5/8 at the default boost, zero false
fires, zero neutral drift across three runs (the 560 ms tier measured 6/8 —
the one lost term decodes as its real-word homophone at the shorter
lookahead).

What we're offering / asking

1. The plumbing PR here — diff below, applies to main (checked against
   HEAD today). We'll open it if the tier is wanted.
2. A mobius PR adding a parameterized encoder-tier export
   (--chunk-mel-frames 16 --att-context 70,1 --precision FLOAT16) so any
   trained lookahead is a one-command conversion, plus the FLOAT16 note.
3. The artifact: we can hand you the built nemotron_coreml_160ms
   directory (encoder_int8.mlmodelc + metadata + the chunk-independent files
   from your 560 ms tier), or you re-run the two commands from your own
   pipeline — the delta to your existing script is
   encoder.set_default_att_context_size([70, 1]) before
   setup_streaming_params() and the three chunk constants. Publishing it as
   nemotron_coreml_160ms on
   FluidInference/nemotron-speech-streaming-en-0.6b-coreml is the one step
   only you can take — the auto-download path needs the subdir to exist.

Plumbing diff

diff --git a/Sources/FluidAudio/ASR/Parakeet/Streaming/Nemotron/NemotronChunkSize.swift b/Sources/FluidAudio/ASR/Parakeet/Streaming/Nemotron/NemotronChunkSize.swift
index eaa040a..5fbc853 100644
--- a/Sources/FluidAudio/ASR/Parakeet/Streaming/Nemotron/NemotronChunkSize.swift
+++ b/Sources/FluidAudio/ASR/Parakeet/Streaming/Nemotron/NemotronChunkSize.swift
@@ -4,13 +4,15 @@ import Foundation
 public enum NemotronChunkSize: Int, Sendable, CaseIterable {
     case ms2240 = 2240  // 2.24s - default; highest throughput (+50% RTFx w/ B1 vs 1120ms), WER-neutral
     case ms1120 = 1120  // 1.12s - the trained chunk; lower latency
-    case ms560 = 560  // 0.56s - lowest latency tier
+    case ms560 = 560  // 0.56s - low latency tier
+    case ms160 = 160  // 0.16s - lowest latency; native att_context [70,1] lookahead
 
     public var repo: Repo {
         switch self {
         case .ms2240: return .nemotronStreaming2240
         case .ms1120: return .nemotronStreaming1120
         case .ms560: return .nemotronStreaming560
+        case .ms160: return .nemotronStreaming160
         }
     }
 
diff --git a/Sources/FluidAudio/ModelNames.swift b/Sources/FluidAudio/ModelNames.swift
index 188b56f..3c2811e 100644
--- a/Sources/FluidAudio/ModelNames.swift
+++ b/Sources/FluidAudio/ModelNames.swift
@@ -24,6 +24,7 @@ public enum Repo: String, CaseIterable, Sendable {
     case nemotronStreaming2240 = "FluidInference/nemotron-speech-streaming-en-0.6b-coreml/2240ms"
     case nemotronStreaming1120 = "FluidInference/nemotron-speech-streaming-en-0.6b-coreml/1120ms"
     case nemotronStreaming560 = "FluidInference/nemotron-speech-streaming-en-0.6b-coreml/560ms"
+    case nemotronStreaming160 = "FluidInference/nemotron-speech-streaming-en-0.6b-coreml/160ms"
     /// Parakeet Unified 0.6B (FastConformer-RNNT). One checkpoint serves both
     /// offline (15 s window) and streaming inference; streaming uses a
     /// chunked-attention encoder re-run over a [left|chunk|right] window
@@ -99,6 +100,8 @@ public enum Repo: String, CaseIterable, Sendable {
             return "nemotron-speech-streaming-en-0.6b-coreml/1120ms"
         case .nemotronStreaming560:
             return "nemotron-speech-streaming-en-0.6b-coreml/560ms"
+        case .nemotronStreaming160:
+            return "nemotron-speech-streaming-en-0.6b-coreml/160ms"
         case .parakeetUnified:
             return "parakeet-unified-en-0.6b-coreml"
         case .diarizer:
@@ -147,7 +150,7 @@ public enum Repo: String, CaseIterable, Sendable {
             return "FluidInference/parakeet-realtime-eou-120m-coreml"
         case .kokoroAne, .kokoroAneZh, .kokoroAneJa:
             return "FluidInference/kokoro-82m-coreml"
-        case .nemotronStreaming2240, .nemotronStreaming1120, .nemotronStreaming560:
+        case .nemotronStreaming2240, .nemotronStreaming1120, .nemotronStreaming560, .nemotronStreaming160:
             return "FluidInference/nemotron-speech-streaming-en-0.6b-coreml"
         case .nemotronMultilingual:
             return "FluidInference/Nemotron-3.5-ASR-Streaming-Multilingual-0.6b-CoreML"
@@ -187,6 +190,8 @@ public enum Repo: String, CaseIterable, Sendable {
             return "nemotron_coreml_1120ms"
         case .nemotronStreaming560:
             return "nemotron_coreml_560ms"
+        case .nemotronStreaming160:
+            return "nemotron_coreml_160ms"
         case .lseendAmi:
             return "optimized/ami"
         case .lseendCallHome:
@@ -229,6 +234,8 @@ public enum Repo: String, CaseIterable, Sendable {
             return "nemotron-streaming/1120ms"
         case .nemotronStreaming560:
             return "nemotron-streaming/560ms"
+        case .nemotronStreaming160:
+            return "nemotron-streaming/160ms"
         case .sortformer:
             return "sortformer"
         case .parakeetCtc110m:
@@ -1303,7 +1310,7 @@ public enum ModelNames {
             return ModelNames.TDTJa.requiredModels
         case .parakeetEou160, .parakeetEou320, .parakeetEou1280:
             return ModelNames.ParakeetEOU.requiredModels
-        case .nemotronStreaming2240, .nemotronStreaming1120, .nemotronStreaming560:
+        case .nemotronStreaming2240, .nemotronStreaming1120, .nemotronStreaming560, .nemotronStreaming160:
             return ModelNames.NemotronStreaming.requiredModels
         case .parakeetUnified:
             // Variants: nil/"fp16" (streaming), "offline"/"offline-fp16" (batch).
diff --git a/Sources/FluidAudioCLI/Commands/ASR/Parakeet/Streaming/NemotronBenchmark.swift b/Sources/FluidAudioCLI/Commands/ASR/Parakeet/Streaming/NemotronBenchmark.swift
index 10951c9..f8052a0 100644
--- a/Sources/FluidAudioCLI/Commands/ASR/Parakeet/Streaming/NemotronBenchmark.swift
+++ b/Sources/FluidAudioCLI/Commands/ASR/Parakeet/Streaming/NemotronBenchmark.swift
@@ -67,9 +67,10 @@ public class NemotronBenchmark {
                     case 2240: config.chunkSize = .ms2240
                     case 1120: config.chunkSize = .ms1120
                     case 560: config.chunkSize = .ms560
+                    case 160: config.chunkSize = .ms160
                     default:
                         logger.warning(
-                            "Invalid chunk size: \(ms)ms. Valid options: 2240, 1120, or 560. Using default 2240ms.")
+                            "Invalid chunk size: \(ms)ms. Valid options: 2240, 1120, 560, or 160. Using default 2240ms.")
                     }
                 }
             case "--help", "-h":
@@ -96,7 +97,7 @@ public class NemotronBenchmark {
                 --max-files, -n <count>   Maximum files to process (default: all)
                 --subset, -s <name>       LibriSpeech subset (default: test-clean)
                 --model-dir, -m <path>    Path to Nemotron CoreML models
-                --chunk, -c <ms>          Chunk size: 2240, 1120, or 560 (default: 2240)
+                --chunk, -c <ms>          Chunk size: 2240, 1120, 560, or 160 (default: 2240)
                 --help, -h                Show this help
 
             Chunk Sizes:
diff --git a/Sources/FluidAudioCLI/Commands/ASR/Parakeet/Streaming/NemotronTranscribe.swift b/Sources/FluidAudioCLI/Commands/ASR/Parakeet/Streaming/NemotronTranscribe.swift
index 54a19d2..0700e98 100644
--- a/Sources/FluidAudioCLI/Commands/ASR/Parakeet/Streaming/NemotronTranscribe.swift
+++ b/Sources/FluidAudioCLI/Commands/ASR/Parakeet/Streaming/NemotronTranscribe.swift
@@ -52,9 +52,10 @@ public class NemotronTranscribe {
                     case 2240: config.chunkSize = .ms2240
                     case 1120: config.chunkSize = .ms1120
                     case 560: config.chunkSize = .ms560
+                    case 160: config.chunkSize = .ms160
                     default:
                         logger.warning(
-                            "Invalid chunk size: \(ms)ms. Valid options: 2240, 1120, or 560. Using default 2240ms.")
+                            "Invalid chunk size: \(ms)ms. Valid options: 2240, 1120, 560, or 160. Using default 2240ms.")
                     }
                 }
             case "--help", "-h":

[Alex-Wengg]

we didn't converted 160 ms chunks bc it was too unnotable of audio chunk while offering weak WER