Skip to content

Fixes OOM Errors - too high RAM usage by VAD - #1198

Merged
MahmoudAshraf97 merged 4 commits into
SYSTRAN:masterfrom
Purfview:patch-3
Dec 12, 2024
Merged

Fixes OOM Errors - too high RAM usage by VAD#1198
MahmoudAshraf97 merged 4 commits into
SYSTRAN:masterfrom
Purfview:patch-3

Conversation

@Purfview

@Purfview Purfview commented Dec 10, 2024

Copy link
Copy Markdown
Contributor

fixes #1193 and #1169
VAD implementation consumes humongous memory amount [original Silero doesn't have this problem]

This PR should fix the OOM problem.
Alt solution could be removing lru_cache.

@MahmoudAshraf97

Copy link
Copy Markdown
Collaborator

Thanks, for reference microsoft/onnxruntime#11627

BTW, there is still room for OOM on smaller ram systems caused by this:

batched_audio = batched_audio.reshape(-1, num_samples + context_size_samples)
encoder_output = self.encoder_session.run(None, {"input": batched_audio})[0]
encoder_output = encoder_output.reshape(batch_size, -1, 128)

the input here is (num_segments, 576) and num_segments depends on the audio length and it's unbounded, we should replace this with a for loop over num_segments with a large batch size (10k for example or the lowest number where we start to lose speed), this is helpful for very long audios

Purfview and others added 4 commits December 12, 2024 14:17
Reported problems:
#1193
#1169

VAD implementations consumes humongous memory amounts [original Silero doesn't have this problem]

This PR should fix the OOM problem.
Alt solution could be removing 'lru_cache'.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

OOM when using VAD

2 participants