Skip to content

Commit aa88ffc

Browse files
authored
Merge pull request #891
Update memory-related docs.
2 parents 404370b + 8b2cc3e commit aa88ffc

1 file changed

Lines changed: 66 additions & 1 deletion

File tree

docs/src/api/memory.md

Lines changed: 66 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -181,4 +181,69 @@ julia> xd * xd # Can be used with HIP libraries.
181181

182182
!!! note
183183
Passing `own=true` keyword will make the wrapped array take the ownership of the memory.
184-
For host memory it will unpin it on destruction and for device memory it will free it.
184+
For device memory it will free it on destruction.
185+
For host memory it will unregister (unpin) it on destruction.
186+
Multiple wraps of the same host pointer are reference-counted,
187+
so the actual unregister only happens when the last wrapper is freed.
188+
189+
## GPU Memory Management
190+
191+
Instances of `ROCArray` are managed by the Julia garbage collector.
192+
When a `ROCArray` becomes unreachable it is eventually collected
193+
and the underlying GPU memory is released.
194+
There is no need for manual memory management in normal use.
195+
196+
### Memory pool
197+
198+
Behind the scenes, a HIP memory pool caches freed allocations to speed up
199+
future requests.
200+
As a result, GPU memory may appear occupied even after arrays have been freed.
201+
When memory pressure is high, the pool automatically releases cached memory.
202+
203+
To query current GPU memory usage:
204+
205+
```julia
206+
AMDGPU.info() # (free_bytes, total_bytes)
207+
AMDGPU.free() # free bytes
208+
AMDGPU.total() # total bytes
209+
AMDGPU.used() # used bytes (total - free)
210+
```
211+
212+
To manually release all cached pool memory back to the system:
213+
214+
```julia
215+
AMDGPU.HIP.reclaim()
216+
```
217+
218+
### Avoiding GC pressure
219+
220+
To explicitly free a `ROCArray` without waiting for the garbage collector:
221+
222+
```julia
223+
AMDGPU.unsafe_free!(x)
224+
```
225+
226+
After this call `x` is invalid and must not be used again.
227+
228+
### Eager GC
229+
230+
When enabled, proactively triggers garbage collection before
231+
allocations when GPU memory pressure exceeds ~75%.
232+
This prevents out-of-memory errors in allocation-heavy workloads at
233+
small runtime cost.
234+
235+
```julia
236+
AMDGPU.eager_gc!(true) # or false
237+
```
238+
239+
### Memory limits
240+
241+
Constrain how much GPU memory usage is allowed.
242+
Values are strings in the form `"8 GiB"`, `"50 %"`, or `"none"` (default).
243+
These are persisted via `Preferences.jl`.
244+
245+
```julia
246+
AMDGPU.hard_memory_limit!("8 GiB") # hard cap checked before every allocation
247+
AMDGPU.soft_memory_limit!("6 GiB") # advisory limit for the memory pool
248+
AMDGPU.hard_memory_limit!("none") # disable (default)
249+
```

0 commit comments

Comments
 (0)