Custom Policy Backend Support

[kakky-hacker]

Is your feature request related to a problem? Please describe.

Currently, it is difficult to integrate a custom reinforcement learning backend into ML-Agents without modifying the core package.

The IPolicy implementations are effectively internal, and there is no official extension point to inject a custom policy from an external package. As a result, users who want to experiment with alternative backends (e.g., calling a reinforcement learning implementation from a native DLL via FFI, instead of using the built-in Python-based workflow) are forced to maintain a fork or directly modify core classes such as BehaviorParameters.

This creates maintenance overhead and makes it harder to keep up with upstream updates.

Describe the solution you'd like

I would like to have a pluggable policy interface that allows external packages to provide custom IPolicy implementations without modifying ML-Agents core code.

For example, this could be achieved by:

• Introducing a factory or registry mechanism (e.g., IPolicyFactory)
• Allowing BehaviorParameters to delegate policy creation to a registered provider
• Supporting external packages to register custom policy implementations

This would make it possible to swap out the default policy backend while still using ML-Agents’ existing sensor and environment systems.

Describe alternatives you've considered

• Forking ML-Agents and modifying BehaviorParameters directly
  → Works, but introduces long-term maintenance issues

• Implementing the entire agent loop outside of ML-Agents
  → Avoids modification, but loses integration with sensors, actuators, and existing tooling

Both approaches are less maintainable compared to having an official extension point.

Additional context

I am currently working on integrating a native reinforcement learning backend (implemented by Rust and tch-rs) and would like to use it as a drop-in replacement for the default policy.

Having a pluggable policy interface would make this integration clean and reusable, and could also benefit other users exploring alternative ML backends.

I would be happy to contribute a PR if this direction aligns with the project’s design goals.

[src-debug]

I think you have to design a Pluggable Policy Interface that introduces a factory-based registry system which will allow you to inject your custom Rust/tch-rs backend as a drop-in replacement without modifying the core ML-Agents package.

The Solution: Registry & Factory Pattern

This involves three key components that decouple policy creation from the BehaviorParameters class:

1. IPolicyFactory Interface: A new public interface that defines how a policy is created. Your external package implements this to wrap your Rust FFI logic.
2. PolicyRegistry: A static global registry where external packages can "sign up" their custom backends during Unity's initialization.
3. BehaviorType.Custom: An extension to the existing behavior enum that tells ML-Agents to look up a policy in the registry instead of using the built-in Sentis or Remote backends.

How to Implement Your Rust Backend

Your implementation will now look somewhat like:

1. Implement IPolicy: Create a C# class that implements IPolicy and handles the FFI calls to your Rust tch-rs library.
2. Create Factory: Implement IPolicyFactory to instantiate your custom policy class.
3. Auto-Register: Use Unity's [RuntimeInitializeOnLoadMethod] to register your factory with the PolicyRegistry when the game starts.
4. Configure Agent: In the Unity Inspector, set the Behavior Type to Custom and provide your unique string ID (e.g., rust-tch-backend).

You would no longer need to fork ML-Agents or edit BehaviorParameters.cs and your Rust backend will still get full access to all ML-Agents sensors and actuators.

Please let me know if you have any more questions. Thank you.

[github-actions]

This issue is stale because it has been open for 30 days with no activity.

[github-actions]

This issue was closed because it has been inactive for 14 days since being marked as stale. Please open a new issue for related bugs.