Description
We are currently investigating a growing leak in our net8 app. Doing a dotnet-dump, and using JetBrains dotMemory, we found over 761k objects in the ConcurrentDictionary<object,Expression>, and 695k ConcurrentDictionary<Expression, string> both found in the ConstantExpressionHelper. We believe this is what is causing our leak. Analysis of all the instances show this class overwhelming on top of any other type instance.
Usage:
Our application makes heavy use of the dynamic library - where expression trees are created from input string data provided by outside calls. This has worked well for many years, going back to the early introduction of the dynamic linq library in the framework days.
Inputs change very regularly, based on the applications requirements. Constants are introduced that may only be used once in the application lifetime.
Possible Solution:
After cloning the project, was able to see the ConcurrentDictionary in ConstantExpressionHelper grow without any way to remove collected constant expressions. Proposal would be to use a sliding cache with a reasonable short lifetime (minutes? hours?), instead of an ever-lasting private static ConcurrentDictionary, allowing the system to remove constants that are no longer being called, which will then allow the GC to remove the tress of referenced objects.
Instance Details
Process running under Ubuntu Linux Container, net8.0 framework, approximately 9 days of uptime.
Trends monitoring:
Screen Shots of dotMemory outputs, arranged by size descending:
Event showing 18 megs of Int32 constants
The use of the ConstantExpressionHelper makes total sense, so I can appreciate why it's there. The proposal is just to have a way to let it clear itself out, especially for highly dynamic, long running services. Self-managing is desired, no reason for the developer to write code responsible to clear these things out.
I am more than willing to work on this and create a PR, downside is I don't have all the frameworks installed on my dev machine. Just getting it to compile required changing some csproj to remove all the old legacy things. If you would like a PR, let me know and ill work on that.
Description
We are currently investigating a growing leak in our net8 app. Doing a
dotnet-dump, and using JetBrainsdotMemory, we found over 761k objects in theConcurrentDictionary<object,Expression>, and 695kConcurrentDictionary<Expression, string>both found in theConstantExpressionHelper. We believe this is what is causing our leak. Analysis of all the instances show this class overwhelming on top of any other type instance.Usage:
Our application makes heavy use of the dynamic library - where expression trees are created from input string data provided by outside calls. This has worked well for many years, going back to the early introduction of the dynamic linq library in the framework days.
Inputs change very regularly, based on the applications requirements. Constants are introduced that may only be used once in the application lifetime.
Possible Solution:
After cloning the project, was able to see the
ConcurrentDictionaryinConstantExpressionHelpergrow without any way to remove collected constant expressions. Proposal would be to use asliding cachewith a reasonable short lifetime (minutes? hours?), instead of an ever-lasting private staticConcurrentDictionary, allowing the system to remove constants that are no longer being called, which will then allow the GC to remove the tress of referenced objects.Instance Details
Process running under Ubuntu Linux Container, net8.0 framework, approximately 9 days of uptime.
Trends monitoring:
Screen Shots of dotMemory outputs, arranged by size descending:
Event showing 18 megs of Int32 constants
The use of the
ConstantExpressionHelpermakes total sense, so I can appreciate why it's there. The proposal is just to have a way to let it clear itself out, especially for highly dynamic, long running services. Self-managing is desired, no reason for the developer to write code responsible to clear these things out.I am more than willing to work on this and create a PR, downside is I don't have all the frameworks installed on my dev machine. Just getting it to compile required changing some csproj to remove all the old legacy things. If you would like a PR, let me know and ill work on that.