Skip to content

chore: Update benchmarks - #163

Merged
neuecc merged 1 commit into
Cysharp:masterfrom
filzrev:chore-update-benchmarks
Feb 6, 2025
Merged

chore: Update benchmarks#163
neuecc merged 1 commit into
Cysharp:masterfrom
filzrev:chore-update-benchmarks

Conversation

@filzrev

@filzrev filzrev commented Feb 6, 2025

Copy link
Copy Markdown
Contributor

What's included in this PR

  1. Update BenchmarkDotNet package to latest version
  2. Update System.CommandLine from beta1 to beta4
  3. Modify Program.cs and add launchSettings.json for measuring average cold startup time

Benchmark results

dotnet run -c Release -- --launchCount 20

Method Mean Error StdDev Ratio RatioSD Allocated Alloc Ratio
'ConsoleAppFramework v5' 1.558 ms 0.1154 ms 0.1329 ms 1.01 0.11 400 B 1.00
CliFx 24.850 ms 0.5209 ms 0.5999 ms 16.04 1.25 57640 B 144.10
System.CommandLine 29.528 ms 3.3015 ms 3.8020 ms 19.07 2.79 18720 B 46.80
Cocona.Lite 47.108 ms 0.8073 ms 0.9297 ms 30.42 2.34 59784 B 149.46
Spectre.Console.Cli 54.705 ms 0.8616 ms 0.9922 ms 35.32 2.70 65680 B 164.20
Cocona 134.436 ms 1.5971 ms 1.8392 ms 86.80 6.56 461640 B 1,154.10

It seems System.CommandLine performance is significantly improved.

@neuecc

neuecc commented Feb 6, 2025

Copy link
Copy Markdown
Member

I see, so NamingConventionBuilder gives good results...?
I have no idea what the correct way to use System.CommandLine is, but it's interesting anyway.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants