perf!: do not create copies of dir entries by KSXGitHub · Pull Request #285 · KSXGitHub/parallel-disk-usage

KSXGitHub · 2025-07-01T06:19:28Z

Breaking Change

AccessEntry has been removed from error_report::Operation.
Generic parameter Name of tree_builder::Info has been replaced with NameIter.
Generic parameter Name of TreeBuilder has been replaced with NameIter.

github-actions · 2025-07-01T06:26:40Z

Performance Regression Reports

commit: 783cfad

--quantity=block-size --max-depth=10 --min-ratio=0.01 --no-sort

Command	Mean [ms]	Min [ms]	Max [ms]	Relative
`pdu`	102.8 ± 0.7	101.8	104.3	1.12 ± 0.01
`pdu-0.12.0`	91.5 ± 0.6	90.4	93.3	1.00

Logs

Benchmark 1: pdu
  Time (mean ± σ):     102.8 ms ±   0.7 ms    [User: 104.0 ms, System: 293.3 ms]
  Range (min … max):   101.8 ms … 104.3 ms    28 runs
 
Benchmark 2: pdu-0.12.0
  Time (mean ± σ):      91.5 ms ±   0.6 ms    [User: 56.4 ms, System: 296.5 ms]
  Range (min … max):    90.4 ms …  93.3 ms    32 runs
 
Summary
  pdu-0.12.0 ran
    1.12 ± 0.01 times faster than pdu

JSON

{
  "results": [
    {
      "command": "pdu",
      "mean": 0.10281047641857144,
      "stddev": 0.0006752263359548726,
      "median": 0.10271400574,
      "user": 0.10397709714285716,
      "system": 0.29329136571428566,
      "min": 0.10176427874,
      "max": 0.10427257874000001,
      "times": [
        0.10289212974,
        0.10176427874,
        0.10246754474,
        0.10286234874,
        0.10204095774000001,
        0.10271020174,
        0.10238513674000001,
        0.10385906474,
        0.10208743474000001,
        0.10394535274000001,
        0.10267083074000001,
        0.10312483674,
        0.10287174574,
        0.10208412874,
        0.10232481474,
        0.10325602974,
        0.10274628474,
        0.10265829074,
        0.10412443274000001,
        0.10185434174,
        0.10427257874000001,
        0.10209522074,
        0.10271780974,
        0.10329889174000001,
        0.10359587874000001,
        0.10250697674,
        0.10285465474000001,
        0.10262114174
      ],
      "exit_codes": [
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0
      ]
    },
    {
      "command": "pdu-0.12.0",
      "mean": 0.09147583602125,
      "stddev": 0.0006201648283807041,
      "median": 0.09141706774,
      "user": 0.0563971775,
      "system": 0.29645967375,
      "min": 0.09039130174,
      "max": 0.09330191074000001,
      "times": [
        0.09116341474,
        0.09061541874000001,
        0.09129890874,
        0.09093501974,
        0.09057480674,
        0.09156440874,
        0.09071805674,
        0.09191862574000001,
        0.09113216174000001,
        0.09187452274,
        0.09194213174,
        0.09141604974,
        0.09091070574,
        0.09141808574,
        0.09203597174,
        0.09143179874,
        0.09228481874,
        0.09137187074,
        0.09159449274,
        0.09157235074,
        0.09039130174,
        0.09255973574000001,
        0.09103284274000001,
        0.09154589874,
        0.09148198674000001,
        0.09191221374,
        0.09242129774,
        0.09102147374,
        0.09129503974,
        0.09112465274,
        0.09136477774,
        0.09330191074000001
      ],
      "exit_codes": [
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0
      ]
    }
  ]
}

--quantity=apparent-size --max-depth=1 --min-ratio=0.01 --no-sort

Command	Mean [ms]	Min [ms]	Max [ms]	Relative
`pdu`	100.9 ± 12.3	94.3	132.4	1.15 ± 0.14
`pdu-0.12.0`	87.8 ± 0.7	86.9	89.8	1.00

Logs

Benchmark 1: pdu
  Time (mean ± σ):     100.9 ms ±  12.3 ms    [User: 80.3 ms, System: 293.6 ms]
  Range (min … max):    94.3 ms … 132.4 ms    31 runs
 
  Warning: Statistical outliers were detected. Consider re-running this benchmark on a quiet system without any interferences from other programs. It might help to use the '--warmup' or '--prepare' options.
 
Benchmark 2: pdu-0.12.0
  Time (mean ± σ):      87.8 ms ±   0.7 ms    [User: 55.7 ms, System: 287.1 ms]
  Range (min … max):    86.9 ms …  89.8 ms    33 runs
 
Summary
  pdu-0.12.0 ran
    1.15 ± 0.14 times faster than pdu

JSON

{
  "results": [
    {
      "command": "pdu",
      "mean": 0.10093710173096775,
      "stddev": 0.012287430445830637,
      "median": 0.09545421286,
      "user": 0.08028180967741937,
      "system": 0.29355121612903223,
      "min": 0.09425074586,
      "max": 0.13243146086000002,
      "times": [
        0.09568934286,
        0.09543283386,
        0.09544669886,
        0.09556071186,
        0.09547096486000001,
        0.09556284886000001,
        0.09506226086,
        0.09495582986000001,
        0.09600068386,
        0.09545421286,
        0.09611994586,
        0.09442848686000001,
        0.09523775086,
        0.09490114086,
        0.09453052886,
        0.09544995486,
        0.09460622686,
        0.09425074586,
        0.09429014886,
        0.11378722386000001,
        0.13116425286,
        0.11831246786,
        0.13243146086000002,
        0.13181763486,
        0.12054745386,
        0.09596289186,
        0.09594847786,
        0.09464351886,
        0.09547585386,
        0.09516498386,
        0.09534261486000001
      ],
      "exit_codes": [
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0
      ]
    },
    {
      "command": "pdu-0.12.0",
      "mean": 0.0877991516781818,
      "stddev": 0.0006655444299082072,
      "median": 0.08768777086,
      "user": 0.05574582727272726,
      "system": 0.2870592454545455,
      "min": 0.08687878086,
      "max": 0.08982392486,
      "times": [
        0.08783713086,
        0.08768777086,
        0.08747560386,
        0.08784428386,
        0.08792716986,
        0.08796493886,
        0.08784599186,
        0.08775713886,
        0.08740133886,
        0.08723363386,
        0.08768059886,
        0.08786411086,
        0.08982392486,
        0.08732284586,
        0.08757514186,
        0.08759477986,
        0.08784018586,
        0.08938271686,
        0.08855055586,
        0.08805027386,
        0.08947350686,
        0.08776553486,
        0.08764431386,
        0.08765929686,
        0.08725368086,
        0.08771310886,
        0.08736893586,
        0.08710001386,
        0.08696305886,
        0.08766444786,
        0.08711660786,
        0.08811058186,
        0.08687878086
      ],
      "exit_codes": [
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0
      ]
    }
  ]
}

--quantity=apparent-size --max-depth=10 --min-ratio=0.01 --no-sort

Command	Mean [ms]	Min [ms]	Max [ms]	Relative
`pdu`	104.1 ± 1.0	102.1	106.5	1.13 ± 0.01
`pdu-0.12.0`	92.3 ± 0.8	90.3	94.2	1.00

Logs

Benchmark 1: pdu
  Time (mean ± σ):     104.1 ms ±   1.0 ms    [User: 97.6 ms, System: 300.2 ms]
  Range (min … max):   102.1 ms … 106.5 ms    28 runs
 
Benchmark 2: pdu-0.12.0
  Time (mean ± σ):      92.3 ms ±   0.8 ms    [User: 58.1 ms, System: 293.8 ms]
  Range (min … max):    90.3 ms …  94.2 ms    31 runs
 
Summary
  pdu-0.12.0 ran
    1.13 ± 0.01 times faster than pdu

JSON

{
  "results": [
    {
      "command": "pdu",
      "mean": 0.1041418226442857,
      "stddev": 0.0009907315808749223,
      "median": 0.10407273418000001,
      "user": 0.09764768428571428,
      "system": 0.30018680857142854,
      "min": 0.10213127618000001,
      "max": 0.10650694318000001,
      "times": [
        0.10464852618,
        0.10388320518000001,
        0.10352611718,
        0.10308081718,
        0.10356259518000001,
        0.10279717918,
        0.10372628418,
        0.10458056318,
        0.10294761818,
        0.10213127618000001,
        0.10399409118000001,
        0.10427809218,
        0.10381384218,
        0.10388228318,
        0.10415137718,
        0.10491232618,
        0.10421001418,
        0.10359203318,
        0.10337576618000001,
        0.10424706318,
        0.10650694318000001,
        0.10617118618,
        0.10448534118000001,
        0.10508957718,
        0.10333118118000001,
        0.10496716818,
        0.10584697118,
        0.10423159518000001
      ],
      "exit_codes": [
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0
      ]
    },
    {
      "command": "pdu-0.12.0",
      "mean": 0.09226974695419357,
      "stddev": 0.0008492499442072674,
      "median": 0.09212968518,
      "user": 0.05806975225806451,
      "system": 0.29379633161290314,
      "min": 0.09025736818,
      "max": 0.09417273518000001,
      "times": [
        0.09417273518000001,
        0.09344239918000001,
        0.09245797618,
        0.09173354618,
        0.09177504918,
        0.09232946318,
        0.09297938718,
        0.09203406218,
        0.09230436218,
        0.09212968518,
        0.09169285518,
        0.09343173818,
        0.09056113218,
        0.09025736818,
        0.09204893018,
        0.09189553018,
        0.09192745118000001,
        0.09207416718,
        0.09239347418,
        0.09209117918000001,
        0.09319048118,
        0.09313665118,
        0.09316206618,
        0.09321672018,
        0.09197828018000001,
        0.09231546718,
        0.09131779418000001,
        0.09093891818000001,
        0.09269756318,
        0.09265146518,
        0.09202425718
      ],
      "exit_codes": [
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0
      ]
    }
  ]
}

--quantity=apparent-size --max-depth=1 --min-ratio=0.01

Command	Mean [ms]	Min [ms]	Max [ms]	Relative
`pdu`	103.8 ± 11.1	94.3	139.8	1.17 ± 0.13
`pdu-0.12.0`	88.4 ± 1.1	87.5	93.5	1.00

Logs

Benchmark 1: pdu
  Time (mean ± σ):     103.8 ms ±  11.1 ms    [User: 80.3 ms, System: 294.5 ms]
  Range (min … max):    94.3 ms … 139.8 ms    31 runs
 
Benchmark 2: pdu-0.12.0
  Time (mean ± σ):      88.4 ms ±   1.1 ms    [User: 54.1 ms, System: 290.3 ms]
  Range (min … max):    87.5 ms …  93.5 ms    33 runs
 
Summary
  pdu-0.12.0 ran
    1.17 ± 0.13 times faster than pdu

JSON

{
  "results": [
    {
      "command": "pdu",
      "mean": 0.10379374585806453,
      "stddev": 0.011104302546835924,
      "median": 0.0981882696,
      "user": 0.08026363548387096,
      "system": 0.29447656709677417,
      "min": 0.0943415006,
      "max": 0.13980796160000003,
      "times": [
        0.09550896960000001,
        0.09649503960000001,
        0.1011407356,
        0.0950864936,
        0.09599976660000001,
        0.1008470266,
        0.0955424596,
        0.09505107460000001,
        0.0951104656,
        0.0943415006,
        0.0945175406,
        0.09450034460000001,
        0.0955863866,
        0.09465416160000001,
        0.09580098760000001,
        0.0951552276,
        0.11599887960000001,
        0.1214407596,
        0.12239597160000001,
        0.13980796160000003,
        0.1135160236,
        0.09766143860000001,
        0.0981882696,
        0.1002589886,
        0.11334956060000001,
        0.11562567160000001,
        0.1111084586,
        0.1083569496,
        0.1062995956,
        0.1107943526,
        0.10746505960000001
      ],
      "exit_codes": [
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0
      ]
    },
    {
      "command": "pdu-0.12.0",
      "mean": 0.08843758638787878,
      "stddev": 0.0011304284154665305,
      "median": 0.0881324666,
      "user": 0.05414554848484848,
      "system": 0.29031851333333325,
      "min": 0.08745561560000001,
      "max": 0.0935097166,
      "times": [
        0.0876446986,
        0.0886297756,
        0.0881324666,
        0.08752342960000001,
        0.0880827306,
        0.0886249496,
        0.0884504006,
        0.0886560846,
        0.08810323860000001,
        0.08825911260000001,
        0.08881780660000001,
        0.0882227976,
        0.0907521686,
        0.0935097166,
        0.0879857126,
        0.08844862360000001,
        0.0878604486,
        0.0878050456,
        0.08769854660000001,
        0.0875075696,
        0.08897214460000001,
        0.0883561836,
        0.08789948160000001,
        0.08916928560000001,
        0.08951913560000001,
        0.08804849760000001,
        0.0877117946,
        0.08877965560000001,
        0.0882356266,
        0.08747844560000001,
        0.08745561560000001,
        0.0879678366,
        0.0881313246
      ],
      "exit_codes": [
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0
      ]
    }
  ]
}

--quantity=block-size --max-depth=10 --min-ratio=0.01

Command	Mean [ms]	Min [ms]	Max [ms]	Relative
`pdu`	103.2 ± 1.4	101.6	109.1	1.12 ± 0.04
`pdu-0.12.0`	92.3 ± 3.1	90.5	106.5	1.00

Logs

Benchmark 1: pdu
  Time (mean ± σ):     103.2 ms ±   1.4 ms    [User: 99.7 ms, System: 299.1 ms]
  Range (min … max):   101.6 ms … 109.1 ms    29 runs
 
Benchmark 2: pdu-0.12.0
  Time (mean ± σ):      92.3 ms ±   3.1 ms    [User: 60.2 ms, System: 292.9 ms]
  Range (min … max):    90.5 ms … 106.5 ms    32 runs
 
  Warning: Statistical outliers were detected. Consider re-running this benchmark on a quiet system without any interferences from other programs. It might help to use the '--warmup' or '--prepare' options.
 
Summary
  pdu-0.12.0 ran
    1.12 ± 0.04 times faster than pdu

JSON

{
  "results": [
    {
      "command": "pdu",
      "mean": 0.10318983005517242,
      "stddev": 0.0014051937700782788,
      "median": 0.10294413240000001,
      "user": 0.09966900344827585,
      "system": 0.29912510965517236,
      "min": 0.10156161340000001,
      "max": 0.10911081640000002,
      "times": [
        0.10246098040000001,
        0.10911081640000002,
        0.10331475140000002,
        0.10309228840000001,
        0.10385840840000002,
        0.10286621940000001,
        0.1038443804,
        0.10192699540000001,
        0.10310878640000001,
        0.10156161340000001,
        0.10358500840000001,
        0.10342945840000001,
        0.10227444140000001,
        0.10162456640000002,
        0.10294413240000001,
        0.10224899640000001,
        0.10288060040000001,
        0.10403892640000001,
        0.10488542940000001,
        0.10433100840000001,
        0.10395459040000002,
        0.1028244964,
        0.10299496740000001,
        0.10213215640000001,
        0.10211689740000002,
        0.10224672140000002,
        0.10287261840000002,
        0.10365887440000002,
        0.10231594140000001
      ],
      "exit_codes": [
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0
      ]
    },
    {
      "command": "pdu-0.12.0",
      "mean": 0.09230000793125001,
      "stddev": 0.0031444567517014714,
      "median": 0.0915340349,
      "user": 0.060227775,
      "system": 0.292916545,
      "min": 0.09046727840000002,
      "max": 0.10647619340000002,
      "times": [
        0.09147680140000002,
        0.09133453640000001,
        0.09141655140000002,
        0.09174149540000001,
        0.09167984440000002,
        0.09158019540000001,
        0.09165263840000001,
        0.09126122340000001,
        0.09169449940000002,
        0.09151711840000001,
        0.09155095140000001,
        0.09303426040000001,
        0.09230164240000001,
        0.09134368240000001,
        0.09105251040000001,
        0.09201770340000001,
        0.0916570304,
        0.0918984614,
        0.09069153440000001,
        0.09136950240000001,
        0.09155702240000001,
        0.10647619340000002,
        0.10121670340000001,
        0.0910297864,
        0.0911762744,
        0.09113482240000001,
        0.09118992740000001,
        0.0928592064,
        0.09107751640000002,
        0.09170919940000001,
        0.09143414040000002,
        0.09046727840000002
      ],
      "exit_codes": [
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0
      ]
    }
  ]
}

KSXGitHub · 2025-07-01T19:49:11Z

Horrible performance. Closed.

feat(api)!: stop reporting dir entry errors

d2719ec

perf!: do not create copies of dir entries

84b2321

KSXGitHub force-pushed the do-not-copy-dir-entries branch 2 times, most recently from d81be04 to 84b2321 Compare July 1, 2025 11:09

KSXGitHub mentioned this pull request Jul 1, 2025

Randomly killed on large volume #277

Open

KSXGitHub added the wontfix This will not be worked on label Jul 1, 2025

Merge branch 'master' into do-not-copy-dir-entries

6e40c45

KSXGitHub closed this Jul 1, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

perf!: do not create copies of dir entries#285

perf!: do not create copies of dir entries#285
KSXGitHub wants to merge 3 commits intomasterfrom
do-not-copy-dir-entries

KSXGitHub commented Jul 1, 2025 •

edited

Loading

Uh oh!

github-actions Bot commented Jul 1, 2025 •

edited

Loading

Uh oh!

KSXGitHub commented Jul 1, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

KSXGitHub commented Jul 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Breaking Change

Uh oh!

github-actions Bot commented Jul 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Performance Regression Reports

Uh oh!

KSXGitHub commented Jul 1, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

KSXGitHub commented Jul 1, 2025 •

edited

Loading

github-actions Bot commented Jul 1, 2025 •

edited

Loading