[CLI] hf cache verify --fail-on-missing-files fails while corresponding folders are here

[Mitix-EPI]

Describe the bug

hf cache verify seems to warn for folders, even if any modification has been done to the downloaded model.
Here are full replication:

Reproduction

Current version: 1.3.2

1- Download model

hf download intfloat/multilingual-e5-large-instruct --local-dir test

2- Verify

HF_DEBUG=1 hf cache verify intfloat/multilingual-e5-large-instruct --fail-on-missing-files --local-dir test

Logs

Request f0cb15b8-fbaa-47a6-bedc-aedd6e65d3c5: GET https://huggingface.co/api/models/intfloat/multilingual-e5-large-instruct/tree/main?recursive=true&expand=false (authenticated: False)
Send: curl -X GET -H 'accept: */*' -H 'accept-encoding: gzip, deflate' -H 'connection: keep-alive' -H 'host: huggingface.co' -H 'user-agent: huggingface-cli/1.3.2; hf_hub/1.3.2; python/3.10.9' -H 'x-amzn-trace-id: f0cb15b8-fbaa-47a6-bedc-aedd6e65d3c5' -d 'https://huggingface.co/api/models/intfloat/multilingual-e5-large-instruct/tree/main?recursive=true&expand=false'
Missing files (present remotely, absent locally):
  - 1_Pooling
  - onnx
⚠️  39 local file(s) do not exist on the remote repo. Use --fail-on-extra-files for details.
❌ Verification failed for 'intfloat/multilingual-e5-large-instruct' (model) in /home/.../test.
  Revision: main

System info

- huggingface_hub version: 1.3.2
- Platform: Linux-6.2.0-060200-generic-x86_64-with-glibc2.35
- Python version: 3.10.9
- Running in iPython ?: No
- Running in notebook ?: No
- Running in Google Colab ?: No
- Running in Google Colab Enterprise ?: No
- Token path ?: /home/.../.cache/huggingface/token
- Has saved token ?: False
- Configured git credential helpers: cache
- Installation method: hf_installer
- httpx: 0.28.1
- hf_xet: 1.2.0
- gradio: N/A
- tensorboard: N/A
- ENDPOINT: https://huggingface.co
- HF_HUB_CACHE: /home/.../.cache/huggingface/hub
- HF_ASSETS_CACHE: /home/.../.cache/huggingface/assets
- HF_TOKEN_PATH: /home/.../.cache/huggingface/token
- HF_STORED_TOKENS_PATH: /home/.../.cache/huggingface/stored_tokens
- HF_HUB_OFFLINE: False
- HF_HUB_DISABLE_TELEMETRY: False
- HF_HUB_DISABLE_PROGRESS_BARS: None
- HF_HUB_DISABLE_SYMLINKS_WARNING: False
- HF_HUB_DISABLE_EXPERIMENTAL_WARNING: False
- HF_HUB_DISABLE_IMPLICIT_TOKEN: False
- HF_HUB_DISABLE_XET: False
- HF_HUB_ETAG_TIMEOUT: 10
- HF_HUB_DOWNLOAD_TIMEOUT: 10
- HF_XET_HIGH_PERFORMANCE: False