feat: pass skip_sha256=True to hf_xet for bucket uploads (#3900)

* feat: pass skip_sha256=True to hf_xet for bucket uploads

Bucket uploads don't need SHA-256 in the shard metadata (the sha_index
GSI is only used for LFS pointer resolution, which doesn't apply to
buckets). Pass skip_sha256=True to hf_xet.upload_files() and
upload_bytes() in the bucket upload path to skip the SHA-256
computation, removing the main CPU bottleneck on non-SHA-NI instances.

Depends on: huggingface/xet-core#679

Co-authored-by: Lucain <Wauplin@users.noreply.github.com>

* test: use real bucket upload instead of mocks for skip_sha256 test

Replace the two mock-based tests with a single integration test that:
- Creates a real Bucket on staging Hub
- Uploads files from both filepath and bytes in a single batch
- Wraps (not mocks) hf_xet.upload_files and hf_xet.upload_bytes to
  verify skip_sha256=True is passed
- Verifies files are actually uploaded by listing the bucket tree

Co-authored-by: Lucain <Wauplin@users.noreply.github.com>

* test: skip skip_sha256 test when hf_xet doesn't support it yet

The test wraps the real hf_xet functions, so it fails when the
installed hf_xet predates the skip_sha256 parameter (xet-core#679).
Use inspect.signature to detect support and pytest.skip accordingly.

Co-authored-by: Lucain <Wauplin@users.noreply.github.com>

* test: handle built-in functions in skip_sha256 signature check

hf_xet.upload_files is a compiled built-in function, so
inspect.signature() raises ValueError. Catch it and skip the test
when the signature can't be introspected (older hf_xet).

Co-authored-by: Lucain <Wauplin@users.noreply.github.com>

* fix: gracefully fall back when hf_xet lacks skip_sha256 support

Use try/except TypeError around upload_files/upload_bytes calls with
skip_sha256=True, falling back to calls without it for older hf_xet
versions. TypeError for unknown kwargs on compiled functions is raised
before any I/O, so the fallback is safe.

Update test to check call_args_list[0] (the first attempt always
includes skip_sha256=True) instead of requiring the function to
accept it.

Co-authored-by: Lucain <Wauplin@users.noreply.github.com>

* better like this

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Lucain <Wauplin@users.noreply.github.com>

Author: Wauplin

src/huggingface_hub/hf_api.py

@@ -12213,6 +12213,7 @@ def token_refresher() -> tuple[str, int]:
                     token_refresher,
                     progress_callback,
                     "bucket",
+                    skip_sha256=True,
                 )
                 for upload_info, op in zip(xet_upload_infos, add_path_operations):
                     op.xet_hash = upload_info.hash
@@ -12229,6 +12230,7 @@ def token_refresher() -> tuple[str, int]:
                     token_refresher,
                     progress_callback,
                     "bucket",
+                    skip_sha256=True,
                 )
                 for upload_info, op in zip(xet_upload_infos, add_bytes_operations):
                     op.xet_hash = upload_info.hash

tests/test_xet_upload.py

@@ -339,6 +339,41 @@ def test_upload_folder_create_pr(self, api, repo_url) -> None:
             assert Path(local_path).read_bytes() == Path(filepath).read_bytes()
 
 
+@requires("hf_xet")
+class TestBucketXetUploadSkipSha256:
+    """Test that bucket uploads pass skip_sha256=True to hf_xet."""
+
+    def test_skip_sha256_passed_for_bucket_uploads(self, api, tmp_path):
+        """Upload from both filepath and bytes to a real bucket, verifying skip_sha256=True is passed."""
+        from hf_xet import upload_bytes as real_upload_bytes
+        from hf_xet import upload_files as real_upload_files
+
+        bucket_url = api.create_bucket(repo_name(prefix="bucket"))
+        bucket_id = bucket_url.bucket_id
+
+        test_file = tmp_path / "test_file.bin"
+        test_file.write_bytes(b"file content for bucket test")
+
+        with patch("hf_xet.upload_files", wraps=real_upload_files) as spy_upload_files:
+            with patch("hf_xet.upload_bytes", wraps=real_upload_bytes) as spy_upload_bytes:
+                api.batch_bucket_files(
+                    bucket_id,
+                    add=[
+                        (str(test_file), "from_path.bin"),
+                        (b"bytes content for bucket test", "from_bytes.bin"),
+                    ],
+                )
+
+                assert spy_upload_files.call_args_list[0].kwargs.get("skip_sha256") is True
+                assert spy_upload_bytes.call_args_list[0].kwargs.get("skip_sha256") is True
+
+        uploaded = {e.path for e in api.list_bucket_tree(bucket_id)}
+        assert "from_path.bin" in uploaded
+        assert "from_bytes.bin" in uploaded
+
+        api.delete_bucket(bucket_id)
+
+
 @requires("hf_xet")
 class TestXetLargeUpload:
     def test_upload_large_folder(self, api, tmp_path, repo_url: RepoUrl) -> None: