Skip to content

Commit dd89b78

Browse files
committed
Update CHANGELOG.md and docs
1 parent ceb7496 commit dd89b78

3 files changed

Lines changed: 41 additions & 6 deletions

File tree

CHANGELOG.md

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,29 @@
1+
# 8.0.0, 2026-06-27
2+
3+
See [the migration docs](MIGRATING_FROM_OLDER_VERSIONS.md) for details on breaking changes.
4+
5+
- Remove deprecated top-level smart_open() function (PR [#928](https://github.com/piskvorky/smart_open/pull/928), [@ddelange](https://github.com/ddelange))
6+
- Remove deprecated concurrency.create_pool and ConcurrentFuturesPool (PR [#931](https://github.com/piskvorky/smart_open/pull/931), [@ddelange](https://github.com/ddelange))
7+
- [hdfs] Support host and port in HDFS URIs (PR [#168](https://github.com/piskvorky/smart_open/pull/168), [@vvaten](https://github.com/vvaten))
8+
- Remove stale Python 2.7 references in comments and docstrings (PR [#932](https://github.com/piskvorky/smart_open/pull/932), [@ddelange](https://github.com/ddelange))
9+
- Remove deprecated compression.tweak_close() (PR [#933](https://github.com/piskvorky/smart_open/pull/933), [@ddelange](https://github.com/ddelange))
10+
- Simplify inspect_kwargs() by dropping getargspec fallback (PR [#934](https://github.com/piskvorky/smart_open/pull/934), [@ddelange](https://github.com/ddelange))
11+
- [s3] Pass session_kwargs as a dict to s3.iter_bucket (PR [#930](https://github.com/piskvorky/smart_open/pull/930), [@ddelange](https://github.com/ddelange))
12+
- [gcs] Remove deprecated buffer_size/line_terminator params (PR [#935](https://github.com/piskvorky/smart_open/pull/935), [@ddelange](https://github.com/ddelange))
13+
- [gcs] Remove no-op _blob.terminate shim in Writer (PR [#936](https://github.com/piskvorky/smart_open/pull/936), [@ddelange](https://github.com/ddelange))
14+
- [s3] Remove deprecated transport params from open_uri (PR [#937](https://github.com/piskvorky/smart_open/pull/937), [@ddelange](https://github.com/ddelange))
15+
- Remove smart_open_lib backwards-compat re-exports (PR [#929](https://github.com/piskvorky/smart_open/pull/929), [@ddelange](https://github.com/ddelange))
16+
- [s3] Remove deprecated smart_open.s3_iter_bucket shim (PR [#927](https://github.com/piskvorky/smart_open/pull/927), [@ddelange](https://github.com/ddelange))
17+
- Refresh stale RaRe-Technologies URLs and boto wording (PR [#938](https://github.com/piskvorky/smart_open/pull/938), [@ddelange](https://github.com/ddelange))
18+
- Document #926 breaking changes in MIGRATING (PR [#939](https://github.com/piskvorky/smart_open/pull/939), [@ddelange](https://github.com/ddelange))
19+
- Add compression_kwargs parameter to smart_open.open() (PR [#940](https://github.com/piskvorky/smart_open/pull/940), [@ddelange](https://github.com/ddelange))
20+
- [s3] Drop host/port from S3 URIs, drop s3u scheme (PR [#943](https://github.com/piskvorky/smart_open/pull/943), [@ddelange](https://github.com/ddelange))
21+
- [gcs] Use gcs:// as canonical GCS scheme, keep gs:// as alias (PR [#944](https://github.com/piskvorky/smart_open/pull/944), [@ddelange](https://github.com/ddelange))
22+
- Parse versionId from s3:// URIs into version_id (PR [#945](https://github.com/piskvorky/smart_open/pull/945), [@ddelange](https://github.com/ddelange))
23+
- Add pre-commit and migrate from flake8 to ruff (PR [#946](https://github.com/piskvorky/smart_open/pull/946), [@ddelange](https://github.com/ddelange))
24+
- Migrate RST docs to Markdown (PR [#947](https://github.com/piskvorky/smart_open/pull/947), [@ddelange](https://github.com/ddelange))
25+
- Add type annotations and ty type checker (PR [#948](https://github.com/piskvorky/smart_open/pull/948), [@ddelange](https://github.com/ddelange))
26+
127
# 7.7.1, 2026-06-26
228

329
- [http,azure] Optimize forward seeks within buffered data to avoid redundant requests (PR [#923](https://github.com/piskvorky/smart_open/pull/923), [@ddelange](https://github.com/ddelange))

README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -382,7 +382,7 @@ fout = open(
382382

383383
### Iterating Over an S3 Bucket's Contents
384384

385-
Since going over all (or select) keys in an S3 bucket is a very common operation, there's also an extra function `smart_open.s3.iter_bucket()` that does this efficiently, **processing the bucket keys in parallel** (using multiprocessing):
385+
Since going over all (or select) keys in an S3 bucket is a very common operation, there's also an extra function `smart_open.s3.iter_bucket()` that does this efficiently, **processing the bucket keys in parallel** (using multithreading):
386386

387387
```python
388388
>>> from smart_open import s3

howto.md

Lines changed: 14 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -194,11 +194,19 @@ It is possible to save both CPU time and memory by sharing the same resource acr
194194

195195
```python
196196
>>> import boto3
197+
>>> import botocore.client
197198
>>> from smart_open import open
198-
>>> tp = {'client': boto3.client('s3')}
199+
>>> # this mirrors the adaptive-retry config smart_open uses internally when it
200+
>>> # creates a thread-safe client for you (see smart_open.s3)
201+
>>> config = botocore.client.Config(
202+
... max_pool_connections=64,
203+
... tcp_keepalive=True,
204+
... retries={'max_attempts': 6, 'mode': 'adaptive'},
205+
... )
206+
>>> transport_params = {'client': boto3.client('s3', config=config)}
199207
>>> for month in (1, 2, 3):
200208
... url = 's3://nyc-tlc/trip data/yellow_tripdata_2020-%02d.csv' % month
201-
... with open(url, transport_params=tp) as fin:
209+
... with open(url, transport_params=transport_params) as fin:
202210
... _ = fin.readline() # skip CSV header
203211
... print(fin.readline().strip())
204212
1,2020-01-01 00:28:15,2020-01-01 00:33:03,1,1.20,1,N,238,239,1,6,3,0.5,1.47,0,0.3,11.27,2.5
@@ -207,7 +215,8 @@ It is possible to save both CPU time and memory by sharing the same resource acr
207215

208216
```
209217

210-
Clients are thread-safe and multiprocess-safe, so you may share them between other threads and subprocesses.
218+
Clients are generally thread-safe, so you may share a single client across threads ([ref](https://github.com/boto/boto3/blob/1.38.41/docs/source/guide/clients.rst?plain=1#L111)); note they cannot be shared across processes.
219+
Note that long-lived clients can leak memory ([boto/boto3#1670](https://github.com/boto/boto3/issues/1670)), so prefer creating a client per operation (or periodically recreating it) for long-running processes.
211220

212221
## How to Write to S3 Efficiently
213222

@@ -235,8 +244,8 @@ import boto3
235244
from smart_open import open
236245

237246
with tempfile.NamedTemporaryFile() as tmp:
238-
tp = {"writebuffer": tmp}
239-
with open("s3://bucket/key", "w", transport_params=tp) as fout:
247+
transport_params = {"writebuffer": tmp}
248+
with open("s3://bucket/key", "w", transport_params=transport_params) as fout:
240249
fout.write(lots_of_data)
241250
```
242251

0 commit comments

Comments
 (0)