Instance-level dispatch table

[ogrisel]

This change is necessary to make it natural to subclass CloudPickler with a per-instance, custom dispatch table.

joblib/loky#260

I also took the opportunity to make it such that dispatch_table are isolated by default: meaning that mutated one does not impact other instances.

I still need to document this change in a changelog entry.

ping @pierreglaser @tomMoral @jakirkham you might be interested in this.

[codecov]

Codecov Report

❌ Patch coverage is 50.00000% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 82.67%. Comparing base (d3279a0) to head (d6a0263).
⚠️ Report is 90 commits behind head on master.

Files with missing lines	Patch %	Lines
cloudpickle/cloudpickle_fast.py	50.00%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #395      +/-   ##
==========================================
- Coverage   91.61%   82.67%   -8.95%     
==========================================
  Files           3        3              
  Lines         656      658       +2     
  Branches      135      135              
==========================================
- Hits          601      544      -57     
- Misses         34       90      +56     
- Partials       21       24       +3     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[jakirkham]

Thanks Olivier! 😄 This seems like a nice improvement. Had a small question below

[jakirkham]

Should we have an internal variable to point to this?

[ogrisel]

That's a good suggestion. I still need to find the time to understand the cause of the PyPy segfault.

[ogrisel]

Actually before considering merging this change we might also want to do some benchmark. Maybe ChainMap is not that fast when doing many consecutive getitems with maybe misses on the first map which will be empty most of the time.

[ogrisel]

Note that we found a workaround for the problem we got in the subclass in loky: joblib/loky#272. Still this behavior is really unexpected so instance level isolation and lazy init of the dispatch_table might be more intuitive for other subclassing this Pickler.

[jakirkham]

Yeah this change still seems reasonable to me FWIW 🙂

[faucct]

Out project has just stumbled on this issue. What is left to do? Do you need any help completing this?

[jakirkham]

Probably still the PyPy segfaulting issue noted here ( #395 (comment) )

[ogrisel]

@pierreglaser this PR requires to be updated (or reworked from scratch) now that we merged #517.

I plan to tag 3.0.0 today and publish to pypi.org and conda-forge on Monday, so I don't want to do that work for 3.0.0.

But if anybody is interesting in working on this, we could cut a quick 3.1.0 release in the coming weeks.