Skip to content

Support a maximum batch size per DataSource - #76

Merged
1 commit merged into
masterfrom
peter-max-batch-size
Nov 9, 2016
Merged

Support a maximum batch size per DataSource#76
1 commit merged into
masterfrom
peter-max-batch-size

Conversation

@peterneyens

Copy link
Copy Markdown
Contributor
  • Add the maxBatchSizePhase interpreter which splits fetches to data sources with a maximum batch size in to multiple sequential fetches. This required a change to Concurrent to return a InMemoryCache instead of a DataSourceCache making it possible to combine multiple caches for sequential batches.
  • Refactor the processConcurrent method which is used in the coreInterpreter (previously called interpreter) by using XorT instead of nested if / else and Option.fold and by extracting some helper methods.
  • Give some members of the Fetch constructors a better name (eg. FetchMany.as to FetchMany.ids)
  • Create the Fetch.multiple method which makes it possible to create a Fetch[List[...]] directly (using FetchMany underneath). This can currently (and previously) be done by using sequence or traverse, but multiple doesn't have the overhead of inspecting the Fetch structure to find independent fetches which can be parallelized.

I changed `Concurrent` to have a `NonEmptyList` instead of a `List`,
which led to some issues with the invariance of `NonEmptyList` and the
existential type `FetchQuery[_, _]`, for the moment I changed it to
`NonEmptyList[FetchQuery[Any, Any]]`.
@codecov-io

Copy link
Copy Markdown

Current coverage is 75.87% (diff: 92.04%)

Merging #76 into master will increase coverage by 6.13%

@@             master        #76   diff @@
==========================================
  Files             8          8          
  Lines           195        228    +33   
  Methods         190        226    +36   
  Messages          0          0          
  Branches          5          2     -3   
==========================================
+ Hits            136        173    +37   
+ Misses           59         55     -4   
  Partials          0          0          

Powered by Codecov. Last update 26f7b45...cb60e82

@ghost

ghost commented Nov 9, 2016

Copy link
Copy Markdown

Outstanding work, well done! :shipit:

@ghost
ghost merged commit 58848d0 into master Nov 9, 2016
@ghost
ghost deleted the peter-max-batch-size branch November 9, 2016 10:42
This pull request was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants