Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 36 additions & 5 deletions docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -235,9 +235,13 @@ Note how the two independent data fetches run in parallel, minimizing the latenc
Fetch.run[IO](fetchMulti).unsafeRunTimed(5.seconds)
```

## Caching
## Deduplication & Caching

When fetching an identity, subsequent fetches for the same identity are cached. Let's try creating a fetch that asks for the same identity twice.
The Fetch library supports deduplication and optional caching.
By default, fetches that are chained together will share the same cache backend, providing some deduplication.

When fetching an identity twice within the same `Fetch`, such as a batch of fetches or when you `flatMap` one fetch into another, subsequent fetches for the same identity are cached.
Let's try creating a fetch that asks for the same identity twice, by using `flatMap` (in a for-comprehension) to chain the requests together:

```scala mdoc:silent
def fetchTwice[F[_] : Concurrent]: Fetch[F, (String, String)] = for {
Expand All @@ -246,12 +250,39 @@ def fetchTwice[F[_] : Concurrent]: Fetch[F, (String, String)] = for {
} yield (one, two)
```

While running it, notice that the data source is only queried once. The next time the identity is requested, it's served from the cache.
While running it, notice that the data source is only queried once.
The next time the identity is requested, it's served from the internal cache.

```scala mdoc:silent
val runFetchTwice = Fetch.run[IO](fetchTwice)
```
```scala mdoc
runFetchTwice.unsafeRunTimed(5.seconds)
```

This will still fetch the data again, however, if we call it once more:
```scala mdoc
runFetchTwice.unsafeRunTimed(5.seconds)
```

If we want to cache between multiple individual fetches, you should use `Fetch.runCache` or `Fetch.runAll` to return the cache for reusing later.
Here is an example where we fetch four separate times, and explicitly share the cache to keep the deduplication functionality:

```scala mdoc:silent
//We get the cache from the first run and pass it to all subsequent fetches
val runFetchFourTimesSharedCache = for {
(cache, one) <- Fetch.runCache[IO](fetchString(1))
two <- Fetch.run[IO](fetchString(1), cache)
three <- Fetch.run[IO](fetchString(1), cache)
four <- Fetch.run[IO](fetchString(1), cache)
} yield (one, two, three, four)
```
```scala mdoc
Fetch.run[IO](fetchTwice).unsafeRunTimed(5.seconds)
runFetchFourTimesSharedCache.unsafeRunTimed(5.seconds)
```

As you can see above, the cache will now work between calls and can be used to deduplicate requests over a period of time.
Note that this does not support any kind of automatic cache invalidation, so you will need to keep track of which values you want to re-fetch if you plan on sharing the cache.

```scala mdoc:invisible
executor.shutdownNow()
Expand All @@ -264,4 +295,4 @@ For more in-depth information, take a look at our [documentation](https://47degr

Fetch is designed and developed by 47 Degrees

Copyright (C) 2016-2019 47 Degrees. <http://47deg.com>
Copyright (C) @YEAR_RANGE@ 47 Degrees. <http://47deg.com>
8 changes: 4 additions & 4 deletions microsite/docs/docs.md
Original file line number Diff line number Diff line change
Expand Up @@ -142,7 +142,7 @@ object Users extends Data[UserId, User] {
latency[F](s"One User $id") >> CF.pure(userDatabase.get(id))

override def batch(ids: NonEmptyList[UserId]): F[Map[UserId, User]] =
latency[F](s"Batch Users $ids") >> CF.pure(userDatabase.filterKeys(ids.toList.toSet).toMap)
latency[F](s"Batch Users $ids") >> CF.pure(userDatabase.view.filterKeys(ids.toList.toSet).toMap)
}
}
```
Expand Down Expand Up @@ -379,7 +379,7 @@ object Posts extends Data[PostId, Post] {
latency[F](s"One Post $id") >> CF.pure(postDatabase.get(id))

override def batch(ids: NonEmptyList[PostId]): F[Map[PostId, Post]] =
latency[F](s"Batch Posts $ids") >> CF.pure(postDatabase.filterKeys(ids.toList.toSet).toMap)
latency[F](s"Batch Posts $ids") >> CF.pure(postDatabase.view.filterKeys(ids.toList.toSet).toMap)
}
}

Expand Down Expand Up @@ -614,7 +614,7 @@ object BatchedUsers extends Data[UserId, User]{
latency[F](s"One User $id") >> CF.pure(userDatabase.get(id))

override def batch(ids: NonEmptyList[UserId]): F[Map[UserId, User]] =
latency[F](s"Batch Users $ids") >> CF.pure(userDatabase.filterKeys(ids.toList.toSet).toMap)
latency[F](s"Batch Users $ids") >> CF.pure(userDatabase.view.filterKeys(ids.toList.toSet).toMap)
}
}

Expand Down Expand Up @@ -652,7 +652,7 @@ object SequentialUsers extends Data[UserId, User]{
latency[F](s"One User $id") >> CF.pure(userDatabase.get(id))

override def batch(ids: NonEmptyList[UserId]): F[Map[UserId, User]] =
latency[F](s"Batch Users $ids") >> CF.pure(userDatabase.filterKeys(ids.toList.toSet).toMap)
latency[F](s"Batch Users $ids") >> CF.pure(userDatabase.view.filterKeys(ids.toList.toSet).toMap)
}
}

Expand Down