Skip to content

Cross-building with Scala 3 #443

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
benderpremier merged 18 commits into from
Apr 3, 2021
Merged

Cross-building with Scala 3 #443

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
6f36a95
adapt the build to crosscompile to scala3
benderpremier Feb 19, 2021
cd951f6
explicitely use Any and adapt for comprehension without better monadi…
benderpremier Feb 19, 2021
83fde52
Syntax Migration Warning method unsafeToFuture must be called with ()…
benderpremier Feb 19, 2021
be4763c
scala 3 on fetch-debug
benderpremier Feb 19, 2021
cf80aa6
change clashing symbol :: to :-:
benderpremier Feb 19, 2021
b79c92c
adapt fetch-example for available libraries
benderpremier Feb 22, 2021
9308a4a
scalafmt
benderpremier Feb 22, 2021
d3e1c45
move versions
benderpremier Feb 25, 2021
0b723c4
deactivate example while waiting for deps to be published
benderpremier Feb 25, 2021
c2fcfa9
use 2.13 as base version
benderpremier Feb 25, 2021
badd0c9
remove global cross versions
Mar 2, 2021
bdbdd34
remove code coverage
Mar 2, 2021
af457c8
test cross versions
Mar 2, 2021
692541d
remove remnant url
Mar 23, 2021
ad6708a
uncomment the tests
Mar 27, 2021
ffa1c5a
Merge branch 'master' into ISSUE-439-build-with-scala3
juanpedromoreno Mar 31, 2021
2abeab0
merge master
Apr 2, 2021
13942f1
update to RC2
Apr 2, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 19 additions & 5 deletions build.sbt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,39 +1,53 @@
ThisBuild / scalaVersion := "2.13.2"
ThisBuild / crossScalaVersions := Seq("2.12.11", "2.13.2")
ThisBuild / scalaVersion := scala213
ThisBuild / organization := "com.47deg"

addCommandAlias("ci-test", "scalafmtCheckAll; scalafmtSbtCheck; mdoc; testCovered")
addCommandAlias("ci-test", "scalafmtCheckAll; scalafmtSbtCheck; mdoc; ++test")
addCommandAlias("ci-docs", "github; mdoc; headerCreateAll; publishMicrosite")
addCommandAlias("ci-publish", "github; ci-release")

lazy val scala212 = "2.12.12"
lazy val scala213 = "2.13.5"
lazy val scala3Version = "3.0.0-RC2"
lazy val scala2Versions = Seq(scala212, scala213)
lazy val allScalaVersions = scala2Versions :+ scala3Version

skip in publish := true

lazy val fetch = crossProject(JSPlatform, JVMPlatform)
.crossType(CrossType.Pure)
.settings(commonCrossDependencies)
.settings(crossScalaVersions := allScalaVersions)

lazy val fetchJVM = fetch.jvm
lazy val fetchJS = fetch.js.disablePlugins(ScoverageSbtPlugin)
lazy val fetchJS = fetch.js
.settings(crossScalaVersions := scala2Versions)

lazy val `fetch-debug` = crossProject(JSPlatform, JVMPlatform)
.crossType(CrossType.Pure)
.dependsOn(fetch)
.settings(commonCrossDependencies)
.settings(crossScalaVersions := allScalaVersions)

lazy val debugJVM = `fetch-debug`.jvm
lazy val debugJS = `fetch-debug`.js.disablePlugins(ScoverageSbtPlugin)
lazy val debugJS = `fetch-debug`.js
.settings(crossScalaVersions := scala2Versions)

lazy val `fetch-examples` = project
.dependsOn(fetchJVM, debugJVM)
.settings(skip in publish := true)
.settings(examplesSettings: _*)
.settings(crossScalaVersions := scala2Versions)

lazy val microsite = project
.dependsOn(fetchJVM, debugJVM)
.settings(docsSettings: _*)
.settings(skip in publish := true)
.enablePlugins(MicrositesPlugin, MdocPlugin)
.settings(crossScalaVersions := scala2Versions)

lazy val documentation = project
.dependsOn(fetchJVM)
.settings(skip in publish := true)
.settings(mdocOut := file("."))
.enablePlugins(MdocPlugin)
.settings(crossScalaVersions := scala2Versions)
2 changes: 1 addition & 1 deletion docs/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Fetch

[![Join the chat at https://gitter.im/47deg/fetch](https://badges.gitter.im/47deg/fetch.svg)](https://gitter.im/47deg/fetch?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge) [![codecov.io](http://codecov.io/github/47deg/fetch/coverage.svg?branch=master)](http://codecov.io/github/47deg/fetch?branch=master) [![Maven Central](https://img.shields.io/badge/maven%20central-1.2.1-green.svg)](https://oss.sonatype.org/#nexus-search;gav~com.47deg~fetch*) [![License](https://img.shields.io/badge/license-Apache%202-blue.svg)](https://raw.githubusercontent.com/47deg/fetch/master/LICENSE) [![Latest version](https://img.shields.io/badge/fetch-1.2.1-green.svg)](https://index.scala-lang.org/47deg/fetch) [![Scala.js](http://scala-js.org/assets/badges/scalajs-0.6.15.svg)](http://scala-js.org) [![GitHub Issues](https://img.shields.io/github/issues/47deg/fetch.svg)](https://github.com/47deg/fetch/issues)
[![Join the chat at https://gitter.im/47deg/fetch](https://badges.gitter.im/47deg/fetch.svg)](https://gitter.im/47deg/fetch?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge) [![Maven Central](https://img.shields.io/badge/maven%20central-1.2.1-green.svg)](https://oss.sonatype.org/#nexus-search;gav~com.47deg~fetch*) [![License](https://img.shields.io/badge/license-Apache%202-blue.svg)](https://raw.githubusercontent.com/47deg/fetch/master/LICENSE) [![Latest version](https://img.shields.io/badge/fetch-1.2.1-green.svg)](https://index.scala-lang.org/47deg/fetch) [![Scala.js](http://scala-js.org/assets/badges/scalajs-0.6.15.svg)](http://scala-js.org) [![GitHub Issues](https://img.shields.io/github/issues/47deg/fetch.svg)](https://github.com/47deg/fetch/issues)

A library for Simple & Efficient data access in Scala and Scala.js

Expand Down
14 changes: 7 additions & 7 deletions fetch-debug/src/main/scala/debug.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@ object debug {
} yield lastRequestEnd - firstRequestStart
val durationDoc =
duration.fold(Document.empty: Document)((d: Long) =>
Document.text("Fetch execution") :: showDuration(d)
Document.text("Fetch execution") :-: showDuration(d)
)

durationDoc :/: Document.nest(
Expand All @@ -80,9 +80,9 @@ object debug {
} yield l - f

val round =
Document.text(s"[Round ${n}]") :: roundDuration.fold(Document.text(""))(showDuration(_))
Document.text(s"[Round ${n}]") :-: roundDuration.fold(Document.text(""))(showDuration(_))

round :: Document.nest(
round :-: Document.nest(
2,
pile(r.queries.map(showRequest))
)
Expand All @@ -91,9 +91,9 @@ object debug {
def showRequest(r: Request): Document =
r.request match {
case FetchOne(id, d) =>
Document.text(s"[Fetch one] From `${d.name}` with id ${id}") :: showDuration(r.duration)
Document.text(s"[Fetch one] From `${d.name}` with id ${id}") :-: showDuration(r.duration)
case Batch(ids, d) =>
Document.text(s"[Batch] From `${d.name}` with ids ${ids.toList}") :: showDuration(
Document.text(s"[Batch] From `${d.name}` with ids ${ids.toList}") :-: showDuration(
r.duration
)
}
Expand All @@ -110,14 +110,14 @@ object debug {
Document
.text(
s"[ERROR] Identity with id `${id}` for data source `${q.data.name}` not found"
) :: showRoundCount(
) :-: showRoundCount(
err
)
case UnhandledException(exc, log) =>
Document
.text(
s"[ERROR] Unhandled `${exc.getClass.getName}`: '${exc.getMessage}'"
) :: showRoundCount(
) :-: showRoundCount(
err
)
}
Expand Down
8 changes: 4 additions & 4 deletions fetch-debug/src/main/scala/document.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,10 +33,10 @@ case class DocCons(hd: Document, tl: Document) extends Document
* @version 1.0
*/
abstract class Document {
def ::(hd: Document): Document = DocCons(hd, this)
def ::(hd: String): Document = DocCons(DocText(hd), this)
def :/:(hd: Document): Document = hd :: DocBreak :: this
def :/:(hd: String): Document = hd :: DocBreak :: this
def :-:(hd: Document): Document = DocCons(hd, this)
def :-:(hd: String): Document = DocCons(DocText(hd), this)
def :/:(hd: Document): Document = hd :-: DocBreak :-: this
def :/:(hd: String): Document = hd :-: DocBreak :-: this

/**
* Format this document on `writer` and try to set line
Expand Down
13 changes: 7 additions & 6 deletions fetch-examples/src/test/scala/DoobieExample.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,8 @@ object DatabaseExample {

def createTransactor[F[_]: Async: ContextShift] =
for {
(conn, trans) <- (connectionPool[F](1), transactionPool[F]).tupled
connAndTrans <- (connectionPool[F](1), transactionPool[F]).tupled
(conn, trans) = connAndTrans
tx <-
H2Transactor
.newH2Transactor[F](
Expand Down Expand Up @@ -138,13 +139,13 @@ class DoobieExample extends AnyWordSpec with Matchers with BeforeAndAfterAll {
createTable(tx) *> authors.traverse(addAuthor(_)(tx))
})
.void
.unsafeRunSync
override def afterAll(): Unit = transactor.use(dropTable(_)).void.unsafeRunSync
.unsafeRunSync()
override def afterAll(): Unit = transactor.use(dropTable(_)).void.unsafeRunSync()

"We can fetch one author from the DB" in {
val io: IO[(Log, Author)] = Fetch.runLog[IO](Authors.fetchAuthor(1))

val (log, result) = io.unsafeRunSync
val (log, result) = io.unsafeRunSync()

result shouldEqual Author(1, "William Shakespeare")
log.rounds.size shouldEqual 1
Expand All @@ -156,7 +157,7 @@ class DoobieExample extends AnyWordSpec with Matchers with BeforeAndAfterAll {

val io: IO[(Log, List[Author])] = Fetch.runLog[IO](fetch)

val (log, result) = io.unsafeRunSync
val (log, result) = io.unsafeRunSync()

result shouldEqual Author(1, "William Shakespeare") :: Author(2, "Charles Dickens") :: Nil
log.rounds.size shouldEqual 1
Expand All @@ -171,7 +172,7 @@ class DoobieExample extends AnyWordSpec with Matchers with BeforeAndAfterAll {

val io: IO[(Log, List[Author])] = Fetch.runLog[IO](fetch)

val (log, result) = io.unsafeRunSync
val (log, result) = io.unsafeRunSync()

result shouldEqual Author(1, "William Shakespeare") :: Author(2, "Charles Dickens") :: Nil
log.rounds.size shouldEqual 2
Expand Down
16 changes: 8 additions & 8 deletions fetch-examples/src/test/scala/GraphQLExample.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,9 +32,9 @@ case class Project(name: Option[String], languages: List[String], collaborators:
case class Repo(name: String)

class GraphQLExample extends AnyWordSpec with Matchers {
implicit val executionContext = ExecutionContext.Implicits.global
implicit val t: Timer[IO] = IO.timer(executionContext)
implicit val cs: ContextShift[IO] = IO.contextShift(executionContext)
implicit val executionContext: ExecutionContext = ExecutionContext.Implicits.global
implicit val t: Timer[IO] = IO.timer(executionContext)
implicit val cs: ContextShift[IO] = IO.contextShift(executionContext)

def countFetches(r: Request): Int =
r.request match {
Expand Down Expand Up @@ -119,7 +119,7 @@ class GraphQLExample extends AnyWordSpec with Matchers {

"We can interpret queries" in {
val io = Fetch.runLog[IO](runQuery(query))
val (log, result) = io.unsafeRunSync
val (log, result) = io.unsafeRunSync()

result shouldEqual Organization(
"47deg",
Expand All @@ -135,7 +135,7 @@ class GraphQLExample extends AnyWordSpec with Matchers {

"We can interpret queries with only languages" in {
val io = Fetch.runLog[IO](runQuery(langsQuery))
val (log, result) = io.unsafeRunSync
val (log, result) = io.unsafeRunSync()

result shouldEqual Organization(
"47deg",
Expand All @@ -148,7 +148,7 @@ class GraphQLExample extends AnyWordSpec with Matchers {

"We can interpret queries with only collaborators" in {
val io = Fetch.runLog[IO](runQuery(collabsQuery))
val (log, result) = io.unsafeRunSync
val (log, result) = io.unsafeRunSync()
result shouldEqual Organization(
"47deg",
List(
Expand All @@ -163,7 +163,7 @@ class GraphQLExample extends AnyWordSpec with Matchers {

"We can interpret queries with no nested joins" in {
val io = Fetch.runLog[IO](runQuery(orgQuery))
val (log, result) = io.unsafeRunSync
val (log, result) = io.unsafeRunSync()
result shouldEqual Organization(
"47deg",
List(Project(Some("fetch"), List(), List()), Project(Some("arrow"), List(), List()))
Expand All @@ -175,7 +175,7 @@ class GraphQLExample extends AnyWordSpec with Matchers {

"We can interpret queries with a limited number of repositories" in {
val io = Fetch.runLog[IO](runQuery(repoQuery))
val (log, result) = io.unsafeRunSync
val (log, result) = io.unsafeRunSync()

result shouldEqual Organization(
"47deg",
Expand Down
40 changes: 26 additions & 14 deletions fetch/src/main/scala/fetch.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -156,7 +156,10 @@ object `package` {
y: BlockedRequest[F]
): BlockedRequest[F] =
(x.request, y.request) match {
case (a @ FetchOne(aId, ds), b @ FetchOne(anotherId, _)) =>
case (a: FetchOne[Any, Any], b: FetchOne[Any, Any]) =>

@cb372 cb372 Mar 22, 2021

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a bit unfortunate. I can't claim to fully understand all this code, but as far as I can tell, it was taking advantage of bugs in the Scala compiler's GADT support in order to compile?

I had a quick look but I have a feeling that any attempt to address this properly would become a rewrite of most of the library. We could at least try to suppress the erasure warnings?

@pepegar pepegar Mar 22, 2021

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Something we can try if the @ operator doesn't work is to rewrap the pattern again if needed?

      case (FetchOne(aId, ds), FetchOne(anotherId, ds2)) =>
        val a = FetchOne(aId, ds)
        val b = FetchOne(anotherId, ds2)

@benderpremier benderpremier Mar 27, 2021

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried that but I am getting the same compilation error:

[error] -- [E007] Type Mismatch Error: /Users/damien/code/fetch/fetch/src/main/scala/fetch.scala:167:48
[error] 167 |          val combined   = combineIdentities(a, b)
[error]     |                                                ^
[error]     |                      Found:    (b : fetch.package.FetchOne[I$2, A$2])
[error]     |                      Required: fetch.FetchQuery[I$1, A$1]

The compiler wants to prove that the type parameters of FetchOne a and b are the same but it can't. We lose those parameters when we use FetchRequest.

@benderpremier benderpremier Mar 27, 2021

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll try to see if we can keep track of the I and A type params along the way. I gave it a quick go now and it doesn't seem so straightforward.

@benderpremier benderpremier Mar 27, 2021

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I spent some time this afternoon trying to keep track of the I and avoid the FetchOne[Any, Any]. However, I end up bubbling that type parameter all the way up in the API (in the pure and apply method of the Fetch object) which I don't think we want to do.

I am not sure there is a way around using Any here. If you look at how the Fetch are created, we widen to Any in the fetch constructor methods (apply, optional) and then cast with asInstanceOf[A]. Looks like this is where we loose the I we would need here.

Also the monad instance for fetch, implicit def fetchM[F[_]: Monad]: Monad[Fetch[F, *]] would have to be reworked if we push the I all the way up which seems complicated.

val aId = a.id
val ds = a.data
val anotherId = b.id
if (aId == anotherId) {
val newRequest = FetchOne(aId, ds)
val newResult = x.result.flatMap(() => y.result)
Expand All @@ -180,7 +183,9 @@ object `package` {
BlockedRequest(newRequest, newResult)
}

case (a @ FetchOne(oneId, ds), b @ Batch(anotherIds, _)) =>
case (a: FetchOne[Any, Any], b: Batch[Any, Any]) =>
val oneId = a.id
val ds = a.data
val combined = combineIdentities(a, b)
val newRequest = Batch(combined, ds)
val newResult = CombinationSuspend((r: FetchStatus) =>
Expand All @@ -198,7 +203,9 @@ object `package` {

BlockedRequest(newRequest, newResult)

case (a @ Batch(manyId, ds), b @ FetchOne(oneId, _)) =>
case (a: Batch[Any, Any], b: FetchOne[Any, Any]) =>
val ds = a.data
val oneId = b.id
val combined = combineIdentities(a, b)
val newRequest = Batch(combined, ds)
val newResult = CombinationSuspend((r: FetchStatus) =>
Expand All @@ -215,7 +222,8 @@ object `package` {
)
BlockedRequest(newRequest, newResult)

case (a @ Batch(manyId, ds), b @ Batch(otherId, _)) =>
case (a: Batch[Any, Any], b: Batch[Any, Any]) =>
val ds = a.data
val combined = combineIdentities(a, b)
val newRequest = Batch(combined, ds)
val newResult = x.result.flatMap(() => y.result)
Expand Down Expand Up @@ -473,9 +481,10 @@ object `package` {
T: Timer[F]
): F[(Log, A)] =
for {
(log, cache) <- (ref[F, Log](FetchLog()), ref[F, DataCache[F]](cache)).tupled
result <- performRun(fa, cache, Some(log))
e <- log.get
logAndCache <- (ref[F, Log](FetchLog()), ref[F, DataCache[F]](cache)).tupled
(log, cache) = logAndCache
result <- performRun(fa, cache, Some(log))
e <- log.get
} yield (e, result)
}

Expand Down Expand Up @@ -529,9 +538,11 @@ object `package` {
T: Timer[F]
): F[(Log, DataCache[F], A)] =
for {
(log, cache) <- (ref[F, Log](FetchLog()), ref[F, DataCache[F]](cache)).tupled
result <- performRun(fa, cache, Some(log))
(e, c) <- (log.get, cache.get).tupled
logAndCache <- (ref[F, Log](FetchLog()), ref[F, DataCache[F]](cache)).tupled
(log, cache) = logAndCache
result <- performRun(fa, cache, Some(log))
eAndC <- (log.get, cache.get).tupled
(e, c) = eAndC
} yield (e, c, result)
}

Expand Down Expand Up @@ -591,8 +602,9 @@ object `package` {
if (performedRequests.isEmpty) Applicative[F].unit
else
log match {
case Some(l) => l.modify((oldE) => (oldE.append(Round(performedRequests)), oldE))
case None => Applicative[F].unit
case Some(l) =>
l.modify((oldE) => (oldE.append(Round(performedRequests)), oldE)).void
case None => Applicative[F].unit
}
} yield ()
}
Expand All @@ -607,8 +619,8 @@ object `package` {
T: Timer[F]
): F[List[Request]] =
blocked.request match {
case q @ FetchOne(_, _) => runFetchOne[F](q, ds, blocked.result, cache, log)
case q @ Batch(_, _) => runBatch[F](q, ds, blocked.result, cache, log)
case q: FetchOne[Any, Any] => runFetchOne[F](q, ds, blocked.result, cache, log)
case q: Batch[Any, Any] => runBatch[F](q, ds, blocked.result, cache, log)
}
}

Expand Down
8 changes: 4 additions & 4 deletions fetch/src/test/scala/FetchAsyncQueryTests.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ class FetchAsyncQueryTests extends FetchSpec {

val io = Fetch.run[IO](fetch)

io.map(_ shouldEqual Article(1, "An article with id 1")).unsafeToFuture
io.map(_ shouldEqual Article(1, "An article with id 1")).unsafeToFuture()
}

"We can combine several async data sources and interpret a fetch into an IO" in {
Expand All @@ -41,7 +41,7 @@ class FetchAsyncQueryTests extends FetchSpec {

val io = Fetch.run[IO](fetch)

io.map(_ shouldEqual (Article(1, "An article with id 1"), Author(2, "@egg2"))).unsafeToFuture
io.map(_ shouldEqual (Article(1, "An article with id 1"), Author(2, "@egg2"))).unsafeToFuture()
}

"We can use combinators in a for comprehension and interpret a fetch from async sources into an IO" in {
Expand All @@ -58,7 +58,7 @@ class FetchAsyncQueryTests extends FetchSpec {
Article(1, "An article with id 1"),
Article(2, "An article with id 2")
)
).unsafeToFuture
).unsafeToFuture()
}

"We can use combinators and multiple sources in a for comprehension and interpret a fetch from async sources into an IO" in {
Expand All @@ -83,7 +83,7 @@ class FetchAsyncQueryTests extends FetchSpec {
Author(3, "@egg3")
)
)
).unsafeToFuture
).unsafeToFuture()
}
}

Expand Down
Loading