Fetch

Running data fetches optimized by the Fetch library in Freestyle programs can be achieved through the algebra and interpreter that are provided by the freestyle-fetch package. freestyle-fetch allows you to run Fetch computations when interpreting a free program, using the target runtime monad as their runtime type.

Familiarity with fetch is assumed, take a look at the Fetch documentation if you haven’t before. We’ll use a trivial data source for the examples:

import _root_.fetch._
// import _root_.fetch._

import _root_.fetch.implicits._
// import _root_.fetch.implicits._

import cats.data.NonEmptyList
// import cats.data.NonEmptyList

object OneSource extends DataSource[Int, Int]{
 def name = "One"
 def fetchOne(id: Int): Query[Option[Int]] = Query.sync({
    println(s"Fetching ${id}")
    Option(1)
 })
 def fetchMany(ids: NonEmptyList[Int]): Query[Map[Int, Int]] = batchingNotSupported(ids)
}
// defined object OneSource

def fetchOne(x: Int): Fetch[Int] = Fetch(x)(OneSource)
// fetchOne: (x: Int)fetch.Fetch[Int]

Let’s start by creating a simple algebra for our application for printing messages on the screen:

import freestyle._
// import freestyle._

import freestyle.implicits._
// import freestyle.implicits._

@free trait Interact {
  def tell(msg: String): FS[Unit]
}
// defined trait Interact
// defined object Interact

Then, make sure to include the Fetch algebra FetchM in your application:

import freestyle.fetch._
// import freestyle.fetch._

import freestyle.fetch.implicits._
// import freestyle.fetch.implicits._

@module trait App {
  val interact: Interact
  val fetches: FetchM
}
// defined trait App
// defined object App

Now that we’ve got our Interact algebra and FetchM in our app, we’re ready to write the first program:

def program[F[_]](
  implicit app: App[F]
): FreeS[F, Int] =  for {
    _ <- app.interact.tell("Hello")
    x <- app.fetches.runA(fetchOne(1))
    _ <- app.interact.tell(s"Result: ${x}")
  } yield x
// program: [F[_]](implicit app: App[F])freestyle.FreeS[F,Int]

To run this, we need to create an implicit interpreter for our Interact algebra:

import cats.Monad
// import cats.Monad

implicit def interactInterp[F[_]](
  implicit ME: Monad[F]
): Interact.Handler[F] = new Interact.Handler[F] {
  def tell(msg: String): F[Unit] = {
    println(msg)
    ME.pure(())
  }
}
// interactInterp: [F[_]](implicit ME: cats.Monad[F])Interact.Handler[F]

Now we can run the program to a Future. Check how the result from the fetch is printed to the console:

import scala.concurrent._
// import scala.concurrent._

import scala.concurrent.duration._
// import scala.concurrent.duration._

import scala.concurrent.ExecutionContext.Implicits.global
// import scala.concurrent.ExecutionContext.Implicits.global

Await.result(program[App.Op].interpret[Future], Duration.Inf)
// Hello
// res0: Int = 1

Running fetches

A handful of operations for running fetches are exposed in the FetchM algebra.

runA

We’ve already seen FetchM#runA, which runs a fetch that returns the final result:

def program[F[_]](
  implicit app: App[F]
): FreeS[F, Int] =  for {
    _ <- app.interact.tell("Hello")
    x <- app.fetches.runA(fetchOne(1))
    _ <- app.interact.tell(s"Result: ${x}")
  } yield x
// program: [F[_]](implicit app: App[F])freestyle.FreeS[F,Int]

Await.result(program[App.Op].interpret[Future], Duration.Inf)
// Hello
// res1: Int = 1

There is a variant of runA where a cache can be specified: FetchM#runAWithCache. In the following example, we simply pass an empty in-memory cache, but you can pass your own cache or a resulting cache from a previous fetch:

def program[F[_]](
  implicit app: App[F]
): FreeS[F, Int] =  for {
    _ <- app.interact.tell("Hello")
    x <- app.fetches.runAWithCache(fetchOne(1), InMemoryCache.empty)
    _ <- app.interact.tell(s"Result: ${x}")
  } yield x
// program: [F[_]](implicit app: App[F])freestyle.FreeS[F,Int]

Await.result(program[App.Op].interpret[Future], Duration.Inf)
// Hello
// res2: Int = 1

runE

Fetch tracks its internal state with an environment of type FetchEnv. The FetchM#runE allows us to run a fetch and get its final environment out, ignoring its result:

def program[F[_]](
  implicit app: App[F]
): FreeS[F, FetchEnv] =  for {
    _ <- app.interact.tell("Hello")
    env <- app.fetches.runE(fetchOne(1))
    _ <- app.interact.tell(s"Result: ${env}")
  } yield env
// program: [F[_]](implicit app: App[F])freestyle.FreeS[F,fetch.FetchEnv]

Await.result(program[App.Op].interpret[Future], Duration.Inf)
// Hello
// res3: fetch.FetchEnv = FetchEnv(InMemoryCache(Map((One,1) -> 1)),Queue(Round(InMemoryCache(Map()),FetchOne(1,One),1,832076479917643,832076480056242)))

There is a variant of runE where a cache can be specified: FetchM#runEWithCache. In the following example, we simply pass an empty in-memory cache, but you can pass your own cache or a resulting cache from a previous fetch.

def program[F[_]](
  implicit app: App[F]
): FreeS[F, FetchEnv] =  for {
    _ <- app.interact.tell("Hello")
    env <- app.fetches.runEWithCache(fetchOne(1), InMemoryCache.empty)
    _ <- app.interact.tell(s"Result: ${env}")
  } yield env
// program: [F[_]](implicit app: App[F])freestyle.FreeS[F,fetch.FetchEnv]

Await.result(program[App.Op].interpret[Future], Duration.Inf)
// Hello
// res4: fetch.FetchEnv = FetchEnv(InMemoryCache(Map((One,1) -> 1)),Queue(Round(InMemoryCache(Map()),FetchOne(1,One),1,832077652678410,832077652812444)))

runF

If we’re interested in both the final environment and the fetch result, we can use FetchM#runF to get a (FetchEnv, A) pair.

def program[F[_]](
  implicit app: App[F]
): FreeS[F, FetchEnv] =  for {
    _ <- app.interact.tell("Hello")
    r <- app.fetches.runF(fetchOne(1))
    (env, x) = r
    _ <- app.interact.tell(s"Result: ${env}")
  } yield env
// program: [F[_]](implicit app: App[F])freestyle.FreeS[F,fetch.FetchEnv]

Await.result(program[App.Op].interpret[Future], Duration.Inf)
// Hello
// res5: fetch.FetchEnv = FetchEnv(InMemoryCache(Map((One,1) -> 1)),Queue(Round(InMemoryCache(Map()),FetchOne(1,One),1,832078860383605,832078860691652)))

There is a variant of runF where a cache can be specified: FetchM#runFWithCache. In the following example, we simply pass an empty in-memory cache, but you can pass your own cache or a resulting cache from a previous fetch:

def program[F[_]](
  implicit app: App[F]
): FreeS[F, FetchEnv] =  for {
    _ <- app.interact.tell("Hello")
	r <- app.fetches.runFWithCache(fetchOne(1), InMemoryCache.empty)
	(env, x) = r
	_ <- app.interact.tell(s"Result: ${env}")
  } yield env
// program: [F[_]](implicit app: App[F])freestyle.FreeS[F,fetch.FetchEnv]

Await.result(program[App.Op].interpret[Future], Duration.Inf)
// Hello
// res6: fetch.FetchEnv = FetchEnv(InMemoryCache(Map((One,1) -> 1)),Queue(Round(InMemoryCache(Map()),FetchOne(1,One),1,832080326460954,832080326611959)))

Tips

Reuse cache between fetches

One of the things to consider when interleaving fetches in free programs is that the default in-memory cache won’t be shared between fetch executions on different steps of the program. Notice that the message from OneSource that says "Fetching 1" is printed twice:

def program[F[_]](
  implicit app: App[F]
): FreeS[F, Int] =  for {
    _ <- app.interact.tell("Hello")
	x <- app.fetches.runA(fetchOne(1))
	_ <- app.interact.tell(s"Result: ${x}")
	y <- app.fetches.runA(fetchOne(1))
  } yield x + y
// program: [F[_]](implicit app: App[F])freestyle.FreeS[F,Int]

Await.result(program[App.Op].interpret[Future], Duration.Inf)
// Hello
// res7: Int = 2

However, we can run a fetch getting both the environment and the result with runF, and pass the resulting cache to subsequent fetch runs. Note how the "Fetching 1" message is only printed once:

def program[F[_]](
  implicit app: App[F]
): FreeS[F, Int] =  for {
    _ <- app.interact.tell("Hello")
    r <- app.fetches.runF(fetchOne(1))
	(env, x) = r
	_ <- app.interact.tell(s"Result: ${x}")
	y <- app.fetches.runAWithCache(fetchOne(1), env.cache)
  } yield x + y
// program: [F[_]](implicit app: App[F])freestyle.FreeS[F,Int]

Await.result(program[App.Op].interpret[Future], Duration.Inf)
// Hello
// res8: Int = 2