The Environment Functions pattern utilizes algebraic effects to provide easy dependency inversion at a more granular level with less boilerpalte than other patterns. This pattern should be easy to replicate in other AE systems. It is technically possible to use this pattern in some other contexts, say an MTL system or a “super monad” one, but the ergonomics are worse. In some systems it’s simply not feasible, usually if there’s no way to define a monad.

The core idea of the pattern is to define function types that are effect polymorphic, allowing implementations to utilize effects that aren’t leaked into abstractions. The abstractions will literally just be function types that we will alias for easier referencing, and the implementations will be actual functions with specific effects.

A more technically correct name was considered, “polymorphic function dependencies”, because the important point of Environment Functions isn’t that they’re required in the environment (though they usually should be), but rather that they’re effect polymorphic function dependencies. However, “Environment Functions” is more concise/memorable, and was the original name I came up with.

It’s often the case that dependency inversion patterns do a good job of hiding some implementation details, like what dependencies are required to instantiate some specific implementation or what backend/data retrieval process is being exactly utilized. However, they are bad at hiding other implementation details, namely most effects. If your repository’s implementation has functions that require async or has explicit error possibilities, those details will leak up into the abstraction, otherwise the implementation’s function signature wouldn’t align with the abstraction.

While it is true that sometimes our abstractions might have error possibilities, it’s also sometimes true that implementations have error possibilities that are irrelevant for the domain, but that still end up being represented in the domain (at least abstractly). Other effects fare even less well, async/future/suspension generally just get leaked directly into the abstractions.

This article will describe 7 different versions of a fetchAndSaveGame function, where we fetch a game from an external API and save it to our local database. The “full” examples will be single code snippets, but within a real codebase they’d usually be split up across different files/folders.

Table of Contents Link to heading

The format of the first 7 sections is [effect system] : [pattern]. “basic” means essentially no effect system. In the clean architecture there’s minimal adoption of some effects (namely Either), but no system for combining/orchestrating effects, they just have to be manually wrapped/unwrapped/mapped.

basic : single function Link to heading

def fetchAndSaveGame(gameId: Long) =
  val response = requests.get(s"https://somegame.com/games/$gameId")
  if response.statusCode != 200 then return false

  try 
    val game = upickle.default.read[Game](response.body)
    run(query[Game].insertValue(lift(game)))
    true
  catch
    case _: Exception => false

This version is straightforward, but the dependencies are very tightly coupled. The ctx the database uses has to exist in the enclosing scope for this function to even compile, and the function signature gives no other information about what dependencies/effects are being utilized. This function just fires off a bunch of untracked side effects.

basic : clean architecture Link to heading

This version is much longer than the first, so after showing the whole snippet I’ll go section by section explaining it.

case class GameGatewayFailure(error: String | Throwable)
case class GameRepositoryFailure(error: String | Throwable)

trait GameGateway:
  def fetch(gameId: Long): Either[GameGatewayFailure, Game]
    
trait GameRepository:
  def save(game: Game): Either[GameRepositoryFailure, Unit]

class HttpGameGateway(api: requests.type) extends GameGateway:
  override def fetch(gameId: Long) =
    val response = api.get(s"https://somegame.com/games/$gameId")

    if response.statusCode != 200 then
      Left(GameGatewayFailure("Invalid status code " + response.statusCode))
    else try
      Right(pickle.default.read[Game](response.body))
    catch
      case e: Exception => Left(GameGatewayFailure(
        s"Fails with body: ${response.body} & message: ${e.getMessage}"))

class PostgresGameRepository(ctx: PostgresJdbcContext[SnakeCase])
    extends GameRepository:
  import ctx.*
  override def save(game: Game) =
    run(query[Game].insertValue(lift(game)))
      .mapError(e => GameRepositoryFailure(e.getMessage))

def fetchAndSaveGame(gameId: Long, gateway: GameGateway, repo: GameRepository) =
  for
    game <- gateway.fetch(gameId)
  _    <- repo.save(game)
  yield game

As was stated in the beginning of the article, we’ll need domain error types defined for our implementations to transform their errors into. This isn’t ideal, as we’re leaking the fact that errors exist at all into our domain layer when they’re not relevant for the function we’re creating

case class GameGatewayFailure(error: String | Throwable)
case class GameRepositoryFailure(error: String | Throwable)

We’ll then create some traits, GameGateway and GameRepository. These will respectively define abstract functions fetch and save for retrieving games from a gateway, and saving them into a repository

trait GameGateway:
  def fetch(gameId: Long): Either[GameGatewayFailure, Game]
    
trait GameRepository:
  def save(game: Game): Either[GameRepositoryFailure, Unit]

We’ll have an implementation for each, one named HttpGameGateway and the other PostgresGameRepository. They both have to map their respective error (Left) channels into types the domain can understand

class HttpGameGateway(api: requests.type) extends GameGateway:
  override def fetch(gameId: Long) =
    val response = api.get(s"https://somegame.com/games/$gameId")

    if response.statusCode != 200 then
      Left(GameGatewayFailure("Invalid status code " + response.statusCode))
    else try
      Right(pickle.default.read[Game](response.body))
    catch
      case e: Exception => Left(GameGatewayFailure(
        s"Fails with body: ${response.body} & message: ${e.getMessage}"))
            
class PostgresGameRepository(ctx: PostgresJdbcContext[SnakeCase])
    extends GameRepository:
  import ctx.*
  override def save(game: Game) =
    run(query[Game].insertValue(lift(game)))
      .mapError(e => GameRepositoryFailure(e.getMessage))

Our new fetchAndSaveGame function will still take a gameId, and it’ll also take a gateway and repo

def fetchAndSaveGame(gameId: Long, gateway: GameGateway, repo: GameRepository) =
  for
    game <- gateway.fetch(gameId)
    _    <- repo.save(game)
  yield game

Compared to the original, the dependencies are managed explicitly and the errors are specified and aggregated correctly. The two major downsides here are how verbose it is, and how leaky the effects are from the implementations. To further illustrate this, let’s say we want asynchronicity.

mtl-cats : clean architecture Link to heading

Using mtl-style cats, we’ll introduce asynchronicity with the IO monad. If you’re unfamiliar with monad transformers, you can read about them in the monad transformers section in my Effect Tracking article.

case class GameGatewayFailure(error: String | Throwable)
case class GameRepositoryFailure(error: String | Throwable)

trait GameGateway:
  def fetch(gameId: Long): EitherT[IO, GameGatewayFailure, Game]

trait GameRepository:
  def save(game: Game): EitherT[IO, GameRepositoryFailure, Long]
    
class HttpGameGateway(client: Client[IO]) extends GameGateway:
  override def fetch(gameId: Long) =
    EitherT(
      client.expect[Game](s"https://somegame.com/games/$gameId").attempt
    ).leftMap(err => GameGatewayFailure(err.getMessage))

class PostgresGameRepository(
  ctx: DoobieContext.Postgres[SnakeCase], 
  transactor: Transactor[IO]
) extends GameRepository:
  import ctx.*
  override def save(game: Game) =
    EitherT(
      run(query[Game].insertValue(lift(game))).attempt.transact[IO](transactor)
    ).leftMap(err => GameRepositoryFailure(err.getMessage))
          

type GameFailure = GameGatewayFailure | GameRepositoryFailure
def fetchAndSaveGame(gameId: Long, gateway: GameGateway, repo: GameRepository) =
  for
    game <- gateway.fetch(gameId).leftMap[GameFailure](identity)
    _    <- repo.save(game).leftMap[GameFailure](identity)
  yield game

This is another improvement, certain errors before that were silent and waiting for runtime to fail are now explicitly tracked, and we have async support in the implementations. The issue was we had to propagate this information upwards to our domain; notice the fetch and save data types are different now. It’s fine for the domain to know about the existence of the langauge or framework that is universal to the entire app (so knowing that cats exists isn’t a problem), the issue is that specific changes in the implementation causes specific required changes in the domain.

mtl-cats : environment functions Link to heading

Despite not being AE systems, Cats and ZIO are both powerful enough to use the Environment Functions pattern, it’s just quite verbose. Now that our errors don’t have to be propagated up, we don’t need data types for the specific errors (No more GameGatewayFailure or GameRepositoryFailure). We would only want those if our main program fetchAndSaveGame was operating on them somehow. Previously we had to explicitly widen the error channels, but that wasn’t real domain logic it was wiring code.

type FetchGame[F[_]] = Long => F[Game]
type SaveGame[F[_]] = Game => F[Long]

type HttpErrors = ResponseException[String, Error] | Throwable
type HttpT = ReaderT[EitherT[IO, HttpErrors, *], SttpBackend[IO, Any], *]

def httpFetchGame: FetchGame[HttpT] = gameId =>
  for
    client <- ReaderT.ask
    request = basicRequest.get(uri"https://somegame.com/games/$gameId")
      .response(asJsonEither[String, Game])
    response <- ReaderT.liftF(EitherT(request.send(client).attempt))
    game <- ReaderT.liftF(EitherT.fromEither[IO](response.body))
  yield game

type PostgresContext = (DoobieContext.Postgres[SnakeCase], Transactor[IO])
type PostgresT = ReaderT[EitherT[IO, SQLException, *], PostgresContext, *]

def postgresSaveGame: SaveGame[PostgresT] = game =>
  for
    (ctx, transactor) <- ReaderT.ask
    result <-
      import ctx.*
      ReaderT.liftF(EitherT(run(
        query[Game].insertValue(lift(game))).attempt.transact[IO](transactor)))
  yield result

def fetchAndSaveGame[F[_]: Monad](gameId: Long) =
  for
    (fetch, save) <- ReaderT.ask[F, (FetchGame[F], SaveGame[F])]
    game <- ReaderT.liftF(fetch(gameId))
    _ <- ReaderT.liftF(save(game))
  yield game

This isn’t too bad to use abstractly, fetchAndSaveGame is simple enough. If we want to change the stack at all for a specific implementation, it doesn’t change our core code at all. The issue here comes when we want to run the code. First, we need to provide a specific monad for our function to operate on. There’s no easy way to compose them, so we need to conceptually combine them.

What we have being returned from fetchAndSaveGame is ReaderT[F, (FetchGame[F], SaveGame[F]), Game]. For ReaderT, you can read the type parameters as ReaderT[Effects, Environment, Result]. So F are the effects we’re operating with, and the reader operates with a FetchGame[F] and SaveGame[F] being provided. FetchGame and SaveGame both return something that operates within F as well. So the meaning of all of this is we need to provide a type F that both our FetchGame and SaveGame implementation can map to.

The httpFetchGame and postgresSaveGame functions operate in similar monad stacks. They have the form ReaderT[EitherT[IO, SomeErrors, _], SomeEnvironment, _], so essentially all we need to do is combine the errors and combine the environments, then we have a single monad that both of our functions can map to:

type AppErrors = HttpErrors | DbFailure
type AppEitherT = EitherT[IO, AppErrors, _]
type AppContext = 
  (SttpBackend[IO, Any], DoobieContext.Postgres[SnakeCase], Transactor[IO])
type AppT = ReaderT[AppEitherT, AppContext, _]

val fetchAndSaveGameWithId1InHttpAndPostgresReaderT = fetchAndSaveGame[AppT](1)

The inferred type of fetchAndSaveGameWithId1InHttpAndPostgresReaderT is kind of insane:

Kleisli[Kleisli[AppEitherT, AppContext, _], (Long => Kleisli[EitherT[IO, AppErrors, _], (SttpBackend[IO, Any], DoobieContext.Postgres[SnakeCase], Transactor[IO]), Game], Game => Kleisli[EitherT[IO, AppErrors, _], (SttpBackend[IO, Any], DoobieContext.Postgres[SnakeCase], Transactor[IO]), Long]), Game]

substituting the aliases in makes it a lot more clear:

ReaderT[AppT, (FetchGame[AppT], SaveGame[AppT]), Game]

actually providing the required environment (FetchGame[AppT], SaveGame[AppT]) isn’t trivial though, because we don’t have a FetchGame[AppT], we have a FetchGame[HttpMonad]. This needs to be converted, as does the SaveGame one:

val fetchAndSaveGameWithId1WithHttpAndPostgres = 
  fetchAndSaveGameWithId1InHttpAndPostgresReaderT
    .run((
      gameId => ReaderT.ask[AppEitherT, AppContext].flatMap: ctx =>
        ReaderT.liftF(httpFetchGame(gameId).run(ctx._1)
          .leftMap[AppErrors](identity)), 
      game => ReaderT.ask[AppEitherT, AppContext].flatMap: ctx => 
        ReaderT.liftF(postgresSaveGame(game).run((ctx._2, ctx._3))
          .leftMap[AppErrors](identity))
  ))

run is how you provide an environment, and our environment requires two functions (fetch and save game). For each function we ask for the current context within our set of effects, and then we unwrap/rewrap the reader/either stack by only providing the subcontext that’s required (when we’re doing ctx._1 etc.), and then widening the error list to include all the valid errors (AppErrors).

We’ve now “unwrapped” the outer ReaderT by providing the necessary environment (fetch/save game), and as a result the new type above is:

AppT[Game]

It’s great if you can follow along with all the type trickery above, but this type of machinery is not the kind of thing you’d want to see in your app. You can create nice helper names that make unwrapping/rewrapping/nested wrapping all nicer, but ultimately you’d need to be pretty comfortable with the mtl style and you’d still have a lot of wiring code/helpers that shouldn’t ultimately be necessary.

kyo : environment functions Link to heading

Kyo is an algebraic effect system. It’s not as “free” as some other ones, but it’s far more performant as a result (we’re not building some giant AST that we later interpret). Environment Functions give us back some of that lost power when comparing to more free AE systems, and is a perfect replacement for something like Clean Architecture if coming from OO:

type FetchGame[S] = Long => Game < S
type SaveGame[S] = Game => Long < S

def httpFetchGame(gameId: Long) =
  Requests(_.response(asJson[Game]).get(uri"https://somegame.com/games/$gameId"))

def postgresSaveGame(game: Game) =
  Env.use[Postgres[SnakeCase]]: ctx =>
    import ctx.*
    ZIOs.get(run(query.insertValue(lift(game))))

def fetchAndSaveGame[S](gameId: Long) =
  for
    game <- Env.use[FetchGame[S]](_(gameId))
    _ <- Env.use[SaveGame[S]](_(game))
  yield game

If you’re interested in a specific deep dive, feel free to check them out either now or after the article:

The “core” is still very small, we just have generic FetchGame and SaveGame aliases, as well as fetchAndSaveGame which also has a type parameter. S in this latter context will be the total set of effects that both FetchGame and SaveGame (that aren’t already defined in those type aliases, in this example it’s none for both).

Env is akin to a Reader effect (it provides an environment), but in the context of algebraic effects it feels as ergonomic as most dependency injection frameworks, without requiring runtime reflection or compile time code generation. For example, when we later specialize fetchAndSaveGame to httpFetchGame & postgresSaveGame, that expression will have the environment requirements of postgresSaveGame, without them needing to be explicitly referenced at all. You’ll see it in the type (as all other effects will appear), but you only need to worry about it when you want to provide it, which could be at the very top of the application where you provide most of your dependencies.

What you end up with is a core layer that knows nothing about the dependencies or other effects that it’s using, and doesn’t have to explicitly pass, handle, or explicitly mark any of those effects. No leaky async or checked exceptions.

Note: if you’re running into issues with Abort.run or mapAbort complaining about ResponseException not having a ConcreteTag, you can just add .mapLeft(_.getMessage) after asJson[Game]. The error says they might alleviate this restriction in the future, so by the time you’re reading/trying this, it might not be an issue.

kyo : clean architecture Link to heading

case class GameGatewayFailure(error: String | Throwable)
case class GameRepositoryFailure(error: String | Throwable)

trait GameGateway:
  def fetch(gameId: Long): Game < (Async & Abort[GameGatewayFailure]) 

trait GameRepository:
  def save(game: Game): Long < (Async & Abort[GameRepositoryFailure]) 
    
class HttpGameGateway(client: Requests.Backend) extends GameGateway:
  override def fetch(gameId: Long) =
    Requests(_.response(asJson[Game].mapLeft(_.getMessage))
        .get(uri"https://somegame.com/games/$gameId"))
      .mapAbort(err => GameGatewayFailure(err))
      .handle(Requests.let(client))

class PostgresGameRepository(ctx: Postgres[SnakeCase]) extends GameRepository:
  import ctx.*
  override def save(game: Game) =
    ZIOs.get(run(query[Game].insertValue(lift(game))))
      .mapAbort(err => GameRepositoryFailure(err))

def fetchAndSaveGame(gameId: Long, gateway: GameGateway, repo: GameRepository) =
  for
    game <- gateway.fetch(gameId)
    _ <- repo.save(game)
  yield game

This is very functionally similar to mtl-cats : clean architecture. Moving away from monad transformers to algebraic effects means less boilerplate, and our effects compose much more nicely. Specifically if you look at fetchAndSaveGame, we no longer need to leftMap[GameFailure](identity), which literally just said map the left channel to itself but with a wider type.

This is more verbose than kyo : environment functions. In this clean architecture version, we need to create traits and classes that implement those traits, all so the function type has to leak. If we wanted to require Async for example in environment functions, the type aliases are perfectly capable of requiring it. It’s just there’s no requirement in the usage of those abstractions for it to be async. If we wanted to fetch multiple things at the same time without blocking, maybe that’d be a good time to add that requirement. We also need to map our error channels to these domain error types just to get our functions to type check.

kyo : single function Link to heading

Algebraic effect systems have so much built in effect tracking that there’s much less need for these patterns.

def fetchAndSaveGame(gameId: Long) =
  for
    game <- Requests(_.response(asJson[Game])
      .get(uri"https://somegame.com/games/$gameId"))
    _ <- Env.use[Postgres[SnakeCase]]: ctx =>
      import ctx.*
      ZIOs.get(run(query.insertValue(lift(game))))
  yield game

When compared to the original, tightly coupled single function at the start of this article, the kyo is actually shorter! The original for comparison:

def fetchAndSaveGame(gameId: Long) =
  import ctx.*
  val response = requests.get(s"https://somegame.com/games/$gameId")
    
  if response.statusCode != 200 then return false
    
  try 
    val game = upickle.default.read[Game](response.body)
    run(query[Game].insertValue(lift(game)))
    true
  catch
    case _: Exception => false

The return type, Boolean, exposes extremely minimal information about that function. There’s no straightforward way to know that the function interacts with http and sql. While kyo specifies the effects in use via the return type, and said type is still completely inferred:

Game < (Async & Env[Postgres[SnakeCase]] &
  Abort[FailedRequest | ResponseException[String, Error] | SQLException])

You can change the http/sql implementation by either using Requests.let for the former, or Env.run for the latter. You could have them completely mocked out, or you could even use quill’s more general sql types if you know you don’t need postgres specifically, allowing the caller to decide if they want to use postgres/mysql/etc.

For posterity, it would be nice to have a similar Databases abstraction, just like the Requests abstraction in kyo. I’ve started looking into building this, and if I actually end up developing it, this is what I would want the end code to look like:

def fetchAndSaveGame(gameId: Long) =
  Requests(_.response(asJson[Game]).get(uri"https://somegame.com/games/$gameId"))
    .flatTap(game => Databases(_.insert(game)))

When to actually use this pattern Link to heading

There will be cases where you do truly want or need very abstract orchestration, but you probably don’t need to reach for it as a global default in these fully typed effect systems. If you use the Environment Functions pattern with kyo, and you end up testing a fully general fetchAndSaveGame, really all you end up testing is that for comprehensions work correctly, and if they don’t then the language has a giant bug. That sort of testing is usually out of scope for these high level apps. If you instead mocked out Requests and Postgres, then you could test something like the correct number of http calls were made and the correct number of db inserts were made, and what happens when deserialization fails (e.g make sure no insertion happens, etc.).

You can do all that testing without making any real http or db calls, with the single function kyo variation.

So for a simple example like fetchAndSaveGame, I wouldn’t actually use this pattern. I’d only suggest it if you have complex abstract orchestration, and by hiding the implementation details you either make the code more clear, and/or you gain the ability to test that orchestration does what you expect.

Statistics Link to heading

Effect SystemPatternCharacter CountLine LengthAsync CapabilityTracked Dependencies & ErrorsNo Leaky Implementations
basicsingle function30410✗✗✓1
basicclean architecture114633✗✓2✗
mtl-catsclean architecture113532✓✓2✗
mtl-catsenvironment functions114533✓✓✓
kyoenvironment functions45916✓✓✓
kyoclean architecture98027✓✓2✗
kyosingle function2638✓✓✓1
kyo (with made up Databases abstraction)single function1643✓✓✓1

  1. No implementations to leak, just a single function ↩︎ ↩︎ ↩︎

  2. Dependencies are tracked but not automatically wired. They’re just arguments to a function or constructor. Errors are mapped to core error types, so important context is often lost or leaked. ↩︎ ↩︎ ↩︎