Reactive Streams in Scala meet a Game Engine - Part 1

Mikołaj Koziarkiewicz

Series' Table of Contents

Intro

It’s always interesting to see a new (or newly fashionable) technique being applied to new and unexpected areas, and see whether it makes sense (and what makes or breaks it).

Such is the case with Scala (reactive) streaming and computer game creation. A while ago, I stumbled upon a talk by Aleksandar Prokopec, which planted the seed of inspiration for this blog series. And what set that seed on a path of growth was another presentation, this time by Michał Płachta [1].

So, why another thing on this?

Well, of course it wouldn’t be fun if the blog series was just a same-scenario rehash. Here’s what’s going to be different - while the aforementioned talks were more-or-less "from-scratch" affairs, creating a game engine and/or a streaming platform along the way - I will be going the "lazy" route, and attempting to join together a ready-made game engine with a ready-made streaming framework.

The primary advantage of this kind-of-unique[2] setup is that the focus almost by definition falls on the paradigm mismatch problem, and thus it’s easy to see what one must solve to make the whole shebang work.

And there’s a related blessing in disguise - because we’re operating between two rigid boundaries of stable libraries, any domain-related idiosyncrasies will be easily identifiable, and have a reduced chance of rising up later on to bite is in the hind parts.

Now, let’s talk framework choices.

Decisions, decisions

Game Engine

It may come as a surprise to some of the readers, but JVM game programming is not exactly a barren field, despite the infamous "Java is slow" stereotype. Probably the greatest influence in recent years has been the rise of Android, hence why a high proportion of JVM game engines provide support for both "mobile" and "desktop" targets[3].

In any case, two engines seem to be the strongest contenders at the moment: libGDX and jMonkeyEngine. Both have been present on the scene for several years, and offer comprehensive game development feature sets[4].

The choice, however, falls on libGDX, for two reasons: it has explicit support for 2D rendering, and it’s the one I’m actually familiar with.

I’m going to expand on libGDX in upcoming installments. Right now, let’s roll over to the other side for the…​

Streaming Implementation

Apart from the obvious choice of Akka Streams, I was also monitoring the progress of Swave.

I recommend following its development closely. It is, in implementation and design, significantly less complex than Akka Streams. Swave can therefore serve as a convenient stepping stone in beating the learning curve, when adapting to the streaming paradigm.

However, Swave is still shaping up, its documentation is somewhat lacking[5], and I have a sneaking suspicion that some unique features of its "competitor" (like blueprinting) might come in handy, so we’re sticking with Akka Streams.

OK, so the tech base is decided now. All that remains is one tiny detail - just what is the end goal and how to achieve it?

Some ground rules

Let’s start with the goal itself, since it’s pretty obvious - a natural way to verify whether something works (or doesn’t), is through a practical proof-of-concept. Therefore, during this series, I’m going to work towards a fully-playable simple game, that uses streaming as its core operational model.

This leads nicely to the preferred form of the series, i.e. a development blog - meaning:

  • installments will be posted (hopefully) regularly,

  • with a soft word-length cutoff,

  • mistakes will be made prominent (and prominently made), including potentially avoidable ones, in order to showcase them and their solutions[6].

Finally, to spice things up a bit, I’m going to deliberately forgo gathering "technical" inspiration from the aforementioned related talks and projects (like the additional abstractions introduced by Dr. Prokopec), in order to see where the problem domain lends toward convergent solution creation.

Up Next

In the next installment, we’re going to cover defining the scope of the PoC game. Stay tuned!

1. Source code here.
2. There are at least two similar projects. One went in the direction of providing an Rx* translation layer. It seems to be inactive for about two years now. Another, implemented in Kotlin, seems to be under active development. Both use the Rx* family of libraries.
3. Be aware 'though that since Android supports "native applications", a lot of Android games are written in C/C++ instead.
4. To answer a - perhaps - lingering question: Minecraft, the poster child of JVM games, uses neither. It has a proprietary engine, with several native bridges for OpenGL support etc.
5. Do see the presentations 'though for a good feel of what it offers.
6. Also making for a nice excuse in case of eggregious blunders.

Painless, type-safe config file parsing with Macwire and Ficus

Mikołaj Koziarkiewicz

Intro

Scala has a distinct advantage of an "official" configuration file format, i.e. HOCON. Even better, the niceness transcends the officialdom - HOCON is quite well-defined, and it has lots of useful of features. In other words, it’s simply a good format.

Of course, any Scala developer past their Day 3 can tell you of a certain little quirk - the reference parser implementation is a Java library. Even though Java-Scala interop is actually mostly tolerable once one learns the ropes, it still remains slightly annoying.

This post will describe a possible approach for reducing that annoyance by synergizing two well-known libs.

The following is intended for devs already minimally familiar with Macwire. If you’re not, take a look at the README. Reading the the Introduction will suffice.

Config

There are several popular pure-Scala HOCON parsing libraries available. We’re going to use Ficus.

As a library, it’s extremely straightforward - specify the config file, the subpath within it, the type you want it to be parsed into - the lib will do the rest. It even supports extraction into semi-arbitrary data types, like so:

server {
    host: "example.com"
    port: 1024
  }
case class ServerConfig(host: String, port: Int)

import net.ceedubs.ficus.Ficus._
import net.ceedubs.ficus.readers.ArbitraryTypeReader._

ConfigFactory.load().as[ServerConfig]("server")
//emits ServerConfig("example.com", 1024)

Ad Rem

First Approach

So, keeping with the theme, let’s wire up a life support system for plants. Say we have two HTTP services - one for moisture control, and the other handling miscellaneous sensors:

services.scala
class HydroService(val config: ServerConfig) {
    //logic goes here
}

class SensorService(val config: ServerConfig) {
   //logic goes here
}

where, again, the config class is:

case class ServerConfig(host: String, port: Int)

So, our config file could look like this:

app {
  hydro {
    host: "hydro.example.com"
    port: 8215
  }

  sensors {
    host: "sensors.example.com"
    port: 1777
  }
}

To make things sweet and simple, let’s encapsulate the the entire (relevant) config into a container class:

AppConfig.scala
class AppConfig(val hydro: ServerConfig, val sensors: ServerConfig)

case class ServerConfig(host: String, port: Int)

We’ll now take the first stab at the application runner:

Run.scala
import com.softwaremill.macwire._
import com.typesafe.config.ConfigFactory

object Run extends App {

  lazy val rawConfig = ConfigFactory.load()

  import net.ceedubs.ficus.Ficus._
  import net.ceedubs.ficus.readers.ArbitraryTypeReader._

  lazy val config: AppConfig = rawConfig.as[AppConfig]("app") (1)

  import config._ (2)

  lazy val hydroService = wire[HydroService]
  lazy val sensorService = wire[SensorService]

  println(s"HydroService configured with ${hydroService.config}") (3)
  println(s"SensorService configured with ${sensorService.config}")

}
1 It is a good idea for your app’s config to be contained in a dedicated subpath, mostly for reasons of avoiding namespace clashes, but also due to technical difficulties of loading the "root path".
2 Bringing ServerConfig instances into scope.
3 Debug statements.

Obviously, we’ll end up with:

> runMain Run
[info] Updating {file:}root-201604ficusplay...
[info] Resolving jline#jline;2.12.1 ...
[info] Done updating.
[info] Compiling 3 Scala sources to target/scala-2.11/classes...
[error] src/main/scala/Run.scala:24: Found multiple values of type [ServerConfig]: [List(sensors, hydro)]
[error]   lazy val hydroService = wire[HydroService]
[error]                               ^
[error] src/main/scala/Run.scala:25: Found multiple values of type [ServerConfig]: [List(sensors, hydro)]
[error]   lazy val sensorService = wire[SensorService]
[error]                                ^
[error] two errors found

because Macwire cannot distinguish between the two ServerConfig instances pulled into scope with import config._.

So, what can we do?

Tagging to the rescue

Well, it turns out Macwire possesses the notion of Qualifiers. In intent, they are identical to JSR 330 qualifiers, the only difference is that they operate on types instead of annotations (since Macwire performs its DI based on types).

OK, so we need to qualify our two service dependencies with some sort of tagging types. We could create marker traits for that. However, we notice that we:

  • already have two distinct types

  • that unambiguously convey our intent.

I’m talking, obviously, about the service types themselves!

OK, so let’s tag those config dependencies:

services.scala
import com.softwaremill.tagging._

class HydroService(val config: ServerConfig @@ HydroService)

class SensorService(val config: ServerConfig @@ SensorService)

Of course, for MacWire to know where to get these dependencies from, we have to tag the config class as well:

AppConfig.scala
import com.softwaremill.tagging._

class AppConfig(val hydro: ServerConfig @@ HydroService, val sensors: ServerConfig @@ SensorService)

case class ServerConfig(host: String, port: Int)

OK, looks like we’re all set. Let’s run our app now, aaand:

[info] Compiling 1 Scala source to target/scala-2.11/classes...
[error] src/main/scala/Run.scala:20: Cannot generate a config value reader for type
com.softwaremill.tagging.@@[ServerConfig,HydroService], because value readers cannot be auto-generated for types with type parameters.
Consider defining your own ValueReader[com.softwaremill.tagging.@@[ServerConfig,HydroService]]
[error]   lazy val config: AppConfig = rawConfig.as[AppConfig]("app")

Still no go.

Rescuing the tagging

The error message pretty much spells out the problem [1]. Since the config instances are now of type @@[ServerConfig,XService], Ficus is unable to find a way to construct instances of their types.

Via the error message, we’re offered a solution of implementing a ValueReader, which is what Ficus depends on when transforming config files into instances. Fortunately, from our previous attempt, we know that Ficus already has ValueReader objects in scope capable of generating case classes like ServerConfig [2]. Additionally:

  • ValueReader has a map method,

  • MacWire provides a taggedWith[TTag] helper that converts any type TType into @@[TType, TTag].

Let’s put those pieces together and add our custom reader for tagged types:

implicit def taggedReader[TType, TTag](implicit reader: ValueReader[TType]) = reader.map(_.taggedWith[TTag])

After we add the above to run, we should end up with:

> runMain Run
[info] Compiling 1 Scala source to target/scala-2.11/classes...
[info] Running Run
HydroService configured with ServerConfig(hydro.example.com,8215)
SensorService configured with ServerConfig(sensors.example.com,1777)
[success] Total time: 2 s, completed 2016-04-30 15:08:12

And we’re pretty much done!

One final improvement that we can make is to better convey the relationship between the transformed type and ValueReader. We do this by using the following equivalent form of our reader:

implicit def taggedReader[TType: ValueReader, TTag] = implicitly[ValueReader[TType]].map(_.taggedWith[TTag])

In the end, our app class looks like this:

Run.scala
import com.softwaremill.macwire._
import com.softwaremill.tagging._
import com.typesafe.config.ConfigFactory
import net.ceedubs.ficus.readers.ValueReader

object Run extends App {

  lazy val rawConfig = ConfigFactory.load()

  import net.ceedubs.ficus.Ficus._
  import net.ceedubs.ficus.readers.ArbitraryTypeReader._


  implicit def taggedReader[TType: ValueReader, TTag] = implicitly[ValueReader[TType]].map(_.taggedWith[TTag])

  lazy val config: AppConfig = rawConfig.as[AppConfig]("app")

  import config._

  lazy val hydroService = wire[HydroService]
  lazy val sensorService = wire[SensorService]

  println(s"HydroService configured with ${hydroService.config}")
  println(s"SensorService configured with ${sensorService.config}")

}

Summary

We’ve managed to get Macwire and Ficus happily working together to parse HOCON config files, in a type-safe manner.

Note that creating a master config object in larger, multi-module projects, is a Bad Idea™. However, that’s not a problem, since you can just pass on the "raw" config (from typesafe-config) onto the relevant submodules, and have Ficus generate the actual config instances there.

Overall, this approach scales well (architecturally), and requires only the minimal boilerplate necessary for type safety.

You can find the full code example on GitHub.

1. If only more Scala libs took this degree of care with compile-time messages…​
2. Since the original approach went pass the "Ficus stage" successfully, and only emitted an error during wiring.