Typelevel Ecosystem

With the imminent upcoming release of Cats v1.0 is incredibly amazing how the Scala’s pure FP ecosystem around it has matured. It’s worth mentioning and remarking the hard work of the entire community behind these Typelevel projects such as Cats Effect, Fs2, Http4s and Circe. And I must confess, it’s been and currently is my favorite tech stack in Scala ūüôā

Cats Effect

It is described as the standard IO for the Cats ecosystem in order to manage both synchronous and asynchronous (callback-driven) effects. The current version is v0.5 dependent on Cats v1.0.0-RC1.

It currently is the main effect type F[_] : Effect used by the latest versions of Fs2, Http4s and Circe.

Fs2

Or Functional Streams for Scala (formerly Scalaz-Stream), is a pure functional streaming I/O library, currently focusing the effort on the upcoming release of the version 0.10 (current version is v0.10.0-M8 dependent on Cats Effect v0.5). It has suffered many transformations but it has now reached a mature state which I consider production ready.

The main effect type used in previous versions was fs2.Task, and previously scalaz.concurrent.Task when it was Scalaz Stream, but it has now been removed in favor of the new standard cats.effect.Effect.

Circe

Described as “A Json library for Scala powered by Cats”, has also upgraded to the latest version of Cats v1.0.0-RC1 offering the milestone¬†v0.9.0-M2, and on the way to release its stable version. It is the default library chosen by Fs2 and Http4s for Json manipulation.

Doobie

“A pure functional JDBC layer for Scala and Cats” that provides¬†a principled way to construct programs (and higher-level libraries) that use JDBC. The latest version is v0.5.0-M9 dependent on the latest Fs2 v0.10.0-M8 and Cats 1.0.0-RC1.

Http4s

“A minimal and idiomatic Scala interface for HTTP”, as described in the main web site. In an incredible team effort, they are currently maintaining four different versions (0.15.x, 0.16.x, 0.17.x and 0.18.x), hats off to the team! The latest version is v0.18.0-M5 dependent on the latest Fs2 v0.10.0-M8 and Cats Effect v0.5.

Use Case Scenarios

In the last few months I’ve been working on Smart Backpacker, an application for travelers where you can find information such as Visa Requirements, Currency Exchange and Airline’s Baggage Policy. If you are an Android user you can try it out right now (download it here). However, iOS users must wait until the end of the year where I hope we can have it ready (currently being developed by my friend @sheinix).

The backend is written in Scala¬†(as you might already guessed) using a few of the libraries described above such as Http4s, Cats Effect and Circe. The Android App has been written 100% by me (I must say I hate dealing with XML and Android layouts) but it wasn’t so bad to do it while learning Kotlin at the same time. I plan to release both projects as open source next year so stay tuned!

Also, recently I’ve been challenged to solve an exercise involving the creation of a User Http API on top of an existent backend service. The requirements, in my understanding, require authentication so I went for¬†Http4s Auth (first time using it) and as I expected works like a charm! Special thanks to @rossabaker who was extremely helpful and friendly in the Http4s Gitter channel¬†when I asked for help regarding the Auth feature.

And last but not least, my personal project fs2-rabbit has also been updated to the latest Fs2 & Cats Effect versions. Oh! And it’s now making use of the latest sbt v1.0.3 ūüôā

Final Thoughts

I’m just super excited about all of what’s going on in the Scala ecosystem right now and currently looking for a new challenge in my professional career. The future looks brighter than ever… Keep on rocking Scalars!

Cheers,
Gabriel.

CRUD and error handling with Http4s

Hi everybody!

Last year I wrote an introductory post to Http4s and a few months later I started using it in the company I was working for. Results have been very good so far regarding readability in a pure-functional-style, usability, throughput and performance in production environments so I would recommend the use of it.

In this post I want to show you how to create a¬†simple CRUD (Create, Read, Update, Delete) demo, always needed when creating an API. And in this kind of operations you will always need to handle expected errors from the backend and returning the appropriate HTTP Status Code and response. So let’s start!

EDIT: This post has been written in October 2016 using Http4s v0.14.6 for the examples and things have changed quite a lot ever since. For a newer version using Http4s v0.18.x (Cats v1.0.0-RC1) please check out this Giter8 Template, this repo and this blog post about all the updates.

CRUD for users

In this demo we will create a CRUD for an User (username, email and age). We will support the following operations:

  1. Find all the users sorted by username
  2. Create a new user:
    • If the username already exists return a descriptive error.
    • Otherwise return the user with the designated id.
  3. Find user by id:
    • Return either the user or a descriptive message indicating user not found.
  4. Edit the age of a user by id:
    • Return either a successful code or a descriptive message indicating user not found.
  5. Delete user by id:
    • Return either a successful code or a descriptive message indicating user not found.

HTTP Status Code Responses

So far this is the definition of the most common use of a crud with some validations. Let’s see how we define the HTTP Endpoints for all these operations:

  1. GET /users | /users?sort=ASC | /users?sort=DESC
    • Returns a 200 http status code with a list of all the users.
  2. POST /users
    • If the username is already in use returns a 409 http status code indicating the duplication.
    • Otherwise creates the user and returns a 201 http status code and¬†the user with id in the body.
  3. GET /users/{id}
    • If the user¬†exists it¬†returns a 200 http status code and the user itself.
    • Otherwise it returns a 400 http status code indicating user not found.
  4. PUT /users/{id}
    • If the user exists it returns a 202 http status code.
    • Otherwise it returns a 400 http status code indicating user not found.
  5. DELETE /users/{id}
    • If the user exists it returns a 204 http status code.
    • Otherwise it returns a 400 http status code indicating user not found.

OK, show me the code!

We will start by creating the main app running the server, as indicated by the official¬†documentation, it’s recommended to extend the ServerApp trait that will handle the shutdown and clean up of the resources automatically:

package com.gvolpe.http.server

import com.gvolpe.http.server.endpoint.UserHttpEndpoint
import org.http4s.server.{Server, ServerApp}
import org.http4s.server.blaze._
import scalaz.concurrent.Task

object HttpApi extends ServerApp {

  override def server(args: List[String]): Task[Server] = {
    BlazeBuilder
      .bindHttp(8080, "localhost")
      .mountService(UserHttpEndpoint.service)
      .start
  }

}

By default we use the Blaze server as it is the native backend supported by http4s but it’s also possible to use different servers, however that topic is out of the goal of this post.

Our UserService trait defines the CRUD operations always returning a scalaz.concurrent.Task[A] as shown below:

trait UserService {
  def save(user: User): Task[Unit]
  def findAll(sort: UserSorting): Task[List[User]]
  def find(id: Long): Task[User]
  def remove(id: Long): Task[Unit]
  def edit(id: Long, age: Int): Task[Unit]
}

We will not care about the implementation of this service because what it matters in this case is the returning types and the possible expected errors that a Task might contain when it runs. However you can take a look at our default implementation on GitHub (link at the end of the post).

In the code shown above the type UserSorting is just an ADT defining two possible values, either Asc or Desc.

Main User HTTP Endpoint

So this is how our UserHttpEndpoint object referenced by the main server app looks like. First of all, we define the codecs available implicitly in the scope of the object. For this purpose we are using the amazing Circe library using automated codec derivation.

implicit def circeJsonDecoder[A](implicit decoder: Decoder[A]) = jsonOf[A]
implicit def circeJsonEncoder[A](implicit encoder: Encoder[A]) = jsonEncoderOf[A]

Then we define the main resources of the endpoint as an HttpService that represents a PartialFunction[Request, Task[Response]]:

val service = HttpService {
  case GET -> Root / "users" : ? SortQueryParamMatcher(sort) =>
    Ok(UserService.findAll(UserSorting.from(sort)))
  case req @ POST -> Root / "users" =>
    req.decode[UserForm] { userForm =>
      val user = User(Random.nextInt(1000), userForm.username, userForm.email, userForm.age)
      UserService.save(user).flatMap(_ => Created(s"User with id: ${user.id}")).handleWith(errorHandler)
    }
  case GET -> Root / "users" / LongVar(id) =>
    Ok(UserService.find(id)).handleWith(errorHandler)    
  case DELETE -> Root / "users" / LongVar(id) =>
    UserService.remove(id).flatMap(_ => NoContent()).handleWith(errorHandler)
  case req @ PUT -> Root / "users" / LongVar(id) =>
    req.decode[UserAgeForm] { ageForm =>
      UserService.edit(id, ageForm.age).flatMap(_ => Accepted()).handleWith(errorHandler)
    }
}

In the code shown above we are using a SortQueryParamMatcher and an errorHandler variable that we didn’t see yet, so this is how is defined:

object SortQueryParamMatcher extends OptionalQueryParamDecoderMatcher[String]("sort")
  
private val errorHandler: PartialFunction[Throwable, Task[Response]] = {
  case UserNotFoundException(id)              => BadRequest(s"User with id: $id not found!")
  case DuplicatedUsernameException(username)  => Conflict(s"Username $username already in use!")
}

Adding a new validation would be just adding a new case in the partial function, very simple isn’t it?

Final thoughts

As I said at the beginning of this post I totally encourage the use of this powerful library if you like to write pure functional code and at the same time having great performance results. The documentation has improved a lot since the last year so please give it a try and let me know if you would like me to write about any other feature of this library not covered either here or in the official docs.

As always, a demo project for this post is available on GitHub, check it out here!

Until next post!
Gabriel.

PS: I’m currently on a world trip for undetermined¬†amount of¬†time and sometimes with limited connection to Internet so the gaps between posts might be bigger but I’ll try to keep on writing. I suppose I’ll be back in the industry next year at some point… Cheers from India!

HTTP API on top of Scalaz Streams

In the last days I’ve been playing with http4s which is defined as a minimal, idiomatic Scala interface for HTTP.

It’s a powerful library, type safe, composable and asynchronous. And it supports different servers like Blaze, Jetty and Tomcat.

Although the project remains a lot of work it’s always good to give it a try.

CREATING THE FIRST HTTP SERVICE

I started creating a basic service that returns a simple string when accessing the root level (localhost:8080/). This is how it looks:

object HomeService {

  def apply(): HttpService = service

  private val service = HttpService {
    case GET -> Root =>
      Ok("Http4s API")
  }

}

And here is the main application that runs the Blaze server:

object Api extends App {

  BlazeBuilder.bindHttp(8080)
    .mountService(HomeService(), "/")
    .run
    .awaitShutdown()

}

Fair enough to get a server running and serving a GET resource. Until here we have the “hello world” example of an HTTP service. After this we can create more services and add them to the server by invoking the mountService function of the server builder.

What I did was to create two similar services for Products and Users serving just mocking data. However the main difference is that the ProductService serves Json data using Play Json and the UserService exposes Json data using Circe.

PRODUCT SERVICE

This is the code for one of the GET resources for the ProductService:

case GET -> Root =>
  val products = List(Product(1, "Book"), Product(2, "Calc"), Product(3, "Guitar"))
  Ok(Json.toJson(products))

It’s very handy. However to get this code working you need some implicit values in the scope:

  • The Writer for the Play Json library.
  • The EntityEncoder[T] for http4s.

To accomplish this requirements I created the following object that is imported in the ProductService:

object PlayJsonImplicits {

  implicit val playJsonEncoder: EntityEncoder[JsValue] =
    EntityEncoder
      .stringEncoder(Charset.`UTF-8`)
      .contramap { json: JsValue => json.toString() }
      .withContentType(`Content-Type`(MediaType.`application/json`, Charset.`UTF-8`))

  implicit val productJsonFormat = Json.format[Product]

}

USER SERVICE

This is the code for one of the GET resources of the UserService:

case GET -> Root / id =>
  Ok(User(id.toLong, s"User$id", s"user$id@mail.com").asJson)

And here it happens something similar to the service above. In this case we need to import the Circe implicit values and to provide an EntityEncoder[T]. This is how it looks:

import io.circe.generic.auto._
import io.circe.syntax._
object CirceImplicits {

  implicit val circeJsonEncoder: EntityEncoder[CirceJson] =
    EntityEncoder
      .stringEncoder(Charset.`UTF-8`)
      .contramap { json: CirceJson => json.noSpaces }
      .withContentType(`Content-Type`(MediaType.`application/json`, Charset.`UTF-8`))

}

Until here we have a few services serving Json data but the most attracting feature of this library is the streaming one. So let’s move on some examples.

STREAMING DATA

Http4s was built on top of Scalaz Streams and every Request is transformed into an asynchronous Scalaz Task[Response]. This means that you can use any function that returns an async Task as a Http Response using the helpers provided by http4s.

Here we have an example extracted from the StreamingService:

private val service = HttpService {
  case GET -> Root =>
    val streamingData = Process.emit(s"Starting stream intervals\n\n") ++ dataStream(10)
    Ok(streamingData).chunked
}

private def dataStream(n: Int): Process[Task, String] = {
  implicit def defaultScheduler = DefaultTimeoutScheduler
  val interval = 100.millis
  time.awakeEvery(interval)
    .map(_ => s"Current system time: ${System.currentTimeMillis()} ms\n")
    .take(n)
}

To send the chunked response we need to add the Transfer Encoding header as below. What I did was to create an implicit class with a “chunked” function to make thinks easier.

response.putHeaders(`Transfer-Encoding`(TransferCoding.chunked))

And that’s it! Find out more and what is possible to do with Scalaz Streams by taking a look at the examples.

Another cool feature of this library is the Web Sockets support, but I’m not covering this topic now. However you’ll find a very basic example of WS connection in the sample project linked below. And maybe you want to take a look at this demo too.

TEST COVERAGE

As always this development stage is for me the most important. That’s why all the services are fully tested with a test coverage close to ~100% (actually Coveralls has some bugs and it’s showing only 92% but take a look at the coverage report! If you run “sbt clean coverage test” the coverage report shows 97.78%).

This is how it looks one of the unit tests for the ProductService:

"Get the list of products" in {
  val request = new Request()
  val response = service.run(request).run

  response.status should be (Status.Ok)
  val expected = """ [{"id":1,"name":"Book"},{"id":2,"name":"Guitar"}] """.trim
  response.body.asString should be (expected)
}

We are creating a Request and running the ProductService to get the response. Then we have assertions for the Status and the Body.

SAMPLE PROJECT

Check out the complete project on Github!.

Important Note: Only runs under Java 8.

Find out more examples in the official http4s ExampleService.

CONCLUSION

At this moment the documentation it’s a bit poor but I hope to find a better one in the future and many other improvements. Nevertheless I know the guys are working really hard on this powerful library.

Well, this was just a quick research, you are always invited to go deeper and deeper!

Until next post!
Gabriel.