Writing a Scala object mapping library – Part 2: Shapeless
I’ve been writing an object mapping library in Scala for fun. In the second part of this blog series I’ll be examining if using shapeless was the right choice.
In the introduction to this blog series, I explored how you could create object mapping code using Scala. Recap: using a simple approach seems to be unmaintainable, because the amount of mapping code increases rapidly depending on two factors:
- The size of your data types (the quality axis)
- The number of data types (the quantity axis)
In this part of the series, we try a different approach.
Possible solution #2: shapeless’
At this point I started thinking of using an external library, but to keep myself challenged, a library that already does all the mapping was not something I was looking for. A quick search for such a library didn’t give me much anyway, apart from maybe this Stack Overflow question. So time to get back to typing.
I already mentioned the shapeless library in the introduction. It is described as Generic programming for Scala, so that promises a nice base for a generic solution to the boilerplate problem.
I won’t dive deeply into the magical world of shapeless, but one of the features the library offers is converting Scala case class instances to
HList instances. A
HList is a datatype for heterogenous lists of fixed length, meaning you can have a list containing elements of different types. For instance:
HList instances can be used to represent arguments to a function. Since a case class constructor is a function, you can use
HList instances as a projection of a case class constructor call, and vice versa. The shapeless library offers this through the use of
Okay, that’s nice. But one thing, field names are ignored. Not of much use when you want to map case class fields based on name. The code below will work, but does not give the desired result; the fields are mapped regardless of field name:
And another thing, it requires the two separate case classes to have fields of the same type in the same order. So this will not work:
It will fail during compilation:
See those expanded types in the compilation error? The
HList representation of
Int :: String :: HNil while the one for
String :: Int :: HNil. Type mismatch. Compilation failed. Sad face 🙁
There’s another tool in the shapeless box that’s a better candidate for name-based field mapping, called
LabelledGeneric. What it does is almost the same as
Generic, but with the added nicety of tagging a field name to the element types in the
HList representation of case class fields. It does so through some macro magic that inspects the case class type during compilation. You can use it like this:
But still, when the code continues like below…
Failure. Very bad.
Type mismatch. Compilation failed. Sad face 🙁
What to do? Well…
turtles implicits all the way down
Ideally, for mapping objects, I’d like to have a nice syntax. Something like
Foo(1,. It should be available not only for
Foo instances, but for other case classes too. Sounds like a task for an implicit class:
Now, what is needed to map one case class instance to an instance of another case class? Maybe this should be the way to go:
- Get a
HListrepresentation of the
LabelledGeneric[Source], let’s call this
sourceHList. It will contain the data, but also information about the field names the separate data elements belong to.
- With the help of
LabelledGeneric[Target], get the type of the
HListrepresentation of the target case class, let’s call this
- For each component type in
TargetRepr, look up the corresponding element in
sourceHlist, and build a value of type
TargetRepr. Let’s call that value
- Get a fresh, shiny
Sounds simple enough:
- Get source representation
- Get target representation type
The tough part in step 3 is that it involves traversing a HList type.
Traversing a list value is simple. Below is a useless function that copies a list in a contrived way. You’d never need something like that in application code (unless you get paid by the line).
This is the bread and butter of functional programming in Scala: pattern matching and recursion. Can we use pattern matching and recursion at the type level, at least for
shapeless.HList types? Turns out we can:
Whew… It has generics, implicits, and it’s a lot more to type than that
copyList() function. And in the end it doesn’t even do anything useful! I’ll give an overview of what happens when you use the code.
HListCopy.copyHList() can be called with a
When the compiler encounters the call to
.copyHList(), it notices the absence of the
hListCopier argument in the call. Since it’s an implicit argument, it will try to find the best candidate for that argument with the specified type of
In this case, it will look for an instance of
HListCopier[Int :: String :: HNil]. The method the compiler uses to find that implicit is discussed in more depth than I will on Stack Overflow, but it boils down to inserting a call to either
HListCopier.hNilCopier will return a
HListCopier[HNil] which doesn’t match the type of
HListCopier[Int ::, so the compiler will try
String :: HNil]
HListCopier.nonEmptyHListCopier[A,. That one promises to return a
HListCopier[A :: Rest] which is generic over both the head and the tail of the
HList type, and possibly matches
HListCopier[Int. After this first implicit lookup, the code will be equivalent to this:
:: String :: HNil]
HListCopier.nonEmptyHListCopier[A, Rest] in turn has its own implicit argument, asking for a
HListCopier[Rest], which in this case expands to
HListCopier[String. Again, this matches with a call to
Now for the next implicit lookup, the implicit argument for
HListCopier[Rest] expands to
HListCopier[HNil]. We have
HListCopier.hNilCopier which promises to return exactly that, so it will be called:
At this point, there are no more implicits to look for, and the resulting code typechecks and runs fine.
That’s a lot of work for emulating a call to
But it offers a starting point for the actual field name-based mapper we have as our goal.
Here it is:
Now let’s use it:
It compiles, and does what we want, awesome.
But what if we try to use it in a way it isn’t supposed to be used?
Compilation fails, as it should:
The downside here is in the error message:
could not find implicit value for parameter ccMap: CaseClassMap[Foo,Baz]. It’s cryptic, not very user-friendly. I’d like to know what exactly is causing the compiler to give up. An error message that informs me about the fields that cannot be mapped would be way better, especially when dealing with mapping many fields.
The next part of this series will discuss how to generate more helpful compiler errors. Stay tuned!