Writing a Scala object mapping library – Part 2: Shapeless

In the introduction to this blog series, I explored how you could create object mapping code using Scala. Recap: using a simple approach seems to be unmaintainable, because the amount of mapping code increases rapidly depending on two factors:

• The size of your data types (the quality axis)
• The number of data types (the quantity axis)

In this part of the series, we try a different approach.

Possible solution #2: shapeless’ LabelledGeneric

At this point I started thinking of using an external library, but to keep myself challenged, a library that already does all the mapping was not something I was looking for. A quick search for such a library didn’t give me much anyway, apart from maybe this Stack Overflow question. So time to get back to typing.

I already mentioned the shapeless library in the introduction. It is described as Generic programming for Scala, so that promises a nice base for a generic solution to the boilerplate problem.

I won’t dive deeply into the magical world of shapeless, but one of the features the library offers is converting Scala case class instances to HList instances. A HList is a datatype for heterogenous lists of fixed length, meaning you can have a list containing elements of different types. For instance:

These HList instances can be used to represent arguments to a function. Since a case class constructor is a function, you can use HList instances as a projection of a case class constructor call, and vice versa. The shapeless library offers this through the use of Generic:

Okay, that’s nice. But one thing, field names are ignored. Not of much use when you want to map case class fields based on name. The code below will work, but does not give the desired result; the fields are mapped regardless of field name:

And another thing, it requires the two separate case classes to have fields of the same type in the same order. So this will not work:

It will fail during compilation:

See those expanded types in the compilation error? The HList representation of Foo is Int :: String :: HNil while the one for Bar is String :: Int :: HNil. Type mismatch. Compilation failed. Sad face 🙁

Label me…

There’s another tool in the shapeless box that’s a better candidate for name-based field mapping, called LabelledGeneric. What it does is almost the same as Generic, but with the added nicety of tagging a field name to the element types in the HList representation of case class fields. It does so through some macro magic that inspects the case class type during compilation. You can use it like this:

But still, when the code continues like below…

Failure. Very bad.

Type mismatch. Compilation failed. Sad face 🙁

What to do? Well…

It’s turtles implicits all the way down

Ideally, for mapping objects, I’d like to have a nice syntax. Something like Foo(1,
"quux").mappedTo[Bar]. It should be available not only for Foo instances, but for other case classes too. Sounds like a task for an implicit class:

Now, what is needed to map one case class instance to an instance of another case class? Maybe this should be the way to go:

1. Get a HList representation of the Source object through LabelledGeneric[Source], let’s call this sourceHList. It will contain the data, but also information about the field names the separate data elements belong to.
2. With the help of LabelledGeneric[Target], get the type of the HList representation of the target case class, let’s call this TargetRepr.
3. For each component type in TargetRepr, look up the corresponding element in sourceHlist, and build a value of type TargetRepr. Let’s call that value targetHList.
4. Get a fresh, shiny Target object through LabelledGeneric[Target].from(targetHList).

Sounds simple enough:

So it’s

1. Get source representation
2. Get target representation type
3. ???
4. Profit

The tough part in step 3 is that it involves traversing a HList type.

Traversing a list value is simple. Below is a useless function that copies a list in a contrived way. You’d never need something like that in application code (unless you get paid by the line).

This is the bread and butter of functional programming in Scala: pattern matching and recursion. Can we use pattern matching and recursion at the type level, at least for shapeless.HList types? Turns out we can:

Whew… It has generics, implicits, and it’s a lot more to type than that copyList() function. And in the end it doesn’t even do anything useful! I’ll give an overview of what happens when you use the code.

HListCopy.copyHList() can be called with a HList instance:

When the compiler encounters the call to .copyHList(), it notices the absence of the hListCopier argument in the call. Since it’s an implicit argument, it will try to find the best candidate for that argument with the specified type of HListCopier[Source].

In this case, it will look for an instance of HListCopier[Int :: String :: HNil]. The method the compiler uses to find that implicit is discussed in more depth than I will on Stack Overflow, but it boils down to inserting a call to either HListCopier.nonEmptyHListCopier[A,
Rest] or HListCopier.hNilCopier. HListCopier.hNilCopier will return a HListCopier[HNil] which doesn’t match the type of HListCopier[Int ::
String :: HNil], so the compiler will try HListCopier.nonEmptyHListCopier[A,
Rest]. That one promises to return a HListCopier[A :: Rest] which is generic over both the head and the tail of the HList type, and possibly matches HListCopier[Int
:: String :: HNil]. After this first implicit lookup, the code will be equivalent to this:

HListCopier.nonEmptyHListCopier[A, Rest] in turn has its own implicit argument, asking for a HListCopier[Rest], which in this case expands to HListCopier[String
:: HNil]. Again, this matches with a call to HListCopier.nonEmptyHListCopier[String,
HNil]:

Now for the next implicit lookup, the implicit argument for HListCopier[Rest] expands to HListCopier[HNil]. We have HListCopier.hNilCopier which promises to return exactly that, so it will be called:

At this point, there are no more implicits to look for, and the resulting code typechecks and runs fine.

That’s a lot of work for emulating a call to Predef.identity()…

But it offers a starting point for the actual field name-based mapper we have as our goal.

Here it is:

Now let’s use it:

It compiles, and does what we want, awesome.

But what if we try to use it in a way it isn’t supposed to be used?

Compilation fails, as it should: