ScalacollectionmethodsflatMap andflatten aremorepowerfulthanmonadicflatMap andflatten

Amonadisanimplementationofoneoftheminimalsetsofmonadiccombinators,satisfyingthelawsofassociativityandidentity.

Hereisamonadtraitimplementingmonadiccombinatorsunit andflatMap

trait Monad[F[_]]{def unit[A](a: ⇒ A): F[A]def flatMap[A,B](ma: F[A])(f: A ⇒ F[B]): F[B]def map[A,B](m: F[A])(f: A ⇒ B): F[B] = flatMap(m)(a ⇒ unit(f(a)))def flatten[A](mma: F[F[A]]): F[A] = flatMap(mma)(ma ⇒ ma)…

}Andhereisamonadtraitimplementingmonadiccombinatorsunit,map andflatten

trait Monad[F[_]]{def unit[A](a: ⇒ A): F[A]def map[A,B](m: F[A])(f: A ⇒ B): F[B] def flatten[A](mma: F[F[A]]): F[A]def flatMap[A,B](ma: F[A])(f: A ⇒ F[B]): F[B] = flatten(map(ma)(f))…

}

Theflatten functiontakesanF[F[A]]andreturnsanF[A]

def flatten[A](mma: F[F[A]]): F[A]

Whatitdoesis“removealayer”ofF.

TheflatMap functiontakesanF[A]andafunctionfromAtoF[B]andreturnsanF[B]

def flatMap[A,B](ma: F[A])(f: A ⇒ F[B]): F[B]

WhatitdoesisapplytoeachAelementofmaafunctionfproducinganF[B],butinsteadofreturningtheresultingF[F[B]],itflattensitandreturnsanF[B].

Inthefirstmonadtrait,flatten isdefinedintermsofflatMap:

def flatten[A](mma: F[F[A]]): F[A] = flatMap(mma)(ma => ma)

SoflatteningisjustflatMappingtheidentityfunction x => x.

Inthesecondmonadtrait,flatMap isdefinedintermsofmap andflatten:

def flatMap[A,B](ma: F[A])(f: A ⇒ F[B]): F[B] = flatten(map(ma)(f))

SoflatMappingafunctionisjustmappingthefunctionfirstandthenflatteningtheresult.

flattening isjustflatMappingidentity– flatMapping ismappingandthenflattening

trait Monad[F[A]]{def unit[A](a: ⇒ A): F[A] def flatMap[A,B](ma: F[A])(f: A ⇒ F[B]): F[B] def map[A,B](m: F[A])(f: A ⇒ B): F[B] = flatMap(m)(a ⇒ unit(f(a)))def flatten[A](mma: F[F[A]]): F[A] = flatMap(mma)(ma ⇒ ma)

}

val listMonad = new Monad[List] {override def unit[A](a: ⇒ A) = List(a)override def flatMap[A,B](ma: List[A])(f: A ⇒ List[B]): List[B] = ma flatMap f

}

WecannowuselistMonad’sflattenmethodtoflattenaList ofLists:

assert(listMonad.flatten(List(List(1,2,3),List[Int](),List(4,5,6))) == List(1,2,3,4,5,6))

SimilarlyforothercollectionslikeSet,Vector,etc:

val setMonad = new Monad[Set] { … }val vectorMonad = new Monad[Vector] { … }assert(setMonad.flatten(Set(Set(1,2,3),Set[Int](),Set(4,5,6))) == Set(1,2,3,4,5,6))assert(vectorMonad.flatten(Vector(Vector(1,2,3),Vector[Int](),Vector(4,5,6))) == Vector(1,2,3,4,5,6))

Butwhatwecannotdoismixtypes.E.g.wecan’tflattenaList ofSets:

assert(listMonad.flatten(List(Set(1,2,3),Set[Int](),Set(4,5,6))) == List(1,2,3,4,5,6))^

error:typemismatch;found:List[scala.collection.immutable.Set[Int]] required:List[List[?]]

ThereasonisthatthesignatureofflattenexpectsanF[F[A]],notanF[G[A]].E.g.itexpectsaList[List[A]]oraSet[Set[A]],notaList[Set[A]]

IfweinstantiatethefirstMonadtraitusingList’sownflatMapmethodthenthetraitgivesusaflattenmethodforfree

Wecanflatten F[F[A]],butnotF[G[A]].e.gwecanflattenList[List[A]],notList[Set[A]].

AmonadcanflattenF[F[A]],butnotF[G[A]]

trait Monad[F[A]]{def unit[A](a: ⇒ A): F[A] def map[A,B](m: F[A])(f: A ⇒ B): F[B]def flatten[A](mma: F[F[A]]): F[A]def flatMap[A,B](ma: F[A])(f: A ⇒ F[B]): F[B] = flatten(map(ma)(f))

}

val listMonad = new Monad[List] {def unit[A](a: ⇒ A): List[A] = List(a)def map[A,B](m: List[A])(f: A ⇒ B): List[B] = m map fdef flatten[A](mma: List[List[A]]): List[A] = mma.flatten

}

WecannowuselistMonad’sflatMapmethod,toflatmapalistwithafunctionthatcreatesalist:

assert(listMonad.flatMap(List(1,0,4)){case 0 => List[Int]() case x => List(x,x+1,x+2)} == List(1,2,3,4,5,6))

SimilarlyforothercollectionslikeSet,Vector,etc:

val setMonad = new Monad[Set] { … }val vectorMonad = new Monad[Vector] { … }assert(setMonad.flatMap(Set(1,0,4)){case 0 => Set[Int]() case x => Set(x,x+1,x+2)} == Set(1,2,3,4,5,6))assert(vectorMonad.flatMap(Vector(1,0,4)){case 0 => Vector[Int]() case x => Vector(x,x+1,x+2)} == Vector(1,2,3,4,5,6))

Butwhatwecannotdoismixtypes.E.g.wecan’tflatmapaList withafunctionthatcreatesaSet :

assert(listMonad.flatMap(List(1,0,4)){case 0 => Set[Int]() case x => Set(x,x+1,x+2)} == Set(1,2,3,4,5,6))^ ^

error:typemismatch;found:scala.collection.immutable.Set[Int] required:List[?]

ThereasonisthatthesignatureofflatMapoperatesonanF[A]andafunctionthatcreatesanF[B],notaG[B].E.g.itoperatesonaList[A]andafunctionthatcreatesaList[A],notaSet[A].

IfweinstantiatethesecondMonadtrait,usingList’sownmap andflattenmethodsthenthetraitgivesusaflatMapmethodforfree

Wecan flatMap F[A]withafunctionthatcreatesanF[B],notonethatcreatesaG[B].e.g.wecan flatMap List[A]withafunctionthatcreatesaList[B],notonethatcreatesaSet[B].

AmonadcanflatMap F[A]withafunctionreturningF[B],butnotwithafunctionreturningG[B]

Theflatten andflatMap methodsofourmonadinstancesdon’tsupportmixingoftypes.e.g.thefollowingdoesnotcompile:

assert(listMonad.flatten(List(Set(1,2,3),Set[Int](),Set(4,5,6))) == List(1,2,3,4,5,6))assert(listMonad.flatMap(List(1,0,4)){case 0 => Set[Int]() case x => Set(x,x+1,x+2)} == List(1,2,3,4,5,6))

Theflatten andflatMap methodsofList, ontheotherhand,doallowmixingoftypes.e.g.thefollowingworks:

assert(List(Set(1,2,3),Set[Int](),Set(4,5,6)).flatten == List(1,2,3,4,5,6))assert(List(1,0,4).flatMap{case 0 => Set[Int]() case x => Set(x,x+1,x+2)} == List(1,2,3,4,5,6))

Infact List supportsevenmoremixingoftypes:

assert(List(Set(1,2,3),Vector[Int](),List(4,5,6)).flatten == List(1,2,3,4,5,6))assert(List(1,0,4).flatMap{case 0 => Set[Int]() case x => Vector(x,x+1,x+2)} == List(1,2,3,4,5,6))

Howdotheflatten andflatMap methodsofScalacollectionssupportmixingoftypes?

Themonadicflatten andflatMap methodsdon’tsupportmixingoftypes,buttheflatten andflatMap methodsofScalacollectionsdo

Fromhttps://docs.scala-lang.org/overviews/core/architecture-of-scala-collections.html

Almostallcollectionoperationsareimplementedintermsof traversals and builders.Traversalsarehandledby Traversable’s foreach method,andbuildingnewcollectionsishandledbyinstancesofclass Builder.

trait Builder[-Elem, +To] Buildersaregenericinboththeelementtype, Elem,andinthetype, To,ofcollectionstheyreturn.…Youcanaddanelement x toabuilder b with b+=x.There’salsosyntaxtoaddmorethanoneelementatonce,forinstance b+= (x,y).Addinganothercollectionwith b++=xs worksasforbuffers.The result() methodreturnsacollectionfromabuilder.…[flatMap]usesa builderfactory that’spassedasanadditionalimplicitparameteroftype CanBuildFrom.

def flatMap[B, That](f: A => scala.collection.GenTraversableOnce[B])(implicit bf: CanBuildFrom[Repr, B, That]): That

…CanBuildFrom isafactoryforabuilder:

trait CanBuildFrom[-From, -Elem, +To]

CanBuildFrom represents builder factories. It has three type parameters:• From indicates the type for which this builder factory applies• Elem indicates the element type of the collection to be built• To indicates the type of collection to build

Fromhttps://www.scala-lang.org/blog/2017/05/30/tribulations-canbuildfrom.html

CanBuildFrom is probably the most infamous abstraction of the current collections. It is mainly criticised for making scary type signatures.

TheflatMap andflatten methodsofScalacollectionsrelyontraversals,collectionbuildersandbuilderfactories

List(1,0,4).flatMap{case 0 => Set[Int]() case x => Set(x,x+1,x+2)}

Thefollowingimplicitbuilderfactory intheList companionobjectisselected:

implicit def canBuildFrom[A]: CanBuildFrom[Coll, A, List[A]] = ReusableCBF.asInstanceOf[GenericCanBuildFrom[A]]def newBuilder[A]: Builder[A, List[A]] = new ListBuffer[A]

Thefollowing flatMap definitionin List isselected

final override def flatMap[B,That](f:A=>GenTraversableOnce[B])(implicit bf: CanBuildFrom[List[A],B,That]):That

bf:builderfactory creatingabuilder (aListBuffer),thatcanbeusedtobuildaList.

Iftheselectedlistbuilderfactory bf isReusableCBF,thenList’sflatMap doesn’tusethefactoryatall!Instead,itdoesitsownlistbuilding:

Foreachlistelement, flatMap adds(tothelistitisbuilding)theelementsofthetraversable createdbyapplyingf tothelistelement.

Iftheselectedbuilderfactoryissomeotherfactory,then flatMap delegatestothe flatMap intraitTraversableLike:

def flatMap[B, That](f: A => GenTraversableOnce[B])(implicit bf: CanBuildFrom[Repr, B, That]): That = {def builder = bf(repr) val b = builderfor (x <- this) b ++= f(x).seqb.result

}

flatMap first uses builder factory bf to create a list builder. Foreach list element, flatMap then gets the builder to add (to the list itis building) the elements of the traversable (e.g. a Set) created byapplying f to the list element.

HowList’s flatMapmethodbuildsaList

ReusableCBF delegatescreationofabuilder tothismethod

NOTE:f doesnotreturnaList,itreturnssomethingthatcanbetraversed usingitsforeachmethod.

List’sflatMap method

NOTE:f returnssomethingthatcanbetraversed usingitsforeachmethod

Vector(1,0,4).flatMap{case 0=>Set[Int]() case x=>Set(x,x+1,x+2)}

Thefollowingimplicitbuilderfactory intheVector companionobjectisselected:

def newBuilder[A]: Builder[A, Vector[A]] = new VectorBuilder[A]implicit def canBuildFrom[A]: CanBuildFrom[Coll, A, Vector[A]] =

ReusableCBF.asInstanceOf[GenericCanBuildFrom[A]]

ThefollowingflatMap definitioninTraversableLikeisselected:

def flatMap[B, That](f: A => GenTraversableOnce[B])(implicit bf: CanBuildFrom[Repr, B, That]): That = {def builder = bf(repr) val b = builderfor (x <- this) b ++= f(x).seqb.result

}

bf:builderfactorycreatingabuilder (aVectorBuilder)thatcanbeusedtobuildaVector.

Foreachvectorelement,flatMap getsVectorBuilder toadd(tothevectoritisbuilding)theelementsofthetraversable (e.g.aSet) createdbyapplyingf tothevectorelement.

HowVector’sflatMapmethodbuildsaVector

ReusableCBF delegatescreationofabuildertothismethod

Set(1,0,4).flatMap{case 0=>Vector[Int]() case x=>Vector(x,x+1,x+2)}

ThefollowingflatMap definitioninTraversableLike isselected:

def flatMap[B, That](f: A => GenTraversableOnce[B])(implicit : CanBuildFrom[Repr, B, That]): That = {def builder = (repr) val b = builderfor (x <- this) b ++= f(x).seqb.result

}

whichdelegatescreationofabuilder tothefollowingmethodinabstractclassImmutableSetFactory

def newBuilder[A]: Builder[A, CC[A]] = new SetBuilder[A, CC[A]](empty[A])

whichdelegatestothefollowingmethodinabstractclassGenSetFactory

def setCanBuildFrom[A] = new CanBuildFrom[CC[_], A, CC[A]] {def apply(from: CC[_]) = from match {case from: Set[_] => from.genericBuilder.asInstanceOf[Builder[A, CC[A]]]case _ => newBuilder[A]

}def apply() = newBuilder[A]

}

Foreachsetelement,flatMap getsSetBuilder toadd(tothesetitisbuilding)theelementsofthetraversable (e.g.aVector)createdbyapplying f tothesetelement.

HowSet’sflatMap methodbuildsaSet

ThefollowingimplicitbuilderfactoryintheSet companionobjectisselected:

implicit def canBuildFrom[A]: CanBuildFrom[Coll, A, Set[A]] = setCanBuildFrom[A]

bf:builderfactorycreatingabuilder (aSetBuilder)thatcanbeusedtobuildaSet.

Unliketheflattenmethodofourmonadinstances,theflattenmethodofScalacollections,e.g.List/Vector/Set,candothefollowing:

assert(List(Set(1,2,3),Set[Int](),Set(4,5,6)).flatten == List(1,2,3,4,5,6))assert(Vector(Set(1,2,3),Set[Int](),Set(4,5,6)).flatten == Vector(1,2,3,4,5,6))assert(Set(Vector(1,2,3),Vector[Int](),Vector(4,5,6)).flatten == Set(1,2,3,4,5,6))

Howdoestheflattenmethodconvertthenestedcollections,whosetypesdifferfromthetypeoftheenclosingcollection,tocollectionsofthesametypeastheenclosingcollection?

Inalltheabovethreecases,flatten isimplementedintraitGenericTraversableTemplate:

def flatten[B](implicit asTraversable: A => GenTraversableOnce[B]): CC[B] = {val b = genericBuilder[B]for (xs <- sequential)b ++= asTraversable(xs).seqb.result()

}

WhentheenclosingcollectionisList,thefollowingbuilderintheList companionobjectisused:def newBuilder[A]: Builder[A, List[A]] = new ListBuffer[A]

WhentheenclosingcollectionisVector,thefollowingbuilderintheVector companionobjectisused:def newBuilder[A]: Builder[A, Vector[A]] = new VectorBuilder[A]

WhentheenclosingcollectionisSet,thefollowingbuilderinabstractclassImmutableSetFactory isused:def newBuilder[A]: Builder[A, CC[A]] = new SetBuilder[A, CC[A]](empty[A])

Foreachcollection-typedelement(e.g.aSet orVector),flatMap getsthe builder (the ListBuffer /VectorBuilder /SetBuilder)toadd(totheList /Vector /Set itisbuilding)theelementsofthecollection-typedelement(atraversable whoseelementscanbeaccessedwithitsforeachmethod).

animplicitconversionwhichassertsthattheelementtypeofthiscollection,e.g.List/Vector/Set, isaGenTraversableOnce,whichprovidesaforeach methodgivingaccesstoitselements.

Howtheflattenmethodofacollectionhandlesnestedcollectionsoftypesdifferingfromthatofthecollection

b:acollectionbuilder - thetypeofbuilder dependsonthetypeofthecollectiontobebuilt,seebottomofslide

TheflatMap andflatten methodsofScalacollectionscanoperateonelementsofmanytypes,includingOption,Range andMap

Option,Range andMap areallexamplesof GenTraversableOnce

import scala.collection.GenTraversableOnceval o: GenTraversableOnce[Int] = Some(3)val r: GenTraversableOnce[Int] = Range(1,3)val m: GenTraversableOnce[(String,Int)] = Map("1"->1,"2"->2)

Sothe flatMap andflattenmethodsofacollection,e.g.aList,canoperateonthosetypes:

assert(List(Some(1),None,Some(2),None,Some(3)).flatten == List(1,2,3))

assert(List(0,1,2).flatMap{case 0 => None case x => Some(x)} == List(1,2))

assert(List(Range(1,4),Range(4,7)).flatten == List(1,2,3,4,5,6))

assert(List(0,1,4).flatMap{case 0 => Range(0,0) case x => Range(x,x+3)} == List(1,2,3,4,5,6))

assert(List(Map("1" -> 1, "2" -> 2), Map[String,Int](), Map("3" -> 3, "4" -> 4)).flatten == List("1"->1, "2"->2, "3"->3, "4"->4))

assert(List(0,1,3).flatMap{case 0 => Map[String,Int]() case x => Map(s"$x"->x, s"${x+1}"->(x+1))} == List("1"->1, "2"->2, "3"->3, "4"->4))