1 17-214
SchoolofComputerScience
PrinciplesofSo3wareConstruc9on: Objects,Design,andConcurrencyPart5:EtceteraJavalambdasandstreamsCharlieGarrod BogdanVasilescu
2 17-214
Administrivia
• Homework5BestFrameworksavailabletoday• Homework5cdueMonday11:59p.m
3 17-214
KeyconceptsfromTuesday
4 17-214
Producer-consumerdesignpaSern
• Goal:Decoupletheproducerandtheconsumerofsomedata• Consequences:
– Removescodedependencybetweenproducersandconsumers– Producersandconsumerscanproduceandconsumeatdifferentrates
5 17-214
ThemembranepaSern
• Mul9pleroundsoffork-join,eachroundwai9ngforthepreviousroundtocomplete
Image from: Wikipedia
6 17-214
Parallelprefixsumsalgorithm,upsweep
• Computesthepar9alsumsinamoreusefulmanner
[13, 9, -4, 19, -6, 2, 6, 3]
[13, 22, -4, 15, -6, -4, 6, 9]
[13, 22, -4, 37, -6, -4, 6, 5]
[13, 22, -4, 37, -6, -4, 6, 42]
…
7 17-214
Parallelprefixsumsalgorithm,downsweep
• Nowunwindstocalculatetheothersums
[13, 22, -4, 37, -6, -4, 6, 42]
[13, 22, -4, 37, -6, 33, 6, 42]
[13, 22, 18, 37, 31, 33, 39, 42]
• Recall,westartedwith:[13, 9, -4, 19, -6, 2, 6, 3]
8 17-214
Doublingarraysizeaddstwomorelevels
Upsweep
Downsweep
9 17-214
Fork/Join:computa9onalpaSernforparallelism
• Forkataskintosubtasks• Jointhesubtasks(i.e.,waitforthemtocomplete)• Subtasksaredecomposedrecursively
• Thejava.util.concurrent.ForkJoinPoolclass– ImplementsExecutorService– Executes java.util.concurrent.ForkJoinTask<V>or
java.util.concurrent.RecursiveTask<V>or java.util.concurrent.RecursiveAction
• Thethreadsinthefork-joinpooldoworkstealing
10 17-214
AForkJoinexample
• SeePrefixSumsParallelForkJoin.java• Seetheprocessorgo,gogo!
11 17-214
Parallelprefixsumsalgorithm
• Howgoodisthis?– Work:O(n),Depth:O(lgn)
• Compareto:PrefixSumsParallelArrays.java
12 17-214
Parallelprefixsumsalgorithm
• Howgoodisthis?– Work:O(n),Depth:O(lgn)
• Compareto:PrefixSumsParallelArrays.java• Comparetothesequen9alalgorithm:
– SeePrefixSumsSequential.java
13 17-214
Parallelprefixsumsalgorithm
• Howgoodisthis?– Work:O(n),Depth:O(lgn)
• Compareto:PrefixSumsParallelArrays.java• Comparetothesequen9alalgorithm:
– SeePrefixSumsSequential.java– n-1addi9ons– Memoryaccessissequen9al
• Theparallelalgorithm:– About2nusefuladdi9ons,plusextraaddi9onsfortheloopindexes– Memoryaccessisnon-sequen9al
• Thepunchline:– Don'trollyourown– CacheandconstantsmaSer– Thebestparallelimplementa9onwasnobeSerthannaïvesequen9al
14 17-214
Today
• Javalambdasandfunc9onalinterfaces• Javastreams
15 17-214
Lambdas,briefly
• Termcomesfromλ-Calculus– Everythingisafunc9on!
• Alambda(λ)isananonymousfunc9on
16 17-214
DoesJavahavelambdas?
A. Yes,it’shadthemsincethebeginningB. Yes,it’shadthemsinceanonymousclasses(1.1)C. Yes,it’shadthemsinceJava8—thespecsaysso!D. No,neverhad’em,neverwill
17 17-214
Func9onobjectsinJava1.0
classStringLengthComparatorimplementsComparator{privateStringLengthComparator(){}publicstaticfinalStringLengthComparatorINSTANCE=newStringLengthComparator();publicintcompare(Objecto1,Objecto2){Strings1=(String)o1,s2=(String)o2;returns1.length()-s2.length();}}Arrays.sort(words,StringLengthComparator.INSTANCE);
18 17-214
Func9onobjectsinJava1.1
Arrays.sort(words,newComparator(){publicintcompare(Objecto1,Objecto2){Strings1=(String)o1,s2=(String)o2;returns1.length()-s2.length();}});
"ClassInstanceCrea9onExpression"(CICE)
19 17-214
Func9onobjectsinJava5
Arrays.sort(words,newComparator<String>(){publicintcompare(Strings1,Strings2){returns1.length()-s2.length();}});
CICEwithgenerics
20 17-214
Func9onobjectsinJava8
Arrays.sort(words,(s1,s2)->s1.length()-s2.length());
• Theyfeellikelambdas,they’recalledlambdas
21 17-214
Lambdasyntax
Syntax Exampleparameter->expression x->x*x
parameter->block s->{System.out.println(s);}
(parameters)->expression (x,y)->Math.sqrt(x*x+y*y)
(parameters)->block (s1,s2)->{System.out.println(s1+","+s2);}
(parameterdecls)->expression (doublex,doubley)->Math.sqrt(x*x+y*y)
(parametersdecls)->block (List<?>list)->{Arrays.shuffle(list);Arrays.sort(list);}
22 17-214
Methodreferences:moresuccinctthanlambdas
• Asta9cmethod– e.g.,Math::cos
• Anunboundinstancemethod(whosereceiverisunspecified)– e.g.,String::length– Theresul9ngfunc9onhasanextraargumentforthereceiver
• Aboundinstancemethodofaspecificobject– e.g.,System.out::println
• Aconstructor– e.g.,Integer::new,String[]::new
23 17-214
Nofunc9ontypesinJava,onlyfunc3onalinterfaces
• Interfaceswithonlyoneabstractmethod• Op9onallyannotatedwith@FunctionalInterface• Somefunc9onalinterfacesyouknow
– java.lang.Runnable– java.util.concurrent.Callable– java.util.Comparator– java.awt.event.ActionListener– Many,manymoreinjava.util.function
24 17-214
Func9oninterfacesinjava.util.function
BiConsumer<T,U>BiFunction<T,U,R>BinaryOperator<T>BiPredicate<T,U>BooleanSupplierConsumer<T>DoubleBinaryOperatorDoubleConsumerDoubleFunction<R>DoublePredicateDoubleSupplierDoubleToIntFunctionDoubleToLongFunctionDoubleUnaryOperatorFunction<T,R>IntBinaryOperatorIntConsumerIntFunction<R>IntPredicateIntSupplierIntToDoubleFunctionIntToLongFunction
IntUnaryOperatorLongBinaryOperatorLongConsumerLongFunction<R>LongPredicateLongSupplierLongToDoubleFunctionLongToIntFunctionLongUnaryOperatorObjDoubleConsumer<T>ObjIntConsumer<T>ObjLongConsumer<T>Predicate<T>Supplier<T>ToDoubleBiFunction<T,U>ToDoubleFunction<T>ToIntBiFunction<T,U>ToIntFunction<T>ToLongBiFunction<T,U>ToLongFunction<T>UnaryOperator<T>
25 17-214
SomeFunction<String,Integer>
Descrip<on CodeLambda s->Integer.parseInt(s)
Lambdaw/explicitparamtype (Strings)->Integer.parseInt(s)
Sta9cmethodreference Integer::parseInt
Constructorreference Integer::new
Instancemethodreference String::length
AnonymousclassICE newFunction<String,Integer>(){publicIntegerapply(Strings){returns.length();}}
26 17-214
Javastreams
• Astreamisabunchofdataobjects,typicallyfromacollec9on,array,orinputdevice,forprocessing
• Processedbyapipeline– Asinglestreamgenerator(datasource)– Zeroormoreintermediatestreamopera.ons– Asingleterminalstreamopera.on
27 17-214
Streamexamples:Itera9on
//IterationoveracollectionstaticList<String>stringList=...;stringList.stream().forEach(System.out::println);//IterationoverarangeofintegersIntStream.range(0,10).forEach(System.out::println);//Aminipuzzler:whatdoesthisprint?"Helloworld!".chars().forEach(System.out::print);
28 17-214
Puzzlersolu9on
"Helloworld!".chars().forEach(System.out::print);
Prints"721011081081113211911111410810033"
ThecharsmethodonStringreturnsanIntStream
29 17-214
Howdoyoufixit?
"Helloworld!".chars().forEach(x->System.out.print((char)x));
• Nowprints"Helloworld!"
• Morals:– Streamsonlyforobjectreftypes,int,long,anddouble– Typeinferencecanbeconfusing
30 17-214
Streamexamples:mapping,filtering
List<String>longStrings=stringList.stream().filter(s->s.length()>42).collect(Collectors.toList());
List<String>firstLetters=stringList.stream().map(s->s.substring(0,1)).collect(Collectors.toList());
List<String>firstLetterOfLongStrings=stringList.stream().filter(s->s.length()>42).map(s->s.substring(0,1)).collect(Collectors.toList());
31 17-214
Streamexamples:duplicates,sor9ng
List<String>dupsRemoved=stringList.stream().map(s->s.substring(0,1)).distinct().collect(Collectors.toList());List<String>sortedList=stringList.stream().map(s->s.substring(0,1)).sorted()//Bufferseverythinguntilterminalop.collect(Collectors.toList());
32 17-214
Streamexamples:bulkpredicates
booleanallStringHaveLengthThree=stringList.stream().allMatch(s->s.length()==3);booleananyStringHasLengthThree=stringList.stream().anyMatch(s->s.length()==3);
33 17-214
Streamsareprocessedlazily
• Dataispulledbyterminalopera9on,notpushedbysource– Infinitestreamsarenotaproblem
• Intermediateopera9onscanbefused– Mul9pleintermediateopera9onsusuallydon’tcausemul9pletraversals
• Intermediateresultsusuallynotstored– Butthereareexcep9ons(e.g.,sorted)
34 17-214
Easyparallelism:.parallelStream()
List<String>longStrings=stringList.parallelStream().filter(s->s.length()>42).collect(Collectors.toList());
List<String>firstLetters=stringList.parallelStream().map(s->s.substring(0,1)).collect(Collectors.toList());
List<String>firstLetterOfLongStrings=stringList.parallelStream().filter(s->s.length()>42).map(s->s.substring(0,1)).collect(Collectors.toList());
35 17-214
Streaminterfaceisamonster(1/3)
publicinterfaceStream<T>extendsBaseStream<T,Stream<T>>{//IntermediateOperationsStream<T>filter(Predicate<T>);<R>Stream<R>map(Function<T,R>);IntStreammapToInt(ToIntFunction<T>);LongStreammapToLong(ToLongFunction<T>);DoubleStreammapToDouble(ToDoubleFunction<T>);<R>Stream<R>flatMap(Function<T,Stream<R>>);IntStreamflatMapToInt(Function<T,IntStream>);LongStreamflatMapToLong(Function<T,LongStream>);DoubleStreamflatMapToDouble(Function<T,DoubleStream>);Stream<T>distinct();Stream<T>sorted();Stream<T>sorted(Comparator<T>);Stream<T>peek(Consumer<T>);Stream<T>limit(long);Stream<T>skip(long);
36 17-214
Streaminterfaceisamonster(2/3)
//TerminalOperationsvoidforEach(Consumer<T>);//OrderedonlyforsequentialstreamsvoidforEachOrdered(Consumer<T>);//OrderedifencounterorderexistsObject[]toArray();<A>A[]toArray(IntFunction<A[]>arrayAllocator);Treduce(T,BinaryOperator<T>);Optional<T>reduce(BinaryOperator<T>);<U>Ureduce(U,BiFunction<U,T,U>,BinaryOperator<U>);<R,A>Rcollect(Collector<T,A,R>);//MutableReductionOperation<R>Rcollect(Supplier<R>,BiConsumer<R,T>,BiConsumer<R,R>);Optional<T>min(Comparator<T>);Optional<T>max(Comparator<T>);longcount();booleananyMatch(Predicate<T>);booleanallMatch(Predicate<T>);booleannoneMatch(Predicate<T>);Optional<T>findFirst();Optional<T>findAny();
37 17-214
Streaminterfaceisamonster(3/3)
//Staticmethods:streamsourcespublicstatic<T>Stream.Builder<T>builder();publicstatic<T>Stream<T>empty();publicstatic<T>Stream<T>of(T);publicstatic<T>Stream<T>of(T...);publicstatic<T>Stream<T>iterate(T,UnaryOperator<T>);publicstatic<T>Stream<T>generate(Supplier<T>);publicstatic<T>Stream<T>concat(Stream<T>,Stream<T>);}
38 17-214
Incaseyoureyesaren’tglazedyet
publicinterfaceBaseStream<T,SextendsBaseStream<T,S>>extendsAutoCloseable{
Iterator<T>iterator();Spliterator<T>spliterator();booleanisParallel();Ssequential();//MayhavelittleornoeffectSparallel();//MayhavelittleornoeffectSunordered();//Noteasymmetrywrtsequential/parallelSonClose(Runnable);voidclose();}
39 17-214
Itkeepsgoing:java.util.stream.Collectors
...toList()
...toMap(...)
...toSet(...)
...reducingBy(...)
...groupingBy(...)
...partitioningBy(...)...
40 17-214
Optional<T>:anotherwaytoindicatetheabsenceofaresult
ItalsoactsabitlikeadegeneratestreampublicfinalclassOptional<T>{booleanisPresent();Tget();
voidifPresent(Consumer<T>);Optional<T>filter(Predicate<T>);<U>Optional<U>map(Function<T,U>);<U>Optional<U>flatMap(Function<T,Optional<U>>);TorElse(T);TorElseGet(Supplier<T>);<XextendsThrowable>TorElseThrow(Supplier<X>)throwsX;}
41 17-214
Streamprac9ce
• GivenaList<String>words,usestreamsto:– GenerateaList<String>ofallwordscontainingthesubstring"heat"– Determineifanywordcontainsthesubstring"aoeu"(aboolean)
• Challenge:Convertsomeopera9oninyourCarcassonnesolu9ontousestreams…
42 17-214
Streamparallelism:Yourmileagemayvary
• Considerthisfor-loop(.96srun9me;dual-corelaptop)longsum=0;for(longj=0;j<Integer.MAX_VALUE;j++)sum+=j;
• Equivalentstreamcomputa9on(1.5s)longsum=LongStream.range(0,Integer.MAX_VALUE).sum();
• Equivalentparallelcomputa9on(.77s)longsum=LongStream.range(0,Integer.MAX_VALUE).parallel().sum();
• Carefullyhandcra3edparallelcode(.48s)
43 17-214
Whentouseaparallelstream,looselyspeaking
• Whenopera9onsareindependent,and• Eitherorboth:
– Opera9onsarecomputa9onallyexpensive– Opera9onsareappliedtomanyelementsofefficientlyspliSabledata
structures– Roughly:Numberofelements*Cost/element>>10,000
• AlwaysmeasurebeforeandaDerparallelizing!
44 17-214
Whennotto…
• Useaparallelstream…• Useastream...
45 17-214
Summary
• APIdesign:"Funandeasytolearnanduse…?"• Whentousealambda
– Always,inpreferencetoCICE• Whentouseamethodreference
– Almostalways,inpreferencetoalambda• Whentouseastream
– Whenitfeelsandlooksright• Whentouseaparallelstream
– Numberofelements*Cost/element>>10,000• Keepitclassy!
– Javaisnotafunc9onallanguage