What's next in Julia

transcript

What’s next in Jul_

Jiahao Chen MIT Computer Science and Artificial Intelligence Laboratory

julialang.org

Alan EdelmanJeff Bezanson Stefan Karpinski Viral B. Shah

What’s the big deal about Julia ?

julialang.org/benchmarks

It bridges the divide between computer science and computational science

data abstraction

performance

data abstraction

performance

What if you didn’t have to choose between data abstraction and performance?

It’s a programming language designed for technical computing

The key ingredients

Multi-methods (multiple dispatch)

Dataflow type inference

together allow for cost-efficient data abstraction

Object-oriented programming with classes

What can I do with/to a thing?

top uppay fare

pay farelose

methods objects

top uppay fare

pay farelose

methods objects

top uppay fare

pay farelose

class-based OO !classes are more fundamental than methods

Object-oriented programming with multi-methods

top up

pay fare

generic function

objectsmethods

Object-oriented programming with multi-methods

top up

pay fare

generic function

objectsmethods

multimethods !relationships between objects and functions

Multi-methods with type hierarchy

top up

pay fare

generic function

objectsmethods

rechargeable subway pass

single-use subway ticket

is a subtype of

subway ticket

abstract object

generic function

objectsmethods

is a subtype of

subway ticket

top up

pay fare

abstract object

generic function

objectsmethods

is a subtype of

subway ticket

top up

pay farelose

abstract object

The Julia codebase is compact

5,774 lines of Scheme 32,707 lines of C 59727 lines of Julia !+LLVM, BLAS, LAPACK, SuiteSparse, ARPACK, Rmath, GMP, MPFR, FFTW,…

Data types as a lattice

Dana Scott, Data types as lattices, SIAM J. Comput. 5: 522-87. 1976

Number

FloatingPoint Rational

Complex

Float64 BigFloat…

Integer

is a parameter ofis a subtype of

Signed BigInt

Int64…

Signed

Multiple dispatch with type lattice traversal

Number

Float64 BigFloat…

Integer

BigInt

Int64…

Signed

Number

Float64 BigFloat…

Integer

BigInt

Int64…

Signed

Number

Float64 BigFloat…

Integer

BigInt

Int64…

Signed

Number

Float64 BigFloat…

Integer

BigInt

Int64…

no method here try supertype

super(Int64) = Signed

Signed

Number

Float64 BigFloat…

Integer

BigInt

Int64…

no method here either super(Signed) = Integer

Signed

Number

Float64 BigFloat…

Integer

BigInt

Int64…

no method here either super(Signed) = Integer

found a method

Signed

Number

Float64 BigFloat…

Integer

BigInt

Int64…

Signed

Number

Float64 BigFloat…

Integer

BigInt

Int64…

LLVM IR

Machine assembly

aggressive type specialization !compiler generates specialized methods for different input types to the same generic function

unitful computations with essentially no runtime overhead

Keno Fischer Harvard Physics and Mathematics

Type lattice of arrays

DenseArray{T, N}

AbstractArray{T, N}

Array{T, N}

is a subtype of

Array{T,1}===Vector{T}Array{T,2}===Matrix{T}

A single parametric type defines many types of matrices Matrix{Float64}, Matrix{Int64},Matrix{BigFloat}, Matrix{Complex128},Matrix{Rational}, Matrix{Quaternion{Float64}},…

DenseArray{T, N}

AbstractArray{T, N}

Array{T, N}

is a subtype of

DenseArray{T, N}

AbstractArray{T, N}

Array{T, N}

is a subtype of

rotation matrices structured matrices

distributed arrays

Multi-methods for linear algebra

find eigenvalues and eigenvectors

find singular values

find singular values and vectors

find eigenvalues

generic function

objectsmethods

general matrix

symmetric tridiagonal matrix

bidiagonal matrix

Methods can take advantage of special matrix structures

eigvals

eigfact

svdvals

svdfact

Matrix

SymTridiagonal

Bidiagonal

So how does this help us with linear algebra?

Multi-method dispatch on special matrix types

stev!{T<:BlasFloat} calls sgestvdgestvcgestvzgestv

and handles workspace memory allocation

Multi-method dispatch with generic fallbacks

Matrix operations on general rings

Matrix operations on general rings textbook algorithm

Matrix operations on general rings

Matrix factorization types

Iterative algorithms as iterators

for item in iterable #bodyend!

#is equivalent to!

state = start(iterable) while !done(iterable, state) item, state = next(iterable, state) # bodyend

Conjugate gradients (Hestenes-Stiefel)

http://en.wikipedia.org/wiki/Conjugate_gradient_method

Can we write this as an iterator?

state = start(iterable)

done(iterable, state)

state = start(iterable)

done(iterable, state)

state, dx = next(iterable, state)

immutable cg_hs #iterative solver K :: KrylovSpace #wraps A, v0, k t :: Terminator #termination criteriaend immutable cg_hs_state r :: Vector #residual p :: Vector #search direction rnormsq :: Float64 #Squared norm of previous residual iter :: Int #iteration countend start(a::cg_hs) = cg_hs_state(a.K.v0, zeros(size(a.K.v0,1)), Inf, 0) function next(a::cg_hs, s::cg_hs_state) rnormsq = dot(s.r, s.r) p = s.r + (rnormsq/s.rnormsq)*s.p Ap = a.K.A*p α = rnormsq / dot(p, Ap) α*p, cg_hs_state(s.r-α*Ap, p, rnormsq, s.iter+1)end done(a::cg_hs, s::cg_hs_state) = done(a.t, s)!K = method(KrylovSpace(A, b, k), Terminator())x += reduce(+, K) # for dx in K; x += dx; end

Hestenes-Stiefel CG

immutable cg_hs #iterative solver K :: KrylovSpace #wraps A, v0, k t :: Terminator #termination criteriaend immutable cg_hs_state r :: Vector #residual p :: Vector #search direction rnormsq :: Float64 #Squared norm of previous residual iter :: Int #iteration countend start(a::cg_hs) = cg_hs_state(a.K.v0, zeros(size(a.K.v0,1)), Inf, 0) function next(a::cg_hs, s::cg_hs_state) rnormsq = dot(s.r, s.r) p = s.r + (rnormsq/s.rnormsq)*s.p Ap = a.K.A*p α = rnormsq / dot(p, Ap) α*p, cg_hs_state(s.r-α*Ap, p, rnormsq, s.iter+1)end done(a::cg_hs, s::cg_hs_state) = done(a.t, s)!K = method(KrylovSpace(A, b, k), Terminator())x += reduce(+, K) # for dx in K; x += dx; end

Hestenes-Stiefel CG

Abstracts out termination check and solution update steps

Native parallelism constructs

Distributed arrays

IJulia: Julia in IPython Notebook

JuMP: writing simple DSLs in Julia

Iain Dunning Miles LubinMIT

Operations Research

Andreas N. Jensen U. Copenhagen

Economics

Alan EdelmanJeff Bezanson Stefan Karpinski Viral B. Shah

Carlo Baldassi Poly. Torino

Neuroscience

Tim E. Holy WUSTL

Anatomy

Douglas M. Bates Wisconsin-Madison

Statistics

Steven G. Johnson MIT

Mathematics

What's next in Julia

Data & Analytics